ggml : repack block_iq4_nlx8 (AVX)

[ggerganov]

Repack 8x block_iq4_nl into block_iq4_nlx8 + add AVX implementation

• Reuse the existing block_q4_0x8 GEMV/GEMM implementation (the logic is the same, just the lookup table for nibbles -> bytes is different)
• Cleanup some UNUSED macros (not exhaustive)

TODOs:

• Test the __AVX512F__ path after the refactoring

[ggerganov]

@Srihari-mcw Since you have access to AVX512, could you run this branch with an iq4_nl quantizaion and verify that the perplexity is within norm?

[Srihari-mcw]

@Srihari-mcw Since you have access to AVX512, could you run this branch with an iq4_nl quantizaion and verify that the perplexity is within norm?

Sure, will check and get back on the same. Thanks

[Srihari-mcw]

Hi @ggerganov , we tested the model for perplexity with meta llama2 7B model quantized to 'IQ4_NL' and observed the following perplexity in AVX512 Machine (AMD Ryzen 5 7600X). The perplexity seem close enough

model	perplexity (Final estimate PPL)	Commit id
llama 7B IQ4_NL	5.8822 +/- 0.03282	Base - 00131d6e
llama 7B IQ4_NL	5.8828 +/- 0.03283	PR Branch - d1788b72