AI is happening. The future is updating.c4ai-command-r-plus-08-2024-IMat-GGUF
Original Model: CohereForAI/c4ai-command-r-plus-08-2024 Original dtype: `FP16` (`float16`) Quantized by: llama.cpp b3645 IMatrix dataset: here If the model file is big, it has been split into multiple files. In order to download them all to a local folder, run: According to this investigation, it appears that lower quantizations are the only ones that benefit from the imatrix input (as per hellaswag results).
Original Model: CohereForAI/c4ai-command-r-plus-08-2024 Original dtype: `FP16` (`float16`) Quantized by: llama.cpp b3645 IMatrix dataset: here If the model file is big, it has been split into multiple files. In order to download them all to a local folder, run: According to this investigation, it appears that lower quantizations are the only ones that benefit from the imatrix input (as per hellaswag results).