AI is happening. The future is updating.Qwen2.5-Coder-7B-Instruct-SOTA-GGUF
This repo contains State Of The Art quantized GGUF format model files for Qwen2.5-Coder-7B-Instruct. Quantization was done with an importance matrix that was trained for ~1M tokens (256 batches of 4096 tokens) of answers from the CodeFeedback-Filtered-Instruction dataset. These quantised GGUFv3 files are compatible with llama.cpp from February 27th 2024 onwards, as of commit 0becb22
This repo contains State Of The Art quantized GGUF format model files for Qwen2.5-Coder-7B-Instruct. Quantization was done with an importance matrix that was trained for ~1M tokens (256 batches of 4096 tokens) of answers from the CodeFeedback-Filtered-Instruction dataset. These quantised GGUFv3 files are compatible with llama.cpp from February 27th 2024 onwards, as of commit 0becb22