CodeLlama-34B-Instruct-GGUF

GGUF is a new format introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp. GGUF offers numerous advantages over GGML, such as better tokenisation, and support for special tokens. It is also supports metadata, and is designed to be extensible. shell pip3 install huggingface-hub>=0.17.1 shell huggingface-cli download TheBloke/CodeLlama-34B-Instruct-GGUF codellama-34b-instruct.q4_K_M.gguf --local-dir . --local-dir-use-symlinks False shell huggingface-cli download TheBloke/CodeLlama-34B-Instruct-GGUF --local-dir . --local-dir-use-symlinks False --include='*Q4_K*gguf' shell pip3 install hf_transfer shell HUGGINGFACE_HUB_ENABLE_HF_TRANSFER=1 huggingface-cli download TheBloke/CodeLlama-34B-Instruct-GGUF codellama-34b-instruct.q4_K_M.gguf --local-dir . --local-dir-use-symlinks False shell ./main -ngl 32 -m codellama-34b-instruct.q4_K_M.gguf --color -c 4096 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "[INST] Write code to solve the following coding problem that obeys the constraints and passes the example test cases. Please wrap your code answer using Change `-ngl 32` to the number of layers to offload to GPU. Remove it if you don't have GPU acceleration.

翻訳準備中のため原文を表示しています