LongWriter-glm4-9b
Original model link: https://huggingface.co/THUDM/LongWriter-glm4-9b
Model by: THUDM
Quants by: QuantPanda
GGUF quantization for llama.cpp and similar applications.
Example:
./llama-cli -m LongWriter-glm4-9B-Q5_K_M.gguf -p "You are a helpful AI assistant." --conversation
If the model takes too long to load you can reduce the context size with --ctx-size
Example with smaller context size:
./llama-cli -m LongWriter-glm4-9B-Q5_K_M.gguf -p "You are a helpful AI assistant." --conversation --ctx-size 4096
- Downloads last month
- 1,468
Inference API (serverless) is not available, repository is disabled.
Model tree for QuantPanda/LongWriter-glm4-9B-GGUF
Base model
THUDM/LongWriter-glm4-9b
Quantized
this model