Testing
Hyperparameters
- Temperature: 0.9
- Penalize repeat sequence: 1.05
- Consider N tokens for penalize: 256
- Penalize repetition of newlines
- Top-K sampling: 40
- Top-P sampling: 0.95
- Min-P sampling: 0.05
LLaMAcpp Version
- b3527-2-g2d5dd7bb
File
- Cathallama-70B.Q4_0.gguf
Test Cases
| Test Case | Result |
|---|---|
| Ball on cup | OK |
| Door window combination | OK |
| Big duck small horse | OK |
| JSON | OK |
| Killers | OK |
| Dragon | OK |
| Poem | OK |
| Jane faster | OK |
| Shirts | OK |
| Sisters | OK |
| Python snake game | OK* |
| Story | OK |
*best I ever saw on local LLMs including Qwen2 72b at 8bpw, Llama 3 70b 8bpw
Note: See sample generations on the main folder of the repo.