[jondurbin/airoboros-34b-3.2] [HQQ] [8bit]
This repository contains an HQQ (Half-Quadratic Quantization) model trained on [describe the task or dataset].
Original Repo
Model Details
Model ID:
jondurbin/airoboros-34b-3.2
Model Architecture: yi-34b-200k
Training Dataset: Read through original repo
Performance: In progress
nbits
: The number of bits used for quantization:8
.group_size
: The size of the quantization group:128
.quant_zero
: Whether to quantize the zero values:True
.quant_scale
: Whether to quantize the scale values:True
.
- Downloads last month
- 6
Inference API (serverless) is not available, repository is disabled.
Model tree for macadeliccc/airoboros-34b-3.2-hqq-8bit
Base model
jondurbin/airoboros-34b-3.2
Finetuned
this model