wav2vec2-large-mms-1b-kyrgyz-colab
This model is a fine-tuned version of facebook/mms-1b-all on the common_voice_11_0 dataset. It achieves the following results on the evaluation set:
- Loss: 3.6994
- Wer: 1.0028
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 2
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 15
- mixed_precision_training: Native AMP
Training results
Training Loss | Epoch | Step | Validation Loss | Wer |
---|---|---|---|---|
8.657 | 0.59 | 1000 | 5.3559 | 1.0 |
3.7087 | 1.18 | 2000 | 4.2908 | 1.0 |
3.4394 | 1.76 | 3000 | 4.1226 | 1.0 |
3.3918 | 2.35 | 4000 | 4.0324 | 1.0 |
3.3357 | 2.94 | 5000 | 3.9328 | 1.0 |
3.2915 | 3.53 | 6000 | 3.8575 | 0.9998 |
3.245 | 4.12 | 7000 | 3.7375 | 1.0001 |
3.219 | 4.71 | 8000 | 3.7220 | 1.0003 |
3.1878 | 5.29 | 9000 | 3.7172 | 1.0008 |
3.1826 | 5.88 | 10000 | 3.7064 | 1.0012 |
3.1613 | 6.47 | 11000 | 3.7029 | 1.0014 |
3.1548 | 7.06 | 12000 | 3.6923 | 1.0011 |
3.1403 | 7.65 | 13000 | 3.7013 | 1.0010 |
3.1286 | 8.24 | 14000 | 3.6862 | 1.0017 |
3.1184 | 8.82 | 15000 | 3.7129 | 1.0028 |
3.1198 | 9.41 | 16000 | 3.7063 | 1.0022 |
3.1051 | 10.0 | 17000 | 3.7093 | 1.0018 |
3.098 | 10.59 | 18000 | 3.6952 | 1.0031 |
3.1001 | 11.18 | 19000 | 3.7042 | 1.0031 |
3.0984 | 11.76 | 20000 | 3.6972 | 1.0020 |
3.0873 | 12.35 | 21000 | 3.7018 | 1.0026 |
3.0852 | 12.94 | 22000 | 3.7005 | 1.0025 |
3.0902 | 13.53 | 23000 | 3.6972 | 1.0026 |
3.081 | 14.12 | 24000 | 3.6984 | 1.0024 |
3.0777 | 14.71 | 25000 | 3.6994 | 1.0028 |
Framework versions
- Transformers 4.37.0.dev0
- Pytorch 2.1.2+cu121
- Datasets 2.16.1
- Tokenizers 0.15.0
- Downloads last month
- 0
Inference API (serverless) is not available, repository is disabled.
Model tree for zhusupaidai/wav2vec2-large-mms-1b-kyrgyz-colab
Base model
facebook/mms-1b-all
Finetuned
this model