mms-MDPC
This model is a fine-tuned version of facebook/mms-1b-all on the None dataset. It achieves the following results on the evaluation set:
- Loss: 2.9364
- Wer: 70.0250
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 14
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant_with_warmup
- lr_scheduler_warmup_steps: 50
- num_epochs: 4
Training results
Training Loss | Epoch | Step | Validation Loss | Wer |
---|---|---|---|---|
19.864 | 0.06 | 250 | 16.1732 | 100.1101 |
3.687 | 0.12 | 500 | 3.8920 | 100.0 |
3.2582 | 0.17 | 750 | 3.7154 | 100.0100 |
2.7525 | 0.23 | 1000 | 3.3154 | 99.8199 |
2.2947 | 0.29 | 1250 | 2.9790 | 95.2376 |
2.024 | 0.35 | 1500 | 2.7973 | 91.3057 |
1.9249 | 0.4 | 1750 | 2.7070 | 89.1546 |
1.7769 | 0.46 | 2000 | 2.6510 | 88.0640 |
1.7417 | 0.52 | 2250 | 2.6255 | 86.5633 |
1.7468 | 0.58 | 2500 | 2.6030 | 85.4227 |
1.6818 | 0.63 | 2750 | 2.5864 | 84.7624 |
1.6304 | 0.69 | 3000 | 2.5921 | 84.3522 |
1.6494 | 0.75 | 3250 | 2.5835 | 83.5118 |
1.5068 | 0.81 | 3500 | 2.5737 | 82.3712 |
1.6079 | 0.87 | 3750 | 2.5621 | 81.7509 |
1.5069 | 0.92 | 4000 | 2.5641 | 80.9605 |
1.5596 | 0.98 | 4250 | 2.5636 | 80.2801 |
1.5396 | 1.04 | 4500 | 2.5623 | 79.7899 |
1.3875 | 1.1 | 4750 | 2.5761 | 79.5198 |
1.3952 | 1.15 | 5000 | 2.5841 | 79.1596 |
1.3948 | 1.21 | 5250 | 2.5849 | 78.5493 |
1.49 | 1.27 | 5500 | 2.5898 | 78.6693 |
1.3669 | 1.33 | 5750 | 2.5908 | 78.5393 |
1.3488 | 1.38 | 6000 | 2.6110 | 78.2491 |
1.3431 | 1.44 | 6250 | 2.6172 | 77.8589 |
1.3429 | 1.5 | 6500 | 2.6238 | 77.5788 |
1.3683 | 1.56 | 6750 | 2.6229 | 77.4887 |
1.4073 | 1.62 | 7000 | 2.6362 | 76.9485 |
1.2954 | 1.67 | 7250 | 2.6421 | 76.8384 |
1.3793 | 1.73 | 7500 | 2.6392 | 76.5183 |
1.3223 | 1.79 | 7750 | 2.6513 | 76.2281 |
1.2377 | 1.85 | 8000 | 2.6695 | 75.8479 |
1.2889 | 1.9 | 8250 | 2.6720 | 75.6778 |
1.2456 | 1.96 | 8500 | 2.6769 | 75.3877 |
1.2595 | 2.02 | 8750 | 2.6945 | 74.9275 |
1.2332 | 2.08 | 9000 | 2.6904 | 74.8374 |
1.2874 | 2.13 | 9250 | 2.7051 | 74.4372 |
1.2886 | 2.19 | 9500 | 2.6900 | 74.2171 |
1.3229 | 2.25 | 9750 | 2.7075 | 74.2271 |
1.245 | 2.31 | 10000 | 2.7114 | 73.8369 |
1.2316 | 2.37 | 10250 | 2.7207 | 73.4767 |
1.2379 | 2.42 | 10500 | 2.7261 | 73.7669 |
1.1906 | 2.48 | 10750 | 2.7471 | 73.2666 |
1.3066 | 2.54 | 11000 | 2.7522 | 73.3667 |
1.2382 | 2.6 | 11250 | 2.7464 | 73.0065 |
1.2262 | 2.65 | 11500 | 2.7626 | 72.8564 |
1.3256 | 2.71 | 11750 | 2.7778 | 72.7264 |
1.2251 | 2.77 | 12000 | 2.7764 | 72.4762 |
1.2187 | 2.83 | 12250 | 2.7751 | 72.2261 |
1.2674 | 2.88 | 12500 | 2.7987 | 72.1861 |
1.2015 | 2.94 | 12750 | 2.8093 | 72.0160 |
1.1485 | 3.0 | 13000 | 2.8161 | 71.8059 |
1.1686 | 3.06 | 13250 | 2.8240 | 71.8659 |
1.1331 | 3.11 | 13500 | 2.8269 | 71.6658 |
1.1589 | 3.17 | 13750 | 2.8500 | 71.5558 |
1.2203 | 3.23 | 14000 | 2.8497 | 71.4557 |
1.1592 | 3.29 | 14250 | 2.8542 | 71.4057 |
1.1957 | 3.35 | 14500 | 2.8789 | 70.9955 |
1.202 | 3.4 | 14750 | 2.8694 | 71.0255 |
1.1397 | 3.46 | 15000 | 2.8777 | 70.8354 |
1.2431 | 3.52 | 15250 | 2.8925 | 70.5753 |
1.178 | 3.58 | 15500 | 2.8910 | 70.6453 |
1.1716 | 3.63 | 15750 | 2.9085 | 70.6153 |
1.1464 | 3.69 | 16000 | 2.8951 | 70.3752 |
1.1307 | 3.75 | 16250 | 2.9159 | 70.5253 |
1.2094 | 3.81 | 16500 | 2.9170 | 70.1851 |
1.1988 | 3.86 | 16750 | 2.9155 | 70.1151 |
1.2435 | 3.92 | 17000 | 2.9272 | 70.1151 |
1.1212 | 3.98 | 17250 | 2.9364 | 70.0250 |
Framework versions
- Transformers 4.33.2
- Pytorch 2.0.1
- Datasets 2.19.1
- Tokenizers 0.13.3
- Downloads last month
- 0
Inference API (serverless) is not available, repository is disabled.
Model tree for herwoww/mms-MDPC
Base model
facebook/mms-1b-all
Finetuned
this model