bert-base-squad-v1.1-pt-IBAMA_v0.420240914221657

This model is a fine-tuned version of pierreguillou/bert-base-cased-squad-v1.1-portuguese on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 9.2011

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	33	4.6586
No log	2.0	66	4.3610
No log	3.0	99	4.3218
No log	4.0	132	4.3218
No log	5.0	165	4.6298
No log	6.0	198	4.7928
No log	7.0	231	5.1607
No log	8.0	264	5.5669
No log	9.0	297	5.7820
No log	10.0	330	6.1737
No log	11.0	363	6.3348
No log	12.0	396	6.7020
No log	13.0	429	6.7789
No log	14.0	462	7.0030
No log	15.0	495	6.8479
2.1793	16.0	528	7.4477
2.1793	17.0	561	7.3856
2.1793	18.0	594	7.5348
2.1793	19.0	627	7.7080
2.1793	20.0	660	7.8438
2.1793	21.0	693	7.8194
2.1793	22.0	726	7.9835
2.1793	23.0	759	7.9357
2.1793	24.0	792	8.1900
2.1793	25.0	825	8.1193
2.1793	26.0	858	8.2119
2.1793	27.0	891	8.5122
2.1793	28.0	924	8.3852
2.1793	29.0	957	8.5886
2.1793	30.0	990	8.4135
0.373	31.0	1023	8.2952
0.373	32.0	1056	8.4717
0.373	33.0	1089	8.4430
0.373	34.0	1122	8.6253
0.373	35.0	1155	8.5774
0.373	36.0	1188	8.7257
0.373	37.0	1221	8.9166
0.373	38.0	1254	8.8245
0.373	39.0	1287	9.0156
0.373	40.0	1320	9.0460
0.373	41.0	1353	8.8083
0.373	42.0	1386	9.1857
0.373	43.0	1419	9.2103
0.373	44.0	1452	9.0799
0.373	45.0	1485	8.9070
0.0952	46.0	1518	9.1408
0.0952	47.0	1551	9.1571
0.0952	48.0	1584	9.1600
0.0952	49.0	1617	9.2383
0.0952	50.0	1650	9.2011

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0+cu121
• Datasets 3.0.0
• Tokenizers 0.19.1