arabert_baseline_vocabulary_task2_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5710
• Qwk: 0.0911
• Mse: 0.5727

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	5.7084	-0.0071	5.7092
No log	0.6667	4	2.7119	0.0524	2.7198
No log	1.0	6	1.4034	-0.0198	1.4046
No log	1.3333	8	0.6439	0.0396	0.6441
No log	1.6667	10	0.6138	0.0	0.6150
No log	2.0	12	0.6664	0.0	0.6682
No log	2.3333	14	0.6604	-0.0925	0.6596
No log	2.6667	16	0.5976	0.0526	0.5979
No log	3.0	18	0.6250	0.1747	0.6254
No log	3.3333	20	0.6646	0.1021	0.6644
No log	3.6667	22	0.5974	0.2396	0.5967
No log	4.0	24	0.5442	0.0911	0.5440
No log	4.3333	26	0.5490	0.0289	0.5512
No log	4.6667	28	0.5728	0.0	0.5758
No log	5.0	30	0.5435	0.0	0.5463
No log	5.3333	32	0.4958	0.2794	0.4980
No log	5.6667	34	0.4795	0.2105	0.4813
No log	6.0	36	0.4887	0.2167	0.4894
No log	6.3333	38	0.4972	0.2222	0.4980
No log	6.6667	40	0.5094	0.2222	0.5094
No log	7.0	42	0.5328	0.1198	0.5321
No log	7.3333	44	0.5401	0.1064	0.5397
No log	7.6667	46	0.5403	0.1064	0.5409
No log	8.0	48	0.5495	0.0911	0.5517
No log	8.3333	50	0.5683	0.0735	0.5711
No log	8.6667	52	0.5763	0.0735	0.5791
No log	9.0	54	0.5754	0.0911	0.5779
No log	9.3333	56	0.5725	0.0911	0.5748
No log	9.6667	58	0.5717	0.0911	0.5736
No log	10.0	60	0.5710	0.0911	0.5727

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1