PhoBert_Dataset59KCoDuoi

This model is a fine-tuned version of vinai/phobert-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2804
• Accuracy: 0.9582
• F1: 0.9583

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.5115	200	0.1696	0.9382	0.9389
No log	1.0230	400	0.1521	0.9538	0.9539
No log	1.5345	600	0.1459	0.9524	0.9528
0.1561	2.0460	800	0.1362	0.9557	0.9558
0.1561	2.5575	1000	0.1355	0.9577	0.9581
0.1561	3.0691	1200	0.1271	0.9617	0.9618
0.1561	3.5806	1400	0.1243	0.9620	0.9621
0.0913	4.0921	1600	0.1298	0.9598	0.9601
0.0913	4.6036	1800	0.1216	0.9617	0.9619
0.0913	5.1151	2000	0.1332	0.9607	0.9608
0.0913	5.6266	2200	0.1444	0.9609	0.9610
0.065	6.1381	2400	0.1573	0.9608	0.9609
0.065	6.6496	2600	0.1477	0.9608	0.9610
0.065	7.1611	2800	0.1572	0.9610	0.9611
0.065	7.6726	3000	0.1521	0.9605	0.9606
0.047	8.1841	3200	0.1523	0.9596	0.9598
0.047	8.6957	3400	0.1935	0.9591	0.9591
0.047	9.2072	3600	0.1751	0.9576	0.9577
0.047	9.7187	3800	0.1979	0.9573	0.9573
0.0335	10.2302	4000	0.1905	0.9592	0.9592
0.0335	10.7417	4200	0.1870	0.9582	0.9582
0.0335	11.2532	4400	0.1940	0.9585	0.9586
0.0335	11.7647	4600	0.1787	0.9549	0.9551
0.0268	12.2762	4800	0.2240	0.9602	0.9602
0.0268	12.7877	5000	0.2161	0.9572	0.9572
0.0268	13.2992	5200	0.2160	0.9542	0.9544
0.0268	13.8107	5400	0.2299	0.9553	0.9555
0.0177	14.3223	5600	0.2551	0.9579	0.9580
0.0177	14.8338	5800	0.2355	0.9576	0.9578
0.0177	15.3453	6000	0.2467	0.9587	0.9589
0.0177	15.8568	6200	0.2507	0.9593	0.9594
0.0142	16.3683	6400	0.2482	0.9580	0.9581
0.0142	16.8798	6600	0.2600	0.9599	0.9600
0.0142	17.3913	6800	0.2682	0.9566	0.9568
0.0142	17.9028	7000	0.2782	0.9573	0.9574
0.0094	18.4143	7200	0.2826	0.9572	0.9573
0.0094	18.9258	7400	0.2783	0.9583	0.9584
0.0094	19.4373	7600	0.2819	0.9583	0.9584
0.0094	19.9488	7800	0.2804	0.9582	0.9583

Framework versions

• Transformers 4.41.2
• Pytorch 2.1.2
• Datasets 2.19.2
• Tokenizers 0.19.1