w2v-bert-2.0-CV_Fleurs_AMMI_ALFFA-sw-1hrs-v1

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.6602
• Wer: 0.2878
• Cer: 0.1026

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
2.983	1.0	36	1.7051	0.9515	0.3287
1.0291	2.0	72	0.9615	0.4801	0.1423
0.8511	3.0	108	0.9519	0.3846	0.1234
0.6501	4.0	144	0.8922	0.3603	0.1195
0.6038	5.0	180	0.8740	0.3454	0.1152
0.4398	6.0	216	0.8208	0.3440	0.1185
0.3469	7.0	252	0.9068	0.3149	0.1077
0.2994	8.0	288	1.0320	0.3162	0.1079
0.3113	9.0	324	0.9206	0.3428	0.1234
0.2526	10.0	360	0.9717	0.3259	0.1124
0.201	11.0	396	0.9481	0.3150	0.1075
0.1847	12.0	432	1.1182	0.3107	0.1071
0.1647	13.0	468	1.0242	0.3073	0.1079
0.1512	14.0	504	1.0811	0.3072	0.1057
0.1276	15.0	540	1.0980	0.3179	0.1093
0.1089	16.0	576	1.1270	0.3233	0.1121
0.0936	17.0	612	1.1667	0.3084	0.1073
0.0782	18.0	648	1.1668	0.3236	0.1128
0.0792	19.0	684	1.1288	0.3346	0.1158
0.07	20.0	720	1.2678	0.3136	0.1104
0.0648	21.0	756	1.1550	0.3323	0.1172
0.0545	22.0	792	1.3441	0.3029	0.1070
0.0531	23.0	828	1.1785	0.3315	0.1149
0.0446	24.0	864	1.3664	0.3008	0.1082
0.0402	25.0	900	1.2758	0.3408	0.1158
0.0399	26.0	936	1.2697	0.3297	0.1158
0.0421	27.0	972	1.3760	0.3032	0.1072
0.0343	28.0	1008	1.3026	0.3242	0.1166
0.0295	29.0	1044	1.4635	0.3015	0.1071
0.0239	30.0	1080	1.5792	0.2977	0.1045
0.0247	31.0	1116	1.5481	0.3025	0.1074
0.0281	32.0	1152	1.4719	0.3137	0.1120
0.0215	33.0	1188	1.5960	0.3000	0.1070
0.0275	34.0	1224	1.6266	0.3012	0.1088
0.0622	35.0	1260	1.2910	0.3051	0.1068
0.0199	36.0	1296	1.4565	0.3035	0.1062
0.02	37.0	1332	1.4058	0.3064	0.1076
0.0155	38.0	1368	1.5731	0.2924	0.1043
0.015	39.0	1404	1.5122	0.3001	0.1077
0.0087	40.0	1440	1.5725	0.2854	0.1021
0.009	41.0	1476	1.5065	0.3007	0.1064
0.0125	42.0	1512	1.4685	0.3021	0.1072
0.0101	43.0	1548	1.5506	0.2953	0.1031
0.0106	44.0	1584	1.5395	0.3052	0.1083
0.0144	45.0	1620	1.5647	0.2896	0.1031
0.0098	46.0	1656	1.5343	0.2886	0.1032
0.0103	47.0	1692	1.6054	0.2918	0.1037
0.0143	48.0	1728	1.5473	0.2911	0.1054
0.0062	49.0	1764	1.6744	0.2869	0.1023
0.0056	50.0	1800	1.6602	0.2878	0.1026

Framework versions

• Transformers 4.48.0
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0