Learning across training horizons.

Synchronous and packed-8 AWC training with WSD. Five global token budgets, three runtime seeds, and checkpoint continuation.

Analysis & protocol ↗

Earlier cosine-study explorer · Audit & provenance

Winning configurations

Final full-validation mean ± sample SD over seeds 42, 43, and 44. Later winners are conditional on earlier LR pruning; continued horizons share training history.

Method	Horizon	LR	β₁	β₂	Mean ± sample SD	Seed results
Synchronous	20	0.004	0.95	0.98	3.556699 ± 0.002045	90
Synchronous	40	0.004	0.95	0.98	3.452800 ± 0.003136	54
Synchronous	80	0.003	0.95	0.999	3.374770 ± 0.003080	54
Synchronous	120	0.003	0.95	0.999	3.339164 ± 0.003378	36
Synchronous	160	0.003	0.95	0.999	3.317091 ± 0.003773	36
Packed-8 AWC	20	0.006	0.95	0.999	3.593128 ± 0.004363	108
Packed-8 AWC	40	0.006	0.95	0.999	3.476282 ± 0.002687	72
Packed-8 AWC	80	0.003	0.95	0.999	3.395320 ± 0.004652	72
Packed-8 AWC	120	0.003	0.95	0.999	3.356347 ± 0.003996	36
Packed-8 AWC	160	0.003	0.95	0.999	3.332262 ± 0.003611	36

All seed results CSV · All rankings CSV · Dataset JSON · Comparison figure PDF

Winners across horizons

Separately tuned configurations; bars show sample SD over three seeds.

Selected mean gap

Positive values favor synchronous training. Winner settings may differ.

Inspect a horizon

MethodHorizonLearning rateBeta 1Beta 2

Packed-8 AWC · horizon 20: 36 of 36 configurations, 108 complete seed results. Entering LR ceiling: 0.01. Global tokens: 408,027,136.

Ranks apply only within this method and horizon. All eligible triples include seeds 42, 43 and 44. Pruned LRs never re-enter the search.

Rank
1	0.006	0.95	0.999	3.593128	0.004363
2	0.008	0.95	0.999	3.595219	0.006842
3	0.004	0.95	0.999	3.598206	0.004634
4	0.01	0.95	0.999	3.598368	0.008263
5	0.003	0.95	0.999	3.606610	0.007636
6	0.006	0.9	0.999	3.608488	0.002933
7	0.008	0.95	0.98	3.615793	0.003917
8	0.01	0.95	0.98	3.616381	0.005264
9	0.004	0.9	0.999	3.617780	0.003443
10	0.006	0.95	0.98	3.618470	0.002558
11	0.01	0.9	0.999	3.619653	0.001403
12	0.008	0.9	0.999	3.620467	0.007288
13	0.004	0.95	0.98	3.625622	0.003120
14	0.003	0.9	0.999	3.636307	0.010322
15	0.003	0.95	0.98	3.636710	0.004510
16	0.006	0.9	0.98	3.639845	0.004906
17	0.008	0.9	0.98	3.641101	0.001965
18	0.002	0.95	0.999	3.642509	0.005807
19	0.01	0.9	0.98	3.645912	0.003872
20	0.004	0.9	0.98	3.657578	0.007070
21	0.002	0.9	0.999	3.666296	0.005473
22	0.003	0.9	0.98	3.672814	0.001524
23	0.002	0.95	0.98	3.674210	0.003280
24	0.008	0.95	0.9	3.676014	0.002169
25	0.004	0.95	0.9	3.678550	0.006849
26	0.01	0.95	0.9	3.679881	0.004471
27	0.006	0.95	0.9	3.680050	0.004974
28	0.003	0.95	0.9	3.690268	0.005680
29	0.002	0.9	0.98	3.703385	0.009365
30	0.002	0.95	0.9	3.711329	0.006905
31	0.006	0.9	0.9	3.729258	0.006999
32	0.008	0.9	0.9	3.731896	0.004661
33	0.01	0.9	0.9	3.736088	0.006061
34	0.004	0.9	0.9	3.753697	0.005456
35	0.003	0.9	0.9	3.768759	0.012779
36	0.002	0.9	0.9	3.793300	0.009331

Learning-rate response

Filtered beta pairs; mean ± sample SD. Click a point to inspect its seeds.

Full horizon grid

All beta pairs and original LR coverage, independent of filters. Click a measured cell to inspect it.

P = pruned · — = outside the original grid · ? = missing observations.

Grid coverage

β₁ / β₂	0.002	0.003	0.004	0.006	0.008	0.01
0.9 / 0.9	Measured	Measured	Measured	Measured	Measured	Measured
0.9 / 0.98	Measured	Measured	Measured	Measured	Measured	Measured
0.9 / 0.999	Measured	Measured	Measured	Measured	Measured	Measured
0.95 / 0.9	Measured	Measured	Measured	Measured	Measured	Measured
0.95 / 0.98	Measured	Measured	Measured	Measured	Measured	Measured
0.95 / 0.999	Measured	Measured	Measured	Measured	Measured	Measured

Selected configuration

Packed-8 AWC · horizon 20 · LR 0.006 · β₁ 0.95 · β₂ 0.999

Seed	Final full-validation loss	Job ID
42	3.588105793	2479658
43	3.595977846	2479659
44	3.595300243	2479660

Loss viewShow individual seeds

Recorded observations are joined without smoothing. Earlier terminal decays are excluded at continuation boundaries. Diamonds show final full-validation loss; the subset and training losses are different measurements.

Loading recorded curves…

Matched optimizer comparisons

Same horizon, LR, beta pair, and runtime seeds. Positive AWC − synchronous values favor synchronous training. SD describes paired seed differences, not a confidence interval.

LR	β₁	β₂	Sync mean	AWC mean	AWC − sync	Paired SD
0.004	0.95	0.98	3.556699	3.625622	0.068923	0.001143
0.003	0.95	0.98	3.560754	3.636710	0.075957	0.005122
0.006	0.95	0.98	3.563240	3.618470	0.055229	0.005050
0.004	0.95	0.999	3.567112	3.598206	0.031095	0.004094
0.004	0.95	0.9	3.567571	3.678550	0.110979	0.004367
0.008	0.95	0.98	3.568472	3.615793	0.047321	0.003922
0.006	0.9	0.98	3.568826	3.639845	0.071019	0.008073
0.003	0.95	0.999	3.569190	3.606610	0.037420	0.008363
0.002	0.95	0.98	3.569789	3.674210	0.104421	0.006927
0.003	0.95	0.9	3.571311	3.690268	0.118957	0.002893
0.004	0.9	0.98	3.571326	3.657578	0.086252	0.011687
0.006	0.95	0.9	3.572639	3.680050	0.107411	0.004960
0.008	0.9	0.98	3.573613	3.641101	0.067487	0.007670
0.006	0.9	0.9	3.574786	3.729258	0.154472	0.004847
0.002	0.95	0.999	3.575370	3.642509	0.067139	0.008593
0.002	0.95	0.9	3.576116	3.711329	0.135213	0.007406
0.006	0.95	0.999	3.576976	3.593128	0.016152	0.005942
0.004	0.9	0.999	3.579804	3.617780	0.037976	0.007419
0.008	0.95	0.9	3.581085	3.676014	0.094929	0.000683
0.004	0.9	0.9	3.582076	3.753697	0.171621	0.004222
0.003	0.9	0.98	3.583236	3.672814	0.089578	0.005461
0.008	0.9	0.9	3.585790	3.731896	0.146106	0.004229
0.003	0.9	0.9	3.588140	3.768759	0.180619	0.008033
0.002	0.9	0.98	3.590214	3.703385	0.113171	0.002232
0.003	0.9	0.999	3.592575	3.636307	0.043732	0.007245
0.006	0.9	0.999	3.597926	3.608488	0.010563	0.040239
0.002	0.9	0.999	3.600230	3.666296	0.066066	0.005472
0.002	0.9	0.9	3.602496	3.793300	0.190804	0.006478
0.008	0.95	0.999	3.603980	3.595219	-0.008762	0.023035
0.008	0.9	0.999	3.823774	3.620467	-0.203307	0.394059