Raw outputs and scripts are in the GitHub repository (Apache-2.0).

1 · Sparse synchronisation, toy model DiLoCo

A byte-level GPT with 3.3M parameters, 4 simulated workers, 500 steps per worker, 3 seeds, outer optimiser lr 1.0 / momentum 0.5. We tried two datasets: a small local Turkish text (256 KB) and the first 3,280 articles of Turkish Wikipedia (30 MB). Values are the validation-loss gap to the reference run (synchronised every step), in nats; lower is better.

Reference loss: 1.532 on the local text, 1.919 on Wikipedia.

Limit: small model; the outer optimiser was tuned only at H=25. On Wikipedia the effect of H is the same size as seed noise, so we do not claim "it gets steadily worse as H grows". A 12-setting outer-optimiser sweep found no better point.

2 · Qwen3-0.6B Turkish adaptation LoRA

Qwen3-0.6B-Base, LoRA r=16 (all linear layers), 215 steps ≈ 1.76M tokens, RTX 4060 Laptop (8 GB), ≈ 5.6 s per step, peak memory 5.23 GB, adapter 40.4 MB. Turkish evaluation is on a Wikipedia slice not used in training.

Reading: Turkish language modelling improved, English shows a slight loss, and the TurkishMMLU difference is 8 questions, within the standard error (≈ 1.6 points) of a 900-question test. We cannot say "the model improved". Limit: single run; the evaluation text is from the same domain as training.

3 · Adapter DiLoCo simulation

Qwen3-0.6B + LoRA, 4 simulated workers (base shared and frozen), 8 rounds × 25 local steps. Baseline: same total compute, synchronised every step. Metric: Turkish bits per byte on a 60k-token held-out slice.

Final gap 0.0011 (0.09%); ≈ 98% of the baseline gain. Communication (computed): baseline 200 steps × 4 workers × 40.4 MB ≈ 32 GB, DiLoCo 8 rounds ≈ 1.3 GB. Peak memory 5.64 GB.

Limit: single seed, small held-out slice, data slices from the same stream (real volunteers will differ), no network latency, worker loss or malicious contributions, outer optimiser not tuned for this model, compatibility with later standard training not tested.