Mini-batch Sampling Strategies for Long-Tailed Image Classification

Code, per-run logs and figures for the paper:

Mini-batch Sampling Strategies for Long-Tailed Image Classification: An Empirical Study on CIFAR-100-LT Siyu Yuan, University of Bristol arXiv:2609.16365 · PDF

Four mini-batch samplers are compared under a controlled protocol on CIFAR-100-LT with ResNet-32, at imbalance ratios rho in {10, 50, 100} with three seeds each — 36 training runs in total.

The samplers themselves are not new — they were assembled and contrasted by Kang et al. (ICLR 2020). What this study adds is a bias–variance treatment of the mini-batch gradient estimator that yields falsifiable predictions, a controlled comparison in which every strategy sees the identical long-tailed subset and identical initial weights within a seed, per-group training dynamics rather than end-point numbers only, and a diagnosis of a failure mode that aggregate results obscure.

1. Progressive sampling helps the tail, not the average.

At rho = 100 it improves tail-class accuracy by 25% relative to the uniform baseline (13.5% vs 10.8%). Because all strategies share seeds, runs can be paired — and the paired differences separate a robust effect from noise:

(progressive minus uniform, percentage points)

The overall-accuracy lead is not robust; the tail-class gain is. The paper's conclusions rest only on the latter.

2. Class-balanced sampling backfires under severe imbalance.

At rho = 100 it is worse than uniform on every class group — including the tail classes it is designed to help (9.2% vs 10.8%). With only 5 samples per tail class, extreme oversampling inflates the variance of the tail-class gradient contribution by roughly two orders of magnitude. At the milder rho = 50, where the smallest classes hold 9 samples, the failure is confined to head and medium classes.

3. Sampler choice only matters when imbalance is severe.

The spread between the best and worst strategy is 1.6 points at rho = 10 and 6.0 at rho = 100.

Head = classes with more than 100 training samples, Medium = 20–100, Tail = 20 or fewer. All numbers are means over seeds {42, 123, 456}; the reported spread is population sigma.

CIFAR100LT_Experiments.py   All experiment code: CIFAR-100-LT construction, the four
                            samplers, ResNet-32, training loop, evaluation, and the
                            table/figure generation used in the paper.
CIFAR100LT_Run.ipynb        Notebook used to drive the runs.
results/                    36 per-run JSON logs + the generated .tex tables.
figures/                    Generated PNG figures.
paper/                      LaTeX source of the paper and its PDF figures.

pip install -r requirements.txt
python CIFAR100LT_Experiments.py

CIFAR-100 downloads automatically into ./data on first run (not tracked here). A full sweep is 36 runs of 200 epochs. Python 3.10 / PyTorch 2.1.

Determinism is fixed throughout — Python, NumPy and PyTorch seeds, cuDNN deterministic mode, and per-worker DataLoader seeding — so runs are reproducible on identical hardware. Within a seed, the dataset subset and the initial weights are shared across all four samplers, which is what makes the paired comparison valid.

One JSON per run, named results_<strategy>_rho<R>_seed<S>.json:

Naming caveat. Despite the prefix, final_overall_acc, final_head_acc, final_medium_acc and final_tail_acc are recorded at the epoch maximising test_acc, not at epoch 200. For genuine final-epoch values use the last element of the corresponding per-epoch list, e.g. test_acc[-1]. Appendix D of the paper reports both protocols and shows the difference is 0.06–0.30 percentage points, with the tail-class finding holding under either.

Minimal example:

import json, glob

runs = {}
for f in glob.glob("results/results_*.json"):
    d = json.load(open(f))
    runs[(d["strategy"], d["imbalance_ratio"], d["seed"])] = d

# tail accuracy at the best epoch, progressive vs uniform, rho = 100
for s in (42, 123, 456):
    p, u = runs[("progressive", 100, s)], runs[("uniform", 100, s)]
    print(s, round(p["final_tail_acc"] - u["final_tail_acc"], 2))

@misc{yuan2026minibatch,
  title         = {Mini-batch Sampling Strategies for Long-Tailed Image Classification:
                   An Empirical Study on {CIFAR-100-LT}},
  author        = {Yuan, Siyu},
  year          = {2026},
  eprint        = {2609.16365},
  archivePrefix = {arXiv},
  primaryClass  = {stat.ML},
  doi           = {10.48550/arXiv.2609.16365},
  url           = {https://arxiv.org/abs/2609.16365}
}

This work began as an undergraduate individual project in the School of Mathematics, University of Bristol, supervised by Dennis Prangle.

MIT — see LICENSE.