Teaching Small LLMs What to Ignore in Noisy Multi-hop QA

This repository implements a practical pipeline for training a small language model to answer multi-hop questions while ignoring distracting context.

The project targets noisy multi-hop QA on HotpotQA distractor-style data with a compact model:

• Base model: Qwen/Qwen2.5-0.5B
• Finetuning: QLoRA (4-bit) + LoRA adapters
• Budget: single GPU workflow
• Focus: robustness via harder negatives, evidence supervision, and curriculum learning

flowchart LR
    A[HotpotQA samples] --> B[Preprocess + flatten context]
    B --> C[Build data variants]
    C --> C1[Clean]
    C --> C2[Random negatives]
    C --> C3[Hard negatives]
    C3 --> D[Answer-only SFT: E2]
    C3 --> E[Answer + Evidence SFT: E3]
    C1 --> F[Curriculum stage 1]
    C2 --> G[Curriculum stage 2]
    E --> H[Curriculum stage 3]
    F --> I[E4 curriculum model]
    G --> I
    H --> I

Full inference-side EM/F1 is implemented but not completed in this local run.
The table below reports available training-side proxy metrics from outputs/metrics/*.json.

Notes:

• E2 vs E3/E4 train loss is not directly comparable due to different output objectives.
• E3/E4 low eval loss is measured on evidence-formatted validation targets.

.
├── src/                  # Core modules: preprocessing, noisy-data building, training, eval, plotting
├── scripts/              # CLI entry points (train/evaluate/report artifacts)
├── notebooks/            # End-to-end workflow notebooks (01~06)
├── report/               # LaTeX report draft
├── outputs/              # Metrics/tables/figures artifacts (generated)
├── data/                 # Raw/processed/formatted data (generated)
└── checkpoints/          # LoRA checkpoints/adapters (generated)

Python 3.10+ is recommended.

python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install torch transformers peft datasets matplotlib bitsandbytes

Main config file:

• src/config.json

Important defaults:

• max_input_length=2048
• max_output_length=160
• train_subset_size=1200, val_subset_size=200, test_subset_size=200
• lora_r=8, lora_alpha=16, lora_dropout=0.05
• learning_rate=2e-4, num_epochs=1
• evidence_loss_weight=2.0

Unified CLI:

python scripts/train.py --experiment e0
python scripts/train.py --experiment e1
python scripts/train.py --experiment e2
python scripts/train.py --experiment e3
python scripts/train.py --experiment e4

Shortcut CLIs:

python scripts/train_e0.py
python scripts/train_e1.py
python scripts/train_e2.py
python scripts/train_e3.py
python scripts/train_e4_curriculum.py

Run full answer/evidence evaluation:

python scripts/evaluate_models.py \
  --models e0 e1 e2 e3 e4 \
  --settings clean_answer_only random_answer_only hard_answer_only hard_answer_evidence

Run robustness curves (optional):

python scripts/evaluate_models.py \
  --run-robustness \
  --robustness-models e0 e2 e4 \
  --robustness-neg-counts 0 1 2 3 4 \
  --robustness-negative-type hard

Standard mode (uses outputs/metrics/eval_summary.json if present):

python scripts/build_report_artifacts.py

Proxy mode (training-only tables/plots):

python scripts/build_report_artifacts.py --force-proxy

The notebook sequence is:

1. notebooks/01_setup_and_data_check.ipynb
2. notebooks/02_preprocess_hotpotqa.ipynb
3. notebooks/03_build_noisy_and_hardneg_data.ipynb
4. notebooks/04_train_baselines_and_evidence.ipynb
5. notebooks/05_evaluate_and_plot.ipynb
6. notebooks/06_report_tables_and_figures.ipynb

• This repo uses .gitignore to exclude large artifacts (checkpoints, images, generated datasets).
• If you clone from GitHub, you may need to regenerate data/, outputs/, and checkpoints/ locally.

• The currently committed artifacts emphasize training-side proxy metrics.
• Final task-level claims should rely on full inference outputs:
  • Answer EM/F1
  • Evidence Precision/Recall/F1
  • Robustness curves over increasing negatives

Full PDF: Teaching Small LLMs What to Ignore