ThoughtLoop

Deliberate problem solving for coding agents.

Think wider. Build better. Prove it.

ThoughtLoop is one installable Codex skill pack with one adaptive orchestrator, shared graph contracts, and independently callable capability nodes. It searches when alternatives matter, makes tradeoffs explicit, executes economically, and proves results with independent evidence.

The architecture is intentionally simple:

Discover -> Decide -> Execute -> Prove

The stages are adaptive guidance, not a mandatory ceremony. Small tasks stay small. High-risk work gets deeper discovery and stronger proof.

• Why ThoughtLoop
• Canonical graph
• How it works
• Independent nodes
• What's included
• Subagent mode
• Quick start
• Install
• Validate
• Roadmap
• Contributing
• Security
• License
• Acknowledgments

Coding agents tend to fail in two opposite ways: they patch the first plausible idea, or they spend too long exploring without converging. ThoughtLoop gives the agent a small set of boundaries that keep judgment proportional to the task.

flowchart LR
    D[Discover] --> S[Decide] --> E[Execute] --> P[Prove]
    P -. backtrack at the right depth .-> S

The four stages answer different questions:

1. Discover: Which materially different approaches or explanations are worth considering?
2. Decide: What should we do next, and which tradeoffs or unknowns could change that choice?
3. Execute: What is the smallest coherent change that satisfies the contract?
4. Prove: What evidence supports the result, what failed, and what remains unknown?

flowchart TD
    T[Task] --> G{Unknowns or material choice?}
    G -->|No| B[Execute]
    G -->|Yes| F[Gapfinder]
    F --> D[Discover or Investigate]
    D --> S[Decide]
    S --> B
    B --> V[Verify]
    V --> R{Review warranted?}
    R -->|Yes| X[Review]
    R -->|No| O[Final outcome]
    X --> O
    O --> M{Failure or unknown?}
    M -->|Yes| Q[Correct]
    Q --> D

The graph is a set of reusable public nodes plus internal operations owned by the orchestrator. The installed contracts in skills/thoughtloop/references/contracts.md carry observable state; the installed routing policy in skills/thoughtloop/references/routing.md defines the boundaries.

Execution, final judgment, correction, and loop evaluation are internal operations owned by thoughtloop, not separately installed skills.

Every canonical node can be invoked directly when a task needs one capability. The node can also receive state from thoughtloop through the shared contracts. There is one public name per responsibility; modes belong inside the owning node rather than being exposed as duplicate compatibility skills.

Only thoughtloop permits implicit invocation. Its boundary is consequential, decision-sensitive coding work—not coding vocabulary alone. Simple explanations, formatting, trivial edits, routine commands, and tasks already governed by a more specific skill remain explicit or stay with the more specific skill.

Subagent mode is opt-in and budget-aware:

$thoughtloop Use bounded subagents with a balanced budget to redesign this caching layer.

The parent agent keeps ownership of the contract, decisions, edits, evidence synthesis, and final result. Delegated work stays narrow and starts with fresh context. Use lower-cost agents for bounded search or mechanical checks; reserve stronger reasoning for difficult tradeoffs, disagreements, and final decisions.

Delegation is never required. If it cannot add independent signal, keep the work in the parent agent.

Invoke the orchestrator inside Codex:

$thoughtloop Reduce p99 latency in this parsing service without changing its external contract. Explore materially different approaches only if they could change the result, then implement and prove the choice.

Use a specialist directly when you want one stage:

$discover Search for materially different ways to reduce p99 latency.
$investigate Measure the performance bottleneck without editing the repository.
$verify Verify whether this implementation satisfies the stated behavior.
$review Try to break this refactor after its ordinary tests pass.

See the examples/ directory for coding, research, and constrained-writing workflows.

This repository is packaged as a skills-only plugin. The manifest is at .codex-plugin/plugin.json, and the local marketplace entry is at .agents/plugins/marketplace.json.

Install it using your Codex plugin workflow, then invoke $thoughtloop in a task that benefits from deliberate search and verification.

Codex discovers user skills from $HOME/.agents/skills. Install all skills from this checkout with:

python scripts/install_local.py

Use --copy for independent copies and --force to replace existing skill paths. Remove the installed skills with:

python scripts/uninstall_local.py

Run the pack checks locally:

python tests/validate_pack.py
python tests/validate_graph.py
python scripts/calculate_metrics.py examples/sample-loop-log.jsonl

The validator uses only the Python standard library. The metrics script reports signals such as exploration, revisions, regressions, unknowns, cost, tokens, and runtime. Treat those metrics as diagnostic signals, not as a single quality score.

The same checks run in GitHub Actions for pushes and pull requests.

Structural validation proves pack invariants; it does not prove that an agent will choose the right route for every prompt. The labeled corpus in evals/cases.jsonl covers activation, route depth, authorization boundaries, missing evidence, review findings, backtracking, delegation budgets, and standalone operation.

Validate the corpus without invoking an agent:

python scripts/run_behavioral_evals.py --validate-only

Run model-backed traces with a host command when the environment provides one. The runner accepts JSON, JSONL, or noisy host output and scores only observable fields. Missing fields remain UNKNOWN:

python scripts/run_behavioral_evals.py --runner codex exec --output evals/runs/local.json

Use repetitions and a paired control when testing an instruction change:

python scripts/run_behavioral_evals.py \
  --runner codex exec \
  --control-runner codex exec \
  --repetitions 3 --output evals/runs/paired.json

The output includes dimension-level results for activation, route compliance, verdict policy, and delegation limits. A completed baseline can be created with --write-baseline path, which refuses to overwrite an existing file.

The runner captures prompts, observable route/verdict fields when the host returns structured JSON, bounded stdout/stderr, delegation observations, and runtime. It does not claim a pass rate when the host emits no structured observation. The first release baseline is kept in evals/baselines/2.0.0.json and remains pending until a real model-backed run is captured.

ThoughtLoop is deliberately small. The next useful improvements are:

• add an independent qualitative judge for genuinely non-deterministic criteria;
• add more pressure scenarios for urgency, sunk cost, ambiguity, and unavailable evidence;
• publish tagged releases as the pack's contracts stabilize.

Open an issue if you have a concrete use case or evidence that should change the design.

See CONTRIBUTING.md for the repository invariants, validation commands, and pull request expectations.

ThoughtLoop contains instructions that can influence an agent's actions. Do not add secrets, private logs, credentials, or untrusted instructions to skills or examples. See SECURITY.md before reporting a security concern.

ThoughtLoop is released under the MIT License. See LICENSE.

The README structure was adapted from Best-README-Template, one of the most widely used README templates on GitHub. The ThoughtLoop content, workflow contracts, and validation tools are specific to this project.

Back to top