Skip to content

Dashboard Agent V1 — chat, reports, Investigate, Watch - #4418

Draft
kathiekiwi wants to merge 208 commits into
mainfrom
feat/dashboard-agent-flows
Draft

Dashboard Agent V1 — chat, reports, Investigate, Watch#4418
kathiekiwi wants to merge 208 commits into
mainfrom
feat/dashboard-agent-flows

Conversation

@kathiekiwi

@kathiekiwi kathiekiwi commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

An AI assistant in a side panel on every dashboard page, behind the dashboard-agent feature flag. It reads runs, errors, queues, deploys and health through the public API (read-only, delegated user token), answers with rich cards, and can keep watching things after the conversation ends.

What's inside

  • Foundation@internal/dashboard-agent-contracts (trigger:// URI grammar, intents, watch specs, block envelope), investigations + watches tables, head-start reliability fix, eval sample-rate gate.
  • Reportsget_report renders the deterministic health report as a card (metric grid, sparklines, Next steps button row); stale telemetry is flagged and never trusted for advice.
  • Investigate — hypothesis-driven investigation on a live card with system-owned identity and revisions; entry buttons on failed runs, errors, backed-up queues and waiting runs; code-grounded when a repo is connected; server-generated follow-ups (Show code, View similar, Watch for a repeat).
  • Watch (Dashboard Agent: Watch (background condition watches + wake notifications + alerts) #4456) — one-shot background watches with a compact creation card on run/queue/error/health pages: resolution + observed outcome model, exactly-once wake delivery, expiry sweep, optional investigate-on-attention, standing email/Slack/webhook alerts with one-click unsubscribe. Queue conditions: drain, above/below N, stalled, oldest-age SLA.
  • UI — blank-state hero (Ask AI) with a Tab-to-accept placeholder and colored smart prompts, fullscreen mode, one persistent agent spinner across all phases.
  • Smart prompts — page-aware chips on every env page (37 routes, 24 page kinds); investigate/status chips appear only on loader-backed abnormal state.
  • Tooling — navigation, TRQL queries with live charts, deploy correlation, docs answers; golden eval suite; seeder for a live playground project (db:seed:agent-examples, with --heartbeat / --degrade / --recover for demos).

How to review

GUIDEBOOK.md — 10-minute local setup and a hands-on walkthrough of every case. Component gallery at /storybook/agent-ui.

Notes

  • Everything is gated by canAccessDashboardAgent; no behavior change with the flag off.
  • The seeder's --heartbeat mode is a review-stand crutch and will be removed before merge.

ericallam added 30 commits July 13, 2026 20:38
…signals

Gauges are read inside the enqueue/dequeue Lua and returned on the script reply
as a 2-tuple; counters are cumulative odometers. The run-queue Redis carries no
metrics stream of its own.
…counters

entryOrderKey returns a string built with BigInt math so ordering stays correct at real epoch magnitudes. Odometer keys are namespaced by definition name. The consumer reports null lag for a missing consumer group instead of 0, and empty gauge values parse as NaN rather than 0.
…ng order keys

The wait-time quantile materialized view now excludes wait_ms = 0 rows so it matches the count aggregation. order_key accepts a string or a number. Migration comments no longer contain semicolons that split the migration into invalid statements.
…rride

The queues list tolerates a metrics query failure by rendering without metrics and logging a warning. UsageSparkline renders its total override even when every bucket is zero. The queue detail page returns 404 and its loader skips the metrics query when the feature flag is off. The seed script validates bucket size and only writes ClickHouse against a local host.
A bucket-led ORDER BY DESC combined with fillGaps emitted an ascending WITH FILL (positive step, ascending bounds), which produces invalid or empty fills. Skip the gap-fill rewrite for descending orders and let the plain descending query stand. Adds a DESC fillGaps test.
Packs the stream sequence with a 1e6 factor (was 1e5) so up to 1M entries per millisecond per shard fit before a seq could spill into the next millisecond's range, far above what a single Redis stream can produce. ms*1e6 stays within UInt64. Also fixes the webapp mapping test that still expected a numeric order_key after the switch to a BigInt-derived string.
The queues list and queue detail pages now use the shared TimeFilter (any preset period or a custom date range) and everything on the page follows it: header tiles, per queue metric columns, charts, and stats. The custom period buttons, hand rolled chart cards, and duplicated metric fetch loops are replaced by the ChartCard and Chart primitives, UsageSparkline, and a shared useMetricResourceQuery hook. The ClickHouse list queries take an explicit end bound so fixed ranges query only their window.
Queries using deltaSumTimestampMerge failed with an unknown function error, which broke the queue detail stats and the started counts on the built in Queues dashboard.
The queues list header tiles now render the same line chart, grid, and tooltip as the rest of the metrics charts instead of a row sparkline, with the headline value in the tile header. The env saturation tile draws the environment concurrency limit and burst limit as labeled reference lines. Chart tooltips keep a gap between the series label and the value, and the shared line chart gains showDots and referenceLines options.
Adds an Allocation tab to the Queues page (behind the queue metrics UI flag): overview cards, a burst-aware capacity bar showing each queue allocation and its live usage in a distinct color, an inline-editable limits table with per-queue locks, load-weighted auto-balance, and a review dialog that bulk-applies limits as overrides through the existing concurrency system.

The queue list now defaults to Busiest ordering (with Backlog and Name options). ClickHouse ranks queues by activity over the last 15 minutes and returns just the requested page of names, so the cost per page is one small aggregate regardless of environment size; idle queues follow in name order and any failure falls back to name ordering. The classic page keeps plain name order.
The fallback WHERE injection only targeted the top-level SELECT, so a
query shaped as an outer aggregation over a FROM subquery failed to
compile: the time column only exists inside the subquery. Descend into
the subquery so the fallback lands next to the table reference.
Adds two rollups fed from the raw landing table: a per-queue 5-minute
tier and an environment-level 1-minute tier (gauges plus TDigest wait
quantiles). Ranking now reads the 5m tier and returns the page and the
ranked total in one windowed query instead of two scans.

The 5m materialized view reads raw rather than cascading off the 10s
table: deltaSumTimestamp states hold a single first/last segment, so
merging states in an MV's hash-ordered GROUP BY double-counts bridging
spans. For the same reason the env tier carries no counter columns, and
env-wide counter totals must group by queue before summing.
The built-in queues dashboard's enqueued vs started chart merged counter
states across queues, which mixes unrelated cumulative counters and
returns wrong totals; it now merges per queue and sums outside. Env
header tiles and saturation charts read the environment rollup, so their
cost no longer scales with queue count, and coarse-bucket ranges are
served from the 5m rollup automatically. Queue list ranking runs as one
query, time bounds are aligned to the bucket grid, and repeated
auto-refresh reads share ClickHouse query-cache entries.
… rollup

The env rollup's win comes from dropping the queue dimension, not from
coarser buckets: row count is queue-independent (~8640/day/env), so full
10-second granularity stays cheap at any range. Env header tiles and
saturation charts now resolve short-range detail exactly like the
per-queue charts, and the current-value tiles read the latest 10-second
bucket instead of a minute-wide one.
The simulator's --reset only cleared the raw and 10s tables, leaving
stale rows in the 5m and env rollups. It also force-merges the rollups
after seeding so current-value widgets read cleanly.
…view 1-4, 7)

- the tick claim is resumable: a retry of its own generation re-runs the
  tick safely (successor dedups on the stable key), so a crash between
  claim and reschedule can't strand the chain
- the sweeper recovers lost deliveries through the real query: terminal
  rows with pending delivery past a grace are swept too (new partial index)
- finalization moved into the webapp: overdue watches get one last
  authorized check — access_revoked cancels with no wake, satisfied at the
  buzzer fires; the agent-side direct-DB sweeper is deleted
- delete and create serialize on the same per-chat lock; createWatch
  re-checks deletedAt inside its transaction
- an inline-resolved watch stays pending until the narration is persisted;
  the recovery sweep delivers the wake if the turn died first
- fixes a latent bug: listExpiredActiveWatches threw on every call with a
  bound Date (postgres-js), which the old fake-driven test never saw
… diff (round-3 review 5-6)

- a source evidence ref canonicalizes only when the read ledger proves the
  file was read at that exact commit this turn — the snapshot sha never
  stands in; unread citations fail the render by name
- the Show code prompt demands one fenced diff with the minimal change,
  anchored file:line@sha, with the dirty-snapshot caveat; eval case added
…times (round-4 review)

- delivery is an atomic claim (pending -> delivering -> delivered): of two
  concurrent same-generation ticks only the winner appends the wake; a
  deliverer that dies leaves a stale claim the sweep recovers
- inline outcomes prove narration from the transcript itself: the wake
  narrator skips prose only when the persisted history shows this watch's
  schedule_watch output followed by an assistant answer — an unrelated
  later message is not proof
- the tick claim no longer stamps lastCheckedAt; observation times are
  written only with a real result
- losing the agent configuration no longer freezes the lifecycle: sweep
  finalization always runs, only the delivery hand-off waits for config
…on proof (round-5 review)

- every delivery claim carries a unique claim id; release and the fenced
  delivered-mark match on it, so a hung deliverer that was taken over can
  no longer release or complete the new owner's claim (the unfenced mark
  is pending-only, for the one caller that never claims)
- inline-narration proof requires assistant prose after this watch's
  schedule_watch part within the same message — a later unrelated answer
  or text before the tool part no longer suppresses the wake
The base agent branch now ships without watches: schedule_watch, the tick
loop, wake delivery, the expiry sweep, watch alerts (email template, alert
type, unsubscribe), the watches table and its migrations, and every UI
surface (chips, wake banner, toast, unread dot, watching status) are gone.
The complete feature lives on feat/dashboard-agent-watch, stacked on this
branch. The review stand (seeder, heartbeat, guidebook) stays here.
@kathiekiwi kathiekiwi changed the title Dashboard Agent: investigations, watches, alerts, and notifications Dashboard Agent: UI & investigations Aug 1, 2026
kathiekiwi and others added 20 commits August 2, 2026 21:09
…ays built but dormant

Plan detection moves to the billing service (plan-defined limit, billing-period
window, separate ticket). The counter, upgrade block and submit guard remain
wired; with no plan answer the quota resolves to unlimited.
- waiting-run diagnosis: 'unknown' with concurrency evidence in hand no longer
  claims the evidence is missing; an elapsed delay says 'not yet enqueued'
  instead of hiding behind time-from-creation
- queue metrics route: drop the double decode that 500ed on names with a
  literal percent sign
- evidence schema: kind must match the URI's own kind
- seed-queue-metrics: default-binding imports like the other seeders
… runs (TRI-12862)

The queue detail page offers Investigate when the queue is at capacity with
runs waiting or the head-of-line wait passes the existing warning threshold;
the run page's waiting widget offers it whenever it renders. Both post the
visible request in the user's own voice through the existing button.
…, gallery matrix (TRI-12862)

- a card left in_progress when the turn ends is force-settled to inconclusive
  (evidence and hypotheses kept, remediation dropped, honest headline note)
  before the turn persists — a refresh can never read a spinner that never stops
- canonicalization throws surface as named tool errors instead of escaping
- protocol rule: a cause names a mechanism; a restatement behind 'because' is
  not a verdict (eval case with tempting mechanism-free evidence)
- gallery: in_progress-early, concluded-not-code-grounded, degraded-after-
  tool-failure fixtures; the two concluded cells contrast server-decided actions
The route's schemas and helpers moved to reportsApi.server.ts — non-loader
route exports that reach server-only modules fail the vite build (the e2e
jobs' failure), which typecheck doesn't catch.
…cess

The env layout loader queried the feature flag unconditionally; without
agent access the panel never mounts, so the read was wasted — and main's
new environment-ownership test (which stubs a minimal prisma) caught it.
- the empty chat centers a hero: sparkles icon, 'Ask Trigger' at blank-slate
  title size with the Beta badge, a one-line subtitle, a three-row composer
  with the send button inside, and the suggested prompts as a wrapping row
  of buttons colored by meaning (action indigo, status secondary, explain
  tertiary, docs the docs style) — one slot-to-variant mapping
- an Expand button next to Close takes the panel over everything right of
  the nav bar, like a page: no route, no modal, no remount — the chat keeps
  its transport and draft text; content stays mounted underneath; the
  transcript column gets a prose max-width; the preference persists
- storybook: hero states at panel and fullscreen widths
…op geometry

- the blank state's field shows the top suggested prompt as its placeholder;
  Tab drops it into the field as editable text, never sending it
- the send and stop buttons share identical square geometry
… land

- cmd+J is contextual: closed opens the panel, open starts a new chat;
  closing is Esc or the header's x — the New chat tooltip now shows cmd+J
  (displayed once, registered once)
- in-flight tool work renders as a bare spinner line, not a bordered pill —
  chips are for artifacts that stay, progress is transient
- error evidence and navigate targets normalize the API's friendly id to
  the raw fingerprint, so View similar failures opens the error page
  instead of 'Error not found'
The header button is the ask-ai Button variant (dot-matrix logo, 'Ask AI');
the hero title follows suit.
…er the agent works

One component in the spinner primitives; chat progress, pending tools, the
history thinking/watching markers, panel loading, chart loading and testing
hypotheses all use it, so agent activity reads as the agent rather than
generic loading.
- AgentSpinner rests on the playlist's first shape, so mounting shows no
  logo-head flash — a spinner is born spinning
- the pending indicator keeps one stable element across tool changes: the
  label swaps, the animation never restarts
…estarts

The pending tool line, the generic activity row and the investigation card's
own progress collapse into a single ChatProgress mounted once at the end of
the live turn: phases only swap its label (card phrase > tool phrase >
activity), decided in the pure progress-line module. ChatPendingTool is gone;
the card renders no spinner of its own; AgentSpinner has exactly one live
render site.
Only the runs list and run detail described themselves to the dashboard
agent, so every other page fell to "other" and offered generic chips.
Add handle mappers for the errors list, an error group, the queues list,
a queue, the deployments list and a deployment — loader data only, no
added queries — plus list page kinds in the contracts and an optional
deployment status. Investigate chips now appear for an unhealthy queue
and a deploy that didn't land.
Only the runs, errors, queues and deployments pages described themselves
to the dashboard agent; everything else fell to "other" and offered the
generic chips. Add handle mappers for the remaining 37 env-scoped routes
and 24 page kinds in the contracts, so each page offers an explain and a
docs question about what it actually shows.

Investigate and status chips stay gated on loader data: a scheduled task
with no schedule attached, all its schedules disabled, a paused queue, a
batch whose runs failed, a wait token past its timeout, a bulk action
still running, a spent quota, a prompt pinned to an override, a session
whose run failed. Loader data only, no added queries, no new signals.
…ions + alerts) (#4456)

Stacked on #4418 — the diff against that branch is the complete Watch
feature, extracted so the base agent PR can land without it.
@kathiekiwi kathiekiwi changed the title Dashboard Agent: UI & investigations Dashboard Agent V1 — chat, reports, Investigate, Watch Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants