Skip to content

chat: Add experiment-backed MCP authentication hints setting - #336484

Draft
Alexandru Dima (alexdima) wants to merge 1 commit into
mainfrom
agents/suppress-mcp-message-in-vscode
Draft

Alexandru Dima (alexdima) wants to merge 1 commit into
mainfrom
agents/suppress-mcp-message-in-vscode

Conversation

@alexdima

Copy link
Copy Markdown
Member

Summary

Recurring proactive MCP authentication reminders can interrupt otherwise unrelated conversations. Add a way to suppress only these reminders so we can evaluate their effect on retention without disabling MCP or bypassing authentication.

  • Add application-scoped chat.mcp.authenticationHints.enabled, defaulting to true, with experiment: { mode: 'auto' }. The shared registration covers editor chat and the Agents Window.
  • Gate new proactive reminders and update existing reminders when the setting changes. Suppression does not mark a server as authenticated or a reminder as actioned.
  • Preserve silent authentication, manual sign-in, authentication prompts blocking actual tool calls, and progress for user-initiated authentication already in flight.
  • Record chat.mcp.authenticationHintEligible once per open conversation in both variants. Its only event-specific property is the effective hintsEnabled boolean; it records no server names, URLs, credentials, or conversation content.

Experiment / rollout

Treatment variable: config.chat.mcp.authenticationHints.enabled

  • Control: true (existing behavior)
  • Treatment: false (suppress proactive reminders)

Explicit user settings take precedence over experiment defaults. The eligibility event supports identifying the affected population in either arm instead of selecting only users who saw a reminder. Review the new telemetry event before rollout, and target only builds containing this change. This PR does not create or enable a remote experiment and does not change the default experience.

Validation

Rebased onto current main (de1a430a2f3).

  • npm run gulp compile-client — passed, including client type-checking.
  • Hygiene and lint on all six changed files — passed.
  • 16 focused Electron unit tests — passed, covering default/enabled/disabled behavior, eligibility deduplication, live configuration changes, preserved silent and tool-call authentication, and existing reminder rendering.
./scripts/test.sh \
  --run src/vs/workbench/contrib/chat/test/browser/agentSessions/agentHostChatContribution.test.ts \
  --run src/vs/workbench/contrib/chat/test/browser/widget/chatContentParts/chatMcpAuthenticationContentPart.test.ts \
  --run src/vs/workbench/contrib/chat/test/browser/agentSessions/stateToProgressAdapter.test.ts \
  --grep 'mcp auth prompt|ChatMcpAuthenticationContentPart|authentication-required invocation'

Try locally

Set "chat.mcp.authenticationHints.enabled": false to suppress reminders. Toggle it while a reminder is visible to verify that hiding it does not authenticate the server, and that authentication required by an actual tool call is still actionable.

To exercise experiment assignment instead, leave the setting unset and use:

"experiments.override": {
  "config.chat.mcp.authenticationHints.enabled": false
}

Allow proactive MCP authentication reminders to be suppressed without changing silent sign-in or authentication prompts for tool calls. Record eligibility in both variants to support a retention experiment.

Co-authored-by: Copilot <[email protected]>
Copilot AI balanced review requested due to automatic review settings September 16, 2026 20:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The setting is narrowly applied, preserves required authentication paths, and has comprehensive focused coverage.

Review effort: Balanced (auto)
Findings: None

Note

Copilot is running an experiment and ran this review at Balanced.

What changed in this PR

Adds experiment-backed suppression of proactive MCP authentication reminders while preserving required authentication flows and measuring eligibility.

Changes:

  • Registers the application-scoped authentication-hints setting.
  • Gates reminder creation and visibility on live configuration.
  • Adds telemetry deduplication and focused behavioral tests.
File Description
src/​vs/​workbench/​contrib/​chat/​common/​constants.ts Defines the setting key.
src/​vs/​workbench/​contrib/​chat/​browser/​chat.shared.contribution.ts Registers the experimental setting.
src/​vs/​workbench/​contrib/​chat/​browser/​agentSessions/​agentHost/​agentHostSessionHandler.ts Gates reminders and records eligibility telemetry.
src/​vs/​workbench/​contrib/​chat/​browser/​widget/​chatContentParts/​chatMcpAuthenticationContentPart.ts Updates existing reminder visibility dynamically.
src/​vs/​workbench/​contrib/​chat/​test/​browser/​agentSessions/​agentHostChatContribution.test.ts Tests gating, telemetry, and preserved authentication behavior.
src/​vs/​workbench/​contrib/​chat/​test/​browser/​widget/​chatContentParts/​chatMcpAuthenticationContentPart.test.ts Tests reminder rendering and live suppression.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants