chat: Add experiment-backed MCP authentication hints setting - #336484
Draft
Alexandru Dima (alexdima) wants to merge 1 commit into
Draft
Alexandru Dima (alexdima) wants to merge 1 commit into
Alexandru Dima (alexdima) wants to merge 1 commit into
Conversation
Allow proactive MCP authentication reminders to be suppressed without changing silent sign-in or authentication prompts for tool calls. Record eligibility in both variants to support a retention experiment. Co-authored-by: Copilot <[email protected]>
Copilot started reviewing on behalf of
Alexandru Dima (alexdima)
September 16, 2026 20:58
View session
Contributor
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
The setting is narrowly applied, preserves required authentication paths, and has comprehensive focused coverage.
Review effort: Balanced (auto)
Findings: None
Note
Copilot is running an experiment and ran this review at Balanced.
What changed in this PR
Adds experiment-backed suppression of proactive MCP authentication reminders while preserving required authentication flows and measuring eligibility.
Changes:
- Registers the application-scoped authentication-hints setting.
- Gates reminder creation and visibility on live configuration.
- Adds telemetry deduplication and focused behavioral tests.
| File | Description |
|---|---|
src/vs/workbench/contrib/chat/common/constants.ts |
Defines the setting key. |
src/vs/workbench/contrib/chat/browser/chat.shared.contribution.ts |
Registers the experimental setting. |
src/vs/workbench/contrib/chat/browser/agentSessions/agentHost/agentHostSessionHandler.ts |
Gates reminders and records eligibility telemetry. |
src/vs/workbench/contrib/chat/browser/widget/chatContentParts/chatMcpAuthenticationContentPart.ts |
Updates existing reminder visibility dynamically. |
src/vs/workbench/contrib/chat/test/browser/agentSessions/agentHostChatContribution.test.ts |
Tests gating, telemetry, and preserved authentication behavior. |
src/vs/workbench/contrib/chat/test/browser/widget/chatContentParts/chatMcpAuthenticationContentPart.test.ts |
Tests reminder rendering and live suppression. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Recurring proactive MCP authentication reminders can interrupt otherwise unrelated conversations. Add a way to suppress only these reminders so we can evaluate their effect on retention without disabling MCP or bypassing authentication.
chat.mcp.authenticationHints.enabled, defaulting totrue, withexperiment: { mode: 'auto' }. The shared registration covers editor chat and the Agents Window.chat.mcp.authenticationHintEligibleonce per open conversation in both variants. Its only event-specific property is the effectivehintsEnabledboolean; it records no server names, URLs, credentials, or conversation content.Experiment / rollout
Treatment variable:
config.chat.mcp.authenticationHints.enabledtrue(existing behavior)false(suppress proactive reminders)Explicit user settings take precedence over experiment defaults. The eligibility event supports identifying the affected population in either arm instead of selecting only users who saw a reminder. Review the new telemetry event before rollout, and target only builds containing this change. This PR does not create or enable a remote experiment and does not change the default experience.
Validation
Rebased onto current
main(de1a430a2f3).npm run gulp compile-client— passed, including client type-checking../scripts/test.sh \ --run src/vs/workbench/contrib/chat/test/browser/agentSessions/agentHostChatContribution.test.ts \ --run src/vs/workbench/contrib/chat/test/browser/widget/chatContentParts/chatMcpAuthenticationContentPart.test.ts \ --run src/vs/workbench/contrib/chat/test/browser/agentSessions/stateToProgressAdapter.test.ts \ --grep 'mcp auth prompt|ChatMcpAuthenticationContentPart|authentication-required invocation'Try locally
Set
"chat.mcp.authenticationHints.enabled": falseto suppress reminders. Toggle it while a reminder is visible to verify that hiding it does not authenticate the server, and that authentication required by an actual tool call is still actionable.To exercise experiment assignment instead, leave the setting unset and use: