AIAgentConform: open-source static package checks for Agent Plugins 1.0.0 - feedback welcome #83

Hi everyone,

I built AIAgentConform, an open-source CLI for conformance testing of AI agent protocols and standards. The first release, v0.1.1, supports static package checks for Agent Plugins 1.0.0.

It implements 20 deterministic rules covering manifests, package structure, standard skills, and MCP server configuration, with:

• Stable rule IDs and specification references.
• Actionable diagnostics.
• Machine-readable JSON output and exit codes for CI.

The aim is to check both schema requirements and justified static/semantic requirements, while keeping package checks distinct from client behavior.

For example, extension values are checked for the required object shape, but their namespace-specific contents remain opaque. Specification ambiguities are documented rather than turned into invented rejection rules.

I tested pinned package snapshots from Context7, Sentry, and GitHub’s C#/.NET Development plugin. All passed the applicable checks. A separate synthetic package exercised standard skills and stdio configuration; five independently introduced defects were detected with the expected rules and exit codes.

These are static package results, not runtime interoperability or full client-conformance certification. The tool does not execute bundled code or contact declared MCP endpoints.

Install with Node.js 22.12 or newer:

npm install --global [email protected]
aiconform --help

https://github.com/rao-anant/aiagentconform

https://www.npmjs.com/package/aiagentconform

I’d welcome feedback from plugin authors and client implementers, particularly on rule interpretations, false positives, and unclear diagnostics. If you try it on a package, please share what worked or what needs improvement.

This is an independent community project, not an official certification tool.