FAI Changelog & Releases
Version history for the FrootAI ecosystem — MCP Server, VS Code Extension, Website, and Solution Plays.
v6.8.1
LatestMay 2026- Closes RAG Gap 6 (Prompt Testing / Dry Run) — the last unaddressed gap from the original 16-gap roadmap. Now 16/16 gaps closed.
- New MCP tool prompt_dry_run (FAI Engine 7 → 8 tools, total 51 → 52): test a system prompt with a single sample query without deploying. Calls Azure OpenAI when AZURE_OPENAI_ENDPOINT is set, else uses a deterministic shim that burns zero tokens.
- Returns the response + lightweight self-checks (citations, abstention, refusal, length, optional groundedness hint via lexical overlap with `context`/`groundTruth`) + iteration suggestions for the next prompt revision.
- Mirrored in the Python MCP server (frootai_mcp.server.prompt_dry_run) for full feature parity.
- Tests: 483 passing (was 469, +14 unit tests for prompt_dry_run input validation, both backends, all checks, and Markdown rendering).
v6.8.0
FeatureMay 2026- Plan A — Telemetry hardening: trackTool now bridges to OpenTelemetry counters/histograms automatically when OTEL_EXPORTER_OTLP_ENDPOINT is set; per-tool sliding-window p50/p95/p99/error counters; concurrency math fix (workers ≤ items)
- Plan B — Coverage expansion: 5 new components (event-grid-publisher, signalr-realtime, azure-monitor-query, service-bus-sender, azure-aks-deployment) → 20 components; C# templates for 6 high-traffic components (managed-identity, app-insights, keyvault, openai-streaming, search-hybrid, content-safety); 4 new plays (07 Multi-Agent Service, 13 Fine-Tuning, 22 Multi-Agent Swarm, 33 Voice AI Agent); C# pinned deps for plays 01, 03, 06, 09
- Plan D — Live Evaluation v2: streaming endpoint modes (sse, ndjson, plain-text), azure-ai-evaluation backend (per-metric judge calls aligned with the Azure AI Evaluation SDK), historyPath JSONL trend log, CI quality-gate workflow (.github/workflows/ai-eval-gate.yml)
- Plan C — Marketplace integration: plugin.schema.json extended with components / playTemplates / bicepTemplates / reviewRules arrays; new plugin-loader module discovers .github/plugins/, plugins/, node_modules/@frootai-plugin/*; scaffold_component MCP tool now merges plugin-supplied components when workspaceRoot is passed; SECURITY: review-rule modules execute JavaScript — disable via FAI_PLUGIN_LOAD_DISABLED=1
- Plan E — VS Code extension: 6 new commands surfacing the v6.7 tools (frootai.analyzeWorkspace, frootai.scaffoldComponent, frootai.getPlayConfig, frootai.getDependencies, frootai.generateBicep, frootai.runEvalLive). Total commands: 46 → 52
- Tests: 469 passing (was 445, +24)
v6.7.0
LatestMay 2026- MCP Server v6.7.0: 51 tools across 8 categories (Knowledge 6, Live 4, Agent Chain 3, Ecosystem 10, FAI Engine 7, Scaffold 7, Workspace 1, Marketplace 13)
- embedding_playground upgraded: 3 backends — azure-openai (real text-embedding-3-small 1536-dim cosine), char-ngram (TF-IDF cosine, captures morphology), jaccard (legacy back-compat). auto-selects azure-openai when AZURE_OPENAI_ENDPOINT is set
- Closes Gap #12 — embedding_playground was misleadingly named (Jaccard, not embeddings)
- Honest naming: every result tags backend, vectorDim, similarityType, latencyMs
v6.6.0
FeatureMay 2026- New tool: run_eval_live — POST test cases to a live AI endpoint, score responses with simple (zero-dep) or LLM-as-judge backend, return aggregate mean/p95 vs threshold
- 5 metrics: groundedness, relevance, fluency, coherence, safety (configurable subset)
- Sandboxed test set + evidence file paths (no symlinks, no .. escapes), bounded concurrency (default 4, max 16), AbortSignal-based timeouts
- Closes Gap #11 — run_evaluation only checked pre-computed scores, never actually ran them
v6.5.0
FeatureMay 2026- New tool: generate_bicep — play-specific IaC with Managed Identity + RBAC + tags + (optional) private endpoints
- 16 Azure resource definitions: openai, aisearch, containerapp(env), keyvault, storage, cosmos, speech, communication, docint, contentsafety, appinsights, loganalytics, redis, apim, appservice
- 8 plays mapped: 01, 03, 04, 06, 09, 14, 17, 21
- Closes Gap #10 — Bicep was generic skeleton, not play-specific
v6.4.0
FeatureMay 2026- Three new tools: scaffold_component, get_play_config, get_dependencies
- Component catalog (15): managed-identity-init, application-insights-init, azure-keyvault-secret, azure-openai-streaming, azure-openai-structured-output, azure-search-hybrid, azure-speech-stt, azure-speech-tts, fastapi-websocket-audio, communication-services-call-automation, document-intelligence-ocr, content-safety-check, azure-cosmos-client, event-hub-consumer, prompt-shields-defense
- Play templates expanded to 8 plays (added 06, 09, 14, 17) with configs + pinned deps
- Closes Gaps #5, #6, #9 — app code, play-specific configs, pinned dependencies
v6.3.0
MajorMay 2026- Code-aware agent chain rewrite: agent_build / agent_review / agent_tune now play-aware + code-aware + chain-handoff capable
- 12-rule code review engine: no_hardcoded_api_keys, uses_managed_identity, temperature_within_bounds, has_retry_logic, has_error_handling, has_app_insights, content_safety_enforced, pii_detection_enforced, citations_in_rag, abstention_message_set, bicep_security_posture, no_hardcoded_secrets
- Play templates with architecture, code patterns (py/ts), config defaults, pinned deps
- Findings include severity + file + line + suggestion + optional fix patch
- 100% backward-compatible — old call sites preserved via param-presence detection
- Closes Gaps #2, #3, #7 — agent_build not play-aware, agent_review not code-aware, chain handoff broken
v6.2.0
FeatureMay 2026- New tool: analyze_workspace — sandboxed workspace inspection
- Detects: solution-play / plugin / monorepo / frootai-source / fai-engine-consumer / unknown
- Inventories agents, instructions, skills, hooks, prompts, workflows, plugins
- Scans Bicep for 20 Azure service patterns + Managed Identity / Private Endpoints / Tags posture
- Gap analysis with play-specific rules (e.g., Play 04 needs Speech + Communication Services)
- Closes Gap #8 — no workspace context awareness
v6.1.0
FeatureMay 2026- New knowledge module: V1 — Voice & Speech AI (Real-Time Conversational Systems)
- 10 sections: Azure AI Speech, Communication Services, real-time audio, voice agent patterns, SSML, STT→LLM→TTS pipelines, telephony, WAF alignment, anti-patterns
- FROOT framework expanded to FROOT+V (6 layers) — Foundations · Reasoning · Orchestration · Operations · Transformation · Voice
- Knowledge bundle: 17 modules / 720 KB (was 16 / 664 KB)
- Closes Gap #4 — search_knowledge had no voice/speech content
v5.0.1
StableApril 2026- MCP Server v5.0.1: 45 tools — FAI Engine (wire_play, inspect_wiring, validate_manifest + 3 more), Scaffold (scaffold_play, create_primitive, smart_scaffold), Marketplace (marketplace_search/browse/install/publish + 8 more)
- HTTP/SSE transport: npx frootai-mcp http — remote MCP server, multi-client sessions, API key auth
- TypeScript full migration: strict mode, 38 vitest tests passing, ESM engine modules
- OpenTelemetry: opt-in via OTEL_EXPORTER_OTLP_ENDPOINT — custom FAI metrics (tool calls, quality evals, plays wired)
- MCP Sampling + Elicitation: server can request LLM completions and structured user input
- FAI Marketplace via MCP: search, install, publish, compose, resolve dependencies for 77+ plugins
- Health probes: /healthz (liveness) + /readyz (readiness) — standard k8s/Container Apps format
- Docker updated for HTTP mode with engine + schemas + marketplace included
v4.0.0
MajorApril 2026- 104 solution plays (was 30) — each with 44 files avg, 4,132 lines avg, 15/15 file types pass quality gates
- 861 standalone primitives: 238 agents (200+ lines each), 176 instructions, 322 skills, 10 hooks, 85 plugins
- MCP Server v4.0.0: 45 tools (added list_primitives, get_play_detail, embedding_playground)
- VS Code Extension v2.1.0: 16 commands, standalone sidebar
- Python SDK v4.1.0 + Python MCP v4.0.0: 104 plays synced
- Docker v4.0.0: Multi-arch (amd64 + arm64), 16 knowledge modules bundled
- FAI Engine: 8 modules operational (manifest-reader, context-resolver, primitive-wirer, hook-runner, evaluator, mcp-bridge, index, test)
- Website: 178 pages, mobile-optimized, Plausible analytics, i18n (10 locales), A/B testing (5 tests)
- Community: 8 partner integrations (ServiceNow, Salesforce, SAP, Datadog, PagerDuty, Jira, Slack, Teams)
- Certification program config: L1-L4 (Foundations → Expert) with study modules, labs, exams, badges
- Enterprise edition config: private marketplace, SSO, usage dashboard, governance, air-gapped support
- 30 emerging tech agents: OpenTelemetry, GraphRAG, A2A, Cloudflare, Vercel, Supabase, Deno, Bun, LangChain, CrewAI
v3.1.2
PatchMarch 2026- Added 6 AI ecosystem tools (model catalog, pricing, cost estimation, model compare, play compare, semantic search)
- Agent chain tools: agent_build → agent_review → agent_tune
- VS Code Extension: Auto-chain agents command
- Website: Added adoption page, eval dashboard, feature spec
- Docker: Multi-arch support (amd64 + arm64)
v3.0.0
MajorFebruary 2026- MCP Server v3: 45 tools (6 static + 4 live + 3 chain + 6 compute + 4 AI ecosystem + 2 browse)
- Solution Plays expanded to 30 with DevKit + TuneKit + SpecKit
- .github Agentic OS: 19 files, 4 layers, 7 primitives per play
- VS Code Extension v1.0.0: 16 commands, standalone sidebar
- Website: 25 pages, Agent FAI chatbot with GPT-4.1
v2.0.0
MajorJanuary 2026- MCP Server v2: Added live tools (fetch_azure_docs, fetch_external_mcp)
- Knowledge base expanded to 16 modules (664 KB)
- AI Glossary: 200+ terms
- Solution Plays: 3 ready + 17 skeletons
v1.0.0
InitialDecember 2025- MCP Server v1: 6 static tools
- 10 knowledge modules (5 FROOT layers)
- Initial website with Docusaurus
- VS Code Extension: Basic sidebar