Play 36
Multimodal Agent
Vision + text + code — analyze images, screenshots, diagrams alongside natural language.
Vision + text + code agent that analyzes images, screenshots, diagrams, and documents alongside natural language input. GPT-4o Vision processes visual content, Azure AI Vision handles specialized image analysis, Blob Storage manages media assets, and Container Apps hosts the agent runtime. Supports use cases from UI testing to architectural diagram analysis to document verification with cross-modal reasoning.
Architecture Pattern
Multimodal agent: vision + text + code understanding, cross-modal reasoning
Azure Services
DevKit (.github Agentic OS)
- agent.md — root orchestrator with builder→reviewer→tuner handoffs
- 3 agents — Multimodal Builder (gpt-4o), Reviewer (gpt-4o-mini), Tuner (gpt-4o-mini)
- 3 skills — deploy (103 lines), evaluate (104 lines), tune (107 lines)
- 4 prompts — /deploy, /test, /review, /evaluate with agent routing
- .vscode/mcp.json — FrootAI MCP with Vision + OpenAI key inputs + envFile
TuneKit (AI Config)
- config/openai.json — gpt-4o vision model config, image tokens
- config/vision.json — image processing params, resolution, formats
- config/guardrails.json — content safety for images, PII in screenshots
- evaluation/eval.py — Cross-modal accuracy >85%, Image understanding >80%
Tuning Parameters
Machine evidence
FrootAI evidence lifecycle
This is an internal evidence maturity label, not third-party certification, accreditation, legal compliance, or a production guarantee. Missing or expired evidence demotes automatically; catalog claims cannot promote a play.
This play currently has design evidence only. A runnable scenario, endpoint evaluation, and build receipts are the next contiguous gates.
Repo Intelligence
v1A no-clone, revision-pinned map for agents and humans. Observed evidence is separated from inferred flow so the output stays useful without pretending to be a full call graph.