Play 95
Multimodal Search Engine v2
Unified search across images, text, code, and audio with cross-modal reasoning.
Unified search across images, text, code, and audio with cross-modal reasoning. A user can search by uploading an image and get related code snippets, documentation, and audio explanations. Combines Azure AI Search vector indexes across modalities with GPT-4o for cross-modal synthesis.
Architecture Pattern
Cross-modal search: query decomposition - modality-specific indexing - vector fusion - cross-modal synthesis - relevance feedback
Azure Services
DevKit (.github Agentic OS)
- agent.md — root orchestrator with builder→reviewer→tuner handoffs
- 3 agents — Multimodal Builder (gpt-4o), Reviewer (gpt-4o-mini), Tuner (gpt-4o-mini)
- 3 skills — deploy (254 lines), evaluate (101 lines), tune (227 lines)
- 4 prompts — /deploy, /test, /review, /evaluate with agent routing
- .vscode/mcp.json — FrootAI MCP with OpenAI + AI Search inputs + envFile
TuneKit (AI Config)
- config/openai.json - cross-modal synthesis and query expansion prompts
- config/search.json - fusion weights, index configs, diversity scores
- config/guardrails.json - relevance minimums, latency budgets
- evaluation/eval.py - Cross-modal NDCG >0.75, Latency <500ms
Tuning Parameters
Machine evidence
FrootAI evidence lifecycle
This is an internal evidence maturity label, not third-party certification, accreditation, legal compliance, or a production guarantee. Missing or expired evidence demotes automatically; catalog claims cannot promote a play.
This play currently has design evidence only. A runnable scenario, endpoint evaluation, and build receipts are the next contiguous gates.
Repo Intelligence
v1A no-clone, revision-pinned map for agents and humans. Observed evidence is separated from inferred flow so the output stays useful without pretending to be a full call graph.