Play 18
Prompt Management
Medium🔧 Skeleton
Version control, A/B test, and rollback prompts across environments.
Manage prompts like code. Git-backed versioning, A/B testing with traffic splitting, automated quality scoring per version, rollback to any previous version. Cosmos DB stores prompt versions and experiment results. GitHub Actions CI/CD deploys prompt updates across dev/staging/prod.
Architecture Pattern
Prompt versioning, A/B testing, CI/CD, rollback, experimentation
Azure Services
Prompt FlowGit/GitHubGitHub ActionsAzure AI FoundryCosmos DB
DevKit (.github Agentic OS)
- agent.md — root orchestrator with builder→reviewer→tuner handoffs
- 3 agents — Prompt Mgmt Builder (gpt-4o), Reviewer (gpt-4o-mini), Tuner (gpt-4o-mini)
- 3 skills — deploy (111 lines), evaluate (101 lines), tune (114 lines)
- 4 prompts — /deploy, /test, /review, /evaluate with agent routing
- .vscode/mcp.json — FrootAI MCP with OpenAI + Cosmos DB inputs + envFile
TuneKit (AI Config)
- config/prompts.json — prompt versions, active versions
- config/ab-test.json — A/B weights, experiment tracking
- infra/ — Prompt Flow templates
Tuning Parameters
Prompt versionsA/B traffic weightsRollback rulesVersion scheduling
Estimated Cost
Dev/Test
$50–150/mo
Production
$300–1K/mo