Operations
Real-Time Monitoring
Monitor annotation sessions, agent executions, and platform health through live dashboards. Set up alerts for quality drops, track throughput in real time, and manage operations from a unified command center.
Key Capabilities
Live Dashboards
Real-time dashboards display annotation throughput, active sessions, quality metrics, and cost tracking. Filter by project, annotator cohort, or time window with sub-second refresh rates.
Agent Monitoring
Watch AI agents execute tasks in real time through the command center grid view. Monitor step-by-step progress, inspect screenshots, and intervene manually when agents encounter unexpected states.
Alert System
Configure alerts for quality drops, throughput anomalies, budget thresholds, and annotator issues. Alerts deliver via webhook, email, or Slack with configurable severity levels and cooldown periods.
Command Center
A unified operations view showing all active environments, running agents, and annotation sessions. Drill into any session for detailed logs, screenshots, and action history with full audit trails.
Usage
# Stream real-time events for a project
curl -N https://api.commandagi.com/v1/projects/proj_abc123/stream \
-H "Authorization: Bearer $COMMANDAGI_API_KEY" \
-H "Accept: text/event-stream"
# Events received:
# event: annotation
# data: { "annotator": "ann_01", "type": "comparison", "timestamp": "..." }
#
# event: quality_alert
# data: { "level": "warning", "metric": "agreement", "value": 0.62 }
#
# event: agent_step
# data: { "agent_id": "agt_x7k", "step": 12, "action": "click" }
# Configure webhook alerts
curl -X POST https://api.commandagi.com/v1/projects/proj_abc123/alerts \
-H "Authorization: Bearer $COMMANDAGI_API_KEY" \
-d '{
"metric": "gold_accuracy",
"threshold": 0.85,
"direction": "below",
"channel": "webhook",
"url": "https://your-app.com/hooks/quality-alert"
}'