Workforce
Annotator Management
Manage your annotation workforce end-to-end. Handle annotator applications and approval workflows, process payments automatically, target specific demographics, and track individual performance over time.
Key Capabilities
Approval Workflows
Annotators apply through a structured onboarding flow with qualification tests and demographic surveys. Review applications, approve or reject with feedback, and manage waitlists by project or domain.
Payment Processing
Pay annotators per label or per comparison with automated batch payouts. Set project-level rates, apply bonuses for high-quality work, and download itemized payment reports for accounting.
Demographic Targeting
Define demographic requirements for each project including age range, region, language, and domain expertise. The platform automatically routes tasks only to qualified annotators matching your criteria.
Performance Tracking
Monitor each annotator's accuracy, throughput, consistency, and response time across projects. Identify top performers for specialized tasks and surface underperformers for retraining or removal.
Usage
# List annotators with performance stats
curl https://api.commandagi.com/v1/annotators \
-H "Authorization: Bearer $COMMANDAGI_API_KEY" \
-G -d "status=active" -d "tier=gold" -d "include=stats"
# Response
# {
# "annotators": [
# {
# "id": "ann_01",
# "tier": "gold",
# "status": "active",
# "stats": {
# "total_annotations": 8420,
# "gold_accuracy": 0.98,
# "avg_response_ms": 2400
# }
# }
# ]
# }
# Approve a pending annotator
curl -X POST https://api.commandagi.com/v1/annotators/ann_15/approve \
-H "Authorization: Bearer $COMMANDAGI_API_KEY" \
-d '{ "tier": "bronze", "projects": ["proj_abc123"] }'