Docs/API/Evaluation

Evaluation API

Score content against taste profiles to determine aesthetic alignment. Returns a score from 0 to 1 with confidence metrics.

How Scoring Works

The evaluation endpoint uses multiple signals to compute a taste alignment score:

Exemplar Matching

Direct matches against labeled exemplars provide the highest confidence scores.

Embedding Similarity

Cosine similarity between the frame's embedding and the profile's centroid.

Latent Scores

Bradley-Terry rankings derived from pairwise comparisons.

Constraint Matching

Verification against explicit constraints from Q&A sessions.

POST/v1/profiles/:id/eval

Evaluate a frame against a taste profile. Returns a score between 0 (poor match) and 1 (excellent match) with confidence metrics.

Path Parameters

Parameter	Type	Description
id	string	Required. The profile ID to evaluate against

Request Body

Field	Type	Description
frameUrl	string	Required. URL of the image to evaluate
embedding	number[]	Optional. Pre-computed embedding vector for the frame

Request

curl -X POST https://api.commandAGI.com/v1/profiles/prof_abc123/eval \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "frameUrl": "https://example.com/image-to-evaluate.jpg"
  }'

Response

{
  "score": 0.847,
  "confidence": 0.92,
  "details": {
    "latentScore": 0.823,
    "constraintMatch": 5,
    "exemplarSimilarity": 0.891
  }
}

Response Fields

Field	Type	Description
score	number	Overall taste alignment score (0.0 to 1.0)
confidence	number	Confidence in the score (0.0 to 1.0)
details.latentScore	number \| null	Score from Bradley-Terry model (if available)
details.constraintMatch	number	Number of constraints matched
details.exemplarSimilarity	number \| null	Cosine similarity to exemplar centroid

Score Interpretation

0.8 - 1.0Excellent match - strongly aligns with taste

0.6 - 0.8Good match - mostly aligns with taste

0.4 - 0.6Neutral - neither matches nor conflicts

0.2 - 0.4Poor match - conflicts with taste

0.0 - 0.2Strong conflict - opposite of taste

Using Pre-computed Embeddings

For higher accuracy and faster evaluation, you can provide a pre-computed embedding vector for the frame. This enables direct similarity comparison with the profile's embedding centroid.

Tip: Use a CLIP or similar vision model to generate embeddings. The embedding should be normalized (unit length) for best results.

Request with Embedding

curl -X POST https://api.commandAGI.com/v1/profiles/prof_abc123/eval \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "frameUrl": "https://example.com/image.jpg",
    "embedding": [0.12, -0.34, 0.56, -0.78, ...]
  }'

Batch Evaluation

To evaluate multiple frames, make separate API calls. For high-volume evaluation, consider using the SDK which provides batch methods with automatic rate limiting.

JavaScript SDK Example

import { TasteOptimizer } from 'commandAGI';

const client = new TasteOptimizer({ apiKey: 'to_...' });

const frames = [
  'https://example.com/image1.jpg',
  'https://example.com/image2.jpg',
  'https://example.com/image3.jpg',
];

const results = await client.profiles.evalBatch('prof_abc123', frames);

// results = [
//   { score: 0.847, confidence: 0.92, ... },
//   { score: 0.234, confidence: 0.85, ... },
//   { score: 0.912, confidence: 0.97, ... },
// ]

Error Codes

Code	Error	Description
`400`	frameUrl is required	Request must include a frameUrl
`400`	Profile has no data	Profile has no effective snapshot to evaluate against
`404`	Profile not found	The specified profile does not exist