Comet

Comet is a web platform for AI developers, researchers, and enterprise teams with two product families: Opik for LLM and agent observability and evaluation, and an MLOps platform for experiment tracking and model management.

Key capabilities include:

• Opik LLM observability and evaluation: trace logging for LLM calls and agents, test suites and assertions, 30+ built-in evaluation metrics, datasets and experiments, an agent playground, and online evaluation and alerts for production monitoring.
• Agent and prompt optimization: prompt and tool optimization toolkits and a prompt library with versioning.
• Experiment tracking (MLOps): log training runs, compare results, manage datasets and a model registry, with hyperparameter search.
• Model production monitoring (Enterprise) to detect data drift and investigate model behavior in deployed environments.

Pricing: Freemium. Opik is open source and free to self-host; Opik Free Cloud ($0, 25k spans/month), Pro Cloud ($19/month, 100k spans/month) and custom Enterprise. MLOps has a Free plan, Pro at $19 per user/month (up to 10 users) and custom Enterprise.

Notable for combining open-source Opik with an MLOps platform, with cloud, self-hosted, managed and on-premise deployment options. Check comet.com for current prices.