ClassiRoute

ClassiRoute was built to solve the 'which model is good enough?' question that every team faces when building on multiple LLM providers.

Teams defaulting to premium models (GPT-4o, Claude Opus) burn budget on trivial requests. Defaulting to cheap models fails on complex prompts.

The goal was automated cost-aware routing that learns what prompts actually need — without manual rules or routing tables.

No easy way to determine prompt complexity before sending it to an LLM.

Hardcoded routing rules don't scale and miss edge cases.

Multi-provider dispatch adds complexity — different APIs, auth, and response formats.

The challenge was building a routing system with 92%+ accuracy, sub-50ms overhead, and graceful fallback when the ML model is uncertain.

ML-powered classification from day one — XGBoost classifier with companion confidence regressor.

22-dimensional feature extraction covering length, readability, patterns, and structural signals.

Plugin-based provider architecture via a Provider Registry for extensibility.

Cascading fallback chain: strong→mid→weak on failure, auto-escalation on low confidence.

Virtual Keys bundling three model tiers (weak/mid/strong) per key with per-tier provider config.

Backend: Python 3.13, FastAPI, async SQLAlchemy + asyncpg for async database operations.

ML: XGBoost 3.2+ classifier (92.31% accuracy) and regressor for confidence scoring, pickle-serialized models.

Frontend: React 19, TypeScript, Tailwind CSS v4, Radix UI, Recharts for analytics dashboards.

Database: PostgreSQL 16 with Alembic migrations and async driver.

Providers: OpenAI-compatible, Anthropic, and Gemini via a unified plugin architecture.

Infrastructure: Docker multi-stage builds, Render backend, Vercel frontend, Neon PostgreSQL.

ML-powered routing beats heuristic rules once you have sufficient training data — the model captures nuance that manual thresholds miss.

Confidence scoring is as important as classification accuracy; auto-escalating low-confidence predictions prevents silent quality degradation.

Provider abstraction pays for itself the moment you add a second provider — the plugin registry pattern made Anthropic and Gemini integration trivial.

Performance benchmarking (15,000 prompts) proved 57.9% cost reduction with 96.8% quality parity — numbers that make the case for ML routing concrete.