Arbiter
LLM quality-aware cascade router. Routes inference requests through a tiered model hierarchy - cheap models first, escalating only when quality thresholds demand it. Built with DeepEval for continuous quality measurement and Prometheus/Grafana for production observability. Not a toy: deployed against real traffic with PostgreSQL for decision logging and Redis for routing cache.
View on GitHub