A hybrid accuracy and token routing system built for the AMD Developer Hackathon (Act II — Track 1). It evaluates queries, executes deterministic Python AST and SymPy solvers first (at 0 token cost), and routes complex reasoning tasks to cloud LLMs with self-consistency voting and local fallback.
Routing every incoming query to high-parameter cloud LLMs causes excessive latency, ballooning token costs, and avoidable hallucinations for deterministic math. Conversely, relying solely on small local models drops accuracy significantly on complex reasoning and code generation.
OptiRoute introduces a tier-based hybrid routing architecture. It utilizes sub-millisecond TF-IDF classification, executes zero-token Python solvers (math, AST code parsing), routes complex queries to remote models (Fireworks AI) with Chain-of-Thought voting, and falls back to a local quantized Qwen 1.5B GGUF model for offline resilience.
Instantly solves math (SymPy, regex), fact lookups (59 verified facts), sentiment rules, and AST code fixes at 0 token cost in < 1ms.
Ultra-lightweight domain classifier (< 10MB model weight) that accurately categorizes incoming prompts into 8 specialized domain types.
Identifies spatial puzzles, tricky wordings, and logic traps, ensuring trick queries bypass shortcuts and route directly to deep reasoning models.
Executes multi-candidate self-consistency voting (3 calls for math, 2 calls for logic) with Chain-of-Thought system prompts and answer extraction.
Routes dynamically based on model strengths (e.g., Kimi for code execution, Minimax for complex logic, Gemma for factual reasoning).
Embeds local Qwen2.5 1.5B GGUF via llama-cpp-python as a thread-safe emergency fallback to guarantee 100% service uptime.
agent/classifier.py — High-speed TF-IDF domain classifier (< 10MB weight)agent/trap_detector.py — Semantic puzzle & trap detector overrideagent/router.py — Core remote-first router with deterministic zero-token shortcutsagent/evaluator.py — SymPy math evaluator, AST code parser, and output validatorsagent/remote_model.py — Fireworks AI API client with CoT, voting, and model fallbackagent/local_model.py — Thread-safe local Qwen2.5 1.5B GGUF fallback runnerstreamlit_app.py — Interactive web application displaying real-time routing diagnosticsDockerfile — Multi-stage container setup ready for Hugging Face Spaces deploymentRemote API provider for high-performance LLMs (Minimax, Kimi, Gemma 4).
Quantized local LLM fallback integrated via llama-cpp-python.
Deterministic mathematical symbolic computation and Python AST code structure validation.
Diagnostic web dashboard and automated container deployment for Hugging Face Spaces.