Policy Complexity Gradient and Model Routing BoundaryURL copied
Not all policies are equally hard to migrate. An HTTP proxy policy with no custom configuration maps cleanly. A quota policy with conditional logic and a JavaScript callout that modifies headers mid-chain does not. The routing layer needs a reliable way to classify policies before it routes them — otherwise it routes complex policies to the lightweight model and produces plausible but incorrect code.
The routing boundary cannot be determined upfront. It emerges from the first 20–30 migrations: which policy types consistently produce high-confidence correct output from the lightweight model, and which require the heavyweight model to get right. The initial configuration should be conservative — route ambiguous cases to the heavyweight model — and tighten as empirical data accumulates.
Option comparison: static type-based routing vs. dynamic confidence-based routing
Static type-based routing assigns each policy type a fixed model tier based on known complexity (e.g., HTTP proxy → lightweight, custom script → heavyweight). Simple to implement; degrades gracefully; does not require a scoring step before routing. Disadvantage: a simple custom script might be handled well by a lightweight model, but the static rule always routes it up.
Dynamic confidence-based routing runs a fast pre-assessment pass with the lightweight model, inspects the confidence score and reasoning, and re-routes to the heavyweight model only if the score falls below a threshold. More cost-efficient at scale. Disadvantage: adds latency (two model calls for ambiguous cases) and requires a reliable pre-assessment score — which is circular if the score is what we're trying to establish.
Recommendation: start with static type-based routing. Switch to dynamic confidence-based routing once the corpus is large enough to validate the pre-assessment scores.