The task
Sitka operates a document intelligence pipeline that classifies incoming operational records before extraction and review. The reference system is accurate, expensive, and too slow at peak volume. You receive a frozen evaluation harness, a representative corpus, model access, and a cost ledger that prices every token and external call.
Your job is to redesign the decision path. You may route examples, cache stable work, distill prompts, introduce deterministic checks, or choose not to invoke a model at all. The winning system will preserve difficult edge cases rather than merely optimize the average document.
Submit a reproducible package and a short technical memo. Judges should be able to understand the tradeoffs, rerun the system, and identify exactly where you chose speed, cost, or caution. Cleverness that cannot survive inspection will not score well.
Hard constraints
- Maximum 5 scored submissions per entrant; the final scored submission is binding.
- Package must run in the supplied container with no outbound network access except the provided model gateway.
- Median wall-clock latency must remain below 1.8 seconds on the hidden workload.
- No training on, manual labeling of, or attempts to infer the hidden test set.
- Deliver one repository archive plus a technical memo of no more than 1,500 words.
AI policy
AI is allowed. We provide it. We log every prompt.
Use the provided environment as aggressively as you like. Judges evaluate your decisions, verification discipline, and final system—not whether you typed every token yourself.
Judging rubric
Cost reduction
Measured against the reference ledger on the hidden workload.
Accuracy retention
Weighted toward rare and high-consequence classes.
Robustness
Behavior under malformed, ambiguous, and adversarial inputs.
Engineering judgment
Reproducibility, clarity, and quality of tradeoff decisions.