Machine Learning Engineer

About Direct Inference

What you'll own

• Improve request classification across code, JSON, document, vision, long-context, reasoning, flash, and pro traffic.
• Build evaluation datasets, scoring workflows, and regression checks for answer quality.
• Analyze production traces and failure cases while preserving privacy and the zero-knowledge contract.
• Partner with inference engineering to translate eval findings into serving improvements.
• Develop tools that compare quality, cost, latency, and capability across candidate serving paths.

Projects you might ship

• Build a durable eval set for one high-value request slice and wire it into a repeatable scoring workflow.
• Improve request-type classification by finding ambiguous production examples and turning them into tested policy behavior.
• Create analysis tooling that compares frontier baselines, serving candidates, and Direct Inference outcomes without exposing private customer data.

What we're looking for

• You have experience with ML systems, evals, classifiers, LLM applications, ranking, or model-quality measurement.
• You can design practical evaluations that are useful to engineering, not just nice-looking benchmarks.
• You are comfortable with Python and production engineering constraints.
• You can reason about model behavior, product outcomes, and data quality together.
• You communicate uncertainty clearly and know when a metric is lying.

Nice to have

• Experience with Langfuse, prompt/eval workflows, human review pipelines, or model-comparison tooling.
• Practical familiarity with coding-agent, document, vision, structured-output, or reasoning workloads.
• A taste for evals that are boring, reusable, and operationally useful.

Your first 90 days

• Improve or expand one request-type quality evaluation suite.
• Trace a quality regression from example failures to a concrete serving or classifier fix.
• Add an eval workflow that helps future model or policy changes ship with confidence.