About Us

Preference Model is building automated ML research engineering.

Existing frontier models are brittle when applied to real-world ML tasks. The present bottleneck is the lack of high-quality RL training environments. Our first step is to build RL environments that reflect real-world complexity, with diverse tasks and robust reward functions.

Our founding team has previous experience on Anthropic’s data team building data infrastructure, and datasets behind Claude. We are partnering with leading AI labs to push AI closer to achieving its transformative potential.

About the Role

AI models have gotten good at narrow coding tasks but still fail at the complex, judgment-heavy parts of software engineering: working in a large codebase with real conventions and technical debt, making the right tradeoff on a system design problem, or navigating a multi-step task with ambiguous stakeholders. As a Senior Software Engineer on the RL Environments team, you will build the environments that expose those failures and help models improve on them.

You will own our most complex tasks end-to-end: environments with multi-step workflows, realistic stakeholder interactions, large codebases, and challenging system design problems. You will work closely with a small team of engineers and directly with our founders, and you will ship environments that go into the training loops of frontier models at our partner labs. This is independent, high-ownership work with regular feedback.

What You Will Do

• Design, build, and refine RL tasks across their full lifecycle, from ideation through grading, failure analysis, and iteration.

• Own the hardest environments on the roadmap: multi-step workflows, realistic stakeholder interactions, large codebases with real conventions and technical debt, and system design problems.

• Direct coding agents heavily in your day-to-day work, evaluate their output critically, and recognize when they are failing in subtle ways.

• Distinguish genuine model capability gaps from grader or environment issues, and redesign tasks to target deeper, more subtle failure modes.

• Contribute to the shared infrastructure and tooling that the environments team depends on.

• Mentor newer engineers on the team as it grows
  

What We are Looking For

• Deep software engineering experience across multiple domains, with genuine expertise in at least one specialty: infrastructure, distributed systems, performance, security, compilers, databases, or similar.

• Proficiency in Python.

• Extensive hands-on experience with coding agents (Claude Code, Cursor, Codex, or similar), including an intuition for where they cut corners and how to direct them well.

• Strong intuition for how models behave, even without prior ML or AI experience. You can anticipate where a model will take shortcuts and design around that.

• Comfort working independently on complex, ambiguous problems with minimal direction.

• Track record of owning work end-to-end in previous roles.

You may be a good fit if one of the following applies

• You have been a senior or staff engineer at a company known for engineering rigor (e.g., a frontier lab, infrastructure startup, or systems-heavy team) and want to apply that experience to model training.

• You have deep specialty expertise in an area that current models struggle with (distributed systems, low-level performance, security, compilers) and can design tasks that expose those weaknesses.

• You have been an early engineer at a previous startup, shipped independently, and want to do it again in AI.

• You have spent significant time building with coding agents, written about their failure modes, or contributed to agent evaluation work.

What We Offer:

• Competitive cash and equity compensation (>90th percentile)

• Ownership and autonomy in a fast moving startup environment

• Opportunity to work with top machine learning engineers

• Health, vision, dental, benefits

• 401K match

• Visa sponsorship & relocation support available

We value diverse perspectives and experiences. If you're excited about this role but don't check every box, we still encourage you to apply.