How to staff the engineering under an RL roadmap

Key takeaways

• The bottleneck under an environments roadmap is usually infrastructure engineering rather than task authoring. Provisioning, orchestration, and the harness decide whether a researcher queues a sweep before dinner or waits three days for one.
• A grader is an adversarial surface. Any check an agent can satisfy without doing the work will eventually be satisfied that way, and catching it takes someone who reads trajectories.
• Contributor pools are built to move task volume: per-task or per-hour pricing, fixed short terms, anonymous supply. That structure is why the person who finally understood your environments is gone before the next hillclimb.
• Seniority buys adversarial imagination on graders, distributed-systems debugging on flaky rollouts, and people who can run a pod. The market produces very few of the third.
• Demand for this engineering is lumpy by construction. Size a permanent core to the environment library and the harness, then flex a senior surge layer around each hillclimb.

Why this is hard to staff

Your roadmap says the model gets better at some class of tasks by the end of the quarter. Underneath that sits a pile of software nobody outside the team can see: the sandbox each rollout runs in, the queue that fans out ten thousand of them, the grader that decides whether a rollout counts as a success, and the trace store a researcher opens when a number moves and nobody can say why. You can buy help for it. The market will show you two very different things when you go looking, and both of them will be described to you as engineers.

The frame: Two supply markets wearing the same words

One market sells task volume. Its unit of production is a completed task, priced per task or per contributor-hour, drawn from a screened pool with a QA layer over the top. The machinery around it, recruiting funnels and sampling rates and throughput dashboards, is tuned for exactly that conversion, and it's genuinely good at it.

The other market sells engineering capacity. Its unit of production is a working system, delivered by named people who commit code into your repository and answer for whether it holds up.

Both markets put "engineer" in the job title. The work that blocks an environments roadmap mostly sits in the second market. The supply that's easy to find mostly sits in the first.

What the engineering under an RL program actually is

Environment provisioning. Every rollout needs an isolated, reproducible starting state: a container or microVM with the repo at a known commit, dependencies resolved, services up, filesystem and clock in a defined condition. Reproducibility is the whole game here. If the same task behaves differently across two runs, the grader's verdict stops meaning anything and every number downstream inherits that noise. The work is image building, snapshot and warm-pool strategy so cold starts stay off the critical path, and the unglamorous job of pinning a dependency tree that upstream keeps moving.

Secure networking. Agents reach for the network. Some tasks require it, most don't, and a task that lets an agent browse to the answer teaches nothing. You need per-task egress control, credential injection that doesn't end up in the transcript, certificate handling for services the environment stands up itself, and isolation firm enough that one rollout can't observe another. It's ordinary platform security work applied to a workload whose entire purpose is to try things.

Job orchestration. A hillclimb means thousands of rollouts across a fleet, with retries, timeouts, preemption, and artifact collection. Whether a researcher can queue a sweep in the evening and read it in the morning is an orchestration question long before it's a compute question. Cost lands here too. Rollouts burn real money, and the gap between a naive fan-out and a scheduled one shows up on the bill every month.

Tasks and graders. A task is a starting state, a specification, and a grader. The grader is where the difficulty concentrates. A grader that verifies a proxy will be satisfied through the proxy: tests get edited, assertions get monkeypatched, an expected output file gets written directly, a known input gets special-cased. The defenses are mechanical once you've been burned by them. Run the grader from a tree the agent can't write to. Hold out cases it never sees. Diff the workspace for edits to the verification code. Randomize inputs so memorizing one case doesn't pay. None of that is exotic. It requires someone who assumes the agent will cheat and goes looking on purpose.

Eval pipelines. Training environments and evals often get built by the same people and serve opposite purposes, so they have to be kept apart in practice. An eval set is held out, versioned, and run through a harness deterministic enough that two runs a month apart are comparable. It needs enough tasks that the delta you're reading isn't sampling noise, and enough spread in difficulty that the model doesn't pass everything or fail everything, since a task set at either extreme can't tell you whether a change helped. Contamination is a live concern. So is eval awareness, where a model behaves differently once it works out that it's being tested.

Observability. Nearly every complaint researchers make about bought environments traces back to somebody not reading the trajectories. Making them readable is engineering: storing the full step sequence with tool calls, observations, and the grader's verdict; a viewer that doesn't fall over on a four-hundred-step run; failure clustering so a pattern surfaces without a human reading two hundred traces to find it.

CI for the environment library. Environments rot. A base image stops building, an upstream API changes a response shape, a pinned package gets yanked, and a task that passed last month now fails for reasons that have nothing to do with the model. The gate that catches this is cheap and almost never built early enough. On every commit, run a reference solution against the task and require it to pass, then run a do-nothing agent and require it to fail. When both pass, the task is broken, and it's been quietly inflating your pass rate since the day it broke.

Task authoring is the only part of that list the human-data market is built to sell. The rest is ordinary backend, infrastructure, and platform engineering done under an unusual constraint. Grader design sits on the engineering side of that line even though it looks like task work from the outside, and that misread is where a lot of scoping goes wrong.

Why the contributor pools top out

The large human-data vendors are good at what they're designed for. Their business converts a big pool of screened contributors into completed tasks at a predictable rate. Four structural features follow from that design, and all four work against you once what you need is engineering.

The unit sold is a task or an hour. Priced that way, the pool has to be fungible. You don't interview anyone, you don't keep anyone, and the vendor's incentive runs toward interchangeability, because interchangeability is what makes throughput predictable.

Terms are short by design. Public contributor programs routinely post fixed three-month engagements, several with a hard cap inside a year. Whoever finally internalized your reward design and your repo layout hits a term boundary and rotates out. The replacement restarts the ramp, and nothing transfers, because nobody documents why grader v4 exists for the benefit of a contributor who won't be there in April.

Rates compress, and the senior end leaves first. Posted rates for coding contributors span roughly $15 to $200 an hour across public programs, with review-tier work at the bottom and a marketplace ceiling few people reach at the top. Programs under margin pressure have cut posted rates mid-engagement. The contributors who leave first when that happens are the ones with other options, which is the exact population you were trying to buy.

Nobody in that model produces pod leads. This is the gap that bites hardest at scale. A contributor pool has QA reviewers and program managers. It doesn't grow engineers who can own an environment family, set the bar inside it, take four or five people, and answer for whether the family holds up. That role is what an environments team runs short of as it passes a dozen engineers, and a model that rotates people out on a fixed clock has no path to it.

How to tell task-labeling supply from engineering supply

Vendors on both sides of this line use identical language in a first call. These questions separate them. Each one has a tell rather than a right answer.

"What does the person you'd put on this do in week one?" Engineering supply describes reading your repo and shipping something small: a fix to the provisioner, a second grader for an existing task family. Task supply describes onboarding to a spec, calibrating against reviewers, then hitting a throughput target.

"Where does the work land?" Ask whether the output is commits in your repository under your review, or rows in the vendor's platform that you export. Both are legitimate products. Only one leaves you with a system you can run after the engagement ends.

"Tell me about the last grader that got gamed." This separates the two markets faster than anything else you can ask. An engineering answer is a story: what the agent did, how it surfaced in the traces, what changed in the grader. A task-volume answer is about sampling rates and reviewer agreement.

"Who owns provisioning and orchestration?" If the vendor's scope stops at authoring tasks against infrastructure you supply, they can't take the piece that's actually blocking you. Name that scope out loud in the call and watch whether they claim it or route around it.

"Do I interview named people, and can I keep them?" Pool models can't offer this without breaking the pool. Follow it with the engagement term and what happens at the end of one.

"What's the pricing unit?" Per task and per contributor-hour price a throughput business. Per engineer prices capacity. The unit tells you which business you're buying from regardless of what the deck says.

"What do your own job postings say?" Look up the vendor's careers page before the call. Titles built around trainer, tutor, or coding specialist recruit for the task market. Titles built around environments, infrastructure, or platform engineering recruit for the other one. Vendors describe themselves accurately to candidates far more often than they do to buyers.

What seniority actually buys here

Senior is a slippery word in staffing, so it's worth naming what it buys on this specific work.

Adversarial imagination on graders. A capable mid-level engineer writes a grader that verifies the task got done. A senior one writes the grader and then spends an hour trying to pass it without doing the task. That hour is most of the difference between a grader that holds and one your researchers quietly stop trusting in month three.

Debugging the rollouts that fail for reasons that aren't the model. Rollout fleets are distributed systems and they fail like distributed systems: contention on a shared cache, a timeout that only fires under load, nondeterminism in a dependency, a container leaking state into the next run. Chasing that down is a different skill from anything in a task-authoring job description. Left alone, it surfaces as unexplained variance in every result you report.

Knowing what the researcher will ask next. Environments get built by one group and used by another, and the good ones come from people who've sat on the research side of that handoff. It shows up in small choices: what gets logged, how a failure gets labeled, whether the trace makes a failure mode obvious or leaves it to be reconstructed by hand.

People who can run a pod. At five engineers you need good engineers. At twenty you need three or four who can own an environment family and answer for it. Those people are usually made rather than hired. They come out of senior engineers who've spent a few months inside your environments and then take a team, which is only possible if the staffing model lets anyone stay that long.

Staffing that flexes between hillclimbs

Demand for this engineering is lumpy by construction. A hillclimb opens with a burst: a new environment family stood up, a task set authored against it, the harness extended to cover a tool surface it didn't have before. Through the run the load shifts toward triage and observability work, which is smaller and shaped differently. The run ends, analysis happens, the next target lands, and the burst repeats.

Headcount sized to the peak sits idle in the trough. Headcount sized to the trough turns every hillclimb into a two-month hiring delay. Teams that have run more than a couple of these tend to land on the same split.

A permanent core owns the library and the harness. Small, on staff, stable. The provisioner, the orchestration layer, and the eval harness are long-lived systems, and someone has to hold them in their head across quarters. This is the group to keep off contract.

A surge layer arrives with each hillclimb. Senior engineers who stand up the new environment family and its task set, then stand down. What makes this affordable is continuity. The same named people come back for the next climb, so the second engagement skips the ramp entirely. A pool that reshuffles between engagements charges you that ramp every single time.

Contract shape matters more than rate. Ask a vendor whether you can go from two engineers to twelve and back inside a quarter without renegotiating. If the answer involves a fresh statement of work each way, the model doesn't fit the demand curve you actually have, and you'll end up carrying people through troughs to avoid the paperwork.

What to do next

Take the seven surfaces from the section above, write them down, and mark which ones your own team owns today and which are unstaffed. Most teams find the gap sits in provisioning, orchestration, and CI for the environment library, and that they've been buying task volume against it because task volume is what the market kept offering. Bring the marked list into your next vendor call and use it as the scope question. A vendor who can only take the task-authoring column will tell you so inside two minutes.

Staff the layer above the task queue

A.Team matches senior, AI-native engineers to environments, tasks, and evals work, with a lead who owns delivery. Under 2% of applicants are accepted into the network of 11,000+ builders, and you interview the people you get by name.

On our largest environments engagement, at an RL-environments and agent-training infrastructure company, 45 senior builders were ramped and running inside a week, and 193 started across the engagement. Many of them grew into pod leads running their own teams. A separate engagement at an AI-agent-tooling company began as a one-builder pilot and became a 26-person team of engineers, QA, and PMs in under 60 days. Both run under NDA, which is why they're described here and not named. Yours would run the same way.

Scope Your Environments Work