Claiming and the lease

Claiming

POST /api/v1/runner/jobs/claim   { "leaseSeconds": 600, "waitSeconds": 25 }

Answers a job, or 204 No Content when the queue holds nothing for this Runner.

Claiming is SELECT … FOR UPDATE SKIP LOCKED over queued jobs. Two Runners claiming at the same instant take different jobs; neither waits for the other.

waitSeconds asks to be told rather than to ask again

Without it the Server answers an empty queue at once and the Runner decides when to try again — up to thirty seconds later, on the shipped backoff. With it the Server holds the request open until a job is claimable, and answers 204 only when the wait runs out.

Nothing else changes, and that is the point. A job still leaves through one claim, still under SKIP LOCKED, and still cannot be handed to two Runners. There is no socket, no subscription and no reconnection logic — a dropped long poll is simply a request to make again.

Three things an implementation has to get right:

• The Server clamps it, as it clamps the lease. Asking for more is answered sooner, not refused. This Server's ceiling is 300 seconds.
• Each deadline is shortened by up to a sixteenth, so a fleet that started together does not answer its 204s in the same instant and ask again as one. A Runner always hears back at or before what it asked for.
• Your own request timeout must exceed the wait, or an ordinary empty answer arrives as a failed request.

Being told is an optimisation and not a guarantee. The Server may answer 204 at any moment, and a Runner that is never told finds the work on its next ask. Nothing about correctness rests on it.

204 asks four questions and answers with one word

A job is offered only if all four hold, and a Runner cannot tell which of them left the queue empty:

• the problem's type is one this Runner declared, by string equality;
• the problem's external flag equals this Runner's, in both directions;
• the work's pools and this Runner's share at least one tag, where an empty list on either side means default;
• no other attempt of the same submission is already being judged.

Only the third is set by an operator, and it is what reserves a room of machines for one contest. The work's pools are the round's where it has any and its activity's otherwise, read at claim time — so retagging redirects work that is already waiting.

The fourth is not about this Runner at all. It exists because the row lock hands two claimable jobs to two Runners — correctly, having no reason to know they are the same source — so without it a rejudge issued while an attempt was in flight would have one submission judged by two Runners at once, and where the Runner forwards, the same solution submitted twice to somebody else's service.

A Runner is never told about the waiting sibling and never sees it. The attempt simply waits, and the report that ends the running one is what releases it. A rejudge does not interrupt an attempt a Runner is holding — it ends only a sibling that is still queued, which by construction nobody holds, and that one leaves in a state a Runner never meets.

What a claimed job carries

{
  "jobId": "…", "submissionId": "…", "attempt": 1,
  "leaseToken": "…", "leaseExpiresAt": "…",
  "problemType": "standard-io@1", "problemVersionId": "…",
  "packageFileId": "…", "packageSha256": "…",
  "files": [{ "name": "source", "fileName": "main.py",
              "fileId": "…", "sha256": "…", "sizeBytes": 42 }],
  "props": { "type": "standard-io@1", "language": "python3" },
  "config": { "type": "standard-io@1", "languages": ["python3"],
              "limits": { "timeMs": 1000, "memoryBytes": 268435456 } }
}

The three opaque documents

The Server reads none of them. It checked that each is an object under 256 kB when a manager or a participant wrote it, and that is the whole of its involvement. What is inside belongs to the problem type.

Three consequences an implementation has to get right:

• The configuration chain is two layers, not three. The Server merges nothing; the Runner performs the only merge there is — the package's own configuration with config over it. problemVersionProps is not that layer under another name.
• A missing props is an infrastructure failure, not a guess. A Runner that finds no language refuses the job as infrastructure: the job arrived incomplete, and the submission stays rejudgeable. Guessing a default would mean reporting a compilation error in a language nobody chose.
• A language the assignment excludes is refused by the Runner, as a verdict. The Server cannot refuse it, because the language is one member of a document it does not read. The participant chose it, their code may be perfect, and what they broke is a rule of the activity.

Files, and two members worth naming

fileName is what the submission is called, and it is load-bearing rather than cosmetic: a Runner may refuse a file whose extension the chosen toolchain does not accept. The Server does not name pasted source, because it does not know the language: the Client sends the name.

packageFileId and packageSha256 are the empty string, not absent, when the problem version carries no package. Treat that as an infrastructure failure rather than fetching an empty id: there is nothing to judge against, and it is not the solution's fault.

The lease

leaseSeconds on the claim is a request, not an instruction. The Server clamps it to [60, 3600] and answers with leaseExpiresAt, the deadline it actually granted.

The floor exists because a lease that could be shortened below what a real evaluation takes would hand the same job to a second Runner while the first was still working.

A note on clocks worth stealing from the reference implementation: it renews on a timer, at a quarter of the granted lease, and does no deadline arithmetic at all. Three renewals therefore fit inside every lease and two may fail in a row with the deadline still ahead. Comparing leaseExpiresAt against the local clock would make every deadline a function of how well two machines agree about the time, and a Runner is explicitly allowed to be a machine nobody administers closely. A duration is immune to skew; an instant is not.

The lease is the whole recovery story

A Runner that dies holds a job nobody will finish. The Server's reaper returns it to the queue when the lease expires. Each delivery is counted, and past the cap — five deliveries in this Server — the job is failed with a reason rather than retried for ever, because retrying for ever is how one bad package stops an installation.

So a job is queued, running on exactly one Runner, or finished — and it comes back to the queue four ways: a release, a lease that expired, a report carrying an infrastructure failure, and a key an administrator revoked while the Runner held the job. Only the first three are the Runner's own doing; the fourth is the Server acting on it, which is why approval is checked on every call rather than only at the handshake. A report with a result is the only thing that ends a job properly, and the delivery cap is the only thing that ends it without one. Which of the four hands the delivery back is what the sections below add.

A rejudge is not on it: it makes a new job, and ends only a queued sibling — never one a Runner is holding.

Giving a job back

POST /api/v1/runner/jobs/{jobId}/release
{ "leaseToken": "…" }

It means one thing: this Runner is stopping. The job returns to the queue immediately, any Runner may claim it, and nothing is recorded against the submission — it was not judged, and a release is not a result. The delivery the claim counted is given back, because being shut down is an operator's doing and a participant's attempts are not the operator's to spend.

There is no reason field, and that is the design. Every other way a job comes back without a result is a report, which carries a description of what went wrong: a Runner that cannot judge a job has something to say about it, and one being stopped has not.

Three releases are free. Past that a release counts the delivery like everything else and the cap above ends it — a Runner crash-looping under a supervisor gives jobs back exactly as an operator's restart does, and nothing here tells them apart except how often it happens.

And a claim you were never heard from about is free too, bounded the same way and for the same reason. The Server commits the handout before writing the answer to it, so an answer lost in between leaves a job you own and cannot release, never having learned the lease token. It is reclaimed when the lease expires and the delivery is given back — the first three times. Anything you did answer against, even once, counts: a renewal or a progress call is what says the job arrived.

A Runner whose lease has already gone is refused with runner.lease.stale, which is a success from its side: the job it was giving back is already back.

Renewing

POST /api/v1/runner/jobs/{jobId}/lease      { "leaseToken": "…", "leaseSeconds": 600 }

Renewing never shortens. The Server keeps the later of the deadline already granted and the one now asked for.

Replacing the deadline outright would be a trap worth seeing: a Runner that claimed for ten minutes and then renewed asking for one would move its own deadline eight minutes closer, and the obvious implementation — a short ping on a short interval — would leave itself a permanent sixty-second margin. One pause longer than that hands the job to a second Runner while the first is still computing.

POST /api/v1/runner/jobs/{jobId}/progress

reports progress. The contract says it renews nothing, so a Runner still working says so with a lease call and never leaves its deadline to a progress note.

This Server is more generous, and that is not something to build on: a progress call goes through the same code as a renewal and pushes the deadline out by the lease the job was granted — never shortening it, by the rule above. An implementation that relied on that would hold its job here and lose it against a Server that does only what the contract asks.

The third row is narrower than the others. A lease, release or progress call folds an unparseable token into runner.lease.stale — a token that cannot be a token is one the caller does not hold. A report answers 422 and names it, because a report carries a result somebody computed, and losing that to a typo deserves the more exact word.

runner.lease.foreign needs a token to reach at all: the token is compared before the owner, and a Runner's id and its lease token are written together and cleared together. A presented token that matches a job this Runner does not own is one that leaked from a live Runner.

A job id nobody issued answers 404, and a job somebody else holds answers 403 with one of the codes above. That is the opposite of the rule for files, deliberately: there the id is the whole authorization decision and the bytes are what somebody would want, so the answer says as little as it can. Here the audience is a Runner an administrator approved, the ids are random, and one bit — this id names a job — buys a diagnostic every operator needs. Without it, a Runner meeting a 404 where its lease used to be could not tell a reclaimed job from a Server that has forgotten it, and would retry the renewal for the rest of the evaluation.

Why the lease token matters twice

The leaseToken is not a job id under another name. It does two jobs, and both are load-bearing.

It makes a late report refusable rather than an overwrite. A Runner that woke up after its lease expired holds a token the Server has already retired. Its report is refused with runner.lease.stale instead of landing on top of a result somebody else computed in the meantime. Without that, the recovery mechanism would be worse than no recovery: two Runners judging the same submission, and the slower one winning.

It makes reporting idempotent. The Server keys the stored result on the lease token, so a report resent after a lost connection answers the same result rather than creating a second one. Reporting says what that looks like.

An implementation should therefore carry the lease token from the claim through to the last retry of the report, and treat a refusal naming a stale or foreign lease as "this job is no longer mine" — drop it silently, report nothing, and go back to claiming.