How it works

Upload a polyman problem, or a contest of them, and the service runs polyman verify on it: it compiles everything, generates and validates the tests, runs every solution and checks that each one gets the verdict its tag promises. Then it goes through the contest guidelines.

Prepare the zip

A problem is a polyman folder: a Config.json and the files it names. If the problem lives on the problem platform, pull it first:

polyman remote pull 123456 ./my-problem
cd my-problem && zip -r ../my-problem.zip .

For a contest, put one folder per problem in the zip (one level deep, or inside a single top folder):

contest.zip
  A/Config.json ...
  B/Config.json ...

You don't need testlib.h, compiled binaries or generated tests: they are made fresh, and polyman's compile cache (.polyman/) is ignored.

What gets checked

Generators compile and generate every test in the script.
The validator compiles, passes its own tests, and accepts every test.
The checker compiles and passes its own tests.
Every solution compiles and runs on every test, with the time and memory limits from Config.json.
Each solution's verdicts match its tag, which the check page calls its label (the same tags as on the problem platform):

Tag	Must
MA	pass every test (the main correct solution)
OK	pass every test
WA	get Wrong Answer on at least one test
TL	exceed the time limit on at least one test
ML	exceed the memory limit on at least one test
RE	crash on at least one test
PE	get Presentation Error on at least one test
TO	be correct, though it may run out of time
RJ	fail at least one test, for any reason

Contest guidelines

A problem can run correctly and still not be ready for a round. These checks read the files; the must-fix ones decide whether it's marked ready.

Check	Severity
The statement has a story, input and output	Must fix
At least 10 tests	Must fix
Has an answer checker	Must fix
Has an input validator	Must fix
Has a main correct solution	Must fix
The answer checker passes its own tests	Recommended
Correct solutions in C++ and Java, plus a wrong one	Recommended
Has a written tutorial	Recommended
Has topic tags	Nice to have

API

The page uses the same API you can call from a script or CI.

curl -F problem=@my-problem.zip https://SERVICE_URL/api/verify

The answer streams one JSON object per line while it works:

{"type":"start","problems":["my-problem"],"polyman":"3.0.0"}
{"type":"step","problem":"my-problem","step":1,"title":"Compiling Generators"}
{"type":"problem","problem":"my-problem","ok":true,"seconds":19.3,"report":{...},"readiness":[...]}
{"type":"done","ok":true,"seconds":19.4}

report is exactly what polyman verify --json prints (or null with an error if the run timed out or crashed), and readiness is the list of guideline checks. The last line is done, or {"type":"error"} if the service itself failed.

Route	What it does
`POST /api/verify`	multipart field `problem`: the zip
`GET /api/samples`	the sample problems
`POST /api/samples/NAME/verify`	check a sample
`GET /api/version`	polyman version and commit
`GET /health`	liveness

Answers you might get instead: 400 with the reason (the zip or Config.json was refused), 413 (over 20 MB), 429 (busy with another check; retry after the retry-after seconds).

Limits and privacy

Up to 20 MB per zip and 26 problems per contest; 120 s per problem.
Every path in Config.json has to stay inside the problem folder, and names (solutions, testsets, generators) have to be plain names.
Your code runs as a user of its own with no route to the internet. Nothing it starts outlives the check.
Your upload is deleted when the check ends. Nothing is stored.
The figures on the front page (go-ahead ledger, pipelines, agent carbon, incidents) are read by your browser from public gitlab.com pages and the project's public API, without signing in. Nothing about you is sent with them.