Live on Google Cloud Run

Is this contest problem ready to use?

polyman verify judges a programming-contest problem the way a real contest would. Every solution runs on the problem's own tests with real compilers, in a locked-down sandbox, and you get a plain answer about what is wrong.

Built and shipped on GitLab: AI agents change this service only after a person says go ahead. See the records

This sample has one planted mistake. Can the check find it?

Each solution below is labelled with how it should behave. Run the check to see which one breaks its label, on which tests, and what to do about it. It takes about 20 seconds. Live check · sample problem “wrong-tag”

Not run yet

Built and shipped on GitLab

AI agents write changes to this service on GitLab, and they start only after a person says go ahead. CI rebuilds the record of every go-ahead from GitLab's own data, so you can check it yourself.

polyman 3.0.0 shipped 8 bugs its 967 unit tests missed. On GitLab, the judges found all 8 and the fixes shipped in 3.0.1. Before and after

Go-ahead ledger

Every push an agent made, judged against who said go ahead. Hash-chained, so a record can't be changed later.

Pipelines

The builds, tests and deploys behind this page.

Agent carbon

The carbon the review agents emit per merge request, measured as Software Carbon Intensity.

Judges

The corpus, the Polygon round trip and mutation testing judge polyman itself, on every merge request and every night.

Check your own problem

Choose a .zip of a polyman problem folder, or drop one here. A contest of up to 26 problems works too, up to 20 MB.

Make one with polyman remote pull ID ./problem, then zip the folder. How to prepare it

Your upload is deleted when the check ends. Nothing is stored.

What the check does

polyman's own ten steps, in three parts. These are the steps you watch tick off in the card.

  1. Make the tests, then prove them

    It builds the generators, generates every test, builds the validator, tests it, and checks that every generated test is valid.

  2. Prove the checker

    The problem's own answer checker is built and run on its own tests, because it decides every answer that follows.

  3. Judge every solution

    Each solution is compiled and run on the tests with the problem's time and memory limits, in a sandbox with no internet, then compared with its label.