Hostinger’s September 25, 2026 coding experiment described models missing a constraint that was already present in a repository. Instead of borrowing a leaderboard result, use that finding to build a small acceptance exercise around your own application’s rules.
1. Pick work with an observable contract
Choose a change your team understands, such as adding an optional field to a form and its API. It should cross enough layers to reveal incomplete work, while remaining small enough for a reviewer to inspect. Run the exercise in an isolated repository copy with test services and no production data.
Write acceptance criteria before the agent starts. The field should persist when supplied, older clients should still work, and validation failures should use the established response format. Include one existing local rule found elsewhere in the codebase, such as how blank values are handled.
Prepare an independent check for that rule. Keep an old solution out of the agent’s input. The goal is to observe whether it discovers the contract, not whether it can imitate a previously completed patch.
Record the initial revision, dependency state and task description. These make another attempt meaningfully comparable instead of accidentally easier.
2. Require evidence before edits
Ask the agent to identify the files that define the API contract and a comparable existing implementation. Its explanation should distinguish inspected code from assumptions. Plausible filenames alone are not evidence that it understood the repository.
Use a review sheet like this:
| Requirement | Evidence |
|---|---|
| Compatibility | The old request still succeeds |
| Validation | Invalid input gets the expected error |
| Authorization | An unauthorized user remains blocked |
| Consistency | Existing local conventions are followed |
| Completeness | Both form and API behavior are covered |
Then allow implementation within a predefined budget. Record every human hint or correction rather than quietly treating assisted success as autonomous success.
Do not score the agent by patch size. A concise change may solve the entire problem; a large patch may still omit the behavior that matters. Review changed dependencies and unrelated edits as separate costs.
3. Judge the patch, including your own repair work
Run the project’s normal checks plus the independent case prepared before the exercise. Inspect access control and any data migration directly. Tests authored by the same agent can reproduce its misunderstanding, so passing them is not sufficient evidence.
Record satisfied criteria, regressions and human repair time. If comparing models, restore the same starting state and keep tools and limits consistent. A small internal trial helps with your workload; it does not establish a universal ranking.
Adopt the agent for this particular maintenance category only when the result meets the contract and another developer can explain the patch. Merging and deployment should still follow your usual review process.
Sources: Hostinger’s production coding experiment.