check_solution
ClaudeCheck a grid against a puzzle's clues, using the same rule as the benchmark grader. Reports which rows and columns do not match.
Overview
An open-source benchmark of how well LLMs solve nonogram puzzles, from 5x5 to 20x20. Query every result through a REST API or MCP server.
Tools
compare_models
ClaudeSide-by-side core overall and per-size accuracy, cost, latency and token results for model or family names.
get_leaderboard
ClaudeModels ranked by accuracy; defaults to all effort levels for compatibility.
get_model_puzzles
ClaudeWhich puzzles one model solved, missed, timed out on, or has not run.
get_model_results
ClaudeAccuracy, cost, latency and token use for one model, broken down by grid size.
get_puzzle
ClaudeOne puzzle, including the clue text models were prompted with. The reference solution is only included on request; some puzzles have several valid solutions.
get_puzzle_results
ClaudePer-model outcomes for one puzzle. Answers are omitted unless requested.
list_families
ClaudeModel families, available efforts and best variants.
list_providers
ClaudeProvider ids, names, families and variant counts.
list_puzzles
ClaudeThe benchmark puzzles with their ids and row/column clues.
list_runs
ClaudeIndividual benchmark runs (one model on one puzzle), optionally with the raw prompt and model output.
