Start with an offline request plan.

The kit includes 12 synthetic tasks, a search runner and a review sheet. It installs nothing and uses an existing working Python interpreter.

Download evaluation kit

This kit uses REST POST /actions/search with a session JWT. For MCP JSON-RPC and OAuth, use the MCP reference.

1. Download and unpack

Open the downloaded archive and use a terminal inside the whatsdo-developer-evaluation-kit folder. Read README.md and QUICKSTART.md before running a command.

2. Inspect all requests offline

sh run.sh search --dry-run

This lists the search plan without credentials or network calls. The task questions are synthetic. They do not establish that requested fields exist.

3. Confirm test access

A live search requires an existing authorized test account. Set WHATS_DO_SESSION_JWT locally. Keep the token out of chat, shared documents and review sheets.

Request an evaluation conversation

4. Run one authorized test case

sh run.sh search --limit 1 --output ./private-results

Here --limit 1 selects one synthetic test case. It does not set the MCP search result limit. This command makes a network request to the existing REST search endpoint. A 401 indicates an access problem, not an empty search. No booking, payment or mutation is performed by the runner.

5. Review the response

Keep raw responses private. Check the actual place identity, location, category, source information and missing fields before running more cases.

Create and score the review sheet