# How we test search and booking | WhatsDo

URL: https://whatsdo.ai/methodology

Skip to contentBookingUse casesDocsBlogFor businesses

Sign inConnect

BookingUse casesDocsBlogFor businessesSign in

# How we test search and booking

We give the same San Francisco requests to WhatsDo and to other search services. A judge reads two answers without knowing whose they are and picks the one that fits the request better.

## Results on 91 requests

Wins for WhatsDo against wins for the other service. Ties are not counted. Against Google Places the result is close.

- WhatsDo vs Exa46–16
- WhatsDo vs Keenable56–13
- WhatsDo vs Google Places28–19
- Exact place asked by name10 of 12found in the first five results

## How the test works

- 1. Real requests91 requests people make in San Francisco: food, mood, nearby, nightlife, kids, hotels, typos and follow-ups.

- 2. Same request everywhereEach service gets the same words and returns its first five places.

- 3. Blind judgeAn AI judge sees two lists without service names. We remove our ratings and booking labels so the judge cannot tell which list is ours.

- 4. Judged twiceEvery pair is judged a second time with the lists swapped. A verdict counts only when both rounds agree.

## What a good answer is

The judge looks only at fit: the right kind of place, the mood or occasion asked for, the price and the area. Length and detail do not count.

An empty answer beats a list with nothing relevant. A guide page counts only for the places it names.

## Where we still lose

Other services still give better answers for hotels, event venues, activities and places for pets. These gaps come from our catalog and tags, and we work on them first.

- Hotels
- Event venues
- Activities
- Pets

## Booking

Every booking request ends with one of three answers, and the agent shows it to the person as is.

- Confirmed
- Waiting for the venue
- Not possible

## Limits

- San Francisco only, where our catalog is built out: 22,379 places, 1,206 of them take booking requests.
- One judge model, relevance only. Pairs where the two rounds disagree are left out.
- Other services got the request with the city added when it was missing.

## Try the same requests with your agent

Connect→Get API key→

© WhatsDo

## Product

MCPPlacesBookingPayments

## Build

ConnectDocsUse casesBlog

## WhatsDo

How we testFor businessesTalk to usPrivacy policyTerms of serviceCookie policySite map