How to test a shopping website with an AI agent

Define a small, repeatable test

You do not need to buy something to contribute useful compatibility evidence. Start with an ordinary task you actually want to perform: find a named item, inspect a service, choose a size, or prepare a cart for review.

Note the agent, version if available, date, country or region, and whether the test starts signed in. Decide which actions you authorize and where the agent must stop. These details help other people understand why their results might differ.

Record only the tasks actually attempted

AgentShopMap uses the same seven capability fields in the merchant dashboard, contribution form, and contribution API. You can complete a report by selecting task outcomes; written notes are optional.

Mark an observed task as Works, Partially works, Blocked, Doesn’t work, or Unsure. Leave unattempted tasks blank. Reaching a later page does not prove the agent performed every earlier step: a direct product link or guest checkout may skip search or login.

• Website access: could the agent load and interact with the site?
• Product / content browsing: could it browse or search for the requested item?
• Product / service details: could it open and inspect the relevant detail page?
• AI agent login: if attempted and authorized, could it complete the sign-in flow?
• Add to cart: did the chosen item and options appear in the cart?
• Agent checkout: could it navigate the checkout flow to the point you authorized?
• Transaction completion: was a transaction actually completed with authorization? Do not infer this from reaching payment.

Describe the stopping point accurately

If something stops working, identify the task and the reason where possible. The report form distinguishes an explicit block, CAPTCHA, interface problem, authentication requirement, payment issue, and another limitation.

For example, a cart can work while checkout requires a human CAPTCHA response. Record the successful cart task and the checkout limitation separately. Do not mark the entire website as blocking all agents.

Human assistance is useful context. Record whether a person had to intervene, rather than presenting an assisted result as fully autonomous.

Submit, then update after a new test

Open the merchant’s Contribute action, select the agent, and choose your observed task results. Remove personal information, payment details, order numbers, and session tokens from any notes or screenshot. The report enters moderation before it contributes to public summaries.

An account has one current report for each merchant and agent. Update that report after a genuinely new test instead of submitting duplicates to increase the count. For repeated agent-led contributions, connect a scoped agent key from your account and follow the testing guide, rate limits, and moderation workflow.