How to optimize your site for AI agents, not just crawlers
AI browsers like Comet and Atlas now click, fill forms, and complete tasks on a site instead of just reading it. Here is what that changes for SEO.
Search traffic used to mean one thing: a person clicks a blue link. In 2026, a growing share of visits to any given site come from something else, an AI agent acting on a person's behalf. Tools like Perplexity Comet, OpenAI's Atlas, and Claude's browser extension do not just read a page and summarize it. They click buttons, fill out forms, and try to finish a task before handing the result back to the user. That changes what it means for a page to be optimized.
Why agents are different from crawlers and chatbots
A search crawler indexes a page once and moves on. A chatbot answering a question fetches a page, reads the text, and generates a response. An agent does both of those things, then keeps going. It might add an item to a cart, submit a contact form, or compare prices across several tabs. If any step in that chain breaks, the agent stalls or gives up, and the site never gets credit for the task it could have completed.
This is still an early, fast-moving practice. There is no single standard yet for how agents identify themselves, what they are allowed to do on a page, or how a site should signal what is safe to automate. Treat everything below as directional, not settled, and expect it to keep changing.
What breaks for an agent that a person barely notices
Agents run into a specific set of failures, most of them small enough that a human visitor works around them without thinking.
- Buttons and links with no visible text, just an icon or a background image, so the agent has nothing to read
- Forms that only submit correctly after a client-side script event that a driven browser doesn't trigger the same way a real click does
- Multi-step checkouts or logins gated behind a CAPTCHA, which stops bad bots and legitimate agents alike
- Content that only appears after a hover, a scroll animation, or a modal that opens on a delay
- Critical details, like price or availability, that load from a script after the page's initial render
Most of the fixes are accessibility fixes
The useful part of this is that it overlaps heavily with basic web accessibility, which most sites already have reasons to invest in regardless of AI traffic.
- Use real buttons and links, not a div with a click handler, so the semantic role is unambiguous
- Label every form field with a proper label element or an aria-label, not just placeholder text
- Keep the text that identifies an action readable: "Add to cart" tells an agent more than an icon alone
- Render critical content, especially price, availability, and the primary action, in the initial HTML instead of behind a script
- Save CAPTCHAs for flows with real fraud risk, not for browsing, comparison, or availability checks you actually want automated
Structured data still helps, cautiously
Schema.org includes action types, such as ReserveAction and OrderAction, meant to describe what a page lets a visitor do. Support for these among AI agents is inconsistent, and no major agent vendor has published a reliable spec for how it reads them. Add them if the data is already modeled and the cost is low, but do not treat them as a guarantee that an agent will find or use them correctly.
Deciding which agents can act on your site
robots.txt still governs crawling, but it was never built to separate "read this page" from "take an action on this page." Some agent products send a distinct user agent string, so check server logs for names like Perplexity-User or ChatGPT-User showing up alongside form submissions and clicks, not just page fetches. Where you want to block automated purchases or account changes, enforce that in application logic (rate limits, authentication, confirmation steps), not only in robots.txt. robots.txt is a request, not a lock.
An agent that can't find the button doesn't file a complaint. It just closes the tab.
Where to start this week
No one can fully test for every agent today, and the tooling to do it is still thin. Start with the changes that also help human visitors, then treat anything more speculative as cheap to build and cheap to change.
- Audit your top landing and conversion pages for content or actions that depend on JavaScript execution or hover states
- Check that every actionable button and link has real, visible text describing what it does
- Search a week of server logs for known agent user agents and see which pages they hit and where they stop