Pagination and infinite scroll: how to keep content crawlable

A product list with 40 pages or a blog archive with a load-more button looks fine to a visitor. To a crawler, it can look like page one and nothing else. Pagination mistakes are one of the quietest ways sites lose index coverage, because everything still works when a person clicks around. The problem only shows up when you check how much of a paginated series actually made it into the index.

Why pagination still matters

A long list of products, articles, or listings has to be split across pages for load time and usability. Google still needs to reach every page in that series to index what's on it, and AI engines need a real URL to fetch when they want to cite a specific item buried on page 6. If a series only truly exists as page one plus some JavaScript state, everything past it is effectively invisible.

What happened to rel=next and rel=prev

For years, sites added rel="next" and rel="prev" tags to hint that a set of pages formed a series. Google confirmed years ago that it no longer uses these tags for indexing. They were never a canonicalization signal in the first place, just a discovery hint, and Google now treats each paginated page as its own independent page that has to stand on its own merits.

What to do instead

• Give every page in the series its own real, indexable URL, such as /category?page=2, not a URL fragment or a state that only exists in JavaScript.
• Link between pages with plain anchor tags that have an href, not buttons that only work after a script runs.
• Use a self-referencing canonical on each page. Page 2 should canonicalize to itself, not back to page 1.
• Write a distinct title and meta description for each page where you can, even if it's just adding "page 2" to an existing pattern.

Infinite scroll and load-more buttons

Infinite scroll feels smooth for a reader, but crawlers don't scroll and they don't reliably click buttons. If new content only appears after a scroll event or a click, and there's no other way to reach it, that content may never get crawled. The fix isn't to abandon infinite scroll. It's to give it a paginated skeleton underneath.

• Update the URL with the History API as the user scrolls, so each batch of content has a real, shareable address.
• Make sure requesting a page URL directly, server-side, returns that page's content, not an empty shell that waits for a script to fill it in.
• Don't make a load-more button the only path to later content. Pair it with plain paginated links, even if they're visually hidden for most users.

Canonical tags in a paginated series

A common mistake is canonicalizing every page in a series back to page 1. That tells search engines and AI crawlers that pages 2 through 40 don't really exist as separate pages, which means anything only listed on those later pages can quietly drop out of the index. Each page in a paginated series should canonicalize to itself. Save real canonicals for actual duplicates, like a page reachable through two different URL parameters.

When a view-all page helps

For a list short enough to load without hurting performance, a single view-all page can be worth adding alongside the paginated version. It consolidates ranking signals onto one URL, and it gives AI engines a single page that contains the full list, which suits how they tend to favor a page that answers a query completely over one that requires following a chain of links. For a catalog with thousands of items, a view-all page usually isn't worth it. It will drag down load time and Core Web Vitals more than it helps.

A page that only appears after a click doesn't exist to a crawler.

If you run a site with any paginated series, product lists, blog archives, forum threads, pull up page 3 or 4 in an incognito window with JavaScript disabled. If the content is still there, a crawler can likely find it too. If the page is blank, that's the gap to fix first: give each page a real URL, a self-canonical, and a plain link pointing to it.