{No thinking needed}Hand this tutorial to your agent in one click

A key section of this article is locked, and an agent opening the link cannot see it either. Unlock it first, then copy the full text for your agent in one click.

Let Your AI Agent Reveal a Rival's Search Traffic, Free

Share

Intermediate tutorial: skip paid SEO tools. Have your AI agent read a rival's public sitemap and count its page types, check the Wayback Machine to see what it has been adding lately, open pages to judge template depth, verify demand with Google autocomplete, and compare everything with your own site to tell you which page to build first. Includes real results from two sites and 3 prompts you can copy and paste.

Machine translation. Read the original

What you get

Give your AI agent a competitor's URL, and it uses only free public data to tell you which kinds of pages bring that site Google search traffic, which type it has been adding more of lately, and which search demands sit behind them. Then you get three to five steps you can follow. There's no SEO tool to buy, no account to register, and no API key. The article includes three prompts you can copy directly.

Set your expectations first: free data can't give you exact figures like "how many searches a month" or a "keyword difficulty score". What it tells you is where your rival is putting its effort, and whether people are searching for these topics. That's enough to get the direction right. Deciding which page to build is still up to you.

Five-step process: count pages, check the timeline, read templates, verify demand, write the playbook

Why free data is enough to see the direction

Paid SEO tools also estimate their traffic numbers from rankings. Often you can see the game without the numbers, because the rival's website shows its approach openly:

  • Sitemap: most websites publish a list of every page they want Google to index. Whichever page type appears far more often than the product count justifies is the one the site built in bulk for search.
  • Web archive: the Internet Archive's Wayback Machine records when each page was first archived. That shows which page types only started appearing in large numbers in recent years.
  • Search box autocomplete: Google's search box completes phrases people commonly search for. If a word completes into a string of long-tail phrases, real people are searching for it.

In November 2026 we ran this method on two real sites:

Real results: Allbirds category pages are 60% of its site, Zapier integration pages are 96%

The shoe brand Allbirds has over two thousand pages and only about three hundred of them are product pages, yet over thirteen hundred are category pages, with men's, women's, colors, and materials repeated in the URLs. That shows it is building pages one by one for searches like "black running shoes" and "wool running shoes". The automation tool Zapier has over 100,000 pages, 96% of which are "how to connect app A with app B" integration pages, generated in bulk from one template. The Allbirds finding matches what a paid SEO tool shows too: its color-based category pages are the most worth learning from.

Step 1: hand this process to your AI agent

We wrote the whole process as a Skill (a specialized instruction manual for AI agents): free-seo-teardown. It includes a small script that uses only Python's built-in features, so nothing needs to be installed. Send this to your AI agent:

Install this Skill: https://cwyz.com/api/skills/free-seo-teardown/raw
1. Read the whole thing first, then tell me in plain language what it will do and which websites it will visit.
2. Save seo_teardown.py from it into a folder I agree to, and tell me the location before you save it.
3. Check whether Python 3 is on my computer. If it isn't, ask me first whether I want to install it.
4. Once it's installed, run census once with example.com to test it, and tell me whether it works.

Step 2: pick one rival and let it run start to finish

Don't pick the biggest company in the industry. Pick one that's a bit ahead of you and about your size. Its approach is one you can actually learn from.

Sign in to unlock this key section

New accounts get 10 free credits, plus 1 more each day you sign in (none on days you don't). Spend 1 credit to unlock and it stays open for you.

The second point asks it to open pages itself, because counts alone can't show quality. If the rival's category page is just one line of intro and a row of products, you have a real chance to win by writing more carefully.

Step 3: for traffic estimates, get a second source yourself

To see rough traffic and backlink figures, open Ahrefs' two free tools yourself: Website Traffic Checker and Backlink Checker. Enter your rival's URL, copy the top pages and the pages with the most backlinks from the results, and send them to your AI agent:

This is [competitor] data I copied from Ahrefs' free tools (third-party estimate):
[paste the top pages and the pages with the most backlinks]
Please compare it with the report you just gave me: which conclusions it supports, which ones need changing, and then update the playbook.

These two tools have human verification, so you open them yourself and keep your AI agent away from them.

After you get the report

Only build the first page in the report's ranking. Then wait a few weeks to see whether it shows up with impressions in Search Console before deciding whether to build that type of page in bulk. For how a new site reads this data, see A New Site With Zero Backlinks Uses a Four-Week Experiment to Find Pages Worth Betting On. To keep watch on a rival's new moves every week, see Have Your AI Agent Watch Your Competitors Every Week.

Notes

  • More pages doesn't mean more traffic: the count only shows where the rival is betting. Some bulk pages never rank, so always open the pages in step 2.
  • Wayback numbers only show trends: pages that were never archived are missed, and old pages that were deleted still get counted.
  • Some sites don't publish a sitemap. In that case your AI agent will read the navigation bar instead, and the conclusions will be rougher.
  • Traffic from a brand name can't be copied: big brands get many searches for their own name, and no page you build can win that traffic.

References

Done with this one? Mark it to track your path.Sign in
Save it for laterSign in
Questions about this piece?Ask AI about this

Comments (0)

No comments yet. Start the conversation.

Sign in to join the discussion.Sign in