Open SERP Checker: a self-hosted Google rank tracker

By Muninder Adavelli. Scraping, SEO, and proxies for over a decade.

1. Set up an Open SERP Checker that works in less than a minute

Two setup paths. Docker is the fastest. Python is for when you want to hack on the code.

Docker (recommended)

Python (from source)

Pick your engine, pick your proxy

Bing works out of the box with no proxy. Add a keyword, pick bing, hit go. It uses your local IP, which Bing treats pretty generously (good for 15 to 50 keywords a day from the country your box physically lives in). Fine for a solo starter list.

The moment you need something more, you need a proxy. US Bing results from a European box? Proxy. Tracking real Google ranks? Proxy. Production-scale volume on either engine? Proxy. In every case, the setup is the same: paste residential proxy credentials into the project settings, change the engine dropdown, done. The first Google run also fetches the Camoufox browser (~500 MB) so budget a minute for the warm-up.

Full engine configuration is in docs/engines.md in the repo.

If you've used a rank tracker before, this is no different

Same process, same mental model: create a project, set defaults, add keywords, pick a schedule. The only real difference is you buy a residential proxy and paste the credentials into the project settings instead of trusting that your SaaS vendor's "unlimited" tier is actually unlimited.

If you're new to rank tracking

1. Create a project. One project per site or per hypothesis you're testing.
2. Set the project defaults. Country, language, device, engine. Everything downstream inherits these unless a specific keyword overrides them.
3. Add keywords. Paste a list, upload a CSV, or add them one by one in the UI. Each keyword can override project defaults if the test needs it.
4. Set the scrape schedule. Daily, weekly, hourly, or ad-hoc. The scheduler respects per-engine rate limits.
5. Let it run. The database fills up. The dashboard renders. The CSV exports work. Open Claude, point it at data/ranks.db, and ask.

Every fetch leaves a raw HTML snapshot in data/snapshots/. If Google changes the SERP tomorrow, you can reparse last year's data without re-fetching anything.

2. What it does

Not a feature list for marketing. The surface area of the tool, flat and factual, with what's shipped and what's queued.

3. What's included

Five engines ship in the box. The scrapers need proxies for Google; the APIs don't.

Proxies for the Google engine

The Google engine needs rotating residential proxies to pass JS-challenge checks reliably. So far I have only measured one provider at the depth required to publish a number:

4. Why I built this

A lot of variables define success in SEO, and the same applies to AI-SEO now. I have always believed the dependable route to success is coming up with hypotheses based on data that is out there, and testing those hypotheses. To make that happen, you need a SERP checker without restrictions.

But every SERP checker in the market today is a rank tracker. They are made to monitor keyword ranking positions. You update your keyword list; they show your position changes over time. None of them, by default, are hypothesis testers or research assistants.

That is the conceptual reason I built this. There are measurable reasons too.

Rank trackers are too expensive, even just for rank tracking

The restrictions are what make the already-high prices worse. 200 keywords tracked daily, 75 keywords daily, these caps are the thing that pushes you up into a $100-plus-per-month plan when the cost of the underlying rank data is a fraction of a cent.

SERP APIs are the same shape, differently packaged. The cheapest option is Serper.dev at $0.001 per query, but you buy $50 of credits upfront. That is a tax on experimentation: before you have measured anything, you have committed money to a vendor.

Rank trackers are not customizable, and SERP APIs give you what they want to give you

SERP APIs are fine for a lot of workloads, but you get the fields the vendor decided to expose, in the shape the vendor decided to expose them in. If your experiment needs a format the vendor doesn't emit, you build an adapter around their API instead of building the experiment.

Rank trackers are worse. You get a dashboard, a few CSV columns, and whatever the vendor's reporting roadmap considers interesting. You end up shaping your experiment around what the vendor measures, not what Google actually renders on the SERP.

What this unlocks

Open SERP Checker is customizable. Raw HTML snapshots on disk, parsed JSON next to them, a SQLite database you can grep, join, export, or point Claude at. If an experiment needs a new field, you add it to the parser. If a competitor comparison needs a different shape, you write the SQL. The tool stays out of the way; the SEO question stays in focus.

Open SERP Checker costs less. Not because the cost disappears, but because the cost is a residential proxy bill (which you can minimize, swap, or share across tools) rather than a vendor's margin.

Open SERP Checker acts as a research assistant, not a monitoring dashboard. The data model is "every SERP I have ever fetched, forever, in a format you can query." Monitoring falls out of that for free. Hypothesis testing becomes the thing you can actually do.

5. How I got this working after 6 failed attempts

Scraping Google at a hobbyist budget is not easy. Before writing a line of this tool, I spent weeks measuring what actually works. The tool you are reading about is the artifact; the writeups below are the receipts.

• I scraped Google SERPs with proxies, at Serper's price: the overview, 20/20 pass at ~$0.0007 per SERP on Decodo rotating.
• curl_cffi did not help me scrape Google SERPs: the TLS-fingerprint approach that didn't work, and why.
• Google blocks SERP scrapers at two layers: TLS and sorry: the two-layer blocking model that explains why datacenter IPs die on Google.
• My Playwright Google scraper pass rate on residential: the honest Chromium baseline before switching to Camoufox.
• Does humanizing a scraper help with Google SERPs: the humanize A/B test, measured not asserted.
• When residential pools burn your Google scraper: the pool-burn failure mode that no vendor tells you about.
• I finally scraped Google SERPs with Camoufox: the parked-tab mechanism that stuck.

Every design decision in Open SERP Checker (Camoufox as the default browser for the Google engine, parked-tab mechanism, residential-rotating as the recommended proxy mode) traces back to a measured result in one of those posts. The tool is not asking you to trust it. The research is the trust.