How to find every Shopify store in a list of 10,000 domains

You have a list of domains. A conference attendee export, a directory scrape, a spreadsheet from a colleague.
You want to know which of them run Shopify, which run WordPress, which have HubSpot installed, and which are on
Cloudflare. That single question sits behind a lot of sales prospecting, competitor tracking and agency audits.

The mechanics are not complicated, and it is worth understanding them before you decide whether to build or buy.

What a website tells you without being asked

Every page you request hands over more than the HTML you asked for:

• Response headers. Server: cloudflare and a cf-ray header are Cloudflare. X-Powered-By: PHP/8.2 is self-explanatory. Shopify sends x-shopid.
• Cookies. _shopify_y, __utma, wp-settings-* each pin a platform.
• HTML and meta tags. <meta name="generator" content="WordPress 6.9.7"> is about as direct as it gets.
• Script sources. cdn.shopify.com, googletagmanager.com/gtm.js, js.hs-scripts.com.
• DNS records. MX records name the mail provider. TXT records carry verification tokens for every SaaS tool the company has ever set up, which is a surprisingly complete picture of their stack.

Wappalyzer popularised a fingerprint format for exactly this, and the open
webappanalyzer project maintains those fingerprints publicly: at the
time of writing, 7,613 technologies with the patterns that identify each one.

The two things that make a naive implementation wrong

If you write this yourself over a weekend, you will hit both of these.

Technologies are injected at runtime. Take Cloudflare's own marketing site. It uses Google Tag Manager, but
there is no <script src="...gtm.js"> anywhere in the HTML. Instead an inline script does this:

const gtmContainerSrc = "https://www.googletagmanager.com/gtm.js?id=GTM-NDGPDFZ";
// ...
var s = document.createElement('script');
s.src = gtmContainerSrc;
document.head.appendChild(s);

A fingerprint matcher that only looks at <script src> attributes sees nothing. The same applies to headless
storefronts: Gymshark is a Shopify store, but its front end is Next.js and the only Shopify URLs on the page are
in an og:image tag and a srcset attribute.

The fix is to treat vendor URLs found anywhere in the document as evidence, at reduced confidence, rather than
only trusting script tags.

Fingerprint patterns carry metadata you have to parse correctly. Patterns look like this:

(?:cdn\.|\.my)shopify\.com\;confidence:50

That trailing \;confidence:50 is a tag, not part of the regex. Split on the wrong thing and the regex keeps a
trailing backslash, fails to compile, and gets silently dropped. In the database as it stands, 223 patterns carry
a confidence tag and 1,392 carry a version tag, so getting this wrong costs you every version number and a good
chunk of your detections. It is exactly the kind of bug that produces plausible-looking output that is quietly
missing a third of the answer.

Doing it without writing it

I published this as an Apify Actor so you can run it on a list without maintaining any of the above:
Tech Stack Detector. It costs $0.02 per website and
unreachable sites are free.

curl -X POST "https://api.apify.com/v2/acts/clearfetch~tech-stack-detector/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["gymshark.com", "techcrunch.com", "notion.so"]}'

Every detection comes back with the evidence behind it, so you can audit rather than trust:

{
  "name": "Cloudflare",
  "confidence": 100,
  "categories": [{ "id": 31, "name": "CDN" }],
  "evidence": [
    { "type": "headers", "key": "server", "pattern": "^cloudflare$", "match": "cloudflare" },
    { "type": "dns", "key": "ns", "pattern": "\\.cloudflare\\.com", "match": "hera.ns.cloudflare.com" }
  ]
}

Filtering a thousand domains down to "everything running Shopify" is then a one-liner:

from apify_client import ApifyClient

client = ApifyClient("YOUR_TOKEN")
run = client.actor("clearfetch/tech-stack-detector").call(run_input={"urls": domains})

for site in client.dataset(run["defaultDatasetId"]).iterate_items():
    if "Shopify" in site.get("techNames", []):
        print(site["url"], site["company"] if "company" in site else "")

Where this approach stops working

Be clear about the limits before you rely on it:

• No JavaScript execution. Technologies that leave no trace in the HTML, headers, cookies or DNS, and are only visible as a window global after the page boots, will be missed unless another fingerprint implies them. Running a real browser would catch those, at roughly fifty times the cost per page.
• One page per site. Something that only loads on a checkout or pricing page will not show up if you only scan the homepage. Pass deeper URLs when it matters.
• Confidence is not certainty. A vendor URL inside an inline script is good evidence, not proof. Raise the minimum confidence if you would rather have fewer, harder detections.

Knowing those boundaries is the difference between a list you can act on and a list you have to re-check.