If you have tried pytrends recently you know the story: TooManyRequestsError on the first call, dailytrends returning 404, and an unmaintained repo. Google redesigned Trends in 2024, moved the "Trending Now" feed to a different backend, and started returning 429 to anything that looks like a script.
The good news is that the site is still a plain JSON API underneath. Here is the map I put together while building a browserless client with nothing but httpx.
The two-step token dance (explore → widgetdata)
Every Trends chart is loaded in two requests:
-
GET /trends/api/explore?hl=en-US&tz=-540&req=<JSON>— you describe what you want (keywords, geo, timeframe, property) and Google answers with a list of widgets, each carrying atokenand arequestobject. -
GET /trends/api/widgetdata/<kind>?token=<token>&req=<JSON(widget.request)>— you send the widget's own request back with its token and get the data.
The req for explore looks like this:
import json
def explore_req(keywords, geo="", timeframe="today 12-m", category=0, prop=""):
return json.dumps({
"comparisonItem": [{"keyword": k, "geo": geo, "time": timeframe} for k in keywords],
"category": category,
"property": prop, # "" web, "images", "news", "youtube", "froogle"
}, separators=(",", ":"))
Widget IDs tell you which endpoint to call:
| Widget id | Endpoint | What you get |
|---|---|---|
TIMESERIES |
/trends/api/widgetdata/multiline |
timelineData[] with time, formattedTime, value[], isPartial
|
GEO_MAP (or GEO_MAP_0 … per keyword) |
/trends/api/widgetdata/comparedgeo |
geoMapData[] with geoCode, geoName, value[]
|
RELATED_QUERIES_0 |
/trends/api/widgetdata/relatedsearches |
two rankedList entries: TOP then RISING |
RELATED_TOPICS_0 |
same endpoint, keywordType: ENTITY
|
topics — empty for logged-out sessions (see below) |
Every response starts with the anti-JSON-hijacking prefix )]}'\n — strip it before json.loads.
Two details that cost me an evening:
- The token is bound to the exact
requestJSON you send back. ChanginguserConfig→ 401. The only fields you may edit areresolution(COUNTRY/REGION/CITY/DMA) andincludeLowVolumeGeoson the geo widget. - Explore stamps
userConfig.userType. Logged-out HTTP sessions getUSER_TYPE_SCRAPER; a real Google login getsUSER_TYPE_LEGIT_USER. Time series, geo and related queries work fine for scrapers, but related topics return{"rankedList": []}. If you need topics you need your own cookies — I don't ship that.
Where Trending Now went
/trends/api/dailytrends and /trends/api/realtimetrends are gone (404). The new Trending Now page is a Google "batchexecute" app:
POST https://trends.google.com/_/TrendsUi/data/batchexecute?rpcids=i0OFE&source-path=/trending&hl=en-US&rt=c
Content-Type: application/x-www-form-urlencoded
f.req=[[["i0OFE","[null,null,\"US\",0,\"en-US\",24]",null,"generic"]]]
The inner argument list is [null, null, geo, 0, hl, hours]. The response is chunked: )]}'\n\n<length>\n[["wrb.fr","i0OFE","<json string>",...]] — read the length prefix, parse the chunk, then parse the JSON string inside it. Each trend item is a positional array:
[title, null, geo, [start_ts], [end_ts] | null, null, search_volume, null,
pct_increase, [breakdown_keywords], [category_ids], [[article_id, lang, geo], ...],
normalized_title]
I keep the mapping in one parse_trending() function and treat it as "will break someday". Note that the category argument in the request does not filter server-side (I got 566 vs 567 items with and without it), so filter on category_ids client-side.
There is also /trending/rss?geo=US, a plain RSS feed with ht:approx_traffic and ht:news_item, which makes a nice fallback when the batchexecute format changes.
Staying under the 429 radar
Google Trends rate-limits per IP and, more subtly, per session:
- Warm up first:
GET https://trends.google.com/(302 →/trends/) sets theNIDcookie. Endpoints work without it on a fresh IP, but sessions without it hit 429 far sooner. -
relatedsearchesis the touchiest endpoint — it 429s on the first call in a burst but is fine with ~5 s spacing. - On 429 (an HTML "Error 429" page, no
Retry-After): back off 5 → 10 → 20 s with jitter, drop the cookie jar and re-warm, and if you have a proxy pool, build a new client on a new IP. - Cache explore results per (keywords, geo, timeframe): the tokens stay valid for a while, so three widgets cost one explore call, not three.
In production I keep ≥ 2 s between calls and rotate the proxy session every handful of requests. A keyword with 12 months of history, regions and related queries takes about 15 seconds end to end.
What you get
A keyword becomes rows like these:
{"type": "interestOverTime", "keyword": "ai agents", "geo": "US", "date": "2026-09-07", "value": 83, "isPartial": true}
{"type": "interestByRegion", "keyword": "ai agents", "geo": "US", "geoCode": "US-CA", "geoName": "California", "value": 100}
{"type": "relatedQuery", "keyword": "ai agents", "kind": "rising", "query": "ai agents framework", "value": "Breakout"}
{"type": "trending", "geo": "US", "rank": 1, "title": "…", "searchVolume": 500000, "startedAt": "2026-09-14T02:10:00Z", "relatedQueries": ["…"], "newsArticles": [...]}
That is enough to rebuild everything pytrends used to give you, plus the Trending Now feed it never had.
Source: https://github.com/m2kyungmin/apify-actors/tree/main/google-trends-scraper (the endpoint table at the top of trends_client.py is the living version of this post). Hosted, pay-per-row version: https://apify.com/kyungminlee/google-trends-scraper.
This article was originally published by DEV Community and written by Kyungmin Lee.
Read original article on DEV Community