Two Reddit Scrapers, One Query, Four Posts in Common

Copy this line to your agent to search Reddit for a keyword.

set up https://monid.ai/SKILL.md and use apify trudax/reddit-scraper-lite for a search term

Somebody on r/apify asked why a Reddit scraper returns posts that do not match their keywords. On 2026-09-08 we put the same query through two different Reddit endpoints in the same minute, with the same sort order, and checked. The results were not irrelevant. Every post came back on topic. The two providers still returned different sets, and they agreed on four posts out of ten. This guide runs through Monid, the OpenRouter for agent tools.

Apify Reddit scraper returns irrelevant posts that don't match search keywords, am I doing something wrong?

Probably not, and on our measurement the results were not irrelevant either. The real problem is a different one.

What we ran

Query rag pipeline evaluation, posts only, sort by relevance, on 2026-09-08. Ten posts came back from apify/trudax/reddit-scraper-lite. We then checked every title and body for the three query terms.

What came back

All ten titles carried at least two of the three terms. All ten bodies carried at least three. The top result was "Not a developer. Accidentally built a RAG pipeline anyway"; the last was "Fresh Grad Solo Project: Am I over-engineering my RAG pipeline evaluation?". Six of the ten sat in r/Rag, the rest in r/LangChain, r/learnmachinelearning and r/dotnet.

On a specific multi-word technical phrase, relevance was not the failure mode. So if you are seeing junk, the first thing to check is the query rather than the actor.

The three things that actually cause the symptom

A short or generic term. Reddit's own relevance ranking widens aggressively on one-word queries, and a scraper faithfully returns what Reddit ranked. A search for rag and a search for rag pipeline evaluation are different problems.

searchCommunities or searchUsers left on. The actor takes separate booleans for posts, comments, communities, users and media. Leave the extra ones true and you get subreddit and profile records mixed into what you assumed was a list of posts, distinguishable only by the dataType field you probably dropped.

Sorting by new and reading it as relevance. New returns whatever was posted most recently that matched at all, which on a broad term is mostly noise. The sort is the single highest-leverage argument in the schema.

The one that surprised us

We asked for maxItems: 15 and received 10 rows, and were billed for 10. The limit is a ceiling, not a target, and Reddit's search surface simply had fewer results it was prepared to hand over.

📖 See also Reddit Scraper: How to Get Reddit Data After the API Lockdown

Is there any alternative to apify for Reddit?

Yes, and running both at once is how the more interesting problem became visible.

The second route

tikhub/api/v1/reddit/app/fetch_dynamic_search takes a query and a search_type, bills per call rather than per result, and returns in about four seconds against roughly three minutes for the browser-based actor.

It also returns something structurally different, which matters more than the speed.

The same query, two answers

Same term, same relevance sort, same minute:

Four posts in common. Six were unique to one provider and three to the other.

What that actually means

Neither of them returns "the" Reddit results for a keyword. Both are reading a ranked, personalised, session-dependent surface, and two readings taken a minute apart differ.So if you monitor a brand or a keyword on one Reddit scraper, you are seeing roughly half of what a second scraper would have shown you on the same query. That is not a bug in either product. It is what search over a ranked feed is.

The practical response

Run two, deduplicate on post id, and accept that even the union is a sample. Two readings of one platform priced honestly against each other is the exercise in Apify vs TikHub for TikTok scraping. That is the same argument for redundancy made in not depending on one scraping vendor, and Reddit is where it shows up fastest because the ranking changes under you.

How do you pull Reddit posts for a keyword?

Three steps, and discovery is free.

For agents

Grab an API key at app.monid.ai, then paste this to your agent and hand it the key:

set up https://monid.ai/SKILL.md

It learns the whole discover, inspect, run workflow itself. More in the agent quickstart.

For humans

npm install -g @monid-ai/cli
monid keys add -k <your-api-key> -l main

Step 1. Search with the narrow route first

What it does. One call, one flat page of ranked posts, no browser.

The endpoints. tikhub/api/v1/reddit/app/fetch_dynamic_search, billed per call, requires query.

The call.

monid run -p tikhub -e /api/v1/reddit/app/fetch_dynamic_search \
  --query '{"query": "rag pipeline evaluation", "search_type": "post", "sort": "RELEVANCE"}'

What comes back. Reddit's GraphQL response. The posts are not at the top level: walk the tree for nodes whose __typename is SearchPost and take the post object off each one.

What it costs. A fraction of a cent per call, and the charge does not move with the number of posts. Current figures at monid.ai/tools.

Step 2. Search with the broad route when you need the body text

What it does. Renders the page, so you get full post bodies and media links.

The endpoints. apify/trudax/reddit-scraper-lite, billed per result with a flat fee per run.

The call.

monid run -p apify -e /trudax/reddit-scraper-lite -w \
  -i '{"searches":["rag pipeline evaluation"],"searchPosts":true,"searchComments":false,"searchCommunities":false,"searchUsers":false,"sort":"relevance","maxItems":15,"skipComments":true}'

What comes back. A flat array with title, body, communityName, upVotes, upVoteRatio, numberOfComments and url on every row. Note the four search* booleans set explicitly rather than left to default.

What it costs. A fraction of a cent per result plus a small flat fee per run, so the flat fee dominates on tiny pulls. Keep maxItems honest.

Step 3. Merge on the post URL, not the title

What it does. Turns two partial views into one list you can trust more than either.

The call. No endpoint, just the join:

seen = {}
for row in apify_rows:
    seen[row["url"]] = row
for post in tikhub_posts:
    seen.setdefault("https://reddit.com" + post["permalink"], post)

Titles are edited and re-posted; the permalink is stable. Deduplicating on a normalised title is what makes two copies of the same thread look like two threads.

📖 See also Is There an Alternative to Apify for Scraping Reddit?

Which fields survive the switch between providers?

Almost none of the names, and that is the tax nobody budgets for.

The same data under different names

Note upVoteRatio against upvoteRatio. One capital letter, and a mapping written by hand will pass review and return undefined in production. The same shape of failure as the enum mismatch described in a wrong enum returns zero.

The field one has and the other does not

The GraphQL route carries viewCount, awardings, flair, languageCode and a moderation block that the flat actor does not expose at all. Views in particular are the one Reddit metric no other route gives you.

On our seven rows, viewCount was null on every one. The field exists in the shape and carried nothing. That is worth knowing before you design a dashboard around it: a field being present in the schema is not a promise that it is populated, and the only way to find out is a call.

The nesting is a real cost

The flat actor gives you rows[i].title. The GraphQL route gives you the same title six levels down, under search.dynamic.components.main.edges[0].node.children[i].post.postTitle, and the shape of that tree is Reddit's to change. Walking for __typename === "SearchPost" rather than indexing a fixed path is the difference between a parser that survives a Reddit release and one that does not.

The enum that fails loudly, which is the good case

Passing sort: "relevance" returns HTTP 422 with a body naming the legal set: RELEVANCE, HOT, TOP, NEW, COMMENTS. Lowercase fails. This is the behaviour you want, and it is worth contrasting with an endpoint that accepts a bad filter and quietly returns an empty set instead.

One wrinkle: the provider's 422 arrives inside a Monid run whose own status reads COMPLETED, because the call was delivered successfully and the provider answered. Check providerResponse.httpStatus, not just the run status.

Which endpoint should I use for which job?

Every row was verified with monid inspect on 2026-09-08. The table gives billing shape rather than figures, because shape is what drives design and current numbers live on monid.ai/tools.

The billing split is the whole design decision here. Per call means a wide, frequent sweep across many keywords costs the same whether each returns two posts or two hundred. Per result means a deep pull on one term is priced by what you actually keep. Most monitoring wants the first and most research wants the second.

When is a Reddit scraper the wrong tool?

Three cases, and the first one catches people out.

You need every post, not a ranked sample. Search endpoints read a ranking. If you need completeness on a subreddit, crawl the subreddit by URL with a date limit instead of searching, and even then accept that removed and quarantined content will not appear. Our own measurement, four common posts out of seventeen, is the evidence for how far a ranked sample is from a census.

You are doing anything commercial with the text at scale. Reddit's terms restrict redistribution and model training on their content, and the 2023 pricing change to the official API was the company deciding to charge for exactly that. Reading public pages to understand a market is a different activity from bulk-licensing a corpus, and if you are doing the second you want a license, not a scraper.

You want author-level identity. A username is not a person, and joining Reddit activity to a real identity is both technically unreliable and an ethical problem. If the job is to find people rather than to understand a conversation, the LinkedIn route is the honest one.

You want the recurring complaint, not the corpus. Pulling the posts is the cheap half. Reading a few thousand of them to find the problem that keeps coming back is the work, and it is a summarisation job rather than a retrieval one. Products built for Reddit pain point research run that pass for you, which is a different purchase from an endpoint and the better one when you want the conclusion rather than the rows.

And the disclosure: this is Monid's blog and we sell per-call access to both endpoints compared above. The finding in this article is that one of them is not enough, which happens to argue for buying two things from us, so weigh it accordingly. The measurement is reproducible in two calls if you want to check it yourself.

Conclusion

The complaint that started this article was that a Reddit scraper returns irrelevant results, and on a specific multi-word query it did not. Every post we got back was on topic. That is worth saying plainly, because the fix people reach for is switching vendors, and switching vendors does not fix a query that is too broad or a sort left on new.

What we found instead is harder to notice and matters more. Two providers reading the same keyword in the same minute agreed on four posts out of ten. Reddit search is a ranked, shifting surface, not a table, so a single scraper is a sample of a sample. If your brand monitoring runs on one endpoint, it is not wrong so much as partial, and it has no way to tell you which half it missed.

Free next step: run the same keyword through both endpoints and diff the URLs. monid discover -q "reddit" lists what is available and costs nothing, and inspect shows each schema and price before a single call bills. Start at monid.ai.

FAQ

Last updated September 2026.