50k Tweets, No X API Tier, One Afternoon

Copy this line to your agent to pull a large set of tweets without an X API tier.

set up https://monid.ai/SKILL.md and use tikhub /api/v1/twitter/web/fetch_search_timeline to page a keyword into tens of thousands of tweets

I pulled roughly 50,000 tweets in one afternoon, from a seed of a few keywords and a handful of handles, and I never signed up for an X API tier. The whole thing ran through pay-per-call TikHub endpoints on Monid, a pay-per-call data API marketplace where one key and one wallet reach hundreds of external data endpoints (scraping, enrichment, social data, search) without a separate signup per vendor. This is the honest story of how it went, what I did with the tweets, and the parts I would tell a friend to watch out for.

TL;DR

• I needed a big tweet corpus for a research project and X priced me out, so I paged TikHub endpoints on Monid instead of buying a tier.
• Three endpoints did all the work: fetch_search_timeline for keyword search, fetch_user_post_tweet for a single account's posts, and fetch_user_tweet_replies when I wanted the conversation around a post.
• Discover and inspect are free. You only pay on run, at the price shown before you commit, so 50k tweets landed in single-digit dollars territory (current numbers at monid.ai/tools).
• Paging is a cursor loop: run, read the next cursor, run again, until the seed is exhausted or I hit my ceiling.
• The unglamorous 20 percent of the job is pacing the calls, deduping by tweet ID, and not being a jerk to the platform.

The wall I hit

The project was a language study. I wanted to see how people actually phrase a certain kind of complaint over time, which meant a lot of real tweets around a cluster of keywords, plus the full posting history of a few accounts that kept showing up. Nothing exotic. A corpus, basically.

Then I went to price the official route. If you have looked recently, you know the shape of it. The tiers jump from almost-free-but-useless to a monthly floor that only makes sense if tweets are your whole business. My project was one afternoon of collection and then weeks of me squinting at text in a notebook. Paying a recurring enterprise-ish bill to feed a one-time corpus felt absurd, and the free lane would have had me rationing calls like a drought. So I closed the tab and went looking for a side door.

Set up once

For agents

Grab an API key at app.monid.ai, then paste this to your agent and hand it the key:

set up https://monid.ai/SKILL.md

It learns the whole discover, inspect, run workflow itself. More details in the agent quickstart.

For humans

npm install -g @monid-ai/cli
monid keys add --label main --key <your-api-key>

More details in the CLI quickstart.

The afternoon, actually

I already had a Monid wallet from other data work, so this was not a new vendor relationship. It was another endpoint on a key I already had. A free monid inspect showed me the request schema and the price per call before I spent a cent, which is the check I always run first.

The search endpoint is the one I leaned on hardest. You give it a keyword and a mode, and it returns a page of matching tweets:

monid inspect -p tikhub -e /api/v1/twitter/web/fetch_search_timeline
monid run -p tikhub -e /api/v1/twitter/web/fetch_search_timeline \
  -i '{"keyword":"my phrase","search_type":"Latest"}' --query '{"count":20}' -w

I ran that once with a tiny count, opened the JSON, and read ten tweets by hand against what X showed me in the browser. They matched: text, timestamps, the engagement numbers, the author handles. That is the only trust check that matters, and it passed on the first try.

For the accounts I wanted in full, I switched to the user-posts endpoint. Same idea, one screen name in, a page of that account's tweets out:

monid run -p tikhub -e /api/v1/twitter/web/fetch_user_post_tweet \
  -i '{"screen_name":"someaccount"}' --query '{"count":20}' -w

And when a particular post was doing something interesting and I wanted the replies under it, fetch_user_tweet_replies gave me the conversation instead of just the top-line post. Three endpoints, one wallet, no client to wire up.

Paging to tens of thousands

One call gives you a page. Fifty thousand tweets is that page, over and over, following the cursor. Each response carries a next-cursor value, and you feed it back in to get the following page. In pseudocode the loop is embarrassingly simple: run, grab the cursor from the response, run again with that cursor, stop when the cursor stops changing or I hit the count I set for myself.

Because I set up the key with the skill, I let my agent drive the loop. I told it the keywords, the handles, and the ceiling, and it paged each seed, saved every page, and stopped where I told it to. For a long async pull you save each page as it lands:

monid runs get -r <RunID> -o page-07.json

The reason this scales without a budget meeting is the billing. Each call is a fraction of a cent to a few cents of territory, and cost tracks the calls I make and nothing else. Tens of thousands of tweets, paged at a small count each, came out in single-digit dollars. There was no seat, no minimum, no tier I would forget to cancel in three months. I watched the wallet tick down in amounts I would not think twice about, which is a very different feeling from staring at a subscription page. Current per-endpoint prices live at monid.ai/tools.

What I actually did with 50k tweets

The corpus was the boring, load-bearing part. Once it existed, the fun started.

I deduped by tweet ID (more on why in a second), then ran the text through a simple pass that pulled the phrasings I cared about and bucketed them by month. The keyword pull gave me the wide cross-section of how strangers phrase the thing, and the per-account pulls gave me a few deep timelines I could read as individuals changing their language over time. The reply endpoint turned out to be the sleeper: the way people respond to a complaint carried as much signal as the complaint itself, and I only got that because pulling replies was one more endpoint on the same wallet, not a second integration to justify.

None of that analysis is special. The point is that the data-collection wall, the thing that usually kills a project like this before it starts, just was not there.

The honest caveats

I would be lying if I said it was pure magic, so here is the part I would actually warn a friend about.

Pace your calls. Paging a cursor as fast as your loop allows is rude and asking for trouble. I put a small sleep between pages and ran the big pulls in the background while I did other things. There is no prize for finishing the pull in nine minutes instead of ninety.

Dedupe by tweet ID, always. Overlapping keyword searches and a user timeline that intersects your search results will hand you the same tweet more than once. I keyed everything on the tweet ID and dropped repeats before counting anything, because a corpus with silent duplicates quietly lies to you about frequency, which was the entire thing I was measuring.

And respect the platform and the people in it. This is public text, but public is not the same as consequence-free. I kept the corpus private, used it for aggregate language patterns rather than putting individuals on blast, and did not go anywhere near private or deleted content. The endpoint reads what a logged-out person could see, and I treated the output like the public record it is.

FAQ

Do I need an X or Twitter developer account? No. The tweets come through TikHub endpoints on Monid, so there is no X developer signup and no monthly API tier. Your Monid key is the only credential.

How did the cost stay so low? Every call is billed pay-as-you-go at a fraction of a cent to a few cents, shown by monid inspect before you run. Tens of thousands of tweets, paged at a small count each, land in single-digit dollars. Live prices are at monid.ai/tools.

How do you actually get past one page? Follow the cursor. Each response returns a next-cursor value; feed it back into the next run and repeat until it stops advancing or you hit your own ceiling.

Can I mix keyword search, user timelines, and replies in one project? Yes, and that is the nice part. fetch_search_timeline, fetch_user_post_tweet, and fetch_user_tweet_replies all bill from the same wallet, so combining them is free of vendor boundaries.

If you have a corpus-shaped project stuck behind an API tier you cannot justify, this is the side door. Inspect an endpoint, pull twenty tweets, check them against the app by hand, and if they match, page it out. It costs pocket change to find out, over at monid.ai.