Build a Keyword-to-Tweets Scraper on X This Afternoon

Copy this line to your agent to scrape recent tweets for a keyword.

set up https://monid.ai/SKILL.md and use apify to scrape recent X tweets matching a keyword and rank them by engagement

You can stand up a working keyword-to-tweets scraper for X (formerly Twitter) in an afternoon, no developer account and no proxy pool, using a single verified endpoint on Monid billed per tweet at a fraction of a cent. Monid is a pay-per-call data API marketplace: one integration and one wallet reach hundreds of external data endpoints across scraping, enrichment, social data, and search. This walkthrough tracks a keyword, pulls recent tweets, ranks them by engagement, and drops the result into a dataset you can rerun tomorrow. Every command below is copy-paste ready.

TL;DR

• One endpoint, apidojo/tweet-scraper, takes search terms, handles, list URLs, tweet URLs, or conversation IDs and returns full tweet records billed per result.
• maxItems sets how many tweets you get and is exactly what you pay for, so cost is predictable before you press run.
• Each record carries text, timestamp, engagement metrics, and author metadata, which is everything you need to rank by real reach instead of follower count.
• Advanced filters (minimumFavorites, start/end, tweetLanguage, onlyVerifiedUsers) run server-side, so you fetch signal and skip noise.
• The whole pipeline is five steps and stays in single-digit dollars for a daily keyword tracker.

What you are building

The finished job takes a keyword like "vision pro", pulls the most recent few hundred matching tweets, sorts them by likes plus retweets plus replies, and writes the winners to a JSON file. Rerun it on a schedule and you have a lightweight social-listening feed for any topic, competitor, ticker, or product launch.

Why not the official X API?

Worth being precise here. The X developer platform gates full-archive and recent search behind paid tiers with monthly caps and app review, and the free tier is effectively write-only. For a one-off keyword tracker that is a lot of ceremony and a recurring bill before you have pulled a single tweet. The endpoint below is maintained by scraping specialists on Apify, and Monid meters it per result, so the access and maintenance problem stays theirs and you pay only for tweets you actually pull.

Set up once

For agents

Grab an API key at app.monid.ai, then paste this to your agent and hand it the key:

set up https://monid.ai/SKILL.md

It learns the whole discover, inspect, run workflow itself. More details in the agent quickstart.

For humans

npm install -g @monid-ai/cli
monid keys add --label main --key <your-api-key>

More details in the CLI quickstart.

Step 1: Find the endpoint

Search the catalog for the capability. Discovery is free and returns endpoints ranked with provider, description, price, and a verified tag.

monid discover -q "twitter x tweets scrape search"
# -> apify /apidojo/tweet-scraper   (verified, per result)
#    plus tikhub + other X endpoints, ranked

The top hit, apidojo/tweet-scraper, is the workhorse. It is the only one you need for this build.

Step 2: Read the schema before you spend

Inspect the endpoint to see its real input fields and current price. This step is also free, and it is where you confirm the field names instead of guessing them.

monid inspect -p apify -e /apidojo/tweet-scraper
# -> input schema + per-result price (free)

The fields that matter for a keyword tracker:

• searchTerms: an array of queries. This is your keyword input.
• maxItems: the cap on returned tweets, and the exact number you are billed for.
• sort: Latest, Top, or Latest + Top. Use Latest for a fresh feed, Top when a broad term returns too little.
• tweetLanguage: an ISO 639-1 code like en to keep one language.
• minimumFavorites, minimumRetweets, minimumReplies: engagement floors applied server-side.
• start and end: date bounds for the window you care about.

There are more (onlyVerifiedUsers, onlyImage, onlyVideo, author, mentioning, geotaggedNear), and the same input object also accepts twitterHandles, startUrls, and conversationIds if you would rather sweep an account or a thread than a keyword.

Step 3: Point it at your keyword

Now the only step that bills. Pass your keyword as searchTerms, cap the pull with maxItems, and sort by Latest. Add a minimumFavorites floor so you skip the long tail of zero-engagement noise.

monid run -p apify -e /apidojo/tweet-scraper \
  -i '{"searchTerms": ["vision pro"], "sort": "Latest", "maxItems": 200, "minimumFavorites": 5, "tweetLanguage": "en"}'
# -> Run ID: 01HXYZ...

monid runs get -r 01HXYZ... -o tweets.json
# -> COMPLETED: up to 200 recent English tweets matching "vision pro",
#    each with text, timestamp, engagement metrics, and author metadata
#    billed per result

Keep maxItems small while you are dialing in the query. Every tweet in the array is a billed result, so a 200-tweet test costs a fraction of what a 10,000-tweet run does, and the schema is identical, so nothing changes when you scale up.

Step 4: Rank by engagement, not follower count

The reason to pull the metrics is to sort by them. Each record returns like, retweet, reply, and view counts, so you can rank by actual reach and ignore how many followers an account claims. A quick pass with jq gives you the top tweets:

jq 'sort_by(-(.likeCount + .retweetCount + .replyCount))
    | .[:20]
    | map({text, likeCount, retweetCount, url})' tweets.json > top20.json

That single expression sorts the dataset by combined engagement, keeps the top twenty, and trims each row to the fields worth reading. You now have a ranked, human-readable digest of what a keyword actually surfaced today.

Step 5: Make it a daily feed

Wrap steps 3 and 4 in a shell script, timestamp the output file, and hand it to cron or Task Scheduler. Because sort: "Latest" plus a start date only returns tweets newer than your last run, each day pulls the incremental window rather than the whole history:

monid run -p apify -e /apidojo/tweet-scraper \
  -i '{"searchTerms": ["vision pro"], "sort": "Latest", "maxItems": 300, "start": "2026-07-08", "tweetLanguage": "en"}' \
  -w -o "tweets-$(date +%F).json"

The -w flag waits inline so the script blocks until the run finishes, which is what you want inside a scheduled job. Point searchTerms at a Boolean query (the endpoint accepts the same advanced search operators as X itself, so "vision pro" OR "apple headset" -is:retweet works) and one job covers a whole topic cluster.

What a keyword tracker costs

Per-result billing makes the math simple: you pay for the number of tweets you keep, and maxItems is that number. A daily run at a few hundred tweets lands in fractions of a cent to a few cents per day, so a month of continuous topic tracking stays in single-digit dollars, with no subscription and no monthly minimum. Widen maxItems for a launch-day spike, narrow it on quiet days, and the bill tracks the pull. Current per-endpoint pricing lives at monid.ai/tools, and because inspect shows the price before you run, there are no surprises. The magnitudes below are a starting point, not a quote.

FAQ

Do I need an X or Twitter developer account? No. You integrate Monid once and fund one pay-as-you-go wallet. The provider handles X access, proxies, and parser upkeep, so you never touch a developer portal.

Can I search by keyword and by account from the same endpoint? Yes. apidojo/tweet-scraper takes searchTerms for keyword tracking, twitterHandles for account sweeps, startUrls for any profile, search, list, or tweet URL, and conversationIds for a thread, all in one input object.

How do I control cost? With maxItems. It caps the number of returned tweets and is exactly what you are billed for, so you set your ceiling before the run starts. Add minimumFavorites or start/end to spend only on tweets that clear a bar.

What fields come back per tweet? Tweet text, timestamp, tweet and profile URLs, engagement metrics (likes, retweets, replies, views), author and profile metadata, media attachments, language, and geolocation tags where present.

Can an agent run this pipeline? Yes. Monid ships as an MCP server, so an agent handed "track vision pro on X daily" can discover the endpoint, inspect the schema, and run the pull on its own. See monid.ai/SKILL.md.

Try it

Run a free monid discover for the X data you want, inspect apidojo/tweet-scraper to confirm the fields and price, then pull 200 tweets for a keyword you know well. If the ranked digest matches what you would find scrolling the app, wire it into a daily job and move on. Start at monid.ai.