Job Scraping Software: There Is No Single Job Feed
Postings are fragmented across boards by design, and no endpoint covers all of them. Pick per board, and the schema differences are the real work.
Copy this line to your agent to pull job postings from a board without writing a scraper.
set up https://monid.ai/SKILL.md and use builtin /search_jobs to pull postings for a role
Search for job scraping software and every result assumes there is one thing to buy. There is not, because there is no single place postings live. Indeed, LinkedIn, Wellfound, Built In and a long tail of niche boards each hold a different slice, none of them syndicates completely to the others, and the aggregator that appears to solve this has its own cost. The real work is not the scraping, it is deciding which boards matter and reconciling their schemas, running through Monid, the OpenRouter for agent tools.
What is job scraping software?
The category name promises something that does not exist, which is why the ranked lists in it disagree so badly.
What people picture
One tool, one query, every job posting matching it. A single feed you subscribe to.
What is actually available
Three kinds, and they are not substitutes.
A per-board endpoint. Somebody has built and maintains extraction for one board and sells it per call or per result. Coverage of that board is excellent and coverage of every other board is zero.
A library you run. Open source packages that know how to parse several boards, running on your infrastructure with your addresses. JobSpy is the best known. You get breadth and you own the operations, the blocking and the maintenance.
An aggregator's results. Google Jobs collects postings from many boards and presents them in one place. Reading that is one query instead of ten, and the next-but-one section is about what you give up.
The distinction that decides everything
Only the first has somebody else maintaining the parser. Only the second gives you breadth on day one. Only the third gives you one query. Nothing gives you all three, and every comparison article in this category is implicitly picking one and not saying so.
📖 See also Job Postings API: Turning Hiring Into a Buying Signal
Why is there no endpoint that covers every job board?
Because fragmentation is the product, not an accident, and it is enforced from several directions at once.
Boards compete on exclusivity
A job board's value to an employer is the candidates it reaches. Its value to a candidate is postings they cannot see elsewhere. Complete syndication would destroy both sides of that, so boards actively differentiate their inventory. Wellfound has startup roles that never touch Indeed. Built In has tech postings organised by city. Niche boards exist precisely because the big ones do not cover their vertical well.
The employer posts differently to each
The same role appears on three boards with three titles, three descriptions and sometimes three salary presentations, because the poster tailored each. Even where inventory overlaps, the records are not the same record, which is what makes deduplication hard later.
Nobody is licensed to aggregate everything
Complete aggregation would need agreements with every board, and boards have limited incentive to grant them. The aggregators that exist do so by crawling rather than by licence, which puts them in the same position as anybody else and gives them the same coverage gaps.
So the useful question changes
Not "which tool covers everything" but "which three boards actually contain the roles I care about". That question has an answer and it is usually a short list. Once you have it, the tooling question mostly evaporates.
Per-board vs library vs aggregator: what actually differs
| Aspect | Per-board endpoint | A library you run | An aggregator |
|---|---|---|---|
| Boards covered | One, well | Several, as maintained | Many, incompletely |
| Who fixes a redesign | The provider | You | The aggregator |
| Exit address | Included | You buy it | Included |
| Schema | The board's, normalised | Whatever the library returns | Flattened, lossy |
| Cost shape | Per call or per result | Servers plus salaries | Per call |
| Good for | Two or three boards that matter | Broad sweeps you operate | A first look |
The fourth row is the one that bites. An aggregator flattens every board into one schema, which is convenient and drops the fields that made a particular board worth reading. The same three-way sort applied to scraping generally is in Web Scraping Tools: Which Kind Do You Actually Need?.
How do you collect postings from several boards?
Three steps, two of them free.
For agents
Grab an API key at app.monid.ai, then paste this to your agent and hand it the key:
set up https://monid.ai/SKILL.md
It learns the whole discover, inspect, run workflow itself. More in the agent quickstart.
For humans
npm install -g @monid-ai/cli
monid keys add -k <your-api-key> -l main
Step 1. Find which boards are covered before you pick boards
What it does. Turns "which boards should I watch" from a guess into a list constrained by what is actually reachable.
The call.
monid discover -q "job board aggregator search"
monid discover -q "linkedin job postings search"
What comes back. Running exactly this on 2026-08-30 turned up dedicated endpoints for Indeed, Wellfound, Built In, Idealist and web3.career, plus a LinkedIn job search actor. Every one is a separate endpoint with its own price and its own billing shape, which is the fragmentation in this article made concrete rather than argued.
This is also the check worth running before writing any scraper at all, and it is the subject of Browser Automation When the Site Has No API.
What it costs. Nothing. Discovery never bills.
Step 2. Pull one board and read the schema you actually get
What it does. Produces real records, which is the only way to judge whether a board's fields support your use.
The endpoints. builtin/search_jobs, billed per call.
The call.
monid run -p builtin -e /search_jobs -w --query '{"query": "data engineer"}'
What comes back. Running that on 2026-08-30 returned records carrying id, title, url, company, company_url, company_logo, work_type, location, salary, experience_level and posted.
Three of those are worth calling out. work_type came back as "Hybrid" and "In-Office", and experience_level as "Senior level": those are the board's own normalised taxonomies, not free text, and reconstructing them from a rendered page means writing and maintaining a classifier. salary came back null on both records, which is the honest state of salary data across this whole category and something to design around rather than assume.
What it costs. About a cent per call, billed per call rather than per posting, so a broad query costs the same as a narrow one. Current figures at monid.ai/tools.
Step 3. Reconcile before you store
What it does. Handles the part that is genuinely hard, which is not the collection.
The call. No endpoint. A decision about identity:
def posting_key(job):
return (
normalise(job["company"]),
normalise(job["title"]),
job.get("location", "").split(",")[0].strip().lower(),
)
What comes back. One row per real role instead of three. Note what is absent: the board's own id is useless as a cross-board key, and the URL is different everywhere. Company plus normalised title plus city is crude and works better than anything more clever, because the descriptions genuinely differ between boards.
What it costs. Nothing, and skipping it means every downstream count is inflated by however many boards you queried.
📖 See also Ship a LinkedIn Job-Alert Bot With One Metered API Call
Give this to your agent
Set up https://monid.ai/SKILL.md, and then use Monid to pull data engineer postings from built in, wellfound and linkedin, deduplicate on company plus title plus city, and give me one row per role with which boards it appeared on.Should you scrape Google Jobs instead?
It is a reasonable shortcut with a specific cost, and the cost is not the price.
What you gain
One query instead of several. Google Jobs aggregates postings from many boards and company career pages, including small sites you would never think to add to a list. For a first look at a market, or for coverage of a long tail you cannot enumerate, that breadth is genuinely hard to reproduce.
What you lose
The board's own fields. Google normalises everything into a common shape, which means the taxonomies that made a specific board useful, work type, seniority level, funding stage on startup boards, do not survive the trip. You get a posting; you do not get the board's opinion about it.
You also inherit an aggregation you cannot inspect. When a role is missing you cannot tell whether the board did not have it, Google did not index it, or it was deduplicated against something else. With per-board collection an absence is at least attributable.
The practical split
Use the aggregator for discovery, when the question is "what is out there" and you do not yet know which boards matter. Use per-board endpoints for monitoring, once you do, because that is where the fields and the attribution live.
Running both is also reasonable and cheaper than it sounds, since the aggregator query is one call.
Which endpoint should I use for which job?
| Endpoint | What it does | Input | Output | Best for | Billing |
|---|---|---|---|---|---|
builtin/search_jobs | Built In postings | A query | Title, company, work type, seniority, posted | Tech roles by city | Per call |
indeed/search_jobs | Indeed postings | A query | Listing records | The largest general board | Per call |
indeed/get_job_details | One posting in full | A posting id | Full description | Enriching a shortlist | Per call |
wellfound/search_jobs | Startup roles | Role and filters | Listing records | Startup hiring | Per call |
apify/harvestapi/linkedin-job-search | LinkedIn postings | Titles and filters | Listing records | LinkedIn coverage | Per result |
idealist/search_jobs | Nonprofit roles | A query | Listing records | A vertical the big boards cover badly | Per call |
Every row was verified with monid inspect on 2026-08-30. The table gives billing shape rather than figures; shape drives design and current numbers live on monid.ai/tools.
Note the split between per-call and per-result in that table, because it changes how you query. Per-call endpoints reward broad queries: one call returns whatever it returns, so narrowing the search saves nothing. Per-result endpoints reward narrow ones, because you pay per posting returned. Running the same query shape against both is how a job pipeline develops a bill nobody can explain.
When should you not collect postings yourself?
Four cases.
You need historical postings. Everything here collects what is live now. Questions about what a company was hiring for last year need an archive, and several vendors sell exactly that. No collection you start today reaches backwards.
You need complete coverage of a market. If the deliverable is "every posting in this vertical", the fragmentation above is a hard problem and buying a compiled dataset beats assembling one. That trade-off, dataset versus per-record call, is worked through in B2B Data Providers: Buy the Dataset or Call the API?.
The board has an official partner programme. Some boards license their inventory properly. If your volume justifies the conversation, sanctioned access with a contract is better than any collection route on every axis that matters.
You want the people rather than the postings. Hiring signals are one input to a prospecting motion and the enrichment side of it is a different set of endpoints, covered in People Data Labs, Apollo, ZoomInfo: Which Should You Actually Buy?.
Your question is about one company. If you want to know whether one employer is hiring, their careers page is authoritative, complete and free, and a job board is a lossy copy of it.
There is also a case for the libraries this article has been quiet about. If you want breadth across many boards immediately and you already operate infrastructure, JobSpy and its equivalents give you that in an afternoon and cost nothing but the running. What you take on is the blocking, the maintenance and the addresses, which is the same trade as everywhere else in this category, and the address half of it is priced out in Do You Still Need a Rotating Proxy in 2026?.
And the disclosure: you are reading Monid's blog and we sell per-call access to tools, so the case we argue best is the one where two or three boards matter and you would rather not operate a scraper. Where you need everything, or need history, buy a dataset.
Conclusion
There is no job scraping software because there is no single job feed, and the fragmentation is deliberate rather than a gap in the market. Boards compete on inventory they hold exclusively, employers post different records to each, and no aggregator is licensed to collect them all. So the ranked list of tools you were about to read is a list of answers to a question that has not been narrowed yet.
Narrow it first. Which two or three boards actually contain the roles you care about is a question with a short answer, and once you have it the tooling decision is mostly made. After that the real work is not collection, it is reconciliation: the same role arrives from three boards as three records with three titles, and the key you deduplicate on determines whether every number downstream is right or inflated.
Free next step: run monid discover -q "job board aggregator search" and see which boards are already covered. It costs nothing, and the list is usually shorter and more specific than the one you would have assembled from a roundup. Start at monid.ai.
FAQ
What is JobSpy and should you use it?
An open source Python library that scrapes several major job boards through one interface, and it is genuinely useful if you want breadth quickly. What it does not include is the part that makes scraping hard: it runs from your addresses, so blocking is yours, and it tracks board changes only as fast as its maintainers do. Use it when breadth matters more than reliability and you have somebody to own it; use per-board endpoints when a silent failure would matter.
Does Indeed have an official API?
Indeed has historically operated partner and publisher programmes rather than an open developer API, and access has narrowed over the years, which is why so much of this category exists at all. If your use case and volume fit a partner arrangement, that is the right route and worth asking about, because sanctioned access removes an entire class of risk. For everyone else the practical answer is a third-party endpoint, and the honest framing is that you are choosing it by elimination rather than preference.
How often should you re-collect job postings?
Daily is the usual instinct and it is usually too often for the value. Postings have a lifespan of weeks, so a daily sweep pays several times for the same record and the marginal freshness rarely changes a decision. Twice weekly catches almost everything that matters at a fraction of the cost, and the exception is if the posting date itself is your signal, in which case daily is defensible and should be scoped to a short keyword list.
How do you deduplicate the same job across boards?
Company plus normalised title plus city, and accept that it is crude. The board identifiers are unrelated to each other and the URLs are all different, so there is no shared key to join on. Descriptions differ enough between boards that text similarity produces false negatives, and titles differ enough that exact matching does too, which is why the crude key beats the clever one here. Keep the source board on each row so you can tell which slice a count came from.
Last updated August 2026.
