How do I read a page that blocks a plain fetch?

Question

Accepted Answer

a real browser session, and what it costs per step read <url> with tinyfish on monid and give me the items on it. it is a page where nothing is in the html until javascript runs, so tell me how many items came back with real values in them. do not judge it by the byte count or by the status: judge it by whether the fields are populated the comparison: on one page the $0.011 route and plain curl returned the same 5,806 bytes and a string equality check said true; on another the bytes differed by 367 and nothing had rendered either way what that means: the script that builds the page was still sitting there unexecuted, so an agent that strips tags and reads text gets nothing back the free one: returned all ten items as 1,080 bytes of clean markdown, which is the whole job, for $0.0000 do not diff the bytes: that 367-byte gap was a CDN performance beacon injected because the request looked like a browser, so on any fronted site the byte check reports a difference that means nothing the rule: count the populated fields. Both routes answered HTTP 200 on both pages, and on two different targets the paid one rendered 0 of 12 and 0 of 10

Browser Automation

Try it with your agents

How-to guide

Browser Automation

Try it with your agents

How-to guide

Related categories