Opening and scraping invisible tabs

I have a beta add-on that opens Quora tabs and scrapes them like so:

It them immediately closes them

Is there a way to open tabs or windows using WebExtensions API so that the user doesn’t see them but yet I can still run content scripts inside them to scrape?

You can hide tabs. But I think that may still trigger a flash of the tab appearing, since I’m not sure you can immediately hide a tab after creating it.

Alternatively you can of course use fetch() to load a page if you don’t have to rely on it running some JS. And lastly, you can embed it in an iframe (in your popup, bg page or the current tab), though properly sandboxing that is a challenge.

However if you can’t use fetch() because it loads in something via JS, I’d suggest reverse engineering to the point where you use the endpoint the page uses to get the data. That’s just as volatile as DOM scraping.

2 Likes

Thanks @freaktechnik I particularly like using the endpoint, will look into that :grin: