webRequest Blocking Response and Cooperative Multitasking

Add-ons Development
#1

Hello!

I make an addon that scans inbound images to see if they’re NSFW and then replaces them with SVG placeholders if they are. It uses neural nets so the calculations are necessarily quite expensive. Given the nature of the use case, I must use BlockingResponse as well as fully wait for the resource to load before I can start scanning.

Based on some limited tests I’ve done, it seems that the inbound requests never stack quite like I thought they might - that is to say, even though I’m using await and async for good chunks of it, my inbound queue of work never seems to grow over size 1. This surprised me a bit and got me wondering if webRequests go in parallel at all when used in this BlockingResponse mode. (“Blocking” in this sense could be interpreted in a couple different ways.) Does anybody know what the defined behavior here is?

Additionally, assuming that I’m not just doing something incorrect here, it seems that the only way to solve this would be to essentially have more than one background.js context doing some form of cooperative multitasking, which is I believe only possible with more than one addon. I’ve done some initial tests that lead me to think this would give a significant boost potentially. Is there any way to have more than one background.js context in an addon?

Thanks for your time!

1 Like
#2

What specific event handler are you using?
So you are saying that if you block the first response for 1 minute then all requests are blocked and waits in the queue? That would be bad.

Note that JavaScript is kind of “single thread”, in Firefox I believe all addons runs in a same thread. Also usage of async / await doesn’t automatically mean parallel, usually it just means do it later in this thread - when you can.
The true parallel is when you create a new Worker and send it some work.

I don’t think you can have more background scripts, also I don’t think you can access this API from the worker.
But you may be able to register multiple handlers, but I’m not sure if that will help, especially if the requests are blocked and waits for each other.

#3

I’m using browser.webRequest.onHeadersReceived.addListener. I think if I register multiple listeners here they will “chain”.

However, I did some further log checking and the stacking is occurring a bit differently than I expected: multiple requests are coming through but something else appears to be the bottleneck. So while the initial symptom I listed is still true, the cause is not webRequest at all, which is good. That means I can solve the problem in different ways. I still don’t know the exact meaning of “blocking” in BlockingResponse, but at least it is does not seem to be the limiter I thought it might.

#4

Update for anyone who might be curious about such things. I decided to do my processing in extension pages that have their own process; I register my extension page “processors” with background.js and post messages back and forth with a port. So far my prototype seems to be working pretty well, and has the advantage that it could be easy to spin up more processors basically at will.