Can Firefox kill my background script if it's out of memory?

[juraj.masiar]

Hello,

For many weeks I’m trying to solve one strange bug that is being reported by several users every day, but I’m unable to reproduce it.

After releasing several new versions with improved error reporting I identified the issue as this:

• my add-on sends message to the background script (requesting few megabytes of images) and it will receive one of these two errors:
  "Could not establish connection. Receiving end does not exist."
  or
  "A request was aborted, for example through a call to IDBTransaction.abort."

These errors tends to happen to the users running 32-bit version of Firefox 58, this means they don’t have enough memory since Firefox is now upgrading to 64-bit automatically now if you have at least 4GB or RAM.

So my question is, can Firefox kill my background script (and cause the first error) or abort pulling images from IndexedDB, due to insufficient resources?

[rugkx (rugk)]

According to this thread:

How long are background scripts valid for?

I am looking to add a feature to my web extension that will give the user a notification after x amount of times using my add-on, asking for them to rate my add-on. When the user clicks the button and is directed to the extension’s AMO page, I want the notification to be forever hidden. I will obviously need to use my background.js file to implement this. I was just curious how long the background.js file will keep variables active for before they are reset? I’ll be storing a variable in the ba…

…Chrome may kill them on low-memory, but Firefox (currently!) does not do so.
So, to be good/future-proof, you should maybe still rather code it in a way that does not rely on having background pages always accessible.

Basically, I’d say storing multiple MB images in a background script is not really a good idea. Remember it stays in memory all the time (if it does, of course), so this can be a big waste of RAM.

[juraj.masiar]

It’s always a trade off - if I have them in memory, then I can access them fast. If I will be pulling it from IndexedDB, it may affect response time. But the truth is, I’ve never tested that because for most of the users the image data will have only few MB (like below 10MB).

But regardless, I’m still getting these error reports (from Firefox users) and I did suspect that once the browser gets out of memory, it will just start killing processes with the most used memory and my add-on would be probably first even if it’s just few MB.
But if there is no such thing, then I would say in the case of insufficient memory the internal call of malloc will fail and the process (thread) stops (every background script is running in own thread, right?).

[rugkx (rugk)]

Actually what you should test it whether keeping them in RAM is really any useful and really improves performance. I just doubt so, and I doubt performance really matters so much for a web extension.

Also 10MB is not much, but if every add-on in my FF would use it, it would use maybe 1GB or so… So with such add-ons then users scream “Firefox is eating all my RAM!”, well…, maybe you can consider to do it differently.

Also “for most of the users”… So these ones that don’t belong to “most of them” do not matter or what? Their RAM can be filled up to infinite and crash/freeze their Firefox, computer or whatever…

[juraj.masiar]

You are right, I should have tested it back than. And I will do it now.

But just to clarify - my add-on is a speed dial - the New tab replacement. And the default New tab page is extremely fast, so I wanted to have the thumbnails to be loaded as fast as possible.

Regarding most of the users - I just pulled statistics from my cloud server and 99.6% users is below 10MB (and most of them way below that) and from the others only one is above 32MB. So it’s not as bad as it sounds.

But anyway, I will do the tests and if there is only a few ms difference, I will turn off the cache. And now when I’m thinking about it, I think there will be a very very low difference…

Thank you for pointing it out. I will post the results soon

EDIT:
Tests suggests that querying data from IndexDB using:
store.index(GROUP_INDEX).getAll(GROUP_ID)
will add around 20ms delay on my PC with 4,4GHz CPU.
Now the question is - is it worth delaying every opening of New tab with only 20ms to save only few MB of memory?

[rugkx (rugk)]

Nobody will notice a load time of 20ms… But needs to be tested in practice.

Actually, I wonder how Firefox built-in tab page does it. You may try to choose a similar way/they likely have tested and explained, why/how they did it the way they do.

[juraj.masiar]

Actually it’s additional 20ms, not total time.

But I just realized - that time can be saved by using IndexedDB directly from the target JavaScript file - so I don’t need to transfer the result data from background script which saves me about 20ms!

So it seems like I have my answer now. I will fix this in my next release and there will be another memory friendly add-on out there

Thank you for your mentoring!

EDIT:
After releasing new version, I found out one important thing - the IndexedDB is not accessible from Private Browsing tab !!! The good news is that it can be still accessed through background script.
Another good news is that my add-on memory consumption went down from 30MB to 5MB on my PC.

EDIT 2:
One more thing - if you decide to use IndexedDB from other than the background script - make sure to include the onupgradeneeded handler!!! Because if you think that background script will always run before anything else, then think again.

[rugkx (rugk)]

Oh, yeah, sure. That seems great!