Background script / extension page during add-on upgrade

Add-ons Development
#1

Hello,

I’ve just faced quite difficult situation after I decided to use IndexedDB also from my extension page.

In my new version I did a database upgrade and updated onupgradeneeded handler - in my background script. However I didn’t include this handler in my extension page, because it won’t initialize database until it receives custom ready message from the background script which happens only after DBOpenRequest.onsuccess handler fires (which is after IndexedDB upgrade happens).

Now what happened is that for some users the database was not upgraded! This could only happened if extension page opened database with new version number which would lead to inconsistent state due to missing onupgradeneeded handler (I’ve upgraded database several times before and this is first time I faced this problem).

I want to know how is this possible.
I’ve just validated that even with 10 seconds delay in my background script - the extension page won’t touch database, so there is no way this is a race condition.

The only possibility is that the extension page was updated and opened BEFORE background script was updated or that the old background script was still running.

Can somebody explain how update process work behind the scenes?

I don’t have onUpdateAvailable listener so the update should not happen when Firefox starts.

I remember I’ve filed a bug related to update process some time ago: https://bugzilla.mozilla.org/show_bug.cgi?id=1436041

#2

I had similar issues with my extension - though I never actually waited for the background script to tell me that the DB is ready in extension pages. I opted to just make a common DB handler file that also included the upgrade and creation handlers: https://github.com/freaktechnik/justintv-stream-notifications/blob/master/src/database-manager.js

#3

The database-manager module is a very good idea. When I started this project I thought database is not accessible from extension pages (I thought it has same restrictions as content scripts) so I put it all in background script.

Anyway I just received two more error reports from the new version where I included onupgradeneeded handler in the extension page and I did increment version number on both places (and I tested it that it works if I delay background script and remove the waiting restriction).

Any ideas what can I do? I really need a help because I have no idea how to fix this :frowning:

This is my upgrade handler new code:

const store = e.target.transaction.objectStore(DB_IMAGE);
if(!store.indexNames.contains(GROUP_INDEX)) 
    store.createIndex(GROUP_INDEX, GROUP_INDEX, { unique: false });

And the errors I’m receiving are:
name: NotFoundError
message: The operation failed because the requested database object could not be found. For example, an object store did not exist but was being opened.

After executing this query using the new index:

db.transaction(DB_IMAGE, DB_OP_READONLY).objectStore(DB_IMAGE).index(GROUP_INDEX).getAll(groupId);
#4

I always have my extension pages grab a reference to the background page and then have the background page initialize the view.

The advantage of this is that you can forgo all messaging (with potential race conditions and whatnot) and can always use the same instance of your modules/objects across all contexts (which can considerably speed up page loads, at a cost of higher idle RAM usage).
The latter might also avoid your problem.

The only real disadvantage is that you can not access the background page from private windows or container tabs.

This script handles those edge-cases rather well, I think: https://github.com/NiklasGollenstede/web-ext-utils/blob/master/loader/_view.js

#5

I remember the getBackgroundPage() - first I was so happy, I spend several hours with refactoring and then I found out it doesn’t work in Private Browsing mode :smiley: . I was the one writing it to the MDN full of anger. My add-on is being used by many users in Private Browsing mode so I can’t really use it (but it would be amazing).

Anyway I have a feeling that what is happening here is again something unrelated to my code and related to some buggy profile of few users - I’ve received so far 10+ reports where the onupgradeneeded didn’t fired (even when defined in both scripts) which is a very low number compared to the total number of users. One of them even wrote me that his Firefox crashed when click a reload button that executes browser.runtime.reload() and that sounds like a pretty broken profile to me. I really hate this kind of issues…

EDIT:
Is there something I can do for the users with this issue except for reinstalling add-on? All that needs to be done is execute the onupgradeneeded code without changing the version number - can I do that?

EDIT 2:
There are now two guys that said that their Firefox crashed after executing the add-on reload browser.runtime.reload(). I thought add-ons these days cannot crash Firefox just like that (and I hope somebody is reading those crash reports :slight_smile: ).

#6

You could request the crash report IDs. This page shows them how to find that: https://support.mozilla.org/kb/firefox-crashes-asking-support

Then if you could post them here or in a new thread, maybe someone will be able to spot a bug.

#7

Great idea! Thank you for that.

However I just received e-mails from them and there are no crash reports from that day because it wasn’t a crash - because if you have only one tab opened and it’s the tab with my add-on page then reloading add-on means closing all add-on pages - so Firefox was just closed.

EDIT:
I have some new info:

  1. users that upgraded to 9.8 version and had this issue were fixed by 9.9 release - so re-running onupgradeneeded helps!
  2. some users that upgraded directly to 9.9 release (because I disabled 9.8 version after the reports) were affected by this issue
  3. total amount of users reported this error (about 25) is so far only 0.05% of all users - and since this is related to the extremely buggy IndexedDB API, I think this may be actually a big success - even if I receive 25 more till tomorrow (when most of the users should be upgraded).

Now since re-running onupgradeneeded helps, I have a crazy idea about adding a piece of code that would just try to increment the DB version when it receives this error and store it in local storage - but it feels extremely hacky especially if users starts to restore old backups of settings with lower number (and who knows what other problems could rise…).

#8

I’m able to reproduce this bug by executing multiple web-ext run commands - this will open multiple separate Firefox instances - some of them even without add-on installed but some of them with exactly this issue - with missing database objects!

And in the browser console there is this line: 1532592867995 addons.xpi WARN Addon with ID admin@fastaddons.com_GroupSpeedDial already installed, older version will be disabled.

Now unless this is related to web-ext tool, I’m pretty sure this is a IndexedDB bug.
Reported now with video demonstration and example add-on:

1 Like
#9

Could you run web-ext with debugging output, I have a feeling this is https://github.com/mozilla/web-ext/issues/1137 which will lead to invalid DBs since the extension gets no chance to properly init the DB before it is unloaded again.

#12

After executing several instances with --verbose I ended up with two console windows running with one Firefox window with error. These are two console outputs:
https://pastebin.com/akPndPE0
https://pastebin.com/4sEH536E

But if this can happen with web-ext, I think it could still happen during add-on upgrade as well.

#13

From those logs it seems like it is indeed an issue with the random Firefox profiles, yes.