WebExtensions, fetch API & Tracking Protection

Hi,

it is not possible to make a fetch request to Reddit or other websites which are on the Tracking Protection list in Firefox, if Tracking Protection is enabled in Firefox.

See https://github.com/cadeyrn/bookmarks-organizer/issues/26 and https://bugzilla.mozilla.org/show_bug.cgi?id=1308640 for details.

It seems to be possible if I request a host permission for Reddit but this can’t be a solution for my add-on, I can’t add thousands of different host permissions to allow all domains on the TP list.

So what can I do to check if these bookmarks are broken?

1 Like

To get this straight: You have bookmarked URLs which load and are not blocked if they are opened in a tab (which means they are not “broken”) but can not be fetched or XHRd from the background process even if you set the *://*/* permission?


Towards the suggestion of iframes in the GitHub issue: that won’t work with some CSPs and/or frame settings.

Correct. As described on Bugzilla:

the domain must be explicitly listed in permissions, all_urls permission does not allow the extension to bypass TP.

That’s a very annoying behaviour for an add-on like my Bookmarks Organizer because bookmarks are marked as broken even if they are okay. So there is a high data loss risk…

OK. Just a quick idea:
You could probably work Erich response times here.
If the request is reflected due to the teaching list (or the hosts file or a local packet filter), the rejection should happen pretty much instantly.
Most other failures should take some time to happen and also shouldn’t be cached by the browser (I’m not sure if missing DNS entries are cached).

Also, if the affected pages load correctly in tab, can’t you just load the page in a background tab?

Does “load the page in a background tab” means that executing the bookmark check open 5000 tabs if there are 5000 bookmarks? If yes, then it’s no option. If no, can you please explain what you mean? Thanks.

Well, I would first do a fetch check and take it’s time. If that fails quickly, I’d see that as an Indicator for a blocked URL.
Only for those URLs, I would open a single or a couple of tabs and consecutively test all questionable URLs in those. Also, the page load can be canceled as soon as webNavigation.onCommitted occurs.
If you want to hide the loading tabs, you can open a popup window in the background. It is actually possible to open additional tabs in those, which can’t be accessed by the user (a bug, I guess).

I doubt there are (m)any users with 5000 blocked bookmarks.

It’s no option to open visible tabs, neither in the main window nor in a popup. My add-on checks for broken bookmarks, it’s not expected to open any tabs. If the add-on opens a popup window in the background there is still a popup windows, right? I need a completely invisible method. And I don’t want to depend on bugs. Also I don’t think that “if that fails quickly” is a reliable method to use for checks. :wink:

Thank you for your suggestions.

1 Like

While timeouts should be avoided whenever possible, they are still a commonly used tool. Especially with network applications, which yours is.

Anyway. The only other thing I can think of is using some kind of external proxy. You do run your own server. You could ask that server to do the check (spammers may be an issue), or you could probably also forward the links to google translate and see if that service is able to access the link (privacy is an issue here).

We’re running into this issue as well with our RSS reader. Did you find a good solution? A proxy is not a great alternative since it introduces an unnecessary step and won’t work for authenticated sites…

A user tries to add a Reddit feed, and the AJAX fails due to tracking blocking. The AJAX is done in the background page of the extension.

By now, you should be able to request host permissions at runtime:

https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/permissions/requesthttps://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/permissions/request

The origins property may include permissions that match a subset of the hosts matched by an optional permission: for example, if optional_permissions include “*://mozilla.org/”, then permissions.origins may include “https://developer.mozilla.org/”.

So you’ll need to include *://*/* (or <all_urls>) and hope that it isn’t excluded from the above documentation.

*://*.* is not a valid match pattern.

Indeed. Corrected it to *://*/*.