FilterResponseData bad decoding from cached file

Add-ons Development
#1

There is a problem when trying to decode the stream filter data that is passed to the filter response data if the requested file is cached.

This problem has been detailed in this link: https://stackoverflow.com/questions/52128329/decode-string-from-typedarray-cached-data-passed-via-webrequest-filterresponseda

And to make it short, the text decoding of the cached data is resulting in this:

����������20180901034956%������������������������������������������������ ����~�������v�a�r

Not only the entire text is filled with nulls, but the data is prefixed with what I believe to be cache data information - that’s everything from the start of the string until just before the letter “v”.

Is this how the API is suppose to work for cached files data or is the decoding being done incorrectly? Is there a way to do this correctly and how can we identify whether or not the data is being received from a cached file or not?

#2

That sounds like a bug - generally webRequest doesn’t even act on cached data, since the modifications by a webRequest filter should already be cached.

#3

I disagree on the part about not acting on cached data seeing that the extension will not be able to modify the data until the file cache expires and downloads a new one again or this was not what you meant by what you said?

If it is a bug then I expect the fix to return the cached data the same way a non-cached file data is returned. I would appreciate if anyone provides a link for that bug if it is already filed or links it afterwards after filing it.

#4

@jorgev Can you or someone else take a look at this? Is this known, a bug or working as intended?

#5

This can’t possibly be intended.
Except for the prefix, the string looks like it is UTF-16 encoded (out some other two bye format, dunno why Firefox would change it).
You could decide the first few bye as binary string, build a RegExp to test for these prefixes and then decide how to treat the rest of the file.

#6

I have no idea about doing that what you suggested. Up until today when that question popped on my feed I thought it was how it should behave and gave up because I was not able to get it to work and the documentation didn’t say anything about it. I didn’t even noticed it was only happening to cached files. I am only now able to go back to using that API again just because that person explained how he kinda solved this problem. thing is, like him, I also don’t know if that’s the right thing to do and if it is a bug then it should be fixed, if it is not then the right way to handle this should be properly documented.

#7

I don’t really see any concrete questions in your last post. What exactly is is you don’t understand / want to know?

#8

In my case I only want to read the response data, I don’t want to modify any of it. Aside from the text decoding issue at hand, I don’t think it is possible to read the stream filter data without writing to it. At least in my tests, whenever I didn’t write to the filter, the page I was listening to would never load, it would stay blank.

Is it possible to listen to the response data without having to write to the filter?

#9

The API doesn’t provide a way to only read. You will have to write everything you read.