SendResponse from background script not being received by content script

[odebroqueville2 (Olivier)]

I’m sending the message ‘contentScriptLoaded’ from a content script once a web page has finished loading.

The background script then checks if it needs to send a response

1. for an AI prompt (action: ‘askPrompt’)
2. for an HTTP POST request (action: displaySearchResults’). In this case, the response needs to be sent after the form data has been submitted and the result obtained.

The problem I’m encountering is that the content script is receiving a response: true. It doesn’t seem to receive the response ‘displaySearchResults’ from the background script or at least not in a timely manner. The data resulting from submitForm is correctly logged in the console for the background script.

I have spent a full day trying to solve this problem, but I can’t find the solution.

Here is the console.log from the content script:

The code extracts can be seen here:

CS BUG Code Only.pdf (47,1 Ko)

[juraj.masiar]

Looking at this single line already tells me, you are doing something wrong:

browser.runtime.onMessage.addListener(async (message, sender, sendResponse) => {

Make sure to read the docs, it has some super useful tips!

runtime.onMessage - Mozilla | MDN

One “potential” issue is the “async” - which will cause the Function to always return a Promise (even if you don’t return anything, it will still be a Promise<undefined>).

This brings us to the “feature” that causes this behavior - instead of using “sendResponse”, return a Promise with a value, and this value will be send back to the caller.

So what to do:

1. remove “sendResponse”
2. remove “async”
3. remove “return true” at the end of the handler, it won’t be needed.
4. refactor your code into something like this:

browser.runtime.onMessage.addListener((message, sender) => {
  switch(action) {
    case 'openModal': return myOpenModalAsyncFunction(sender);
    case 'otherModal': return otherAsyncHandler();
    case 'not_expeting_reply?': someJob(); return; // returning undefined, not a promise! So sender won't receive a reply.
    case 'need_reply_but_not_async?': return Promise.resolve(9);  // sends back number 9
  }
});

async function myOpenModalAsyncFunction(sender) {
  // ...
}

This way, your handler won’t consume ALL messages, only those that needs consuming.
And you will still be able to use async/away syntax.

[odebroqueville2 (Olivier)]

Hi Juraj, thank you for your quick response. I used async in the listener because one of the cases included an await:

const result = await submitForm(finalFormData);

I don’t quite understand what I should do with this line that would be required before I return the Promise, as you’ve explained.

[juraj.masiar]

Ideally, each “case” block, should be a function call.
So what you should do, is extract the lines for each case into own function, and call it in the case blocks, just like I showed before.

These functions can be made async, so your existing code that uses await will still work.

And if you want the sender to receive a response, you simply return the value that the async function call returns - for example:
case 'otherModal': return otherAsyncHandler();
This will return to the sender whatever the function returns (as long as the function is async).

Note that encapsulating logic into functions is in general a good thing to do, instead of having one huge switch.

[odebroqueville2 (Olivier)]

Sorry for being a bit of a novice developer, but can you please give me an example to understand what you mean by:

Note that encapsulating logic into functions is in general a good thing to do, instead of having one huge switch .

Do you mean, for example, that I should have:

function someFunction(action, data) {
...switch (action)/case code here...
}

UPDATE: Cool! I got the code working thanks to your suggestions. I’m still getting one odd error though:

Error: Promised response from onMessage listener went out of scope

Simeon was right! Coding is hard! There are a lot of small details to master, until your code starts looking well structured.

[juraj.masiar]

By encapsulating I mean taking the code that runs in your case and putting it in own function.
For example, instead of this:

switch(action) {
  case 'a': 
    console.log('hello');
    sendResponse('world');
    return true;
}

You do this:

switch(action) {
  case 'a': return runConsoleAndGetWorld();
}

async function runConsoleAndGetWorld() {
  console.log('hello');
  return 'world';
}

The error you pasted sounds like you are still using “sendResponse”.

[dotproto (Simeon Vincent)]

Offhand I don’t know all of the patterns that will cause “Error: Promised response from onMessage listener went out of scope” to get logged, but I do know of one concrete example.

1. Load this extension in Firefox
2. Visit https://example.com.
3. Open devtools on example.com and look for the “Error: Promised response from onMessage listener went out of scope” in the console. It should appear in less than 30 seconds.

manifest.json

{
  "name": "Listener went out of scope",
  "version": "1.0",
  "manifest_version": 3,
  "background": {
    "scripts": ["background.js"]
  },
  "content_scripts": [{
    "matches": ["*://*.example.com/*"],
    "js": ["content.js"]
  }]
}

background.js

// Note that `sendResponse` is defined but not used
browser.runtime.onMessage.addListener((message, sender, sendResponse) => {
  console.log("message", message);
  return true;
});

content.js

(async function() {
  const res = await browser.runtime.sendMessage("ping");
  console.log("response", res);
})();

To fix this we just need to make sure that the onMessage listener returns a response by calling sendResponse(). Calling sendResponse is not optional – even the response won’t be used, you still need to call sendResponse() (even with no arguments) to tell the browser that the message handler completed it’s work.

background.js

browser.runtime.onMessage.addListener((message, sender, sendResponse) => {
  console.log("message", message);
  sendResponse("pong");
  return true;
});

[odebroqueville2 (Olivier)]

I updated my background script (please see code below) to include a sendResponse, but after a while I’m still getting (from the content script console logs):

Error: Promised response from onMessage listener went out of scope

    browser.runtime.onMessage.addListener((message, sender, sendResponse) => {
        const action = message.action;
        const data = message.data;
        if (logToConsole) console.log(`Message received: action=${action}, data=${JSON.stringify(data)}`);
        switch (action) {
            case 'openModal':
                handleOpenModal(data);
                break;
    …
            case 'hidePageAction':
                browser.pageAction.hide(sender.tab.id);
                break;
            case 'showPageAction':
                browser.pageAction.show(sender.tab.id);
                break;
            case 'contentScriptLoaded':
                return handleContentScriptLoaded(data);
            default:
                console.error('Unexpected action:', action);
                return false;
        }
        sendResponse({});
        return true;
    });

    async function handleContentScriptLoaded(data) {
        if (logToConsole) console.log('Content script loaded. Sending response.');
        // Send a response to the content script
        const { domain, tabUrl } = data;
        if (logToConsole) console.log(`Tab url: ${tabUrl}`);
        if (logToConsole) console.log(`Prompt: ${promptText}`);

        if (aiUrls.includes('https://' + domain)) {
            return Promise.resolve({
                action: "askPrompt",
                data: { url: tabUrl, prompt: promptText }
            });
        }

        // Check if tabUrl is in the list of search engine URLs 
        for (let id in searchEngines) {
            if (id.startsWith('separator-') || id.startsWith('link-') || id.startsWith('chatgpt-') || searchEngines[id].isFolder) continue;
            const searchEngine = searchEngines[id];
            if (targetUrl.startsWith(tabUrl) && searchEngine.formData) {
                let finalFormData;
                let formDataString = searchEngine.formData;
                if (formDataString.includes('{searchTerms}')) {
                    formDataString = formDataString.replace('{searchTerms}', selection);
                } else if (formDataString.includes('%s')) {
                    formDataString = formDataString.replace('%s', selection);
                }
                const jsonFormData = JSON.parse(formDataString);
                finalFormData = jsonToFormData(jsonFormData);

                if (logToConsole) {
                    console.log('Form data string:');
                    console.log(formDataString);
                    console.log(`Selection: ${selection}`);
                    console.log(`id: ${id}`);
                }
                return await submitForm(finalFormData);
            }
        }
        return false;
    }

[odebroqueville2 (Olivier)]

I have a small question re the code: given that myOpenModalAsyncFunction is an async function, how come you don’t have to write:

case 'openModal': return await myOpenModalAsyncFunction(sender);

[juraj.masiar]

Your code looks much better now.
But you still didn’t removed sendResponse, you don’t needed it, and using it causes a bad design patterns (spaghetti code).
Also returning “true” is not needed (you need to return “true” only when using sendReponse).
So simply remove these two:

        sendResponse({});
        return true;

Regarding the question:

case 'openModal': return await myOpenModalAsyncFunction(sender);

Don’t use await.
As mentioned before, when you call async function, it will return a Promise.
Using await tells the code to wait for the Promise to resolve. But you don’t need that.

The handler will send the caller a response, only if you return a Promise (which is what your async function always returns).

That’s why you use:

case 'openModal': return myOpenModalAsyncFunction(sender);

Let’s brake it down, just in case, so that you see what I mean:

case 'openModal': 
  const reponse = myOpenModalAsyncFunction(sender);  // reponse is a Promise
  return reponse;

Now, about your handleContentScriptLoaded function, since it’s an async function, it always returns a Promise. So inside it, you don’t need to use:

            return Promise.resolve({
                action: "askPrompt",
                data: { url: tabUrl, prompt: promptText }
            });
// instead:
            return {
                action: "askPrompt",
                data: { url: tabUrl, prompt: promptText }
            };

You can simply return the object you need, because it will be wrapped in a Promise automatically. But even if you wrap it in a Promise.resolve, it will still work the same.

It’s best to study Promises before using them, it’s not that hard, and there is a lot of materials about it on the MDN:

Using promises - JavaScript | MDN

[stig (Stig Nygaard)]

I don’t want to interrupt or confuse your discussion too much, and I haven’t studied the posted code here in details…

BUT if you are making a cross-browser webextension, isn’t it necessary to use the sendResponse() method? That was the conclusion I came to when making one of my extensions cross-browser compatible.
Also according to https://developer.mozilla.org/docs/Mozilla/Add-ons/WebExtensions/API/runtime/onMessage , “response with promise” is not supported by Chrome/Edge.

I have however also always found messaging between extension scripts rather confusing, and don’t want to state anything as a fact. So I hope I don’t just ad further confusion to the subject?..

[odebroqueville2 (Olivier)]

I followed your instructions and removed Promise.resolve() both in handleContentScriptLoaded and submitForm functions. So, now I’m just returning an object or false. I also removed (which I had added after reading Simeon’s response):

        sendResponse({});
        return true;

Yet, whether the above 2 lines are removed or not, I keep getting the same error appearing after a while:

Error: Promised response from onMessage listener went out of scope

[juraj.masiar]

Are you sure you are not calling sendResponse anymore?
Also, can you share a bit of code that sends the message? I suppose it’s in the content script?

[juraj.masiar]

@stig

BUT if you are making a cross-browser webextension, isn’t it necessary to use the sendResponse() method?

You are (almost) completely right! Chrome doesn’t support it.
https://issues.chromium.org/issues/40753031

However, this can be fixed with the webextension-polyfill, which perfectly polyfills this behvaior in Chrome.

You should almost always go for the readability when writing code. And using callbacks (in general) is a legacy pattern, now replaced with Promises (in most cases…).

[odebroqueville2 (Olivier)]

Yes, there are no sendResponse remaining in my code. Here is the background script:

browser.runtime.onMessage.addListener((message, sender, sendResponse) => {
    const action = message.action;
    const data = message.data;
    if (logToConsole) console.log(`Message received: action=${action}, data=${JSON.stringify(data)}`);
    switch (action) {
        case 'openModal':
            handleOpenModal(data);
            break;
…
        case 'hidePageAction':
            browser.pageAction.hide(sender.tab.id);
            return Promise.resolve({});
        case 'showPageAction':
            browser.pageAction.show(sender.tab.id);
            return Promise.resolve({});
        case 'contentScriptLoaded':
            return handleContentScriptLoaded(data);
        default:
            console.error('Unexpected action:', action);
            return false;
    }
});


async function handleContentScriptLoaded(data) {
    if (logToConsole) console.log('Content script loaded. Sending response.');
    // Send a response to the content script
    const { domain, tabUrl } = data;
    if (logToConsole) console.log(`Tab url: ${tabUrl}`);
    if (logToConsole) console.log(`Prompt: ${promptText}`);

    if (aiUrls.includes('https://' + domain)) {
        return {
            action: "askPrompt",
            data: { url: tabUrl, prompt: promptText }
        };
    }

    // Check if tabUrl is in the list of search engine URLs 
    for (let id in searchEngines) {
        if (id.startsWith('separator-') || id.startsWith('link-') || id.startsWith('chatgpt-') || searchEngines[id].isFolder) continue;
        const searchEngine = searchEngines[id];
        if (targetUrl.startsWith(tabUrl) && searchEngine.formData) {
            let finalFormData;
            let formDataString = searchEngine.formData;
            if (formDataString.includes('{searchTerms}')) {
                formDataString = formDataString.replace('{searchTerms}', selection);
            } else if (formDataString.includes('%s')) {
                formDataString = formDataString.replace('%s', selection);
            }
            const jsonFormData = JSON.parse(formDataString);
            finalFormData = jsonToFormData(jsonFormData);

            if (logToConsole) {
                console.log('Form data string:');
                console.log(formDataString);
                console.log(`Selection: ${selection}`);
                console.log(`id: ${id}`);
            }
            return submitForm(finalFormData);
        }
    }
    return false;
}

async function submitForm(finalFormData) {
    let data = '';
    const controller = new AbortController();
    const timeoutId = setTimeout(() => controller.abort(), 10000); // Timeout set to 10 seconds

    try {
        const response = await fetch(targetUrl, {
            method: 'POST',
            body: finalFormData,
            signal: controller.signal // Signal for aborting the fetch on timeout
        });

        clearTimeout(timeoutId); // Clear timeout once response is received

        // Check if the response is successful (status code in the 200–299 range)
        if (!response.ok) {
            throw new Error(`Error: ${response.status} ${response.statusText}`);
        }

        data = await response.text();
        if (logToConsole) console.log('Data:', data);
        if (data) {
            return {
                action: "displaySearchResults",
                data: data
            };
        } else {
            return false;
        }

    } catch (error) {
        if (error.name === 'AbortError') {
            console.error('Request timed out');
        } else {
            console.error('Fetch error:', error);
        }
        throw error; // Re-throw the error to ensure the calling code handles it
    }
}

And here is the part of the content script that is sending the message:

    // If the web page is for an AI search, then send a message to the background script and wait for a response
    if (logToConsole && aiUrls.includes('https://' + domain)) console.log(`AI search engine detected: ${domain}`);
    const response = await sendMessage('contentScriptLoaded', { domain, tabUrl });
    if (response.action === 'askPrompt') {
        try {
            const { url, prompt } = response.data;
            await ask(url, prompt);
        } catch (err) {
            if (logToConsole) console.log(err);
            await sendMessage('notify', notifySearchEngineNotFound);
        }
    } else if (response.action === 'displaySearchResults') {
        try {
            const results = response.data;
            const html = document.getElementsByTagName('html')[0];
            const parser = new DOMParser();
            const doc = parser.parseFromString(results, 'text/html');
            if (logToConsole) console.log(results);
            if (logToConsole) console.log(doc.head);
            if (logToConsole) console.log(doc.body);
            html.removeChild(document.head);
            html.removeChild(document.body);
            html.appendChild(doc.head);
            html.appendChild(doc.body);
        } catch (err) {
            if (logToConsole) console.log(err);
            await sendMessage('notify', notifySearchEngineNotFound);
        }
    } else {
        if (logToConsole) console.error("Received undefined response or unexpected action from background script.");
        if (logToConsole) console.log(`Response: ${response}`);
    }

[juraj.masiar]

The code looks more less fine, just a few more points:

Using return Promise.resolve({});, will reply to the caller with empty object, but why?
It may be better to return nothing (if the code that send that message doesn’t expect any response), or return the promise that the async function returns, for example, instead of this:

        case 'hidePageAction':
            browser.pageAction.hide(sender.tab.id);
            return Promise.resolve({});

Simply use:

case 'hidePageAction': return browser.pageAction.hide(sender.tab.id);

This has one other benefit - if the function fails (throws an exception), it will be propagated to the original code that send the message, so you’ll see what failed and why.

Also, your default case returns false, it should return undefined, or nothing, or just use break;.

Now, regarding the error about “went out of scope”, could you trace where exactly it originates? What code is executed right before it shows? And does it break anything?

[odebroqueville2 (Olivier)]

I returned a Promise because the documentation doesn’t say that browser.pageAction.hide returns a Promise or is async. So I wasn’t sure what to do!

Re “went out of scope”, it’s a bit strange because it appears some 20-30 seconds after the content script of newly created tab has received the response data (corresponding to the results of an HTTP POST request) from the background script. Also, the line indicated from which the error would appear to originate from is line 1 of the background script!

Here is the console.log:

When I click on background.js:1…, this is what I get:

which I guess corresponds to the background.js code minimised! Which I can’t make anything of! I don’t know how to debug beyond console logging!

Luckily the error doesn’t seem to break anything.

[juraj.masiar]

This is actually the reason why I always rename the “background.js” script into project specific name, like “background_ah.js” when I work on my AutoHiglight.
Because Firefox logs messages also from other extensions, and since everyone uses “background.js”, you can’t tell whether the error comes from your script or not.

[odebroqueville2 (Olivier)]

Thank you so much for your help, Juraj. I changed the name of my background script to background-cs.js and the error kept coming from background.js. I then disabled all extensions and re-enabled them one by one. Turns out that Mozilla’s extension Orbit seems to be responsible for the trouble. The thing is that Orbit places, well, an orbit on each new web page that loads. My content script removes the head and body elements of the html document and appends the result of the HTT POST to the html element. By doing so, I guess I’m removing something used by Orbit. I may not be doing things in the best of ways, because the reason I’m loading the domain corresponding to the website from which I’m doing an HTTP POST request is to preserve the same styling that’s applied throughout that website.

Anyway, you’ve been a great help to me, so thanks again. Out of curiosity, which one of your extensions is paid? If I have any use for it I may gladly adopt it. Be well.

https://addons.mozilla.org/en-US/firefox/addon/orbit-summarizer/

[juraj.masiar]

No need to repay me in any way Olivier, I’m happy I could help.
Plus I always learn something new this way, (and the fake internet points counts too ).

But if want to try some of my extensions, I can highly recommend Scroll Anywhere, Search Result Previews and Auto Highlight. These I would recomment to anyone, some of my others are meant only for freaks like me .