Hi, everyone
I’m working on a firefox extension that has a native part. But I keep getting the same error: “Native application tried to send a message of 1684611707 bytes, which exceeds the limit of 1048576 bytes.”
The native part is a C program and right now I only use to send a small json string. I’ve googled this for a long time without luck. So anyone knows what can be the cause of this kind of error??
Thanks a lot!
This sounds like an encoding issue with the message, if the string is below the number of bytes Firefox seems to think are in the message.
I try to send message like this
fprintf(stdout, “{\“id\”: 23453}”)
which is a valid json string, right??
Oh, and I’m on Windows7, is there anything about the encoding I should care on windows platform?
As https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Native_messaging outlines, the first 32 bits of your message must be a unit with the length of the string you are sending in bytes (same as the incoming message format).
I obtain the same error, native program is in NodeJS language.
according to edge inspector, the stdout is perfectly similar to stdin message, which is received properly from the native app (different strings).
this is the “inspector” debug output:
data from extension to native app:
[39, 0, 0, 0, 34, 87, 72, 89, 32, 73, 84, 32, 68, 79, 69, 83, 78, 39, 84, 32, 87, 79, 82, 75, 32, 65, 83, 32, 69, 88, 80, 69, 67, 84, 69, 68, 32, 63, 63, 63, 63, 63, 34]
data sent from the native app to extension:
[39, 0, 0, 0, 34, 73, 32, 65, 77, 32, 83, 84, 82, 85, 71, 71, 76, 73, 78, 71, 32, 116, 111, 32, 115, 111, 108, 118, 101, 32, 116, 104, 105, 115, 32, 112, 114, 111, 98, 108, 101, 109, 34]
extension error:
Disconnected due to an error: Native application tried to send a message of 1394626127 bytes, which exceeds the limit of 1048576 bytes.
NodeJS function to send message from native app to extension:
let test = () => {
let jsonString = JSON.stringify("I AM STRUGGLING to solve this problem");
let jsonBuffer = Buffer.from(jsonString);
console.log("JSON BUFFER", jsonBuffer);
let lengthBuffer = Buffer.alloc(4);
lengthBuffer.writeUInt32LE(jsonBuffer.length);
console.log("LENGTH BUFFER", lengthBuffer);
let data = Buffer.concat([lengthBuffer, jsonBuffer]);
console.log("SENDING DATA", data);
process.stdout.write(data);
};
debug output (from NodeJS):
JSON BUFFER Buffer(39) [34, 73, 32, 65, 77, 32, 83, 84, 82, 85, 71, 71, 76, 73, 78, 71, 32, 116, 111, 32, 115, 111, 108, 118, 101, 32, 116, 104, 105, 115, 32, 112, 114, 111, 98, 108, 101, 109, 34]
LENGTH BUFFER Buffer(4) [39, 0, 0, 0]
SENDING DATA Buffer(43) [39, 0, 0, 0, 34, 73, 32, 65, 77, 32, 83, 84, 82, 85, 71, 71, 76, 73, 78, 71, 32, 116, 111, 32, 115, 111, 108, 118, 101, 32, 116, 104, 105, 115, 32, 112, 114, 111, 98, 108, 101, 109, 34]
I have also tried to reverse byte order, but another error arose: a “zero” value to stdout…
Any advise ?
The example completely ignores the endianness of the OS, which you should detect using os.endianness https://nodejs.org/api/os.html#os_os_endianness (for both reading and writing), since the messages uses “native byte order”.
However, given that your incoming message seems to have the same endianness, this is probably not the issue.
How are you running your native application? With a wrapper script? Maybe that’s outputting some data that’s interfering with the stdout stream from node? Or are there any console.logs or errors?
thank you very much for your attention.
I have checked with node REPL: endiannes is LE (windows 10, intel CPU).
How can I get the std in/out stream ?
I know only the console.log function or Edge Dev Tools log.
the entire log of the browser is:
BACKGROUND SENDING: WHY IT DOESN'T WORK AS EXPECTED ????? [background.js:25:11](moz-extension://0cf8c74a-2275-4297-bfee-5ae6e558488f/background.js)
Disconnected due to an error: Error: Native application tried to send a message of 1394626127 bytes, which exceeds the limit of 1048576 bytes.
the entire log of MS Edge DevTools console is:
ON STDIN
VM103 nativeMessaging.js:50 ON STDIN
VM103 nativeMessaging.js:62 Reached end of stream.
VM103 nativeMessaging.js:17 PROCESS DATA
VM103 nativeMessaging.js:20 STRING DATA Buffer(43) [39, 0, 0, 0, 34, 87, 72, 89, 32, 73, 84, 32, 68, 79, 69, 83, 78, 39, 84, 32, 87, 79, 82, 75, 32, 65, 83, 32, 69, 88, 80, 69, 67, 84, 69, 68, 32, 63, 63, 63, 63, 63, 34]
VM103 nativeMessaging.js:27 PAYLOAD SIZE 39
VM103 nativeMessaging.js:34 INSIDE IF
VM103 nativeMessaging.js:39 flushChunksQueue
VM103 nativeMessaging.js:43 JSON PARSE WHY IT DOESN'T WORK AS EXPECTED ?????
VM103 nativeMessaging.js:77 JSON BUFFER Buffer(39) [34, 73, 32, 65, 77, 32, 83, 84, 82, 85, 71, 71, 76, 73, 78, 71, 32, 116, 111, 32, 115, 111, 108, 118, 101, 32, 116, 104, 105, 115, 32, 112, 114, 111, 98, 108, 101, 109, 34]
VM103 nativeMessaging.js:81 LENGTH BUFFER Buffer(4) [39, 0, 0, 0]
VM103 nativeMessaging.js:84 SENDING DATA Buffer(43) [39, 0, 0, 0, 34, 73, 32, 65, 77, 32, 83, 84, 82, 85, 71, 71, 76, 73, 78, 71, 32, 116, 111, 32, 115, 111, 108, 118, 101, 32, 116, 104, 105, 115, 32, 112, 114, 111, 98, 108, 101, 109, 34]
VM103 nativeMessaging.js:71 About to exit with code: 0
node script nativeMessaging.js
let payloadSize = null;
// A queue to store the chunks as we read them from stdin.
// This queue can be flushed when `payloadSize` data has been read
let chunks = [];
// Only read the size once for each payload
let sizeHasBeenRead = () => Boolean(payloadSize);
// All the data has been read, reset everything for the next message
let flushChunksQueue = () => {
payloadSize = null;
chunks.splice(0);
};
let processData = () => {
console.log("PROCESS DATA");
// Create one big buffer with all all the chunks
const stringData = Buffer.concat(chunks);
console.log("STRING DATA", stringData);
// The browser will emit the size as a header of the payload,
// if it hasn't been read yet, do it.
// The next time we'll need to read the payload size is when all of the data
// of the current payload has been read (ie. data.length >= payloadSize + 4)
if (!sizeHasBeenRead()) {
payloadSize = stringData.readUInt32LE(0);
console.log("PAYLOAD SIZE", payloadSize);
}
// If the data we have read so far is >= to the size advertised in the header,
// it means we have all of the data sent.
// We add 4 here because that's the size of the bytes that old the payloadSize
if (stringData.length >= (payloadSize + 4)) {
console.log("INSIDE IF");
// Remove the header
const contentWithoutSize = stringData.slice(4, (payloadSize + 4));
// Reset the read size and the queued chunks
console.log("flushChunksQueue");
flushChunksQueue();
const json = JSON.parse(contentWithoutSize);
console.log("JSON PARSE", json);
// Do something with the data...
test(json);
}
};
process.stdin.on('readable', () => {
console.log("ON STDIN");
// A temporary variable holding the nodejs.Buffer of each
// chunk of data read off stdin
let chunk = null;
// Read all of the available data
while ((chunk = process.stdin.read()) !== null) {
chunks.push(chunk);
}
});
process.stdin.on('end', () => {
console.log('Reached end of stream.');
processData();
});
process.stdout.on('end', () => {
console.log('STDOUT END');
});
process.on('exit', (code) => {
console.log(`About to exit with code: ${code}`);
});
let test = (json) => {
let jsonString = JSON.stringify("I AM STRUGGLING to solve this problem");
let jsonBuffer = Buffer.from(jsonString);
console.log("JSON BUFFER", jsonBuffer);
let lengthBuffer = Buffer.alloc(4);
lengthBuffer.writeUInt32LE(jsonBuffer.length);
console.log("LENGTH BUFFER", lengthBuffer);
let data = Buffer.concat([lengthBuffer, jsonBuffer]);
console.log("SENDING DATA", data);
process.stdout.write(data);
};
batch file that launches node script: (if I remove echo of, the native messaging port is disconnected at the startup)
@echo off
node --inspect "D:\\nativeMessaging\\node_server\\nativeMessaging.js"
background script
const port = browser.runtime.connectNative("ping_pong");
/*
Listen for messages from the app.
*/
port.onMessage.addListener((response) => {
// let view = new DataView(response, 0);
console.log("Received: " + response);
});
port.onDisconnect.addListener((p) => {
if (p.error) {
console.log('Disconnected due to an error:', p.error);
}
});
/*
On a click on the browser action, send the app a message.
*/
let data = "WHY IT DOESN'T WORK AS EXPECTED ?????";
browser.browserAction.onClicked.addListener(() => {
console.log("BACKGROUND SENDING: " + data);
port.postMessage(data);
});console.log will go to the same stdout as Firefox wants to read, since there is only one. Anything a node script logs as output is stdout (except for errors that are in stderr). So you likely want to remove your console.logs.
thank you very much
I have removed all the console.log calls, and moved processData() right after the “while(chunk…)” cycle, and now background script receives correctly native messages !