Beginner's questions about embedding SpiderMonkey

Hello, first time poster here. I’m looking at SpiderMoney for embedding in an audio application, and I have some things I’d like to clarify before I get started. I’ve been using a much smaller JavaScript parser JUCE::JavascriptEngine, but it is quite limited and so I need to replace it. I’ve been doing a lot of research over the past couple of weeks (V8, ChakraCore, etc.), and SpiderMonkey has the API that makes most sense to me and seems to be generally well suited to my project. I’m fairly competent at C++ and the JUCE library, but I’ve never worked with a package as heavy as SpiderMonkey before, so it’s quite possible that I’ll be making some basic mistakes. Please correct me if I do.

I will first give some context about my program. I’m building an audio application which lets the user sequence events by writing code. For instance, the user might write

play(50);
wait(1);
start(next());

which would play Midi note 50, wait for one second, and then start the next bit of code. Rather than write my own DSL, I’m using a Javascript for the parsing. (From the example I’ve given here this seem that this is not a good decision, but for reasons I won’t go into here, Javascript is a actually great fit). The user will also be able to write their own functions, store variables, etc, just like in regular code. The situation here is that the code I’ll be executing will mostly be quite short, but it will be executing frequently, and I really want the process to be as light as possible, since the application will have a lot of other heavy audio processing to deal with.

Now for my questions.

  1. One of the reasons why I wanted to use SpiderMonkey is because of the JIT compiler and other optimizers. In some of the documentation, I read that SpiderMonkey first parses code to an Abstract Syntax Tree which can later be executed much more efficiently. My question is: is it possible to parse the code once and then store the AST somehow for execution later? What I’m thinking of here is that you pass in some code as a string and the AST could be created only once. Then the AST could be executed many times, without the expensive parsing. It might be, though, that in SpiderMonkey the AST only exists under the hood, and that the user never has access to it. If that is the case, is there some other way that I can achieve this “parse once, execute many times” model?

  2. In JUCE::JavascriptEngine, code can be run in two different ways: “execute” (which can set new variables, and returns success / failure status), and “evauate” (which basically just runs a function and returns the result). Are there similarly two different modes in SpiderMonkey? If not, what is the simplest way of running a line of Javascript code and returning the result in C++?

  3. Related to the last question, I want to know if it is possible to define a global JS function which returns custom C++ types. I see that JS::Value is the object that deals with JS values. This can hold basic types, but can this hold a pointer to a custom type also? for example, in JUCE::JavascriptEngine, I can define a global function play which returns in C++ a pointer to a class PlayExecution, which my C++ app can then deal with accordingly. Is this possible in SpiderMonkey also?

  4. Building SpiderMonkey is the thing that scares me most about it. I’ve found two different guides for building SpiderMoney:
    1: https://github.com/mozilla-spidermonkey/spidermonkey-embedding-examples/blob/esr91/docs/Building%20SpiderMonkey.md

    2: https://firefox-source-docs.mozilla.org/js/build.html

    The two seem to differ quite a lot. Which one should I be using? Are there any other resources I’m missing? (I’m using Windows, btw). Also, a simple question: how large is it once its built?

Thanks for taking the time to read my post. If you can answer just one of these questions, or set me straight on why they are misguided, I’ll be very grateful.

2 Likes

Hi, welcome! To answer your questions:

  1. You’re able to parse code to a JSScript* with JS::Compile. The script represents the bytecode and other information. You can then execute this script multiple times with JS_ExecuteScript.
    (There are also ways to cache bytecode on disk. This is an area that has changed a lot since ESR 91 though and it’s probably not worth it yet for your relatively small scripts.)

  2. JS_ExecuteScript executes a top-level JSScript* and returns the result. The Evaluate APIs take a source code string instead of a script. There’s also an API to compile a function instead of top-level script ( JS::CompileFunction) which you then call with JS_CallFunctionValue.

  3. You can use a JS object to wrap your C++ class. You can do this by defining your own JSClass with 1 reserved slot. You can then use this slot to store a pointer to your PlayExecution class (store PrivateValue(PlayExecution*) in the reserved slot). Other C++ functions can then get the PlayExecution* pointer out of this JS object.

  4. It’s probably best to follow the embedding-examples docs.

These are some quick pointers to get you started, but we’re happy to answer more questions. We also have a Matrix channel where other embedders hang out as well, SpiderMonkey at chat.mozilla.org

A few additional notes:

  • There are different tiers of JIT compilation. The engine will not do the most optimized compilation immediately, both because it is rarely needed in most applications and because it requires type information that is gathered during execution in earlier tiers. There are ways of configuring it to do JIT compilation earlier, though. For an audio application, you might need to play with it some and figure out how to get it “warmed up” enough before playback that really needs to be both fast and dropout-free. But it is very possible that you will not need more than the baseline interpreter. It’s more than fast enough for most things.
  • You are correct that SpiderMonkey is heavy. The library is rather huge. Be sure you’re not including debug information when measuring its size, but even without that be prepared to be surprised at just how big it is, in the 30-50MB range. And it’s not set up to omit parts in order to slim it down.
  • I think V8 and ChakraCore are similar, though it’s possible they have the ability to be trimmed down (I don’t know).
  • You will at some point have to deal with integrating with the garbage collector, which is a whole topic by itself. Both in making sure the engine can tell what is and isn’t garbage, as well as scheduling when it runs. (If possible, you’ll want to trigger it to run when it’s not going to underrun your audio.)
  • SpiderMonkey is not a realtime JS engine. It does get used for soft (like, really soft) realtime applications, but it reserves the right to pause when it feels the need. GC is probably the most common source of pauses, and most but not all GC pauses can be given a (best effort) bound.

Thanks for these detailed responses, they’re very helpful. Just to clarify, I won’t be using SpiderMonkey on any realtime audio thread. I have separate threads for audio and for JS parsing, and I have already worked out solutions for thread synchronization.

I think I will start by trying to run the JIT on this same thread. If I understood the documentation correctly, there I should be able to set JSContext to do this. My idea here would be that the parsing thread does all the heavy lifting the first time it encounters a function, and then calls only the optimized bytecode thereafter (unless the code changes, in which case it resets). I suppose that I haven’t factored for the multiple levels of optimization in this reasoning though. I’m guessing that I can use the ReadOnlyCompileOptions in JS::Compile to define some of this behavior? (In any case, this seems like a detail I could probably work out later).

I also haven’t given too much thought to garbage collection, but there is a lot in the documentation that I haven’t gone through yet, so I’ll start there. I’m guessing that I can use JS::Handle to ensure that a variable doesn’t get deleted. Is it possible to to use JS::Handle with a custom class stored in PrivateValue, the way jandem described?

I feel like all of these things will fall into place, but the thing that intimidates me the most right now is just building the library. I got OK at building software on Linux, but now that I’m working on Windows and Visual Studio, I’m pretty clueless. Are there any precompiled binaries out there that I can download? If not, my questions are going to be on the level of "how do I install GNU make, zlib, libffi and autoconf" and “can I do this on a PC where I don’t have administrator privileges?” (lol)

JSRuntimes are specific to a single thread, and JSContext is 1-1 with
JSRuntime. The runtime is capable of parsing on another thread, but I’m
not familiar enough with that to talk about it. Alternatively, you could
save out bytecode-compiled data on one runtime and load it into another.
You probably ought to see if parsing time even matters for your project
before exploring any of those, though.

JS::Handle doesn’t itself keep anything alive. JS::Handle<JSObject*>,
for example, is just a fancy JSObject** where you’re promising via the
C++ type system that the pointed-to JSObject* is protected in some way
(protected from being considered dead as well as updated if a GC moves
it). Usually the actual protection comes from storing the pointer into a
JS::Rooted<JSObject*> in a local variable, though you can also get
JS::HandleJS::Value from the stack (the runtime keeps the stack alive
for you, so pointers to pointers on the stack are fine.) Anything else
is typically copied into a Rooted before being used. (And passing a
Rooted into a function that accepts a Handle will autoconvert Rooted ->
Handle.)

You can’t automatically use Handle or Rooted with your own C++ class
pointers, though it is possible to specialize some templates if you need
to. Whether that makes sense depends on how you want to manage your C++
objects’ lifecycles.

For building, you mostly want to run mach bootstrap to get the
libraries and compilers needed, then mach build to compile. I think
bootstrap works on Windows? Sorry, even though I’ve set up a Windows
box, I don’t touch it much. The build instructions are more up to date
than my brain.

1 Like

So are you saying that the whole engine runs on a single thread, which is (presumably) whichever thread calls into it? If so, this will certainly make things a lot easier.

Can you clarify what the difference is between “keeping something alive” and “protecting it from being considered dead”?

The github instructions don’t call for mach, but it’s good to know I have this option. I have some time this weekend and I’ll try to build it then. Will I be able to get winget to install the necessary packages?

Hello, I’m returning to this after a while. Unfortunately I still haven’t gotten SpiderMonkey built, but I have some time this weekend to work on it. My plan is to use a Fedora machine and follow the instructions on github, using autoconf2.13. However, I do most of my development on Windows, so I’d like to target the build for Windows, not Linux. Can anyone advise me how to edit the makefile so I can achieve this?