Mark was kind enough to give me and Ian a quick walkthrough of his testing workflow this morning. Here are some notes on getting set up, as well as some notes on what’s currently broken. Once this matures a bit, I’ll add it to the docs in the pageshot repo.
General Stuff
Gecko setup
Start by setting up your Firefox dev environment. The bootstrap.py script is fantastic and saves many hours of confusion and stress.
The Try server
You’ll eventually want to push to try, which is a server that fronts Mozilla’s testing cluster. You will need to get vouched for level 1 commit access to do so. Read through the required steps here and ask me if you have any questions. Mark has agreed to vouch, so needinfo :standard8 in the Bugzilla bug.
Firefox review process
Mozilla now uses MozReview for uploading patches. There are good docs for it, and it’s under active development.
The mercurial vs git thing
Mozilla picked mercurial as its VCS a few years ago; here are some docs about how hg works at mozilla. If you prefer git, Mark recommends using git-cinnabar to translate between git locally and hg on the server. git-cinnabar is also nicely documented and under active development.
Page Shot specific stuff
Currently, lots of tests are failing on try: here’s Mark’s initial try run; all the oranges and reds are breakage we have to sort out.
If you want to run these tests locally:
- Get a local copy of m-c (see the ‘general stuff’ section above)
- Apply the patch from bug 1346825 to your local copy of m-c
- Mark’s landed a script in the pageshot repo that exports the code into Firefox. This incantation should work:
PAGESHOT_BACKEND=https://pageshot.dev.mozaws.net EXPORT_MC_LOCATION=../gecko make export_addon
- Run individual tests locally like this:
./mach build && ./mach test dom/network/tests/test_network_basics.html
- You can also attach the debugger to local tests:
./mach mochitest --jsdebugger dom/browser-element/mochitest/priority/test_HighPriority.html
- Run tests remotely using try syntax (see
./mach try --helpor the try chooser). The try server takes a long time for a full run (1-2 hours), so you can just run the linux x64 opt tests instead. Takes about 20 minutes, and seems to hit all the broken tests. Often tests will fail on try, but not locally, so plan your time accordingly
about the broken tests
There seem to be two classes of broken bugs: those that break predictably, because Page Shot either adds a context menu item, or a toolbar button; and those that break because of something related to WebExtensions toggling prefs. The latter tests seem to live under the /dom/ module, and seem to affect APIs that can be preffed on, like navigator.connection. Some of the test timeouts seem to belong to the latter category.
