Engineering Effectiveness Newsletter (June 2021 Edition)

Welcome to the June (and first) edition of the Engineering Effectiveness Newsletter! The Engineering Effectiveness org makes it easy to develop, test and release Mozilla software at scale. See below for some highlights, then read on for more detailed info!


  • Content process crashes generated via Windows Error Reporting are now being intercepted!
    • Early data suggests around 10% of crashes were previously being missed
    • Many new bugs being found by these crashes, including security sensitive ones
    • We are possibly the first FOSS project to do this
  • The minimum version of Python is now 3.6, no more Python 2!
    • All mach commands were converted to Python 3
    • Feel free to start using 3.6+ features like f-strings and async/await in your Python
  • A new about:third-party page has been created for shedding light on code injected by third parties
    • Bugs in these modules often get blamed on Mozilla
    • Previously these were invisible to users, now they can be used to help diagnose support issues
  • Google’s Fuzzilli (a coverage-guided fuzzer) has been integrated into our JS engine and deployed on our fuzzing infrastructure!
    • Will help find additional bugs and vulnerabilities in the JS engine
  • There are some exciting changes coming with respect to lint and formatting in Phabricator!
    • Automatic code formatting on landing, currently only Python is supported but support for more languages coming in H2
    • Warnings are now displayed by reviewbot. These are optional fixes which won’t cause a back out. This is the first time they are being surfaced.
  • Speaking of Phabricator, the e-mails it generates have gotten a major overhaul!
    • They’re now less noisy, less cluttered and packed with only the most relevant context, so you can tell at a glance exactly what is needed from you
    • Now enabled as the default setting

Detailed Project Updates

Bugzilla and Bugbug

Build System and Mach Environment

CI and Treeherder

  • marco improved the test selection algorithm and enabled it for mach try
    • Running mach try should now provide much better results by default
  • sclements worked with oremj to migrate Treeherder from heroku to gcp/cloudOps with very little down time
  • sclements also hit a milestone with Push Health
    • Push Health is a developer-friendly alternative to Treeherder’s default jobs view
    • Bug reports and feature requests welcome!
  • andi added Firefox builds using Clang trunk for linux64
    • These are currently running as a daily cron task on central
    • This will allow us to identify issues caused by upstream clang changes much sooner
  • andi also added awsy and browsertime performance tests to monitor regressions in Firefox built with clang-trunk vs the official clang toolchain
  • Thanks to masterwayz’s contribution to finish migrating tests from OSX 10.14 -> 10.15
  • jmaher moved the majority of Windows 7 32 bit tests to Windows 10 32 bit
  • jmaher also supported the WebRender team in their efforts to expand test coverage
    • As well as starting to turn off non WebRender tests
  • ahal ported the standalone taskgraph module to Python 3
    • Fenix’s taskgraph has also been migrated, but other consumers still left todo
  • ahal also enabled many more tasks in support of Fission

Crash Management

Fuzzing and Sanitizers

  • decoder integrated fuzzilli (a coverage-guided JIT fuzzer by Google P0) into our JS engine and deployed it in our internal fuzzing infrastructure.
  • tsmith added support to handle OOMs and hangs to Grizzly
    • Existing fuzzers using Grizzly will get this automatically

Lint, Static Analysis and Code Coverage

OS Integration and Security

  • tkikuchi added a new about:third-party page to visualize third-party modules injected into Firefox processes
    • flopez also helped update SUMO docs to explain how to use it
    • This sheds some light on previously invisible modules injected by third parties


  • bdahl added support for the standard 14 fonts (which are not always embedded in PDF files)
  • bdahl also added support for tagged PDFs to improve screen reader accessibility
  • calixte added support for XFA forms
    • These are still used by several governments (like Canada)
    • Test it out by setting “pdfjs.enableXfa” to true in about:config.

Phabricator, moz-phab, and Lando

Release Engineering and Management


Thanks for reading and see you next month!