Engineering Effectiveness Newsletter (June & July 2024 Edition)

Welcome to the June and July edition of the Engineering Effectiveness Newsletter! The Engineering Effectiveness org makes it easy to develop, test and release Mozilla software at scale. See below for some highlights, then read on for more detailed info!

Highlights

  • Dave Lawrence added a reminder system for bugs to Bugzilla.
  • Benjamin Mah improved BugBot’s product and component classification feature by using the Fenix component model to move bugs out of Fenix::General.
  • Heitor Neiva studied Zucchini, a diff tool we could use for Firefox partial updates. By switching to it, we could make partial updates 30% smaller on Windows (10% on other platforms). Proposal - Firefox Partials Diffing Tool.
  • Evgeny Pavlov and Greg Tatum have been working on training the next batch of languages: 23 translation models were trained for 17 languages, 16 models were shipped in production, 12 models are in progress.

Detailed Project Updates

Bugzilla and Bugbug

Build System and Mach Environment

  • Multiple people added Python 3.12 support to the build system

CI and Treeherder

  • Andrew Halberstadt migrated Firefox CI’s old configuration repo to a new one on Github. This reduced deployments from 20 down to 2 minutes and lowered the barrier for external contributions.
  • Gabriel Bustamante added a Python toolchain for Linux on AArch64.

Lint, Static Analysis and Code Coverage

  • Code Review Bot work on speeding up end-to-end times from patch upload to have linting and similar tasks executed continues under Marco with patches by contractor Teklia. Older data has been purged, monitoring of end-to-end times set up by Sebastian.

PDF.js

  • Calixte Denizet implemented the new alt-text flow for using ML to guess the alt-text for an added image.

Firefox Translations

  • Evgeny Pavlov and Greg Tatum have been working on training the next batch of languages: 23 translation models were trained for 17 languages, 16 models were shipped in production, 12 models are in progress.

  • Greg Tatum investigated the feasibility of porting the translations engine to ONNX to share infrastructure with the GenAI work. This proved not to be feasible, but one outcome is a plan to substantially slim down our reliance on the existing engine.
  • Greg Tatum continued improving the training dashboards (pictured above) and did cost estimates for training, identifying the most expensive parts of the pipeline.
  • Greg Tatum shipped attribute support (such as aria attributes) for Firefox Translations.

Phabricator , moz-phab, and Lando

  • Connor Sheehan ported a significant amount of Mercurial server-side hooks into Lando.
  • A WIP data extract from Phabricator into BigQuery for deep data analysis has been deployed.

Release Engineering and Management

  • Ben Hearsum made it possible to configure Taskcluster scopes and grants more granularly in our Taskcluster configuration scripts
  • Pete Moore and Ben Hearsum got to the bottom of a tricky issue on Taskcluster workers that prevented us from moving them off of a deprecated version
  • Gabriel Bustamante updated Ship-It to dynamically handle release milestone dates and product version numbers. This enhancement removes the need for repetitive Ship-It pull requests to update hard-coded values, eliminates rote deployments, and automates the Product Details API rebuild, all of which were manually performed on release day.
  • Gabriel Bustamante added a manpage to the Mozilla Firefox .deb package.
  • Heitor Neiva studied Zucchini, a diff tool we could use for Firefox partial updates. By switching to it, we could make partial updates 30% smaller on Windows (10% on other platforms). Proposal - Firefox Partials Diffing Tool.
  • Julien Cristau upgraded RelEng’s Taskcluster workers from Python 3.9 to Python 3.11

Version Control

  • Julien Cristau improved the hg.mozilla.org deployment automation, including adding support for deploying to a Rocky 9 host.

mozregression

  • Zeid Zabaneh added Python 3.12 support to mozregression.

Thanks for reading and see you next month!