Engineering Effectiveness Newsletter (November 2021 Edition)

[ahal (Andrew Halberstadt)]

November 2021

Welcome to the November edition of the Engineering Effectiveness Newsletter! The Engineering Effectiveness org makes it easy to develop, test and release Mozilla software at scale. See below for some highlights, then read on for more detailed info!

Highlights

• A massive refactor around how Mach manages its Python environment is wrapping up.
  • Separate virtualenvs for individual commands and checkouts.
  • No more failures due to a Python dependency being the wrong version.
  • No more mach create-mach-environment.
  • Significantly eases the burden on tool authors.
• The stack walker used in CI and on developer machines has been re-written in Rust.
  • The new one is faster, more flexible, more robust and provides a human friendly output format!
  • Will soon be used by crash-stats.mozilla.com as well.
• Explorations into Intel CET (Control-flow Enforcement Technology) are yielding security bugs.

Contributors

• Jane Kotovich (janekotovich)
• Sivadeilra
• Devon Burriss (dburriss)
• Giovanni Tangredi (giovannitangredi)
• Luca Barbato (lu-zero)
• Marco Ballario (marco-ballario)
• masterwayz

Detailed Project Updates

Bugzilla and Bugbug

• Sebastian enabled auto tracking approval for expiring telemetry probes.
• Calixte updated the autonag bot to notify about P1 bugs with no assignees.
  • Please use this as an opportunity to re-triage P1 bugs that are not actually P1 (see Priority Definitions - Firefox Source Docs documentation).

Build System and Mach Environment

• Glandium fixed artifact builds so they now work on Apple Silicon Macs.
  • This was previously blocked by the new code signing requirements introduced in MacOS 11.
• Mitch released MozillaBuild 3.4, upgrading bundled software to recent versions.
• Mitch improved Mach so it now automatically manages its Python environment, and uses a different virtualenv for each worktree
  • No more unexpected failures when out of date or manual ./mach create-mach-environment invocations.
• Mitch also resolved a major source of inconsistencies when importing Python packages
  • This allows for the simplification of cases like symbols_archive.py
• Alex improved compatibility with Python 3.10, specifically around the collections module.

CI and Treeherder

• ahal cleaned up the test task configuration by removing ambiguity around where tasks run, revamping the variants system, exposing data about the execution environment of a task and re-factoring the transforms.

Crash Management

• Aria Beingessner rewrote the stack walker we use in CI and on developer machines to Rust.
  • The new one is faster, more flexible, more robust and provides a human friendly output format!
  • She is also deploying it to Socorro with the help of willkg and will soon be used by crash-stats.mozilla.com.
• Calixte documented Clouseau, a tool that helps quickly triage large numbers of crashes.
  • If you find yourself triaging crashes often, give Clouseau a shot!

Fuzzing and Sanitizers

• The fuzzing team has been exploring Intel CET (Control-flow Enforcement Technology).
  • This has been finding real security issues in the field and bugs are being filed.
  • Team is looking for ways to make these types of issues easier to pull out of crash telemetry.

Lint, Static Analysis and Code Coverage

• sivadeilra fixed a long standing grcov issue related to symlink usage on Windows.
• Devon Burriss (dburriss) documented how to add a new language to rust-code-analysis.
  • Check out the article we published on SoftwareX journal for more info about rust-code-analysis.
• Giovanni Tangredi (giovannitangredi) added minimum and maximum implementations for the cyclomatic, the nexit and the cognitive metrics in rust-code-analysis
• Luca Barbato (lu-zero) slightly refactored the mk_enum macro, enabling the removal of a dependency from rust-code-analysis
• Marco Ballario (marco-ballario) moved C integration tests to their rightful place in rust-code-analysis.
• sylvestre added a linter to our CI which prevents the trojan-source vulnerability.
• Standard8 enabled some more ESLint rules on xhtml files under dom/.
  • no-undef and no-unused-vars

OS Integration and Security

• gcp updated jemalloc to use guard pages before the allocation area.
  • This helps prevent buffer underrun exploits.
• Nika Layzell joins the team and will be working on improving our IPC infrastructure.

PDF.js

• Jane Kotovich fixed an issue in XFA where some nodes must be removed from the template DOM because they’re unbound.
• Calixte fixed the long-standing intermittent bugs we had in the pdf.js test suite because of XFA.
• Calixte added support for rich text in annotations.
• Brendan added a way to track 2D path bounding boxes to improve performance and memory usage with certain PDFs.

Phabricator , moz-phab, and Lando

• Mitch tweaked Phabricator Emails to include watchers when sending emails to reviewer groups.

Release Engineering and Management

• Release Engineering would like to:
  • Welcome Gabriel Bustamante who joined full time last week
  • Welcome chidinma who will be improving test migration tooling as part of Outreachy
  • Say farewell to bhearsum who has moved to the Desktop Installation team
• Release Management would also like to welcome Donal Meehan!
• UpdateVerify now pre-fills the cache asynchronously thanks to hneiva!
  • This change improved UV performance by ~19%, saving ~8 minutes on average for each ongoing beta and paves the way to refactor UV to async Python.
• sclements is moving Focus for Android releases to shipit.
  • This will standardize the release process and make lives easier.
• sclements and others got the ball rolling on standalone taskgraph docs.
  • These docs will focus on lowering the barrier to entry for new users.

Thanks for reading! We’ll be taking a break this month, so we’ll see you next year for the combined December/January edition.