How to train your Fennec (to be cuter & cuddlier)

Mozillians
#1

I was drafting a blog-post, while it hit me… Discourse would be rather better platform for this topic.

Chhota Fennec (Li’l Fennec / छोटा फेनक) is a new experimental Mozilla project, being designed for participation (hence this category seemed most suited for posting this). The core agenda behind the project, is to make Firefox for Android slimmer, leaner and faster on lower end devices, primarily targeting Indian not-yet-web-native users.

There are several aspects of making Fennec more feasible for that target audience. Loweing the APK size, removing unnecessary features (and downloadable components on demand), server side compression (with care for privacy) etc. are the lower to higher hanging fruits to achieve the goal.

N.B: This post explicitly discusses my today’s trial of dismembering the APK & discovering which all parts contribute into the “download size” of Fennec & how.

Foreword

I’m using apktool to unpack yesterday’s nightly (42.0a1 x86) build, on a *nix host system. There will be just enough information for anyone who wants to give this another try. I’m open to questions which dig deeper, but initially will not go into the technical details, and focus on summarizing my findings in layman’s terms - easy to be understood & communicated with.

Basic Details

Firefox for Android is built for 3 different platforms (binary compatibility). Each weighs around 45MiB in download size, and around 100-110MiB while expanded/installed.

For a regular person on 2G GPRS/EDGE (~300kbps working bandwidth) this is roughly a 20min download - IF Murphy doesn’t strike (but we all know, Murphy does strike - right?). Also, 100MiB free space on entry level Android phones are a considerable amount (even if available/allocable).

It may look like a far-fetched, blown-out-of-proportion argument to make Firefox for Android more popular, or a strawman fallacy from a personal perspective of who already has better/faster devices and internet (2G ought’a be just an edge case at best - because who doesn’t have 4G, right?), but this is a raging problem & primary reason for people to not opt for Firefox on their Android devices. Here’s a proof (notice any common theme?):

StatCounter-browser-IN-monthly-201406-201506.png854×500 187 KB

Weight Distribution

[root@debs fennec]# du -h --max-depth=1
62M     ./smali       # 62 MiB
364K    ./original
664K    ./lib
24K     ./unknown
37M     ./assets      # 37 MiB
7.9M    ./res         # 8 MiB
107M    .             # Total Unpacked Size

These are the sizes of the directories inside the unpacked APK . Clearly we can focus on the /smali, /assets & /res now.

So, looking inside /smali for things which count in MiB:

[root@debs fennec/smali]# du -h | grep [0-9]M
# snipped
11M    ./android/support
# snipped
23M    ./org/mozilla/gecko
# snipped
23M    ./com/google/android/gms
# snipped
62M    .
  1. I’m no pro at GMS (Google Mobile Services), but a hell lot of them (fitness, wearables, plus…), if not all, can be simply gotten rid of.
  2. Gecko is next in the line bulky stuff that’s included in the package. Discussed up next.
  3. A lot of content inside support are non-essential, and can be gotten rid of.

Similarly, inside assets we basically have the XUL libs. It’s the thing that makes Firefox’s (and its addon’s) interface work (and the thing we’re trying to get rid of, for a while):

[root@debs assets]# du -h | sort -hr
9.2M    omni.ja              # 9 MiB of XUL/XBL/PNG overloads
26M     x86/libxul.so      # 26 MiB... oh noes!
1.1M    x86/libnss3.so
# snipped

The res directory primarily has all the cool effects, animations, and the interface images, icons, logos of various sizes (for different resolutions) etc. I’m not getting further into this, but I hope it’s agreeable that we don’t need too much eye-candy for a lean browser.

More about Gecko

[root@debs smali/org/mozilla/gecko]# du -h --max-depth=1 | sort -hr
23M    .
3.9M    ./sync
2.2M    ./home
1.5M    ./widget
1.4M    ./fxa
1.3M    ./background
1.2M    ./gfx
1.2M    ./db
748K    ./toolbar
576K    ./webapp
552K    ./tabs
516K    ./preferences
304K    ./util
268K    ./overlays
256K    ./prompts
236K    ./menu
236K    ./favicons
172K    ./health
160K    ./animation
136K    ./mozglue
124K    ./updater
112K    ./tabqueue
112K    ./distribution
84K    ./tokenserver
84K    ./browserid
80K    ./mdns
76K    ./javaaddons
64K    ./sqlite
64K    ./firstrun
56K    ./lwt
28K    ./trackingprotection
8.0K    ./tiles

The amount of background services we run (sync, telemetry, healthreport etc), others have made usable browser-clients of that size-factor. Clearly a lot of components can be safely removed without a regular user even noticing anything. All that’s required, is to be able to browse web-pages. Period.

OK, I’m lost now…

No, you’re not. Here’s an approximated summary of what all we do, and projected results:

Component        Size (MiB)       ~ removable %      Saved (MiB)
GMS              23                 70%              16
Gecko            23                 50%              12
Support          11                 50%               5
Interface        10                 50%              5
XUL Lib          26                100%              26
Eye candy         8                 75%               6
Total                                                70

So with a ballpark estimation, we just saved 65MiB of expanded app size, which is about ~30MiB reduced from APK size as well (that is 65% of the initial 45MiB).

There are further possibilities of removing miscellaneous not-important-to-have components, like WebRTC, shumway and stuff like them. Combined, they’re 10MiB+ as well.

Bottomline

It doesn’t seem to be an unreachable goal to reduce 70% of the size of Firefox for Android to fit the purpose of feature-limited browsing experience. A sub-10MiB browser is likely to increase in popularity manifold in bandwidth deprived mass markets like India (and pretty sure, it’s not ONLY India, where this challenge is faced).

FWIW, there will be many volunteer developers as well, willing to contribute to make this happen. We simply need to design the project making it mutually beneficiary for both Mozilla and the contributors.

2 Likes
#2

Placeholder first comment, for unforeseen changes/fixes/updates.
Ignore this comment for now.

#3

Hey Soumya. Thanks for looking into this.

You should be doing your analysis on release builds. Compare, say, Nightly 40:

http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/2015/05/2015-05-09-08-12-31-mozilla-central-android-api-11/en-US/

to Beta 40:

http://ftp.mozilla.org/pub/mozilla.org/mobile/releases/latest-beta/android-api-11/en-US/

You’ll see that the release build is 8MB smaller (30MB compared to 38MB). There’s a lot of stuff that goes into a Nightly that doesn’t reach release; the updater, debug symbols, etc.

You should also be careful how you calculate sizes. du is very filesystem-dependent, and it’s also ignoring compression. We deliberately package libxul.so and omni.ja with seekable compression, so they’re never extracted from the APK (which is why Fennec doesn’t take 100MB on your phone when it’s installed). The raw disk sizes are useful for relative comparisons to the same file over time, but not in absolutes.

To speak momentarily to some of your more specific analysis:

  • We’re unlikely to remove the Google support libraries unless the feature that depends on them is removed, but we have a bug on file to use granular versions: Bug 1115004. In the mean time we rely on ProGuard to strip out code that isn’t used. We exclude them from the constrained Gingerbread build, because the Android features that we integrate with that need Play Services aren’t available on Gingerbread.

  • You’re unfortunately totally wrong about omni.ja. It’s the JS side of Gecko (libxul.so being the other side); probably 10% of it is removable, not 100%, and it compresses very well.

  • You can ignore the whole smali directory. It’s not present in release builds. If you want to see how much compiled Java code we ship for the entire app, use jar tvf on the APK. You’ll see that classes.dex is 4.7MB for Firefox 40 in Beta. This is why I don’t suggest much time be spent on trying to remove Java features; probably the entire compiled FxA code is smaller than the welcome image on the login screen.

1 Like
#4

Noted; I should’ve taken the current release build itself probably. Ignoring ./smali going forward.
Although, approx 1MB size increase per release versio itself is a weird growth metric! :stuck_out_tongue:

Please expand on this, so that I can answer better.
FWIW, I’ve calculated the compressed size of omni.ja contents (~9MiB) - expanded, they’re about 30MiB.

  1. Files can’t be used in their compressed format. If omni.ja isn’t being expanded while being installed, it’s (taking a guess) being expanded in-memory at the application runtime. Which implies:
  • Slower startup speed
  • Higher memory consumption

  1. Even though omni.ja seems well compressed, it carries a lot of baggage, which simply aren’t required & can be dropped. More on that later.

  2. More on libxul.so later too.

Good time to take a look and figure out how they can be removed. As mentioned before, most of those services (if not all) aren’t required for a lean/minimal browser.

  1. I’m probably not wrong about this. Consider the current direction of ditching XUL. In which case, both libxul.so & omni.ja are almost completely useless.
  2. Please don’t be mistaken, as I’ve not called omni.ja as JS overload.
  • Almost 30% of omni.ja are fonts - which can definitely be removed, with a little consideration.
  • Another 30% are the modules - most of which can simply be dropped for Chhota Fennec.
  • If we’re moving away from XUL, all the XUL/XBL files are totally unnecessary.
  • Like fonts, most of the various-resolution PNGs can be chucked (SVGs?).
  1. At most 20% of omni.ja are the JS/JSM.
  • Try du -ah | grep "\.jsm" | grep [0-9][0.9]K | sort -rh for JSM on expanded omni-dir
  • Try du -ah | grep "\.js$" | grep [0-9][0.9]K | sort -rh for JS on expanded omni-dir
  • If most modules are removed, they’re also removed as part of it.
  1. Hence, I doubt "probably 10% of it is removable" is a valid statement in Chhota Fennec’s context.

When we’re targeting a ~10MiB for the whole browser, 4.7MiB is almost half that size. Can’t simply be ignored. But point taken that it might take considerable amount of work-hours to get that done. In that case, why don’t we start with the low hanging fruits, which wouldn’t need too much of efforts to start generating results?

#5

I’m probably not wrong about this. Consider the current direction of ditching XUL. In which case, both libxul.so & omni.ja are almost completely useless.

XUL and Gecko are not the same thing. Ignore that the library is called ‘libxul’. libxul.so and omni.ja are everything: networking code, JS execution, HTML layout. Many of those browser features are wholly or partly implemented in JS. There are some bits of XUL and XBL stuff hanging around in Fennec’s omni.ja, but it’s not much.

You also can’t reject things like images in omni.ja out of hand — where do you think things like network error pages come from? And we use images from omni.ja in Fennec’s UI.

There’s stuff to remove here (e.g., the editor images that we use in about:config are a motivation to ship about:config as an add-on), but that’s a game of inches.

A version of Firefox for Android without Gecko would be a whole new product, and a lot of work, and as such it’s simply not on the table.

Easily separable chunks of functionality (e.g., omni.ja fonts) can be split out and shipped separately, but that separation is a long way from “don’t need it, delete it”.

When we’re targeting a ~10MiB for the whole browser, 4.7MiB is almost half that size.

Firstly, that’s not compressed size.

Secondly, we are not targeting 10MB for the whole browser. A 10MB Gecko-based browser is not a realistic goal right now. A 10MB non-Gecko-based browser already exists — there are plenty of them on Google Play, right? It doesn’t make sense for us to build one.

1 Like
#6

A 10MB non-Gecko-based browser already exists — there are plenty of them on Google Play, right? It doesn’t make sense for us to build one.

Now that brings up a really interesting, strategic and maybe philosophical question.

If our most important, north-star goal as an organization is to hold long-term relationships that help people and promote the Open Web…

…and most of the devices in the world that access the web are Android powered…

…and our ability to have Mozilla/Firefox on those devices is compromised by the size of our current browser ([1] see below for caveat on this)…

…and the theoretical limit on the size of Fennec is governed by the size of Gecko…

…then, why wouldn’t we consider, as Mozilla, building a Android browser not powered by Gecko? [2]

Maybe a distraction, but…thoughts?!

[1] Granted this is a big assumption, that the size of the browser is substantially limiting to both distribution partnerships and user-initiated installs.

[2] Note, I’m not making an “if we build it they will definitely come” argument, more of a “if what we are building prevents them from coming, then building something different could give us a chance.”

1 Like
#7

It might be worth flipping that chain of reasoning at 90°: would we help achieve Mozilla’s goals by investing in a commodity WebKit browser?

Generally our feeling is no, for lots of reasons:

  • it’s a lot of investment when we have lots and lots of other things to work on. Building a WebKit browser from scratch and promoting it is a multi-million-dollar endeavor. We’re making those investments in places that will move the needle further.
  • it’s not clear if there’s much point. Pushing WebKit doesn’t advance the health of the web, so we’d only do it if it’s the only option (see iOS).
  • the competitive outlook isn’t compelling. In an established commodity market, could we build an offering that would displace users from Opera Mini or the phone’s stock browser in enough numbers to be visible in charts? I doubt it.
  • the timescale we’d be looking at means we’d be targeting a shrinking niche. If we shipped a 15MB WebKit browser in 2016, alongside a 25MB Gecko-based Fennec that had more features, would anybody choose the small one? And would they be better off if they did, versus using Dolphin or UC?
  • it would detract from our clarity of message and ability to affect the direction of the web via mainline Firefox on Android. Imagine having two browsers with two entirely different web engines and codebases that have the same name!

I joke a little that it would be cheaper to buy new phones for people than to build them a new browser. It’s definitely true that it would be cheaper to buy preloads than to build a smaller browser.

So our focus is on taking the wedge we have now — Fennec — and making it more effective in all markets. Solutions that involve shipping multiple browsers or different rendering engines aren’t solutions at all, because they don’t make financial sense, don’t drive product adoption, and don’t
advance the Mozilla mission.

1 Like
#8

I wanted to respond to this single point separately.

…and our ability to have Mozilla/Firefox on those devices is
compromised by the size of our current browser ([1] see below for caveat
on this)…

Mozilla’s goal is to not make people use Firefox. It turns out that most of the time we think it’s a pretty good way to advance our mission of a free and open internet accessible to all, and of course most of my job is making Firefox succeed, but Firefox itself is a means to an end.

That means we can take the very pragmatic view that maybe it’s best to cede the market for 15MB browsers, because it’s not going to achieve our goal. Instead we can focus on what we can achieve.

1 Like
#9

What is the distribution of download speeds in India?

For example:
20% of population has zero cell reception
25% of population has < 300kbps
40% has 300kbps - 500kpbs
5% has 500kbps - 1mbps
5% has 1mbps - 3mbps
3% has 3mbps - 5mbps
2% has greater than 5mbps

#10

According to a survey (not a well trusted source, but seems pretty darn accurate):

  • Around 30% has less than 256kbps
  • Around 65% has less than 2mbps
    • Which means 95% under 2mbps
  • Around 3% has less than 5mbps
  • Rest 2% only has a usable Internet connection
    • Like me, on a 100mbps unlimited… like a boss, in Bangalore/India!
    • And when I’m back home, in my village… no 3G there yet. So, just GPRS/EDGE @ 160kbps

Also the wall street journal post goes along the same line:

Although, please keep in mind, these are promised speeds - not at actual experience. Add the disruption of services, jitteriness, frequent disconnection etc. making the whole thing a far worse a mess than the already shitty it appears up front.

#11

Some progress on Fennec download size:

https://bugzilla.mozilla.org/show_bug.cgi?id=1269440

#12

On a related note. The recent decrease (Not so recent actually)

Pasted image675×864 167 KB

And the relevant commit