To avoid burying the lede, I’ll come right out and say Riot/Matrix is my first choice and it wasn’t close.
Disclaimer: I’ll admit I didn’t give Rocketchat and Slack the same fair shake as the other two. The former due to poor first impressions and lack of activity. The latter due to previous familiarity and bias (against).
In terms of the interface, I’d say Riot and Mattermost are a wash. At first I liked Mattermost’s interface better, but after spending a bit of time tweaking my settings I managed to make Riot look much more appealing and less cluttered. I learned that a lot of these settings can have a server default, so we can fix this for people if we want. But overall they both seem to tick most of the major boxes in terms of usability, and getting into the minor details here isn’t very interesting to me.
The main reason to switch away from IRC is to make it possible to fight abuse. And here Riot/Matrix seems head and shoulders above the others. One feature I particularly liked is the ability to mute users. This means if anyone is experiencing abuse either in channel or in direct messages, they can take matters into their own hands without needing to wait for an admin/moderator to step in. This feature didn’t seem to exist in Mattermost. Matrix also has a handy moderation guide. While I haven’t tested out everything myself, it looks like Matrix supports many ways to deal with spam and abuse. In contrast I searched the Mattermost docs for “ban”, “abuse”, “kick”, “moderation”, “moderator” but none of those came up with any relevant hits.
All that being said, the biggest draw to me is how dedicated Matthew and the Matrix team are to making this work. From standing up the moderation guide, to fixing bugs in near realtime to simply hanging out and answering questions. It’s clear they are dedicated and willing to support Mozilla (at least for the time being). It has been a very pleasant surprise and I think both orgs will benefit from the relationship (Mozilla gets support, Riot/Matrix gets a large user study). The interactions I’ve witnessed so far have made me confident that any problems Mozilla might have with Riot/Matrix will be taken seriously and addressed relatively quickly.
I have several other reasons for preferring Riot/Matrix too, though they aren’t criteria in the doc so I’ll just list them quickly.
- Read-only guest mode
- E2E Encryption (*not quite fully implemented in Riot yet)
- Open protocol / not a silo
- Federation. I know this was a non-goal, but being able to jump into the channels of other open source projects that we interact with (and letting them jump into ours) is useful. If we can enable this without making the moderation story harder, it would be nice to do.
- IRC / Slack bridges (I think Mattermost supports these too)
- Other bridges (not so relevant to Mozilla, but nice for individuals who might want them)
- Several clients to choose from
- Markdown (Mattermost also has this)
But even without all these extras, from my experimentation Riot/Matrix seems like the best choice for Mozilla.