Synchronous Messaging at Mozilla - Trial Servers & Feedback

Mozillians
#31

I feel that 1. it is a huge missed opportunity to not experiment
with Zulip, 2. a huge shame to consider a proprietary platform
that is Slack among the 4 candidates. It is still early in the game. The supposed IAM
feature depends on having SAML support, which is only ~100 LOC
away to have the feature implemented [1]. I am not affiliated with
Mozilla or Zulip, but as a bystander I can’t bear to watch this
neglect to happen! I can help with the man-hour to implement the
required feature(s).

[1] https://github.com/zulip/zulip/issues/716#issuecomment-530141795

1 Like
#32

That’s awesome. Gnome did the same recently. Would be nice if all the big names in the FOSS community (Mozilla, Debian, RedHat/Fedora, etc) also join Matrix. :slight_smile:

6 Likes
#33

I find https://chat.inkscape.org incredibly inviting because I have anonymous read access to a wide variety of interesting conversations going on about Inkscape. This inspired me to create an account and join in the fun. :tada:

In contrast, all the proposed chat solutions for Mozilla feel like a walled garden. :european_castle:

#34

As per #synchronicity on moznet, Matrix (and Rocket.Chat too, i think?) both have guest access mode which can enable anonymous read-only access. For Matrix, it just needs to be enabled in the server settings in Modular.

2 Likes
#35

We used Mattermost at Mozilla Foundation for a little over a year before the company migration to Slack. We had a lot of non-technical contributors for which IRC wasn’t an ideal candidate. It worked, but wasn’t a favorite compared to Slack.

Mobile support left a lot to be desired at the time, the apps were slow and buggy. It lacked some features and nice-to-haves that you might expect from a more mature platform. Not sure if that is still the case.

I will say, from experience it’s better to have everyone on one platform if you can - although understandably this doesn’t always happen. With the creation of the Slack there was a duration of a few months where contributor engagement was low because employees would understandably choose to just use the one platform. Which led to delayed responses increasingly as time went on.

Contributors ended up being folded into the MozFest Slack, and tied back into the official Mozilla slack through the #contribute-code channel. This seemed to be a good solution, it was never fully tested because shortly after was the transition away from learning products (phase out of Thimble, etc)

1 Like
#36

Good to know. @ryanwse do you happen to have the list of requirements that were considered when analyzing these tools in the past? It would helpful to frame some conversation here.

Thanks!

1 Like
#37

I love all three options for an open source code and I like it in the next order:

1-Rocket
I think it is more personalized and have a installer to linux and app movil, i like it.

2-Mattermost
This remember a slack, i very like that has installer in gnu/linux but this is not too much necessary, i think. I don’t like that it isn’t very customizable.

3-Riot
In this I don’t like it too much but it’s not bad, mmmm i like this but it is no my favorite option.

1 Like
#38

Around 2016/2017, the Mozilla Philippines Community tried to have a self-hosted Mattermost instance. We abandoned the project because:

  • Most of the local Mozillians are in Slack;
  • Mattermost needs an above average server specs

In my opinion, Mattermost is the better option for us to promote FOSS and at the same time, adopt it as an organization.

#39

After 10 days, here my impressions:

I’ve been daily testing the three options, but I have to admit that I stopped opening RocketChat after a few days, from my point of view its set of of features and UI seem too simplistic for the need we have.

Mattermost seems to have a nicer UI/UX and parity with known slack workflows, but I have to admit when you look a bit deeper, its back-end is not as flexible as I would like it to be. Being have to enforce our CPG and easily handle moderation is a must for me, and it’s seems mattermost is not specially strong on this one (no report, no moderation, no clear way to control a public space where random people will join).

Riot/Matrix initially looked like a less-advanced UI/UX as mattermost, but I have to say that the core matrix team have been collecting all our feedback and even implementing fixes in realtime. This has been a surprise to be and demonstrates a potential for further engagement in case we need paid-modifications to suit our needs. Simple theming and changing some defaults seems to be totally doable, and these were my only cons when talking about the web client.

When talking about backend, Matrix seems to be the most flexible and mature one, not only in terms of moderation tools and policy (which are really strong) but other things like potential federation and integrations.

I suspect a lot of people will provide feedback only based on the main clients UI, but I think that the minor personal preferences we might have can easily changed to fit out needs (specially in the case of riot).

That’s why I would recommend we go with matrix/riot and just understand the minor UX changes needed, since I’m confident the backend will serve us better in the long term than the other options.

Cheers.

13 Likes
#40

I wanted to wait until I post my feedback, but I really couldn’t have done it any better than Nuke just did. I fully, whole-heartedly agree with everything he said.

5 Likes
#41

Hi,

Going to add more information about Mattermost, specially related to security.

Mattermost is really careless about deps in their project components having vulnerabilities, quick example:

The desktop client is not exception, I have done a few checks time ago and found deps with vuln not being taken care, and is being something that keeps happening from time to time.

Be sure to check the vuln record and how they proceed to deal with this type of issues, for example:

Hope this brings a different point to consider.

6 Likes
#42

To avoid burying the lede, I’ll come right out and say Riot/Matrix is my first choice and it wasn’t close.

Disclaimer: I’ll admit I didn’t give Rocketchat and Slack the same fair shake as the other two. The former due to poor first impressions and lack of activity. The latter due to previous familiarity and bias (against).

Interface

In terms of the interface, I’d say Riot and Mattermost are a wash. At first I liked Mattermost’s interface better, but after spending a bit of time tweaking my settings I managed to make Riot look much more appealing and less cluttered. I learned that a lot of these settings can have a server default, so we can fix this for people if we want. But overall they both seem to tick most of the major boxes in terms of usability, and getting into the minor details here isn’t very interesting to me.

Moderation

The main reason to switch away from IRC is to make it possible to fight abuse. And here Riot/Matrix seems head and shoulders above the others. One feature I particularly liked is the ability to mute users. This means if anyone is experiencing abuse either in channel or in direct messages, they can take matters into their own hands without needing to wait for an admin/moderator to step in. This feature didn’t seem to exist in Mattermost. Matrix also has a handy moderation guide. While I haven’t tested out everything myself, it looks like Matrix supports many ways to deal with spam and abuse. In contrast I searched the Mattermost docs for “ban”, “abuse”, “kick”, “moderation”, “moderator” but none of those came up with any relevant hits.

Support

All that being said, the biggest draw to me is how dedicated Matthew and the Matrix team are to making this work. From standing up the moderation guide, to fixing bugs in near realtime to simply hanging out and answering questions. It’s clear they are dedicated and willing to support Mozilla (at least for the time being). It has been a very pleasant surprise and I think both orgs will benefit from the relationship (Mozilla gets support, Riot/Matrix gets a large user study). The interactions I’ve witnessed so far have made me confident that any problems Mozilla might have with Riot/Matrix will be taken seriously and addressed relatively quickly.

Extras

I have several other reasons for preferring Riot/Matrix too, though they aren’t criteria in the doc so I’ll just list them quickly.

  • Read-only guest mode
  • E2E Encryption (*not quite fully implemented in Riot yet)
  • Open protocol / not a silo
  • Federation. I know this was a non-goal, but being able to jump into the channels of other open source projects that we interact with (and letting them jump into ours) is useful. If we can enable this without making the moderation story harder, it would be nice to do.
  • IRC / Slack bridges (I think Mattermost supports these too)
  • Other bridges (not so relevant to Mozilla, but nice for individuals who might want them)
  • Several clients to choose from
  • Markdown (Mattermost also has this)

But even without all these extras, from my experimentation Riot/Matrix seems like the best choice for Mozilla.

10 Likes
#43

I agree with everything @ahal said. Riot is also for me my first choice and Mattermost my second.

Here’s one thing I’d add that hasn’t been mentioned but I think is important for me.

App client

An advantage with Riot is that they have a native app available for Android and iOS. The other chat platforms only offer a wrapper of their web version. I’ve tested the Mattermost one and it doesn’t offer a great user experience. Personally, the new platform should have a mobile app that is seen as a first-class citizen by their developers

Also, on desktop there’s a few native clients for Matrix that integrate well with the OS they’re targeting (e.g Fractal with Gnome on Linux). It also has a plugin for Weechat if you like terminal apps.

There’s also Matterhorn for Mattermost but the developers have no intention to integrate SAML/SSO connection and that’s a dealbreaker for me.

6 Likes
#44

Agree with everything said by @nukeador and @ahal about Matrix.

Matrix aligns with Mozilla Mission compared to other options we have, it is a open protocol that enables interoperability and federation, which is a must needed thing to improve the internet health in the current times.

8 Likes
#45

Hi, everyone: we’re now less than 24 hours away from the shutdown of our chat system trial instances. Tomorrow morning - October 9th, morning Eastern time, and with little or no further notice - those machines will be shut down and permanently deleted . As much as I would love to leave these things spinning until it was time to announce the successful candidate, we’ve made a specific commitment about how user data would be handled on these services and we are going to follow through on that commitment.

We’re grateful for everyone’s participation and the feedback we’ve received so far, and we’ve learned a lot. If you have any information from those instances that you would like to preserve, you need to act on that immediately.

Again, thank you for all of your time, effort and feedback. I’m looking forward to the next steps in the process.

6 Likes
#46

And there it is. As announced, we’ve shut down our trial servers.

I’d like to thank everyone who’s participated in this process and given us your feedback. I’m grateful for the time and effort everyone’s put in to testing these services, and we’ve learned a lot.

I hope that we’ll be to distill it all down, make a decision and announce our next steps shortly.

Again, thank you.

4 Likes
#47

I missed the trial (my bad). FWIW there’s some commentary in Reddit:

#48

Slack

I rarely need/needed to use instances of Slack, but the experiences made me uneasy in ways that were difficult to quantify/describe. There was, I think, an underlying sense of “Yet Another System”; plus an overarching sense of things occurring in relative isolation.

Matrix

IHMO this will be a fine choice for Mozilla.

To counter the many plus points, my only significant criticism is of the notifications UX in the https://riot.im/app/ web client.

(For me, the long-standing shortfalls require use of a last-century technology – e-mail – to plug the gaps in an otherwise very pleasantly forward-looking solution. YMMV; I’m in https://riot.im/app/#/room/#riot-web:matrix.org for feedback.)

HTH

Re: Publicly visible streams · Issue #4817 · zulip/zulip and linked issues, I imagine that the current lack of true public visibility was an additional factor in exclusion of Zulip from the final contest.

Also: re: https://chat.zulip.org/#narrow/stream/7-test-here/topic/push.20notifications I couldn’t get push notifications. @rht join me there, if you like; I may be missing something.

#49

The blog post says that “assuming we can gather that support” the new service will be set up on December 1st. Did that happen?

#50

Being Canadian I didn’t account for American Thanksgiving in my calculations. So I won’t have an announcement for you today, unfortunately. I expect (though cannot promise) to have more to say very shortly.