Migrating AMO forums to Discourse

Archives Community Ops/Discourse
#1

Continuing the discussion from Special Project Planning - 2015-05-05:

I checked with @r_oVhPfcJCUUC5wbm6i4_C2Q and the script does migrate users. It imports them as discourse users so they can log in using Persona with the same email address as they have listed in the old forum.

With regards to migrating threads, there’s two approaches we can take:

  • Use the import script as-is and let it migrate all of the categories and threads. This would be the easiest option but we’d then have to clean up all the different categories, which could take a while.
  • Get @leo or somebody else who knows ruby to edit the script so that it just dumps all of the posts into one category, which we could put in the archives category. It wouldn’t be pretty to browse, but it would be archived and searchable. (I suspect most people hit old threads via search anyway)

The other thing we’d need to think about are the new categories we’d want. From experience, I’ve found that less is more as too many categories can become confusing, but that comes later in the roadmap.

Where should we get started?

/cc @clouserw @jorgev

2 Likes
#2

Can we have an Archives::Add-ons subcategory, so that they aren’t mixed with other archived posts? I think that’s good enough, and better than having to deal with the old category tree.

I can help coming up with new categories.

#3

Cool, @leo will adapt the script. He just needs 100% confirmation that you want every thread from the old forum in a single category. We can make the script sort the posts however you’d like it to.

#4

I think it’s fine to move all of them to that single category. There may be a handful that we want to move to the real categories, but we can deal with them manually.

From the original forum I would skip migrating anything in the Introductions section, since they’re mostly useless threads.

#5

Thanks for starting this thread, @yousef. Are there clear next steps from where we are? I’d like to get a timeline and roadmap going if no one else has one already. I can build some kind of a roadmap if everyone can let me know what their steps are. Off the top of my head:

@yousef : How long will the script changes take (please note passwords are not in the AMO db so they won’t be available)? Is there a transitional notice needed on the site? Are any other changes needed? How long does the script take to run?

@jorgev - have you notified the forum users? Do you need a transitional period? I assume we can set it read-only before we start the transition? Do we need redirects in place? Do we need a blog post or a SUMO article? Have you chatted with Legal at all about any concerns there?

Thanks

#6

I haven’t notified them yet, but I think I’ll start moving them in that direction soon. First thing I need to do is set up new forum categories here. Then I can tell people to start posting here instead, ideally with some approximate date of when we will close the old forum. We will definitely need a redirect when we don’t need the old forum anymore, and a blog post is in order.

I haven’t talked to Legal about this, but I think we should loop them in about the migration. I’ll look into that.

#7

@leo can answer that first question since he’s doing all of the coding wizardry. Once we have the script we can set a one hour maintenance window to get it migrated. I’m not sure how long it will take for the script to run but since it will be working from a local DB dump it shouldn’t take too long.

As for passwords, we don’t store any either so that’s not an issue. We just store their email address which Persona verifies belongs to them.

Feel free to ping me on IRC (yalam96) any time and I can help out with setting up the email addresses and such.

#8

They won’t take too long at all, it’s really a matter of making sure I didn’t break anything, and finding the time to make the changes/test (which I won’t really have the time to do rest of this week and this weekend). I’m confident everything will be ready to do the migration in the first week of June.

I have a few questions for @jorgev first, though, from reading through the script:

  1. Do you want to import attachments and images (which includes avatars)?
  2. Do you want to import private messages?
  3. Do you want to import banned users and their posts?

In order to test the script I’ll need:

  1. A db dump.
  2. A tarball of the phpbb installation, if the answer to #1 above was yes.
#9

“No” to all 3.

#10

@yousef Here are the sub-categories we would like to have, with their descriptions:

  • Announcements. Important announcement for the add-ons community.
  • Development. Technical discussions about add-on development.
  • addons.mozilla.org. Discussions about AMO add-on reviews and bugs.
  • Themes. Discussions about background themes and complete themes.
  • Add-on Support. Need help with an add-on? Ask here!
#11

I’ve made the categories, feel free to tweak them if needed.

#12

I’ve edited the script, now I need that db dump to test and make sure I haven’t done anything stupid:

(Original here if you want to diff.)

#13

Thanks @leo. Jorge has started a conversation with our legal team to make sure we have the thumbs up here. As soon as that happens, we’ll get it over to you. Thanks.

1 Like
#14

Since the conversation with Legal has been really slow, we’re going to split up this migration and start telling people in the old forum to use the new one. I’ll start by announcing it in the old forum today, and then in the blog sometime next week. The plan is to make the old forum read-only on July 1st. Hopefully we’ll see some progress on the content migration bit before then.

For now, heads up on (hopefully!) many new users and posts.

2 Likes
#15

This is a relevant discussion about pros-cons about migrating old forums content :wink:

https://meta.discourse.org/t/is-forum-migration-really-worth-it/7644

#16

@r_oVhPfcJCUUC5wbm6i4_C2Q’s latest question aside… :smile:

We met with legal today and they were open to the possibility of migrating the old data. I’m going to get a dump of the database and look into sanitizing it. Thanks @leo for posting the script. I’m going to try to just create a dump with the columns you care about which should put legal more at ease.

Overall though, as Jorge said, this doesn’t block us so we’re moving forward on directing people to this forum. Thanks everyone!

2 Likes
#17

Is forum migration really worth it? Good question.

In the meetings we’ve had about this no one has been passionately for importing and after review it’s a decent chunk of work at an awkward time (the entire company is at a meeting all of next week) and it means several more meetings and reviews (eg. a rapid risk assessment with our IT department).

Jorge and I talked this morning and we agreed that it’s something we don’t need to pursue at all. Our plan is to maintain the current forums in a read-only state for a period of time (while pointing people to the new forums) and then eventually just shut them down.

Thanks @leo for finding the script, but we’re not going to use it this time. Thanks to everyone for all your help around planning and migrating.

2 Likes
#18