Testing Translate.Next – Phase 1

Welcome to the topic about the first Phase of testing Translate.Next. If you haven’t done so already, please read our Testing Translate.Next topic first.

In this topic, we’ll explain the testing plan for this first phase. This is also where we will be collecting all your feedback.

Phase 1 Testing Plan

This testing will happen exclusively on our Pontoon staging site. We have not yet turn Translate.Next on on production, and only intend to do so once this phase is done and we have confidence in the state of our code.

Timeline

This testing Phase will last until the conditions to start Phase 2 are met, but will be split in rounds. For the first round, we ask that testers perform their tests before Sunday, April 14th. We will then gather your feedback, turn it into bugs, prioritize those, and work on fixing the most important issues. We will notify all interested testers (here and by email) when we start the next round for this Phase.

Preliminary steps

These steps must be completed before any testing.

  1. Go to Pontoon’s stage website
  2. Log in the same way you would do on production
  3. Go to the translate page for your preferred locale and the project of your choice
  4. Make sure Translate.Next is turned on (look in the top right corner)

Once you have completed these steps, please run one, some, or all of the following instructions.

Regular localization

Follow your usual workflow to get to strings that need to be translated into your preferred locale, and translate some of them. Please send translations with the “Make Suggestions” option on and off to test both cases. Try to use all the features you would normally use on Pontoon.

Reviewing

Search for strings that need to be reviewed (for most teams that would be unreviewed strings), and review them as you would normally. Approve some, reject some, propose better translations when appropriate, and so on.

Search

Search for a specific string in content, and make sure that results are coherent (one easy way to verify that is to run the same search on production and verify that the results match).

Machinery

Look for strings to translate, and use the Machinery tab to translate them. Please test as many different options as possible: from Translation Memory, from Machine Translation, with or without changes, etc.

Logged out

Log out of the website. Make sure you are still on Translate.Next (you can use the navigation bar for example, it is different than in the old Translate page). Then verify that you cannot make any changes (sending, approving or rejecting translations).

Push the boundaries

To say it bluntly, we want you to break it. If you can find a behavior that doesn’t work as it does on the old Translate page, and it is not referenced in our bugs list, then you have likely found an issue. When that happens, please let us know by commenting here. See the Testing Translate.Next topic or the Report template below for more information about how to report an issue.

Why test on stage

This is important: this testing is happening on our stating instance, which means that none of the work you are going to be doing will be reflected in production data, or in any of the products you might contribute to.

We understand that this is not ideal, and that is part of the reason why we are only asking a small number of users to help us with this step. But, we do not want to risk breaking your team’s workflow, or a product’s integrity, because of potential bugs in our code. As soon as we are confident that our code is safe, we will turn to Phase 2, which will happen on production, where all your contributions will be shared with the rest of your team.

Report template

<ins>URL:</ins>

<ins>Description:</ins>

<ins>Expected outcome:</ins>

<ins>Steps to reproduce:</ins>
1. 
2. 
3. 
3 Likes

Thanks Peter for these first 3 bugs:

Hi Adrian. Good job with the new view.

I have filed a few bugs I have discovered when trying to break Pontoon as you asked. If you want to share with the others, feel free to add them there. I myself already lost the tabs when writing the descriptions and testing Pontoon for the next possible mis-behaviour.

If there is anything not clear in those reports, I will be happy to describe better. My English is not the best when I have two bugs two report at once. :wink:

2 Likes

Thanks a lot for all those bug reports Michal! It’s very very useful! :slight_smile:

I’ll be adding those bugs to our list, so no need to paste them all here.

Hi!
Here are a few things I ran into. I tried to break things but couldn’t, I believe that’s a good sign :slight_smile:

  1. URL: https://mozilla-pontoon-staging.herokuapp.com/translate/fr/mozillaorg/firefox/new/quantum.lang/?status=missing&string=172716
    Description: When I filtered the strings to see which ones were missing, the first one didn’t open by default. I just got a blank screen on the right half of the editor.
    Expected outcome: The first string should open automatically for translation.
    Steps to reproduce: filter the strings to display missing strings

  2. URL: any
    Description: search results aren’t highlighted.
    Expected outcome: When searching for a term or even just a few characters, they’re usually highlighted within the strings that appear as search results. This was not the case.
    Steps to reproduce: search for any term

  3. URL: any
    Description: I submitted a few suggestions, but they never showed up as “unreviewed” when I filtered the strings.
    Expected outcome: suggestions should appear as unreviewed
    Steps to reproduce: make a suggestion

2 Likes

Hey @Lizie, thanks a lot for your feedback!

Here are the bugs corresponding to the problems you reported:

  1. This one will probably come as part of https://bugzilla.mozilla.org/show_bug.cgi?id=1539228
  2. That’s https://bugzilla.mozilla.org/show_bug.cgi?id=1511357
  3. I can’t reproduce that one, can you give me more precise steps? I tried on Firefox, on a string that already had an approved translation, and that worked fine. I then tried on a missing string, and again I could see it when selecting the “unreviewed” filter. :confused:

Hi @adngdb!

  1. Alright, now that you mentioned it I can kind of see what you meant in the original bug description but my non-tech brain hadn’t made the connection at all :wink:

  2. Indeed, I missed that one. Sorry!

  3. I can’t seem to reproduce it either, I’m not sure what happened. Now the string shows up both when “unreviewed” and “missing” are selected (by the way, I’m guessing this is not the time or place to bring this up but in my opinion, unreviewed strings shouldn’t appear as “missing”. Just my 2 cents).

Well that was useless. I couldn’t find anything else though! Good luck with the actual bugs that have been found so far! :wink: