outage - 14.02.2017

(John Giannelos) #1

What happened

On 14 Feb between 10:32:51PM and 11:05:51PM we had an outage on portal. The issue was triggered because of the way we handle RSS feeds for the mozilla portal. Posts from are aggregated to homepage. In the past this was being done in the frontend but after PR 1304 we switched to feedparser and started parsing RSS feeds in the backend. The outage was caused by server being down.

Workaround server got back up after 30 mins so webops considered this issue resolved

Next steps

Improve homepage code to gracefully cache/timeout reps planet feed parsing.