Outage of the Public Photon Cloud in region US East on October 20th, 2018

Our team works very hard to achieve a very high QoS – even the shared Photon Public Cloud has an uptime of 99.9% and higher in all available regions. We are proud of this and serve more than 300 Million players every month.

On Saturday we hit a snag in one of our 13 regions: We apologize for any inconveniences that may have occurred and we have applied corrective measures to prevent similar issues in the future.

Affected Products: Public Cloud / Premium Cloud

Affected Regions: US East

Downtime (UTC): Sat 1:30am – Sat 08:00am // 6.5h

Root cause:  Memory issue on master servers in the US East region.

Problem: An alert-endpoint was misconfigured and not triggered for the US East environment. Hence our 24×7 NOC team was not alerted and could not resolve the issue.

Resolution:  The issue was resolved by the German team that immediately replaced the master server.

Our operations team took immediate actions

• have double checked all alert configurations to make sure they are correct
• installing fallback alerts to our 24×7 NOC

We are deeply sorry! We will do the best possible to keep highest standards and continue to invest into our monitoring, alerting and automation.