Service Outage: 09/17/2020

Slickstream had a major service outage today affecting all customers. The outage started around 12:30PM Pacific Time and lasted approximately 30 minutes. At that point, service should have returned to normal on all sites.

During the outage, Slickstream was probably absent from most pageviews. But in some cases, Slickstream may have been present, but with search not functioning correctly.

This outage occurred immediately following a software upgrade. Normally, our updates are done in a way that can compartmentalize problems. Unfortunately, today's update was part of a major restructuring we are doing so that we can continue to scale with the tremendous growth we've been experiencing. The changes today required a complete switchover affecting all sites at once. The upgrade itself went smoothly and was completed by around noon today without affecting any service. Unfortunately, about 30 minutes later, our back-end systems went into an overload condition that we failed to anticipate. That has been temporarily resolved while we study the underlying problem and service has been fully restored.

We're very sorry for this. While we've only had a small number of outages to date, we know that any outage is unacceptable and we're working hard to learn from our mistakes. We will continue to strive to maintain 100% uptime in future.