Increased latency due to database load

Incident Report for Planning Center

Resolved

This week we made a two line, seemingly benign code change that had unexpected performance implications when placed under the high load of Sunday morning traffic. We've reverted the changes and our server loads (specifically our MySQL servers) are back to their typical load levels, resulting in normal response times. Kudos to our development, operations, and support teams for quickly identifying the issue and implementing the fix.

Posted Sep 25, 2016 - 08:33 PDT

Monitoring

The issue has been identified and a fix has been made. We're keeping an eye on things to ensure we're back to 100%.

Posted Sep 25, 2016 - 08:20 PDT

Investigating

We are investigating a slow down in some apps due to increased load on our database.

Posted Sep 25, 2016 - 07:31 PDT