Resolved -
This incident has been resolved.
Jun 8, 19:45 UTC
Monitoring -
We have deployed patches to our database to optimize the problematic queries, and are seeing stability at this point.
Jun 8, 18:18 UTC
Identified -
We have identified several optimizations to prevent lock situations on the database and are in the process of testing and deploying the changes.
Jun 8, 17:13 UTC
Update -
We are continuing to work to optimize performance here. Based on our findings, we expect that this service degradation should only be affecting a small number of customers.
Jun 8, 16:07 UTC
Investigating -
Given a certain set of queries, the Postgres database reader nodes consume a high amount of memory, causing the operating system to terminate the Postgres master process. We're currently working to add reader nodes and optimize the problematic queries.
Jun 8, 15:32 UTC