Additionally, we encountered an unrelated issue with a small number of sites on our database cluster using a larger than normal number of long-running database queries, in turn leading to database performance issues more widely across Onyx.
For the LiteSpeed issues, we rolled back to an earlier version to resolve the issues whilst we work with LiteSpeed to resolve this issue in the latest version of the software.
The database issues have been tracked down to a faulty plugin on the sites concerned, and we have worked with the clients to remove this and find alternatives whilst the plugin vendor patches the issue. Additionally, we have taken the opportunity to boost database capacity by bringing online additional clusters, and put the sites with the busiest queries on their own cluster to minimise the risk of this recurring. Our team are also developing additional layers of caching to enhance object and query performance.
28th May 2020
No incidents reported
27th May 2020
Database performance issues
We're aware of an issue with reduced database performance and are working on this now. We'll provide an update shortly.
Edit: 18:49 - Performance has returned to normal levels and we're continuing to monitor. Thanks for your patience.
Slow database queries
We're aware of there being an issue with slow queries on the database cluster, we hope to have this resolved shortly.
Edfit 15:09: This is now resolved and we're continuing to monitor. We're in the process of implementing a long-term fix for this now and apologise for any inconvenience caused.