Perth Primary Storage Disruption

Resolved

At approximately 22:35 AWST, Zettagrid engineers were alerted to a primary storage controller failure in the Perth zone.

Indications at the time suggested that the failover between controllers occurred and as expected workloads were running on the redundant controller without impact to customer workloads.

We have since had a small number of reports that workloads sensitive to storage latency may have been impacted by this failover.

We have been working with the storage vendor to identify the root cause which has indicated the disruption to be related to a bug in the storage controller software which under very specific circumstances can cause a controller to become unresponsive and reboot. Zettagrid engineers are working with the vendor to install the latest release which addresses this issue.

We do not expect any service disruptions as a result of the software upgrade, however, all related works will be posted to the status page. You may wish to subscribe to alerts for these upgrades, once posted, or all future works by clicking 'Subscribe To Updates'.

If you do have any VMs in Perth zone which are not performing as expected, you may need to initiate a reboot of that VM in the first instance. If you continue to experience issues, please contact the Operations Centre on support@zettagrid.com or 1300 597 656.

Began at: