Issues with Network connectivity in Melbourne Zone

Resolved
{:closed=>"Closed", :complete=>"Complete", :false_alarm=>"False Alarm", :identified=>"Identified", :investigating=>"Investigating", :open=>"Open", :recovering=>"Recovering", :resolved=>"Resolved", :scheduled=>"Scheduled", :underway=>"Underway"}
After 6 days, 1 hour, and 35 minutes

During routine BAU datacentre work to prepare replacement of equipment nearing end of life, a SFP module was removed from said equipment which resulted in an unexpected fatal error taking all services attached offline.

Upon identification of the fatal error, Onsite engineers began to repatch affected services into the replacement infrastructure already installed while offsite engineers migrated customer configurations.

Additional mitigation work was done during change CHG-305 on the 29th of March to fully complete the migration off the nearing EOL infrastructure.

Resolved
After 1 day, 2 hours, and 5 minutes

We've now resolved the incident. Thanks for your patience.

{:closed=>"Closed", :complete=>"Complete", :false_alarm=>"False Alarm", :identified=>"Identified", :investigating=>"Investigating", :open=>"Open", :recovering=>"Recovering", :resolved=>"Resolved", :scheduled=>"Scheduled", :underway=>"Underway"}
After 53 minutes

Technicians have resolved all known issues related to this incident and monitoring to confirm all is OK.
Further details will be posted here once we have them.

Recovering
After 28 minutes

We've fixed the core issue, and are resolving secondary issues. Most connectivity has ben restored except some midband ethernet providers services which are being resolved.

Investigating

We have identified an issue with network connectivity in our Melbourne Zone. Technicians have identified a problem with network hardware and are working on resolving the issue. We do not have an ETA on resolution yet.

Began at:

Affected components
  • AUS - Melbourne
    • Virtual Data Centre (VDC)
    • Virtual Server (VPS)
    • Veeam Backup
    • ImageArchive & VDC Backup
    • Disaster Recovery Replication (VMware, Veeam, Zerto)
    • MyCloud (VMware Cloud Director)
    • Ethernet (Business Fibre)
    • Simtex VoIP and Fax
    • NBN & DSL
    • 4G Wireless Broadband
    • Internet Transit