Is Deft Down Right Now? Check if there is a current outage ongoing.

We've identified that traffic has rerouted successfully, and all packet loss has cleared.

Summit engineers have taken action to reroute traffic from the problematic carrier, and we are observing the packet loss clearing. Local traffic did not appear to be impacted. We are monitoring for continued stability.

Summit engineers believe this was isolated to transpacific traffic via a single carrier. We have taken actions to reroute traffic.

The Summit engineering team is investigating Network packet loss and latency in the Sydney region.

We've identified that traffic has rerouted successfully, and all packet loss has cleared.

Summit engineers have taken action to reroute traffic from the problematic carrier, and we are observing the packet loss clearing. Local traffic did not appear to be impacted. We are monitoring for continued stability.

Summit engineers believe this was isolated to transpacific traffic via a single carrier. We have taken actions to reroute traffic.

The Summit engineering team is investigating Network packet loss and latency in the Sydney region.

Waveserver pair between ORD1 and ORD6 was restored to service after a cold boot of the appliances. Traffic is now being passed properly once again.

Limited degraded network performance for select customers with wave circuit transit between Elk Grove Village (DF-CH1 - Digital Realty Trust CH1), and Chicago 350 Cermak (EQ-CH1).

Microsoft 365 services continue to return to functional status and our email and voice communications are back online. We will monitor these issues throughout the evening in the event of any lingering issues. Thank you for your patience.

We are currently experiencing an outage affecting Microsoft 365 services, including email and voice communications. This disruption may impact our ability to communicate effectively, including ticketing responses. Microsoft has acknowledged the issue and is actively working to resolve it. We will provide updates as more information becomes available.

Deft Engineers have identified an issue with one of our transit providers and are working on shifting traffic off the carrier. They will conduct further investigation with the carrier and provide updates as more information becomes available.

This incident has been resolved.

We have applied a vendor-suggested fix that appears to have mitigated the issues. We will continue to monitor and test to validate resolution.

Engineers are implementing a workaround and will continue to test resolution of the issue.

We have isolated the issue to our Chicago Managed Object Storage environment

Engineers are currently investigating an issue with our Managed Object Storage.

This incident has been resolved.

The incident has been resolved. Please contact Deft if you have any issues.

We are continuing to work on a fix for this issue.

The issue has been identified and a fix is being implemented.

Telia has notified us that service is restored. All services are normalized and optimal at this time..

Telia has notified us that technicians are on site and troubleshooting. ServerCentral will keep our links to Telia disabled until we have received an all clear from them. Next update at 19:00 CDT

Telia has notified us they have isolated at least one fiber cut in the Phoenix AZ area and are dispatching a fiber crew. Next update at 17:00 CDT

Telia has informed us they are having a major outage involving multiple backbone link failures involving a fiber cut. We have disabled our connections to Telia until we receive an all clear from them. Next update at 15:00 CDT

Our monitoring system has detected performance issues inside Telia's network, and we have proactively lowered the preference of their customer routes to mitigate performance issues. Update at 13:00 CDT

Full service has been restored as of 10AM CDT. No additional work expected.

Fiber splicing has been delayed, new ETR is Sunday afternoon.

Fiber crews have located the cut, are on site and actively working. ETR 10:30CDT.

The Upstream Carrier has identified a general location of the cut, Flatonia Texas, and has technicians en route. No ETR is available at this time.

Traffic between SJC and IAD on ServerCentral's Network was automatically rerouted around the affected circuit.

This issue is part of a larger incident due to a fiber cut. Our carrier is attempting to locate the exact position where the damage has occurred.

The ServerCentral Network Operations Center reported an alert of loss of signal on a wave between SJC and IAD.

We have bypassed the affected third party component and mail is flowing normally again. Some previously sent mail will be delayed.

We are currently seeing issues with inbound email to servercentral addresses. Please use portal.servercentral.com or call us at 312 895 3005 for urgent support issues.

Fiber has been repaired and service normal for 12 hours.

A fiber cut has been identified and splicing crews have been dispatched. Next update at 19:00 CDT

Our internal monitoring has noted an issue with Telia in the US Southwest, we are proactively moving traffic away to mitigate any potential issues. Next update at 13:00 CDT

This incident has been resolved.

We are working with our upstream providers on increased latency in AMS3.

This issue has been resolved, all phone services are fully restored.

An issue is currently preventing outside parties from dialing direct DIDs for users at ServerCentral. This is affecting attempts to call individual employees but does not affect our support or main office numbers. Our team is working to restore the DID dialing now for end-users and should have services running again. ETA is 30 minutes.

We have normalized peering with Telia at this time, but at a reduced traffic level. The ServerCentral NOC continues to closely monitor.

Telia reports they are seeing packet loss on routers in New York and Los Angeles at the moment due to a major transmission outage in the US. We will provide additional details when they are available. We are routing around Telia at this time and all connectivity should be optimal.

Due to multiple reports related to performance issues via Telia, we have moved traffic away to mitigate any issues. We are reaching out to the Telia NOC to get further details.

Issue resolved - affected customers have been contacted.

We are experiencing traffic loss in our London pop due to as yet undetermined router issues. Under investigation

An optical component was replaced upstream, traffic has been shifted back and is normalized. No impact was observed.

ServerCentral Network Engineering was alerted to the loss of a single circuit of a redundant pair in SJC. Traffic automatically shifted to the second member.

This incident has been resolved.

All affected links and paths are back online. We are investigating with our vendors.

We are looking into multiple link outage in the Chicago area. We are still investigating at this time.

At approximately 6:50PM Central Time, engineers have resolved the outage and service is restored to the Enterprise Cloud ORD6 zone. If you are still experiencing issues, please open a support case via https://portal.servercentral.com or by emailing support@servercentral.com. Engineers are working to identify the root cause in the meantime.

At approximately 6:30PM Central time, the Enterprise Cloud ORD6 availability zone experienced a partial outage. Engineers are investigating the cause now.

This incident has been resolved.

Network Engineering has located issues on a 1g copper card in slot 11 of ar4.ord6. Affected customers have been alerted. The Card will be restarted and customers on this card will see a loss of redundancy for a few minutes during the card reboot.

ServerCentral Network Engineering has been alerted to an issue on AR3.ORD6 and AR4.ORD6.

The Amsterdam event has been isolated and resolved.

We are seeing a large volume of traffic in Amsterdam and in the process of investigating.

This incident has been resolved.

We have corrected the user associations and continue to review the incident to confirm there are no other problems.

We have discovered an issue which disassociated some customer contacts from their company, limiting access for some customer contacts to our portal. We are working on re-assocating these contacts now.

Our transit issue in London is resolved and has remained stable. Redundancy has since been restored. No further impact is expected.

Update: LHR We are continuing to pursue our upstream transit issues in London. We have reduced redundancy on our backbone at this time.

We're experiencing upstream issues in London. We're actively investigating and working with our upstream providers to resolve this issue.

A routing anomaly occurred at roughly 11:30 CST on our ar3 and ar4 aggregation routers in Chicago ORD6. Customers on these routers may have been affected for a few minutes, routing has normalized.

#Reason for Outage Report **Date of Event:** February 22, 2017<br> **Event Start:** 21:56 CDT<br> **Event End:** 22:23 CDT<br> **Duration:** 27 minutes <br> **Affected Devices:** portal.servercentral.com<br> **Master Ticket:** N/A<br> **Service Impact:** Outage for portal.servercentral.com and customer API access<br> **Root Cause:** Incorrect configuration deployment to switch<br> ##Summary: ServerCentral maintains a web-based portal for interacting with our customers and prospects in order to ensure the highest level of customer service and responsiveness for our staff. Within this system we provide access to our support ticketing backend, billing resources, and customer inventory information via our customer portal (https://portal.servercentral.com) to all known and registered users of our existing customers. Additionally we offer customers access to this data via a Restful API hosted at the same URL provided on an as-needed basis. On February 22, 2017, at approximately 21:56 CDT, the ServerCentral IT department was staging a change to the core switch supporting the primary customer portal system in preparation for a maintenance to be scheduled and announced. The configuration change was in anticipation of a scheduled test of automatic failover capabilities of a security component protecting the customer portal. This upgrade was to be announced via a login message presented to all users and via an email notification to relevant contacts as soon as the details were arranged. During the staging process, the proposed changes were inadvertently pushed into production immediately instead of staged for later review by the relevant staff involved. This was done through human error and is not a normal part of the ServerCentral change management procedures. Approximately 1 minute after the configuration change was made the error was detected by monitoring systems and reported to the ServerCentral IT department. System Administrators accessed the switch and rolled-back the changes once they were detected, restoring connectivity to the primary application servers. Changes were committed at 22:22 CDT and the process completed and access was restored at 22:23 CDT. Based on the assessment of the Compliance Officer and as the real cause of the disruption was identified so quickly and was easily resolved, ServerCentral chose not to pursue the implementation of the business continuity plan for the customer portal or API access. The recovery process would have taken longer than rolling back the switch changes, and as such it was opted to correct the issue with the primary systems instead of implementing the fail-over. ##Resolution: It is now and has always been the policy of ServerCentral to observe the full change management plan maintained by our Security, Risk, and Compliance committee and adopted by all departments. Changes are staged, vetted, approved, and scheduled by internal teams prior to being placed into production using an industry-leading approach to securing processes on our production systems. Moving forward the team managing the systems that run our internal and customer-facing applications will be re-trained on the change management procedure and their work will be observed by more senior staff on an ongoing basis.

Connectivity has been restored by system administrators and all systems are now fully operational.

Network connectivity to the customer portal is currently unavailable, systems administrators are investigating the cause and working swiftly toward a resolution.

A router in our Tokyo pop experienced an unexpected restart. It is now recovered and stable, we are investigating the cause of the restart.

Our Tokyo POP is experience hardware failure. We are currently investigating.

All routing is normalized at this time. Issue resolved.

ServerCentral Engineering has seen network performance issues related to one of our carriers. We have proactively suspended service from them until they send us an all clear. Next update at 12:00 CDT

Degraded hardware has been restored to proper function. No more updates.

We have identified degraded hardware and working to restore.

At shortly after 22:00 CDT some routes from the Elk Grove Village Datacenter, mostly to AWS and Comcast were dropped. Routing has re-converged, and we are expecting no further issue

Services temporarily degraded in European region due to long haul Transit issue. Issue occurred between Jul 26 00:50 and Jul 26 00:56 UTC Service has since been restored.

A router in IAD experienced a router anomaly and recovered as expected. This resulted in a line card restart. Single threaded customers experienced a brief drop in connectivity as the line card restarted.

ServerCentral personnel are investigating possible network issues in the IAD network. More details to be posted as they are available.

One of our redundant core routers in Ashburn, VA experienced an issue between 01:24AM and 01:44AM CDT. Redundant services failed over expected. Single homed customers may have experienced a brief loss of connectivity as the router recovered.

ServerCentral personnel are investigating a network issues in Ashburn, VA. More details to be posted as they are available.

We are declaring this situation resolved.

A core router in Amsterdam has experienced an un-expected routing restart. It has recovered and we are working with out vendor to identify the cause and mitigate against future occurrences.

The Network Software event is resolved. One router was affected resulting in an outage for single homed customers. We apologize for the inconvenience this may have caused.

We are currently investigating this issue.

The Network event is resolved. One router was affected resulting in an outage for single homed customers. We apologize for the inconvenience this may have caused.

We are investigating a network event in Tokyo, Japan. Senior network engineering staff are currently working to isolate and resolve the issue.

The incident has been resolved at approximately 6:05PM CDT. We apologize for the inconvenience this may have caused.

Our engineers have identified the issue and are working to bring the environment back online.

Our engineers are still currently investigating the interruption of service.

At approximately 4:50PM Central time, the Enterprise Cloud ORD6 availability zone experienced a partial outage. Engineers are investigating the cause now.

Level3 has stopped leaking Comcast routes to NTT.

ServerCentral is now blocking Comcast routes that were leaked from Level3 to NTT. Level3 is still leaking these routes, which may be causing congestion on Level3's network.

ServerCentral has removed the affected routes from this issue, and traffic should begin flowing normally to Comcast at this time.

ServerCentral has identified an issue with Comcast, and is working with them to resolve. We are working to reroute the affected traffic.

We are investigating reports of latency on our Network

We are continuing our follow up investigation with our vendor, but expect no further impact at this time.

We saw the routing process on one of our two Core Routers in 350 Cermak bounce at 13:25 UTC today. We are investigating the event with our vendor.

Efforts continue in working with our vendor to determine the root cause of the incident.

Senior Network Engineering continues to monitor the situation closely.

We are placing the affected devices under a configuration freeze until such time as our consultation with our vendor reaches a positive conclusion.

We saw the routing process on one of our two Core Routers in 350 Cermak bounce at 23:56 UTC today. We are investigating the event with our vendor.

Repairs are complete on our DWDM system, and we are back to normal.

ServerCentral has disabled a segment of our DWDM system on riser fiber between Floors 1 and 5 at 350 E Cermak. No customer capacity has been affected, but we have lost some redundancy within the building. We expect to have it repaired shortly.

Comcast has implemented a temporary solution as they deal with multiple fiber cuts on their backbone

The ServerCentral NOC monitoring systems have detected issues within Comcast's network. We are working to minimize the impact to our customers.

This issue has been resolved. All services are now operational.

We have resolved the issue and all system functionality is being restored.

We have Identified the issue and are actively working to resolve it.

ServerCentral Staff are currently examining an issue related to Veeam Cloud Connect and Backup Repositories located in our Elk Grove Village Datacenter. Further details will be posted to status.servercentral.com.

This issue has been deemed resolved by our Managed Services team and all functionality is operating normally.

Services have been restored and are resuming normal activity. We are continuing to monitor the platform to ensure stability.

We have identified the issue and are working to restore service.

We are investigating reports of an issue relating to issues connecting to the Veeam Cloud Connect repository in our Elk Grove Village area datacenter.

This incident has been resolved.

Services have been restored and are resuming normal activity. We are continuing to monitor the platform to ensure stability.

We have identified the issue and are working to restore service.

We are investigating reports of an issue relating to connecting to the Veeam Cloud Connect repository in our Elk Grove Village area data center.

This incident has been resolved.

We have identified the issue and are working to restore service.

We are investigating reports of an issue relating to connecting to the Veeam Cloud Connect repository in our Elk Grove Village area data center.

This issue was narrowed down to an incorrect configuration applied earlier this evening and has been resolved.

ServerCentral Turing Group is exploring a higher than usual amount of bandwidth entering our network. Some customers may experience slower than normal network response times. We are currently investigating this issue.

Storage latency has returned to normal levels. Enterprise Cloud is operating normally at this time.

We are continuing to investigate this issue.

ServerCentral Managed Services staff are investigating reports of increased storage latency affected Enterprise Cloud ORD.

We've moved back to our primary provider.

Calls to our main IVR and NOC have been rerouted to an alternate phone provider while we troubleshoot the issue with our primary vendor.

Everything continues to be stable. We will follow up with vendor but expect no further issues.

SCTG Network Operations have identified a brief interruption to one of our redundant dark fiber paths between 9:29 AM CDT and 9:42 AM CDT. All services are restored. We are monitoring and pursuing this with the vendor.

CenturyLink has advised that they are taking corrective action and expect things to be clearing soon if not already. For our part, we are leaving them in a non-preferred state for the time being.

A fix has been implemented and we are monitoring the results.

CenturyLink have advised they have identified the issue and are taking corrective measures.

SCTG NOC has monitored trouble with the CenturyLink / Level 3 (AS3356) network. We have taken steps to mitigate but there are still issues remaining due to the size and scope of their network.

Looking Glass software was replaced with updated

This incident has been resolved.

We are continuing to monitor for any further issues.

We are continuing to monitor for any further issues.

Currently our Looking glass, https://lg.deft.com is down for maintenance

This incident has been resolved.

Service continues to be stable at this time. Deft Network Engineering will re-enable the circuit in question after business hours and continue to monitor.

Deft's 24/7 NOC has monitored an event with one of our upstream IP Transit circuits. Customers may have noticed a brief interruption in a portion of their internet traffic. We have disabled this circuit for the time being to mitigate any further impact.