QuadraNet

Is QuadraNet Down Right Now? Check if there is a current outage ongoing.

QuadraNet is currently Operational

Last checked from QuadraNet's official status page

Historical record of incidents for QuadraNet

Report: "DTLA Maintenance"

Last update
resolved

This incident has been resolved.

identified

Dear Customers, Thank you for your continued patience as we work through our maintenance in our Los Angeles facilities. The issues have affected multiple services at both locations. We are working 24/7 to resolve this, but as expected this is not enough to resolve all. Once your services are restored for our dedicated server customers only, please submit for an SLA credit via the Neo portal. For further escalations, you can contact our CEO and Owner Jon Eaves, directly at jon.eaves@quadranet.com. We apologize for the extended duration and any downtime you have experienced.

identified

The maintenance window has taken longer than expected, but we have a team on-site working diligently to get every customer back up and running. Please refrain from opening multiple tickets or calling into the data center. Our team is doing everything they can to assist each customer as quickly as possible.

identified

Our team is making excellent progress in getting all bare metal customers back online. Some single colocation customers are still experiencing issues, and we have all hands on deck to restore service for everyone affected as quickly as possible.

identified

The recovery operations still in progress, more detailed report shall be rendered by the end of the event. We appreciate your patience.

identified

Our team has identified some connectivity issues relating to our maintenance being performed today. We are working to resolve this as quickly as possible.

monitoring

We appreciate your continued patience while we work though this maintenance window. We hope to have everyone back online as quickly as possible.

monitoring

Thank you for your continued patience while we work through this maintenance window. Our team has run into a few challenges that were not a planned part of the maintenance. We appreciate your business and understanding.

monitoring

We are still undergoing our network and cabinet restructuring at our Los Angeles facility and will provide additional updates as they become available.

monitoring

Please note, we are restricting all non critical access to the data center during this time.

monitoring

We are currently undergoing our first round of maintenance at our DTLA facility. We are half way through this process and customers should start to regain access in waves shortly.

Report: "Chicago Network Outage"

Last update
resolved

This incident has been resolved.

monitoring

Our team has been able to restore service connectivity to 90% of effected customers. We hope to have this finalized by end of day.

identified

Thank you for your continued patience. Our on-site team has replaced a core device. Our networking team will begin modifying the configuration and get service restored shortly.

identified

We appreciate your patience during the downtime in Chicago. We are in constant communication with our on-site partners and are urgently working to restore service as quickly as possible.

identified

We have identified an issue with our Core Node around 3PM PST today and are actively working with our team to get this back online. We apologize for the delay.

Report: "Urgent Update: Service Disruption in LAX Facilities Due to Cogent Network Issue"

Last update
postmortem

Cogent was having memory issues in their Core Routers at the Data Center and an upgrade was completed to resolve the issue. This issue has now been resolved.

resolved

This incident has been resolved.

identified

There is currently an ongoing connectivity issues impacting our LAX facilities, which we’ve traced to a critical network disruption involving Cogent Communications. Please accept our sincere apologies for the inconvenience this has caused. Current Status: Our engineering team is working closely with Cogent Communication to resolve the disruption affecting both of our Los Angeles peering points. We’ve escalated the issue to Cogent’s senior technical team and are collaborating closely with their engineers to expedite resolution. At this time, Cogent has confirmed the root cause and is prioritizing repairs. Next Steps: We will provide a detailed update as soon as possible. We deeply regret any inconvenience this may have caused and appreciate your patience.

Report: "WHMCS access situation"

Last update
resolved

This incident has been resolved.

identified

Dear Customers, For those amongst our clients who are using the WHMCS part of our CRM. For the time being as a temporary solution, please use our legacy NEO portal for any helpdesk requirements and service communications via https://neo.quadranet.com, if you don't have the logins to NEO, you may simply email to support@quadranet.com. We will keep you up to date once WHMCS issue has been rectified. Apologies for the inconvenience caused, QuadraNet Team

Report: "Networking Related Issue"

Last update
resolved

This incident has been resolved.

investigating

We are currently experiencing a networking issue. This is being worked on by our networking engineers, once resolved a RFO will be provided.

Report: "Dallas Outage"

Last update
resolved

This incident has been resolved.

monitoring

Our team has successfully replaced the Aggregation Switch, and services are gradually coming back online. If you are still experiencing any network-related issues, please open a ticket in the support portal.

identified

We are currently addressing a power issue with our Aggregation Switch. To expedite the resolution, we have brought in additional network engineers to our data center. Our team is working diligently, and we are confident that service will be restored as quickly as possible.

identified

We are currently experiencing a power issue with our Aggregation Switch. Our team is fully engaged, both locally and remotely, to resolve this issue as quickly as possible. We are confident that service will be restored within the next 12 hours.

identified

We are experiencing a power issue with our Aggregation Switch. Our team is actively working on a solution to restore service as quickly as possible. At this time, we do not have an estimated time of resolution.

Report: "LAX100 DWDM Ring (Partial Outage)"

Last update
resolved

We are in the process of transitioning the remaining individual channels for specific customer's to our 530 W. DWDM side of the ring at this time. Ongoing communication for those customer's will be conducted via QuadraNet's support portal as the broader issue from this incident has been repaired. https://neo.quadranet.com 1-888-5-QUADRA

monitoring

Please find an update below: Our fiber engineering team has identified damage to the fiber and is dispatching a repair crew to repair the damage. We will provide updates as they become available. Regards, Crown Castle Fiber NOC

monitoring

Connectivity has been restored by isolating the impacted DWDM path. We are still working to restore a small amount of customers that utilize direct DWDM channels on our 624 S Grand Multiplexer, however.

investigating

We are currently experiencing an outage on one of our DWDM legs that connect between 624 S Grand and 6171 W Century BLVD. Issue is being investigated now.

Report: "Miami Network Impact"

Last update
resolved

We are pleased to inform you that connectivity has been fully restored. Our network team is in the process of compiling a detailed incident report, which will be available for customers upon request. If you require this report, please submit a formal request through the NEO portal. Thank you for your patience and understanding.

monitoring

We experienced a brief recovery, but the network remains unstable. Our teams are fully engaged and working diligently to resolve the issue. We will continue to monitor the situation closely and provide updates as we make progress. Thank you for your patience, and we apologize for any inconvenience this may cause.

monitoring

The network is back to normal, currently being monitored.

identified

We are aware of ongoing network interruptions affecting our Miami datacenter. Our team is actively investigating the issue and working diligently to restore full service as soon as possible. We appreciate your patience and will provide further updates as progress is made. Thank you for your understanding.

Report: "Chicago Network Impact"

Last update
resolved

Root Cause: NTT upstream was down causing a routing issue leading to a short outage. Status: Network is back to normal state. Next Step: Our team is monitoring the situation and inspecting the network to ensure this issue doesn't occur again.

identified

We are aware of the ongoing network issue in our Chicago facility. Our senior network architect is actively working with the onsite team to isolate and resolve the problem. Further updates will be provided as progress is made. Thank you for your patience.

Report: "[DFW] Restoration Complete - Monitoring"

Last update
resolved

This incident has been resolved.

monitoring

We have made repairs to our critical network infrastructure in order to restore primary public network connectivity to downstream customers, we are still monitoring the situation at this time so this case will remain open until further notice. If you are still experiencing public traffic connectivity difficulties, please open a support ticket with QuadraNet at https://neo.quadranet.com

monitoring

A fix has been implemented and we are monitoring the results.

identified

QuadraNet's CORE router has failed to energize after the abrupt power failure in the Dallas Datacenter. We are currently working on a solution at this time.

identified

Power has been restored in the Dallas datacenter and we have partially been restored. QuadraNet is continuing to work with onsite engineers to resolve any outstanding issues. Thank you.

Report: "[DFW] Utility Issues / Heat Issues"

Last update
resolved

This incident has been resolved.

identified

We are continuing to work on a fix for this issue.

identified

At approximately 11:00 AM CT, The DAL data center proactively transferred to generator power on a ATS3 due of excessive temperatures on one of the upstream utility transformers. The transfer was successful. After further investigation of the transformer, insulation damage showing exposed electrical conductors was discovered and Oncor Electric, our upstream utility provider disconnected the utility feed to enable further investigation of the transformer. At approximately 1:58 PM CT, the generator servicing ATS 3 had a fan belt failure. As a result, any non-redundant, single corded critical loads associated with UPS #10 and #12 ran out of battery reserve at approximately 2:24 PM CT. Generator and Electrical vendors are actively engaged. A spare generator fan belt is on site and the generator vendor is currently working to replace the failed component. MTTR on the belt replacement is approximately 3:30 PM CT. Due to the utility and subsequent generator failure, temperatures in the data center will increase. The local team is currently working to re-feed HVAC units, associated with ATS 3, from an alternate power source within the facility. The HVAC refeed process should be complete by 3:45 PM CT.

Report: "NEO Portal Availability Notice"

Last update
resolved

Update: NEO portal issue solved. Full functionality has been restored. Thank you for your patience!

monitoring

The NEO portal is currently experiencing partial outages. While some features may be operational, others are not functioning properly. We are aware of this issue and are actively working to isolate and restore full functionality. If you encounter any difficulties performing urgent actions, such as a remote OS reinstallation, please contact us directly for assistance. Should you be unable to file a request via NEO, you may call our 24x7 NOC at 1-888-578-2372. We will continue to provide updates on this page as we make progress. Thank you for your patience and understanding.

Report: "Seattle Network Impact"

Last update
resolved

This incident has been resolved.

investigating

Cogent network has been restored. Everything should be up now. Root cause: Cogent had a planned network maintenance which wasn't supposed to have any impact. Unfortunately - an unexpected impact has been occurred causing a total of two hours outage. We sincerely apologize for the inconvenience and thank you for your patience during this incident. Should you still experience any issues, please raise a ticket or call our support center. Thank you.

investigating

Our network team is working to activate a backup link while Cogent is offline. Update shall follow.

investigating

Currently, our network team has observed a network impact within our Seattle facility, could cause some parts of our Seattle network to be unreachable. This is being investigated and an update shall follow.

Report: "[EWR] - Network Outage"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are currently investigating a potential network issue that is impacting our New Jersey (EWR) datacenter and we are working towards a resolution.

Report: "Network Interruption in Miami"

Last update
resolved

Issue: We experienced connectivity issues due to a TeleFornia upstream issue. Resolution: The issue has been resolved. All traffic has been successfully rerouted. Outage Period: From 1:30 AM PST until 2:50 AM PST. Status: Resolved.

investigating

Update: The initial root cause appears to be stemming from the upstream provider. Our team is actively investigating the matter. We appreciate your patience and understanding.

investigating

Currently observed network interruption across several sections of Miami network, our engineers working on this and more details to follow. Thank you.

Report: "Emergency Maintenance to QuadraNet Portal Services"

Last update
resolved

Completed.

investigating

Maintenance completed. Portals are back to normal. Thank you.

investigating

Customer fronting portals are currently offline for emergency maintenance. We apologies for any inconvenience caused. Thank you.

Report: "Seattle Partial Impact"

Last update
resolved

Update: Status: Network is now recovered to normal state. Outage Period: 2:15 AM - 2:45 AM PST / 30 minutes of partial network outage (on/off) Root Cause: Core router degraded performance (was very slowly responding to any input). Solution: A hard reboot of the router was needed. -------------------- Next Step: Network team will be monitoring the network performance to ensure stability and take any actions needed if any further anomalies to be detected. -------------------- We apologize for any inconvenience this outage may have caused. Thank you for your patience. Thank you.

investigating

Our engineers are currently investigating an issue that is causing partial network outages in our Seattle location. The root cause appears to be an anomaly at the core router level. We apologize for any inconvenience this may be causing. Our team is working diligently to identify the problem and implement a fix as quickly as possible. We will provide further updates here on the status page as we have more information to share. Thank you for your patience while we work to restore full network connectivity in Seattle.

Report: "Network Outage in Seattle"

Last update
resolved

The total outage time was from 0h20 AM to 3h AM (PST). The network has been confirmed as stable for now. Our team is still investigating to get the complete details on the root cause. We will provide further updates once we have finalised the investigation. Thank you for your patience and understanding during this outage. Best regards, The QuadraNet Team

investigating

We are pleased to report that the network has returned to normal operation since the last 15 minutes. Our team is still investigating with Seattle to get the complete details on the root cause. Thank you for your patience and understanding during this outage. We will provide further updates as more information becomes available. Best regards, The QuadraNet Team

investigating

We are currently experiencing a network outage in our Seattle location. Our network team is actively investigating the issue and working to resolve it as quickly as possible. We apologize for any inconvenience this may cause and will provide updates as more information becomes available. Thank you for your patience and understanding.

Report: "Network Stability Impact in Seattle"

Last update
resolved

The issue has been resolved and our network is now functioning normally. Our network team is currently working with Seattle/Cogent to determine the underlying cause of the unexpected outage. Duration of Total Outage: 1 hour (from 4:13 AM to 5:15 AM) We sincerely apologize for any inconvenience caused and appreciate your patience and understanding during this time. Thank you for your cooperation!

investigating

We are continuing to investigate this issue.

investigating

We are writing to inform you that our Seattle location has been experiencing network stability issues for the past 30 minutes. Initial observations indicate that the issue is related to a recent upgrade that was performed by one of our bandwidth providers, Cogent. Cogent was upgrading one of the route processors on their core and aggregation routers, as recommended by the manufacturer. This work was supposed to begin at 9:00 AM UTC on June 22, 2023, and was not expected to impact our customers. However, we are now experiencing some network instability as a result of this upgrade. Our network engineers are currently working with the Seattle and Cogent teams to isolate the issue and restore full service as soon as possible. We apologize for any inconvenience this may cause. We will provide updates as they become available. Thank you for your patience.

Report: "Chicago Minor Impact, Partial Packet Loss."

Last update
postmortem

Faulty cable connected the switch caused packet loss. Cable replacement fixed the problem. Total partial packet loss time: 3 Hours Absolute outage time: 55 Minutes

resolved

Faulty cable replaced. No further partial packet loss can be observed. We will continue to monitor to ensure the link is healthy. Thank you for your patience during this incident. Incident status: Resolved.

investigating

Certain devices may experience a complete packet loss as our engineers replacing a faulty cable at this very moment. We will report back once replacement is accomplished.

investigating

Update: Our network engineers are still working on the incident. We would like to clarify that the network is not down, it is just a minor section of our Chicago network with a partial packet loss.

investigating

We have identified a small section of the Chicago network with unstable connectivity, which is causing partial packet loss. Network engineers are working to isolate the issue and restore full connectivity. Updates will follow.

Report: "Chicago Network Stability"

Last update
resolved

Issue: Partial packet loss in Chicago Date: 2023-05-18 Time: 10:00 PM - 3:15 AM PST Total Impact: 5h:15 minutes Root Cause: Distributed Denial of Service (DDoS) attack Course of Action: Mitigated the DDoS attack. Action Plan: An internal meeting will be held between network engineers and the facility CTO to discuss the impact of the DDoS attack and enhance the mitigation process to minimize future impacts. Thank you for your patience during this time. We appreciate your business. If you have any questions or concerns, please do not hesitate to contact us.

investigating

Our network team is still working on resolving the partial packet loss issue. More updates to follow. Thank you.

investigating

Our monitoring has detected network stability issues causing partial packet loss across certain sections of our Chicago network. Our network engineers already investigating the anomaly and further updates shall follow.

Report: "Miami: Partial packet loss on TELEFONICA upstream"

Last update
resolved

Incident: Partial packet loss on TELEFONICA upstream in Miami Date: 2023-04-28 Time: 0:55 AM PST Resolution: Resolved at: 1:25 AM PST Total Impact: Approx. 30 Minutes Impact: - Partial packet loss on outbound traffic for clients routed on TELEFONICA upstream - No impact on inbound traffic - No impact on clients routed on other upstreams Details: We experienced partial packet loss on our TELEFONICA upstream for about 30 minutes, from 0:55 AM PST to 1:25 AM PST. We quickly identified and resolved the issue, and all services are now restored. We apologize for any inconvenience this may have caused. As always, we are committed to providing our customers with a reliable and stable network. Next Steps: We will continue to monitor our network and will update this status page as needed.

monitoring

Impact: The partial packet loss caused some disruption to our services. However, we were able to quickly resolve the issue and restore full service to our customers. Resolution: We deactivated the problematic upstream and are currently working with our upstream partner to resolve the issue. We will continue to monitor the stability of our network and will update this report as needed.

investigating

Currently investigating possible anomalies within some sections of our Miami network. Network stability issues being reported and within an on going investigation. Update shall follow.

Report: "AMS Network Outage"

Last update
resolved

This incident has been resolved.

identified

We are currently experiencing network connectivity disruptions in our Amsterdam datacenter and we are working to identify and resolve the problem.

Report: "LAX13 - UPS2 Power Incident"

Last update
resolved

The issue has been identified as a faulty power supply unit located within LAX13-Z2.UPS-2 during the power transfer between sources. Currently the critical power vendor has replaced the faulty power supply unit and have normalized the unit back to UPS protected power at this time. Please open a ticket regarding this incident should you require additional information.

monitoring

QuadraNet has identified a customer impacting power incident located in our LAX13 datacenter pod related to our UPS-2 equipment. Power has currently been normalized at this time and we are working with impacted customers via our ticketing system in order to resolve any issues that are currently outstanding for equipment. Please utilize the QuadraNet NEO control panel to open a ticket located at: https://neo.quadranet.com or email support@quadranet.com for additional assistance. We currently have a critical power vendor located onsite working to determine the root cause of this incident and additional information will be provided once it is available. Regards, QuadraNet Management

Report: "LAX9 - UPS2"

Last update
resolved

The latest update was sent in error. The work for the UPS-2 repair has been completed in full and all customers have been returned to UPS protected power at this time. Thank you and please let us know if you need any further assistance.

identified

The defective unit has been full removed from the site and we are still operating in maintenance bypass at this time. Estimated time to complete the work has been updated to: Saturday June 25th, 2022: 3:00PM PST. Please open a support ticket if you have any questions.

identified

Critical Power Vendor has arrived at the datacenter to begin the removal of the defective UPS unit. No customer impact is expected as a result of this work at this time. We will provide updates as they become available.

identified

Due to a defective Power Module in LAX9 UPS2, related to the critical power even that happened last night (6/22/2022). We have currently isolated this unit and removed it from our critical power topology. Any customer equipment connected (without redundancy) to LAX9-UPS2 is currently operating in an unprotected state. QuadraNet has secured a replacement unit expeditiously and we anticipate to have this incident resolve and the replacement unit installed as follows: Friday June 24th, 2022: Critical Power Vendor will arrive onsite to begin the process of removing the defective UPS unit. Friday June 24th, 2022: Critical Power Vendor will deliver the new unit onsite to QuadraNet's LAX9 datacenter zone and begin assembly. We anticipate all work to be completed by 11PM PST on 6/24/2022 - however, if this timeline changes we will provide updates. We have been assured by our vendor that all work will be completed by no later than 11PM PST on 6/25/2022. Thank you,

Report: "LAX100 Chiller Status"

Last update
resolved

This incident has been resolved.

identified

We are currently experiencing a fault with our chiller system and we have engaged our chiller vendor to perform an emergency dispatch to rectify the issue. We anticipate the vendors will arrive at approximately 6:30PM PST

Report: "Support Portal Maintenance"

Last update
resolved

This incident has been resolved.

investigating

We are currently performing ongoing work on our support portal. You may notice times of inaccessibility during this work.

Report: "[ATL01] Power Outage"

Last update
resolved

All repairs have been completed to our edge router and traffic has been gracefully shifted back to it without service interruption.

monitoring

At this time we have restored network connectivity to our environment(s) and some customers have also been restored. At this time we will begin the recovery of customers that are not currently restored. If you have an open (pending) ticket regarding this incident, we will review your request and work to restore your service (if not already restored). The status of this situation is being changed at this time as QuadraNet has successfully deployed our disaster recovery playbook and we will continue efforts to restore customers who are still impacted by the original power failure.

identified

QuadraNet has not been successful at this time in restoring our edge routing equipment from the power failure. At this time we will be exercising all available options in our disaster recovery plan in order to restore service to our customers. Updates will be provided once they are available.

identified

At this time we have not been successful in restoring our routing equipment and we are still troubleshooting this with the datacenter engineers.

identified

At this time QuadraNet network engineers are working with local engineers to restore service to our routing equipment. Due to the hard power disconnect, it has resulted in the failure of several components in our edge routing equipment in this datacenter. We will provide updates as they become available.

identified

Engineers have been provided access to the building after being cleared by the Atlanta Fire Department. We are currently working directly with the engineers onsite to restore service.

identified

The datacenter has informed us of the following: "A contractor working on the 2nd floor of the 55 Marietta building cut into a water main and flooded the 2nd and 1st level of the building. The flooding extended to the generator room which took down the datacenter generator and ATS (Automatic Transfer Switch). The Atlanta Fire Department is currently on the scene and is not allowing access to the building until it has been cleared for safety." QuadraNet is currently working diligently with our datacenter partner in the Atlanta location to restore service. We will provide additional updates as they become available. Thanks,

identified

The issue has been identified and a fix is being implemented.

investigating

The datacenter at 55 Marietta has informed us of a power event that has impacted the entire datacenter floor. We are currently awaiting updates from the datacenter engineers.

Report: "BR2.LAX Emergency reboot"

Last update
resolved

issue is now resolved

monitoring

Patching completed, traffic switched over to BR2 and we are monitoring.

identified

All traffic moved to BR1 successfully with no service interruption, Patches are being applied on BR2

identified

Our network team identified an issue with BR2.LAX100 ad require an emergency reboot to apply patches. network degradation is expected for under 5 minutes

Report: "LAX DC - 530 - Network Degradation"

Last update
resolved

This incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

Network Engineers have retracted announcements to our upstreams for the impacted prefixes at this time in order to restore network stability. We are currently monitoring the situation and will make any necessary adjustments as required.

identified

The issue has been identified and a fix is being implemented.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We are currently tracking a large network attack in our Los Angeles datacenter that is impacting network performance. Network Engineers are working diligently to restore service to impacted customers.

Report: "VEST DDoS Emergency Maintenance"

Last update
resolved

This incident has been resolved.

identified

Restoration time updated to: 7:30PM PST

identified

QuadraNet has placed our VEST DDoS protection offering under emergency maintenance. During this maintenance -- all protection will be disabled and null routes will be applied to network attacks, rather then the VEST DDoS filtering service. We anticipate service restoration by: 6:30PM PST (Subject to change)

Report: "Fiber Cut"

Last update
resolved

ZAYO has indicated that they will have their fiber cut repaired by potentially 2PM PST 3/13/2021 - QuadraNet at this time, has rerouted and migrated the impacted uplinks in order to restore service to impacted customers. Thanks,

identified

The issue has been identified and a fix is being implemented.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

Zayo dark fiber cut 20 miles away from facility, this issue effected only customer with Shared VLAN's across LAX100 <>DTLA facilities. We are working on a solution to remedy the situation via our redundant fibers.

Report: "LAX9 - Network Maintenance"

Last update
resolved

This incident has been resolved.

investigating

We are performing an emergency maintenance in order to rollback a portion of the changes that were performed on Saturday (December 19th, 2020) You should expect partial degradation of service while traffic reshifts to the designated uplinks

Report: "LAX9 - Network Maintenance"

Last update
resolved

We are currently investigating this issue.

Report: "LAX13 (DTLA - 530 W) Network Outage"

Last update
resolved

This incident has been resolved.

identified

The issue has been identified and a fix is being implemented.

investigating

We are currently investigating this issue.

Report: "[LAX - 530 W 6th ST] Onsite Visits Temporarily Restricted"

Last update
resolved

This incident has been resolved.

monitoring

Hello, In an effort to protect QuadraNet staff and our customer(s); At this time QuadraNet is restricting all onsite visits to our 530 W. 6th ST datacenter except for QuadraNet personnel due to on-going protests in the downtown area which may pose a risk. Any customer wishing to visit our datacenter should submit a request to support@quadranet.com instead. Thank you,

Report: "[*] - Network Performance"

Last update
resolved

This incident has been resolved.

investigating

We are continuing to investigate this issue.

investigating

All system are still stable and running. if this continue we will close this incident ticket within the next 16 hours.

investigating

JTAC team discovered a new JunOS bug, patch applied VIA ISSU. Quadranet Engineers continue to monitor and work with the JTAC team. all systems are FULLY operational and we are keeping this incident report running for at least 24 hours to better updates our client.

investigating

Issue is Isolated to our Edge BR1 Router, we are working with the juniper JTAC team to isolate and solve the issue in hand. Current status is fully operational but we are monitoring the situation.

investigating

We are continuing to investigate this issue.

investigating

We have detected a network performance issue in our Downtown Los Angeles facility. Engineers have been notified and are currently working to identify the issue. At present, the issue appears to be isolated to Downtown Los Angeles. Customers with redundant uplinks may have not been impacted.

Report: "Amsterdam Degraded network"

Last update
resolved

Networking performance has been restored. This was the result of a misconfiguration on our upstream's router that resulted in packets being dropped sparsely. If you are experiencing any further difficulties - please email support@quadranet.com

identified

We are currently experiencing degraded network performance in our Amsterdam location. We are working to restore stable service at this time.

Report: "[Degraded Networking] New Jersey"

Last update
resolved

Cogent has repaired the fiber cut and we have re-enabled our BGP sessions with AS174. Thank you for your patience.

identified

Cogent Technicians are en-route to 110 B Meadowlands to investigate the outage. We are currently awaiting updates.

identified

Cogent's status page has been updated to reflect that this outage is affecting 110 B Meadowlands Pkwy, Secaucus,NJ. There is still currently no ETR at this time for repairs.

identified

This has been identified as a fiber cut on Cogent's side in NYC. They are currently working to repair the cut and have no ETR at this time. We are continuing to work with them to determine the ETR for service restoration. Updates will follow.

investigating

We are currently investigating degraded networking performance in New Jersey. Currently it appears that Cogent has had an outage upstream and we have retracted our routes with them until this issue has been resolved. We are continuing to work on this issue.

Report: "Miami - Power Disruption"

Last update
resolved

This incident has been resolved.

identified

Dear QuadraNet Customer(s), EMERGENCY UPS MAINTENANCE - SERVICE IMPACTING Location: 36 NE 2nd ST Miami, FL 33132 Date: Saturday, June 8th 2019 Start Time: 4:30PM PDT End Time: 5:30PM PDT Expected Outage: 10 minutes Risk: Critical load will be shut down and transferred to another UPS. Description: QuadraNet critical power vendors will be transferring the critical load out of wrap around bypass back to protected load on a secondary standby UPS unit. QuadraNet engineers are on standby to assist with bringing up equipment and assisting customers after the cut over. QuadraNet recommends that all customers power off their equipment during this maintenance window to avoid any data corruption. We will place a 20 minute notice on http://status.quadranet.com before the electricians transfer power back to protected load If you have any questions regarding the maintenance at any point, please contact our support department at support@quadranet.com or 1-888-5-QUADRA. We appreciate your understanding and patience during this maintenance and welcome all feedback. Thank You, Management Team QuadraNet Enterprises, LLC. www.QuadraNet.com 1-888-5-QUADRA support / sales @quadranet.com

identified

We are continuing to work on a fix for this issue.

identified

We have been able to temporarily restore service(s) until the electrical contractors arrive onsite. We are running in maintenance bypass at this time. Updates to follow.

identified

A faulty input breaker has been detected for UPS-2. Electrical contractors are en route to the data center to transfer critical load to UPS-4 (Standby UPS). Current estimated time for electrical contractors to arrive onsite is 45 minutes.

investigating

We have detected a power disruption in our Miami facility. Engineers have been notified and are currently working to identify the issue. At present, power service to affected customers' equipment is offline, and the issue appears to be isolated to A powered cabinets. Redundant power service has not been impacted.

Report: "[New Jersey] Network Performance Degradation"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

Network Engineers have traced the issue and have been able to repair it. A final reboot will commence now to fully restore the networking functionality in this location. CONNECTIVITY WILL BE INTERRUPTED DURING THIS EVENT.

identified

Network Engineers have encountered an error with the firmware that was booted - a rollback has been commenced.

identified

The EWR-CORE01 router is being upgraded now - internet connectivity will be disrupted during this event.

identified

A firmware update is being pushed to the ERW-CORE01 to rectify the issue. More information to follow.

investigating

Network Engineers are currently investigating a network performance issue within our New Jersey facility that is interrupting network traffic for a handful of customers. More information to follow soon.

Report: "ORD-C1 to ORD-CH3 [Migration]"

Last update
resolved

All devices are reporting online at this time and we are marking this incident as resolved. Please open up a ticket with us at support@quadranet.com.

monitoring

All cabinets are energized and our team is working though a small handfull of devices that are still reporting down.

identified

Hello, We are currently going to go over our maintenance window slightly - the final two cabinets are being cabled at this time and energized. We should have all equipment back online very soon.

Report: "Dallas Network Maintenance [Rev - 3]"

Last update
resolved

This incident has been resolved.

identified

All customers have been restored at this time.

identified

CORE01 has been re cabled and sessions are beginning to restore. Customer access is being restored at this time.

identified

Original CORE01 has been reracked into cabinet and blades are being inserted at this time. Further updates to follow.

identified

The network topology is being reverted at this time due to extenuating circumstances with the limitations imposed on one of the new routing appliances. Further information will be provided once this has been completed.

identified

An additional supplemental network appliance is being configuring now to offload routes from the impacted appliance. Network engineers are currently still working to remedy this issue.

identified

The issue has been tracked back to a limitation on one of the routing appliances installed. The amount of IPv6 routes in the system have reached a hard limitation, which is resulting in some IPv4 routes being improperly managed by the appliance. Network engineers are currently reducing the amount of IPv6 routes in order to restore IPv4 functionality for all customers.

identified

The issue has been identified and a fix is being implemented.

investigating

There are currently a small handful of devices still impacted due to a small announcement issue on one of our subnets. Network engineers are working to repair this - all other customers should have restored connectivity at this time, including Infracloud users.

Report: "DDoS Mitigation Outage"

Last update
resolved

This incident has been resolved.

investigating

We are currently investigating this issue.

Report: "Control Panel Upgrade"

Last update
resolved

This issue is now resolved, the upgrade is complete. Apologies for the lack of notification, this will not occur again.

monitoring

QuadraNet's control panel is experiencing a minor upgrade. Please reach out to 213-614-9371 x1 for support.

Report: "Century - Network Performance"

Last update
resolved

This incident has been resolved.

monitoring

At midnight site time we are going to make some modifications. We do not anticipate any issues.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We have detected a network performance issue in our Century facility. Engineers have been notified and are currently working to identify the issue. At present, the issue appears to be isolated to a specific router. Customers with redundant uplinks have not been impacted.

Report: "[LAX100] Utility Power Outage"

Last update
resolved

Utility power was restored at 11:45AM local time and has been stable since that point. We are now marking this issue as resolved.

monitoring

QuadraNet has been made aware of a utility power outage at our Century facility. At this time load has been transferred to our generators and no customer impact is expected. The ETA for power restoration is 1:30PM local time and we will provide an update if this changes.

Report: "LAX3-UPS1 - Power Outage"

Last update
resolved

This incident has been resolved.

monitoring

Utility power has been restored to all circuits fed from LAX3-C1-UPS1. Customer equipment is fully energized.

identified

Work is done, being checked for faults and will energize in about 10 minutes.

identified

We are waiting for a proper ETA from electricians. We apologize for the increasing estimates of completion.

identified

Further delays in this repair, at least 30 minutes additional repair time.

identified

The repair has been delayed by about 15 minutes. We still anticipate a 10:30AM restoration but the window has extended to 10:45AM.

identified

Electricians expect power to be restored by 10:30AM. Updates to follow.

identified

The problem has been back traced to a failed main breaker, electricians are currently replacing the breaker.

identified

Technicians are currently working to restore power, bypassing the failed UPS unit.

identified

A power outage has occurred on our LAX3-UPS1 system. We are currently working with our vendor to restore power to the unit.

Report: "ORD-C1 - Outage"

Last update
resolved

This incident has been resolved.

monitoring

The incident has been identified and resolved. Services are restoring now.

investigating

At this time we are investigating an outage in our Chicago facility. Further information will be provided as it is discovered.

Report: "Packet Loss in Miami"

Last update
resolved

This issue is now resolved, the last of the providers has made adjustments to fully block attack traffic toward their infrastructure.

monitoring

QuadraNet has discovered an issue affecting connectivity in the Miami area. We have reached out to our upstream providers and are working on a solution.