Is Paperspace Down Right Now? Check if there is a current outage ongoing.

The issue was identified as affecting only a specific cohort of users. We have verified that all the systems including the Paperspace Console are operating as expected. Any remaining access concerns will be handled on case-by-case basis. We appreciate your patience and understanding.

We are currently investigating an issue that is preventing users from accessing the Paperspace console. Our team is working to resolve the issue as quickly as possible. We apologize for any inconvenience and will provide updates as we progress.

The issue was identified as affecting only a specific cohort of users. We have verified that all the systems including the Paperspace Console are operating as expected. Any remaining access concerns will be handled on case-by-case basis. We appreciate your patience and understanding.

We are currently investigating an issue that is preventing users from accessing the Paperspace console. Our team is working to resolve the issue as quickly as possible. We apologize for any inconvenience and will provide updates as we progress.

The scheduled maintenance has been completed.

Scheduled maintenance is currently in progress. We will provide updates as necessary.

Paperspace will be performing upgrades to the underlying systems powering your machine(s) in the AMS region. If your machine(s) are running during the maintenance window, they will be rebooted once. Additionally, start, stop, and create operations for all Ampere-based machines may be temporarily impacted during this period. If your machine(s) are expected to be affected, you will have received an email notification and should see a banner in the console. If you did not receive an email or see a banner but believe your machine(s) may be impacted, please reach out to Support for further assistance.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are investigating an issue affecting Core VMs in the NY2 region, which may cause accessibility issues. Our team is working to resolve the issue as quickly as possible. We apologize for any inconvenience and will provide updates as we progress.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

We are seeing that our API is having degraded performance, also, access to the Paperspace Console might be intermittent. Our engineers are investigating the root cause. We appreciate your patience.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

We are continuing to work on a fix for this issue.

We've identified the issue and are working on a fix. Some users might still experience issues while we are implementing the fix.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are continuing to investigate this issue.

We are currently investigating a network issue that is preventing users from interacting with Gradient notebooks and Core VMs.

This incident has been resolved. Thank you for your patience and cooperation.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are continuing to investigate the issue preventing users from logging into their Paperspace accounts. We sincerely apologize for the inconvenience this has caused and appreciate your patience as we work toward a resolution.

We are currently investigating an issue that is preventing users from logging into their Paperspace account. We apologize for the inconvenience caused.

Our engineers have applied a fix and the issue is considered resolved. If you continue to experience problems, please contact our Support Team. We apologize for the inconvenience experienced.

We are continuing to monitor for any further issues.

Our engineers have implemented a solution and we are currently monitoring the results.

Our engineers have identified the issue and are actively working on implementing a fix. We appreciate your patience as we work to resolve this issue.

We are continuing to work on a fix for this issue.

We are continuing to work on a fix for the issue. We apologize for any inconvenience caused.

The issue has been identified and a fix is being implemented.

We are currently investigating an issue that is preventing users from interacting with their CORE machines in CA1. Our engineers are working on a fix. We apologize for any inconvenience caused.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are currently investigating a network issue that is preventing users from interacting with Gradient notebooks.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are currently investigating a network issue that is preventing users from interacting with Gradient notebooks.

This issue has been resolved now. Thank you for your patience and cooperation.

Our engineers implemented a fix and we're monitoring the situation. If you're still experiencing issues, please contact our Support Team.

A fix has been implemented and we're monitoring the results. Thank you for your patience and cooperation.

Our engineers have identified a networking issue affecting Paperspace Bare-Metal services caused by our upstream provided. Our network team is actively working with our upstream provider to mitigate the issue. Further updates will be provided as we implement corrective measures.

We are investigating a Network Issue that's affecting our Bare-Metal users in the NY Region. We apologize for the inconvenience.

Our engineers have resolved the issue. Please contact our Support Team if you encounter any problems with your CA1 CORE Virtual Machine.

Our engineers have implemented a fix and events should start processing as usual. Customers that still experience issues, please wait a few minutes and try again or contact our Support Team.

We are currently investigating an issue that is preventing CORE machines in CA1 from processing events. Our engineers are working on a fix. We apologize for any inconvenience caused.

This incident has been resolved. Thank you for your patience and cooperation.

We are continuing to monitor for any further issues. We appreciate your patience and cooperation.

Our engineering team has implemented a fix and we are monitoring the results. We appreciate your patience and cooperation.

Our engineers have identified the issue and are working on a fix for this issue. We apologize for the inconvenience caused.

The issue has been identified and a fix is being implemented.

We're investigating an issue where users aren't able to interact with the Paperspace Console and other Paperspace Services. We apologize for any inconvenience caused.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

Several reports have been done regarding machines not being accessible on both Core and Gradient, we are investigating the root cause of this issue, and your patience is appreciated.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are currently investigating an issue that is preventing news users and teams from accessing the billing page.

The issue has been resolved. If you experience any problems, please contact our Support Team.

Our engineers implemented a fix and we're monitoring the situation. Users should be able to interact with their VMs as normal. If you experience any issues, please contact our Support Team.

Our engineers are investigating an issue that prevents NY2 Region users from interacting with their Virtual Machines. We apologize for the inconvenience.

The incident was resolved by our engineers. If you continue to experience issues, please contact our Support Team.

Our engineers have implemented a fix. Users should be able to access Gradient assets as normal. If you experience any problems, please contact our Support Team.

We are currently investigating an issue that is preventing users from interacting with Gradient notebooks. We apologize for the inconvenience caused.

The issue affecting both CORE and Gradient users in the NY2 Region has been resolved. If you continue to experience problems, please contact our Support Team.

Our engineers have implemented a fix and we're monitoring the situation. If you continue to experience issues, please let us know.

Our engineers are currently working on a fix. We appreciate your patience.

We're investigating a possible network issue that's affecting CORE and Gradient services in the NY2 Region. We apologize for the inconvenience experienced.

This incident has been resolved, free notebooks are working well. We appreciate your patience and cooperation. Thank you.

A fix has been implemented and we are monitoring the results.

We are continuing to work on a fix for this issue.

We've identified the issue with Free notebooks being stuck in a pending state and are working on a fix. Some users might still experience issues while we are implementing the fix.

This incident has been resolved.

We are continuing to monitor for any further issues.

A fix has been implemented and we are monitoring the results.

We are continuing to work on a fix for this issue.

Our engineers have identified generalized issues with the network connectivity, we are working on a solution to ensure connectivity to all our users. We appreciate your patience.

# Paperspace NY2 network outage Postmortem ## Incident Summary On June 16th, a significant number of virtual machines in the NY2 region became unavailable and went into Read-Only \(RO\) mode, which affected network availability. By 18:20 UTC on the same day, network connectivity was restored to the affected virtual machines. However, affected virtual machines continued to be unavailable until 12:23 UTC on June 17th as they were still in RO mode. The affected virtual machines were restored to Read-Write \(RW\) mode between 5:58 and 12:23 UTC on June 17th. By 12:24 UTC, affected virtual machines were available and network access to customers was restored.  ## Incident Details ### Root Cause The root cause of this unexpected service unavailability is related to a core switch failure in NY2 that caused a large number of machines to go into RO mode. ### Impact As a result of the failure of the core switch, there was a spike in traffic that adversely impacted network performance. The virtual machines went into RO mode as a result of the loss of network access to the Network File System on which they resided. Since virtual machines were in RO mode, customers were unable to perform any write operations on them, resulting in service disruptions around 10:30 UTC on June 16th, 2024.  ## Remediation Actions A number of efforts are underway to try to prevent these types of failures from occurring again, including a network redesign and installation of new equipment. On behalf of Paperspace, we apologize for the disruption to your services and appreciate your understanding.   If you have any questions or concerns, please open a ticket with our[ Customer Support team](https://docs.digitalocean.com/support/paperspace/#open-a-ticket.).

Our engineers have resolved the issue. If you continue to experience issues, please contact our Support Team.

We've implemented a series of fixes and are monitoring the results. While Core is already fixed, the Gradient platform will continue to have degraded performance since it is still taking time to recover. We will continue working on fixes for it.

The issue has been identified, it appears to be an issue with our network in NY2. A fix is being implemented.

We are continuing to investigate this issue.

We are continuing to investigate this issue.

We are continuing to investigate this issue.

We are continuing to investigate this issue.

We are currently investigating a network issue that is preventing users from interacting with Gradient notebooks and Core VMs based in NY2 region.

Our engineers have resolved the issue. If you continue to experience issues, please contact our Support Team.

A fix has been implemented and we are monitoring the results.

We've identified the issue and are working on a fix. Some users might still experience issues while we are implementing the fix.

We are currently investigating an issue that may prevent users from interacting with CORE in the NY2 Region. We apologize for the inconvenience caused.

Our engineers have resolved the issue. If you continue to experience issues, please contact our Support Team.

Our engineers have identified the issue that's preventing some users from interacting with Gradient Notebooks and are in the process of fixing it. Thanks for your patience.

Our engineers are investigating an issue that is affecting some Gradient Notebook users. We apologize for the inconvenience.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are currently investigating an issue that is preventing users from interacting with Gradient.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

We are currently investigating an issue that is preventing users from interacting with their Console as expected. This includes logging in, turning their machines on and off, and interacting with the API. We are working on getting this fixed ASAP.

This incident has been resolved.

Our engineers have addressed the issue. Most users should be able to use their CORE Virtual Machines in the AMS1 Region. If you continue to experience issues, please reach out to our Support Team.

Our engineers are working to resolve a networking issue that may prevent some users in the AMS1 Region interacting with their CORE Virtual Machines. We apologize for the inconvenience.

The issue has been resolved by our engineers. We apologize for any inconvenience caused. If you continue to experience issues, please contact our Support Team.

Our engineers have addressed the issue. Most users should be able to use their CORE Virtual Machines in the AMS1 Region. If you continue to experience issues, please reach out to our Support Team.

Our engineers have addressed the issue. Most users should be able to use their CORE Virtual Machines in the AMS1 Region. If you continue to experience issues, please reach out to our Support Team.

Our engineers have identified a potential electrical issue that may be impacting users of the AMS Region CORE Virtual Machine and are currently in the process of rectifying it. We apologize for any inconvenience this has caused you.

This incident has been resolved.

We are continuing to monitor for any further issues.

A fix has been implemented and we are monitoring the results.

We are continuing to investigate this issue.

Our engineers are investigating an issue where users may not be able to interact with Gradient Notebooks and Core VMs in NY2

We experienced a brief network disruption affecting GRADIENT services at 20:53 EDT. This issue has been resolved and all systems are back online. We apologize for any inconvenience caused. If you are still seeing issues, please reach out to Support.

At this time we are confident that the incident has been resolved. If you are still seeing issues, please reach out to Support.

We've implemented a fix and are monitoring the results. Users should be able to load the console again.

We are currently investigating an issue that is affecting Core and Gradient users. During this time users will face some issues while attempting to access the resources and may receive 429 errors. Our engineers are working on a fix. We apologize for any inconvenience caused.

This incident has been resolved.

We are monitoring the situation. Some users may experience issues with Gradient and Free instances continue to run at a lower capacity. We apologize for the inconvenience.

We continue to keep an eye on the situation. Gradient Free instances are currently running at a lower capacity. We apologize for the inconvenience.

We continue to monitor the situation. Users may encounter issues starting Free Gradient instances. We apologize for the inconvenience.

We've implemented a fix and are monitoring the results.

Our engineers have identified and working on a fix for GRADIENT. We apologize for the inconvenience caused.

We have implemented a fix for CORE and continue to investigate GRADIENT. We apologize for the inconvenience caused.

We are continuing to work on a fix for this issue.

Our engineers are still working on resolving this issue. We appreciate your patience and apologize for the inconvenience.

Our engineers continue investigating the issue. We apologize for the inconvenience caused.

Our engineers are investigating an issue where users may not be able to interact with Gradient Notebooks. We apologize for the inconvenience caused.

Our engineers have resolved the issue. If you encounter any issues, please contact our Support Team.

Our engineers continue to work on resolving this issue. We apologize for any inconvenience caused.

We are currently investigating an issue that may prevent users from interacting with the CORE API. We apologize for the inconvenience caused.

The issue affecting CORE and GRADIENT users in the NY2 Region has been resolved. If you experience any further issues, please contact our Support Team.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are still investigating a network issue that may prevent users from interacting with CORE and Gradient in the NY2 Region. We apologize for the inconvenience caused.

We are currently investigating a network issue that may prevent users from interacting with CORE and Gradient in the NY2 Region. We apologize for the inconvenience caused.

The issue has been resolved by our engineers. If you need further assistance, please contact our Support Team.

Our engineers have implemented a fix. If you continue to experience issues, please reach out to our Support Team.

Users may experience issues starting or stopping CORE Virtual Machines in the NY2 Region. Our engineers are already working on a fix. We sincerely apologize for any inconvenience that may have been caused.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

Our engineers are currently investigating an issue causing slower than usual response times on services in the NY2 Region.

The issue affecting users in the NY2 Region has been resolved. If you experience any further issues, please contact our Support Team.

Our engineers are currently investigating an issue causing slower than usual response times on services in the NY2 Region.

This incident has been resolved.

We are continuing to monitor for any further issues.

A full fix has been implemented for Core and Gradient. We are monitoring the results.

We are continuing to see residual effects on Gradient. Our engineers are continuing to implement a fix.

A fix has been implemented and we are monitoring the results.

We are currently seeing slower performance than usual on Gradient and CORE in NY2, we appreciate your patience while our engineers work on it.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

We've identified the issue and are working on a fix. CORE VMs functionality has returned. Gradient users might still experience issues while we are implementing the fix.

We are currently investigating an issue that may prevent users from interacting with CORE and Gradient in the NY2 Region. We apologize for the inconvenience caused.

Our engineers have fixed this issue. If you experience any problems, please contact our Support Team.

Our engineers applied a fix to the slow performance issue in the NY2 Region and continue to monitor the situation. Please let us know if you continue to experience any problems.

Our engineers continue to work on fixing this issue. We appreciate your patience.

Our engineers continue to work on fixing this issue. We appreciate your patience.

Our engineers continue to work on a fix for this issue. We apologize for any inconvenience caused.

Our engineers continue to work on fixing this issue. We appreciate your patience.

Our engineers continue to work on a fix for this issue. We apologize for any inconvenience caused.

Our engineers continue to work on fixing this issue. We appreciate your patience.

Our engineers are still working on a solution for this problem. We apologize for any inconvenience this may have caused.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are currently investigating a network issue that is preventing users from interacting with notebooks and affecting connectivity to CORE VMs.

Our engineers have resolved this issue. If you keep experiencing problems, please reach out to our Support Team.

Our engineers have implemented a fix. If you experience issues, please reach out to our Support Team.

Our engineers have identified the issue and are working on a fix. We appreciate your patience.

We are investigating an issue preventing users from interacting with their Paperspace CORE Machines and Gradient Notebooks. We apologize for the inconvenience caused.

The issue has been resolved. For any further issues, please contact our Support Team.

We've implemented a fix and are monitoring the results.

This incident has been resolved.

Our engineers have resolved the issue, but we will continue to monitor the situation. If you require assistance, please don't hesitate to contact our Support Team.

Our engineers are currently working to fix the issue that is preventing users from accessing their virtual machines. Thanks for your patience.

We are investigating an issue preventing users from starting or shutting down their Virtual Machines. We apologize for the inconvenience caused.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are continuing to investigate this issue.

We're currently investigating the issue. Users may find issues while creating or accessing Core VMs in the CA1 region.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

We are continuing to investigate this issue.

We are currently investigating an issue that is preventing CORE machines in CA1 from starting. We are working to restore service as soon as possible. We apologize for any inconvenience caused.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

We are continuing to investigate this issue.

We're currently investigating the issue to get it fixed ASAP. Some users might be facing issues while trying to run GPU+ machines in CA1 region.

Our engineers have resolved the issue. If you experience any problems, please contact our Support Team.

We are actively monitoring the issue. For any questions or concerns, please reach out to our Support Team.

Our engineers have resolved the issue and are continuing to monitor the situation. If you experience additional problems, please reach out to our Support team.

Our engineers are actively working on a solution for this issue. For any inquiries, please contact our Support Team. We appreciate your patience and apologize for the inconvenience.

Our engineers are currently working to resolve an issue that is affecting Free IPU instances. We apologize for any inconvenience this may cause.

The engineers have resolved the issue with vGPUs. If you continue to experience problems, please contact our Support team.

Our engineers have implemented a fix and we're monitoring the situation. If you experience further issues, please contact our Support team.

We have identified an issue preventing users from using their vGPU (GRID) Virtual Machines. We are currently working on a fix and apologize for any inconvenience caused.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are currently investigating an issue may prevent users from interacting with CORE and Gradient in the NY2 Region. We apologize for the inconvenience caused.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

We are continuing to investigate this issue.

We are currently investigating an issue is preventing users from interacting with Gradient IPU Notebooks.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

At 9:38pm our engineers became aware of an issue affecting Core and Gradient in NY2 due to Network maintenance. We are working to restore service as soon as possible.

This incident has been resolved.

We are currently investigating an issue that is preventing users from interacting with their desktop streams on Linux VMs as expected.