env0

Is env0 Down Right Now? Check if there is a current outage ongoing.

env0 is currently Operational

Last checked from env0's official status page

Historical record of incidents for env0

Report: "Issue with creating deployments in env0 SaaS"

Last update
resolved

This incident has been resolved.

investigating

We are currently investigating a sporadic issue with creating deployments in the env0 SaaS. env0 agents are currently unaffected.

Report: "Issue with creating deployments in env0 SaaS"

Last update
resolved

This incident has been resolved.

monitoring

We've have resolved the issue and are currently monitoring the fix. The env0 SaaS should be fully operational

investigating

We are currently investigating a sporadic issue with creating deployments in the env0 SaaS. env0 agents are currently unaffected.

Report: "Issue with creating deployments in env0 SaaS"

Last update
Resolved

This incident has been resolved.

Investigating

We are currently investigating a sporadic issue with creating deployments in the env0 SaaS.env0 agents are currently unaffected.

Report: "Deployments fail in SaaS runners"

Last update
resolved

This incident has been resolved.

identified

A recent docker image upgrade caused a dependency issue, our teams are working on a fix.

Report: "Errors while entering the app"

Last update
resolved

Outage resolved.

monitoring

Revert is done. Monitoring...

identified

Issue identified in one of our services. Revert in progress (~20m)

investigating

We are continuing to investigate this issue. Auto revert is upcoming.

investigating

We are seeing several errors when customers access the app. The issue is in investigation. Auto revert is upcoming.

Report: "Deployments are not started"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We noticed that deployments are not getting started, our team is working on a fix for the issue.

Report: "We're seeing some deployments either failing to start or report completion"

Last update
resolved

This incident has been resolved.

investigating

We are currently investigating this issue.

Report: "We're seeing some deployments either failing to start or report completion"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are currently investigating this issue.

Report: "Request Timeout Errors"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We noticed some timeout errors in our system. Our team is on it.

Report: "Timeout Errors"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We identified some timeout errors in our system, our team is working on it.

Report: "Issues with GitLab integration"

Last update
resolved

This incident has been resolved.

identified

The issue has been identified and a fix is being implemented.

Report: "An issue handling workflow file from GitLab Enterprise Edition"

Last update
resolved

This incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We have identified and are working to resolve an issue when trying to deploy custom flows fetched from GitLab Enterprise Edition. We will roll out a fix shortly.

Report: "Deployment fails when using commit hash"

Last update
resolved

This incident has been resolved.

identified

We currently having some issues completing deployment when using git refs (commit hashes) in place of branch name. The issue has been identified and a fix is being issued.

Report: "Errors in accessing env0 UI"

Last update
resolved

This incident has been resolved.

identified

We've identified an issue with accessing the UI of env0 - our team is working on rolling out a fix

Report: "PR/CD Status checks not reported on GitHub"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We have identified an issue with how we report git providers status checks. We are looking into rolling out a fix within the next 30 minutes

Report: "Issues performing clone actions with GitHub"

Last update
resolved

The source of the issue was found and the incident has been resolved. All operations with GitHub.com are now working as usual

investigating

There are sporadic issues when cloning from GitHub in env0, either when git cloning a GitHub repository, or when downloading a module with a GitHub source. A small number of such git clones fail due to network issues with GitHub.com This seems to be an issue in GitHub, though non has been reported as of yet. We are looking further into the issue

Report: "PR Plans and CD fail to report status check"

Last update
resolved

This incident has been resolved.

identified

We have identified an issue with how we report git providers status checks. We are looking into rolling out a fix within the next 30 minutes

Report: "Some deployments fail to start"

Last update
resolved

This incident has been resolved.

investigating

We are currently investigating this issue.

Report: "Deployments fail to start"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are currently investigating this issue.

Report: "Errors reported on the env0 application UI"

Last update
resolved

This incident has been resolved.

identified

The issue has been identified and a fix is being implemented.

investigating

We are currently investigating this issue.

Report: "env0 UI issues"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

Users are experiencing issues in env0 UI, our RnD members are investigating it.

Report: "Environment service degradation"

Last update
resolved

env0 is back to operational state

investigating

Environment service degradation - querying environments and deploying are not fully available

Report: "Deployment service degradation"

Last update
resolved

The issue is resolved and the application is operational

investigating

We face issues with the deployment service that cause steps not to log and the frontend to not load deployment relevant data.

Report: "UI app auto updates degradation"

Last update
resolved

This incident has been resolved.

monitoring

We rolled out a fix, currently monitoring.

identified

We identified an issue with our AWS AppSync integration. This means the env0 UI auto updates are degraded. Some customers experience stuck loading stats, loading deployment logs, or stating deployment. As a temporary workaround, you can try to click on our “refresh” button (2 blue circular arrows). That should load your data right away. We are deeply sorry. Our team is monitoring and working on a fix.

Report: "Degraded performance starting new deployments"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

Users may experience failures and timeouts when triggering new deployment.

Report: "Gitlab Service Disruption"

Last update
resolved

This incident has been resolved.

identified

Gitlab webhooks and GitOps flows are interrupted because of ongoing incident in gitlab. For more information you can go to: https://status.gitlab.com/

Report: "Delays in Deployments"

Last update
resolved

This incident has been resolved.

identified

A fix was implemented and we are trying to monitor if the issue is resolved.

identified

The issue has been identified and we are working on a fix.

investigating

We are continuing to investigate the issue together with AWS support. It looks like this is an issue with our EKS cluster. Customers who are using the Self-hosted agent should not be affected.

investigating

We are seeing a lot of deployments delays. At first look, it seems to be coming from AWS Step function API where we are getting a lot of network errors and timeouts. We are currently contacting AWS to find out what is the root cause of this issue.

Report: "Increased Error Rates and Delays in Deployments due to AWS"

Last update
resolved

This incident has been resolved.

investigating

We're experiencing an issue with deployments, and overall elevated error and delays in the application. The source of the problem is an AWS outage that affects our application, mostly around API Gateway, EKS, Lambda services on "us-east-1" region. For more information about the AWS outage please refer to AWS status page - https://health.aws.amazon.com/health/status We are currently monitoring the outage, and trying to find workarounds for it.

Report: "Incident with starting new deployments"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

Some of the deployments fail to start, we are taking a look.

Report: "AWS issues"

Last update
resolved

Our login service is now fully restored following the recent AWS outage. You can now access our services without any issues.

monitoring

Our service is now fully restored following the recent AWS outage. We want to assure you that we are actively monitoring the situation to ensure continued stability. If you encounter any further concerns, please reach out to our support team.

identified

We are currently experiencing a service disruption with our service. This is due to a significant outage at AWS: https://health.aws.amazon.com/health/status We will keep monitoring and updating

investigating

We are continuing to investigate this issue.

investigating

We are continuing to investigate this issue.

investigating

The system is unstable due to regional issues in us-east1

investigating

We are continuing to investigate this issue.

investigating

Our system is currently unstable due to issues with auth0

Report: "Deployment Logs may be unaccessible"

Last update
resolved

The incident has been resolved

monitoring

We've uploaded a fix, and are monitoring the issue

identified

We're experiencing an issue with deployment logs not being accessible via the env0 UI. We've identified the source of the issue and working on resolving the issue

Report: "Sporadic timeouts when starting new deployments"

Last update
resolved

Deployments on env0 SaaS should now start as usual. Deployment images on env0 SaaS platform will be pulled from mirror till GitHub resolve their outage of ghcr.io.

investigating

We're experiencing issues starting new deployments due to an outage of ghcr.io. Our team's working rerouting image pull to a mirror - deployments may take longer to start and can timeout

Report: "Deployments may experience degraded performance due to GHCR issues"

Last update
resolved

Github has reported the issue as resolved

monitoring

Implemented a short-term mitigation for SaaS users, deployments will be slower but should work

monitoring

We are continuing to monitor for any updates.

identified

Github Packages is experiencing degraded availability. This can cause a significant delay in all deployments - regardless of customers' VCS Providers.

Report: "Elevated errors in Authentication and Management APIs. Issues Logging In"

Last update
resolved

Auth0 authentication services had downtime, which affected env0 as well env0 users had issues logging in

Report: "Deployments may experience degraded performance due to GHCR issues"

Last update
resolved

This incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

We are continuing to monitor for any further issues.

monitoring

We are continuing to monitor for any further issues.

monitoring

Deployments may experience degraded performance due to GHCR issues

Report: "Issue with loading the App"

Last update
resolved

This incident has been resolved.

identified

Due to an issue with a third party provider (MyFonts), The front-end application might not load for certain customers.

Report: "Incident with starting new deployments"

Last update
resolved

The root cause (GHCR) has been resolved and deployments on env0 are starting as usual.

identified

Due to an issue with Github and GHCR, new deployments will not start or take a very long time to start.

Report: "GitHub Enterprise Pull Request Integration Problem,"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We are currently investigating an issue with GitHub Enterprise Pull Request Integration.

Report: "Auth0 flakiness"

Last update
resolved

This incident has been resolved.

identified

auth0 seems to be having increased flakiness, the source seems to be cloudflare which has increased error rates

Report: "Issue with Private Module Registry"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We're seeing failures when trying to deploy Templates using our Private Module Registry. We are currently investigating this issue

Report: "Drift Detection runs fail"

Last update
resolved

This incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We've identified an issue with Drift Detection runs and are working to resolve it.

Report: "Downtime due to third party provider issues"

Last update
resolved

This incident has been resolved.

monitoring

Cloudflare (and consequentially Auth0) have reportedly implemented a fix and monitoring the results. And so are we

identified

The issue has been identified and a fix is being implemented.

Report: "Elevated API Errors"

Last update
resolved

This incident has been resolved.

monitoring

A fix was rolled out successfully and we're monitoring the results

identified

We've identified a possible cause and are looking to roll out a fix

investigating

We're experiencing an elevated level of API errors and are currently looking into the issue.

Report: "we're experiencing issues with Gitlab integrated templates - this might affect your deployments or gitflow triggers"

Last update
resolved

We've rolled out the fix that will allow GitLab tokens to be refreshed. You will need to re-authenticate your using GitLab user/bot with env0 by editing the Template in env0 that integrates with that GitLab user, click the "authorize another GitLab user" and authorize with your user/bot again. Once again we're terribly sorry for the inconvenience - we're in touch with GitLab trying to understand how it came to pass that we were not alerted about this change in their API to avoid such future incidents.

identified

GItLab.com (the Saas) updated their version to 15.x. which deprecates our authorization approach. We are working on a solution, in the meantime, if you must use GitLab you can re-authorize by going into the template, selecting "authorize another Gitlab user"

identified

we're experiencing issues with Gitlab integrated templates - this might affect your deployments or gitflow triggers

Report: "Deployments are taking longer to start"

Last update
resolved

env0 deployments are at peak performance again

identified

Deployments are now operational but may take longer to start than usual. We're still working on getting env0 back to peak performance

identified

Deployments are now operational but may take longer to start than usual. We're still working on getting env0 back to peak performance

identified

env0 deployments will take a long time to start or will time out. The root cause seems to be a global issue with ghcr.io, which is hosting our deployment image. We are looking into it.

Report: "Slowness in env0 deployments"

Last update
resolved

This incident has been resolved. If you have an environment that appears "In Progress" when it has already finish, then clicking on the "abort" button will resolve the issue.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are continuing to investigate this issue. Effects of this issue include slowness in deployments, inability to approve or cancel deployments

investigating

Some customers are currently experiencing slowness in env0, for deployments and drift detections. We are currently investigating the issue

Report: "Slowness in env0 SaaS deployments"

Last update
resolved

This incident has been resolved.

investigating

The issue has been resolved. All new deployments should now start as usual

investigating

Some env0 deployments are currently slower than usual. We are investigating the source of this issue

Report: "Partial System Outage"

Last update
resolved

The incident has been resolved. All env0 components are now fully operational

investigating

We are continuing to investigate this issue.

investigating

Some aspects of env0 are currently not working properly, including secrets, cost, and AWS assume role We are looking

Report: "Partial System Outage"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been deployed ad we're monitoring the results

identified

We've identified a cause for a partial outage in env0 - ETA on a being back online is 15-20 minutes