Historical record of incidents for PeopleGrove
Report: "Site slowness"
Last updateWe are continuing to monitor the slowness, but response times have returned to normal.
We are continuing to investigate this issue.
We are investigating site slowness that is affecting US clients. We will post an update as soon as possible.
Report: "Degraded site performance"
Last updateThe incident has been resolved.
We have identified the cause of the issues, and implemented a fix. Site load times have returned to normal. We are continuing to monitor the issue.
We are continuing to investigate this issue.
We are currently investigating degraded site performance. PeopleGrove platforms may be slow to access. We are investigating and will provide an update shortly.
Report: "Issues accessing sites"
Last updateThe fix is implemented by Cloudflare. All sites are operational and working. We would continue to monitor the results.
We are currently investigating an issue accessing PeopleGrove platform websites. Sites may not be available to users. Issue is coming because service issues with our service provide Cloudflare. We are taking this up with Cloudflare. Latest status of the outage on Cloudflare can be checked at https://www.cloudflarestatus.com/
Report: "Partially degraded performance"
Last updateWe have been monitoring the issue and performance has been normal and the platform is functioning normally.
We have identified the reason for the site slowness, and have implemented a fix. Sites should be functioning normally now. We are continuing to monitor the performance.
We are investigating an issue causing slowness on the PeopleGrove platform. We will provide an update as soon as possible.
Report: "Issues accessing sites"
Last updateGoogle has confirmed that all issues have been resolved. All systems are normal at this time.
The issue is related to a Google Cloud Platform outage: https://status.cloud.google.com/incidents/6PM5mNd43NbMqjCZ5REh. We are continuing to monitor the issue.
All PeopleGrove sites are operational again. We are continuing to monitor performance.
We are currently investigating an issue accessing PeopleGrove platform websites. Sites may not be available to some users. We will provide an update as soon as possible.
Report: "Slow site performance"
Last updateThe incident has been resolved.
A fix has been implemented and we are monitoring site performance. Users should be able to access sites at this time.
We are currently investigating slow site performance for all US-based PeopleGrove. Users may have difficulty in accessing sites. We are working to restore full services as quickly as possible. Thank you for your patience.
Report: "Site performance issue"
Last updateAll loading issues have been resolved, and have seen all metrics return to normal. All sites should be accessible as normal. We are continuing to monitor site performance. We apologize for the inconvenience.
We are still investigating the incident and are implementing a fix to make the service available. We will continue to keep this incident updated. We apologize for the inconvenience and thank you for your patience.
We are currently investigating a performance issue with the PeopleGrove platform. We are seeing an elevated error rate and some users may experience slow load times. We are investigating the issue and will provide an update shortly. We apologize for the inconvenience.
Report: "Bulk Email Issue"
Last updateThe issue with Sendgrid has been resolved, and all email systems are now fully operational.
We are currently experiencing an issue with our service provider, Twilio Sendgrid, which we use to send bulk emails in the system. Until the issue is resolved, all bulk emails and newsletters will not be sent. We apologize for the inconvenience and will provide an update as soon as the service is restored.
Report: "Delayed data"
Last updateAll data queues have been processed all tools are now operating normally.
We are continuing to work through the backlog of delayed events, and expect all data to be updated by later tonight US Pacific time. Again, this incident is only affecting the following admin tools: - Explore Users - All email statistics (such as opens and clicks) - Update History API For the above tools, no data has been lost, but processing has been delayed. All other tools are fully functional for all users and we are not seeing any degraded performance. We apologize for the inconvenience and thank you for your patience.
We have identified an issue with the application which is causing certain data to be delayed. All data is being queued and will be populated within the next few hours. The following admin-facing tools are experiencing delays in displaying data: - Explore Users - All email statistics (such as opens and clicks) - Update History API We apologize for the inconvenience and thank you for your patience.
Report: "Platform performance issues"
Last updatePeopleGrove has experienced slow platform performance from ~12:30PM Pacific time until ~1:15PM Pacific time due to an issue with Google Cloud Networking (https://status.cloud.google.com/incident/cloud-networking/20004). The incident has now been resolved. We apologize for the inconvenience.
Report: "Google Cloud outage"
Last updateAll services are operating normally.
All services have been restored. Google Cloud has restored traffic out of the us-east-1 region, and we have also deployed a fail-over in another region. We apologize for the inconvenience. We are continuing to monitor the situation.
Due to an ongoing incident with our provider Google Cloud, users may be unable to access some or all PeopleGrove sites. We are working as quickly as possible with our providers to restore service. Thank you for your patience. https://status.cloud.google.com/incident/cloud-networking/19015
Report: "Degraded performance"
Last updateThis morning, we experienced higher than normal API failure rates for about 90 minutes. Our engineers identified a performance issue with a database and have deployed a fix. Systems are back to normal.
Report: "Connectivity issues with some custom domains"
Last updateThe issue has been resolved and changes are propagating. The sites with affected custom domains are starting to come back online.
We are continuing to monitor for any further issues.
Some customers who are using a custom domain for their site are currently experiencing issues accessing their site due to an outage with one of our service providers. We are monitoring the situation and those sites should be back up shortly.
Report: "Slow site performance"
Last updateThe site slowness issue has been resolved.
We are currently experiencing slow site performance. Our engineers are looking into the issue and hope to be back to normal as soon as possible.
Report: "Degraded site performance"
Last updateAll systems are back to normal and any issues accessing our sites should be resolved.
Due to an issue with one of our vendors (Heroku, https://status.heroku.com/), out automatic scaling is not working properly and some users are having trouble accessing our platforms. We are investigating the issue and working with our infrastructure providers to resolve as soon as possible.
Report: "Site slowness and increased failed requests"
Last updateFrom about 5am to 9am PT, we experienced an increased number of failed requests to our API, which caused the site to be for some users, and for some admin tools to stop working. The issue was related to a failure of one of our systems to scale to additional resources, causing the system to be overloaded. We have identified and fixed the issue.
Report: "Site Loading Issues"
Last updateThis incident has been resolved.
It appears that our infrastructure providers have resolved the issues. We are continuing to monitor the situation closely.
We are monitoring issues with upstream providers (Heroku, Amazon) which are resulting in slow page loads, failure for images to load, and errors uploading/downloading files. We are continuing to receive updates from Amazon and Heroku regarding the outage. We will keep you updated as this progresses and thank you for your patience.