Historical record of incidents for Keycafe
Report: "Partial cellular service outage"
Last updateWe’re currently experiencing a partial cellular service outage affecting some Smartboxes. As a result, some customers may experience connectivity issues with their Smartbox, including intermittent loss of connection or issues starting new data sessions. Our provider is actively investigating the issue with their upstream partners. We’ll continue to monitor and provide updates as more information becomes available. We apologize for the inconvenience and appreciate your patience.
Report: "Degraded Database Performance"
Last updateWe are experiencing degraded database performance as a service provider is experiencing a partial outage. We will continue to monitor and provide updates.
Report: "2G/3G Connectivity disruption"
Last updateThis incident has been resolved.
Most of the region's power has been restored, and Smartboxes are coming back online. We will continue to monitor the situation until everything is fully restored.
Devices operating on 2G and 3G networks are currently affected by power outages in Spain. This affects devices not in these regions due to data routing through these countries. Our cellular partner is impacted, and devices will remain offline until power is restored in the region. We will monitor the situation and provide updates.
Devices operating on 2G and 3G networks are currently affected by power outages in Spain. This affects devices not in these regions due to data routing through these countries. Our cellular partner is impacted, and devices will remain offline until power is restored in the region. We will monitor the situation and provide updates.
Report: "Service Disruption"
Last updateTimeframe: 10:19 AM to 10:27 AM EST Impact: Approximately 50% of requests failed during this period. Resolution: The issue causing the disruption has been resolved, and all systems are now fully operational.
Report: "Degraded cellular serveice affecting Key exchanges"
Last updateThis incident has been resolved.
We have noticed a few Celular Smartboxes encountering partial service degradation. Our service provider is investigating the matter.
Report: "Service Interruption across Smartboxes and Applications."
Last updateThis incident has been resolved.
Our service provider has reported that they implemented a fix. We will continue to monitor the results.
We are still experiencing interruptions in service. Some customers may experience elevated latencies or intermittent timeouts. Our service provider is working to resolve the issue.
We are still experiencing a partial outage impacting Smartboxes and our website. Our service provider has acknowledged the issue and is investigating.
We are experiencing an outage that is affecting Smartboxes and our website. We will share an update as we have more information.
Report: "Service interruption across Smartboxes and Applications."
Last updateThis incident has been resolved.
Our providers have reported that they implemented a fix. We will continue to monitor the results.
We are still experiencing interruptions in the service, although some requests are coming through. We are continuing to monitor.
We are experiencing an outage that is affecting Smartboxes and our website. Our providers have acknowledged the issue and are working to identify the cause.
Report: "Partial Outage Impacting Small Number of Cellular Network Devices"
Last updateThis incident has been resolved.
We are experiencing a partial outage affecting a small number of cellular network devices. Our service providers are working closely with their partners to resolve the issue as soon as possible.
Report: "Partial outage affecting key exchange"
Last updateThe issue causing transaction timeouts has been fully resolved. All systems are now functioning normally. We appreciate your understanding and patience throughout this incident.
The cause of the transaction timeouts affecting key exchanges has been identified by our service provider, and a fix has been implemented. We are currently monitoring the results to ensure the issue is fully resolved. Thank you for your patience.
We are experiencing a partial service disruption, causing transaction timeouts during key exchanges. Our team is actively investigating and working to resolve the underlying issue. We appreciate your patience as we work toward a solution.
Report: "Partial Outage"
Last updateServices have been restored and are now operating normally. We will continue to monitor the system to ensure stability. Thank you for your patience and understanding.
We are continuing to investigate this issue.
We are beginning to observe some devices and website functionality, and our team is currently investigating.
We are currently investigating a partial outage that is impacting our website and Smartboxes. Our Engineers are investigating.
Report: "Connectivity interruption for some North America Cellular Devices"
Last updateThis incident has been resolved.
Some devices in North America are experiencing network interruptions. Our service provider has identified the problem and is currently working with their team to fix it. Some devices have since come back online and are fully functioning.
Report: "Partial outage affecting website performance and key exchanges"
Last updateResolved Issue: At 9:22 GMT-5, we experienced a partial outage impacting our website's performance and key exchanges. The issue has been identified and a fix is implemented. Current Status: We continue to monitor the system closely to ensure stability
Report: "Network performance issues"
Last updateWe are pleased to inform you that the service issue that was affecting our systems has been successfully resolved. Our service provider has diligently worked to rectify the problem, and we are happy to report that our systems are now operating normally.
A fix has been implemented and the results are being monitored.
The issue has been identified and a fix is being implemented.
Cloudflare has updated its incident status to include a latency issue for the SAM region in addition to the APAC and partial NA that was previously reported. A fix has been implemented for the NA region and the results are being monitored. They are still working to analyze and mitigate this problem being experienced in the SAM and APAC region as soon as possible.
We are aware of the latency is also currently being experienced in some areas of NA in addition to APAC regions. Our providers are working diligently to investigate the issue.
We are continuing to monitor for any further issues.
At present, Cloudflare is diligently investigating the root cause of the network performance concerns that have been reported in the APAC region.
Report: "Partial outage affecting website and app"
Last updateOur team was able to locate and correct the issue. The incident is now resolved.
Our website and app are experiencing a partial outage affecting performance/loading/searches. We have identified the issue and working to get it resolved.
Report: "Connectivity issue affecting some 3 G devices (mostly outside USA)"
Last updateWe are seeing the impact of a fix now, which has resulted in significant recovery.
Our provider is still engaged in troubleshooting with all relevant technical teams. No ETA is currently available, but the issue is being given the highest priority.
Our third-party provider has identified it as a network issue, the connectivity provider is working to solve its issue. Most of the devices affected are outside of the USA.
Report: "Connectivity Incident LTE Devices (recovered)"
Last updateOur service provider reported an LTE connectivity incident at 05:43 PDT. Connectivity was restored at 06:14 PDT.
Report: "Partial Smartbox Outage"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
An incident is affecting our Smartboxes. Our service provider has identified the problem and is working on a fix. This occurred at 10:55 PDT. We will keep you informed.
Report: "System outage"
Last updateThis incident has been resolved.
We are seeing Smartboxes recovering in NA, however, there are still reports of incidents in the EU.
We are monitoring an incident reported by one of our service providers affecting Smartboxes. This started at 05:17 PDT.
Report: "Service provider outage"
Last updateThis incident has been resolved.
Our service provider is experiencing an issue. They are investigating. This is affecting a lot of major websites.
Report: "Third party outage. Investigating"
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "Third-party provider - Cellular Connectivity Incident"
Last updateNetwork traffic has resumed and our providers are monitoring recovery.
Latest report Traffic is beginning to resume. Only areas on 2G/3G are affected. EtherSIM devices are not affected.
Our provider has reported a connectivity issue affecting some areas. Report: We are investigating a cellular outage involving Electrons and E310s. No regional patterns have presented themselves as of yet.
Report: "Third party provider Cellular Outage"
Last updateUpdate from our provider: Traffic has resumed to normal levels. We will continue to monitor resolution closely overnight before marking this incident resolved.
Latest update from our providers: Particle has identified a return in device traffic, which will likely increase over the next hour. We will monitor the stability of this resolution for the next several hours.
The latest update for our provider: This issue is ongoing. We are continuing to engage our partner with the highest priority to secure an ETA for resumption of service.
Latest update from our provider: This issue is ongoing. We are in dialogue with the network provider in question; our provider is leveraging engineering resources toward identifying root cause.
A third-party provider has reported a cellular outage affecting our Smartboxes on this network type in NA. We will continue to monitor.
Report: "Third-party provider wide-spread issue"
Last updateOr service provider has resolved a network issue that affected multiple services across the internet. Our website has returned to normal operation.
We are continuing to monitor for any further issues.
They have implemented a fix and are now monitoring the results.
A third-party provider Cloudflare has discovered a widespread issue with their service impacting our Website. We are monitoring.
Report: "Third-party provider wide-spread issue"
Last updateA third-party provider Cloudflare has discovered widespread issues with their service impacting our Website. We are monitoring.
Report: "Third party outage causing connectivity issues"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
Our service provider reported that A fix has been implemented and they are seeing recovery. The results are being monitored.
A third-party service provider is experiencing a small drop in webhooks/integrations. They are currently assessing the scope of impact and identifying the cause.
Report: "Loading error"
Last updateThis incident has been resolved.
Our provider reported that their engineers are recovering affected routing components.
A third-party provider has acknowledged the connectivity issue and is investigating.
Our website and smartboxes are presenting a connection problem. We are investigating the issue at this time.
Report: "Rolling blackout incident affecting Smart Boxes"
Last updateThe rolling blackout incident which created a disturbance with our service providers while affecting SmartBoxes seems to be resolved.
Our network of SmartBoxes is currently being affected by a rolling backout incident occurring in the US. Some systems have returned to normal while others are still being affected. It is expected that this interruption to our network will continue until the "rolling backout" has ended.
Report: "Connectivity Incident"
Last updateA third-party service provider reported intermittent connectivity loss. The incident is now resolved.
Report: "Third party outage causing connectivity issues"
Last updateThis incident has been resolved.
Performance improvement/recovery is continuing.
We are continuing to monitor for any further issues.
Our third-party provider reported that they are continuing to see signs of recovery. Events latency is still relatively high (the delivery/appearance of Events may take longer than expected at this stage).
The issue has been identified and a fix is being implemented.
Particle is reporting a partial outage. Communication between some devices and our server is being delayed.
Report: "Third party outage causing connectivity issues."
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are currently awaiting updates while our service provider continues to investigating a connectivity-related outage. We will provide an update as soon as one is available.
Report: "Partial outage"
Last updateThis incident has been resolved.
The issue has been identified. All components have stabilized.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We’re experiencing a partial outage. Requests are slow and occasionally timing out.
Report: "Cellular Outage"
Last updateOne of our third-party service providers had a major outage affecting LTE/3G devices. The incident occurred at 18:15PDT and was resolved at 19:23 PDT.
Report: "Third party provider connectivity issue affecting Programmable SMS"
Last updateThis incident has been resolved.
Identified - We are no longer experiencing connection issues, however, Messaging queues are still being processed. An update will follow once the issues have been fully resolved.
A third party provider is experiencing connection issues affecting programmable SMS. 2factor authentication and pickup codes are being affected. The issue is being investigated.
Report: "Smartbox connectivity issues due to third party outage"
Last updateThis incident has been resolved.
Devices are actively recovering; Our service provider is monitoring the recovery process.
Some Smartboxes are experiencing connectivity issues due to a service provider having a partial outage. The issue has been identified and a fix is being implemented.
Report: "Degraded service from provider resulting in offline Smartboxes"
Last updateThis incident has been resolved.
We are seeing where most affected Smartboxes have returned to active status. We are continuing to monitor for further issues.
Our service provider (Particle) has deployed a fix which is currently ongoing. During this time, some devices will still remain at issues status until the fix has been completed. We will continue to update you.
We are continuing to monitor for any further issues.
A major Keycafe service provider is currently experiencing technical issues resulting in some Keycafe Smartboxes appearing offline. They have identified the root cause of the outage and has reported that their systems are returning online, however, the service continues to be degraded across their platform. We will continue to monitor the issue.
Report: "Website and app down due to Heroku (provider) platform Issues"
Last updateThis incident has been resolved.
Heroku (provider) has identified the issue and their Engineering are working to resolve the issue as soon as possible. Keycafe Website and app is not partially operational.
The Keycafe website and app is experiencing an outage due to a platform issue reported by or service providers Heroku. Heroku is currently investigating the matter.
Report: "Service provider outage"
Last updateThis incident has been resolved.
Majority of our Smartboxes have returned to an active status. The few that's left are expected to produce an updated online status.
We are still experiencing a partial outage of our Smartbox network due to a major service provider incident. We will continue to monitor the situation and post updates accordingly.
Some of SmartBoxes are currently being affected by a service provider outage incident. We are investigating.
Report: "Database maintenance"
Last updateKeycafe.com went offline due to database maintenance. The maintenance has been completed. The actual downtime was 3:31 minutes.
Report: "Connection timeouts with tentative email and third party integration failure"
Last updateHeroku has resolved the issue.
We are continuing to work on a fix for this issue.
Latest update: Our service provider Heroku is reporting that dynos in the Cedar US region may be experiencing issues with outbound networking calls. This is causing degraded service to some Keycafe platforms. Status update: https://status.heroku.com/incidents/1896
We are continuing to work on a fix for this issue.
Heroku's Engineers are investigating potential platform Issues. The link to their status page update posted below. We are seeing connection timeouts to both our IP address geolocation service. https://status.heroku.com/incidents/1896
Report: "Amazon Web Services (AWS) outage"
Last updateThis incident has been resolved.
We are continuing to work on a fix for this issue.
Our platforms have regained functionalities and users are now able to do key exchanges tentatively. Degraded performance is still seen in our mobile application.
Keycafe is experiencing a major platform outage due to the Amazon Web services outage. Amazon Web Services (AWS) is a subsidiary of Amazon that provides on-demand cloud computing platforms to individuals, companies, and governments. This outage has affected our Smart Boxes as well as the performance of our website and app. We are monitoring the situation.
Report: "The server update - slow response"
Last updateThe issue has been resolved and all functions have returned to normal.
We will revert to exclude the archiving feature within a few minutes.
We are continuing to work on fixing this issue.
The new archiving feature is causing slow responses and timeouts on the access list
Report: "Degraded Performance"
Last updateSome customers experienced errors attempting to access the website or perform key exchanges. Many other users reported slow response times. All errors were resolved within one hour.
Report: "Partial Outage"
Last updateSome customers experienced errors attempting to access the website or perform key exchanges. All errors were resolved within 10 minutes.
Report: "Partial Outage"
Last updateSome customers experienced errors attempting to access the website or perform key exchanges. All errors were resolved within 10 minutes.
Report: "Partial Email Outage"
Last updateEmail notifications failed to send to some customers for approximately one hour.
Report: "Scheduling plugin error"
Last updateOur logging service reported a minor plugin failure which set us offline from 10:43:55 to 10:47:48 PDT.
Report: "Webhooks to some API users failed"
Last updateAfter a routine upgrade, webhooks to some API users failed. The upgrade has been reverted and notifications are delivering again. All event failed tries were ultimately sent.
Report: "Guest journey loading issues"
Last updateA release was pushed in order to fix Guest Journey loading issues. Estimated downtime was two minutes.
Report: "Mobile number verification issue fix"
Last updateAn update was pushed in order to fix a mobile verification issue. Estimated downtime was two minutes.
Report: "User's app sending access searches in rapid succession"
Last updateWe experienced an outage October 3rd 1:20 pm EST. It appeared to have been caused by one user's app sending in access searches in rapid succession. The issue resolved itself when the requests stopped.
Report: "Support Issues"
Last updateA brief maintenance was required to address the problem.
Keycafe support is unable to help customers with problems related to location inventories.
Report: "API Access Search Errors"
Last updateA brief maintenance was required to address the problem.
Searching for accesses through the API is failing for some users.