Historical record of incidents for Verkada Command
Report: "Unable to link Verkada Data Plans to Gateway devices."
Last updateWe are currently investigating an issue where Verkada Data Plans cannot be linked to Verkada Gateway devices within Command.
Report: "Issues with notification delivery"
Last updateWe are currently experiencing issues with some notifications being delivered. Our engineering team is investigating and we will provide updates soon.
Report: "Issues loading Command"
Last updateWe are currently investigating this issue.
Report: "Some devices are unable to connect to Verkada's backend"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Some devices are unable to connect to Verkada's backend"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Database Maintenance in US Regions"
Last updateThe scheduled maintenance has been completed.
Scheduled maintenance is currently in progress. We will provide updates as necessary.
On Saturday May 24th, at 9:00PM until 10:00PM PDT, we will be performing maintenance on several of our databases in our US Regions. This maintenance will affect the following features:Notifications & Events: May be temporarily delayed for up to 30 minutes.Sensors: Accessing the sensors page in Verkada Command will be temporarily unavailable. Devices will still be recording data.Guests: Guest check-ins will be temporarily unavailable during this period.Mailroom: Accessing mailroom will be temporarily unavailable during this period.Gateway: Temporarily unable to make changes to gateway configuration. Devices will continue to work with existing configuration.Alarms: Temporarily unable to access the Verkada Alarms page.Device Claiming: Temporarily unable to claim new devices to Verkada CommandThis maintenance applies to both our Commercial and Gov regions in the US.We expect this maintenance to take between 15 and 30 minutes to complete.
Report: "Problems adding devices to Command"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating issues with adding new devices to Command.
Report: "Pass App for iOS - Unable to unlock via bluetooth"
Last updateThis incident has been resolved.
Apple has approved v5.1.2 of Verkada Pass for iOS. Please check the App Store for updates.
A new verison of the Pass App for iOS has been uploaded to the App Store and is awaiting review.
The issue has been identified and we are working on a fix. Customers experiencing this issue can still resolve it by launching the app and pulling down to refresh.
We are investigating an issue with Pass App for iOS 5.1.1 which results in users being unable to unlock doors via bluetooth. Launching the app and "pulling down to refresh" should resolve this issue.
Report: "Prod2 region (Europe): Verkada Pass for iOS v5.1 users are unable to login"
Last updateVersion 5.1.1 is now available in the Apple App Store. Affected users should visit the App Store and update to v5.1.1.
The issue has been identified. A new version of the Pass App for iOS has been submitted to the App Store and is awaiting review.
We are currently investigating this issue.
Report: "Issue Loading Command in Gov Region"
Last updateThis incident has been resolved.
We are investigating and issue loading Command in the Gov region.
Report: "Archives not completing"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have identified an issue preventing archives from completing and are working to fix the issue as soon as possible.
Report: "Issue Arming/Disarming in New Alarms"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating an issue causing users to be unable to arm/disarm camera partitions in New Alarms.
Report: "H.265 transcoding failing on some devices"
Last updateThis incident has been resolved. If you are still seeing issues, please hard refresh Command in your web browser (Ctrl + Shift + R on Windows or Cmd + Shift + R on Mac).
A fix has been implemented and we are monitoring the results.
We are currently investigating an issue where h.265 transcoding is failing on some devices.
Report: "Issue Loading Command"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are investigating an issue preventing Verkada Command from loading.
Report: "Delay in disarming Classic Alarm sites in European region Organisations."
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue. Alarm sites successfully arm and disarm in these organisations; however, we have observed behaviour in which consoles and keypads may time out during the operation of disarming a site, despite the site successfully disarming.
Report: "Cameras falsely reporting as offline"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are investigating an issue where Cameras and Intercom devices are reporting as offline, however, they continue to stream and are otherwise fully functional.
Report: "Guest cannot integrate with MS Teams"
Last updateThis incident has been resolved.
Engineering has been notified and is currently working toward a resolution. We will provide additional updates on this page as they become available.
Report: "Unable to create new Command organizations"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "SMS Delivery Issue"
Last updateThis incident has been resolved.
We have identified an issue preventing SMS messages from being delivered and are working to fix the issue.
Report: "PoI Creation Issue"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have identified an issue preventing users from creating person of interest profiles and are working to fix the issue.
Report: "[Access Control] AD32/AD33 Bluetooth approach distance was not being honored (newly created doors only)"
Last updateFrom 19 Dec 2024 to 30 Dec 2024, if a NEW door was added with an AD32 or AD33 reader, the Bluetooth approach distance was not honored. This is now resolved.
Report: "Intermittent issues loading Command"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have had reports of intermittent issues loading Command. This is being actively investigated.
Report: "Unable to view Access Control User List"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "List of Access Controllers in a specific site not available"
Last updateThe incident has been resolved.
When filtering by a site, a list of controllers returns "Something went wrong."
Report: "Viewing Station Issue"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have identified an issue preventing viewing stations from showing up in Verkada Command, and are working to address the issue. We expect to have a resolution within 15 minutes.
Report: "User Permissions Issue"
Last updateThis incident has been resolved.
We have identified an issue preventing permissions changes to users and groups from saving correctly. We are working to fix the issue and will provide an update shortly.
Report: "Access Control tailgating detection is not operational"
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "Delays in SMS deliverability"
Last updateThis incident has been resolved.
We have identified an issue causing delays in SMS deliveries and are working with our vendor to resolve the issue. Customers utilizing SMS notifications are encouraged to switch to email or push notifications, which are not affected by this issue. For SMS Two-factor, we recommend switching to Passkey or TOTP authentication. Help articles to configure these recommendations can be found here: https://help.verkada.com/en/articles/6203427-camera-status-alerts https://help.verkada.com/en/articles/3831845-enable-2fa-for-your-command-account
Report: "Degraded Streaming Issue"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have identified an issue causing degraded streaming performance and are working to fix this issue.
Report: "Issue Loading Command"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating an issue preventing some users from loading Verkada Command.
Report: "Issue Loading Command"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating an issue preventing some users from loading Verkada Command.
Report: "Issues with New User Invites, Password Reset & Magic Link login in EU & AU region Organisations."
Last updateThis incident has been resolved.
We have identified an issue preventing newly added users from logging In, Magic Link login & password resets from operating as expected. We are working to release a hotfix for this issue and will provide an update shortly. This only affects EU and AU region organisations.
Report: "Issue Loading Verkada Command"
Last update**Incident Summary** On November 6th at 7:00 PM PST, our automated monitoring system detected a service degradation impacting multiple services, leading us to declare an internal incident. During this period, users encountered instability in the Command application and API, with some unable to access the Command application at all. **Root Cause** The incident was caused by a surge in load on a permissions database. This spike originated from a suboptimal database operation within our regular release process, which led to excessive lock contention on the database. As a result, database queries and numerous internal requests were blocked or delayed, causing instability across the Command application and API. **Resolution and Mitigations** We quickly identified the source of the elevated database load and have since disabled the responsible operation. To prevent recurrence, we are: 1. Optimizing and refining the specific database operations to improve efficiency. 2. Implementing additional scope restrictions on these operations to minimize database lock contention during high-load scenarios. **Next Steps** We are conducting a broader review of our database operations in the release process to ensure stability under all conditions. Additionally, further adjustments to monitoring thresholds and alerts are underway to enhance early detection and prevent similar issues from impacting our users in the future.
We have resolved an issue causing some users to be unable to load Verkada Command. The erroneous release has been reverted.
Report: "Issues with Alarms Cellular Backup Modules connecting."
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "Issues with Intercom calling via Command."
Last updateThis incident has been resolved.
We are currently observing issues with Intercom calls causing Command Web to display a black screen. The current workaround is to use the Pass App where possible.
Report: "Users unable to login to Command"
Last updateA recent deployment of the licensing API resulted in a number of organizations being locked-out due to invalid licensing state. These organizations would have been identified as invalid by having too few licenses for the number of active devices in their organizations. This issue has been identified and resolved.
Report: "Events Delays"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating an issue causing increased delay in event processing, leading to delays in alert inbox, occupancy trends, and AI search.
Report: "Delay with Access Control door events"
Last updateThis incident has been resolved
Less than 1% of our customers are still experiencing issues. We are continuing to monitor.
We continue to work through our backlog of events. Most customers should see delays of 15 minutes or less.
Customers may experience a delay of up to 1 hour for Access Control door events. We’ve identified the issue and are working on a resolution.
Report: "Sensor alerts not sending or populating events page"
Last updateThis incident was due to misconfiguration of one of the services responsible for delivering alerts for Environmental Sensor’s events. During the outage, alerts generated by sensor devices did not get delivered to downstream systems responsible for displaying events on Command UI and sending email, SMS and push notifications out to users. We have fixed the incorrect configuration and all systems are functioning as expected. We will prevent such issues in the future by adding tests and monitoring relevant metrics to detect problems before impacting customers. Specifically we will monitor alert propagation from service to service and raise alarms when the sending of alerts falls outside of normal traffic patterns.
This incident has been resolved.
We are currently investigating this issue.
Report: "Unable to load Access Control Settings Page"
Last updateA hotfix has been deployed and the issue is resolved.
The issue has been identified and a hotfix is being implemented
Report: "User deletion delays"
Last updateThis incident has been resolved.
We are continuing to work on a fix for this issue.
We are continuing to work on a fix for this issue.
We've identified an issue where Command user deletion requests are delayed and are working to address this.
Report: "Guest background check partial outage"
Last updateThis incident has been resolved.
We have identified an outage with a 3rd party provider preventing background checks in Verkada Guest from succeeding. We will continue to monitor this as provide an update as soon as the functionality is restored.
Report: "Guest background check outage"
Last updateThis incident has been resolved.
We have identified an outage with a 3rd party provider preventing background checks in Verkada Guest from succeeding. We will continue to monitor this as provide an update as soon as the functionality is restored.
Report: "Updates to users not being reflected in Command"
Last updateThis incident has been resolved.
A fix has been implemented and user information is now being accurately reflected.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Issue loading Verkada Alarms"
Last updateThis incident has been resolved.
We have identified an issue preventing users from loading Verkada Alarms, and are actively working to mitigate this issue.
Report: "Intermittent issues loading command"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are currently investigating this issue.
We are continuing to investigate this issue.
We are aware of an issue preventing some users from loading Verkada Command. We are actively investigating this and will provide updates shortly.
Report: "Command is showing incorrect lockout status"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Camera statistics are erroneously reporting as offline"
Last updateCameras are erroneously reporting as offline in device statistics
Report: "Camera statistics are erroneously reporting as offline"
Last updateCameras are erroneously reporting as offline in device statistics
Report: "Newly created Access Users are not showing on the Access Users page in Command"
Last updateThis incident has been resolved.
A fix has been implemented and updates/new users are now being displayed.
The issue has been identified and we are working on a resolution.
Users are being created, but are not being displayed. We are currently investigating.
Report: "Access Control User updates slow to appear in Command"
Last updateThis incident was caused by an accidental underprovisioning of one of the services used to process user updates and propagate them to the low-latency storage used to display user profile and user list in Command. During this outage, all changes made to users \(for example access suspensions, card assignments, etc.\) actually were persisted and propagated to devices, so devices were functioning properly. However, changes were showing up in Command only after a long delay. The underprovisioning was caused by an incorrect deployment configuration for the affected service. We have addressed this issue by fixing bug in the deployment configuration. We will prevent future instances of this issue by adding monitoring for the deployment configuration. That work is in progress.
This incident has been resolved.
A fix has been implemented and we are monitoring the results.
The root cause seems to have been an underprovisioning of one of the services that processes user updates. We have increased the provisioning of this service and the user profile update latency seems to have improved to expected values. We are monitoring to see if the improvement is consistent and durable.
We are currently investigating this issue.
Report: "Issues with Intercom Calls not being delivered to T-Mobile phone numbers"
Last updateAnalytic engines in the US are increasingly labeling phone numbers as spam. We are working with trust systems to prevent these incorrect labels.
We’ve mitigated all known customers impacted by this degradation and will continue to monitor for new occurrences. We will continue to work on a root cause and final resolution with our 3rd party telephony provider. Please contact support if you experiencing similar issues.
We are currently investigating issues with some Intercom calls not being delivered to T-Mobile phone numbers. Our engineering team is working on this now and will provide updates shortly.
Report: "Issue with Alarms webserver communication"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Report: "Issue claiming new cameras into EU region"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have identified the issue and are working to deploy a hotfix for this issue.
We are currently investigating an issue where newly claimed cameras are not coming online in the EU region.
Report: "Intermittent issues with Command"
Last updateThis incident has been resolved.
Our engineering team has implemented a fix and is monitoring the results.
We are investigating intermittent issues with Command logins and device claiming. We will provide updates as soon as possible.