Is Density Down Right Now? Check if there is a current outage ongoing.

This issue has been resolved. No data was lost during the Atlas Building Analytics outage.

Atlas Building Analytics page is down. We have found the cause of the issue and are working to resolve the issue.

The process to remove duplicate events has been completed. Events have been corrected for September 14 at 10:15 a.m. to September 16 at 11:25 a.m. EST.

The next round of data corrections from multiple Entry sensors will run on September 18 at 4 a.m. EST to avoid conflicting with peak hours.

We are continuing to work on a fix for duplicate event removal.

The process to correct count duplications between multiple sensors over a single doorway is expected to run overnight (EST). We will provide more information on resolution timing tomorrow, September 17th.

A data pipeline delay from doorways with multiple Entry sensors (s5b and ELR) occurred from September 14 at 10:15 a.m. to September 16 at 11:25 a.m. EST. The data is processing but will come in two rounds. First, you will see inflated counts due to duplicates from multiple sensors, and then the process to remove duplicates will be run. We recommend not using historical data from September 14-16 until this is resolved. Live data (Safe Displays) counts will be impacted by the missing and duplicated data.

We experienced an AWS IoT connection outage today from 12:05 a.m. to 11:05 a.m. EST. During this time period, our Live displays for affected spaces would have shown the space as grey. These devices would be reported as unhealthy via our public API. The issue is resolved.

The intermittent outages with Density applications have been resolved. We saw brief outages between 9:43- 9:44 a.m. and from 10:00 -10:05 a.m. EST. The root cause has been identified, and a fix has been deployed. No data was lost during this time, but APIs may have returned a 404 error and need to be rerun.

We are experiencing intermittent outages with our backend systems starting at 9:43 a.m. EST. This outage impacts Atlas and all APIs, but no data will be lost.

From 12 p.m. EST on June 24 - 11 a.m. EST on June 25, v2 experienced intermittent 404 errors. This was due to an issue on the backend that has been resolved.

The API documentation is back online. This did not impact any Density APIs, only the documentation.

Our API documentation is currently unavailable. We have discovered an issue with our hosting site for developers.density.io and are working with the partner to resolve it.

This incident has been resolved. If your organization was impacted by our OA historical data downtime, you will receive an individual email with the next steps for API backfills.

We are continuing to work on a fix for this issue.

A number of Open Area sensors stopped uploading count data beginning April 22, 2024. To prevent data loss, all sensors began storing count data in backup local storage. We are actively working to fix the upload issue and expect to have historical data fully restored. There was no impact to live or Entry Density data at any point.

This incident has been resolved. If you use the API to feed a data lake or other record of Open Area count data, you will need to backfill the data from April 24th if it was pulled on April 25th. Please email support@density.io if you need any assistance with this.

There is a backlog in our historical OA pipeline causing a delay in OA data in Atlas and the API. We are implementing a fix, and no data will be lost. Entry data is not impacted.

This incident has been resolved.

We're having an outage with our hosting site. The team is aware and is working to resolve it.

From December 12-20, webhooks were unavailable. This issue has now been resolved, and you do not need to do anything to resume receiving data. If you use webhooks to feed a data lake or other record of count data, you will need to backfill data from December 12-20, 2023. We recommend using the API query space events endpoint. If you need help downloading a CSV from the time frame, please let us know, and we can provide instructions.

When we cut over the live Entry data to our improved count database, the system ran into unexpected errors. This caused a backup in Entry count, which resulted in incorrect live count for about 20% of spaces. This issue was resolved at 4:30 p.m. EST. The only impact to historical data and APIs was a slight delay in arrival, resolved at 3:45 p.m. EST. There is no incorrect data populating in either location. The legacy Dashboard had a bug with timestamps. If you use the legacy platform for building-level data, this is now available. v2 APIs had a bug with representing timestamps on the response. If you ran into timestamp parsing errors on your end, you may have to re-pull data.

The Entry count backlog is currently clearing from the earlier outage. Until this is resolved, live data in Displays will be incorrect, and you may experience jumps in numbers. All v2 and v3 APIs are operating normally. If you use APIs to feed a data lake or other record of count data, and received API errors between 4-4:30 a.m. or 7-9 a.m., you will need to backfill your data. Not all customers received error messages, please check your API instance.

Density performed scheduled maintenance this morning at 3 a.m. EST for v2 API changes and fixes to Entry Live (displays). -Between 4-4:30 a.m. EST there was a full outage across the platform due to the v2 cutover. This was resolved at 4:30 a.m. EST. -Between 7-9 a.m. EST a limited number of customers may have experienced v2 API errors. This was resolved at 9 a.m. EST. -At 5:30 a.m. EST, we began to experience issues with Entry data, related to the fixes for Entry Live. This is ongoing; updates to follow.

We are currently investigating a large number of errors in Entry data.

We faced an hour-long outage for our Entry count data into Atlas and APIs due to an AWS issue. This has been resolved. If you use our v3 API to feed a data lake or other record of count data, you will need to backfill data from 12:09-1:04 EST. We recommend refreshing the data for that time frame using the same endpoints.

Our Entry counts API is down, which is impacting Atlas and APIs. We will update as we uncover the cause.

From November 14-20, webhooks were unavailable. This issue was resolved on November 20 at 5 a.m. EST. You do not need to do anything to resume receiving data. We have also put monitoring in place to prevent this from happening again. If you use webhooks to feed a data lake or other record of count data, you will need to backfill data from November 14-20. We recommend using the API query space events endpoint. If you need help downloading a CSV from the time frame, please let us know, and we can provide instructions.

We have identified a backlog in our webhooks, which is in the process of being cleared.

All systems have been restored and are operating normally.

Backend systems will be running overnight to restore normal functionality. All Entry data is available, but may experience delays or intermittent issues during this time.

We are currently experiencing downtime with our Entry live availability features, including Displays. There will also be a delay in Entry historical data until the issue is resolved. The cause has been identified, and we are working to resolve the issue.

This incident has been resolved. If you use the v2 API to feed a data lake or other record of count data, you will need to backfill the data during this outage. Please let us know if you need any assistance in this.

The Entry count system is down, impacting Dashboard, Displays and v2 count APIs. A fix has been implemented, and we are monitoring.

During a period from 9/5 to 9/9, a select number of API analytics endpoints did not return results for count data. Events (entrances / exits) were still available. No data was lost during this period. Live data (webhooks/websockets) was unaffected.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

Our database provider is currently experiencing an outage, and our analytics API has been affected. No data has been lost, however, space analytics are currently unavailable.

We are currently investigating this issue.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are continuing to investigate this issue.

We are continuing to investigate this issue.

We are seeing degraded performance retrieving data from the Counts API.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

We have enabled basic auth for Portfolio. At this time, we are awaiting the Auth0 incident to resolve to re-enable SSO. Please reach out to support@density.io if you need help.

We are continuing to work on a fix for this issue. Dashboard and Display have basic auth enabled. Please contact support@density.io if you would like to temporarily disable SSO.

We have re-enabled basic authentication (email/password) for customers of Dashboard. Portfolio and Display will be available via basic auth soon. SSO is currently disabled. We will post an update as soon as we have SSO re-enabled.

Auth0 incident available here https://status.auth0.com/incidents/zvjzyc7912g5

We are currently experiencing an issue with a 3rd party provider (Auth0) that provides authentication to the Density Dashboard. We will sharing more updates shortly.

During an automated minor version upgrade of our database, a set of configuration parameters became invalid. The parameters were updated to be consistent with a new localization spec for Postgres.

Reports on the Dashboard were unavailable from May 27 12:00am UTC until May 27 12:30am UTC. The team identified the issue, and pushed a fix to production. No data was lost during this period.

The Density API became unavailable at 11am UTC. The API is back to a stable state as of 1:40pm UTC. There was no loss of data.

This incident has been resolved.

We are continuing to monitor for any further issues.

We have identified the issue. We are currently monitoring the fix. Systems should be available shortly.

We are currently investigating an issue related to a database migration. The API and Dashboard are currently affected. We are still able to receive events from Density sensors and our websocket + webhook servers are operational.