Historical record of incidents for Tenant Turner
Report: "CodeBox code generation outage"
Last updateYesterday we observed a marked improvement in the ability to generate codes and schedule viewings for CodeBox lockboxes, however a very small number of lockboxes did still experience issues. As yesterday evening all boxes were no longer experiencing errors. Please contact support if you continue to notice disruptions in Self-Access viewings.
At this time codes are being generated as expected, however we are awaiting official confirmation from CodeBox Inc. that the outage has been resolved.
CodeBox, Inc. is currently experiencing a code generation error, thus Tenant Turner is unable to generate new one-time codes at this time. If a self-access CodeBox viewing was already scheduled before this outage, tenant leads should still be able to access the rental with their one-time code. Requests for new codes that cannot be generated while the CodeBox API is down will be put into a queue. Once the CodeBox API has recovered, prospective tenants will be notified to schedule their viewings and get new codes. CodeBox acknowledged the outage and is working to resolve the issue.
Report: "Services unavailable"
Last updateWe appreciate your patience during this system outage. Everything is resolved and you should have full access to the site at this time.
We are aware of a wide scale system outage. During this time you may not be able to login to your account and will see an 'Our services aren't available right now' error page. The team is working towards a swift resolution. Currently scheduled appointments will not be affected but leads my experience delays or slowness while scheduling.
Report: "Microsoft Azure is currently experiencing intermittent outages"
Last updateThe incident was resolved in the evening hours of July 30th.
Microsoft Azure, the network that hosts Tenant Turner's servers, is currently experiencing intermittent outages. Access to Tenant Turner is limited at this time in some locations, and lead phone calls to your Tenant Turner phone number may also be impacted. We will continue to monitor the situation and send a notice with the servers are back up and running.
Report: "Microsoft Azure having widespread issues"
Last updateMicrosoft Azure now seems to be in a stable place and systems are back up and running. If a tenant lead had trouble scheduling through Tenant Turner or accessing a rental for a scheduled appointment, you may need to reach out to help reschedule them. Our support team is here to help!
Microsoft Azure, the network that hosts Tenant Turner's servers, is currently experiencing widespread outages. Access to Tenant Turner is limited at this time and lead phone calls to your Tenant Turner phone number may also be impacted. We will continue to monitor the situation and send a notice with the servers are back up and running.
Report: "SentriLock API Outage"
Last updateThis incident has been resolved.
SentriLock is currently experiencing a code generation error, thus Tenant Turner is unable to generate new one-time codes at this time. If a self-access viewing was already scheduled before this outage, tenant leads should still be able to access the rental with their one-time code. Requests for new codes that cannot be generated while the SentriLock API is down will be put into a queue. Once the SentriLock API has recovered, prospective tenants will be notified to schedule their viewings and get new codes. SentriLock acknowledged the outage and is working to resolve the issue.
Report: "CodeBox API Outage"
Last updateCodeBox Inc. let us know that new, one-time code generation was back up and running as of 2:57pm ET. Our team has tested and confirmed that new codes can be generated through Tenant Turner via the API. Prospective tenants will be notified to schedule their viewings and get new codes if they attempted to do so while the CodeBox API was down.
CodeBox, Inc. is currently experiencing a code generation error, thus Tenant Turner is unable to generate new one-time codes at this time. If a self-access CodeBox viewing was already scheduled before this outage, tenant leads should still be able to access the rental with their one-time code. Requests for new codes that cannot be generated while the CodeBox API is down will be put into a queue. Once the CodeBox API has recovered, prospective tenants will be notified to schedule their viewings and get new codes. CodeBox acknowledged the outage at 9:24am ET and is working to resolve the issue.
CodeBox, Inc. is currently experiencing a code generation error, thus Tenant Turner is unable to generate new one-time codes at this time. If a self-access CodeBox viewing was already scheduled before this outage, tenant leads should still be able to access the rental with their one-time code. Requests for new codes that cannot be generated while the CodeBox API is down will be put into a queue. Once the CodeBox API has recovered, prospective tenants will be notified to schedule their viewings and get new codes. CodeBox acknowledged the outage at 9:24am ET and is working to resolve the issue.
Report: "CodeBox API Outage Resolved"
Last updateCodeBox Inc. let us know that new, one-time code generation was back up and running as of 12pm ET. Our team has tested and confirmed that new codes can be generated through Tenant Turner via the API. Prospective tenants will be notified to schedule their viewings and get new codes if they attempted to do so while the CodeBox API was down.
CodeBox, Inc. is currently experiencing a code generation error, thus Tenant Turner is unable to generate new one-time codes at this time. If a self-access CodeBox viewing was already scheduled before this outage, tenant leads should still be able to access the rental with their one-time code. Requests for new codes that cannot be generated while the CodeBox API is down will be put into a queue. Once the CodeBox API has recovered, prospective tenants will be notified to schedule their viewings and get new codes. CodeBox acknowledged the outage at 10:43am ET and is working to resolve the issue.
Report: "Tenant Turner site down"
Last updateThis incident has been resolved.
Tenant Turner is up and fully operational.
We are continuing to investigate this issue.
The Tenant Turner site is currently down. Our team is investigating the issue. We will continue to share updates as we have them.
Report: "CodeBox API unable to generate new codes"
Last updateCodeBox identified the source of the non-responsiveness and corrected it.
The CodeBox API has resumed responding to code generation requests. While this issue is resolved, we'll continue to monitor. Please reach out to our support team if you experience anything unexpected. Thank you!
The CodeBox API is intermittently not responding, thus we are unable to generate one-time codes at this time. Codes that have previously been generated will be sent to prospective tenants. Requests for new codes that cannot be generated while the CodeBox API is down will be put into a queue. Once the CodeBox API has recovered, prospective tenants will be notified to schedule their viewings and get new codes. The CodeBox technical team is investigating.
Report: "CodeBox API Outage"
Last updateThis incident has been resolved.
CodeBox has released a fix for their API issue. We're seeing one-time use codes being generated again. Any prospective tenant who attempted to schedule a showing but was not able to get a code has received an email from Tenant Turner notifying them that they can schedule showings again. We'll continue to monitor and mark this issue as resolved if we do not see any other issues.
Earlier today the CodeBox API stopped responding, thus we are unable to generate one-time codes at this time. Codes that have previously been generated will be sent to prospective tenants. Requests for new codes that cannot be generated while the CodeBox API is down will be put into a queue. Once the CodeBox API has recovered, prospective tenants will be notified to schedule their viewings and get new codes. CodeBox acknowledged the outage at noon ET and is working to resolve the issue.
Report: "CodeBoxes currently unable to generate new codes"
Last updateThe issue with CodeBox generating new codes has been resolved. Please reach out to our support team if you experience anything unexpected. Thank you!
We learned that CodeBox lockboxes are currently unable to generate new codes. This issue impacts the box's ability to generate on-demand codes for vendors, agents, etc. and it impacts the scheduling of new CodeBox self-access appointments. It does not impact viewings that have already been scheduled. We will continue to update this page as we know more.
Report: "Connectivity slowness"
Last updateSystems have returned to normal operating capacity.
We are continuing to investigate this issue.
We received communication from Microsoft that multiple of their hosting customers are having similar server connectivity issues. We continue to monitor.
We are seeing slow server responsiveness and connectivity delays. Customers have reported receiving "This site can't be reached" messages. We are investigating and will provide updates as they are available.
Report: "Microsoft Azure Hosting Service Experiencing DNS Issues"
Last updateThis incident has been resolved. For more details on the Microsoft outage go to, https://techcrunch.com/2021/04/01/microsoft-outage-knocks-sites-and-services-offline/
A fix has been implemented and we are monitoring the results.
Tenant Turner hosting provider is currently having DNS issues that are preventing Tenant Turner from rendering.
Report: "CodeBox code generation failure 2020-04-22"
Last updateThis incident has been resolved.
CodeBox restored their API server, and we are able to once again generate codes. This issue is resolved.
At 2:26pm ET the CodeBox API stopped responding, thus we are unable to generate one-time codes at this time. Codes that have previously been generated will be sent to prospective tenants. Requests for new codes that cannot be generated while the CodeBox API is down will be put into a queue. Once the CodeBox API has recovered, prospective tenants will be notified to schedule their viewings and get new codes. CodeBox acknowledged the outage at 3:00pm ET and is working to resolve the issue.
Report: "CodeBox code generation failure"
Last updateCodeBox restored their API server, and we are able to once again generate codes. This issue is resolved.
The issue has been identified and a fix is being implemented.
At 8:15am ET we reported to CodeBox that we are unable to generate one-time codes because their API is not responding. Codes that have previously been generated will be sent to prospective tenants. Requests for new codes that cannot be generated while the CodeBox API is down, will be put into a queue. Once the CodeBox API has recovered, prospective tenants will be notified to schedule their viewing and get new codes. CodeBox acknowledged the outage at 9:00am ET and is working to resolve the issue.
Report: "CodeBox New Code Generation Issues"
Last updateThis incident has been resolved.
CodeBox has made a security change to their API which is temporarily preventing self access viewing codes from being generated. Codes that have previously been generated will be sent to prospective tenants. Requests for new codes that cannot be generated while the CodeBox API is down, will be put into a queue. Once the CodeBox API has recovered, prospective tenants will be notified to schedule their viewing and get new codes.
Report: "RESOLVED - Intermittent Phone and Website Access Issue"
Last updateThis incident has been resolved.
Website and phone systems are fully operational now. We are continuing to monitor for any further issues.
When did the issue occur? Wednesday, 11/6, from 7:22am EST to 5:26pm EST What Tenant Turner functionality was impacted? When leads called your Tenant Turner phone number they received an application error. Some leads were able to get through and receive text messages, but most did not. A small number of Tenant Turner users also had an issue logging in to Tenant Turner and viewing their Tenant Turner listings on their website. What occurred? On Tuesday, 11/5, we upgraded our web servers to improve the performance and speed of Tenant Turner. On Wednesday, 11/6, at 7:22am EST we started receiving errors in our phone system logs. Our team immediately started investigating the issue and reached out to several different providers for assistance. We found that the errors were originating from OpenDNS* whom had partially blocked one of our subdomains causing the issue to occur. We were able to rebuild a new web server to direct traffic there by 5:26pm EST, which solved the issue. OpenDNS also ended up unblocking our domain early Thursday morning (by 4:30a EST). What are we doing now? OpenDNS has yet to provide a reason for the partial block but it appears that they did so in error. We're working with OpenDNS to understand why the block occurred and ensure they don't block us again. We're also actively monitoring our error logs to know if anything were to happen again. If we see anything in those logs, an update will be posted here. *OpenDNS Global Network is a Cisco subsidiary that provides content filtering for mostly enterprise businesses. OpenDNS is not associated with Tenant Turner nor any of our infrastructure providers, but they are part of the backbone of the Internet. They process an estimated 100 billion internet traffic requests daily from 85 million users through 25 data centers worldwide, which is why our phone systems were impacted without warning.