Is TrackTik Down Right Now? Check if there is a current outage ongoing.

This incident is resolved.

The platform has been restored, and we are actively monitoring the recovery process.

We detected a service-impacting incident at 4:30 PM ET, in the US-West region (California) which our team is actively addressing. The impact is limited to customers in the West coast region of the continental US. The service is in the process of recovering and is expected to resume shortly.

A hotfix has been deployed to mitigate the underlying cause of this issue.

US West infrastructure is now operational again. We are continuing to investigate this issue.

Customers on US West infrastructure may experience problems loading their portals. We are currently investigating the issue.

Our services are now operating normally.

We are continuing to investigate this issue.

Customers on US West infrastructure may experience problems loading their portals. We are currently investigating the issue.

502 Bad Gateway errors have been resolved.

We are continuing to investigate this issue.

We are currently investigating reports of 502 Bad Gateway errors for clients on Infr3.

The DNS has been successfully propagated across all regions, and the service is functioning normally. If you are still having trouble accessing your portal, consider clearing your browser cache or flushing your computer's DNS. Additionally, restarting your phone may help trigger a refresh.

DNS propagation is underway, and portals should now be accessible in most, if not all, regions. This delay occurs due to the DNS propagation process, during which DNS information is updated across the internet. The duration of this delay may vary depending on your internet provider. You can try clearing your browser cache or restarting your phone to prompt a refresh.

We have identified a significant issue affecting the DNS service for the EU domain. Our team has taken corrective actions, and the service is now in the process of being restored. The restoration will occur gradually across different regions. This means some users may experience normal service sooner than others, depending on how quickly the changes propagate through the global DNS network.

A partial disruption occurred for a few minutes for customers on our Asia infrastructure.

A new feature was reverted, rolling back to provide full functionality to Runsheets.

We are continuing to investigate this issue.

A recent fix and deployment were expected to resolve an issue with Runsheets, but it has actually caused further problems. Some clients have reported that runsheets for their mobile users have disappeared since last week, particularly when the runsheet spans over two days. The issue has been identified, and we are currently working on implementing a more permanent fix.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

At approximately 13:15 EST today, we experienced a service disruption that may have affected a limited number of customers in the North America The incident root cause is still under investigation; initial analysis indicates a database performance issue for some requests that caused a disruption of service. During this time, the web portal may have been slow or unresponsive. For any questions or concerns, please do not hesitate to contact our customer support team.

This incident has been resolved.

A few clients may experience issues since last week where Runsheets are disappearing for their Mobile users. It seems to happen when the Runsheet spans over two days The issue has been identified and we are currently working to restore the functionality.

We have experienced a service disruption that has affected our users in our LATAM region from 10:16AM to 10:49AM ET.

This incident has been resolved.

We're currently experiencing issues with SMS Shift Offer responses no longer being received for some customers. We're working to restore services as soon as possible.

Services have been restored.

We are currently experiencing a service disruption that is affecting users in North America East. Our engineering teams are working to get services restored as soon as possible.

This incident has been resolved.

Our upstream provider, Twillio, is currently experiencing an issue that is affecting our ability to send SMS to some customers. We will resolve this incident once we have received confirmation that our upstream provider has resolved the issue.

This incident has been resolved.

We are continuing to monitor for any further issues.

A fix has been implemented and we are monitoring the results.

At approximately 13:22 EST today, we experienced a service disruption that may have affected a limited number of customers in the North American regions. The incident's root cause is still under investigation; initial analysis indicates a database performance issue for some requests, bringing the system to an impacting service disruption. During this time, the web portal may have been slow or unresponsive. For any questions or concerns, please do not hesitate to contact our customer support team.

This incident has been resolved.

Progress has been made in addressing the ongoing issue with outgoing SMS messages and it's filtering. Two-way communication (for example shift offering, or schedule verification) has been resolved. One way communication (for example clock-in and report creation notifications) has improved and we are continuing to work to resolve it with our service partner.

We are currently experiencing reduced SMS functionality due to messages being blocked by US carriers (T-Mobile). This negatively impacts shift offerings by SMS and may impact other SMS functions. We are currently working to resolve this issue with our service partner and will continue to update.

This incident has been resolved. We sincerely apologize for the performance issues that our clients experienced today. We understand how frustrating that can be. We want to assure you that our team is actively investigating the situation to prevent future occurrences. Even after the final resolution was put into place today around 18:06 EST To give you a quick summary of what happened, we detected a high load on our servers. To prevent any further issues and ensure the smooth functioning of our services, we performed system and integrity checks that required us to suspend access for short periods. Hence, the maintenance pages are posted on your portals when you attempt to log in. Once again, we apologize for any inconvenience this may have caused. We truly appreciate your patience and understanding. For any technical issues, do not hesitate to contact the support team. Please do not hesitate to contact your Client Success managers should you require further detail than outlined here. Regards, Team Support

A fix has been implemented, and we are monitoring the results.

We are experiencing a service disruption that may have affected a limited number of TrackTik Cloud customers. The incident's root cause is still under investigation; initial analysis indicates a database performance issue for some requests, causing the system to impact service disruption. The web portal and the Shift App may be unresponsive during this time.

Services are now fully operational again.

Please be aware of performance degradation currently affecting Canadian clients. We'll provide an update once services are fully operational.

Services have been restored.

We are currently experiencing a service disruption that is affecting users in Canada. Our engineering teams are working to get services restored as soon as possible.

This incident has been resolved.

We are currently experiencing a service disruption that may be affecting report PDF generation, data replication and email notifications. Our Engineering team is working to identify the root cause and implement a solution. Users in the US East may be affected.

This incident has been resolved.

At approximately 10:30 AM EST today, we experienced a service disruption that may have affected a limited number of customers utilizing the Run sheet features. Initial analysis indicates a database performance issue for some requests bringing the system to an impacting service disruption. Which has since been resolved During that time, the web portal and mobile apps may have been slow or unresponsiveness to accessing the Run sheets For any questions or concerns, please do not hesitate to contact our customer support team.

The scheduled maintenance has been completed.

We would like to inform you that the Scheduled Maintenance on March 9th, 2023 at 10:15AM EST is taking more time than expected and has been extended. We are working to re-establish services as soon as possible. Thank you for your patience. Please note, this maintenance only affects our VIRGINIA-1 and VIRGINIA-2 clusters. What to Expect -------------- During this maintenance window, users will not be able to access the web-application (website), and features that require a connection will not be available in the TrackTik Guard Tour mobile application. Some examples of the affected features are: - Signing and clocking in - Changing sites/zones - Dispatch/Runsheet completion - Notifications - Viewing Post Orders and Message Board Activities recorded on the mobile application during this period will be uploaded to the server once the connection is re-established. Visit status.tracktik.com for updates or to sign up for maintenance and incident alerts.

This incident has been resolved.

We are currently experiencing a service disruption. Our Engineering team is working to identify the root cause and implement a solution. Users in the US East may be affected. We will send an additional update in 10 minutes.

This incident has been resolved.

At 11:06 am EST today, we experienced a service disruption of 6 minutes that may have affected a limited number of customers in the North American regions. A change in the backend services took an unexpected time to complete. At 11:12 am EST, our Engineering team completed the return of the change; It took the server more time to return to a normal operating state than usual. All systems are back online, and users can access them. All data is safe and restored. If you have any issues logging back in, you can reach our support staff at support@tracktik.com.”

This incident has been resolved.

We are continuing to work on a fix and have identified the issue. A fix is being deployed shortly.

At approximately 16:50 EST today, we experienced a service disruption that may have affected a limited number of customers utilizing the Guarding Suite. The incident's root cause is still under investigation; initial analysis indicates a database performance issue for some requests bringing the system to an impacting service disruption. During this time, accessing zones and certain functionalities may be diminished. For any questions or concerns, please do not hesitate to contact our customer support team.

This incident has been resolved.

We are continuing to investigate this issue.

Numbers calling from Europe may experience issues with IVR. An investigation is currently ongoing.

This incident has been resolved. Amazon AWS closed their incident at 10:24 PM PDT.

Situation seems stabilized and no errors have been reported recently. We are monitoring the situation until our hosting provider close the incident.

We have been informed of an ongoing incident with our hosting provider Amazon AWS in their US-East region caused by a third-party DNS provider. The DNS service being a critical networking component of the internet, this incident causes internet traffic to drop and degrade access to our web application. Amazon AWS is actively working on resolving the issue.

Between 5:07 PM to 5:28 PM EST today, an incident with our replication services made portal access inaccessible for some users in the Eastern US region. Our Engineering team identified and restored services without any loss to user data. The root cause is still under investigation. Once we have more details, we will share them.

This incident has been resolved.

We are now monitoring our App restore in the GooglePlay store. A temporary measure has been put in place. The following link will allow for a direct download of the TrackTik Guard tour app: https://bit.ly/3LMjeFp If prompted select: 'Allow install from unknown sources'

A temporary measure has been put in place. The following link will allow for a direct download of the TrackTik Guard tour app: https://bit.ly/3LMjeFp If prompted select: 'Allow install from unknown sources'

We are continuing to work on a fix for this issue.

An issue has been identified in our Playstore app and we are attempting to restore the ability to locate and install the app. Updates will be provided here as we receive them.

Amazon AWS report an outage in their hosting region US-East-1 impacting only a specific availability zone (AZ). We are not observing any impact on our services and we are monitoring the situation.

Amazon AWS report that they resolved their internet connectivity outage and that the internet traffic is back to normal. We are seeing that our service is also back online for our West coast region.

Amazon AWS is currently working on fixing the situation.

We are currently experiencing internet connectivity disruptions in our US West Amazon region (California) which is linked to the reported Amazon AWS Oregon network issues currently happening. The interruptions are intermittent, we are monitoring the situation and are in contact with Amazon AWS. Not ETA is available from Amazon yet.

This incident has been resolved.

We are continuing to monitor for any further issues.

From Amazon AWS: [4:35 PM PST] With the network device issues resolved, we are now working towards recovery of any impaired services. We will provide additional updates for impaired services within the appropriate entry in the Service Health Dashboard.

Amazon AWS our hosting platform is experiencing a localized networking incident in their US-East region. The issue affects multiple AWS services and our clients may experience degraded service with our portals hosted in the US-East region.

This incident is closed.

Twilio started seeing recovery for inbound calls to Twilio Canada Toll Free phone numbers. We will continue to monitor to ensure a full recovery.

Twilio is still reporting some call failures with Canadian toll free phone numbers. Twilio SMS delivery delays are limited to shortcodes on multiple carriers networks. TrackTik is using 10-Digit Long Code SMS notifications and is not impacted by those SMS delivery delays.

The issue is linked to the service of our sub-processor Twilio.

We have reports of partial outages with our sub-processor Twilio on IVR and SMS services in Canada and for Canadian Toll-Free numbers.

Our Engineering team identified the root cause and resolved the issue.

A fix has been implemented and we are monitoring the results.

We are currently experiencing a service disruption in our US-West Region (California). Our Engineering team is working to identify the root cause and implement a solution.

The issue has been resolved and connectivity has been restored. Between 8:19 AM and 10:59 AM PST Amazon AWS experienced internet connectivity issue with an external provider outside of their network. This impacted internet traffic both to and from the TrackTik Asia-Pacific Region to internet destinations outside Australia. Connectivity from locations within Australia to our Asia-Pacific Region was not impacted

Amazon AWS is continuing to work towards shifting traffic from the external provider to recover from the event. TrackTik portal may experience intermittent outages.

Amazon AWS Sydney is experiencing connectivity issues with an internet provider external to their network. This affects international connectivity from their South East Pacific Region to internet destinations outside Australia. Connectivity within the Australian Region or to internet destinations within Australia is not impacted at this time.

At approximately 4:15 a.m. EST, we experienced a service disruption of 15 minutes that may have affected a limited number of customers in the U.S. regions. The incident's root cause is still under investigation; initial analysis indicates a database performance issue for some requests bringing the system to an impacting disruption of service. During this time, the web portal may have been unavailable, and mobile devices will have uploaded data once the connection has been re-established. For any questions or concerns, please do not hesitate to contact our customer support team.

A change in the time zone code was deprecated causing alerts in our system.

This incident has been resolved.

The issue has been identified and a fix is being implemented.

We are currently experiencing a service disruption with a server and require a reboot that will cause an unexpected downtime of a few minutes. Our Engineering team is working to identify the root cause and implement a solution. Multiple users may be affected. We will send an additional update within the next 15minutes.

On 2020-08-19 approximately @ 7:45 PM EST we experience internal server issues where some client could not access their portal for approximately 15 mins. All systems have been restored and no loss of data occurred.

We are continuing to investigate this issue.

We are continuing to investigate this issue.

We are currently experiencing a service disruption. Our Engineering team is working to identify the root cause and implement a solution. Some US regional portal users may be affected. We will send an additional update in 15 minutes.

We have completed the restoration of the databases and resolved the issue.

We are finalizing the complete restoration of the databases and should have an update and complete resolution shortly.

We are continuing to work on a fix for this issue.

Rollback of the database prior to the maintenance has commenced you may have some discrepancies with activities performed in the last 3-4 hours. We are continuing to roll back and will update on the status

We are currently experiencing a service disruption in our Asian and European region following our database maintenance at 2 PM EST and are working on resolving them We will send an additional update in 15 minutes.

This incident has resolved itself. We suspect discrepancies with our AWS cloud hosting services. For more details or information please reach out to support@tracktik.com

The issue has been identified and we are working to resolve it. The slow performance on some portals comes from the initial download of web assets (js, css files) while browsing in "incognito" with a cleared cache. However, when you access those files directly without incognito, you get normal performance.

We are investigating an issue with the Web Portal Performance that is impacting users on the West Coast infrastructure. You may experience long load times when first attempting to log on to your portal. Next update in 30 minutes.

The error is fixed, and the jobs are executing again. The problem was caused by our cron job server that stopped executing the jobs. we located a misconfiguration in one of the agents that may have caused a temporary failure.

A fix has been implemented and we are monitoring the results.

Around 6:40 PM EST today we have identified delays in message delivery, and are actively investigating. Messages are currently QUEUED and will Resume the items in the queue once we resolve the issue. We will send an additional update in 60 minutes.

on approximately 10:19PM EST PROBLEM: A server instance related to the Canadian region became unresponsive. Our logs indicate that our read database failed and the failover server to manage any perceived downtime also failed. We have identified a flaw in the code logic, and our Engineering team was able to quickly restore the services at 10:41PM EST. We are working directly with Amazon Web Services technicians to identify the root cause and ensure we can trust in the failover system.

A database update caused a temporary issue with some client portals. We are back up 100%

We are continuing to investigate this issue.

We are currently experiencing a service disruption. Our Engineering team is working to identify the root cause and implement a solution. Some Users attempting to login may be affected. We will send an additional update in **15min** minutes.

This incident has been resolved.

A fix has been implemented and all services should now be restored

The issue has been identified and a fix is being implemented.

We are continuing to investigate this issue.

We are continuing to investigate this issue.

We are continuing to investigate this issue.

We are currently experiencing a service disruption. Our Engineering team is working to identify the root cause and implement a solution. **Europe/ North American Infrastructure** users may be affected. We will send an additional update in **15** minutes.

Notifications are now being sent on time. The incident is now closed.

We are experiencing delays in email notifications and report delivery. Though the problem has been identified and resolved, notifications may arrive late for some time.

This incident has been resolved.

We have completed our monitoring and verification and all systems normal. For more info and details please reach out to Support @ support.tracktik.com

We are currently experiencing a service disruption. Our Engineering team is working to identify the root cause and implement a solution. All Clients on our European infrastructure users may have been affected. Downtime had occurred between 11:05 EST - 11:20 EST Everything should be operational now. For more info and details please reach out to Support @ support.tracktik.com

This incident has been resolved.

We are currently experiencing a service disruption. There was an unexpected delay in restoring the Canadian Infrastructure Our Engineering team is working to identify the root cause and implement a solution. We will send an additional update in **10** minutes.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We're experiencing an issue with the Dispatch+ Module (unable to edit reports in view) and are currently looking into the issue.

Clients may not have been able to open PDF attachments sent today between 10AM EST and 2PM EST, at which time the issue was identified and resolved. Clients can retrieve their reports directly using their Client Portal access or have the report manually resent by an administrator or manager of their account from the Operation Reports Dashboard. Please contact Support if you require assistance or clarifications.

Clients with ".net" and ".com" domains may have experienced brief connection issues today between 15:20-15:40. An error occurred that may have disabled access to services that generated a connection error. All services have been restored and are fully functional.