Routific

Is Routific Down Right Now? Check if there is a current outage ongoing.

Routific is currently Operational

Last checked from Routific's official status page

Historical record of incidents for Routific

Report: "Google Cloud Provider downtime affecting Routific services"

Last update
investigating

One of our third party providers, Google Cloud Provider (GCP), is experiencing downtime and thus affecting some Routific services. Log in via Google SSO is not working at the moment, as well as the ability to schedule and optimize a route on the old and new Routific. There may be other services impacted. We are actively investigating and will post another update in 30 minutes time. Thank you for your patience.

Report: "Driver location not available"

Last update
identified

We are currently investigating an issue with our Driver Location Service that is affecting real-time updates in both the Routific Platform and the Delivery Tracker. As a result, some users may be unable to see their drivers location on the map.

Report: "Live tracking not working on new routific"

Last update
investigating

We are experiencing an issue with the live tracking feature on the new Routific platform. This may affect the driver status on the Delivery Tracker. We are investigating this issue and will give another update in the next hour.

Report: "Phone Notifications Not Delivered"

Last update
resolved

We experienced an issue with a third-party service responsible for triggering notifications. As a result, phone notifications were not sent during the affected period. The incident lasted from 2:14 PM to 2:30 PM UTC.

Report: "Phone Notifications Not Delivered"

Last update
Resolved

We experienced an issue with a third-party service responsible for triggering notifications. As a result, phone notifications were not sent during the affected period.The incident lasted from 2:14 PM to 2:30 PM UTC.

Report: "Service Delays: Customer Notifications, Address Lookup, and Delivery Tracker"

Last update
resolved

The system is stable, and all functionalities are operating as expected.

monitoring

We have confirmed that the issue has been resolved and systems are operating normally. We continue to monitor closely to ensure full stability. During the incident, some users may have received duplicate notifications, while others may not have received certain notifications at all.

identified

We have identified an issue that caused delays in our customer notifications, address lookup, and delivery tracking functionalities. The affected systems are now operating normally, and we are currently assessing the full impact to ensure ongoing stability. Our team is closely monitoring the situation to ensure all services remain fully operational. We will provide further updates as more information becomes available.

Report: "Some users may be unable to download CSV solutions on the new Routific."

Last update
resolved

The incident has been resolved.

monitoring

CSV downloads on the new Routific are working again. We are continuing to monitor the situation and will update again in 30 minutes.

investigating

Some users may be unable to download CSV solutions on the new Routific. It’s our highest priority to get up and running again, and our engineers are working on it. We will send another update in 30 minutes.

Report: "Delayed optimization responses due to geocoding processing delay"

Last update
resolved

We experienced a delay in processing geocoding requests between 19:20 and 19:35 UTC on March 27th, which resulted in slower completion times for optimizations. As a result, users may have noticed delayed responses when fetching the `https://api.routific.com/jobs/:id` endpoint, as some optimizations took longer than expected to finalize. The issue has been fully resolved, and all systems are now operating normally.

Report: "We are currently experiencing an outage. Users are not able to optimize routes on beta.routific.com. Its our highest priority to get up and running again. We will send another update in 30 minutes"

Last update
resolved

All Routific’s systems are now working as expected.

monitoring

Optimizations are now working on beta.routific.com again. For users that were experiencing errors trying to optimize, please re-attempt the optimization. We are continuing to monitor the results.

identified

The issue has been identified and we are working on a fix for those experiencing errors on existing optimizations. We will provide another update in 30 minutes

investigating

The ability to optimize new routes on beta.routific.com has been restored. For those still experiencing errors on existing optimizations, please stay tuned

investigating

We are currently investigating this issue.

Report: "Routific (app.routific.com) Outage"

Last update
postmortem

Routific \([app.routific.com](http://app.routific.com/)\) was unavailable intermittently on Friday, February 21, 2025 between the hours of 3 p.m. PST to 4:48 p.m. PST. This was due to an unexpected spike of traffic. We have since made some scalability improvements to handle such loads in the future. We thank you for your patience, and for continuing to work with us.

resolved

We can confirm that the issue has been resolved. Routific is now fully operational. We will provide a full post mortem on Monday. Thank you for working with us on this.

monitoring

We have identified the issue and implemented some changes to fix the problem. Routific is currently operational. We will keep monitoring the site and provide another update in the next 30 minutes.

investigating

We are currently experiencing an outage. Users are not able to access Routific (app.routific.com) and the Routific Driver mobile app. It is our highest priority to get up and running again. We will send another update in 30 minutes.

investigating

We are continuing to investigate this issue.

investigating

Some users were not able to access Routific due to an outage between 3:06PM and 3:10 PM PST. We are currently investigating the issue and will provide an update in the next 30 minutes.

Report: "Issue with Driver Assignment"

Last update
resolved

Timeframe: February 14, 2:35 p.m. PST – 3:10 p.m. PST We experienced an issue during a New Routific platform that impacted the driver assignment to routes functionality. During the noted timeframe, users were unable to view the driver list after clicking the button "Assign to Driver".

Report: "Routific (app.routific.com) password resets not sending"

Last update
resolved

This incident has been resolved.

monitoring

We have determined the root cause, and have pushed a fix. Password resets should be working again for all users on app.routific.com. Thank you for your patience!

investigating

We have determined the root cause, and have pushed a fix. Password resets should be working again for all users on app.routific.com. Thank you for your patience!

investigating

We are investigating an issue where users on app.routific.com are not receiving the password reset emails. Password resets on the new Routific at beta.routific.com are unaffected. Please reach out to support@routific.com if you need to reset your password urgently.

Report: "Temporarily unable to access Customer Profiles"

Last update
resolved

There was a partial outage in accessing customer profiles due to a technical issue which has now been restored. The outage lasted roughly 10 minutes and started at around 3:35 PST. We appreciate your patience and worked promptly to restore the service as we understand it impacts your business.

Report: "Delivery tracker outage"

Last update
postmortem

On January 22, 2025 at approximately 10:16 AM PST, Routific experienced an outage that prevented users from sending and accessing delivery tracking links. The disruption lasted around 45 minutes before full service was restored. Our investigation determined that a bug in our internal messaging system caused a backlog of unacknowledged messages, which overwhelmed the system and led to the outage. Our engineering team quickly identified the root cause and deployed a fix to clear the message backlog and stabilize the platform. We have also improved our monitoring and alerting systems to detect similar issues sooner and avoid future disruptions. Thank you for your patience as we worked to resolve this matter.

resolved

The delivery tracker continues to be stable and fully operational. Our team has identified the root cause of the issue. A bug in our internal messaging system caused an overload, resulting in about 45 minutes of downtime. We have since fixed the issue and will improve our monitoring systems to identify these types of issues earlier to avoid future disruptions.

monitoring

Access to the delivery tracker has been restored. Thank you for your patience! We are still investigating the cause of the outage. We will report our findings as soon as possible.

investigating

We are currently experiencing an issue where delivery tracking links are not being sent to customers. For those who have already received a tracking link, it may not display accurate information about the status of the delivery. Our engineering team is actively working to resolve the issue and restore full functionality as soon as possible. We appreciate your patience and will provide more updates within the next 30 minutes.

Report: "New Routific outage"

Last update
postmortem

On January 22, 2025 at approximately 10:16 AM PST, Routific experienced an outage that prevented users from accessing the new Routific. The disruption lasted around 45 minutes before full service was restored. Our investigation determined that a bug in our internal messaging system caused a backlog of unacknowledged messages, which overwhelmed the system and led to the outage. Our engineering team quickly identified the root cause and deployed a fix to clear the message backlog and stabilize the platform. We have also improved our monitoring and alerting systems to detect similar issues sooner and avoid future disruptions. Thank you for your patience as we worked to resolve this matter.

resolved

The new Routific continues to be stable and fully operational. Our team has identified the root cause of the issue. A bug in our internal messaging system caused an overload, resulting in about 45 minutes of downtime. We have since fixed the issue and will improve our monitoring systems to identify these types of issues earlier to avoid future disruptions.

monitoring

The new Routific is stable and operational. Our team will continue monitoring and provide a report on the cause of the issue soon.

investigating

Access to the new Routific has been restored. Thank you for your patience! We are still investigating the cause of the outage. We will report our findings as soon as possible.

investigating

We are actively working to restore user access to the new Routific. This issue is our team's highest priority. We thank you for your patience as we work to resolve this as soon as possible. The next update will come within 30 minutes.

investigating

We are actively working to restore user access to the new Routific. The issue is affecting most users and our engineering team is working on a fix at the moment. More updates will be provided in the next 30 minutes.

Report: "Driver dispatch partial outage"

Last update
postmortem

On Monday, November 18 starting around 4:37 p.m. PT, some users were not able to access their routes on Routific Driver via the link provided in the SMS dispatch. This partial outage lasted approximately one hour before it was resolved. The issue was caused by our URL shortener malfunctioning. When our internal monitoring systems alerted our engineering team of the issue, we were able to reboot our systems allowing the URL shorteners to work as expected. Our team is now working to push an update to improve the stability of our URL shortener.

resolved

This incident has been resolved.

Report: "Google Play Store app download"

Last update
resolved

Routific mobile apps are now fully accessible via the Google Play Store. You can search for the Routific apps in the Play Store and download via the direct links we send out during the driver invite process.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The ability to download Routific mobile apps from the Google Play Store has been partially restored. Direct download is now available via the email links sent out to your drivers when inviting them. The ability to search for the apps directly on the Google Play Store is still unavailable. This issue does not impact users who already have our apps installed on their devices or who access the app via the email invite link. The direct links are as follows: Routific Driver (compatible with the old Routific) https://play.google.com/store/apps/details?id=com.routific.driverapp&hl=en_CA Routific (compatible with the new Routific) https://play.google.com/store/apps/details?id=com.routific.mobile We will provide another status update on Friday around 9 AM PST.

identified

Our Routific mobile apps have been submitted to the Google Play Store and they’re waiting to be reviewed by the Google Play Store team. Please note that this process may take longer than 24 hours. This issue only affects users attempting to download our apps and does not impact users who already have our apps installed on their devices. We will provide another status update on Friday around 9:00 AM PST.

identified

We are still actively working to restore the ability to download our app from the Google Play Store.

identified

We are actively working to restore the ability to download our app from the Play Store. Please note that this issue only affects new downloads and does not impact users who currently have our application installed on their devices.

Report: "SMS Connectivity Issues in United States and Canada"

Last update
resolved

This incident has been resolved.

monitoring

Our SMS service provider is reporting some SMS notifications being blocked in the U.S. and Canada. We are in contact with them to address the issue and hope to restore deliverability as soon as possible.

Report: "AT&T blocking some SMS messages in the United States"

Last update
resolved

We are no longer experiencing failures to the AT&T network in the United States. After dispatching your project, drivers under this carrier will receive the SMS notification with the link to access their routes without any issues.

identified

We have identified an issue with AT&T in the United States that is preventing some drivers from receiving the SMS link with their route. We are investigating with our SMS provider and will update as soon as we know more. If your driver hasn’t received a notification after you’ve dispatched a route, there is a workaround: Click “Copy Link” on the Dispatch screen and paste the link in an email or text message to send to your drivers. This link will open up the Routific web app on their phone, which can also be used to complete deliveries.

Report: "Temporarily unable to access routific.com"

Last update
resolved

This incident has been resolved.

monitoring

The issue has been resolved. We will continue to monitor

investigating

Access to our homepage www.routific.com is temporarily blocked due to our third party provider having an outage. Users attempting to access app.routific.com or beta.routific.com to manage their routes are unaffected.

Report: "Temporarily unable to access our homepage routific.com"

Last update
resolved

This incident has been resolved.

monitoring

The issue has been resolved and we are continuing the monitor.

investigating

Access to our home page www.routific.com was temporarily blocked due to our third party provider having an outage. Users attempting to access app.routific.com to manage their routes were unaffected.

Report: "SMS notification instability"

Last update
resolved

There has been no new occurrences of this issue and all SMS services should be functioning as expected. We plan to improve our ability to monitor and react to future outages like this in an attempt to limit or prevent the effects on our users.

monitoring

We have isolated two issues around this, the first happened between 20:20 - 21:00 UTC and the second happened between16:30 -16:55 UTC. We discovered that some messages did go through and only a subset of notifications was affected. We are still actively monitoring the situation and working with our provider to better mitigate issues like this in the future.

identified

We had an issue yesterday with our SMS provider that caused a potential degradation in the sending of SMS messages. We are still investigating the potential ongoing impact to notifications.

Report: "Disruption in SMS Customer Notifications for some Users in Australia"

Last update
resolved

After thorough investigation, we have determined that this issue was isolated to a specific group of users. We have been in direct contact with all affected users to resolve the issue as quickly as possible.

investigating

We are currently investigating this issue.

Report: "Account management and webhook email notifications not being sent correctly"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We've identified an integration issue with an external tool responsible for sending email notifications. This is affecting account management and webhook failure email notifications. Consequently, users may not be receiving emails to activate accounts, update password or notifications about failed webhook calls.

Report: "Routing Service Disruption"

Last update
resolved

Around 12:15:46 PST, we detected a connection issue with a downstream provider. This issue resulted in jobs not being processed, consequently timing out. The issue lasted until 12:17:49 PST and self-resolved. We have reached out to the provider to gather more details on what has caused the connection error and will continue to monitor the situation.

Report: "SMS deliverability in the U.S."

Last update
resolved

SMS deliverability has been stable for many days now with all systems fully functioning. Deliverability rates have returned to normal. Thank you for your patience, and thank you for working with us through this.

monitoring

SMS deliverability has been stable for many days now with all systems fully functioning. Deliverability rates have returned to normal. Thank you for your patience, and thank you for working with us through this.

monitoring

SMS deliverability in the United States continues to improve daily. While most customers have experienced normal deliverability rates over the last few days, some are still reporting a 5-10% failure rate due to another issue that has arisen with T-Mobile. We are actively working with our SMS providers and that carrier to resolve the issue. In the meantime, we have several ongoing initiatives aimed at further increasing deliverability rates. Please contact support@routific.com should you have questions or need assistance.

monitoring

SMS deliverability in the United States has improved significantly over the last few days, and we're currently operating at normal deliverability rates. We are continuing to monitor for any further issues.

monitoring

Our SMS provider has informed us that T-Mobile has lifted the block and we should see gradual improvement in SMS deliverability with that carrier over the next few days. We will continue to monitor the situation and provide another update once we are confident this particular issue has been resolved.

monitoring

SMS deliverability has stabilized in Canada, and in the U.S. it appears most of the deliverability issues we continue to face are largely due to T-Mobile blocking about ~25% of our SMS messages on their network. We are continuing to lobby our SMS provider for help in unblocking the messages with the carrier. Currently, switching to email notifications or migrating to our new SMS provider (standard messages only) are two options to increase deliverability. Email support@routific.com if you are interested in either of the two options. Thank you for your patience.

monitoring

Our team has been working on increasing SMS deliverability in the U.S. and Canada over the last few weeks. While we are seeing good progress, some Routific users are still experiencing inconsistent deliverability. We are closely monitoring the situation, and have escalated the issue with our SMS providers. North American telecom companies are increasingly strict on SMS content and perceived spam. We thank you for your patience as we navigate the ever-changing rules and regulations. We will update this page as we learn more.

Report: "SMS notifications not delivering to US & CA customers"

Last update
resolved

Overall message deliverability in the US and CA is much improved with our new primary SMS message provider, and is stable again with our backup provider. We have improved our capability for message delivery by leveraging multiple providers, and will continue tuning and optimizing delivery to provide the best possible SMS delivery rate for our customers. This incident is resolved.

monitoring

Between 6:47am PDT and 7:46am PDT, our SMS deliverability experienced delays and has since recovered.

monitoring

Our systems have alerted us of a possible degradation with SMS deliverability for some customers in the US and Canada. We are working with our SMS service provider to assess the impact and resolve the issue.

monitoring

While our team has successfully resolved the issue impacting customer notification deliverability, we're continuing to collaborate with our SMS provider to address the challenge of users' ability to send customized messaging. Thank you for your ongoing patience and cooperation as we make progress towards a resolution.

monitoring

We are pleased to announce that the deliverability issues affecting standard Routific messages in the US and Canada have been successfully resolved. However, we are still addressing a specific challenge for users sending customized messages. We are collaborating closely with our SMS provider to thoroughly investigate this issue. In the meantime, we recommend utilizing our standard message templates, which have shown reliable deliverability. We appreciate your patience and understanding as we work towards a complete resolution.

monitoring

We are currently experiencing near-normal SMS deliverability rates in the US and Canada. However, we have noted concerns with some valid numbers not receiving messages. We are investigating these reports and have escalated the matter to our providers for swift resolution. Further updates will follow as available. We appreciate your patience and apologize for any inconvenience.

monitoring

Some issues with SMS delivery persist in the US and Canada. We are collaborating with our providers and carriers to address these issues and have implemented their recommended adjustments to enhance message delivery. The effectiveness of these changes is being closely monitored, and updates will be provided as progress occurs. We apologize for the inconvenience and value your continued patience.

monitoring

We have observed some deliverability concerns with both our primary and backup providers that may affect a small percentage of SMS messages in US and CA. Our providers are working with US Carriers to improve this situation. We're very sorry for this ongoing inconvenience.

monitoring

Both primary and backup SMS providers are now online and we’re closely monitoring deliverability on both platforms. Our team has also started to reach out to customers who have customized SMS messages to improve deliverability. Thank you for your patience as we work through the list of those affected.

monitoring

We have continued implementing a migration to a new SMS provider for our US and CA customers. Message deliverability looks good, and all customer messages appear to be getting delivered as expected now. Our customer team will be reaching out to some customers about managing their message template and any customizations, to help ensure you do not experience any future interruption to message delivery. We will continue closely monitoring our SMS delivery status.

identified

We have implemented an initial fix which should see some SMS messages now flowing through to CA and US numbers. This involved switching to a new / backup SMS provider for North America messaging. All messaging outside of North America should continue to be unaffected. We are now monitoring the deliverability of this new provider, and if delivery looks good, we will be looking to migrate more of our US and CA SMS traffic to this new service throughout today. We will provide another update soon.

identified

We have identified the cause for the blocking of our SMS notifications and are working closely with our 3rd party provider to resolve the issue as quickly as possible. We hope to be able to restore services soon.

investigating

SMS notifications to U.S. and Canadian customers are being blocked by our SMS provider. We are very sorry to have this impact your business! It is our top priority to resolve this. In the meantime, we advise you to please switch to email notifications if possible.

investigating

SMS notifications to U.S. and Canadian customers are being blocked by our SMS provider. We are very sorry to have this impact your business! It is our top priority to resolve this. In the meantime, we advise you to please switch to email notifications if possible.

Report: "Optimizations on the New Routific (Beta) are delayed or not completing"

Last update
resolved

The incident with scheduling on the New Routific is resolved.

monitoring

We have taken some remediating action, and initial testing indicates optimization / scheduling on the new Routific is functioning as expected again, but we are monitoring the situation closely.

investigating

We are currently investigating this issue, and have some leads on the cause. We will update again shortly.

Report: "Live Tracking Disruption"

Last update
resolved

The problem with location tracking on Android devices has been successfully identified and resolved. Thank you for your patience and cooperation.

investigating

We are currently investigating an issue affecting live tracking for drivers on Android devices.

Report: "SMS notifications not delivering to US & CA customers"

Last update
postmortem

### **Summary** SMS customer notifications to U.S. and Canadian customers were blocked by Routific’s SMS provider from April 20, 2024 at 9:45 am PDT until April 22, 2024 at 2:21 p.m. PDT. SMS customer notifications are now being successfully delivered to U.S. and Canadian customers. ### **What happened?** On April 21, we were alerted that our SMS provider had suspended our SMS messaging to the U.S. and Canada due to “non-compliant A2P messaging”, which means that some of the messages being sent from the Routific platform were not compliant with guidelines meant to protect customers from unwanted or unlawful messaging. We immediately reached out to the SMS provider and worked with their team to reverse the suspension. ### **What are we doing next?** Routific is committed to improving the reliability and stability of our systems. To prevent such an incident from occurring in the future, we are looking into ways to implement fallback measures should there be an issue with our SMS campaigns again. We also need to improve our monitoring to detect such service disruptions earlier. Looking ahead, we also need to protect our customers relying on SMS notifications by ensuring the Routific platform better enforces A2P compliance, thereby increasing deliverability of lawful and important updates.

resolved

SMS customer notifications are now being successfully delivered in both the U.S. and Canada. We will post more details on the cause of the outage and resolution soon. We understand how frustrating this experience has been for all. We thank you for your patience and understanding.

investigating

We are currently working with our SMS provider to restore SMS customer notifications. We understand this impacts your business, and we appreciate your patience while we actively work to find a solution. Please switch to email notifications for the time being, if possible. We will update this thread tomorrow with further developments.

investigating

SMS notifications to U.S. and Canadian customers are being blocked by our SMS provider. We are very sorry to have this impact your business! It is our top priority to resolve this. In the meantime, we advise you to please switch to email notifications if possible.

Report: "Trouble uploading orders"

Last update
resolved

This incident has been resolved.

monitoring

The issue has been identified and we have shipped a fix. Uploading orders is working again but we will continue to monitor

investigating

We are continuing to investigate this issue.

investigating

We are seeing issues with customers trying to upload orders. We are investigating

Report: "Drivers are unable to open routes - link takes them to Routific Webpage"

Last update
resolved

The third-party provider responsible for the redirection of our driver app links is no longer experiencing intermittent outages. This incident has been resolved

monitoring

Issue Identified: The third-party provider responsible for the redirection of our driver app links is experiencing intermittent outages. As a workaround, users can access the Routific Lite app by navigating to the 'Dispatch' tab and selecting 'Copy Link'. Pasting this link into a browser will open Routific Lite. We are actively monitoring the situation and will provide updates as they become available. Thank you for your understanding.

investigating

We are currently investigating reports of drivers being unable to open routes and the link taking them to the Routific webpage

Report: "Product API stability issues"

Last update
postmortem

At approximately 5:00am pst we discovered an issue with one of our services. At this point we weren't sure of the impact yet and but continued to investigate in order to resolve the issue as soon as possible. We later discovered, around 7:00am pst that the issue we were investigating was affecting access to customer profiles. We tried a few actions that temporarily restored access but ultimately we did not know the root cause yet at this point and the feature soon reverted to its degraded status. At 8:am pst we discovered our database was under a high load and determined this is what was causing the disruption to the Customer Profiles feature. Shortly after we reset our database and this resolved the load issue on the database as well as restoring access to customer profiles. Our action items to help prevent such issues in the future was to increase our monitoring on our database load to be informed much sooner if any issues are occurring. We are sorry for the impact this may have had on your business.

resolved

We have isolated the cause of this issue which was caused by a long running task in our backend and resolved it. Customer profiles are again fully functional. Sorry for any inconvenience caused this morning.

monitoring

We have taken some action to resolve this issue. Customer profiles are currently working again at the moment. We are monitoring the situation closely.

investigating

The ongoing issue is currently affecting customer profiles. We are continuing our investigation to resolve this as soon as possible.

investigating

We are currently experiencing some stability issues within our application. We are working to determine exactly what is affected at this time and apologize for any issues this may cause.

Report: "Dispatching Service Disruption"

Last update
postmortem

At 9:27am PST — We had received multiple reports from our customers being unable to dispatch. After some initial investigation we found this issue had started at approximately 9am PST but our existing systems did not flag this issue for us. Throughout our investigation we found that a recent release introduced a bug which caused the issue, preventing users from being able to dispatch. At 9:35am PST — We reverted our services to the previous version to unblock our customers and ensure that dispatching did indeed work again. Action Items: * Our team will look into our monitoring systems to identify why we weren't notified of this issue earlier and fix this gap to ensure that similar issues in the future can be identified and resolved quickly. We apologize for any impact this had on your businesses today.

resolved

This incident has been resolved.

monitoring

We have deployed a fix to the related bug and services should be restored. We are monitoring this situation closely.

investigating

We introduced a bug with our latest update that impacts dispatching. Our engineers are investigating and will update as soon as its resolved. We are sorry as we know this impacts your business!

Report: "Google SSO Login Issue - Resolved"

Last update
resolved

We were experiencing issues with Google Single Sign-On (SSO) login on our Mobile Driver and Chrome extension applications. Users were unable to log in or may have experienced difficulties during the authentication process. The issue has since been resolved. We apologise for any inconvenience caused.

Report: "Some customers unable to dispatch"

Last update
resolved

We recently performed maintenance and upgrades to one of the services backs the platform's dispatching capabilities. We began slowly migrating a portion of our traffic to use that upgraded service, but identified an intermittent issue with the upgrade prior to releasing the changes to all customers. Prior to reverting the changes, some customers may have received an error while trying to dispatch, but the service should now be stable and our team will continue to monitor.

Report: "Some customers are not able to optimize on Beta"

Last update
postmortem

Starting at 10:09 a.m. PST, some optimizations would be stuck in an “in progress” state due to an internal infrastructure error. At 11:15 a.m. PST, all affected customers could optimize successfully again. For the optimizations being executed at 10:09-11:15 \(and affected by this issue\), they are now in a failed state. Further investigations are being conducted by the engineering team.

resolved

This incident has been resolved.

investigating

We are currently investigating this issue.

Report: "Some customers are not able to optimize on Beta"

Last update
resolved

Some customers are not being able to optimize due to an infrastructure issue

identified

The issue has been identified and a fix is being implemented.

investigating

We are continuing to investigate this issue.

investigating

A fix is being provided

Report: "Unable to load Beta Mobile App (version 2.84.1)"

Last update
resolved

This incident has been resolved.

identified

An issue in the latest version release of the Beta Mobile App has been identified (2.84.1). We are working on a fix. In the meantime, if you have upgraded to this version and are unable to run the app, please uninstall and reinstall the app.

Report: "Routific.com is down and inaccessible"

Last update
postmortem

### Postmortem On September 17th 5:37am - 8:49am p.m. PST, Routific’s main site was down and inaccesible to users. This prevented any user that was logged out of the app from logging in to Routific. ### What happened? WPE which we use to host our site had a major outage \([Incident link](https://wpenginestatus.com/incidents/524219)\) which caused our site to be unavailable. ### What are we doing about it? To prevent this incident from occurring in the future, we will look into ways to ensure our users are still able to access their dispatching portal even in the case of an outage to our home site or one of our providers. We also provided a workaround login url that users could access during this outage in order to login to the Routific application.

resolved

This incident has been resolved.

monitoring

The provider for our main site (www.routific.com) has restored service, and users who may not have been logged in can login again. FYI, the Routific Application was available throughout this period, and if you were previously logged in, you likely were not impacted. Likewise API users would not have been impacted. We will continue monitoring, and be exploring future mitigation strategies to avoid such interruptions. Thank you for your patience.

identified

There is a fix being rolled out by our provider, but we have not seen it take affect yet. We are exploring alternatives to ensure the restoration of our main site and login, should the provider fix not work soon.

identified

The issue appears to be with the provider managing our main site (www.routific.com) and DNS, and we are working with them to restore service as soon as possible. One potential workaround is to visit our Engine / Developer Site: dev.routific.com, login as usual, and then browse to app.routific.com.

investigating

Actually, it appears that app.routific.com is accessible as well as the API. The issue is the main site is not accessible (www.routific.com), so if you are not logged in already, it is possible you may not be able to access the service.

investigating

We are investigating this urgent issue, which may be related to our domain.

Report: "Beta service orders cannot be added"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We have implemented a fix and we will continue monitoring.

identified

We have identified the issue and are currently trying to resolve it.

investigating

Currently, on the Routific beta service, orders cannot be added and order addresses cannot be edited due to the address suggestion not showing up. We are currently investigating this issue.

Report: "Mobile App unavailable on Play Store"

Last update
resolved

The Routific Driver App is available again for download on the Google Play Store.

identified

Our team is working to remediate the situation and make the app available again.

investigating

The Routific Driver app is currently unavailable for download on the Google Play Store. Our team is working to remediate the situation and make the app available again. In the meantime, drivers who are required to install the app need to use the lite app.

Report: "Notifications Delay"

Last update
resolved

The notifications have been stable for the last couple of hours. The incident has been resolved, sorry for the inconvenience.

monitoring

We have identified the issue and deployed a fix. Notifications are now being sent without delays. We will be monitoring now on.

investigating

The notification messages were all sent due to a temporary measure we took, but the delay is still present. We continue investigating the issue.

investigating

We are currently investigating an issue with Notifications. Due to that, notifications are currently delayed.

Report: "Homepage outage"

Last update
resolved

We have found the cause and fixed our recent homepage outage that prevented users from signing in

investigating

We apologize for the current outage of our homepage. We have identified the issue and have shipped a fix. Please rest assured that we are closely monitoring the situation and will provide updates as soon as we have more information. You can continue to access Routific through https://app.routific.com/

Report: "Issue With Geolocation Services"

Last update
resolved

This incident has been resolved.

investigating

The issue has been identified and resolved, and we will continue to monitor the situation closely

investigating

We are currently investigating an issue with our geolocation services. This may result in inaccurate location data or difficulty in accessing certain location-based features in our app (e.g. you may face errors while uploading stops - specifically we have seen issues with mobile country codes). We will provide updates on our progress and will notify you as soon as the geolocation services are restored.

Report: "Potential service issues in the Europe region."

Last update
resolved

Our service provider closed the incident. We were not impacted.

monitoring

We have determined that our services should not be affected by our providers current outages but we are still monitoring the situation as a precaution.

investigating

Our service provider in Europe is experiencing issues right now that could potentially disrupt services. We are monitoring things and will update when the services are stable again.

Report: "Sending customer notifications will be delayed"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

Report: "Sending customer notifications will be delayed"

Last update
resolved

Any messages affected by the delay have been processed, and the system has stabilized.

investigating

We are currently investigating this issue.

Report: "Customer Notifications Service Disruption"

Last update
resolved

We are aware of an outage on our notification service that delayed sending customer notifications (including driver delivery tracking links). Users were also unable to access data in their customer notification templates and configuration settings, and may have been prompted that a recipient’s mobile number was unable to be validated. The incident occurred between 9:54am PST and service was fully restored at 10:20am PST.

Report: "Beta Service Outage"

Last update
postmortem

### Postmortem On February 20th 12:00 a.m. - 4:00 a.m. PST approximately Routific Beta experienced an outage with our ability to optimize, this prevented our customers from scheduling their projects. ### What happened? Our service which listens for and processes requests for scheduling was no longer picking up new requests. ### What are we doing about it? * Investigating the exact cause through reproducing it on our closed environments * Updating our monitoring to better and more quickly identify issues in the future * Better updating our support resources to increase our response time to issues

resolved

This incident has been resolved.

monitoring

We are aware of an outage from last night on our beta platform where users were not able to schedule their projects and have since resolved it. We will continue to monitor closely. This issue happened at approximately 1am PST on February 20th.

Report: "Issues with our Shopify Store App"

Last update
resolved

This issue has been confirmed resolved with many of our users. We will be taking action to ensure we do not experience a similar outage in the future.

monitoring

We have resolved the internal networking issue and our testing indicates the Shopify integration should now be working.

investigating

We have identified a potential concern in our internal network, but are still investigating.

investigating

We are continuing to investigate this issue.

investigating

We are currently investigating reports that users cannot access our Shopify store app service from their Shopify app's.

Report: "Service Disruption"

Last update
resolved

We have restored stability to the affected services.

monitoring

We are continuing to monitor for any further issues.

monitoring

We had an issue that temporarily caused services to be disrupted between 12:26 PST - 13:00 PST, we have resolved it but will continue monitoring.

monitoring

We had an issue that temporarily caused services to be disrupted, we have resolved it but will continue monitoring.

Report: "Updates not sending via web hooks"

Last update
resolved

This incident has been resolved. We are sorry for any inconvenience, and are looking to improve our monitoring on this particular part of our service ASAP to avoid any future occurrences.

investigating

We have identified an issue at 4pm PST, which appears to have begun in the early morning today. We are currently investigating this issue which may affect customers using "web hooks", where updates to routes are not being sent to their services.

Report: "Issues with driver SMS notifications delivery"

Last update
resolved

This incident has been resolved.

monitoring

While we've isolated the delivery concern to an issue between Twilio and the Rogers Canada Network, we've taken steps to workaround this issue and customers' drivers should be receiving their notifications now, based on our testing.

investigating

We are currently investigating reports of issues with driver notifications via SMS, specifically on the Rogers Canada network.