Historical record of incidents for Hydrow
Report: "Service Disruption: Possible Workout Playback Impacts"
Last updateWe are currently experiencing an outage affecting some of our partners, which may disrupt the ability to play workouts. Our team is actively monitoring the situation and working on a resolution. We appreciate your patience and will provide updates as soon as possible.
Report: "Setup Errors for Some Hydrow Rowers Identified"
Last updateWe have monitored for new instances of this error and have not identified any further issues. Hydrow members have also not reported new instances of failed updates. We now consider this incident resolved. If you continue to experience any issues, please contact Hydrow Member Services for assistance.
We’ve resolved the issue that was causing “Update did not succeed” errors during the setup or reactivation of some Hydrow rowers. The fix has been deployed, and we are monitoring to ensure continued stability. If you continue to experience any issues, please contact Hydrow Member Services
A fix has been deployed for this issue and we are monitoring the result. Impacted Rowing machines may need a restart for the change to take effect.
We’ve identified a software issue causing "Update did not succeed" errors for some Hydrow rowers during setup. This affects: - Newly purchased Hydrow rowers being activated for the first time. - A small number of Hydrow rowers being reactivated after a period of inactivity. Our team is actively working to resolve this. We will post additional updates as we have more information.
Report: "Setup Errors for Some Hydrow Rowers Identified"
Last updateWe have monitored for new instances of this error and have not identified any further issues. Hydrow members have also not reported new instances of failed updates. We now consider this incident resolved.If you continue to experience any issues, please contact Hydrow Member Services for assistance.
We’ve resolved the issue that was causing “Update did not succeed” errors during the setup or reactivation of some Hydrow rowers. The fix has been deployed, and we are monitoring to ensure continued stability.If you continue to experience any issues, please contact Hydrow Member Services
A fix has been deployed for this issue and we are monitoring the result. Impacted Rowing machines may need a restart for the change to take effect.
We’ve identified a software issue causing "Update did not succeed" errors for some Hydrow rowers during setup. This affects: - Newly purchased Hydrow rowers being activated for the first time.- A small number of Hydrow rowers being reactivated after a period of inactivity.Our team is actively working to resolve this. We will post additional updates as we have more information.
Report: "Headline: Unable to add products to cart on"
Last updateWebsite and cart operation have returned to normal and appear stable. Marking as resolved. If you have a problem adding to cart on hydrow.com, please reload your browser window. If the problem persists, contact http://support.hydrow.com. The root cause of the problem was with one of our infrastructure partners. Once their error was resolved, our issue was also resolved.
Errors from this incident appear to have stopped. Monitoring a few minutes to be sure things are resolved.
We are aware that some customers are unable to add products to their cart on hydrow.com. There are no reports of problems with the Hydrow application, workouts, workout stats, or mobile apps. We are investigating.
Report: "hydrow.com website not accessbile"
Last updateThe issue has been resolved, and hydrow.com is loading normally. If you have a problem reaching hydrow.com, please reload your browser window. If the problem persists, contact http://support.hydrow.com. The cause of the problem was with one of our infrastructure partners. Once their error was resolved, our issue was also resolved.
Several people are unable to get to hydrow.com, and are seeing an error message when they try to. There are no reports of problems with the Hydrow application - workouts, stats, and mobile apps etc. We are investigating.
Report: "Hydrow Service Outage"
Last updateConfirmed that all operations have returned to normal and all Hydrow members should now be able to work out again via Rower or Mobile app. We will be closing out this incident as resolved.
We are beginning to see normal workout behavior resume for most users. We will be monitoring to ensure the solution is lasting.
We've root caused the issue and will be deploying a fix presently that should improve the issue.
Work to resolve this continues on with our entire team engaged. We will provide an update within the next hour or as soon as more information becomes available to share.
We are continuing to investigate the issue and working to restore service as quickly as possible. We will update again in the next 30 minutes.
We have detected a problem preventing workouts on both our rowers and mobile app. We are investigating and will provide more detail in the next 30 minutes.
Report: "Phone support temporarily unavailable"
Last updateInbound phone lines are open and working again. Resolving this incident.
We are currently investigating a problem with the phone lines to reach Hydrow Member Services Directly If you need to submit a ticket with us while we resolve this issue you can visit https://support.hydrow.com/s/contactsupport We will update with more information shortly.
Report: "Partial Outage - Hydrow is not available error"
Last updateWe have implemented a fix. All Hydrow services have been restored and this incident is now resolved.
We have discovered a root cause for this issue and are implementing a fix. We will provide an additional update when the fix is complete.
We are investigating reports of members being unable to row or otherwise use their Hydrow with the message "Hydrow is not available / Internal server error." This appears to be isolated to new member accounts or guest users on commercial rowers.
Report: "Outage - Members unable to row."
Last updateAll services are returned to normal operation and no new reports of the On-Hydrow error have been received for the last 20 minutes. We are declaring this issue resolved.
We discovered a root cause for this issue and implemented a fix. We see activity already returning to normal but will continue to monitor.
We are investigating reports of members being unable to row or otherwise use their Hydrow with the message "Hydrow Has Stopped" We will update with more info within 30 minutes
Report: "Issue using Hydrow"
Last updateAll issues appeared resolved.
Hydrow service has been restored.
The issue has been identified and a fix is being implemented.
We're currently investigating an issue that may prevent users from using Hydrow.
Report: "Difficulty creating new memberships"
Last updateWe continue to see normal membership creations and no unexpected card declines through our payment processor since ~3pm Eastern Time. We will be marking this incident as resolved.
We have begun to see successful memberships being created in our system again. Including some using cards that were previously being declined. We are continuing to monitor and work with our payment processor to make sure that Hydrow memberships can be created smoothly.
This issue appears to be related to specific types of charges being declined through our 3rd party payment processor. We are currently working with them to determine root cause and resolve.
We are aware of an issue that is currently preventing new memberships from being created on Hydrow. Existing memberships and workouts appear unaffected. We are investigating and will provide updates as we learn more.
Report: "issue with checkout - purchasing is blocked"
Last updateThe issue has been resolved.
Our merchant partner has resolved the underlying issue, and we have corrected the problem it caused in our systems. Customers should be able to purchase normally. Any customers who are having difficulties with their shopping cart should use another browser, or contact Hydrow Support. We apologize for any inconvenience this caused.
We have identified that many people can't checkout purchases on the hydrow.com website. We are investigating. The Hydrow product is not affected.
Report: "Cart issue during checkout"
Last updateThis issue has been fully resolved and we're closing the incident.
All systems are back to normal. We identified the issue. Our merchant partner had a technical issue, which they report is resolved. We are continuing to monitor.
We have identified that many people can't checkout purchases on the hydrow.com website. We are investigating. The Hydrow product is not affected.
Report: "Hydrow.com Website Issue"
Last updateThe problem has been resolved. Hydrow.com is operating normally. We continue to monitor for any additional issues.
We are investigating reports that some users are receiving a 503 error when attempting to navigate to hydrow.com. The Hydrow service is not impacted by this issue; workouts, meter tracking, etc. are not affected.
Report: "Hydrow Service Impacted"
Last updateWe have seen no new incidents of this error in the last 20 minutes. We are considering this issue resolved and closing the incident.
We believe this issue should now be fixed and the symptoms resolved. We are continuing to monitor. We will update again within 30 minutes.
We have now identified the root cause for these error messages and are actively working to resolve. We expect to share another update within the next half hour.
We are currently investigating a number of reports of degraded service for some Hydrow Members. Symptoms may include difficulty signing in to a profile or signing up for membership. We will update here when more information is available.
Report: "Hydrow.com Website Issue"
Last updateThe problem has been resolved. Hydrow.com is operating normally. We continue to monitor for any additional issues.
We are continuing to investigate the problem with hydrow.com, and have identified the immediate cause of the problem. The hydrow.com website pages are working properly, except the shopping cart and purchase process are still not loading. The Hydrow service is not impacted by this issue; workouts, meter tracking, etc. are not affected. We will provide our next update within 30 minutes.
The hydrow.com website is freezing and unresponsive for some US users.
Report: "Degraded Hydrow.com Checkout"
Last updateThe checkout incident has been resolved by our service provider.
We are continuing to work on a fix for this issue.
Our checkout service is experiencing an issue which may result in a degraded checkout experience. Some Hydrow.com visitors may experience issues or delays with checkout, or when moving from cart to checkout. We have observed this specifically with mobile devices. Our service provider has identified the problem and is investigating.
Report: "live rows not running properly"
Last updateOur 6pm Eastern live row is underway with no known issues and we do not anticipate further disruption. We will continuing to monitor for any further problems.
AWS reports that the issue is on its way to resolution. We believe the live row for 6PM Eastern will be successful. We continue to monitor closely.
The 4PM Eastern Live Row was not able to start or end the timer, or record any meters. We apologize for the difficulty. We are not aware of any other impacts to our customers; only live rows appear to be affected. We believe the issue is caused by an outage acknowledged by one of our vendors, AWS. We are monitoring that issue, and waiting for them to resolve it.
We have reports and logging that show a problem with live rows happening this afternoon. We are investigating.
Report: "Time-outs on Shop.hydrow.com"
Last updateAll website activity has remained stable. Updating to resolved.
Shopping cart performance and Checkout appear to be back to normal. Monitoring status.
Customers have reported experiencing long loading times on our shopping website and we are investigating. Workouts and the hydrow app are NOT impacted.
Report: "support.hydrow.com is unavailable"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Our support website is currently unavailable. We appreciate your patience as we investigate the issue.
Report: "Customers are unable to reach our Hydrow store."
Last updateThis incident has been resolved.
The problem has been identified and corrected.
Customers are unable to reach our Hydrow store. We are investigating.
Report: "System Problem Impacting Workouts"
Last updateThis incident has been resolved.
The problem was traced to one of our technology partners, and they have mitigated the problem. We are no longer seeing errors with workouts. We are continuing to monitor.
We are are investigating an issue that is preventing at least some members' workouts.
Report: "Workout Issue Investigation"
Last updateThe problem has been resolved and the incident is closed.
The problem appears to be resolved. We are monitoring the situation.
We are investigating a problem with the Hydrow service that is preventing connections to our backend database. We will continue to post regular updates as we investigate.
We are continuing to investigate this issue.
We are investigating alerts that our system is not operating properly, likely impacting members and their workouts.
Report: "Service Unavailable"
Last updateThis incident has been resolved.
The Issue appears to have been brief. It may have related to an internal data Migration. Service seems to be returning to normal already.
Some Hydrow members are unable to log in currently. We are investigating the service implications and scope of the issue. We will update here as we learn more.
Report: "Hydrow Service Unavailable"
Last updateNormal Service has been restored and has appeared to be stable for the last half hour. We are marking this incident as resolved.
Service appears to be back to normal levels for the last 15 minutes or so. We will continue to monitor, but will call this resolved soon if there are no further issues.
We have clarified the nature of the issue and are working to mitigate. We're starting to see some successful log-ins. We will continue to monitor.
Log-in on Hydrow and related apps remains unavailable. A 3rd party hosting service may be involved in the outage. We are continuing to investigate but do not yet have an ETA for restoration of service.
Service for Hydrow devices and apps is currently unavailable for some users. We are investigating.
Report: "UK Membership Issues"
Last updateThis incident has been resolved.
We are continuing to investigate this issue.
This issue has been resolved. UK Subscriptions are working as normal. Closing the issue.
We are continuing to investigate this issue.
We have heard several reports from Hydrow Members in the UK of difficulties signing up for Hydrow membership or renewing membership. We are investigating and will update ASAP
Report: "hydrow.com website outage"
Last updateThis incident has been resolved.
We have identified the cause and hydrow.com is loading. We are continuing our investigation and testing of the full website.
Direct access to https://shop.hydrow.com/ is working.
We are investigating an outage on hydrow.com which prevents shopping, browsing, and purchases. The Hydrow application (workouts, etc.) and support.hydrow.com appear to be unaffected.
Report: "Hydrow Shop offline"
Last updateShopify has resolved their incident on https://www.shopifystatus.com/ and activity from our web store appears to be back to normal operation.
We are seeing improvement in function from the online store as Shopify resolves their outage. We will continue monitoring the status.
Customers are unable to add items to their shopping carts on shop.hydrow.com. The problem appears to be with our partner, Shopify. They are providing updates on https://www.shopifystatus.com/. We do not know when the issue will be resolved.
Report: "Help Center and Customer Service offline"
Last updateThis incident has been resolved.
Zendesk has resolved their related outage. Help Center operations appear to be back to normal. Continuing to monitor
Zendesk reports having resolved their incident. We are monitoring to ensure our systems are stable again.
We are continuing to investigate this issue.
Both Customer-facing and internal services that rely on Zendesk appear to be down at the moment. Zendesk is investigating an outage on https://status.zendesk.com/ that is likely the root cause
Report: "Workouts Not loading"
Last updateThis incident has been resolved.
We have seen no further abnormalities. This may have been due to an AWS incident earlier in the day. Continuing to monitor.
We are seeing a spike in errors when retrieving workout videos from our content network. Hydrow Members may experience issues when trying to begin a new workout video. We are investigating.
Report: "Long response times & Delayed Log-in on Hydrow."
Last updateAmazon Web Services has mitigated the issue that was causing problems for Hydrow services. The situation appears to be resolved.
Slow response times and delays on Hydrow have continued intermittently as Amazon Web Services continues their attempts to mitigate their outage. We have seen some improvements, but as of yet they do not have an ETA for full recovery of service. We will continue to monitor the situation as they pursue a solution. We will update here when more information is available.
The Amazon Web Services outage has been clearly identified as the root cause of this issue. They are working to fully restore service but Hydrow response times remain slightly degraded. We will update again when there is a change in status on this issue.
Due to a disruption across Amazon Web Services, Hydrow is currently experiencing slow response times, including long delays at login or the inability to login or access Hydrow workouts. We are working with our partners to resolve this issue as quickly as possible, and will update here when we have more information.
Report: "support.hydrow.com unreachable"
Last updatesupport.hydrow.com is fully accessible.
support.hydrow.com is accessible again. The problem appears to have been with our support partner's infrastructure. We are monitoring for any recurrence.
We are currently investigating this issue.
Report: "Workout Issue"
Last updateThis incident has been resolved.
The issue appears to be resolved. It has been more than 20 minutes since we have seen a workout error. The underlying problem appears to be the major internet outage earlier this morning that affected widespread areas of the internet. We will conduct an internal post-mortem to evaluate how the problem could be better avoided, mitigated, and/or managed in the future.
Workouts are now starting. We continue to investigate the problem and are monitoring the resumption of service.
Our video partner is experiencing a service disruption that is affecting Hydrow. We continue to investigate and will update this incident frequently.
We are investigating an issue where workouts are unable to start. Customers are getting a "Something went wrong" error on start of a workout.
Report: "Video Streaming Issue (Test not real)"
Last updateThis incident has been resolved.
We have received reports that several customers are having issues with interrupted video streams. We are investigating.
Report: "This is an example incident"
Last updateWhen your product or service isn’t functioning as expected, let your customers know by creating an incident. Communicate early, even if you don’t know exactly what’s going on.
Empathize with those affected and let them know everything is operating as normal.
As you continue to work through the incident, update your customers frequently.
Let your users know once a fix is in place, and keep communication clear and precise.