Rivery

Is Rivery Down Right Now? Check if there is a current outage ongoing.

Rivery is currently Degraded

Last checked from Rivery's official status page

Historical record of incidents for Rivery

Report: "Potential River failures due to GCP outage"

Last update
identified

We've observed some river failures in the US console due to a current Google outage: https://status.cloud.google.com/ This is impacting rivers utilizing GCP (Google Cloud Platform) as their Filezone and / or use GCP as a Target. Relevant error: "Error while creating new bucket in GCS internal file zone: Google Cloud bucket error: We encountered an internal error. Please try again." We are monitoring the situation and will update once we see the incident has been resolved.

Report: "River failures - can't compare datetime.datetime to datetime.date"

Last update
resolved

We have deployed a fix for the issue that was causing failures in some Rivers using S3 and Azure Blob data sources. A limited number of River runs were impacted by the following error: TypeError: can't compare datetime.datetime to datetime.date The root cause was identified and a fix was successfully deployed. Since then, no further failures have been observed. System performance has remained stable during the monitoring period. This incident is now considered fully resolved. Thank you for your patience and understanding. For any further questions please contact our Support Team.

monitoring

We have deployed a fix for the issue that was causing failures in some Rivers using S3 and Azure Blob data sources. Incident start: May 28, 2025, at 16:00 IST (13:00 UTC) Incident resolved: May 29, 2025, at 17:30 IST (14:30 UTC) The issue resulted in a limited number of River runs failing with the error: TypeError: can't compare datetime.datetime to datetime.date We are currently monitoring the system to ensure full stability. No further failures have been observed since the fix was deployed. We will post a final update once the incident is fully resolved. Thank you for your patience. Data integration (Rivery) suport team

identified

We have identified that the issue is impacting Rivers with S3 and Azure Blob as their data sources. These Rivers may fail with the following error: TypeError: can't compare datetime.datetime to datetime.date We are working on a fix and will update as progress is made. Best Regards Data integration (Rivery) suport team

investigating

We are continuing to investigate this issue.

investigating

We are currently investigating an issue causing failures in a minor portion of River runs. The error message observed is: "TypeError: can't compare datetime.datetime to datetime.date" Our team is actively working to pinpoint the exact configuration that is leading to these failures. Most River runs continue to operate normally. We will provide further updates as soon as more information becomes available. Thank you for your patience. Data integration (Rivery) suport team

Report: "River failures - can't compare datetime.datetime to datetime.date"

Last update
Resolved

We have deployed a fix for the issue that was causing failures in some Rivers using S3 and Azure Blob data sources.A limited number of River runs were impacted by the following error:TypeError: can't compare datetime.datetime to datetime.dateThe root cause was identified and a fix was successfully deployed. Since then, no further failures have been observed. System performance has remained stable during the monitoring period.This incident is now considered fully resolved. Thank you for your patience and understanding.For any further questions please contact our Support Team.

Monitoring

We have deployed a fix for the issue that was causing failures in some Rivers using S3 and Azure Blob data sources.Incident start: May 28, 2025, at 16:00 IST (13:00 UTC)Incident resolved: May 29, 2025, at 17:30 IST (14:30 UTC)The issue resulted in a limited number of River runs failing with the error:TypeError: can't compare datetime.datetime to datetime.dateWe are currently monitoring the system to ensure full stability. No further failures have been observed since the fix was deployed.We will post a final update once the incident is fully resolved.Thank you for your patience.Data integration (Rivery) suport team

Identified

We have identified that the issue is impacting Rivers with S3 and Azure Blob as their data sources. These Rivers may fail with the following error: TypeError: can't compare datetime.datetime to datetime.dateWe are working on a fix and will update as progress is made.Best Regards Data integration (Rivery) suport team

Update

We are continuing to investigate this issue.

Investigating

We are currently investigating an issue causing failures in a minor portion of River runs.The error message observed is: "TypeError: can't compare datetime.datetime to datetime.date"Our team is actively working to pinpoint the exact configuration that is leading to these failures.Most River runs continue to operate normally.We will provide further updates as soon as more information becomes available.Thank you for your patience.Data integration (Rivery) suport team

Report: "ECB Currency Rate Connectors - Status code: 404"

Last update
resolved

After monitoring we see ECB Currency Rate endpoint is now fully operational. The earlier disruption was due to an issue with the upstream source API and all associated data pipelines ("rivers") are running as expected. Should you experience any further issues or have questions, please contact our Boomi support team and we will be happy to assist you! Thank you for your patience and continued trust.

monitoring

The ECB Currency Rate endpoint is now fully operational. The earlier disruption was due to an issue with the upstream API on the source endpoint. All data pipelines ("rivers") are running successfully at this time. We will continue to monitor the situation to ensure continued stability. Thank you for your patience.

investigating

Our team is continuing to investigate the issue affecting ECB currency rate rivers. At this time, our findings suggest the issue is originating from the ECB's API itself. We are monitoring the situation closely and will provide further updates as more information becomes available.

investigating

We are currently investigating an issue causing ECB (European Central Bank) currency rate connectors to fail with a 404 Not Found error. Our team is looking into whether the source of the issue is with the ECB API. We will share updates as soon as we have more information. Thank you for your patience.

Report: "ECB Currency Rate Connectors - Status code: 404"

Last update
Resolved

After monitoring we see ECB Currency Rate endpoint is now fully operational. The earlier disruption was due to an issue with the upstream source API and all associated data pipelines ("rivers") are running as expected.Should you experience any further issues or have questions, please contact our Boomi support team and we will be happy to assist you!Thank you for your patience and continued trust.

Monitoring

The ECB Currency Rate endpoint is now fully operational. The earlier disruption was due to an issue with the upstream API on the source endpoint. All data pipelines ("rivers") are running successfully at this time.We will continue to monitor the situation to ensure continued stability. Thank you for your patience.

Update

Our team is continuing to investigate the issue affecting ECB currency rate rivers. At this time, our findings suggest the issue is originating from the ECB's API itself. We are monitoring the situation closely and will provide further updates as more information becomes available.

Investigating

We are currently investigating an issue causing ECB (European Central Bank) currency rate connectors to fail with a 404 Not Found error. Our team is looking into whether the source of the issue is with the ECB API.We will share updates as soon as we have more information.Thank you for your patience.

Report: "Instagram Source Failures in US and EU Consoles"

Last update
resolved

We are happy to update that a fix has been deployed. Starting April 21, 2025, Instagram’s API changes have affected reporting metrics. The following metrics have been deprecated: - Impressions - Plays - clips_replays_count - ig_reels_aggregated_all_plays_count These are all now consolidated under a new metric: Views. Action Required: Custom Reports: Add the new Views metric, and re-run your River starting from April 20 to refresh your Target Schema. Predefined Reports: We’ve updated them automatically. Please re-run reports from April 20 for complete data coverage. For any questions or assistance, please don’t hesitate to reach out to our Support Team. We’re here to help!

monitoring

Our engineering team has successfully removed the deprecated Instagram metrics from impacted rivers to ensure the pipelines continue running smoothly. Customers who were using the impacted metrics will receive an email from the Rivery Product Team with clear action steps once the updates to support the new Instagram metrics are finalized. Thank you for your patience as we worked to resolve this issue.

identified

Starting April 21, 2025, we are observing failures in source-to-target river connections involving Instagram data sources across both the US and EU regions. These disruptions were caused by changes in the Instagram API. Root cause: Instagram’s API (version 21+) has deprecated certain time series metrics - see more information on the metrics here: https://developers.facebook.com/docs/instagram-platform/reference/instagram-media/insights Insights data handling error: Instagram API original error: '(#100) Starting from version v22.0 and above, the impressions metric is no longer supported for the queried media. Please refer to the documentation at https://developers.facebook.com/docs/instagram-api/reference/ig-media/insights for a list of supported metrics. '. Workaround: To restore functionality and maintain continuity in reporting: The deprecated impressions metric have to be replaced with the supported read metric in all affected pipelines. We are currently active working on a fix including for the predefined reports and will update when deployed.

Report: "502 Errors for Google Analytics Source to Target Rivers"

Last update
resolved

After deploying our recent fix and closely monitoring the system, we are pleased to report that all errors related to Google Analytics have been resolved. All systems are functional and operating normally. Thank you for your patience while we addressed this issue, for any inquiries please reach out to our Rivery Support Team.

monitoring

We deployed a fix at 24/04/2025 | 14:24:42 UTC. Google recently removed access to a discovery URL that our system used to initiate Universal Analytics (UA) reports.GA4 reports are fully functional and not impacted. To address the issue for the users still using UA, we’ve made a targeted change in our system. If you're still using UA, we strongly encourage switching to GA4 reports. We’re continuing to monitor the process on our side to ensure everything runs smoothly.

identified

We have identified the issue and our team is actively working on a fix. We are committed to resolving this as quickly as possible and will provide updates as we make progress. Thank you for your patience.

monitoring

The 502 error appears to originate from the Google Analytics API endpoint. At this time, we believe the issue is external and related to a service disruption on Google’s side. Data rivers are currently running and we are closely monitoring for any further anomalies or disruptions.

investigating

Starting April 23, 2025, at 08:00 UTC, we see an issue affecting source-to-target data rivers connected to Google Analytics. Users may encounter failures with the following error message: "Failed to establish connection to Google Analytics: <HttpError 502 when requesting https://analyticsreporting.googleapis.com/$discovery/rest returned 'Bad Gateway'>". This appears to be caused by a response from the Google Analytics API. We are currently investigating the root cause and will provide updates as more information becomes available.

Report: "Unintended Task Termination During Maintenance Window"

Last update
resolved

Following continued monitoring, we can confirm that as of 09:10 UTC, all rivers have resumed normal functionality and are operating as expected. We apologize for any inconvenience this may have caused and appreciate your patience. If you continue to experience any issues, please don’t hesitate to contact our support team.

monitoring

On April 17, 2025, at 08:30 UTC, a planned instance maintenance window inadvertently caused the termination of active tasks. This affected river runs in both Prod-US and Prod-EU environments between 08:30–09:10 UTC, resulting in tasks being terminated mid-run.Although these tasks still appear as “Running” in the UI, they are not actually executing. Task executions initiated after 09:10 UTC have resumed normal functionality and are running as expected. Impact: - Tasks scheduled between 08:30–09:10 UTC were unexpectedly terminated. - Affected runs remain inaccurately displayed as “Running” in the UI. - No impact on task executions beyond 09:10 UTC. Next steps: - The next scheduled river runs will resume from the start date of the last successful execution. - Engineering is reviewing monitoring and UI accuracy enhancements to prevent recurrence.

Report: "Ticketing System Communication Disruption"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We are currently experiencing a disruption in our ticketing system where customer replies are not being received, and our agents are unable to send out messages through the platform. This issue is affecting all ongoing communications and may result in delays in support responses. We understand the critical nature of this disruption and are actively working with our technical teams and service provider to resolve it as quickly as possible. Thank you for your understanding Rivery Support

Report: "3rd party issue: Facebook Ads errors"

Last update
resolved

This incident has been resolved.

monitoring

.

identified

Facebook Ads API is currently experiencing an issue and rivers may fail with the following error: Facebook Ads Issue - [RVR-FB-110] <!DOCTYPE html> <html lang="en" id="facebook"> <head> <title>Facebook | Error</title> ... Facebook is aware and actively working to resolve the issue. For real-time updates, please follow the Facebook status page here: https://metastatus.com/

Report: "CDC enablement error"

Last update
resolved

This incident has been resolved.

monitoring

We have identified the underlying infrastructure issue affecting CDC enablement and have deployed a fix. The impact was isolated to the CDC enablement process only; all other platform functionalities remained fully operational. We are currently monitoring system performance to ensure stability. Please contact our support team with any questions.

investigating

We're currently investigating an issue that began on March 31, 2025 (16:00 UTC), where attempting to enable CDC—whether on a new or existing river— encounters the error: "Internal error encountered while creating new cdc connector. Please check your connection and contact support." This issue is only affecting CDC during the enablement process. CDC is not impacted on actively running rivers. Our team is actively working to identify the root cause and will provide an update on our progress as soon as possible.

Report: "Rivers Connection to Snowflake Failing"

Last update
resolved

The issue that caused Rivers to fail when connecting to Snowflake has been resolved. A fix has been deployed, and systems are now functioning normally. Failed Rivers will automatically pick up any missed data on their next scheduled run. We are now closing this incident. If you have any questions or concerns, please don’t hesitate to reach out to our Support team. We thank you for your understanding and apologize for the inconvenience caused.

monitoring

We are continuing to monitor for any further issues.

monitoring

Start Time: 14:40 UTC Resolved Time: 15:40 UTC Description: We experienced an issue where Rivers were failing to connect to Snowflake. Users may have seen the following error: "Connection to Snowflake failed. Please check your credentials, your warehouse permissions and your account preferences in Snowflake." A fix has been deployed, and the issue should now be resolved. We are monitoring the situation to ensure continued stability. Thank you for your patience. Rivery Support Team

Report: "High Failure Rates in CDC Rivers"

Last update
resolved

The issue with the CDC connectors has been fully resolved. All CDC streams are now operating as expected, with no data loss, as the connectors are correctly fetching data from the last known produced offset. Thank you for your patience.

monitoring

Incident Update: A remediation has been successfully deployed, and the team is actively monitoring the CDC connectors. The issue has been resolved, and all CDC streams are now functioning correctly. No data loss is expected, as the connectors have resumed fetching data from the last known produced offset. The root cause was identified as internal issues related to assuming roles during the operation of the CDC connectors. This was triggered by an automatic CD change that redirected the CDC connector’s assumed role to incorrect configurations. The team is currently investigating the cause of this change and is working on preventing similar incidents in the future.

identified

The team identified the root cause of the issue and working on remediation.

investigating

Our team is investigating high failure rates in CDC source rivers due to our cloud vendor failures. CDC source rivers might fail or retrieve no data.

Report: "Rivers are failing with the error - "An error occurred (AccessDenied) when calling the AssumeRole operation""

Last update
resolved

AWS AssumeRole Authorization Issue Timeframe: 11:30 AM - 12:52 PM UTC Impact: Rivers began failing due to an authorization error when attempting to assume an AWS IAM role. The error message was: "An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:iam::**************:user/RiveryECS is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::**************:role/Rivery-S3-Role." Resolution: The issue was resolved, and normal operation has been restored. We apologize for any inconvenience caused. Please reach out if you experience any further issues.

Report: "Customers Encountering Errors on River Screens"

Last update
resolved

We experienced an issue between 09:35 and 09:50 UTC, during which customers encountered error messages when navigating any River screen. As soon as the issue was identified, we took immediate action to revert the change, and River screens are now loading successfully. We apologize for the disruption and appreciate your patience. If you continue to experience any issues, please reach out to our support team.

Report: "customer replies are not reflected in Ticketing system"

Last update
resolved

We’ve successfully resolved the issue that was preventing customer comments from being added to the ticketing system. All missing comments have been recovered, and we do not anticipate any data loss. Thank you for your patience and understanding. Best regards, Rivery Support Team

identified

We are currently experiencing an issue with our ticketing system where customer replies are not being received. This issue started approximately 3 hours ago. Our team is actively working to identify the root cause and implement a solution. We will provide updates as soon as the issue is resolved. We apologize for any inconvenience this may cause and appreciate your patience.

Report: "Apple Search Ads Connector Issue"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

Starting January 28, 2025 at 01:00 PM (GMT) Users with scheduled Rivers using outdated groupby fields are experiencing errors. The issue is impacting multiple clients, particularly in the US and EU regions. Following the recent Apple Search Ads Connector upgrade from version 4 to version 5, we have identified an issue affecting scheduled Rivers using outdated field values. The error observed is: 'str' object has no attribute 'get' This issue arises due to a change in the field type for group by fields, which were converted to a multi-select type. Scheduled Rivers are still sending old string values, causing data processing failures.

Report: "Login Issues in EU & US Consoles"

Last update
resolved

This incident has been resolved.

monitoring

We identified that some users were experiencing login issues in both our EU and US consoles starting at 09:15 UTC. However, the issue resolved itself without any intervention from our team, and it appears to have been a momentary disruption. Resolution Details: No direct action was required from our end. The service has stabilized, and users should now be able to log in without issues. Next Steps: Our engineering team will review system logs to investigate any potential root cause. We will continue monitoring the service to ensure stability. If you continue to experience issues, we encourage you to reach out to our. We appreciate your patience and apologize for any inconvenience caused. Best Regards Rivery Support

investigating

We are aware that some users are experiencing login issues in both our EU and US consoles. Issue started at 09:15 UTC Our engineering team is actively investigating the issue and working to resolve it as soon as possible. We will provide updates as soon as we have more information. Thank you for your patience.

Report: "Activity Log Download Issue"

Last update
resolved

After monitoring the deployed fix, we have confirmed that it is effective, and normal functionality has been fully restored. We sincerely appreciate your patience and understanding throughout this process. If you experience any further issues, please don’t hesitate to reach out to our Rivery Support Team.

monitoring

A fix has been successfully deployed to address the issue preventing customers from downloading activity logs in our EU and US environments. The solution has been confirmed to work, and normal functionality has been restored. We are currently monitoring the situation to ensure stability and that no further issues arise. Thank you for your patience and understanding during this time.

identified

We are continuing to work on a fix for this issue.

identified

We have identified the root cause of the issue, which is preventing customers from downloading activity logs in our EU environment. The issue is due to recent changes made by a third-party application we utilize for log management. Our team is actively working on implementing the necessary changes to align with the updated requirements of the third-party app. Thank you for your understanding and patience as we work to resolve this issue.

investigating

We are currently investigating an issue in our EU environment where customers are unable to download activity logs. Incident Details: Start Time: January 28, 2025, at 09:09 AM UTC Impact: Customers in the EU environment cannot download activity logs. Current Status: Our team is actively investigating the root cause and working to resolve the issue as quickly as possible. We apologize for the inconvenience caused and will provide further updates as soon as more information becomes available. Thank you for your patience. Rivery support Team

Report: "Intermittent 504 Errors and Performance Degradation on US Console"

Last update
resolved

This incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are investigating an issue causing some API calls to return 504 Gateway Timeout errors. This is due to an infrastructure issue that we have identified and are working to resolve. Clients may also experience some general slowness during this time. Details: The issue started at 10:37 UTC. Our engineering team is focused on fixing the issue and restoring full functionality as soon as possible. We will provide updates as soon as we have more information. We sincerely apologize for the inconvenience and thank you for your patience while we address this matter. Rivery Support Team

Report: "Temporary Run Failures on US Console (Source-to-Target Rivers)"

Last update
resolved

Between Jan 15, 21:10 UTC, and Jan 16, 05:40 UTC, we experienced sporadic run failures for source-to-target rivers on our US console, specifically during the source-to-FZ (File Zone) task. The issue was caused by a temporary drop in the shared disk’s ability to quickly update metadata. This delay led to occasional failures in the source phase of affected rivers, with runs encountering the error: "[Errno 17] File exists." Impact: The issue only impacted rivers running on our US console. Sporadic failures occurred during the specified timeframe. Any affected runs need to be restarted from the last successful run. Action Required: No action is required from users. Affected rivers will resume automatically on the next execution. Resolution: The issue was fully resolved as of Jan 16, 05:40 UTC, and the system is now operating as expected. We sincerely apologize for the inconvenience caused and appreciate your patience and understanding. If you have any questions, please don’t hesitate to reach out to our support team.

Report: "Apple Appstore Connect: Apps Analytics Rivers are failing "Please contact Rivery Support""

Last update
resolved

The incident has been resolved. The App Store Connect Analytics connector has been successfully redesigned, developed, and deployed in production. Customers are now required to take action to reconnect using the new connection flow. Summary of Work Done: - Cause of the Issue: Apple introduced significant changes to its authentication flow, including the implementation of the SRP (Secure Remote Password) protocol. This deprecated the previous login flow, rendering existing connections unusable. - Solution Implemented: A brand-new connector was developed to comply with Apple’s new authentication and data retrieval requirements. The new flow leverages JWT-based token authentication and adheres to the updated App Store Connect API specifications. The new connector allows users to request and download a wide variety of 12 pre-defined reports (e.g., App Downloads, Purchases, Pre-orders, App Sessions, etc.). Testing & Validation: - Extensive end-to-end (E2E) testing was conducted to ensure stability and compatibility. - Documentation was updated to reflect the new process and reports available. - Deployment: The new connector is now live and ready for customer use. We encourage customers to refer to the updated documentation to follow the new setup process - https://docs.rivery.io/v1/docs/app-store-connect-apps-analytics We Thank you for your patience - please contact Rivery support for any questions or assistance you may need.

identified

We are currently facing an issue where Apple Appstore Connect: Apps Analytics Rivers are failing with the error message "Please contact Rivery Support." This issue has occurred due to an unexpected change Apple implemented in their API. Our R&D team is actively working on a solution, and we will provide updates as soon as the fix is deployed. We apologize for any inconvenience this may cause and appreciate your patience. Please don’t hesitate to reach out if you have any questions. Best regards, Rivery Support Team

Report: "CDC Infrastructure Issue"

Last update
resolved

Between 10:00 and 11:30 UTC, we experienced an infrastructure issue that impacted CDC Rivers. Scheduled runs during this time failed with error message: "Binary Log connector is failing while getting data from the source. The error: HTTPConnectionPool(host='rivery-cdc-management-service.prod.rivery.in', port=8000): Read timed out. (read timeout=20)." The issue has been resolved, and future runs are expected to automatically resume from the last successful execution. No action is required on your end. We apologize for any inconvenience caused and appreciate your understanding. If you have further questions, please contact our support team. Thank you Rivery support Team

Report: "Redshift as Target - Failed to upload the data to Redshift"

Last update
resolved

Some Rivers using Redshift as the target experienced failures with the error: "Failed to upload the data to Redshift." Impact: A small number of customers were affected. Cause: During a recent update aimed at enhancing error detection and troubleshooting for Redshift operations, an unexpected permissions issue occurred, which disrupted some Rivers' ability to complete data uploads. Timeline: Start: December 3rd, 15:00 UTC Resolved: December 4th, 11:00 UTC (following a code rollback) Resolution: The issue has been resolved, and all impacted Rivers should now operate as expected. Next Steps: We are reviewing our processes to prevent similar occurrences in the future. Please feel free to reach out to our support team if you have any further questions or concerns. Thank you for your understanding and patience as we work to continuously improve our platform. Rivery Support Team

Report: "3rd Party: Salesforce Connectors are failing with "Cannot connect to Salesforce""

Last update
resolved

We are pleased to share that the issue affecting Salesforce Rivers has been resolved. Salesforce has updated its status page to confirm that the incident has been closed on its end ( https://status.salesforce.com/generalmessages/1476 ) . Our team has also verified that Salesforce Rivers in Rivery are now completing successfully without errors. If you encounter any further issues or have any questions, please don’t hesitate to contact the Rivery Support Team. Thank you for your patience and understanding. Rivery Support

identified

We have identified that Salesforce rivers are failing with the error "'Cannot connect to Salesforce. Failure reason: http://www.w3.org/TR/html4/loose.dtd:31:2: error in processing external entity reference" We have contacted Salesforce Support, and they are actively working on resolving the issue. For more information, please refer to the Salesforce incident page: https://status.salesforce.com/generalmessages/1476 If you have any questions, feel free to reach out. Best regards Rivery Support team

identified

We have identified that Salesforce rivers are failing with the error "'Cannot connect to Salesforce. Failure reason: http://www.w3.org/TR/html4/loose.dtd:31:2: error in processing external entity reference" We have contacted Salesforce Support, and they are actively working on resolving the issue. For more information, please refer to the Salesforce incident page: https://status.salesforce.com/generalmessages/1476 If you have any questions, feel free to reach out. Best regards Rivery Support team

Report: "Intermittent Data Push Failures for Snowflake Connections Using Private Key Authentication"

Last update
resolved

This incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

We experienced sporadic failures with Rivery when pushing data to Snowflake. This issue impacted a limited subset of our users—specifically those using private key authentication (key pair) to connect to Snowflake. The failures were inconsistent and only affected certain tables at random. Our team has identified the root cause, implemented a fix, and is actively monitoring the situation to ensure stability. We apologize for any disruptions this may have caused and appreciate your patience. If you continue to experience issues, please contact our support team. Thank you for your understanding and trust in Rivery.

Report: "Delayed Execution Times and Rivers in Pending Status - US console"

Last update
resolved

This incident has been resolved.

monitoring

We have identified the root cause of the delayed execution times and pending status for Rivers, and a fix has been deployed. The majority of existing runs are now expected to complete successfully. Please note: A small number of Rivers may continue to display as "running" until they timeout naturally. There is no expected data loss; any Rivers that did not execute during this period will pick up the data in their next scheduled run. We are continuing to monitor the system closely to ensure stable operation. Thank you for your patience, and please reach out if you have any questions. Rivery Support Team

identified

The issue has been identified and we are working to resolve the issue

monitoring

We have identified the root cause of the delayed execution times and pending status for Rivers We are still working to resolve the issue and will update once a fix is implemented

identified

We are currently experiencing delays with River execution times and an increase in Rivers remaining in a pending status. Our team is actively investigating the issue to identify the root cause and restore optimal performance as quickly as possible. During this time, you may notice that: River executions are taking longer than usual to complete. Some Rivers may remain in a pending status for extended periods. We apologize for the inconvenience and appreciate your patience as we work to resolve this. Updates will be provided here as we make progress. Thank you for your understanding. Rivery Support Team

Report: "Currency Service Rivers Experiencing Failures"

Last update
resolved

The issue affecting our Currency Service Rivers has been resolved, and all Rivers are now functioning normally. The problem appears to have originated from a temporary unavailability of the ECB service, which has since been restored. No action was required from Rivery's engineering team. Thank you for your patience and understanding during this incident. Please reach out if you continue to experience any issues. Rivery Support Team

investigating

We are currently experiencing issues with our Currency Service Rivers, resulting in failures. Our team is actively working to resolve the issue as quickly as possible. We apologize for any inconvenience this may cause and appreciate your patience. Further updates will be provided here as soon as they are available. Thank you for your understanding. Rivery Support Team

Report: "Extended Execution Times in EU & US Environments"

Last update
resolved

The issue causing delayed River runs and extended execution times has been fully resolved. This incident started at 06:05 UTC on November 3rd and was completely addressed by 13:00 UTC on November 4th. Due to these delays: * Rivers with shorter timeout settings may have reached their limits and been canceled. * Rivers with longer timeouts were completed successfully, with no data loss expected. The next scheduled River run will retrieve data for any canceled runs, ensuring continued data integrity. We are investigating the root cause and will provide a detailed RCA once our review is complete. We appreciate your patience and understanding. Best regards, Rivery Support Team

monitoring

A fix has been implemented and we are monitoring the results.

identified

We are continuing to work on a fix for this issue.

identified

The issue has been identified and a fix is being implemented.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We are currently experiencing longer-than-usual execution times for Rivers across both our EU and US environments. Our team is actively investigating the issue to identify the root cause and restore optimal performance as quickly as possible. While Rivers are still running, users may experience delays in execution times. We understand the importance of timely data workflows and apologize for any inconvenience this may cause. We will provide updates here as soon as more information becomes available, including an estimated time to resolution. Thank you for your patience and understanding as we work to resolve this issue. Rivery support tream

Report: "AWS Outage impacting Rivery Consoles"

Last update
resolved

The service disruptions caused by the AWS outage have been resolved and all Rivery services are now operating normally. Users should no longer experience the “500 Internal Server Error” when attempting to open the console. Rivers scheduled to run during the AWS outage window (19:27 and 19:52 UTC), which may have failed or encountered errors due to the intermittent nature of the outage. If your Rivers were scheduled to run during this window and failed, the next scheduled run will recovering the missing data by pulling from the last successful run time. Thank you for your patience and understanding during this time. For more details on the AWS incident, visit the AWS Health Dashboard here: https://health.aws.amazon.com/health/status.

investigating

Due to an AWS outage, some Rivery services are impacted. This includes the US and EU console websites intermittently returning the message "500 Internal Server Error". We are monitoring the AWS incident and investigating the full impact on the Rivery platform. Thank you for your patience. See the AWS Health Dashboard here: https://health.aws.amazon.com/health/status

Report: "EU Console Returning 503 - Service Temporarily Unavailable"

Last update
resolved

We have resolved the issue causing a 503 - Service Temporarily Unavailable error on the EU Console. The affected components during this incident were: EU Console During the incident, users were unable to create, update, or change rivers and were also unable to log in to the system. Most scheduled rivers, activities, and new API components continued to function normally. This means that most rivers already created and scheduled ran without any disruptions. Start Time: Still under investigation, will be provided in the RCA. End Time: 07:45 UTC While the issue has been resolved and services are restored, we are continuing to investigate the root cause and will provide a detailed timeline and analysis in the RCA. Thank you for your patience, and we apologize for any inconvenience caused during this period. Rivery Support Team

monitoring

We have deployed a fix for the issue causing the 503 - Service Temporarily Unavailable error on the EU console. The service is now restored, and we are currently monitoring the system to ensure stability. We will continue to observe the situation closely and provide additional information in the next update. Thank you for your patience, and we will keep you informed of any further developments.

investigating

We are currently investigating an issue affecting access to the EU console, where users are receiving a 503 - Service Temporarily Unavailable error. Our team is actively working to identify the root cause and restore service as quickly as possible. We will provide updates here as soon as more information becomes available. Thank you for your patience and understanding. Rivery Support Team

Report: "Scheduled Rivers Not Triggering in US Console"

Last update
resolved

This incident has been resolved.

monitoring

We experienced an issue in the US console between 05:03 and 07:45 UTC, during which scheduled rivers were not triggering as expected. This issue has been resolved, and scheduled rivers are functioning as expected. Our team identified the root cause, applied a fix, and confirmed that all services are operating normally. Please note that the missed runs will not be automatically triggered. However, the data will be processed in the next scheduled run. We apologize for any inconvenience caused and appreciate your understanding. Please feel free to reach out with any question to us by submitting a support ticket Rivery Support Team

Report: "River failures for Netsuite RESTlet using customsearch"

Last update
resolved

After additional monitoring, we see that the rivers are successfully running. We are marking this incident as resolved. Please note that the incident impacted both the the US console (https://console.rivery.io/) and EU console (https://eu-west-1.console.rivery.io/) between 08-10-2024 at 8:30 UTC and 09-10-2024 at 16:30 UTC. Rivers will run normally on the next scheduled runs and capture data from the previous successful river run. Thank you for your patience while we resolved the issue. If you continue to experience any issues or need assistance from our team, please open up a ticket in the console via Help --> Contact Support and our support team will be happy to help you.

monitoring

We have deployed a fix to address the Netsuite RESTlet errors for customsearch river runs. Rivers will run normally on the next scheduled runs and capture data from the previous successful river run. We are monitoring the rivers for any issues. If you continue to experience any issues or need assistance from our team, please open up a ticket in the console via Help --> Contact Support and our support team will be happy to help you.

identified

We’re seeing an increase in errors for rivers using Netsuite RESTlet as a source where customsearch data fails to fetch. The error shown is: "Couldn\'t read customsearch" AND "Please check your access and roles. Error: HTTPSConnectionPool" AND "restlets.api.netsuite.com\', port=443): Read timed out. (read timeout=30)" We have identified the root cause for the issue and will provide updates on the resolution.

Report: "SnapChat Ads Rivers Failing with HTTP 4XX Error"

Last update
resolved

We have deployed a workaround to address the HTTP 4XX errors seen for SnapChat Ads rivers. The rivers will run successfully on the next scheduled run. Thank you for your patience, if you need any assistance from our team, please open up a ticket in the console via Help --> Contact Support and our support team will be happy to help you.

identified

We have identified that the issue is related to changes outside of Rivery's platform, as there have been no recent modifications to our code. We are in communication with SnapChat's support team to gather more information and better understand the root cause. In the meantime, we are actively exploring potential workarounds to address the error. We will update this page as soon as we have a fix or workaround in place. Thank you for your understanding and patience. Rivery Support team

investigating

We are currently facing issues with our SnapChat Ads rivers, which are failing due to HTTP 4XX errors, indicating that the SnapChat Ads API is rejecting the requests. Our engineering team is actively investigating the matter, and we will provide an update as soon as we identify the root cause. If you have any questions, please don't hesitate to reach out. We apologize for any inconvenience this may cause. Best regards, The Rivery Support Team

Report: "Facebook Source River Failures Due to Deprecated Metrics"

Last update
resolved

This incident has been resolved.

monitoring

We have deployed a fix across all environments to address the river failures caused by Facebook's deprecation of certain insights metrics. We are currently monitoring the situation to ensure there are no further issues. The next river run will automatically backfill the data for the failed runs, so no action is required from our customers at this time. Thank you for your patience and understanding.

identified

We have identified that the root cause of the river failures is because Facebook has deprecated certain insights metrics. More details on the deprecated metrics can be found in Facebook’s changelog documentation: https://developers.facebook.com/docs/pages-api/changelog/ We are actively working to resolve this issue and will update you as soon as we have more information.

investigating

We are investigating an issue where some rivers with Facebook as a source as returning the error: “Facebook Error: An error description: (#100) The value must be a valid insights metric.” We are actively looking into this and will update on our findings. Thank you for your patience and understanding.

Report: "Log Download Failure"

Last update
resolved

The issue causing the "no logs found" error when attempting to download logs has been resolved. After a thorough investigation, our team has successfully implemented a solution. Please note that for River Runs in the US console (https://console.rivery.io/) during the period between 09:00 and 12:45 UTC on September 12, 2024, logs may not be available in the activity page. We sincerely apologize for any inconvenience this may have caused and appreciate your understanding. If you require any further assistance, please don't hesitate to open a ticket in the console by navigating to Help → Contact Support. Our support team will be happy to assist you.

identified

We have identified an issue where in the US console (https://console.rivery.io/) we see "no logs found" error when attempting to download logs. The error is being actively investigated by our team, and we will provide further updates as more information becomes available.

Report: "CDC rivers are failing with 'NoneType' object error"

Last update
resolved

This incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

We have identified an issue where CDC rivers encountered a `'NoneType' object has no attribute 'get'` error during execution. Incident Time: The impact occurred between 09:08 UTC and 10:39 UTC. Current Status: The error has been reverted in all affected regions. The next scheduled runs of the rivers will catch any missed streamed data, ensuring no data loss. There is no need to manually re-run failed rivers as future executions will pick up the data. Next Steps: We are actively investigating the root cause to understand what triggered this issue, even though the solution was previously tested thoroughly. Our team is monitoring the situation closely to ensure smooth operation going forward. We will provide further updates as more information becomes available. Thank you Rivery Support Team

Report: "Rivery US Console Outage"

Last update
postmortem

The SingleStore Helios® service utilizes Kubernetes infrastructure to host database clusters. As part of the Managed Service, SingleStore constantly modernizes their networking infrastructure to keep up with the current versions of EKS. On 28-Aug-2024, SingleStore detected an incompatibility in networking configurations in a small number of cells related to supporting customers that have a large number of IP addresses/endpoints, and network setup in less-than-modern network infrastructure in those cells. The impact was limited to 10/65 of their cells, all of which were running on AWS only. The Rivery US console utilized services hosted on one of the impacted cells, resulting in network connectivity issues. SingleStore has restored networking connectivity for the affected clusters and are taking steps to standardize the networking infrastructure on these cells to ensure resilience, and minimize connectivity issues in the future. The Rivery engineering team is also exploring failover solutions in order to minimize distruptions if an underlying system component experiences disruptions in the future.

resolved

We are marking this issue as resolved as the SingleStore team has confirmed that our regions are fully operational and we have seen the US console activity return to normal. Additionally, we have cancelled any river runs that were in progress when the SingleStore outage began to ensure that the next run of your river will proceed as expected. The next scheduled river run will resume from the last successful run, ensuring that no data is missed. Please note that if you are using the “Append Only” loading method, you may notice duplicates in your target system. Thank you again for your patience as we worked to resolve this issue. If you need any assistance from our team, please open up a ticket in the console via Help --> Contact Support and our support team will be happy to help you.

monitoring

We are continuing to monitor for any further issues.

monitoring

We are seeing that service has been restored to the the US Rivery platform (console.rivery.io) and rivers are running as expected as of 19:25 UTC. We are continuing to monitoring the SingeStore Status Page as there is still an indicated service disruption (see: https://status.singlestore.com/) The SingleStore incident started at 17:05 UTC and ended 19:25 UTC on 28-Aug-2024. Due to the outage, river runs that were in progress when the incident began may or may not have completed successfully. In order to ensure all data is sent to your target system, we are cancelling any river runs that were in progress and the next scheduled river run will pick up all data from the last successful run. Note that you may see duplicates in your Target if you are using the Append Only loading method. We will provide additional details about this incident once we have more information from the SingleStore team. Thank you for your patience as we worked to resolve this issue. If you need any assistance from our team, please open up a ticket in the console via Help --> Contact Support and our support team will be happy to help you.

identified

The Rivery US platform is experiencing issues due to an incident impacting one of our underlining system components (see https://status.singlestore.com). There is a high probability that rivers may fail during this period. We are in contact with SingleStore and are actively working to restore all services.

investigating

We are currently investigating an issue with the Activities page in the US Rivery platform (console.rivery.io) returning the error "Something went wrong, please try again later." We will update as soon as we have more information.

Report: "FB Ad Creatives failure - messenger_sponsored_message field nonexisting"

Last update
resolved

This incident has been resolved.

monitoring

Incident Summary: We experienced an issue with FB ad creatives in Rivery, where the error message "Tried accessing nonexisting field (messenger_sponsored_message) on node type (AdCreative)" was triggered. This issue occurred from Monday, August 19th, 21:15 UTC until Wednesday, August 21st, 10:00 UTC. Impact: This problem affected FB ad creative reports when "all fields" were selected in the UI, or when the field ‘messenger_sponsored_message’ was included in the report set. Current Actions: We have deployed a fix, and Rivery runs should now be back to normal. If you wish to fetch the data before the next scheduled run, you can manually re-run the affected rivers. Otherwise, the next scheduled run will retrieve the data as expected. Next Steps: We are closely monitoring the situation to ensure ongoing stability. We apologize for any inconvenience this may have caused and appreciate your patience. For further assistance or inquiries, please submit a support ticket through the console. Thank you for your understanding. Rivery Support Team

identified

Incident Summary: We are currently encountering an issue with FB ad creatives in Rivery. The error message is: "Tried accessing nonexisting field (messenger_sponsored_message) on node type (AdCreative)." Impact: This issue affects FB ad creative reports when "all fields" are selected in the UI, or when the field ‘messenger_sponsored_message’ is included in the report set. Current Actions: Our engineering team has identified the problem and is actively working on a resolution. Next Update: We will update you as soon as a fix is deployed. We apologize for any inconvenience this may cause and appreciate your patience. For further assistance or inquiries, please submit a support ticket through the console. Thank you for your understanding. Rivery Support Team

Report: "Unable to Save/Manually Run Rivers in EU Console"

Last update
resolved

This incident has been resolved.

monitoring

Incident Summary: The issue affecting our EU console, which prevented users from saving configurations or manually running rivers, has been resolved. This incident occurred between 12:50 PM UTC and 13:40 PM UTC. Impact: During the incident, users in the EU region encountered errors and were unable to save configurations or manually execute rivers. Root Cause: The incident was linked to maintenance work previously announced on our status page: https://status.rivery.io/incidents/rb4s6mh43hng Resolution: Our engineering team has confirmed that the disconnection issues causing timeouts have been fully resolved. Next Steps: We will continue to monitor the system to ensure ongoing stability closely. We apologize for the inconvenience this issue may have caused and appreciate your patience during this time. Thank you for your understanding. Rivery Support Team

investigating

Incident Summary: We are currently experiencing an issue with our EU console. Users are unable to save changes or manually run rivers. Our team is aware of the situation and is actively investigating the root cause. Impact: Users in the EU region may encounter errors when trying to save configurations or manually execute rivers. Current Actions: Our engineering team is thoroughly investigating the issue to identify the underlying cause. We are working diligently to implement a fix and restore full functionality as soon as possible. Next Update: We will provide an update as soon as we have more information. We apologize for any inconvenience this may cause and appreciate your patience as we work to resolve the issue. For further assistance or inquiries, please reach out to our support team by submitting a support ticket through the console. Thank you for your understanding. Rivery Support Team

Report: "Google Ads River Failures"

Last update
resolved

After carefully monitoring ur systems we see that the issue affecting the Google Ads River data retrieval has been resolved. If you wish to fetch the data before the next scheduled run, you can manually re-run the affected rivers. Otherwise, the next scheduled run will retrieve the data as expected. Thank you for you patience as we worked on the resolutions and we apologize for any inconvenience caused. Should you continue to experience any issues, please do not hesitate to reach out to our Support Team through the platform via Help --> Contact Support. Thank you for your understanding and continued support - The Rivery team

monitoring

We are continuing to monitor for any further issues.

monitoring

Incident Start Time: 09:30 UTC Incident End Time: 12:50 UTC Description: We experienced failures with Google Ads Rivers following an upgrade made at 09:30 UTC. The issue caused rivers to fail with the error Message: "Setting the page size is not supported. Search Responses will have a fixed page size of '10000' rows." Resolution: At 12:50 UTC, we deployed a fix to address the issue. Rivers should now run as expected without encountering this error. Action Required: If you wish to fetch the data before the next scheduled run, you can manually re-run the affected rivers. Otherwise, the next scheduled run will retrieve the data as expected. We appreciate your patience and apologies for any inconvenience this may have caused you Best Regards Rivery Support Team

Report: "Increased Run Times for Redshift Target Rivers"

Last update
resolved

After monitoring the river runs, we have observed that all systems are operating as expected. We apologize for any inconvenience this may have caused and appreciate your patience as we worked to resolve the issue. If you continue to experience any issues, please do not hesitate to reach out to our Support Team through the platform via Help --> Contact Support. Thank you for your understanding and continued support - The Rivery team.

monitoring

Starting on July 03, 2024, at 01:00 PM UTC, Source to Target river runs in US, EU, and Frankfurt console region experienced longer than usual run times. A fix was deployed at 04:16 PM UTC and all run times are now back to normal. At this time, no action is required and rivers will resume upon the next scheduled run time. Thank you for your patience as we worked to resolve this issue. If you continue to experience any issues, please reach out to our Support Team through the platform via Help --> Contact Support.

investigating

We have identified that rivers with Redshift as a target are experiencing longer than usual runtimes for US console,. We are actively investigating this and we will provide an update as soon as we have more information

Report: "Unable to Save/Manually Run Rivers in EU Console"

Last update
resolved

This incident has been resolved.

monitoring

Incident Summary: The issue affecting our EU console, which prevented users from saving or manually running rivers, has been resolved. The incident occurred between 09:00 AM UTC and 12:00 PM UTC. Impact: Users in the EU region encountered errors and could not save configurations and manually execute rivers. Root Cause: The incident was related to maintenance work previously posted on our status page: https://status.rivery.io/incidents/cryhdgcgysps. This work caused a disconnection and led to timeouts in our EU console upon attempt to save/manually run Rivers Resolution: Our engineering team has confirmed that the disconnection issues causing the timeouts have been resolved. Next Steps: We will continue to monitor the system to ensure stability. We apologize for the inconvenience caused and appreciate your patience during this time. Thank you for your understanding. Rivery Support Team

investigating

Incident Summary: We are currently experiencing an issue affecting our EU console, where users are unable to save or manually run rivers. Our team is aware of the problem and is actively investigating the root cause. Impact: Users in the EU region may encounter errors or be unable to save configurations and manually execute rivers. Current Actions: Our engineering team is conducting a thorough investigation to identify the underlying cause of the issue. We are working on implementing a fix to restore full functionality as quickly as possible. Next Update: We will provide an update as soon as we have more information We apologize for the inconvenience and appreciate your patience while we work to resolve this issue. For any further assistance or inquiries, please reach out to our support team by submitting a support ticket through the console. Thank you for your understanding. Rivery Support Team

Report: "Webhook source data ingestion issue"

Last update
resolved

This issue has been resolved and rivers with Webhook as a Source in US console are now running successfully. We apologize for any inconvenience this may have caused. Please reach out to the Rivery Support Team if you encounter any further issues.

monitoring

Between June 02 13:28:17 UTC and June 04 18:03:09 UTC, Source-To-Target data rivers US console using Webhook as the source did not ingest any data. Data not captured during this period will need to be re-ingested. A fix has been deployed, and all Webhooks are now consuming data again. We apologize for any inconvenience this may have caused. Please reach out to the Rivery Support Team (in the console via Help —> Contact Support) if you encounter any further issues or have any questions.

identified

Starting June 02 13:28:17 UTC we identified an issue affecting Source-To-Target data rivers using Webhook as the source, resulting in data not being ingested. Existing Webhook Rivers: A fix has been deployed, and data capture has now resumed as normal. New Webhook Rivers Initialized on June 2nd, 13:28:17 UTC: We are actively working on a solution. Updates will be provided as soon as more information is available. We apologize for any inconvenience this may have caused and appreciate your patience as we work diligently to resolve the issue.

Report: "Errors with subset of Databricks rivers"

Last update
resolved

On 28-May-2024 between 10:00 - 21:30 UTC, there was an increase of river failures in a subset of source to target where Databricks was selected as the target. The full error message can be found below. Please note that the issue did not affect all rivers with Databricks as a target. At 21:30 UTC, a fix was deployed and the river runs that were previously failing have returned to normal. Any data that was not transmitted during the time of the incident will be captured during the next scheduled run. If you have any questions, please do not hesitate to reach out to our support team (via Help --> Contact Support in the console). Thank you for your patience as we worked to investigate and resolve this issue. "Failed to parse JSON data while loading file. Error: An error raised while trying to create a table raw.tmp_XXXXX: ('42617', '[42617] [Simba][Hardy] (80) Syntax or semantic analysis error thrown in server while executing query. Error message from server: org.apache.hive.service.cli.HiveSQLException: Error running query: [PARSE_EMPTY_STATEMENT] org.apache.spark.sql.AnalysisException: [PARSE_EMPTY_STATEMENT] Syntax error, unexpected empty statement. SQLSTATE: 42617; line 12 pos 34\n\tat org.apache.spark.sql.h (80) (SQLExecDirectW)')"

Report: "Bing Ads - River Failures"

Last update
resolved

This issue has been resolved and rivers with Bing Ads are now running successfully. Please reach out to the Rivery Support Team (in the console via Help —> Contact Support) if you encounter any further issues.

monitoring

A fix to resolve the Bing Ads river failures was deployed at 18:31 UTC today and we have seen river activity return to normal. Any data that was not transmitted during this outage will be captured during the next scheduled run. We are actively monitoring the results and thank you for your patience as we worked to resolve this issue.

identified

We have identified the issue and are currently working on deploying a fix. We will update once the fix has been deployed.

investigating

We are investigating an increase in failures with rivers where Bing Ads is the source, as many runs are returning the below error. However, manually triggering a Source-to-Target (S2T) river with the Bing Ads integration functions correctly. "No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/home/riverfw']" We are actively looking into this and will follow up with additional details as soon as they are available.

Report: "Notice of Service Interruption: Connectivity Issues with Facebook"

Last update
resolved

This incident has been resolved.

monitoring

We encountered connectivity issues with Facebook due to an error stating: "API access disrupted. Go to the App Dashboard and complete Data Use Checkup." Incident Start Time: 1:00 AM UTC Incident End Time: 7:30 AM UTC For Customers Using Rivery's App: Your Rivers should now be operational. For Customers Using Their Own Facebook Apps: According to Facebook's documentation, you must complete specific steps to restore connectivity. Please follow the instructions provided in this guide: https://developers.facebook.com/docs/development/maintaining-data-access/data-use-checkup/?locale=en_US If you are unsure whether you are using your own app or Rivery’s, please check the advanced settings of your Facebook connection. If you see details such as a client ID, secret, and access token, you are using your own app and must follow the steps in the Meta article linked above. If problems persist after completing the necessary steps, please submit a support ticket to the Rivery support team. We apologize for any inconvenience this may have caused and appreciate your cooperation. Sincerely, The Rivery Team

Report: "EU Rivery console is down"

Last update
resolved

After continuing to monitor the Rivery EU console, we can now confirm that this incident is resolved. The Rivery EU console experienced a complete outage on 15-Apr-2024 from 15:38 until 16:59 UTC. Rivers that were scheduled during the window were not initiated. Any rivers that were in progress and did not complete before the outage window will have failed (but may show as still running in the UI). We suggest reviewing your scheduled rivers and either re-running the river if the updated data is needed ASAP or waiting for the next scheduled run to capture the missing data. We apologize for the disruption and our engineering team is performing a full root cause analysis to understand and prevent this issue from occurring in the future. If you have any questions, please contact our Support Team via Help --> Contact Support in the Rivery console.

monitoring

A fix has been implemented and the team is monitoring the results. The console, river runs and api are working as expected again. Any process that should be running at the incident's time was skipped and it is recommended to be run manually.

monitoring

We have deployed a fix and the EU console is back up. We are continuing to monitor and investigate the impacts of this outage.

identified

We identified the root cause of the incident and working to fix it as soon as possible. The incident affects all of the processes in EU console, including river runs, console, and the API.

investigating

It appears that the EU console for Rivery (https://eu-west-1.console.rivery.io/) is currently experiencing downtime. The team is aware of the issue and is actively investigating the cause. We are committed to providing updates as they progress in identifying and resolving the issue. Please stay tuned for further information.

Report: "[Logic Run River] Logic run river steps are failing"

Last update
resolved

The incident has been resolved. Since 08:39 UTC aprx, all of the logic steps rivers working successfully. We're sorry for the inconvenience and we're investigating the flow made the malformed version being deployed to make sure it won't happen again.

monitoring

The team find out the issue was made due an malformed version deployed in one of our internal services. The malformed version reverted successfully and the logic's run river step is working as expected. We're working on a quick fix for the root cause of the issue and will update soon.

identified

We are continuing to work on a fix for this issue.

identified

Logic rivers are failing to run rivers under river steps with error of [RVR-ERR-001]: An error has occurred. Please contact Rivery Support. The team identified the root cause and working on a quick fix.

Report: "YouTube rivers encountering quota limit"

Last update
resolved

In order to circumvent the new YouTube quota limits, we have added a custom application connection option. Please follow the instructions in our documentation here: https://docs.rivery.io/docs/connection-youtube If you have any questions about setting up this new option, please reach out to the Rivery Support Team via the Console (Help --> Contact Support). Thank you for your patience as we worked to resolve this issue.

identified

We have identified an issue where rivers with YouTube as a source are at times returning the following error and failing: Permission Error received from YouTube: 403-The request cannot be completed because you have exceeded your <a href="/youtube/v3/getting-started#quota">quota</a>. This is due to a recent change by YouTube to Rivery's quota limit and we are actively working with the YouTube team to address this issue. Thank you for your patience.

Report: "[EU-Central-1] Console login is unreachable"

Last update
resolved

This incident has been resolved.

monitoring

A fix for the issue has been deployed.

identified

The issue has been identified and a fix is being implemented.

investigating

We're investigating an error of connecting to console login in eu-central-1. We'll provide an update in 30 mins.

Report: "Service Alert: Jira API Outage"

Last update
resolved

Jira has successfully implemented a fix and updated the incident status to monitoring. We have verified that Rivers with Jira as a source are now completing successfully. Thanks, Rivery Support Team

monitoring

We are experiencing an issue with Rivers failing in Rivery when querying the Jira API. The error received is a Status 503 – Service Unavailable. The root cause of this problem has been identified as an incident on Jira, as reported on their status page: https://status.atlassian.com/ Current Status: 🛑 Error: Status 503 – Service Unavailable 🔍 Source: Jira API 📅 Incident Start Time: Jan 18, 2024 - 08:35 UTC What We're Doing: We continuously monitor the situation and will provide updates as new information becomes available. Next Steps: We will keep you informed about any developments via this status page. Once Jira services are restored and operational, we will update the Rivery status page accordingly. We appreciate your continued partnership as a valued customer. Rivery Support

Report: "Instagram Rivers failing due to deprecated fields"

Last update
resolved

After further monitoring, we can confirm this incident is now resolved. Please note that the fields deprecated by Instagram will now return null values in your target table. Below is the list of now-deprecated metrics in each predefined report: Insights Account Lifetime - audience_gender_age_lifetime, audience_locale_lifetime, audience_country_lifetime, audience_city_lifetime Insights Media Image - engagement_lifetime Insights Media Story - exits_lifetime, taps_forward_lifetime, taps_back_lifetime Insights Media Carousel - all metrics starting with the prefix "carousel_" Insights Media All Medias - exits_lifetime, taps_forward_lifetime, taps_back_lifetime, engagement_lifetime, all metrics starting with the prefix "carousel_" Please reach out to our support team via the the Rivery Console (Help --> Contact Support) if you continue to experience any issues, and thank you for your patience as we worked to resolve this issue.

monitoring

Our team has deployed a fix to resolve the error returned by several predefined report that contained deprecated metrics. All rivers utilizing this report should now run successfully - while no updates are needed, the deprecated fields will now return null values in your target table. Below is the list of now-deprecated metrics in each predefined report: Insights Account Lifetime - audience_gender_age_lifetime, audience_locale_lifetime, audience_country_lifetime, audience_city_lifetime Insights Media Image - engagement_lifetime Insights Media Story - exits_lifetime, taps_forward_lifetime, taps_back_lifetime Insights Media Carousel - all metrics starting with the prefix "carousel_" Insights Media All Medias - exits_lifetime, taps_forward_lifetime, taps_back_lifetime, engagement_lifetime, all metrics starting with the prefix "carousel_" We will continue to monitor Instagram rivers to confirm all functionality has been restored as expected. Please do not hesitate to reach out to our support team via the the Rivery Console (Help --> Contact Support) if you continue to experience any issues.

identified

We have identified an issue with Instagram rivers that utilize the Instagram Media Story predefined report. The error returned is "Instagram Error: (#100) These metrics taps_forward, taps_back, exits are no longer supported on version v18+. For a list of alternative metrics, visit https://developers.facebook.com/docs/instagram-api/reference/ig-media/insights." This error appears to be connected to the recent API changes by Instagram that deprecated several metrics from this report. Our team is actively working to resolve this issue and we thank you for your patience.