Historical record of incidents for Findify by Maropost
Report: "Google Cloud and Cloudflare Incident - Some Services Affected"
Last updatehttps://status.cloud.google.com/ https://www.cloudflarestatus.com/ We are currently observing partial service disruption due to a broader incident involving Google Cloud and Cloudflare infrastructure providers. This may result in degraded performance or limited accessibility for some users. Our team is actively monitoring the situation and working to mitigate any impact. We will provide updates as more information becomes available from the affected providers.
Report: "Shopify Access Token"
Last updateContinued monitoring to ensure smooth resolution for all merchants.
The issue has been identified and affected merchants contacted.
There is a issue accessing the Shopify access token affecting some merchants
We are currently investigating this issue.
Report: "Pulsar experiencing issues causing delays in Sync & Analytics"
Last updateThe incident have now been fully resolved. Actions are taken to avoid similar issues.
Sync running as normal
The issue has been identified and a fix is being implemented.
Report: "Hyperdrive issue affects Norce Product Sync"
Last updateNorce has resolved the issue and product sync is now running again.
Norce Support team are working on identifying the issue and restoring the Hyperdrive access
Report: "Temporary Delay in Sync"
Last updateSome clients experience delay in sync updates. The problem has been identified to a memory profile in the sync infrastructure & resolved. The memory has been increased & alarm implemented to prevent future problems. The search API has been unaffected and search, collections & recommendation results has been continuously served.
Report: "Norce Hyperdrive Error"
Last updateThe issue has been resolved by Norce Team.
All Norce hyperdrive feeds are returning errors and can hence not be downloaded/synced. The Norce team is investigating
Report: "Partial issues with custom Sync"
Last updateAn update has been released and the issue is resolved
A few merchants running on custom feed sync is currently experience problems to update products. The root cause has been identified and the fix is being tested.
Report: "Analytics in merchant dashboard are not loading"
Last updateWe currently have an issue with the service that handles Findify analytics. For now Findify is unable to display any analytics in the merchant dashboard. There data is still available and we are working to have it restored. This does not affect the core functionality of search, collections and recommendations on site. Update: The issue was resolved over the weekend and the data is now restored.
Report: "Analytics are not loaded for some of the merchant in the Merchant Dashboard"
Last update10:02 We currently have an issue with the service that handles Findify analytics. For now Findify is unable to display some analytics in the merchant dashboard. The data is still available and we are working to have it restored. This does not affect the core functionality of search, collections and recommendations on site. 10:36 The issue has been resolved.
Report: "Service degradation search"
Last update21:16 We have a service degradation on a number of clients. We are working to identify the root cause. 21:29 This incident has been resolved. We will continue to monitor the performance.
Report: "Access to Admin API"
Last update11:12 We are currently experiencing an issue with the admin API. This may prevent users to log into the dashboard. 11:19 The issue has been identified and a fix is being implemented. 12:42 A fix has been implemented and we are monitoring the results. 13:06 This incident has been resolved.
Report: "Increased Error rate Search"
Last update15:50 Some clients are experiencing an increase error rate. We are looking into the root cause 15:53 This incident has been resolved.
Report: "Depreciated search service"
Last updateWe just experienced a huge traffic spike (larger than we get on black Friday) that for a few minutes resulted in an increased error rate. We have increased capacity in the affected components, fine tuned the autoscaling and will now look into the source of the traffic to understand if we need to prepare for similar or even larger spikes going forward.
Report: "Search API - Access control layer"
Last updateToday in the European morning (North American night) we had an issue in the access control layer, which affected the search api causing it to lose access to the configuration service. The team jumped on the issue immediately and it was solved within 20 minutes during which the service was partially unavailable. We are changing the access control method, to make it less error prone and more resilient in the future.
Report: "Index not fully updated"
Last update16:42 Since a few minutes back we have an issue with indexing. Some clients are affected where indexing is not happening for the whole product catalogue. We are investigating this issue 16:54 The issue has been identified and a fix is being implemented. 16:57 A fix has been implemented and we are monitoring the results. 17:32 This incident has been validated to be resolved.
Report: "Autoscaling Deficiency"
Last updateDuring high volume inflow we experienced delays in the autoscaler which lead to outage of our main load balancer. This resulted in 25min of index disruptance and failure to readily serve product results. The team has scaled all systems manually and all functions are operational. We are further investigating why the autoscaler failed.
Report: "Redis Error affecting authentication"
Last update22:09 An error in Redis are affecting authentication. 22:14 The issue has been identified and a fix is being implemented. 22:19 80% restored 22:23 100 % restored. 22:28 The incident is validated to be fully restored. Monitoring is continued.
Report: "API Disrupted"
Last updateAt 10:47 we published an updated internal URL containing additional data for new functionality. This change was reviewed, tested & validated in our staging environment without any problems. Once released, our internal alarming immediately notified that clients were affected. The change was immediately rolled back but took a few minutes to take effect. At 11:03 service was confirmed to be back up running. We take this very seriously and are investigating why the initial tests didn't correctly reflect the outcome. Until that is resolved we will not publish any updates at all.
Report: "Cluster Degraded Performance"
Last updateWe are experiencing performance issues with Nodes in one cluster and a few clients are affected. Clients have been moved to functional clusters and the team are identifying the root cause.
Report: "Partial outages in the US region"
Last updateIn the afternoon of the 15th June 2023, we had an incident which caused partial outages in the US region. This was caused by an outburst of search requests which overloaded our Redis Connections. The issue was automatically detected by our internal alarming & immediately resolved where full recovery took 30-60 minutes to fully apply. No components were marked as affected by this incident. (.)
Report: "Product Sync Affected Partially"
Last updateA subset of our merchants experienced interruptions in product sync. All other components of Findify were working as intended. The issue was identified and resolved.