Historical record of incidents for nyris GmbH
Report: "Elevated error rate"
Last updateThere was a temporary disruption due to a recurring issue that briefly affected our Search API. A preventive workaround has been implemented, and and search functionality is now available. A long term fix is on the way.
Report: "Elevated error rates"
Last updateThere was a temporary disruption due to scheduled maintenance, which briefly impacted Search API. The issue has since been fully resolved, and search functionality is now available.
Report: "Elevated error rates"
Last updateWe have identified the cause of this intermittent issue and are monitoring the situation while implementing a fix for the conditions under which this issue manifests. Success rates have returned to normal levels.
We are currently investigating an increase in request error rates.
Report: "DEV-2783 Recommend API returns empty result sets"
Last updateThis incident has been resolved.
After deploying the fixes, the Recommendation and Search APIs are working again. We are continuing to monitor the situation.
Search requests return no results after a configuration change. We have identified the issue and are deploying fixes.
Recommendation requests return empty result sets. We have identified the issue and are working on resolving it.
Report: "NYRIS-6191 - similarity search times out"
Last updateSimilarity search was disrupted with search requests timing out or taking 10s or more.
Report: "NYRIS-6007 Occasional timeouts when using visual search or recommendations"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
From 6:30 UTC, we are experiencing decreased availability issues with visual search and recommendation system, causing requests to have high duration or time out in irregular intervals. We are investigating the issue.
Report: "NYRIS-5902"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Many requests to object detection API are failing
Report: "nyris-5891"
Last updateWe are no longer seeing issues with our object detection system.
We have identified the cause of the issue and rolled back our object recognition system. We are continuing to monitor the situation.
We're observing an issue with our object detection service that may cause requests to fail.
Report: "Degraded visual search performance"
Last updateThis incident has been resolved.
We are continuing to work on a fix for this issue.
Impact: Non-OCR search results might be incomplete, search requests are slow.
Report: "Impacted success rate and increased latency"
Last updateDue to issues our CSP experienced in Europe, we observed a drop in request success rate as well as an increase in request latency. The incident has since been resolved. We are continuing to monitor the situation on our end.
Report: "Increased request latency"
Last updateThis incident has been resolved.
The fix has been applied and search is back. We are continuing to monitor the situation.
We are continuing to implement a fix for the issue.
We have identified the cause of the issue and are currently implementing a fix.
We are investigating an issue that may result in increased request latency and failing requests.
Report: "[INFRA-1627] Search API outage"
Last updateThis incident has been resolved.
Our services are back up and we're continuing to monitor the situation.
We have identified the cause of the service outage and have implemented a fix
We are currently investigating an outage of multiple APIs.
Report: "Services not reachable"
Last updateThis incident has been resolved. Due to a network configuration issue, the services were not reachable. This has affected all of the services, resulting in timeouts on requests.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Object recognition outage"
Last updateDue to a network problem, the object detection service was not available. Queries to the service would take >=10s and return a timeout error. This impacts all object detection queries between 31.07. 06:00 and 01.08. 06:00 A fix for the problem has been implemented.
Report: "Timeouts on editing items through the Data API"
Last updateThis incident has been resolved.
We have implemented a workaround and are working on a solution.
We have received reports of timeouts on operations performed through the Data API. The occurrences start as early as 2022-04-28 07:00 UTC. We have identified the problem and working on a solution.
Report: "Object detection fails with error code 500"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Recommendations API returns 404"
Last updateChecks have been implemented to ensure similar problems are detected immediately.
The problem was related to a configuration change from 2022-05-14 and has been fixed.
The issue has been identified and a fix is being implemented.
Report: "INFRA-1315"
Last updateOutage detected: 13.06.2022, 04:34 CEST Outage resolved: 13.06.2022 10:56 CEST Description: Due to a service mesh configuration issue our search APIs were unavailable. We have fixed the issue and are currently monitoring the situation.
Report: "Delay in image processing"
Last updateThis incident has been resolved.
We have tuned the processing configuration and processing has cached up. We continue to monitor the situation for new delays.
Image processing is lagging behind changes performed through feed imports and Data API. This can impact search of newly added items or items with changed images or image URLs.
Report: "Object detection fails with a server error"
Last updateThis incident has been resolved.
We observe a high error rate on the object detection API. We are investigating the issue.
Report: "Metrics on statuspage not updating"
Last updateThe issue has been fixed.
The metrics on status.nyris.io are not updated due to an internal change. The actual services are operational. We are working on a solution.
Report: "Intermittent search outage"
Last updateBetween 2022-04-24 20:50 and 2022-04-25 07:35, the search API could fail with status code 500 for intervals of multiple minutes.
Report: "Increased error rate for region detection endpoint"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented. To our current knowledge, the error only affects invalid requests, which are misclassified as a server errors and provide a wrong status code and error message.
We are observing an increased error rate for the object detection API and are investigating
Report: "recommendations API requires additional parameter"
Last updateWhen using the recommendations API, an optional parameter was marked as required. This resulted in HTTP code 400 with an error message like "The requestIndexIdString field is required." This is fixed on 2022-03-02 at 16:00 UTC
Report: "Customer portal is not reporting requests (NYRIS-5169)"
Last updateThe incident has been resolved. Due to wrong configuration, the request information could not be fetched. The issue started at 2022-02-23 1:10 PM
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Timeout on similarity search"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Requests using similarity search time out after 10s. We are investigating the issue.
Report: "INFRA-1112 text search does not produce results"
Last updateWe found a problem, which resulted in text search producing results. The problem started at 10:29 UTC and is resolved 16:47 UTC
Report: "INFRA-629"
Last updateAn update error caused search requests to fail between 16:25 and 16:56 UTC.
Report: "INFRA-989"
Last updateWe are no longer seeing errors or degraded performance. The incident started on 03.10.2021 at around 16:00.
Search appears to be stabilizing. We are continuing to monitor performance.
We have traced the issue to an error inside our message queuing backend and have applied fixes.
We are investigating reports of search failure for multiple indices.
Report: "INFRA-975 (exact search)"
Last updateImpact: Search requests using exact search would result in errors. Started: 2021-09-26 09:04 UTC Ended: 2021-09-26 11:15 UTC
Report: "DATA-538 data in the index might be out of date"
Last updateThis incident has been resolved.
We observe problems processing imports. The data in the index might be out of date.
Report: "NYRIS-4853 Statistics in the portal show wrong numbers"
Last updateProblem has been fixed and the statistics recalculated.
Statists in the portal dashboard show high number of items and images. The actual search is not affected.
Report: "INFRA-944 /find/v1 returns 500"
Last updateThis incident has been resolved.
Issue is fixed. We will continue to monitor it.
We are currently investigating this issue.
Report: "INFRA-933"
Last updateAffected: Similarity search was unavailable from 01:00 due to a backend issue. Resolved: 13:50.
Report: "INFRA-876"
Last updateThis incident has been resolved.
Fetching requests is sometimes slow
Requests are loading slow or time out.
Report: "CS-200"
Last updateAffected: Search API queries using text recognition. Items recognized by text extracted from the images would not be returned or get a lower score. Resolved: 2021-07-30 8:30 PM
Report: "INFRA-877"
Last updateSearch requests failed due to internal search component unavailability between 7:30 and 8:00 UTC
Report: "INFRA-832"
Last updateCustomer statistics were not shown in the Portal dashboard due to a database issue.
Report: "INFRA-800: Portal does not show statistics"
Last updateDue to an error in the backend, customer statistics are not shown in the portal dashboard.
Report: "INFRA-747"
Last updateSimilarity search requests failed until 2021-04-26 06:40 due to a database problem
Report: "INFRA-744"
Last updateThis incident has been resolved.
We have found the issue and implemented a workaround.
We are experiencing slow responses and timeouts on requests using similarity search.
Report: "INFRA-732 low success rate on B environment"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are observing long response times and some failed search requests. The cause is not known yet
We are currently investigating this issue.
Report: "INFRA-699 database maintenance (online)"
Last updateMaintenance completed. Due to high traffic spikes while availability was reduced due to maintenance, the average request duration rised to 1.59 seconds. It is now returned to <0.40 seconds.
Maintenance on similarity search database has to be performed. No outage expected. The request speed might be affected during component replacement. Expected end of maintenance: 16:00 UTC
Report: "MATCH-192"
Last updateTests have been extended to check the default search modes.
Searches with default x-options did not trigger OCR text search
Report: "INFRA-632"
Last updateSimilarity search requests are slow or time out
Report: "INFRA-639"
Last updateSearch requests using Similarity Search are slow or time out after 10s
Report: "INFRA-555"
Last updateWe are currently investigating this issue.
Report: "Incident INFRA-552"
Last updateDeployment of new search component caused memory limits to be exceeded resulting in a crash of the server.
Searches using similarity search fail