Historical record of incidents for AppZen
Report: "Invoices are failing to integrate to Oracle"
Last updateThis incident has been resolved.
Currently being investigated with Oracle
Report: "Invoices are failing to integrate to Oracle"
Last updateCurrently being investigated with Oracle
Report: "Appzen Expense Audit: Reports may not be actionable for some customers"
Last updateThis incident has been resolved.
We are continuing to investigate this issue.
We have discovered an issue at Appzen where some users/customers may not be able to action on reports. We apologize for any inconvenience as our teams work with urgency to resolve.
Report: "Appzen Expense Audit: Reports may not be actionable for some customers"
Last updateThis incident has been resolved.
We are continuing to investigate this issue.
We have discovered an issue at Appzen where some users/customers may not be able to action on reports.We apologize for any inconvenience as our teams work with urgency to resolve.
Report: "Concur Outage Issue - System Recovery"
Last updateThis incident has been resolved.
Due to a system wide Concur Outage earlier today, we are in the process of recovery to align our Integrations to normal operating functionality. Thank you for your patience as we work to resolve.
Report: "Concur Outage Issue - System Recovery"
Last updateThis incident has been resolved.
Due to a system wide Concur Outage earlier today, we are in the process of recovery to align our Integrations to normal operating functionality.Thank you for your patience as we work to resolve.
Report: "Invoice & Expense Report Ingestion Delay"
Last updateThis incident has been resolved.
We are currently experiencing a degradation in service related to a delay in the ingestion of some Invoices & Expense Reports. We apologize for any inconvenience as our teams work with urgency to resolve.
Report: "Slowness in auditing reports and ingestion of invoices"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We are currently experiencing a slowness in service related to the ingestion of some Invoices & auditing of expense reports. We apologize for any inconvenience as our teams are actively working to resolve this with urgency.
Report: "Invoice & Expense Report Ingestion Delay"
Last updateEA and AAP processing are back to normal processing speed & issue has been resolved.
We are currently experiencing a degradation in service related to a delay in the ingestion of some Invoices & Expense Reports. We apologize for any inconvenience as our teams work with urgency to resolve.
Report: "AI Analytics Outage"
Last updateThis incident has been resolved.
AppZen is currently experiencing an outage with our AI Analytics functionality. We have identified the issue and are working with priority to correct, and will provide updates as soon as they become available. We apologize for any inconvenience and appreciate your patience as we work expediently to resolve.
Report: "Invoice WORKBENCH Navigation Issue"
Last updateThis issue has been resolved.
Workbench only shows the result of 1st page and does not move to the following pages.
Report: "Reports Getting Stuck in Pending Auto Audit status"
Last updateThis incident has been resolved.
We have fixed the issue. Currently, backlogs are getting cleared. The next update will in 2 hours.
We are continuing to work on fixing the issue. The next update is in 2 hours.
We are continuing to work on a fix for this issue.
We are continuing to work on a fix for this issue.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
We have identified an issue with Reports Getting Stuck in the Pending Auto Audit status. The team is currently addressing the matter as a top priority. The Incident Response Team has been activated and is probing the incident's root cause. We will provide you with updates on the progress.
Report: "Total amount on Expense report is not including the amount for Itemised line - Impact for ChromeRiver Customers"
Last updateWe got confirmation from the Chrome River team asking us to redirect the customers to the CR help desk, and they will address it on a customer-by-customer basis. Thank you for your understanding & patience with this matter.
The Chromeriver team has implemented the fix, and all the new reports are crossing over correctly We are still working with them to correct the affected reports We will share the next update in 2 hours
The Chromeriver team has implemented the fix, and all the new reports are crossing over correctly We are still working with them to correct the affected reports We will share the next update in 2 hours
The Chromeriver team has implemented the fix and all the new reports are crossing over correctly We are still working with them to correct the affected reports We will share the next update in 2 hours
The Chromeriver team has implemented the fix and all the new reports should be crossing over correctly We are working with them to correct the affected reports We will share the next update in 2 hours
The Chromeriver team has identified that the issue is with Hotel type line items only We are working with them to fix the issue on priority We will share the next update in 2 hours
We are currently working with the Chromeriver team to find the root cause of the issue. We will share the next update in 2 hours
We have identified an issue with the report total where there is an itemized line in the report. In the ChromeRiver payload, the total amount calculated is not correct. We have escalated the issue with the ChromeRiver team.
Report: "Delay in Updating Post Audit Workflow Status in Concur"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
We are currently seeing improvements in the workflow action and will continue monitoring moving forward.
We have noticed delays in the workflow status updating in Concur after actioning a report in AppZen. This is caused by an error response from the Concur API for some applicable customers. We have opened a case with Concur to investigate further and we will keep you updated. We apologize for the inconvenience caused due to this.
Report: "Autonomous AP - Invoices stuck in Processing"
Last updateWe have Re-run the audit for all the invoices and marking the Incident as resolved
We have fixed the invoices stuck in Validating state We are currently working on the invoice in Processing state
We are currently working on the invoice in Validating state
We have fixed all invoices stuck in Approval Workflow We are currently working on the invoice in Validating state
A fix has been implemented & we are monitoring the same.
We have identified an issue with AppZen's Invoice Processing. The team is currently addressing the matter as a top priority. The Incident Response Team has been activated and is probing the root cause of the incident. We will provide you with updates on the progress.
A fix has been implemented & we are monitoring the same.
The issue has been identified and a fix is being implemented.
We have identified an issue with AppZen's Invoice Processing. The team is currently addressing the matter as a top priority. The Incident Response Team has been activated and is probing the root cause of the incident. We will provide you with updates on the progress.
Report: "Issues with Autonomous AP - Invoice Processing"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have fixed all invoices stuck in Validating state We are currently in the Monitoring phase
The issue is still being investigated by the team on the highest priority. We are working to fix the issue. We will keep you updated on the progress. The next update will be in 2 hours.
The issue is identified by the team. We are working to clear the backlogs. We will keep you updated on the progress.
The issue is still being investigated by the team on the highest priority. We are working to fix the issue. We will keep you updated on the progress. The next update will be in 2 hours.
We have identified an issue with AppZen's Invoice Processing. The team is currently addressing the matter as a top priority. The Incident Response Team has been activated and is probing the root cause of the incident. We will provide you with updates on the progress.
Report: "Reports Getting Stuck in Pending Auto Audit"
Last updateThis incident has been resolved.
We are currently in the Monitoring phase The number of reports stuck in Pending Auto Audit is reducing gradually
The fix was rolled out. We are currently in the Monitoring phase
A fix has been implemented & we are monitoring the same.
The issue has been identified and a fix is being implemented.
We have identified an issue with Reports Getting Stuck in Pending Auto Audit The team is currently addressing the matter as a top priority. The Incident Response Team has been activated and is probing the root cause of the incident. We will provide you with updates on the progress.
Report: "Pending Audit reports"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
Reports continue to be audited by AppZen and the queue is going down. We will continue monitoring and provide further updates.
We have identified some delays in audit times for reports running intermittently. This is being investigated and we will update you soon with more details.
Report: "Unable to view receipts in AppZen"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have identified an issue with the Expense Audit for the US Production Region. You might not be able to view the receipt and our team is investigating and working on resolving the issue on high priority. We will continue to provide further updates. We apologize for any inconvenience caused due to this.
Report: "Concur US Outage - 4th Mar 2024"
Last updateThis incident has been resolved.
Hi There, The Concur issue is resolved & we are back up online. We regret the inconvenience caused due to this. Regards, AppZen Support Team
We are staying in communication with the Concur Dev team and are working to receive further updates.
Concur is experiencing an outage in the US. We will do the best we can to keep you updated on the progress of the situation. This will affect the use of our solution for customers using the Concur platform. You can access the Concur Status Page here. https://open.concur.com/#public_sector
Report: "Issues with AAP Invoice Processing"
Last updateThis incident has been resolved.
Hi There, The issue is identified & fixed by the team. The backlog clearing is in progress and catching up. We will keep you updated on the progress. The next update will be in 2 hours. We regret the inconvenience caused due to this. Regards, AppZen Support Team
Hi There, The issue is identified & fixed by the team. The backlog clearing is in progress and catching up. We will keep you updated on the progress. The next update will be in 2 hours. We regret the inconvenience caused due to this. Regards, AppZen Support Team
Hi There, The issue is identified & fixed by the team. We are working to clear the backlogs. We will keep you updated on the progress. The next update will be in 2 hours. We regret the inconvenience caused due to this. Regards, AppZen Support Team
Hi There, The issue is still being investigated by the team on the highest priority. We are working to fix the issue. We will keep you updated on the progress. The next update will be in 2 hours. We regret the inconvenience caused due to this. Regards, AppZen Support Team
Hi There, We have identified an issue with AppZen's internal database. The issue is being looked upon by the team on the highest priority. We are working to fix the issue. We will keep you updated on the progress. The next update will be in 2 hours. We regret the inconvenience caused due to this. Regards, AppZen Support Team
Report: "AppZen AAP Inbox - Email Delivery Warning"
Last updateThis incident has been resolved.
Hi There, We have identified an issue with AppZen AAP Inbox Email Delivery. Customers may have received an email delivery failure warning. The issue with Email Delivery occurred due to a change in DNS Entry which wiped out the DNS for the appzen.ai domain and resulted in email delivery failure. The issue has been fixed now, and further we are monitoring the incoming emails. No action is required from the customer end for now and we will update if any action is required. We regret the inconvenience caused due to this. Regards, AppZen Support Team
Report: "Issues with Invoice and Expense Report Processing"
Last updateThis incident has been resolved.
Hello, The issue with Expense Audit and Autonomous AP is resolved. Regards, AppZen Support Team
Hello, We have fixed the issue with Expense Audit and Autonomous AP which was impacting the Invoice as well as Expense Report processing. Currently we are monitoring the same. Will provide next update in 2 hrs. We apologise for any inconvenience caused due to this. Regards, AppZen Support Team
Hello, We have identified an issue with Expense Audit and Autonomous AP impacting the Invoice as well as Expense Report processing. We are investigating and working on resolving the issue on high priority. We will continue to provide further updates. We apologize for any inconvenience caused due to this. Regards, AppZen Support Team
Report: "AppZen Expense Audit and AAP Degraded Performance due to AWS Outage"
Last updateThis incident has been resolved.
Hi, Greetings! We are glad to inform you that the report ingestion has been resolved. However, reports are being audited and are in the queue. We have scaled up the audit tool to start pushing through reports quicker and you should see this over the next few hours.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
Hi, Greetings! We would like to enlighten you that AWS is currently experiencing an outage that has an impact on AppZen services such as Expense report ingestion, Invoice ingestion, and Audit. AppZen support is closely monitoring the AWS outage notifications and we will keep you posted with the updates as we receive them. We regret the inconvenience caused due to this. Feel free to reach out AppZen Support if any queries. Regards, AppZen Support Team
Report: "Mastermind Analytics is Down for few customers"
Last updateAs per the latest update, we are happy to confirm that the issue with Mastermind Analytics has been resolved.
We are continuing to work on a fix for this issue.
Our engineers are addressing the issue and working on a solution for the identified issue with Mastermind Analytics. We will continue to provide further updates. Thank you for your cooperation and patience.
We have identified an issue with Mastermind Analytics. Only a few customers may experience an issue accessing Mastermind Analytics and our team is working on resolving the issue on high priority. We will continue to provide further updates. We apologize for any inconvenience caused due to this.
Report: "Delay in Processing Invoices"
Last updateThis incident has been resolved.
Hi, We are glad to inform you that the issue with invoice extraction is resolved. And, the invoices are getting extracted and processed without any issues. We will continue to monitor the status and provide an update once all the invoices in the queue are processed. Regards, AppZen Support Team
Hi, We have identified an issue with invoice extraction which is causing delays in processing invoices. We are working on fixing this issue with the highest priority. Our next scheduled update will be in one hour or sooner should a significant change occur. Regards, AppZen Support Team
Report: "Concur US Outage - 07th Feb 2023"
Last updateThis incident has been resolved.
Hello, We are glad to inform you that this issue has been resolved and the status of the report is getting updated. We will continue to monitor the report processing for the next 2 hours and update once all the reports are processed. Regards, AppZen Support Team
Hello, We are actively working on this issue and we will provide an update on the progress of the restoration in an hour or sooner if any significant change occurs. Regards, AppZen Support Team
Hello, According to the latest update on the outage, Concur support has restarted an application to restore services. We are further checking on the next steps and we will keep you updated on the progress of the restoration. Regards, AppZen Support Team
We are staying in communication with Concur Dev team and are working to receive further updates.
We are continuing to monitor the Concur Status Page for the latest notifications on this ongoing outage.
Concur has been experiencing an outage in the US. This will affect the use of our solution for customers using Concur platform. You can access the Concur Status Page here. https://open.concur.com/
Report: "Unable to login to AppZen - EU Environment"
Last updateHello, As per the latest update, we are happy to confirm that the Login Issue with the AppZen EU region has been resolved. Please keep monitoring the Status Page updates related to the RCA." Regards, AppZen Support Team
Hello, We have identified a Login issue with our EU region (https://eu.appzen.com/console/home.html#/). We are working on resolving this issue with the highest priority and we will keep you posted on the progress on restoration. We regret the inconnivience caused due to this. Regards, AppZen Support Team
Report: "Mastermind Analytics Dashboard Data Issue"
Last updateHi, We have identified the issue and working on a solution for this issue and currently migrating the dashboards in stages. If you observe any issues in existing dashboards and looking for early migration to the new solution, please contact AppZen Support. We regret the inconvenience caused due to this. Regards, AppZen Support Team
We are continuing to work on a fix for this issue.
Our engineers are addressing the issue and working on a solution for the identified issue with Mastermind Analytics. We will continue to provide further updates. Thank you for your cooperation and patience.
Hi, We are working on the investigation and identification of the cause for this issue with Mastermind Analytics. We will continue to provide further updates on the progress of the restoration. Regards, AppZen Support Team
Hi, We have identified an issue with Mastermind Analytics, which is causing the data in Mastermind Dashboards to be out of date by a few days. Our team is working on fixing this issue on high priority and we will continue to provide further updates. Sorry for any inconvenience caused due to this. Regards, AppZen Support Team
Report: "Unable to Login to Amazon Environment | Login Issue"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix will be implemented soon.
We are currently investigating an issue that is causing Amazon Environment users the inability to log in. We will be providing updates soon. Thank you for your patience and cooperation. Apologies for any inconvenience caused.
Report: "Unable to action reports and Status Sync issue"
Last updateHi, We are glad to inform you that the Expense Report Workflow Status issue has been resolved. As per an update from Concur on RCA, this issue was caused due to disabling the AppZen App accidentally by a Concur Engineer. Reenabling the AppZen app at the Concur end resumed the expense reports flow and status sync which fixed this issue. We are following up with Concur in order to get preventive measures to avoid such issues in future. Regards, AppZen Support Team
Hello, We are glad to inform you that the Expense Report Workflow Status issue has been resolved. We will publish RCA for this issue shortly. Apologies for the inconvenience caused and thank you for your patience and co-operation. Regards, AppZen Support Team
Hello Team, This is to inform you all issue has been resolved and we are monitoring further progress. We will keep you updated on the monitoring status and RCA of this issue. We apologize for the inconvenience caused and thank you for your patience and cooorpation. Thank you, AppZen Support Team
We are still following up with Concur and they have assigned the problem with their internal group for further action, we will continue to update the status on further progress. Thank you for your patience and cooperation. Best Regards, AppZen Support Team
Our team is continuously following up with Concur support. Once we hear back from them, we will be sure to update you on progress.
We are continuing to work with Concur team on a fix for this issue.
Hello, Our team has been engaged and the issue has been identified. This appears to be impacting only customers using Concur and we are working with them to resolve the issue. Regards, AppZen Support Team
Hello, We are experiencing an issue with Approve/Reject button functionality in AppZen Expense Audit due to which auditors are unable to Approve/Reject expense reports. The workflow status issue was noticed for some customers after actioning a report is due to Concur API connection issue. We have opened a case with Concur to investigate further. Our team is investigating and working with Concur team on resolving the issue on high priority. We will continue to provide further updates. We apologize for any inconvenience caused due to this. Regards, AppZen Support Team
Report: "Unable to Approve and Reject Expense Reports in AppZen"
Last updateThis incident has been resolved.
Hi, Our engineers have resolved this issue. The auditor needs to clear the Cookies, Browser cache, and History to use AppZen seamlessly. Regards, AppZen Support Team
Hi, We are experiencing an issue with Approve/Reject button functionality in AppZen Expense Audit due to which auditors are unable to Approve/Reject expense reports. Our team is investigating and working on resolving the issue on high priority. We will continue to provide further updates. We apologize for any inconvenience caused due to this. Regards, AppZen Support Team
Report: "Users from some regions are not able to redirect to AppZen URL page and receiving 403 error"
Last updateThis incident has been resolved.
Hi, We have identified and deployed a fix for the issue with unable to redirect to AppZen Page with Error 403. We are currently monitoring the issue and we will keep you posted on further updates. Apologies for the inconvenience caused due to this. Regards, AppZen Support Team
This has been identified and applied to the fix to unblock requests for impacted regions.
Report: "AppZen is Facing Issue During Workflow Refresh due to Concur API blocked"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the progress. We will keep you updated on further result of monitoring.
Critical issue has been resolved and reports are started processing now, stuck reports will take some time to process and complete. We will monitor and keep you updated on further progress. Thank you.
Hi, We have identified an issue with External Workflow status not updating for the Reports for US Region. According to the latest update, our engineers have identified the issue and implementation of the fix is in progress. Apologies for the inconvenience caused. Regards, AppZen Support Team
Hello Team, We are currently working to fix the issue and will keep you updated on further progress. Thank you for your patience and cooperation. Thank you,
Report: "AppZen Services Impacted due to Amazon Web Services (AWS) Services Event"
Last updateHi, As per the latest update, the issue with the AWS service event is resolved, and impacted services are fully recovered and operational now. Regards, AppZen Support Team
Hi, As per the latest update, the issue with AWS service event is resolved. And AWS is currently working on impacted services recovery. We are working on impact Analysis as well as continuously monitoring the AWS Services Status and we will keep you updated on the progress of the restoration. Regards, AppZen Support Team
Hi, AWS is currently experiencing a service event, which is affecting parts of the AppZen services. Specifically, this is impacting Workday customers as well as other AppZen services. We are monitoring further updates on this issue and we will keep you updated. Regards, AppZen Support Team
Hi, AWS is currently experiencing a service event, which is affecting parts of the AppZen services. Specifically, this is impacting Workday customers as well as other AppZen services. We are working on impact Analysis as well as continuously monitoring the AWS Services Status and we will keep you updated on the progress of the restoration. Regards, AppZen Support Team
Report: "Delay in Updating Post Audit Workflow Status"
Last updateThis incident has been resolved.
Hi, We are continuing to observe the intermittent delays in workflow status after actioning a report due to a delay in response from Concur API. We are working closely with the Concur team to come to a resolution and this is on priority. We will be sure to provide continuous updates Apologies for the inconvenience caused. Regards, AppZen Support Team
Hi, We are continuing to observe the intermittent delays in workflow status after actioning a report due to a delay in response from Concur API. Our engineers are working with Concur team to investigate and resolve this issue. Regards, AppZen Support Team
Hi, We are continuing to observe the intermittent delays in workflow status after actioning a report due to a delay in response from Concur API. Our engineers are continuously working to investigate and resolve this issue. Regards, AppZen Support Team
Hi, We have noticed intermittent delays in the workflow status updating in Concur after actioning a report due to a delay in the response from the Concur API for some applicable customers. We have opened a case with Concur to investigate further and we will keep you updated. We apologize for the inconvenience caused due to this. Regards, AppZen Support Team
Report: "Intermittent Login | EU Environment"
Last updateWe sincerely apologize for the login issue for our EU region customers We are pleased to inform you that, we have identified the root cause along with remediation for the issue as below. **Root Cause:** This issue occurred because of the server was running at max database memory. Which caused the code to throw exceptions turning into login failures. **Remediation:** We deployed a change with increasing the Server memory. And prevent server from blocking the login services when memory error is encountered.
This incident has been resolved.
We are continuing to monitor for any further issues.
Hello, Apologies for any inconvenience caused due to this issue. We would like to inform you that, we found out cause of the issue and fixed the same. We have verified EU login and it is working without any issue now. We will keep monitoring this further. Thank you for your cooperation, AppZen Support Team
Hello, We have identified an "Intermittent Login issue with our EU Environment" (https://eu.appzen.com/console/home.html#/) . Our internal team is working on resolving this issue with highest priority. We will update you soon. Thank you, AppZen Support Team
Report: "Expense Reports - Workflow Stuck in PEV due to Concur API Error"
Last updateHi, We sincerely apologize for the issue with report workflow stuck due to Concur API error. We are glad to inform you that we have RCA in place by Concur along with the fix to avoid reoccurrence of this issue. Below is the detailed RCA. **Impact as Reported:** Customers depending on AppZen services observed “Report stuck in PEV”. AppZen integration for Expense Report processing has been interrupted causing numerous customers with Expense reports to be stuck in queue. Impact to customers occurred at different periods during the event duration. Given the nature of the product service impact this was classified as high priority requiring immediate mitigation. **Root Cause:** The event was caused by a bulk creation of system users created on behalf of the customers that occurred on September 21st. These system users are necessary to allow AppZen to make calls to Concur on behalf of the customer for their integration. Specifically, some of these system users created in bulk did not replicate across the Concur Platform completely. This partial data for the system users is the root cause for the API calls to fail with _Access Denied_ error. **Corrective Measures:** - We have mitigated the system user replication issue by forcing a replication to required sub-services. This completed and was verified September 29th and was reverified from logs on September 30th. - We have reviewed our process to create users in bulk finding a deficiency in the handling/replication process of system users. - We have established corrective actions that will address this issue from occurring in the future. Short term mitigation is available immediately. Full remediation to be addressed with improvement in our user creation/management automation. Regards, AppZen Support Team
Hi, We are happy to inform you that the Concur API Issue has been resolved. Apologies for the inconvenience caused. Regards, AppZen Support Team
Hi, We are happy to inform you that the Concur API Issue has been resolved. Apologies for the inconvenience caused. Regards, AppZen Support Team
Hi, We are happy to inform you that the Concur API Issue has been resolved. It will take some time to reflect all the reports in AppZen. Regards, AppZen Support Team
Hello, The fix is in progress as we are in touch with Concur team in order to get this resolved. We will do the best we can to keep you updated on the progress. Thank you for your patience and cooperation. Regards, AppZen Support Team
Hello, The implementation and fix are in progress in collaboration with Concur team. We will make sure to give updates on the progress of the restoration. Thank you for your patience and cooperation. Regards, AppZen Support Team
Hi, We are in touch with concur team in order to get the fix implemented as early as possible. We will keep you updated on the progress of the restoration. Regards, AppZen Support Team
Hi, We are happy to inform you that we have identified the fix for Concur API Issue and currently the fix is being implemented with the help of Concur. The fix will be deployed soon in order to get this resolved. We will keep you updated on the progress of the restoration. Regards, AppZen Support Team
Hi, We are still actively working with Concur on this issue and we will be sure to provide timely updates on the progress. Apologies for the inconvenience caused. Regards, AppZen Support Team
Hi, We have raised and high-priority incident with Concur Support for the Concur API Issue. This specific issue is causing delays for ingestion as well as reports stuck in Pending External Validation. We are working closely with the Concur team to come to a resolution and this is our top priority. We will be sure to provide continuous updates on the progress of the restoration. Apologies for the inconvenience caused. Regards, AppZen Support Team
Hi, We are continuing to observe the intermittent delays in workflow status in Expense Audit after actioning a report due to a delay in response from Concur API. We are continuously following up with Conur on further updates on a resolution. We sincerely apologize for the inconvenience caused due to this. Regards, AppZen Support Team
Hi, We have noticed intermittent delays in the workflow status - Ingesting report as well as reports stuck in Workflow due to a delay in the response from the Concur API issue for some applicable customers. We are continuing to follow up with Concur for further updates. We sincerely apologize for the inconvenience caused due to this. Regards, AppZen Support Team
We are continuing to monitor for any further issues and updates.
We are continuing to monitor for any further issues and updates.
We are continuing to monitor for any further issues.
We have noticed intermittent delays in the workflow status updating in Concur after actioning a report due to a delay in the response from the Concur API for some applicable customers. We have opened a case with Concur to investigate further. According to the update on Concur Status page, there is an issue with their Data Center and currently Concur is working on fixing the issue.
Report: "Report Ingestion Issue for Workday Platform Users"
Last updateHi, We sincerely apologize for the report ingestion failure issue caused for Workday platform users. We are pleased to inform you that, we have identified the root cause along with remediation for the issue as below. **Root Cause:** AppZen has deployed a new feature to retain the auditor feedback for resubmitted reports. During resubmit, for Retain Auditor Feedback feature, code tries to compare the report lines from the previously submitted version of the same report. For workday customers, the external line id value was coming as blank and it was causing to report ingestion failure. For the workday, we temporarily turned this feature off which fixed the report ingestion issue. **Remediation:** We are in process of enhancing the code with more rigorous testing in order to make this feature available for Workday platform customers. Regards, AppZen Support Team
This incident has been resolved.
A fix has been implemented and we are monitoring the results.
Hi, Our engineers have identified the issue for Workday customers and the hotfix has been tested successfully. Currently, our engineers are working towards deploying the fix. We will keep updated on the progress of deploying the fix. Apologies for the inconvenience caused. Regards, AppZen Support Team
We are seeing intermittent report ingestion issues for Workday customers and our Engineers are actively working to fix this issue.
Report: "SSO Login Issue"
Last updateThe incident has been resolved.
A fix has been implemented for the identified issue. Users should be able to log in via SSO. We are monitoring the issue moving forward.
Hello, We have observed that users are unable to log in via SSO. We are currently investigating this issue and we will be providing updates soon. Sorry for any inconvenience caused due to this. Regards, AppZen Support Team.
Report: "AppZen Not Accessible - US Region"
Last updateHi, We are happy to inform you that the access issue with AppZen has been resolved. Regards, AppZen Support Team
A fix has been implemented and we are monitoring the results.
This is to inform you that, we have observed some performance issues with AppZen US Region. You might observe some issues while accessing the application as well as Resetting passwords. Our team is currently working on resolving the issue with high priority. We will be sure to update you on your progress. Thank you for your cooperation.
Report: "Expense Report Audit Stuck"
Last updateHi, We had identified an issue internally with Expense Report auditing. The reports were stuck in the audit queue due to some internal error and it has been resolved. Please expect some delay in processing the expense report audit. We apologize for the inconvenience caused. Regards, AppZen Support Team
Report: "Report Ingestion Delay"
Last updateHello All, Thank you for your patience. We can confirm that, issue has been fixed and there are no pending(stuck) reports to be ingested now. Ingestion is functioning as normal for all new incoming reports. Thank you, AppZen Support Team
Hello All, We have identified an ingestion issue, because of this there may be some delay in report ingestion in AppZen. Our teams are working with high priority. We will provide update on resolution progress. Thank you, AppZen Support Team
Report: "Intermittent Login | US Region"
Last updateHello, We are pleased to inform you that the AppZen Login Issue is resolved. Apologies for the inconvenience caused due to this. Please keep monitoring the Status Page for the updates related to RCA. Thank you, AppZen Support Team
Hello, We have identified an "Intermittent Login issue for the US Region" (https://audit.appzen.com/console/home.html#/) Our engineers are working on resolving this issue with the highest priority. We will keep you posted on the progress of the restoration. Thank you, AppZen Support Team
Report: "Delay in Report Ingestion"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
We will continue monitoring and provide further updates
We are currently seeing improvements after a workaround was implemented by our team. The delay in ingesting reports issue has been resolved and the reports are processing. We are monitoring further moving forward.
We are experiencing a delay in ingestion impacting Concur customers. After touching base internally, we were informed that we are experiencing an API error from Concur, we have reached out to the Concur Support team to get it to fix. We will update this status shortly as we have more information.
Report: "Concur Outage"
Last updateAccording to the latest update from Concur Status Page, the Outage has been resolved.
We are continuing to monitor the Concur Status Page for the latest notifications on this ongoing outage.
Concur is experiencing an outage in the US and EMEA. We will do the best we can to keep you updated on the progress of the situation. This will affect the use of our solution for customers using Concur platform. You can access the Concur Status Page here - https://open.concur.com/
Report: "Delay in Auditing the Reports"
Last updateWe are glad to inform you that the issue with the delay in auditing expense reports has been resolved. Please keep monitoring the Status Page updates related to the RCA.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We had observed an issue with report workflow action and it is delaying the report auditing. Our team has been engaged and we are actively working to resolve the issue. We will continue to provide further updates.
Report: "Delay in Report Ingestion and Audit"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
After verifying, we see that the reports are now ingested. Reports continue to be audited by AppZen and the queue is going down. We will continue monitoring and provide further updates. Thank you for your patience and cooperation.
We had observed an issue with report ingestion and workflow action. We worked with the internal team immediately before updating the status page. Hence this is to inform you about this issue. All pending reports should be ingested within the next 2 hours and everything will be cleared up. This appears to be affecting the Concur customers. We are continually monitoring and we will post a final update on the status soon.
Report: "Mastermind Analytics is Down"
Last updateAs per the latest update, we are happy to confirm that the issue with Mastermind Analytics has been resolved. Please keep monitoring the Status Page updates related to the RCA.
We have identified an issue with Mastermind Analytics. You might not be able to access our product and our team is investigating and working on resolving the issue on high priority. We will continue to provide further updates. We apologize for any inconvenience caused due to this.
Report: "Unable to Open AppZen"
Last updateWe sincerely apologize for the AppZen site not loading issue. We are pleased to inform you that, we have identified the root cause along with remediation for the issue as below. **Root Cause:** A change was implemented to add new infrastructure component and the underlying services were failing to run the services and it caused to load the site contents on AppZen. **Remediation:** We have introduced the process to have new components added in infrastructure only after completing Capability Assessment of underlying services and infrastructure components.
We are happy to inform you that the issue with Expense Audit - Unable to load site contents post log in has been resolved.
A fix has been identified within the issue for the Expense Audit application and the issue is resolved now. We keep monitoring the status of this issue. We continue to provide further updates.
We have identified an issue with Expense Audit and unable to load the AppZen post-login for the US region. You might not be able to access our product and our team is investigating and working on resolving the issue on high priority. We will continue to provide further updates. We apologize for any inconvenience caused due to this.
Report: "AppZen Not Accessible"
Last updateWe sincerely apologize for the access issue with the AppZen. We have identified the root cause along with remediation for the issue as below. **Root Cause:** The load balancer \(LB\) certificate was expired for all environments. It was using an older imported certificate, which expired on 04/18/2021. Hence while logging in we were getting a Bad Gateway error. **Remediation:** We have put a process in place in which there is an automated renewal of site certificates in order to avoid this issue in the future.
As per the latest update, we are happy to confirm that the access issue with AppZen has been resolved.
We have identified an issue with accessing AppZen Expense Audit. You might not be able to access our product and our team is investigating and working on resolving the issue on high priority. We will continue to provide further updates. We apologize for any inconvenience caused due to this.
Report: "Concur Workflow Status Update"
Last updateThe issue with Concur API has been resolved. The Concur Gateway team has lengthened the timeout.
We have monitored the logs and can see improvements currently with no errors.
We are currently seeing improvements and will continue monitoring moving forward.
We are currently seeing improvements and will continue monitoring moving forward.
We are continuing to monitor for any further issues.
This issue is still under monitoring. We will keep you updated on progress of investigation and monitoring.
We have noticed intermittent delays in the workflow status updating in Concur after actioning a report due to a delay in the response from the Concur API for some applicable customers. We have opened a case with Concur to investigate further. We have a proposed solution in place currently and are monitoring it until further information is received in the investigation.
Report: "Delay in Report Ingestion"
Last updateWe have monitored the ingestion for last few hours and we do not see any delay in ingestion. Hence we confirm that issue has been resolved. Closing this alert now. Thank you, AppZen Support Team.
The delay in ingesting report issue has been resolved and we are monitoring further moving forward.
We are currently seeing improvements and will continue monitoring moving forward.
We are experiencing a delay in ingestion impacting Concur customers. We will update this status shortly as we have more information.