Kuali Incident History

We experienced a disruption affecting the Platform and Build Service, which resulted in degraded platform performance and intermittent deployment failures. Our investigation identified a configuration issue that caused the service to operate with less capacity than intended. Our engineers corrected the configuration and restored the expected capacity, after which performance returned to normal. We continued monitoring the environment to confirm that the service remained stable. We are taking additional steps to improve service resilience and reduce the likelihood of similar disruptions in the future.

Our engineers identified a configuration issue that was limiting system capacity below the expected level. The configuration has been corrected, and system performance has improved. We will continue to monitor the environment and are taking additional steps to help prevent this issue from recurring.

We apologize for the disruption. Our engineers have increased system capacity, and performance has improved. The system is now loading as expected. We are continuing to monitor the environment and investigate the underlying cause, and we will provide additional updates as more information becomes available.

Our engineers are still actively working to restore normal service. We will provide another update as soon as there is additional progress to share.

Our engineers are continuing their investigation and working toward a resolution. We will provide another update as soon as additional information is available.

We are continuing to investigate this issue. We will share another update as more information becomes available.

We are investigating a service disruption affecting Platform and Build customers. Users may experience issues loading pages, accessing applications, or completing actions within the system. Our team is actively working to restore normal service. We will provide additional updates as more information becomes available.

We identified a background process that was placing unnecessary load on the database and have since optimized it, improved the associated data service, and completed database performance enhancements to help prevent similar issues going forward.

A deployment-related infrastructure issue caused some application servers to become unhealthy, resulting resulting in intermittent access errors for some users in Kuali Platform products. The affected servers have been removed and service has been restored.

The intermittent performance issues affecting Build/Platform have been resolved, and users should now be able to access apps normally. Our team will continue to monitor the environment to ensure continued stability

We are currently investigating intermittent performance issues with Build/Platform. Users may experience errors or delays when accessing apps. Our team is actively investigating the issue and will provide updates as more information becomes available.

Our team has deployed a fix to resolve the intermittent PDF generation issues. Documents that encountered errors during processing will begin retrying automatically, and any associated notifications will be sent as expected once processing completes. Our investigation indicates the issue was caused by an internal configuration change that disrupted communication between services. We are continuing to validate the root cause, but service has been restored.

We have identified the underlying cause of the intermittent PDF generation failures and are actively deploying a fix. Once the update is complete, queued PDF generation requests will resume. We appreciate your patience and will provide another update as soon as the deployment is complete.

We are continuing to investigate the issue affecting PDF generation. Our team is actively working to identify the root cause and restore consistent PDF generation as quickly as possible. We appreciate your patience and will share another update as soon as we have more information.

We are continuing to investigate the issue affecting PDF generation. Our team is actively working to identify the root cause and restore consistent PDF generation as quickly as possible. We appreciate your patience and will share another update as soon as we have more information.

We are continuing to investigate the issue affecting PDF generation. Our team is actively working to identify the root cause and restore consistent PDF generation as quickly as possible. We appreciate your patience and will share another update as soon as we have more information.

We are aware of an issue causing PDF documents to fail to generate consistently for some users. Our team is actively investigating the cause and working to restore normal functionality as quickly as possible. We will provide additional updates as soon as more information becomes available.

Our team has deployed a fix to resolve the intermittent PDF generation issues. Documents that encountered errors during processing will begin retrying automatically, and any associated notifications will be sent as expected once processing completes.Our investigation indicates the issue was caused by an internal configuration change that disrupted communication between services. We are continuing to validate the root cause, but service has been restored.

We have identified the underlying cause of the intermittent PDF generation failures and are actively deploying a fix. Once the update is complete, queued PDF generation requests will resume. We appreciate your patience and will provide another update as soon as the deployment is complete.

We are continuing to investigate the issue affecting PDF generation. Our team is actively working to identify the root cause and restore consistent PDF generation as quickly as possible. We appreciate your patience and will share another update as soon as we have more information.

We are continuing to investigate the issue affecting PDF generation. Our team is actively working to identify the root cause and restore consistent PDF generation as quickly as possible. We appreciate your patience and will share another update as soon as we have more information.

We are continuing to investigate the issue affecting PDF generation. Our team is actively working to identify the root cause and restore consistent PDF generation as quickly as possible. We appreciate your patience and will share another update as soon as we have more information.

We are aware of an issue causing PDF documents to fail to generate consistently for some users. Our team is actively investigating the cause and working to restore normal functionality as quickly as possible. We will provide additional updates as soon as more information becomes available.

Our environments experienced intermittent performance degradation from approximately 10:00–10:27 AM MT due to an incident with our hosting provider that delayed replacement servers from coming online as expected. This temporarily reduced system capacity and resulted in slower performance. Once the replacement servers were available, performance returned to normal.

Our US environments were briefly unable to load from ~12:54-12:56pm MT due to a temporary issue with our database provider and our monitoring shows that performance has stabilized.

A deployment-related infrastructure issue caused some application servers to become unhealthy, resulting resulting in intermittent access errors for some users in Kuali Platform products. The affected servers have been removed and service has been restored.

The intermittent performance issues affecting Build/Platform have been resolved, and users should now be able to access apps normally. Our team will continue to monitor the environment to ensure continued stability

We are currently investigating intermittent performance issues with Build/Platform. Users may experience errors or delays when accessing apps.Our team is actively investigating the issue and will provide updates as more information becomes available.

We've resolved the issue affecting all environments. The issue was caused by a database migration associated with a recent code deploy that encountered an unexpected error in our US environments. The migration has been corrected, and all services are now operating normally. As a follow-up, we are reviewing the migration process to identify what led to the error and implement additional safeguards to help prevent similar issues in the future. Our apologies for the disruption this caused.

We are continuing to investigate this issue.

We're actively investigating reports of environments not loading for Kuali Build/Platform products. We'll keep you posted as we know more and apologize for the inconvenience.

We've resolved the issue affecting all environments.The issue was caused by a database migration associated with a recent code deploy that encountered an unexpected error in our US environments. The migration has been corrected, and all services are now operating normally.As a follow-up, we are reviewing the migration process to identify what led to the error and implement additional safeguards to help prevent similar issues in the future. Our apologies for the disruption this caused.

We are continuing to investigate this issue.

We're actively investigating reports of environments not loading for Kuali Build/Platform products. We'll keep you posted as we know more and apologize for the inconvenience.

This issue is now resolved and environments in the US region should be back up again. The incident was caused by an unexpected database coordination issue that prevented the database cluster from operating normally, resulting in a temporary service disruption. The issue was identified and the cluster was restored to its expected operating state. We'll be doing a full retro and take the necessary steps to avoid this type of disruption in the future. Our apologies for the disruption this caused your users.

We've identified the issue and putting in a fix to resolve the issue - once fully restored and stable we'll give another notification.

We're actively investigating reports of environments in the US region not loading - we apologize for the inconvenience and we'll let you know more as soon as possible.

This issue is now resolved and environments in the US region should be back up again.The incident was caused by an unexpected database coordination issue that prevented the database cluster from operating normally, resulting in a temporary service disruption. The issue was identified and the cluster was restored to its expected operating state.We'll be doing a full retro and take the necessary steps to avoid this type of disruption in the future. Our apologies for the disruption this caused your users.

We've identified the issue and putting in a fix to resolve the issue - once fully restored and stable we'll give another notification.

We're actively investigating reports of environments in the US region not loading - we apologize for the inconvenience and we'll let you know more as soon as possible.