Historical record of incidents for Brillium
Report: "Assessment Builder Unresponsive"
Last updateThis incident has been resolved.
Access to all systems has been confirmed and tested. We are continuing to monitor systems for an extended period.
Most all system access has been restored. The User Profile applciation is still in the process of coming back up.
We are continuing to monitor for any further issues.
Access to the Brillium application(s) has been successfully restored. All systems are coming back online and should be fully available within the next few moments. Our team will continue to monitor system performance closely to ensure stability. We appreciate your patience and understanding during this interruption.
Our investigation has identified that multiple errors occurred within Brillium’s authentication systems, which disrupted the ability to authenticate between and within Brillium applications and infrastructure. Our engineering team is actively working to restore full service and is making steady progress. We expect resolution to be completed shortly and will continue to provide updates as work is completed.
We are continuing to work on a fix for this issue.
We have identified the root cause of the issue. Restoration efforts are underway by our Systems Operations team. We’ll share additional updates as they become available.
We are currently experiencing a temporary service interruption, and our team is investigating the issue. We are actively working to restore full functionality as quickly as possible.
Report: "Database performance degradation for small number of customers"
Last updateThis incident has been resolved.
All issues have been resolved and tested.
We are continuing to restore the remaining (2) customers affected. We will provide an update when the process is complete.
We are addressing this issue on 2 customers. All others remain unaffected. We expect to be completed shortly.
We are continuing to work on a fix for this issue.
We are continuing to make progress wiht the remaining customers as previously noted.
Approxiamately 4 customers continue to be affected and experience service degredation. The resolution process is continuing and we expect a complete resolution in the next hour. We apologize for the delay adn will provide another udpate shortly.
We are continuing to address the issue for these customers. We expect that the resolution may take an additional 30 minutes. We will probvide an additional update in 15 minutes.
We have identified a database service degradation that affects a very small number of customers (less than 10). The cause was immediately identified and is currently being addressed. We expect this situation to be resolved in less than 60 minutes.
Report: "Assessment Authoring and Administration Access"
Last updateThis incident has been resolved.
A fix has been implemented. We are continuing to monitor the system to ensure operational integrity.
We are currently investigating reports that some customers are not able to reach Assessment Builder (authoring) and Administration. Assessment (delivery) activity appears unaffected.
Report: "Intermittent Performance Issues"
Last update**ISSUE** After a security update, monitoring services reported that some customers may be experiencing intermittent 503 errors. This issue impacted a small number of customers. **ROOT CAUSE ANALYSIS** Investigation by our DevOps team identified the root cause as a configuration issue that affected an underlying systems application web service. **RESOLUTION** A revised web service configuration was deployed to resolve the issue. Further monitoring indicated that the service was functioning normally and as expected. **NEXT STEPS** The DevOps team is conducting further analysis to prevent the issue from resulting in any future service degradation.
Monitoring has confirmed the issue has been resolved.
The issue has been identified and a fix has been implemented. We are currently monitoring systems to ensure all issues are fully resolved.
The source of the issue has been identified. Root cause analysis is currently underway.
We are currently experiencing intermittent issues that may result in a 503 error being displayed. Refreshing the browser will resolve the issue and display the expected screen properly. The systems operations team is currently working to resolve the issue as quickly as possible.
Report: "Assessment Builder Unavailable"
Last updateIn our post-mortem analysis, it was discovered that the "Automatically Enable Minor Upgrade" flag was enabled within AWS’s Database configuration. As a proactive measure, we have since disabled this automated minor upgrade feature. This adjustment ensures that system downtime for upgrades will be conducted in a controlled manner, allowing us to manage and schedule these updates more effectively.
Brillium's database services, hosted on AWS, underwent an unplanned minor security update to fortify the database's integrity. Consequently, the Brillium platform experienced a temporary unavailability while the database was restarted. This process, estimated to take approximately 2 minutes, swiftly concluded, restoring all services to our valued customers without further interruption.
Report: "Database Performance"
Last updateThis incident has been resolved.
The system appears to be fully operational.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are currently investigating a potential database performance issue with Brillium Assessment Builder. We are reaching out to the Amazon AWS team for a status.
Report: "Scheduled Release Maintenance Complete - 2023.07.04 @ 05:00 UTC/GMT"
Last updateThe Brillium Systems Operations team completed the application of scheduled updates and enhancements (R2023.2) to improve your experience on the Brillium platform on 2023.07.04 @ 05:00 UTC/GMT. During this time, access to the following application's components was limited: 1. Assessment Builder 2. Administration 3. APIs 4. Partner Central 5. Integration (Zapier, PowerAutomate, Make, etc) For more details, please view the software Release Notes page here: https://support.brillium.com/knowledgebase/category/?id=CAT-01032
Report: "Incident Investigation"
Last updatePost monitoring confirms this issue has been resolved
An update to address the "General Error" message has been applied. We have been monitoring the system for the last 12 hours with no further indications of this issue.
Following a comprehensive investigation and analysis, we have identified the root cause of the issue. Our findings indicate that the issue lies in the interaction between the application and the database, which is supported by the data collected during the investigation. We have already implemented some temporary enhancements aimed at working around the root cause. Meanwhile, our engineering team will begin introducing a series of permanent solutions over the next several weeks. The system is not expected to experience any downtime or service interruptions during this time.
We have determined that the issue we are experiencing is related to the database. Specifically, the Brillium platform has encountered very infrequent intermittent challenges with connecting to the database causing the General System Error Respondents experiencing. We are currently working closely with AWS to identify the root cause of these issues. Please know that we are actively investigating the matter, and we will provide updates as soon as we have additional information.
Brillium Engineering Team has eliminated any issue with the platform. We are now working with AWS engineers to determine the issue. This issue has been escalated at AWS.
Brillium is currently experiencing a occasional issue during Respondents taking an Assessment. Respondents may receive the following message when this issue occurs: General System Error. We are aggressively investigating to understand the issue so we can deliver a solution to resolve. As we understand more on the issue, we will update the status page to keep you informed on the issue, our solution and a proposed timeframe to resolve.
Report: "502 Gateway Error Reports"
Last update**Background:** Brillium customers experienced an incident from approximately Friday, August 26, 2022 @ 2145 UTC to Saturday, August 27, 2022 @ 0530 UTC. This issue affected customer access to both Brillium Assessment Builder version 10 and Brillium version 11 applications. **Root Cause:** After interfacing with several Amazon AWS engineers and conducting a thorough incident analysis and investigation internally, we have discovered an incompatibility between network configuration components that would present itself as a loss of connectivity even though the systems themselves were in fact nominal. **Steps Taken:** * We communicated actions taken during analysis and remediation phases of the incident via the Brillium Status Page. * We responded to customer inquiries through our support channel * We worked with Amazon support engineers to analyze and identify the root cause * We applied remediation steps to remediate and fully resolve the root cause. * We monitored the systems to ensure its stability. **Mitigation:** The components identified during the investigation are necessary for compatibility in order to support customers on earlier versions of the platform \(i.e. v9, v10\). During the remediation steps, we were able to remove any dependency on the components identified as the root cause. Additionally, we added validation enhancements to our QA process that include compatibility checks in order to prevent any such issue from reoccurring in the future.
We are satisfied that we have resolved the issue and all systems are operating within normal parameters. For more details, we will provide a post mortem on this issue describing what caused the issue and steps we took to remediate and prevent it from possibly occurring again.
All services are available, and we will continue to closely monitor systems.
The platform is back in service but we will continue to monitor and scale up for performance.
We have resolved the issue and slowly bring the platform back online. You may experience some initial slowness but the system will improve its performance as we began to become fully operational. We will post when the system is fully up and running optimally.
Our Engineering Team is currently working with AWS Engineers to discover the root cause of the issue. We will share more information as we gain more insight. We do not have an estimated uptime as of now but will share once we understand the issue.
Brillium is currently experiencing an issue and investigating the issue. We will post an update as soon as we learn more about the root cause of the issue.
Report: "Zapier Integration"
Last updateAt 04:00AM UTC (00:00 AM EST) the critical fixes were applied and tested successfully.
A Software Maintenance update has been scheduled for Friday, July 29 at 04:00 UTC (00:00 AM EST). This update will addresses 2 major issues: - Zapier connector authentication failure issue - invitations created via the Invitation WebAPI with "Send: true" does not result in an email
We have identified and are working on addressing an issue with a Zapier integration. This issue ONLY affects Zapier connector integrations, and does not impact use of the Brillium WebAPI. We are working with the assistance of Zapier support to address the issue. A fix has already been developed and successfully tested by the QA team. We are awaiting a scheduled deployment date and will post it shortly.
Report: "502 Gateway Error Reports"
Last update### Background On June 13 there was an outage event \(very similar to the one on June 8\), occurring between approximately 1300 UTC and 1800 UTC. This event only affected a portion of our customers. Review of our data indicated that specific Amazon AWS cloud service communications between our server systems and authentication resources were failing sporadically during this period, and at times appeared to fail altogether. Our monitoring and system information shows that our systems repeatedly tried to communicate with external resources without success. All systems became operational later in the day. Unlike the previous event on June 8, this one was publicly reported by a few outlets and users on social media. Amazon’s own services appear to have been impacted by the event. ### Steps taken: * We communicated our status and findings, as we learned them, to all customers via our status page * We provided additional details to customers through direct support communications. * We contacted Amazon AWS Support for additional information and confirmation of our findings. * We continued to closely monitor systems for a period after the event, to ensure ongoing stability. ### Public Reports Public reports of the Amazon outage can be found via the links below and these were shared with some customers: * https://www.reuters.com/technology/amazon-down-thousands-users-downdetector-2022-06-13/ * https://knowtechie.com/amazon-is-down-for-a-ton-of-people-right-now/ * https://www.theverge.com/2022/6/13/23166246/amazon-down-error-message-outage ### Mitigation As this event was external to our systems and outside of our control, no direct actions could have predicted or prevented the issues. External network or Internet issues can and will affect access to our systems; however, these types of events are generally rare and often quickly resolved.
Monitoring shows that the issues have been addressed. A full report will be shared via the incident Post-Mortem.
We are continuing to monitor activity. Initial results and testing are positive.
Brillium Systems are currently back in service. Our internal tests show the external networking is presently stable. We will continue to monitor.
Investigations continue. Currently, it does look like there is an issue with the network routing connecting our systems that is causing sporadic issues. Reports have begun to surface that other Amazon systems are experiencing downtime as well, although we do not know the extent of such issues or the relationship to our specific issue. As these systems are outside of our control, our attempted workaround(s) did not sufficiently address the problem. We are currently assisting engineers in further diagnosis any way that we can, in an attempt to help address the issue in the most expedient way.
The system operations team is currently consulting with AWS engineers to determine the root cause of the issue.
We are receiving intermittent reports that some users are receiving 503 errors.
We are continuing to investigate this issue.
We have implemented a workaround to these issues. The systems are available and we continue to investigate and gather information surrounding the root cause.
The systems operations team is provisioning a possible workaround to the issue, while they continue to investigate. An additional status update will be posted in approximately 15 minutes.
We are continuing to investigate the issue
The system operations team is investigating whether an operating system level error used by some of the AWS cloud systems is potentially affecting services. Currently, the system monitoring of Brillium services does not indicate an issue that would be unusual. Although this may lay outside of our direct control, the team is investigating any opportunity to mitigate the effect from our side. We continue to investigate.
We are currently investigating 502 Gateway errors reported by a portion of our customers.
Report: "502 Gateway Timeout Error"
Last update### Background On June 8 there was an event that affected service to all customers between approximately 1300 – 1400 UTC. Immediate review of our monitoring and systems data suggested that the network between our systems and database services suddenly stopped functioning. Our systems repeatedly tried to reestablish communication with these resources without immediate success. The systems once again became operational after approximately ~1 hour. Further review of available data indicated communications between our systems and other Amazon AWS cloud services were not reliably functioning during this time. This presented itself as sporadic errors and/or temporary communication failures between key systems. As these network systems are not within our domain of control, we do not have access to the detailed data that can confirm our conclusions with absolute certainty. Previously, our systems were operating normally until network communication was suddenly lost. Subsequently, we reported this to Amazon AWS support, which later advised that there were elevated API latency issues around the period of this event. While this does align with what our evidence indicates, Amazon AWS Support was not immediately able to positively confirm that these were in fact the specific issues that we experienced. We have requested additional information. ### Steps taken: * We communicated our status and findings, as we learned them, to all customers via our status page. * We provided additional details to customers through direct support communications. * We contacted Amazon AWS Support for additional information and confirmation of our findings. * We continued to closely monitor systems for a period after the event, to ensure ongoing stability. ### Mitigation As this event was external to our systems and outside of our control, no direct actions could have predicted or prevented the issues. External network or Internet issues can and will affect access to our systems; however, these types of events are generally rare and often quickly resolved.
This incident has been resolved.
We are continuing to monitor for any further issues. At present, it continues to appear that this was caused by something related to some of the AWS systems we use. We will be confirming this with them and providing updates within the next 48 hours or so.
Ongoing systems monitoring has begun. Systems are operating normally.
All systems are recovered. We have begin monitoring operations while we gather additional information.
All systems are being tested now (smoke test). We expect an update within a few moments.
We have begun and are continuing to bring up affected customers. Currently we estimate approximately 5 more minutes to address remaining affected instances.
We estimate approximately 5 minutes to address the issue. We are continuing to identify the root cause of the issue. At present, it appears to be a related to the AWS database system infrastructure.
The issue has been identified and the system operations team is working with the AWS team to remedy the situation.
We are currently investigating reports that some customers are unable to access their Brillium assessment Builder instance and receiving a "502 Bad Gateway" error
Report: "Performance Hotfix"
Last updateA critical hotfix will be applied to address the following issues: - IMPROVED - Overall performance improvements to Dashboard, Assessment List, and Assessment Builder [286] - FIX - to address an infrequent error with custom data delivery services jobs [279] - FIX - a Question Tag Analysis Report error that may occur for some customers [281]
Report: "502 Bad Gateway / 504 gateway Timeout Error"
Last updateMonitoring, testing, and analysis show that systems are operating as expected. This issue is resolved. A post-mortem report will be posted when complete.
The application of the hotfix has been completed. The systems will be closely monitored over the next 24 hours.
The root cause of the issue has been identified, and the development team is applying a emergency hotfix to fully resolve the issue. There may be very brief interruption in service for some customers while the patch is being applied. We expect this process to take no more than a few minutes. No further interruption to services is planned or expected.
We are continuing to monitor systems while conducting the root cause analysis.
Brillium Application Builder access has been restored. Investigation to root cause continues. We will post additional findings soon.
Administration, Partner Central, and Profile applications are unaffected and remain operational. Investigation is focused on Application Builder and we will continue to report further progress.
We have received reports from some customers that they are unable to access Brillium with an Error 502 Bad Gateway/504 Gateway Timeout. We are in the process of investigating the issue and will report further progress.
Report: "Temporary Outage for v10 Customers"
Last updateThe environment is operating normally.
A fix has been implemented and we have begun monitoring duties.
Brillium lost one of its network zone that caused a brief outage. Unfortunately, customers that were connected to this zone may have experienced a brief loss in connection to the Brillium platform. We have identified the issue and restored services. Brillium version 10 customers should be operating normally. Some customers connected during the outage, may have needed to reconnect to Brillium.
Report: "Update for Apache Log4j2 Issue"
Last updateAfter conducting a thorough review, we have confirmed and verified that Log4J is not implemented as part of Brillium’s software application services.
This incident has been resolved.
Brillium is aware of the recently disclosed security issue affecting the open-source Apache "Log4j2" utility. We are actively monitoring this issue, and are working on addressing it for any Brillium services which use Log4j2. At the moment, we have identified no use of Log4j2 within our direct platform nor have we identified any vulnerabilities but will continue to monitor. Should you have any question regarding Log4j2, please do not hesitate to contact us as support@brillium.com.
Report: "Infrastructure issues at AWS"
Last updateAWS has resolved all infrastructure issues and operating normally.
AWS has fixed the issues from yesterday. We are closely monitoring the Brillium Platform to ensure there are no longer any impacts from yesterday AWS issues.
No additional update at the moment. We continue to monitor AWS status and will update as we hear more information.
AWS has reported they have identified a root cause of the service API and console issues affecting customers serviced by the US-EAST-1 Region, and are starting to see some signs of recovery. These services are used by and can affect a variety of functions used by Brillium, which may cause intermittent errors. There has been no ETA for full recovery provided at this time, but we will continue to monitor the recovery effort and provide updates.
We are currently seeing issues reported by Amazon Web Services (AWS) which may impact some platform functionality for versions 9, 10, and 11 intermittently. We are closely monitoring the AWS status reports and will provide updates as we receive them.
Report: "Intermittent Outage"
Last updateWe discovered a compute resource in our environment was being over utilized and have remediated the platform issue.
This incident has been resolved.
We believe we have identified the issue, fixed it and now monitoring the fix.
We are continuing to investigate this issue.
Customers on Version 11 of the platform may experience intermittent performance challenges at the moment. We are aware of the issue and working to stabilize the platform. Thank you for your patience
Report: "Assessment Builder - Emergency deployment update"
Last updateThis incident has been resolved.
We have applied a emergency patch to Assessment Builder which may have cause an brief outage for customers that were currently using the platform.
Report: "Emergency Patch Release"
Last updateThere was an emergency patch deployed on 11/09/2021 @ 5:00am EST to fix a critical issue in Assessment Builder application version 11. The system was offline for approximately 2 minutes during this time to deploy the hotfix. The system is operating normal after the patch deployment.
Report: "Maintenance Release - 11.1.2021.1102"
Last update<h3>11.1.2021.1102 Release Notes</h3> This maintenance release includes the following: - General improvements - New Features <hr/> <h4>Brillium Assessment Builder</h4> <h5>FIXES AND ENHANCEMENTS</h5> 1. In addition to existing XML and XLS formats, JSON support was implemented for the export/import of assessments. <img src="https://brillium-www-public.s3.amazonaws.com/status-page/image-20211012-144910.png"/> 2. When creating Workspaces, the name is actively validated to ensure it follows the proper format (i.e. length, acceptable characters, etc.) <img src="https://brillium-www-public.s3.amazonaws.com/status-page/image-20211012-143514.png"/> 3. A new ‘About’ screen has been added to more clearly display the version information of the Brillium applications and API. <img src="https://brillium-www-public.s3.amazonaws.com/status-page/image-20211012-142541.png"/> 4. Active type-ahead search has been added to We improved dropdown components to make finding and selecting items more efficient. <img src="https://brillium-www-public.s3.amazonaws.com/status-page/image-20211026-144238.png"/> 5. Assessment Builder settings tabs have been enhanced with security controls allowing the Brillium Administration application to enable and disable tabs for defined Teams and Users. 6. The jQuery library utilized for the Assessments application has been updated to version 3.6. 7. The Certificate Preview functionality has been improved to now include preview capability of both Standard US Letter and A4 formats. 8. A Workspace Grouping checkbox has been added to the Assessment listing screen. Selecting this option will group the display of Assessments by Workspace. Ths setting is unchecked by default. <img src="https://brillium-www-public.s3.amazonaws.com/status-page/image-20211026-150104.png"/> 9. General enhancements to the Assessment Builder tab screens 10. Improved User Profile update Experience <h5>NEW FEATURES</h5> 1. A default, per-customer Help screen can be added to a Brillium tenant (Brillium Enterprise Elite). Respondents can be redirected to this help page when the help icon is selected, provided another screen is not defined to override it. <h5>Security Updates</h5> 1. Each maintenance release includes general application security updates.
Report: "Authentication Errors Affecting Some Brillium v11 Users"
Last update## Findings Although extremely limited in affect and scope, this issue was caused by a combination of unexpected and unpredictable factors that included assessment configuration, atypical respondent activity, and to an extent, local government policy. On September 16, 2021, we began to receive reports from approximately 30 - 40 customers that were impacted by extremely unusual assessment activity patterns, that originated from customers delivering assessments in a region of Asia, related to a large-scale government economic recovery program. The program required qualifying citizens to complete several assessments in order to receive funds. The population of this region is significant, as were the number of qualifying individuals. The benefits offered through the program resulted in highly unusual activity, with some individuals retaking a single assessment up to 100 times or more, completing hundreds \(in some cases over a thousand\) assessment attempts per-day per-person. The configuration of the assessments was such that it further increased the internal API call volume. The combination of these factors caused the affected Brillium customers to encounter errors when accessing their accounts and delivering assessments. ## Resolution Upon analysis and review, Brillium staff and development teams took the following steps to address these issues. Subsequent acitivty with similar volumes of attempts took place without incident in the Fays that followed. The following steps were taken: * Brillium staff consulted and advised customers located in the source regions of the assessment activity, on improved assessment configuration, resulting in a reduction of unnecessary activity. * Brillium’s development team made significant improvements to the Brillium API, resulting in improved efficiency and increased capacity across all global regions. * Enhanced system metrics and increased monitoring ## Additional Monitoring Although the impact was eliminated within a short time, the development team has monitored systems for an additional 30 days in order to collect and analyze additional assessment and system performance data. The resulting analysis shows that all implemented fixes have adequately increased system performance and eliminated related errors.
All monitoring and tests have completed successfully.
Mitgation efforts appear to be successful. We will continue monitoring systems. All components are operating at nominal levels.
While continuing our investigations of the performance challenges experienced by a small number of our customers, we have narrowed the issue down to the activity related to a small number customers located in the Asia Pacific region. We are working on addressing the impact and will post further updates as we make progress.
a fix is being implemented and monitoring will begin.
We are continuing to work on a fix for this issue. This is only affecting a small number of users.
We are continuing to work on a fix for this issue.
We are looking at a possible issue that could be caused by the web server software itself. The development team is working on a fix that will be tested. While this work is underway, you may see a maintenance page when accessing Assessment Builder or the Administration portal (if applicable). There is currently no estimated completion time, but we will post more information as soon as we have it.
We are investigating a possible reoccurance of the previous issue that prevents authors, administrators and assessment test takers from accessing a small number of accounts.
Report: "Brillium v11 Login Issues"
Last updateAfter closely monitoring systems, the issue appears to be resolved. We have reached out to our cloud services provider to help us with more information on root cause. We will include details in the event post-mortem as soon as possible.
A fix has been implemented and we are monitoring the results
We have intermittent errors regarding communication with the Brillium version 11 authentication system. A handful of customers are currently affected by an issue, preventing the login to Brillium Assessment Builder and the Brillium Administration portal. We are investigating the cause. Currently, a small number of customers are affected (less than 10). No recent changes or deployments have been made, so we are investigating whether this is an external problem affecting these users. This should not affect test takers, only author/administrator users.
Report: "Thinkific Integration Issues"
Last update## Thinkific Connectivity Issue During the morning hours of August 13, 2021 we received a number of reports that assessments launched from Thinkific sites were displaying an error message of "Invalid Access Code". This prevented students progressing further. ### Investigation We opened an investigation and found no issues with the Brillium software services. We followed this action by notifying the Thinkific Support team. Shortly thereafter we were notified that Thinkific Support was working with their Development team to investigate the issue. ### Fix We were notified that Thinkific Development team had applied a fix towards the end of the day on August 13, 2021. We were no longer receiving reports of the problem. ### Monitoring As a matter of process, we monitored activity for 18 hours following the last reports. During this process we reviewed support cases, error logs, and continued to monitor activity. ### Root Cause As this was an external issue stemming from activity coming from a third-party system, we are unaware of the root cause of the issue or the fix applied. From our own internal investigation it appears that there was an issue with the student data sent from Thinkific to Brillium. **NOTE:** This was **NOT a Brillium issue** and had no impact on Brillium operations or availability. This was an external problem with a third-party system that affected a significant number of our Thinkific users. As such, we investigated the issue and report the status and findings as a convenience to our customers.
After further monitoring, and having received no further reports from Thinkific users, the issue is considered resolved. NOTE: This was NOT a Brillium issue and had no impact on operations or availability of services. This was an external problem that affected a significant number of our Thinkific users. As such, we investigated the issue and reported it as a convenience to our customers.
The Thinkific Development team announced that they had applied a fix. We are monitoring activity. We ask customers to notify us and copy Thinkific Support if they continue to experience any errors. To report an issue please visit support.brillium.com.
We are currently awaiting feedback from the Thinkific developer team.
Our investigation, at present, did not show any Brillium issue that is contributing to the error.
We have received several reports of assessments that are launched from Thinkific sites displaying an error "Invalid Access Code" and preventing students from progressing further. We have notified the Thinkific Support team and they are currently working with their Development team to investigate the issue.
Report: "Reported Access Issues"
Last updateWe are currently investigating a widespread internet access issues that are preventing users from reliably accessing the Brillium Platform. We are working with service partners and solution engineers to expedite improved access to the platform. 2021-01-26 11:59 EST It has been reported in several news outlets that a major fiber access connection in New York was disrupted, causing intermittent access to major web and cloud service application providers. This appears to be external to Brillium. 2021-01-26 13:00 EST News outlets continue to report a fiber incident causing issues that affect the East Coast of the United States and cause unreliable access to web applications (including Brillium) and major cloud service providers. 2021-01-26 17:00 EST This incident has been investigated and identified as an external issue disrupting general Internet access to applications on the Internet. Access to services has been improving over the past hour. Brillium has been operating normally throughout the incident for those customers that were able to access the application. Post Mortem Analysis *Services Access Disruption Caused by Widespread Internet Outage* When initial customer reports were brought to the attention of our support staff, we began investigating the issue as an internal problem. Very quickly our investigation made it clear that there was an external Internet issue at fault. The large-scale service disruption that occurred during the morning hours of January 26, 2021, was caused by a major Internet outage in New York, outside of our control and affected a number of other major cloud services and application providers such as Google, AWS, Twiitter, and Slack. The exact cause of the outage somewhat varied among early reports by major news outlets, but was most commonly attributed to a fiber cable in Brooklyn, New York. Access to most major Internet service providers began to resume by early afternoon.
Report: "Assessment Builder Delays, 504 Gateway Error"
Last updateWe have received several reports of delays in Assessment Builder load times and respondent 504 gateway errors. In some instances, customers report that selecting the Save button on Assessment Builder does not save selected changes. We are investigating the issue. 2021-02-03 17:30 EST We have identified an issue where large assessment question set (over 2,000 questions) are taking excessive processing time, delaying some operations. We are currently reviewing options to address the issue. 2021-02-04 13:45 EST We have implemented an optimization that addresses the issue with large Assessment Builder question sets. The resolution is currently in QA. 2021-02-04 16:15 EST QA has confirmed the resolution. We are deploying to limited customer instances for further testing. We expect to deploy a fix to all customers by 09:00 February, 5, 2021. In the meantime performance has been improved across all instances. 2021-02-04 16:58 EST We are continuing to monitor for any further issues. 2021-02-04 23:20 EST Testing is complete and a fix has been applied
Report: "Maintenance - 10.2.20210210+478.b9488a"
Last updateResolved - inconsistent behavior with certificate attachment to respondent emails [D-26] Improved - speed of Assessment Builder page load to resolve reported delayed Save operations [D-40] Resolved - Feedback Summary screen error when for Net Promoter Question Type when using question tags [D-41] Improved - Assessment List Search functionality [D-21] Added - Question numbering added to Assessment Listing [D-29]
Report: "Maintenance Release - 10.2.20210205-467-dd54fd"
Last updateFixed - House Blend randomization mode and selection UI improvements Fixed - Improvement to the retrieval of assessments with large question sets
Report: "Deprecation of Secure Cipher Support for TLS 1.1 and earlier Implemented"
Last updateAs previously announced, on March 2020, TLS 1.0 and 1.1 was deprecated by Apple Safari, Mozilla Firefox, Microsoft EDGE/IE, and Google Chrome. Any secure connection that traverses software or network services that rely on these versions of TLS in order to function, browsers will recognize them as insecure and may not display or function. Further support of deprecated TLS versions can negatively affect the security grading of web applications like Brillium. Brillium recommends that customers implementing tools that require TLS 1.0 or 1.1, follow security guidelines to implement TLS 1.2 as soon as possible. As of February 15, 2021, Brillium version 10 no longer supports TLS 1.0 and 1.1. For more information, please refer to the Brillium Blog article located here: https://blog.brillium.com/deprecation-of-secure-cipher-support-for-tls-1.1-and-earlier .
Report: "Topic and Conditional Feedback May Not Display as Expected"
Last updateWe are investigating reports from some customer instances that topic and conditional feedback do not reliably display as expected on the feedback summary screen. 2021-02-16 14:15 EST We are continuing to investigate this issue. 2021-02-16 16:15 EST We have identified certain conditions that could result in topic and conditional feedback messaging not appearing as expected on the feedback summary screen at the end of an assessment. No other systems appear to be affected. 2021-02-17 11:00 EST A fix has been identified and is now in QA testing. 2021-02-18 00:00 EST [D-57] An update was applied to improve the reliability and display of topic and conditional feedback. Post Mortem Specific combinations of complex topic scoring and conditional feedback rules could result in feedback to not reliably display in conformance with the user intentions. An update was applied that included added logic to ensure feedback displays as intended. [10.2.2021.0217].
Report: "Scheduled Software Update (10.2.20210315+508.b82f18)"
Last updateA software maintenance update that includes the following updates and fixes will be applied: [Fixed] - After creating an assessment, changing the Assessment Type could result in saving a blank version of Assessment Instructions No downtime is expected in association with this update.