Historical record of incidents for Core Practice
Report: "Degraded performance"
Last updateThe incident has now been resolved, and we sincerely apologise for any disruption caused to affected customers. We will continue to review the situation to find ways to strengthen our system’s resilience and prevent similar issues from occurring in the future. Summary Our web application experienced performance issues on March 23, 2025 between 10:30am and 10:51am, due to memory issues on some of our servers. Extra instances were deployed as cover while we investigated the cause. Root Cause Memory constraints on specific instances affected server resources, leading to slow performance. Resolution We added three extra instances to improve capacity and distribution of load while addressing the underlying memory issues. Next Steps We'll implement improved monitoring and review resource allocation to prevent similar issues in the future. We apologise for any inconvenience and thank you for your patience.
Additional servers have been successfully added to the pool to handle the load spikes. Users should now experience normal operation. We apologise for any inconvenience caused during this period of degraded performance. Our team will continue to closely monitor the system as we investigate the root cause of the spikes
We are investigating a server performance issue related to CPU spikes and are adding an additional servers to handle the requests. Users may experience temporary slowness over the next few minutes.
Report: "Degraded performance Core Practice Application"
Last updateThis incident has been resolved. We sincerely apologize for the impact to affected customers. Investigations have assessed the best ways to improve architectural resiliency and mitigate the issues that contributed to this incident and its widespread impact.
A manual fix has been successfully implemented and system performance is now returning to normal levels. We apologise for any inconvenience caused during this period of degraded performance. Our team will continue to monitor the system closely while we investigate the root cause.
We are investigating reports users are experiencing degraded performance, you may experience slow response times when accessing Core Practice. We apologise for the delays and will provide an update in 15 minutes.
Report: "Elevated level of application errors"
Last updateSummary of Impact: On 28th August 2024, between 06:10 PM and 6:40 PM (AEDT), a significant number of users intermittently encountered application errors, which inhibited their ability to access the Core Practice system. Root Cause: Upon investigation, we identified that a small number of dependency requests were returning 500 errors, preventing the Core Practice application from loading successfully. This issue was exacerbated by the recent migration of Core Practice to a new infrastructure architecture, which took longer than expected to diagnose. We found that some server pools were functioning as expected, and steps were taken to reroute all traffic to these unaffected pools. Further investigation revealed that one of the servers in the affected pool had failed to load, prompting us to rebuild the affected server. Mitigation: The Core Practice team is committed to improving infrastructure through the new server migration to enhance security and redundancy by routing traffic to healthy pools. We are currently working to improve our emergency procedures to reduce switchover downtime and are reviewing processes to automate repairs without manual intervention. We apologize for the inconvenience caused and thank you for your cooperation.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
We have identified the affected servers and rerouted all traffic to unaffected servers. We are currently investigating the root cause.
We're experiencing an elevated level of application errors and are currently investigating the issue.
Report: "Elevated level of application errors"
Last updateThis incident has been resolved.
We have recently migrated all our users to new servers to improve performance and security. However, some customers using older versions of Core Practice Bridge may experience elevated errors when connecting. We are aware of the situation and are here to help. Please create a ticket to book a meeting with one of our agents for a Bridge update, or use live chat if the issue is urgent. Thank you for your understanding.
Report: "Elevated level of application errors"
Last updateSummary of Impact: On 2nd July 2024, between 5:45 PM and 6:00 PM AEDT (approx), we identified that one of our database servers experienced an elevated level of application errors. During this time, some users on the affected server were unable to access Core Practice. Root Cause: Upon investigation, we determined that high disk activity on the affected database server was the underlying root cause. The high disk activity caused the database service to shut down. Mitigation: The Core Practice team is dedicated to improving our infrastructure to prevent such issues from recurring, and we have increased server capacity. We will be working to migrate customers to new infrastructure in the coming weeks to ensure better stability and performance. We sincerely apologize for the impact on affected customers. We are committed to enhancing our architectural resiliency and mitigating the issues that contributed to this incident and its widespread impact.
Our systems are now operational. We are currently investigating the issue. Thank you for your patience.
We have identified an issue with one of our servers that resulted in an elevated level of application errors. During this time, some of our users were unable to access Core Practice. We apologize for any inconvenience this may have caused and appreciate your understanding and patience as we work to resolve this issue.
Report: "Optus Outage affecting some users logging into Core"
Last updateOptus network has now been restored and we are expecting all Core services to be operational again. We again thank you for your patience.
We want to inform you of a service disruption affecting SMS delivery and two-factor authentication capabilities, due a nationwide Optus outage. We are currently investigating the full impact on our services. If you are experiencing issues logging in, please reach out to a user who is already logged in to temporarily disable your two-factor authentication. We apologize for the inconvenience and appreciate your patience as we work to resolve this matter. Stay tuned for updates.
Report: "System Outage"
Last updateSummary of Impact: On 25th October 2023, between 11:00AM to 12:30PM (AEDT), a server outage was identified. A significant number of users encountered an "SSL Certificate Expired" warning from their browser, which inhibited their ability to access the Core Practice system. Root Cause: Upon investigation, the expiration of the SSL Certificate was traced back to an unsuccessful automatic renewal. The failure in automatic renewal was attributed to modifications in the validation process of approved domains by our Certificate Authority provider. While we aimed to reduce the outage duration, this incident was exacerbated when it was discovered that the certificate had not been successfully installed across multiple servers. Fortunately, we managed to utilize redundant servers on standby to meet the user demands. Mitigation: The Core Practice team is dedicated to improving our infrastructure to prevent such issues from recurring. We are currently incorporating Application Gateway technology to mitigate the risks tied to certificate management.
We are continuing to monitor for any further issues.
Our system is working expected now and we will continue to monitor the issue. We sincerely apologize for the impact to affected customers.
We are currently experiencing a system outage. Our teams are actively investigating and working on a resolution. We understand how critical our services are to your operations, and we apologize for any inconvenience this may be causing.
Report: "Issue with CP users not able to send SMS/receive SMS through Core"
Last updateSummary of impact: On 18 August 2023 between 10AM-12:30PM, we have identified an issue where some users may not be able to send SMS massages or receive SMS messages through Core with error message "service unavailable. Please try again". This issue prevents two-way SMS from functioning. Root Cause: We have identified the SMS service was not available in Core for some of our users due to our SMS provider is experiencing a technical issue since 10:00 am AEST time today, impacting multiple services including Messaging. Mitigation: Our SMS provider's engineering team has been actively working to resolve the issue since it was reported this morning. According to the update provided as well as testing internally by our team, we confirmed the issue has been resolved and our SMS service is available for our users again. Next Steps: Core Practice and our SMS provider will continue monitoring to ensure a full recovery of our SMS services. We appreciate your understanding and patience during this time.
We have identified the SMS service was not available in Core for some of our users due to our SMS provider is experiencing a technical issue with the Rest API since 10:00 am AEST time today, impacting multiple services including Messaging. Their engineering team is actively working to resolve the issue and providing updates.
We have identified some users may not be able to send SMS massages or receive SMS messages through Core with error message "service unavailable. Please try again." Our team is currently investigating the issue working towards a resolution. We appreciate your patience and understanding.
Report: "Issue with patients not able to reply to SMS sent from Core"
Last updateSummary of impact: On 16 June 2023 between 9AM-3PM, we have identified an issue where some patients are unable to reply to SMS messages sent through Core. This issue prevents two-way SMS from functioning as expected, affecting SMS chat and reply confirmations for appointments. Root Cause: Our SMS provider made some updates this morning in response to compliance with regulatory changes that require the registration of Alphanumeric Sender IDs across the entire industry. Unfortunately, these changes had unforeseeable consequences on standard SMS messages sent from regular numbers. Core Practice has been working closely with our supplier to reverse these changes. Mitigation: Our SMS provider has successfully completed a rollback of the settings. All SMS messages should now be sent from standard numbers, and SMS reply functionality should be working as expected. Next Steps: Core Practice will continue working closely with our supplier and will apply necessary hotfixes in the coming days. We appreciate your understanding and patience during this time.
We have identified patients not able to reply to SMS massages sent through Core for some users. Our team is currently investigating the issue working towards a resolution. We appreciate your patience and understanding.
Report: "Issue with Application Icons"
Last updateSummary of impact: On 13 June 2023 between 10AM-11AM, application icons in Core Practice did not display correctly for some users. Root Cause: After investigation we have identified that this incident is related to some unexpected downtime from our hosting provider for icon package. Mitigation: Currently the issue has been resolved and icons should appear correctly in Core Practice. Please refresh your browser cache. Next steps: Our engineering team is working on a solution with our service provider to mediate this issue for future events.
We have identified application icons in Core Practice may not display correctly for some users. Our team is currently investigating the issue working towards a resolution. We appreciate your patience and understanding.
Report: "Issue with Application Icon"
Last updateService provider have resolved the issue. Icon should appear correctly in Core Practice. Please refresh browser cache.
The issue has been identified and a fix is being implemented.
We have identified Icon Service Provider is currently experiencing an issue which is affecting our ability to provide Icon Service to Core Practice. We will resolve this incident once we have received confirmation the issue has been resolved from our upstream provider.
We have identified application icons in Core Practice may not display correctly for some users.
Report: "Planned maintenance - Completed"
Last updatePlanned maintenance to improve system performance has been successfully completed overnight. The maintenance work involved upgrading our servers and infrastructure, and implementing various optimizations to enhance the overall stability and responsiveness of our system. Our engineers have thoroughly tested the system and we expect everything to be working as expected.
Report: "System outage"
Last updateWe sincerely apologize for the impact to affected customers. Investigations have assessed the best ways to improve architectural resiliency and mitigate the issues that contributed to this incident and its widespread impact.
Core Practice seem to be working as expected, we will continue to monitor server performance. Microsoft Azure have determined the network connectivity issue is occurring with devices across the Microsoft Wide Area Network (WAN). This impacts connectivity between clients on the internet to Azure, as well as connectivity between services in datacenters, as well as ExpressRoute connections. The issue is causing impact in waves, peaking approximately every 30 minutes. For more info: https://status.azure.com/en-us/status
Starting at 06:30PM SYDNEY Time, we're aware of a connectivity issue relating to Service Provider Microsoft Azure. We have temporarily restored some services in Core Practice but some users may still experience degraded performance. We are actively investigating and will share updates as soon as more is known.
Report: "Elevated level of application errors"
Last updateBetween 9:00AM and 10:00AM AEDT (approximately) on 15 Sept 2022, Core Practice monitoring systems have detected high levels of application errors that triggered automatic alerts. Engineers have investigated the root cause and have identified an isolated customer with incorrect settings. Settings was corrected and application errors is now at a normal level. System is now working as expected and no users were affected during this time.
Services are restored. We are continuing to investigate the issue to find the root cause.
We're experiencing an elevated level of application errors and are currently looking into the issue.
Report: "Elevated Application Errors when sending Online Form"
Last updatePerformance should be restored to normal service, sending Online Form email should be working as expected. We apologize for the impact to affected customers. We are continuously taking steps to improve Core Practice platform. We are monitoring system stability during this time.
The issue has been identified and a fix is being implemented.
Summary of impact: From 08:00AM to 12:00PM AEDT (approx) on 22 June 2022, we're experiencing an elevated level of application errors when trying to send Online Form using email or sms. During this time some users may not be able to send Online Form. Root Cause: We are investigating the root cause may be related to third party integration related to short link service. Mitigation: As a workaround we are recommending users to use Online Form Cast feature to send to iPad during this time. Next steps: We are releasing an emergency hotfix, will provide further update once this is released.
Report: "Core Practice Bridge connectivity"
Last updateWe apologize for the impact to affected customers. We are continuously taking steps to improve Core Practice platform. To help ensure such incidents do not occur in the future we have added dedicated Server and new end point to handle Bridge connections. We will monitor stability of new infrastructure before migrating users.
Between 8:00AM and 10:00AM AEDT (approx) on 03 May 2022, we have identified one of our server may have intermittently experienced connectivity issue, during this time your Core Practice Bridge Software for HICAPS or Media Bridge may not work as expected.
Report: "New message status appear for all Appointments"
Last updateIssues has been resolved, service should return to normal operation. We sincerely apologize for the impact to affected customers for the inconvenience.
We have identified the root cause and have created a fix, we will be updating our system and refresh the cache after 6PM
Some users may be experiencing New message status appear for all Appointments. Our Development team is currently investigating issues related to recent update which create incorrect appointment cache.
Report: "Elevated Application Errors"
Last updatePerformance should be restored to normal service.
This issue have been resolved, users should be able to login using two-step authentication and SMS should be working as expected. We will continue to monitor.
We're experiencing an elevated level of application errors When trying to login using two-step authentication or sending SMS. We are currently looking into the issue with SMS provider.
Report: "Elevated Application Errors"
Last updatePerformance should be restored to normal service. Please let us know if you still experience issue.
Affected server was taken out of the pool, services should be restored. Some users may still experience slower than normal performance.
Affected server was taken out of the pool, services should be restored. Some users may still experience slower than normal performance.
Affected server was taken out of the pool, services should be restored. Some users may still experience slower than normal performance.
We are continuing to investigate this issue.
Starting 6AM on 15th March 2021, Core Practice rely on Microsoft Azure Active Directory to authenticate with Microsoft services, including Storage Service to upload Media, Attachment and to encrypt Form Data. We're experiencing an elevated level of application errors and are currently looking into the issue.
Report: "Degraded Performance for some users"
Last updateWe sincerely apologize for the impact to affected customers. Investigations have assessed the best ways to improve architectural resiliency and mitigate the issues that contributed to this incident and its widespread impact.
Summary of impact: On 1 July 2020 between 7AM-8AM, one of our server seem to experienced degraded performance or connectivity issue, during this time some users may not be able to login. Root Cause: We are investigating the root cause may be related to recent software update. Mitigation: Affected server was taken out of the pool, this will allow users to access Core Practice. Next steps: We are investigating for potential software updates to recover the affected server. We are also adding new server in case affected server can not be recovered. During this time you may experience slow performance than usual during peak period.
Report: "Core Practice is down for some users"
Last updateService monitor detected the build-up of unprocessed requests at 9:00AM AEDT. Engineer initiated secondary backup memory server and began reloading the data into memory server by 9:15AM AEDT. The cache refresh process on memory server took 15 minutes to complete and at 9:30AM AEDT most service was restored. The impacted network path was pinpointed and traffic was reconfigured to bypass the failure path. Next steps: We sincerely apologize for the impact to affected customers. Investigations have assessed the best ways to improve architectural resiliency and mitigate the issues that contributed to this incident and its widespread impact. Please perform the following: 1. Reconnect Hicaps and Media Bridge 2.Refresh browser
Lost of network connectivity on our memory server was determined as the underlying root cause, it also coincided with morning peak traffic. This combination of events caused build-up of unprocessed request and longer recovery time. Engineers have routed the traffic to minimize any customer impact.
Between 8:45AM and 9:15AM AEDT (approx) on 18 Nov 2019, we have identified one of our server may have intermittently experienced degraded performance, during this time our users may experience limited functionality or was not available.