Viedoc

Is Viedoc Down Right Now? Check if there is a current outage ongoing.

Viedoc is currently Operational

Last checked from Viedoc's official status page

Historical record of incidents for Viedoc

Report: "Planned downtime on EU instance in an upcoming maintenance window"

Last update
Completed

The scheduled maintenance has been completed.

In progress

Scheduled maintenance is currently in progress. We will provide updates as necessary.

Scheduled

On the date specified below, the Viedoc suite will be taken offline for technical infrastructure upgrades. During the upgrade the full Viedoc suite will be inaccessible for all users. We will do everything we can to limit the downtime as much as possible. In a worst-case scenario the downtime might last up to approximately 1 hour but will likely be shorter. An announcement will be published on this status page when the upgrade is about to start and as soon as completed.EU instance 31st of May 06:00 - 07:00 UTC (08:00 - 09:00 CEST)

Report: "Issues launching the TMF on the US instance"

Last update
resolved

This incident has been resolved and users on the US training and production instance can now launch the TMF as expected again.

investigating

There are currently issues accessing the TMF on the US training (clinictraining.us.viedoc.com) and US production instance (clinic.us.viedoc.com). No other instances are affected. Investigations are ongoing.

Report: "Operational disturbance observed on the European instance"

Last update
postmortem

# Viedoc service issue report **Report author** Ted Lähteenmäki, Manager IT Operations Viedoc Technologies AB  Email: [ted.lahteenmaki@viedoc.com](mailto:ted.lahteenmaki@viedoc.com)  **Report issued** 2025-03-07 **Description** Temporary unavailability of Reports application. **Instances affected** Viedoc 4 Production Instance, Europe. **Duration** 2025-03-03 15:10 UTC to 2025-03-03 16:05 UTC. Incident time stamps on the Viedoc Status Page have been updated to reflect actual start and resolution time after analyzing all log entries related to this incident. **Cause** 2025-03-03 15:10 UTC – Reports application experienced an unexpected restart. This incident was identified as being caused by human error while performing maintenance work. **Impact** Availability only. Data integrity and data confidentiality not affected. **Corrective action** Viedoc Reports dependent resources were restarted and the Viedoc Reports application was available again at 2025-03-03 16:05 UTC. **Preventive action** The routines for maintenance work will be scrutinized to avoid human errors.

resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

Operational disturbance observed in Viedoc Reports on the European instance (v4.viedoc.net). Investigations are ongoing.

Report: "Operational disturbance observed on the European instance"

Last update
resolved

This incident has been resolved.

monitoring

System are back to normal operations. We continue monitoring.

investigating

Operational disturbance observed on the European instance (v4.viedoc.net). Investigations are ongoing.

Report: "Issues generating pdf documents in Viedoc Designer on the European instances"

Last update
postmortem

**Description** Reports on failed pdf generation in Viedoc Designer on the EU Production and Training instance. **Instances affected** Viedoc 4 Production and Training Instance, Europe. **Duration** 2025-01-24 17:34 UTC to 2025-01-25 **Cause** 2025-01-24 17:34 UTC – Pdf documents \(blank CRF, annotated CRF and study workflow\) could not be printed in Viedoc Designer on the EU Training and Production instance. Documents were stuck in 0% progress without further feedback to the user. Analysis showed that documents were processed, although a high peak of print jobs resulted in a queue. During the night between 2025-01-24 and 2025-01-25 the queue had been processed and documents were again printed as expected **Impact** Documents were printed from Viedoc Designer on the EU Production and Training server with a delay during this time period. All print jobs started were completed. **Corrective action** None **Preventive action** TBD

resolved

Pdf documents (blank CRFs, annotated CRFs and study workflow) can now be generated as expected in Viedoc Designer on the European training and production server.

investigating

There are currently issues with generating pdf documents (blank CRFs, annotated CRFs and study workflow) in Viedoc Designer on the European training and production server. This is currently being investigated. No other regions are affected by the issue.

Report: "Problem launching Viedoc on the European production instance"

Last update
postmortem

**Description**                                                                                STS failure, impacting user login services. **Instances affected** Viedoc 4 Production Instance, Europe. **Duration** 2025-01-16 11:46 UTC to 2025-01-16 11:57 UTC. **Cause** 2025-01-16 11.46 UTC – Alerts were received from the automated platform monitoring, saying that the STS \(security token service\), managing authentication of users, unexpectedly stopped due to exceeding its operating capacity. **Impact** The issue led to Viedoc users on the European production instance being unable to access the login page and login to the system. There was no impact to other instances. **Corrective action** The STS was restarted and at 2025-01-16 11.57 UTC the users could access the European production instance as expected again. **Preventive action** Investigation is ongoing to determine the cause of the exception and how to prevent this from happening in the future.

resolved

The incident has been resolved since 11.57 UTC. Users can since then launch Viedoc as expected again.

investigating

We are continuing to investigate this issue.

investigating

Operational disturbance observed in the Viedoc application on the European instance (v4.viedoc.net). Users are currently unable to access the login page and a "runtime error" is displayed to the user. Investigations are ongoing.

Report: "Issues with data imports through the WCF API on the EU and US instance"

Last update
postmortem

**Description** After release of Viedoc 4.80, the WCF API did not accept requests with a large amount of data.  This was for example causing the import of clinical data through the WCF API to fail.  **Cause**  When migrating the WCF service to dot NET core, the maximum size of message accepted was set to default \(65535 bytes\) in the framework.   **Corrective action**  Updated the code to change the message size to a configurable value. And changed the configurable value to accommodate large messages.  ‌ **Preventive action**  The message size is set to max value that can be handled by system.

resolved

The issue has now been resolved. Imports of large data files submitted through the WCF API on the EU and US instances are now working as expected again. This issue was resolved on the EU instance on 2024-10-16 and on the US instance on 2024-10-17.

investigating

Imports of large data files submitted through the WCF API on the EU and US instance are currently failing. Investigation is ongoing and status updates will be provided here. For urgent imports, please contact the support or your product specialist for advice.

Report: "Intermittent login issues observed on the US production instance"

Last update
resolved

Intermittent issues with logging in to the US production and US training instance between 6:29 - 7:07PM CET on 2024-10-22. The incident is now resolved.

Report: "Account Recovery Timeline"

Last update
resolved

This incident has been resolved.

identified

Update of the issue per instance: US instance: Solved 2024-06-19 EU instance: Solved 2024-06-25 JP instance: Solved 2024-08-27 CN instance: Solved 2024-09-03

identified

A fix for the issue has been implemented in Viedoc 4.79. Once Viedoc 4.79 has been released the account recovery link will again be valid for 12 hours. More information about Viedoc 4.79 can be found here: https://www.viedoc.com/support/release-notes/

identified

When resetting the password for another users in Viedoc Admin, the account recovery link is valid only for 3 hours, instead of the expected 12 hours. To avoid issues due to this bug, please ensure that the users use the account recovery link within 3 hours from when the account was reset. If the account recovery link has not been used within 3 hours, the password will need to be reset again to trigger a new email. The user then again has a window of 3 hours to reset the password. The bug was introduced with the release of Viedoc 4.78. We are currently working on a fix to solve this issue and progress will be communicated here. If a user is resetting his/her own password from the login page, the account recovery link is still valid for 3 hours as expected.

Report: "Release of Viedoc 4.79 underway on the CN instance"

Last update
resolved

Viedoc Designer is now available on the CN instance (https://clinic.viedoc.cn/) again. Downtime of Viedoc Designer lasted for approximately 16 minutes.

monitoring

Viedoc Designer will shortly be unavailable on the CN instance (https://clinic.viedoc.cn/) due to the release of Viedoc 4.79. The downtime of Viedoc Designer is expected to last for less than 30 minutes. All other parts of the system will function as expected.

monitoring

Viedoc Designer will shortly be unavailable on the CN instance (https://clinic.viedoc.cn/) due to the release of Viedoc 4.79. The downtime of Viedoc Designer is expected to last for less than 30 minutes. All other parts of the system will function as expected.

monitoring

Approximately 30 minutes from now, Viedoc Designer will be unavailable on the CN instance (https://clinic.viedoc.cn/) due to the release of Viedoc 4.79. The downtime of Viedoc Designer is expected to last for less than 30 minutes. All other parts of the system will function as expected.

Report: "Release of Viedoc 4.79 underway on the JP instance"

Last update
resolved

Viedoc Designer is now available on the JP instance (https://v4jp.viedoc.net/) again. Downtime of Viedoc Designer lasted for approximately 14 minutes.

monitoring

Viedoc Designer will shortly be unavailable on the JP instance (https://v4jp.viedoc.net/) due to the release of Viedoc 4.79. The downtime of Viedoc Designer is expected to last for less than 30 minutes. All other parts of the system will function as expected.

monitoring

Approximately 30 minutes from now, Viedoc Designer will be unavailable on the JP instance (https://v4jp.viedoc.net/) due to the release of Viedoc 4.79. The downtime of Viedoc Designer is expected to last for less than 30 minutes. All other parts of the system will function as expected.

Report: "Issues generating documents in Viedoc Designer on European servers"

Last update
postmortem

**Description**                                                                                    Between the late afternoon of 2024-08-20 and noon 2024-08-21 some of the documents \(blank CRF, annotated CRF, Workflow\) could not be generated in Viedoc Designer. Users who attempted to generate these documents were shown the document generation being stuck at 0%. Once the worker was re-started and corrupt print jobs were removed, the worker was functioning as expected again. **Instances affected** [https://v4.viedoc.net/](https://v4.viedoc.net/); [https://v4training.viedoc.net](https://v4training.viedoc.net) **Duration** Between the late afternoon of 2024-08-20 and noon 2024-08-21. **Cause** As preparation for some technical infrastructure upgrades, the designer worker handling print jobs was re-started in the afternoon of 2024-08-20. When re-started, a print job that was ongoing when the worker was re-started ended up in a corrupt state, not being possible to process by the worker. This blocked the queue and no other print jobs were executed in Designer. Around 9AM on 2024-08-21 the first report came in from a customer that the print jobs were not being processed. **Impact** Between re-starting the worker and resetting the status of the corrupt print job, no print jobs handled by this worker were processed. Once the status of the corrupt print job was reset, the queue was again being processed as expected. Thus, the impact for the customers was a delay in getting these specific print jobs completed. **Corrective action** When the print job queue was reviewed, the invalid print job blocking the queue was detected. This status of this job was then reset so that the worker would start over. After this action the print job queue was processed as normal again and print jobs that had previously not been completed were executed. **Preventive action** Since this issue was triggered by the manual intervention or re-starting the worker, the initial preventive action is just to manually monitor to see that print jobs start executing as expected in case the worker needs to be re-started again. For the future, improved automatic monitoring and re-try attempts will be evaluated.

resolved

This incident has been resolved. Documents can now be generated in Viedoc Designer as expected again. The issue was resolved in the European production instance around 10:30 UTC and for the European training instance around 11:55 UTC.

investigating

There are currently issues with generating documents (blank CRFs, annotated CRFs and study workflow) in Viedoc Designer on the European training and production server. This is currently being investigated.

Report: "Updated release schedule for Viedoc 4.79 for the CN instance"

Last update
resolved

Updated release schedule for Viedoc 4.79. The release is planned for the following dates on the CN instance: 2024-09-03 (all applications except Viedoc TMF) 2024-09-04 (Viedoc TMF)

Report: "Updated release schedule for Viedoc 4.79 for the JP instance"

Last update
resolved

Updated release schedule for Viedoc 4.79. The release is planned for the following dates on the JP instance: 2024-08-27 (all applications except Viedoc TMF) 2024-08-28 (Viedoc TMF)

Report: "Viedoc 4 - EU - Operational Disturbance"

Last update
postmortem

# Viedoc service issue report  **Report author**  Ted Lähteenmäki, Manager IT Operations  Viedoc Technologies AB Email: [ted.lahteenmaki@viedoc.com](mailto:ted.lahteenmaki@viedoc.com)   **Report issued**  2024-07-30  **Description**` `  Disturbance in IdP impacting user login services.  **Instances affected**  Viedoc 4 – Europe.  **Duration**  2024-07-30 13:30 UTC to 2024-07-30 13:35 UTC.  **Cause**  The infrastructure provider \(Microsoft Azure\) experienced a service issue which impacted our IdP login services.  **Impact**  Upon attempting to log in to Viedoc, a “Maintenance mode” message appeared during the times specified above. Data integrity and confidentiality were not affected.  **Corrective action**  This issue was resolved automatically as when the infrastructure provider services were restored.  **Preventive action**  Pending the Root Cause Analysis report from the infrastructure provider.

resolved

This incident has been resolved.

monitoring

The incident was resolved by infrastructure provider Microsoft Azure.

Report: "Viedoc 4 - US - Operational Disturbance"

Last update
resolved

Incident have been resolved by infrastructure provider Microsoft Azure, meaning that the US instance of Viedoc is now operating as expected again.

identified

We have identified that our provider Microsoft Azure is experiencing a disturbance with their service.

investigating

We are currently investigating an operational disturbance.

Report: "Disturbance with text messages sent to US phone numbers from the EU instance"

Last update
resolved

This incident has been resolved.

investigating

There are currently disturbances with text messages sent to US phone numbers from the Viedoc EU instance. This is affecting 2FA text messages and Viedoc Me reminders sent via SMS. Numbers from other regions are not affected.

Report: "Unscheduled maintenance window for Viedoc 3 (legacy platform)"

Last update
resolved

Due to an unforeseen issue with a TLS certificate roll-over procedure, an unscheduled maintenance window had to be implemented between 9:20 -- 10:00 UTC (18:20 -- 19.00 JST). We apologize for any inconvenience this might have incurred.

Report: "Viedoc 4 Europe - Email delivery issues"

Last update
postmortem

# Viedoc service issue report **Report author** Ted Lähteenmäki, Manager IT Operations Viedoc Technologies AB Email: [ted.lahteenmaki@viedoc.com](mailto:ted.lahteenmaki@viedoc.com) **Report issued** 2024-06-24, updated 2024-06-28 **Description** Email delivery was delayed, and subsequently duplicate emails were sent upon email service restoration. **Instances affected** Viedoc 4 Europe **Duration** 2024-06-15 14:00 CEST to 2024-06-17 09:30 CEST **Cause** An annual certificate change was not picked up by one system. Mailmonitor service started to cache up and duplicate emails to be sent. **Impact** Emails were not sent in the timeframe between 2024-06-15 14:00 to 2024-06-17 09:30 CEST. When the certificate was updated on the system all the emails started to be processed. The Mailmonitor service started to send all the emails it had in the queue. \[Update 2024-06-28\] Additionally, some emails were never sent due to this issue. Please refer to the communication log in Viedoc Admin and contact Viedoc Support if you have further questions on emails that were not sent. **Corrective action** System with expired certificate was restarted and the Mailmonitor service was stopped. **Preventive action** The processes for updating the certificate on our systems is updated with clarification to check that new certificates are picked up. The Mailmonitor service will be updated to not duplicate emails in case it resends.

resolved

Email delivery was delayed and subsequently duplicate emails were sent upon email service restoration.

Report: "Viedoc 4 - US - Disturbance in sending emails and SMS"

Last update
postmortem

# Viedoc service issue report **Report author** Ted Lähteenmäki, Manager IT Operations Viedoc Technologies AB Email: [ted.lahteenmaki@viedoc.com](mailto:ted.lahteenmaki@viedoc.com) **Report issued** 2024-06-28 **Description**                                                                                                                                     Disturbance in sending emails and SMS. **Instances affected** Viedoc 4 US **Duration** 2024-06-25 23:55 CDT to 2024-06-26 04:50 CDT \(2024-06-26 06:55 CEST to 2024-06-26 11:50 CEST\) **Cause** An unannounced maintenance by the infrastructure provider \(Microsoft Azure\) impacted our application handling distribution of email and SMS. During the maintenance the infrastructure provider lost our image that should be loaded to run our application, resulting in Emails and SMS not being sent. **Impact** Availability only. Data integrity and data confidentiality were not affected. **Corrective action** The infrastructure provider restarted their worker server handling our instance allowing for redeploying of the application. **Preventive action** Pending the Root Cause Analysis report from the infrastructure provider, additional preventive actions might be implemented.

resolved

This incident has been resolved.

investigating

We are currently investigating this issue.

Report: "Viedoc 4 - EU - Disturbance with Reports"

Last update
resolved

This incident has been resolved.

investigating

We are currently investigating this issue.

Report: "Release of Viedoc 4.79 - TMF"

Last update
resolved

Viedoc TMF is now available again. Downtime of Viedoc TMF lasted for about 50 minutes.

investigating

Viedoc TMFwill now be unavailable due to the release of Viedoc 4.79. The downtime of Viedoc TMF is expected to last for less than 30 minutes. All other parts of the system will function as expected.

Report: "Viedoc Reports disturbance"

Last update
resolved

Between 00:00 and 07:50 CET on 2024-06-26, there were issues with loading Viedoc Reports, preventing it from loading properly for studies using the EU instance. This was resolved on 07:50 CET.

Report: "Viedoc Reports"

Last update
resolved

VIedoc Reports is now available again.

investigating

Viedoc Reports temporarily unavailable on the EU instance. We are currently investigating the issue.

Report: "Release on Viedoc 4.79 underway on the EU instance"

Last update
resolved

Viedoc Designer is now available again. Downtime of Viedoc Designer lasted for approximately 14 minutes.

monitoring

Viedoc Designer will now be unavailable on the EU instance due to the release of Viedoc 4.79. The downtime of Viedoc Designer is expected to last for less than 30 minutes. All other parts of the system will function as expected.

monitoring

Approximately 30 minutes from now, Viedoc Designer will be unavailable on the EU instance due to the release of Viedoc 4.79. The downtime of Viedoc Designer is expected to last for less than 30 minutes. All other parts of the system will function as expected.

Report: "Release of Viedoc 4.79 complete on the US instance"

Last update
resolved

This incident has been resolved.

identified

Viedoc Designer is now available again. Downtime of Viedoc Designer lasted for 21 minutes.

identified

Viedoc Designer will now be unavailable on the US instance due to the release of Viedoc 4.79. The downtime of Viedoc Designer is expected to last for less than 30 minutes. All other parts of the system will function as expected.

Report: "Updated release schedule for Viedoc 4.79"

Last update
resolved

Updated release schedule for Viedoc 4.79. The release is planned for August/September on the JP instance.

Report: "Updated release schedule for Viedoc 4.79"

Last update
resolved

Updated release schedule for Viedoc 4.79. The release is planned for August/September on the CN instance.

Report: "Updated release schedule for Viedoc 4.79"

Last update
resolved

Updated release schedule for Viedoc 4.79. The release is planned for 2024-06-24 – 2024-06-26 on the EU instance.

Report: "Viedoc - US - Disturbance with emails and SMS"

Last update
resolved

Services are fully recovered.

investigating

We are currently investigating this issue.

Report: "Viedoc 4 - Operational Disturbance"

Last update
resolved

Azure China reports that the DNS resolution issue has been mitigated.

monitoring

Our subprovider Azure China experience issues connecting to multiple Azure services due to DNS resolution issues. They have applied a fix and are monitoring the recovery. This can cause intermittent delays in accessing Viedoc.

Report: "Japanese Instance: Degraded performance on certain actions in PMS-studies"

Last update
postmortem

For the Japanese Viedoc instance, degraded performance was reported for specific tasks in the PMS studies. Non-PMS studies were not affected. These included actions such as: - opening subject cards - clicking “manage” for partially submitted forms - saving forms - booklet actions submit, return, and recall. Issues were identified and performance was improved in Viedoc 4.77. With the release of 4.77 on 2023-10-06 this issue was resolved. Performance was immediately restored as expected.

resolved

Viedoc 4.77 has now been released on the Japanese training and production instances. Several fixes for PMS-studies are included, resulting in a significant improved performance.

identified

The release of Viedoc 4.77 has been postponed and is now planned release 5th (JP), 9th (EU) and 10th (CN) of October. The rationale for releasing 4.77 in Japan first is that the release is including some prioritized performance improvements in Japanese PMS studies.

identified

The release of Viedoc 4.77 has been postponed one week and is now planned for release the 3rd - 5th of October.

identified

The release of Viedoc 4.77 has been postponed one week and is now planned for release 26th - 28st of September.

identified

Issues have been identified in the remaining categories ("opening subject cards", "saving forms" and "booklet actions submit, return, and recall"). Performance improvements are currently being implemented. The current plan is to release these improvements in the upcoming release 4.77 (currently planned for 19th - 21st of September)

identified

Issues related to "clicking “manage” for partially submitted forms" have been identified and work is ongoing for fixing those issues. Other actions with degraded performance are still being investigated.

investigating

For the Japanese Viedoc instance in PMS studies there are some actions that currently have degraded performance. These include actions such as: - opening subject cards - clicking “manage” for partially submitted forms - saving forms - booklet actions submit, return, and recall. Investigations are ongoing to identify the problems to be able to improve the performance of these actions. Progress of our investigations will be updated on this page. These performance issues are not present in non-PMS studies or other instances.

Report: "Release of 4.78 on Japanese Instance"

Last update
resolved

Migration of login history for all users is now complete, meaning that the hundred most recent logins for each user is again displayed in the user profile in Viedoc Admin. This also means the release of Viedoc 4.78 is fully completed.

monitoring

The release of Viedoc 4.78 is now completed on the Japanese instance and users can since 09.59 CET login and use Viedoc as normal. Interruption lasted for 27 minutes. Login history of users is still being updated in the background and will be completed within a couple of days. The only impact of this is that the user login history will not display all records until the update is complete. It will be announced on this page when the update of login history for all users is complete.

identified

The release of Viedoc 4.78 is underway on the Japanese instance. During this release, as previously communicated, the system will be down for a short period of time, and it will be impossible to log in or use the system. It is expected that the downtime will be less than 30 minutes. We will do everything we can to make this interruption as short as possible and will update you on the progress here.

Report: "Release Dates of Viedoc 4.78"

Last update
resolved

Viedoc 4.78 has now been successfully released on all Viedoc instances. US servers: 2024-03-11 (released) China servers: 2024-03-14 (released) Europe servers: 2024-03-16 (released) Japan servers: 2024-03-23 (released)

monitoring

Viedoc 4.78 has now been successfully released on all Viedoc instances. US servers: 2024-03-11 (released) China servers: 2024-03-14 (released) Europe servers: 2024-03-16 (released) Japan servers: 2024-03-23 (released)

monitoring

The release of Viedoc 4.78 is scheduled to the following dates: US servers: 2024-03-11 (released) China servers: 2024-03-14 (released) Europe servers: 2024-03-16 (released) Japan servers: 2024-03-23 As previously communicated the system will be down for a short period of time during the release of Viedoc 4.78 and it will be impossible to log in or use the system. It is expected that the downtime will last less than 30 minutes. We are making everything we can to make this interruption as short as possible and will update you on the progress on this page.

monitoring

US servers: 2024-03-11 (released) China servers: 2024-03-14 (released) Europe servers: 2024-03-16 Japan servers: 2024-03-23 As previously communicated the system will be down for a short period of time during the release of Viedoc 4.78 and it will be impossible to log in or use the system. It is expected that the downtime will last less than 30 minutes. We are making everything we can to make this interruption as short as possible and will update you on the progress on this page.

monitoring

The release of Viedoc 4.78 is scheduled to the following dates: US servers: 2024-03-11 (released) China servers: 2024-03-14 (released) Europe servers: 2024-03-16 Japan servers: TBD As previously communicated the system will be down for a short period of time during the release of Viedoc 4.78 and it will be impossible to log in or use the system. It is expected that the downtime will last less than 30 minutes. We are making everything we can to make this interruption as short as possible and will update you on the progress on this page.

monitoring

The release of Viedoc 4.78 is scheduled to the following dates: US servers: 2024-03-11 (released) China servers: 2024-03-14 Europe servers: 2024-03-16 Japan servers: TBD As previously communicated the system will be down for a short period of time during the release of Viedoc 4.78 and it will be impossible to log in or use the system. It is expected that the downtime will last less than 30 minutes. We are making everything we can to make this interruption as short as possible and will update you on the progress on this page.

monitoring

The release of Viedoc 4.78 on the Japanese server has been postponed. New release dates for Viedoc 4.78 to the US, European, Japanese and Chinese instance will be communicated on this page shortly. Japan servers: TBD US servers: TBD China servers: TBD Europe servers: TBD

monitoring

The release of Viedoc 4.78 has been postponed for the US, China and European server. The release is still on track for the Japanese server. Japan servers: Planned for 2024-02-24 (date not changed) US servers: TBD China servers: TBD Europe servers: TBD The release of 4.78 has been moved to a later date. The rationale for postponing the release is that an issue was observed in the final testing stages, and to minimize the risk, this issue will be analyzed and fixed before the release. New release dates will be communicated here as soon as they are available.

Report: "Occasional delay in accessing user details page on EU instance"

Last update
postmortem

**Background:** As part of the Viedoc 4.78 release on 2024-03-16, all user data was moved to a new dedicated database. This transition uncovered a slowdown issue due to a missing index, particularly affecting user detail page load times in Viedoc Admin. Detected on 2024-03-18, this was caused by an overlooked index in the user login history table. This scenario was not anticipated during testing due to its occurrence under specific high-load conditions, as observed in the production instances. **Resolution:** To address this, we added the necessary index to the login history table at 16.51 PM CET on 2024-03-18, promptly restoring normal response times and preventing future recurrences. **Impact:** From 09.00 AM to 16.51 PM CET on 2024-03-18, users experienced delays and, in some cases, timeouts when accessing user details pages.   We apologize for any inconvenience this may have caused.

resolved

During the day 2024-03-18 there have been occasional delays accessing the details page of other users from Viedoc Admin. This led to slower response times and in some cases requests timed out so that an error message was displayed on the screen. This issue was solved 16:51 CET. Performance is back up and the issue is resolved now. Users should be able to access user details page as expected.

Report: "Release of 4.78 on EU Instance"

Last update
resolved

The release of Viedoc 4.78 is now completed on the EU instance and users can again login and use Viedoc as normal. Login history of users is still being updated in the background and will be completed within a couple of hours. The only impact of this is that the user login history will not display all records until the update is complete.

investigating

The release of Viedoc 4.78 is underway on the EU instance. During this release, as previously communicated, the system will be down for a short period of time, and it will be impossible to log in or use the system. It is expected that the downtime will be less than 30 minutes. We will do everything we can to make this interruption as short as possible and will update you on the progress here.

Report: "Release of 4.78 on China Instance Completed"

Last update
resolved

The release of Viedoc 4.78 is now completed on the Chinese instance and users can again login and use Viedoc as normal. Login history of users is still being updated in the background and will be completed within a couple of hours. The only impact of this is that the user login history will not display all records until the update is complete.

monitoring

The release of Viedoc 4.78 is underway on the Chinese instance. During this release, as previously communicated, the system will be down for a short period of time, and it will be impossible to log in or use the system. It is expected that the downtime will be less than 30 minutes. We will do everything we can to make this interruption as short as possible and will update you on the progress here.

Report: "Release of Viedoc 4.78 postponed"

Last update
resolved

The release of Viedoc 4.78 has been replanned to the following dates: Planned for 2024-01-31: US servers Planned for 2024-02-15: China servers Planned for 2024-02-17: Europe servers Planned for 2024-02-24: Japan servers As previously communicated the system will be down for a short period of time during the release of Viedoc 4.78 and it will be impossible to log in or use the system. It is expected that the downtime will last less than 30 minutes. We are making everything we can to make this interruption as short as possible and will update you on the progress on this page.

investigating

The release of 4.78 has been move to a later date. The rationale for postponing the release is that an issue was observed in the final testing stages, and this issue will need to be analyzed and fixed before the release. New release dates will be communicated here as soon as they are available.

Report: "Issues generating user administration log in some studies"

Last update
resolved

This problem has been resolved with the release of hotfix Viedoc 4.77.1. User Administration Logs can now be generated as expected again.

identified

The issue has been identified and a fix is being implemented.

investigating

When generating user administration logs from Viedoc Clinic or Viedoc Admin, an error occurs in a few studies. Investigation is ongoing to identify the cause for this. Most studies are unaffected by this issue.

Report: "Issues uploading files through new ViedocMe"

Last update
resolved

This issue is now resolved with the release of Viedoc 4.77. Viedoc 4.77 was released on the production environments on the following dates: - Japan 2023-10-06 - Europe 2023-10-09 - China 2023-10-12

identified

The fix has been implemented and will be released in Viedoc 4.77. As soon as Viedoc 4.77 is released file upload will work as expected in the new ViedocMe interface.

identified

Uploading files > 2MB fails through the new Viedoc Me interface. Uploading files less than 2MB works as expected. Uploading files in the old ViedocMe interface or the EDC works as expected. The issue has been identified and we are working on a fix.

Report: "SMS test message delivery issues with some US operators"

Last update
resolved

This issue is now resolved with the release of Viedoc 4.77.

identified

The release of Viedoc 4.77 has been postponed and is now planned release 5th (JP), 9th (EU) and 10th (CN) of October. The rationale for releasing 4.77 in Japan first is that the release is including some prioritized performance improvements in Japanese PMS studies.

identified

The release of Viedoc 4.77 has been postponed one week and is now planned for release the 3rd - 5th of October.

identified

The release of Viedoc 4.77 has been postponed one week and is now planned for release 26th - 28st of September.

identified

The current manually-triggered, test, text message from the Viedoc Me activation page contains several words that increase the likelihood of it being caught in the phone operator’s spam filters: “This message is sent from Viedoc to confirm your contact information. Click the following link to login to ViedocMe: {url}”. To reduce the risk of this text message being caught in the filters, the second sentence will be removed in the release of Viedoc 4.77. Thus, after the release of Viedoc 4.77, the new test text message will be: “This message is sent from Viedoc to confirm your contact information.” Viedoc 4.77 is planned for release 19th - 21st of September.

investigating

We are waiting for information from our subprocessors after their investigation on why test text messages are failing to some US numbers. Scheduled reminders are still being sent as expected

investigating

The problem with text messages not being sent to US phone numbers is isolated to the test text messages, triggered manually from the ViedocMe activation page in the EDC. Investigation is still ongoing to identify the problem with the test text messages.

investigating

We are continuing to investigate this issue with our SMS infrastructure providers. Problem still seem contained to Verizon and T-Mobile network users.

investigating

We are continuing to investigate this issue with our SMS infrastructure providers. Problem seems isolated to Verizon and T-Mobile network users.

investigating

We are continuing to investigate this issue with our SMS infrastructure providers.

investigating

We are continuing to investigate this issue with our SMS infrastructure providers.

investigating

We are currently investigating this issue.

Report: "Viedoc Reports not syncing data automatically"

Last update
resolved

Viedoc Reports is now automatically syncing data again per the expected schedule. More information about the expected schedule for data sync can be found in the relevant eLearning: https://help.viedoc.net/c/8a3600/4cdde8/en/

investigating

Viedoc Reports are currently not syncing data automatically. The manual/ad-hoc sync is still working as expected, meaning that data could still be synced manually on demand by any user with access to the Reports tool.

Report: "Viedoc 4 China - Operational Disturbance"

Last update
resolved

This incident has been resolved.

identified

The issue has been identified and a fix is being implemented.

investigating

We are currently investigating an operational disturbance.

Report: "Viedoc 4 Reports - Operational Disturbance"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are currently investigating an operational disturbance.

Report: "China Viedoc 4 Reports - Operational Disturbance"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are continuing to investigate this issue.

investigating

We are currently investigating an operational disturbance.

Report: "Temporary delay of exports and pdf generation in association with the release of Viedoc 4.76 on the CN instance"

Last update
resolved

Viedoc 4.76 has now been released on the CN instance and the export and pdf queue is being processed.

investigating

The release of Viedoc 4.76 is now ongoing on the CN instance. During this release some pdf’s will not be processed and exports will not be generated. Instead pdf’s and exports will be placed in queue. This is expected to last for around 30 minutes, then the pdf generation and exports will work as expected and the queue will be processed. No action is required by the user and pdf’s and exports will automatically be generated when the queue is processed.

Report: "Temporary delay of exports and pdf generation in association with the release of Viedoc 4.76 on the JP instance"

Last update
resolved

Viedoc 4.76 has now been released on the JP instance and the export and pdf queue is being processed.

monitoring

The release of Viedoc 4.76 is now ongoing on the JP instance. During this release some pdf’s will not be processed and exports will not be generated. Instead pdf’s and exports will be placed in queue. This is expected to last for around 30 minutes, then the pdf generation and exports will work as expected and the queue will be processed. No action is required by the user and pdf’s and exports will automatically be generated when the queue is processed.

Report: "Temporary delay of exports and pdf generation in association with the release of Viedoc 4.76 on the EU instance"

Last update
resolved

Viedoc 4.76 has now been released and the export and pdf queue is being processed.

monitoring

We are continuing to monitor for any further issues.

monitoring

The release of Viedoc 4.76 is now ongoing on the EU instance. During this release some pdf’s will not be processed and exports will not be generated. Instead pdf’s and exports will be placed in queue. This is expected to last for around 30 minutes, then the pdf generation and exports will work as expected and the queue will be processed. No action is required by the user and pdf’s and exports will automatically be generated when the queue is processed.

Report: "France Viedoc 4 Reports - Operational Disturbance"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are continuing to investigate this issue.

investigating

We are currently investigating this issue.

Report: "Japan Viedoc 4 Reports - Operational Disturbance"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are currently investigating this issue.

Report: "China Viedoc 4 - Operational Disturbance"

Last update
resolved

This incident has been resolved.

identified

Issue is identified and fix was applied.

investigating

We are continuing to investigate this issue.

investigating

We are currently investigating this issue.