Historical record of incidents for Subsplash
Report: "When processing files using Pulpit AI, some may remain in the “Processing” state longer than expected"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Dashboard login returning error message"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating an issue causing an error message to appear when attempting to login to the dashboard. The error message reads "There was an issue loading this content".
Report: "On-Demand Media Transcoding is Failing at a Higher Rate"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: ""Something went wrong" error messages across multiple Subsplash features"
Last updateThis incident has been resolved.
We are currently investigating an issue causing pages across multiple Subsplash features to display a "Something went wrong" error message. In many cases, refreshing the page will cause it to load correctly.
A fix has been implemented and we are monitoring the results.
We are currently investigating an issue causing pages across multiple Subsplash features to display a "Something went wrong" error message. In many cases, refreshing the page will cause it to load correctly
Report: "Clients Redirected to a Pricing Page After Selecting "Live" in the Dashboard Menu"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Simulated Live Streams Disconnecting"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating an issue causing some Simulated Live Streams to either fail to start, or disconnect. This only affects Simulated Live Streams. True live broadcasts are unaffected.
Report: "Subsplash Messaging Displaying, "Something Went Wrong.""
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating an issue causing Subsplash Messaging to display a "Something Went Wrong" error when attempting to load.
Report: "Gifts not showing up in the Dashboard"
Last updateThis incident has been resolved. Please see the Statements page of you giving dashboard for an important update about generating statements.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
We are currently investigating an issue causing gifts to not show up in the Dashboard, on statements, and on reports. This does not affect Donors' ability to give. Gifts are still being processed normally and this issue only affects their visibility in the Dashboard.
Report: "SMS texts not being delivered"
Last updateFull resolution of this issue will require continued communication with our 3rd party partner. As we continue to work with them, we will provide updates on this issue via the SMS section within the Management dashboard. Please see that section of your dashboard for continued updates.
We’ve identified the issue in a 3rd Party Service provider that our SMS system depends on. We’re actively working with that service provider to resolve the issue as quickly as possible, and will provide updates as progress is made.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are currently experiencing an issue affecting SMS delivery for our USA-based clients.
Report: "Unable to View Messaging Groups in the Dashboard"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating an issue where some Messaging users do not see Groups in the Subsplash Dashboard; however, they are still visible in the Mobile App. Updates will be posted as we identify and resolve this issue.
Report: "Subsplash Live Facebook Syndication Disconnect"
Last updateSubsplash Live streams were unable to syndicate to Facebook due to a change made on Facebook's end, which caused that syndication to time out. This has since been identified and fixed, and will continue to work properly moving forward.
Report: "Messaging Not Loading for Some Users."
Last updateThis incident has been resolved.
All systems are now operational. We appreciate your patience!
A fix is now in place and everything is working as expected. We will closely monitor this resolution to ensure stability.
We believe the underlying issue is with a 3rd party vendor and we are actively working with them to find the root cause and implement a solution.
We are continuing to investigate this issue. Thank you for your patience and understanding.
We are continuing to investigate this issue.
We are currently investigating an issue with Subsplash Messaging. After logging in, some users may experience an error after the page fails to load properly. Our team is actively working on a resolution.
Report: "Subsplash Giving and Live Stream Service Interruption"
Last updateThis incident has been resolved.
We are continuing to monitor the situation and will post another update when there is more information to share.
The Subsplash Giving and Live Stream disruption is resolved. All systems are fully operational. We will monitor the results of this resolution closely.
We are continuing to investigate an issue where some Subsplash Giving and Live Streams result in failures to load.
We are currently investigating an issue where some Subsplash Giving and Live Streams result in failures to load.
Report: "Subsplash Dashboard Service Disruption"
Last updateOn July 23, 2024, Subsplash experienced an unexpected infrastructure change to a core networking system. While this networking system was interrupted, many of our Subsplash backend services were unavailable. The Dashboard, Giving, Live, SnapPages, Mobile Apps, and Messaging were among these services. This interruption affected both Clients managing the services and End-Users interacting with them. Recovering the services required manual intervention to properly reconfigure the networking system. We are prioritizing the improvement of our misconfiguration alerts and network resilience.
This incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are currently investigating an issue preventing Clients from logging into the Dashboard. Clients may receive an error when attempting to log into the Dashboard. This issue is also affecting other Subsplash services from loading such as Mobile Apps, SnapPages, Giving, and Media & Live.
Report: "Giving Data not recording and Batches returning error message"
Last updateThis incident has been resolved. You should see any missing gift or payment from transactions during this incident appear in your dashboard in the next couple days.
A fix has been implemented, and all data related to gifts and payments will record correctly moving forward. Our team is currently working on restoring data that was not recorded due to this issue.
The issue has been identified, and the ability to commit batches has been restored. We are currently working on a fix for Giving Data to be recorded in the Dashboard. This does not affect donor's ability to give gifts or payments.
We are currently investigating an issue causing Giving Data for Donations, Payments, and Transactions to not record, as well as creating batches returning an error message. This does not affect donor's ability to give gifts or payments.
Report: "Fill-In Notes Service Interruption"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
A fix was implemented, however, the issue persists so our team is working to identify a new resolution.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Our team is aware of and actively investigating the interruption with Fill-In Notes.
Report: "Fill-In Notes Service Interruption"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "SnapPages Service Disruption"
Last updateThis incident has been resolved.
Sites are operating as expected and we will continue to monitor until we are assured of a resolution.
We are continuing to monitor for any further issues. In the meantime, clients may experience slower load times.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Missing Fill-In Notes"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
An issue causing Fill-In Notes to display "No Pages" has been identified and a fix is being implemented.
Report: "Church Management Dashboard returning 502 Error"
Last updateThis incident has been resolved, thank you for your patience.
A fix has been implemented and we are monitoring the results.
The issue has been identified and our team is continuing to research its implications to determine a solution.
We are currently investigating an issue causing Church Management Dashboard pages to return a 502 error. This may also impact loading some pages on the Subsplash Dashboard as well.
Report: "Messaging Groups not loading from Group Finder"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are currently investigating an issue causing Messaging Groups to not load from Group Finder page.
Report: "Unable to access Dashboards"
Last updateWe discovered an issue earlier today that caused giving and media analytics, as well as check-in data, to be inaccessible for around 45 minutes. Once it was discovered, it was identified resolved very quickly by our team, and access has been restored.
Report: "Live Streams returning error code instead of starting"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating an issue causing an error code that reads "could not access the specified channel or stream key" when attempting to start a live stream
Report: "Video files failing to upload to the Dashboard"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We are investigating an issue preventing videos from uploading to the Dashboard.
Report: "Web App, Embeds, and Blocks not loading"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are currently investigating an issue causing the Web App, Embeds, and Blocks to show a "502 Bad Gateway" message.
Report: "Giving Dashboard returning 500 error and Giving page continuously loading"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating an issue causing Giving Dashboards to display a 500 error and some Giving pages to display a continuous loading wheel
Report: "Direct links to Media Items and Live Stream items returning error page"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating an issue causing short links to media items and live streams to return an error page that reads "Oops! Page not found." This is affecting direct links to media items and live streams, but should not affect playback in the app or web embeds.
Report: "Unable to upload media to a Media Item"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Media Uploads Failing to Process"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
A fix has been deployed for media failing to upload or transition. We are monitoring this fix.
Report: "Giving Transfers not updating in dashboard"
Last updateA recent change was deployed that caused slow processing of giving data. Transactions with Stripe were not affected, and all affected data in Subsplash systems is expected to be caught up by the end of this week.
This incident has been resolved. It will still take some time for the system to catch up fully and display every transaction and transfer. If you still see missing information on your Giving dashboard, that information should be visible soon.
A fix has been implemented and we are monitoring the results. This issue created a backlog of updates that will take some time to catch up now that the fix has been implemented.
The issue has been identified and our team is working on a fix.
Our team is continuing to investigate an issue causing the Giving dashboard to not reflect recent transactions and transfer reports. This affects transactions and transfers starting Monday, 3/4. We assure you that this is just a display issue, and transactions are successfully running and funds are being transferred to your account. This issue will also delay ChMS syncing.
Our team is currently investigating an issue causing the Giving Transfers page to not update and display the most recent deposit. This is purely a display issue and does not affect funds depositing to bank accounts. You may see a delay in Integration syncing due to this.
Report: "People and Groups experiencing loading issues"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring results. People and Groups pages should now be loading properly.
Our team is currently investigating an issue related to People and Groups not loading properly in Subsplash Dashboards.
Report: "Events tab and events in calendars not loading"
Last updateOn February 2nd and 3rd, an incident occurred that limited viewing or editing event-related data in the dashboard, as well as viewing events in apps and on the web. This occurred due to load on our system that exceeded expected parameters. As a result, our team implemented a solution and system capacities have been increased.
This incident has been resolved.
Events tabs and calendars are now restored, and we are monitoring the results.
The issue has been identified and we are working to develop a fix.
We are continuing to investigate this issue.
Our team is currently investigating an issue where the events tab in your dashboard or events in calendars are not loading, preventing users from creating new events or editing existing events.
Report: "Event embeds not loading on web pages"
Last updateEvent embeds were unable to load on websites. When navigating to web pages containing an events embed, no events would populate on the page. After receiving reports of this issue, our team quickly identified the cause and it has since been resolved.
Report: "Web players (SnapPages and Web App) failing to load files that were recently uploaded."
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Events tab and events in calendars are not loading"
Last updateThis incident has been resolved.
A fix has been implemented and we're monitoring results. New events can now be created and existing events can be edited.
Our team is continuing to investigate this issue.
Our team is currently investigating an issue where the events tab in your dashboard or events in calendars are not loading, preventing users from creating new events or editing existing events.
Report: "SnapPages Web Builder, Apps and Livestreams unable to load."
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Our team is currently investigating an issue where users are unable to load their SnapPages web builder, mobile apps, or live streams.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Delays in recurring gift processing and ChMS syncing"
Last updateAll delays in recurring donations and ChMS integration syncing are resolved and have resumed standard operation.
All recurring donations previously queued have been processed, all future recurring donations will now process on time. ChMS integration syncing has resumed. You can expect to see all transaction syncing with your ChMS within the next several days.
On Monday, January 15th (Bank Holiday/MLK Jr. Day), some recurring donations were delayed. All of these transactions will process if they have not already. Our system immediately began processing delayed recurring donations and all donations are actively in a queue to be processed. No action is needed from clients at this time. All recurring donations will be processed within 24 hours if they have not already been processed. In order to speed up the processing of delayed recurring transactions, we temporarily paused ChMS integration syncing on Wednesday, and since resumed syncing on Thursday 1:00pm PST. No action is needed from clients at this time and they should expect to see all transactions synced with their ChMS within the next several days.
Report: "Live Chat Service Interruption"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating an issue with a service interruption for live chat. Live streaming is not being affected.
Report: "Church Management mailouts not sending"
Last updateThis issue is now resolved
Mailouts are now delivering. We are monitoring to ensure this issue is resolved.
We are currently investigating an issue related to mailouts not being sent.
Report: "Fill-in Notes Outage"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are currently investigating an issue where Fill-in Notes are not loading past notes and new notes are not being created.
Report: "Service Interruption"
Last updateOn October 22, 2023, the Subsplash Platform experienced a service interruption that impacted the functionality of some of our products. During a routine technical update to one of our services, there was a misconfiguration that impacted the autoscaling technology of Subsplash Giving, Websites, and our Dashboard for some of our clients—we saw roughly 1% of traffic requests fail during this time and understand the impact this had on your end users. During the 2 hours of interruption these products operated with an uptime of 99% and 99.99% for the day. Other services were entirely unaffected and maintained an uptime of 99.99%-100%. Our engineers identified the root cause and corrected the issue. The Subsplash team is actively working to prevent this issue from occurring in the future, including expanding our load testing capabilities and implementing better observability into our use of the impacted services in the event of future updates.
This incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating the issues affecting multiple areas of our platform.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are currently investigating an issue affecting multiple areas of the platform, including dashboard access, Giving, websites, and Check-in.
Report: "Messaging and Live Chat Service Interruption"
Last updateService functionality is restored
There is currently a known issue with a third party service used with messaging and live chat. We are monitoring the status of the service and will update here when it's fixed.
Report: "Fill-In Notes Outage"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Church Management - Degraded Experience for some Checkin Clients"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Some clients may be experiencing degraded performance with the Subsplash Checkin app. This issue can be resolved by uninstalling and reinstalling the Subsplash Checkin app. If after uninstalling and reinstalling the Subsplash Checkin app, you find that your labels are printing longer than usual, then please go into your kiosk settings on the check-in app and select the appropriate tape size to match what you are using, then quit/relaunch the app.
We are continuing to investigate this issue.
If after uninstalling and reinstalling the Subsplash Checkin app, you find that your labels are printing longer than usual, then please go into your kiosk settings on the check-in app and select the appropriate tape size to match what you are using, then quit/relaunch the app.
Some clients may be experiencing degraded performance with the Subsplash Checkin app. This issue can be resolved by uninstalling and reinstalling the Subsplash Checkin app.
We are currently investigating this issue.
Report: "Mailout Service Interruption"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have identified the issue with Church Management mailouts failing to send, and our team is currently implementing a fix.
We are currently investigating this issue.
Report: "HTTP Error 500 on some SnapPages websites"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
We are currently investigating an issue causing some SnapPages sites to show a HTTP Error 500
Report: "Events Unable to be Scheduled"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Slow load times on Church Management"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are currently investigating an issue causing some users to experience slow to very slow load times on Church Management.
Report: "Messaging Service Interruption"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified, and we are currently waiting for a resolution from the vendor.
We are currently investigating this issue.
Report: "Giving Performance Degraded"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.