Historical record of incidents for Maxio
Report: "Subscription Next Billing Date Error in UI for Some Subscriptions"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating an error for the Next Billing date that is occurring for some Advanced Billing subscriptions in the UI. This error also prevents Proforma Previews from being generated for impacted subscriptions.
Report: "Maxio Platform 503 Error Access Issues"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
During a scheduled release that was not expected to cause downtime, an issue occurred that is currently causing certain Maxio platform accounts to be temporarily unavailable. Impacted users are receiving a 503 error when attempting to access their accounts. Our engineering team is actively investigating the root cause and working to restore full functionality as quickly as possible.
Report: "Advanced Billing - API Service Interruption on "Create Usage" Endpoint"
Last updateService has been fully restored, but any usage submissions that received a 500 error during the affected periods will need to be resent. Please reach out to our Support team should you require logs of specific usage API calls that were impacted as we are happy to provide these. We apologize for this disruption. We are implementing improvements to our systems to prevent similar issues in the future. Thank you for your patience and understanding.
A fix has been implemented and we are monitoring the results.
We experienced an issue with the Advanced Billing "Create Usage" API endpoint between the 2025-03-31 17:49:38 -0400 and 2025-03-31 18:22:21 -0400, and 2025-03-31 18:42:22 -0400 - 2025-03-31 18:49:36 -0400 where timeouts resulted in 500 errors. This was caused by an unexpected load increase from a separate service, which impacted the performance of the Usage endpoint. The underlying issue has been identified and system performance has stabilized. A full resolution is in progress.
Report: "Webhook Delivery Delays"
Last updateIncident Dates: Start: 03/10/2025 End: 03/11/2025 Root Cause: A rare communication issue between a background server and an internal service caused disruptions in webhook processing. Impact: Some webhook events were not delivered as expected, potentially leading to delays or inconsistencies for systems relying solely on those notifications. Resolution & Mitigation Steps: The issue was identified and resolved. Monitoring has been improved to catch similar issues earlier. Additional internal safeguards have been put in place to prevent recurrence. Actions Needed: We will contact affected customers to inform them of the issue. Customers will be given the option to have any missing webhook events resent. Best Practices Reminder (link below): Webhooks are asynchronous. We strive for timely delivery, but we DO NOT recommend relying on them for time-sensitive events. Do not block users based on webhook responses. The recommended approach is to verify subscription status using the Subscriptions API. https://developers.maxio.com/http/getting-started/about-the-api/webhooks#best-practices
Report: "Maxio Payments: Tokenization Processing Issue"
Last updateIncident Overview: A recent configuration change introduced an unintended impact on payment processing, causing payment profile tokenization failures for a subset of customers. The issue was quickly identified and resolved, restoring normal operations. Root Cause: An internal adjustment altered the way certain payment profile tokenization requests were handled, affecting the communication between systems and leading to failures in storing these payment profiles. Resolution & Next Steps: - The adjustment has been reverted to restore expected functionality. - Additional monitoring has been implemented to detect similar issues earlier. - A review process is in place to improve testing and validation of future changes. Potential Actions Needed: We recommend that customers whose signups or payment profile updates failed retry their transactions. If any issues persist, they should reach out to support for further assistance.
Report: "Brief Service Disruption – February 17, 2025"
Last updateIncident Summary: On February 17, 2025, from 10:31:57 AM to 10:32:13 AM EST, A system change caused an unexpected service disruption of 16 seconds. During this time, some users experienced temporary unavailability via the API and UI, with a limited number of lost connections. Resolution: The issue was quickly identified and resolved. The system is now fully operational. Next Steps: Customers who encountered a 500 error during this period should retry their API requests. We are reviewing our processes to prevent similar occurrences in the future.
Report: "Maxio Platform Brief Outage"
Last updateWe experienced a brief outage due to a configuration change aimed at improving system functionality. The issue was identified and promptly resolved by rolling back the change, restoring full availability. The downtime lasted approximately 10 minutes. We apologize for any inconvenience caused and appreciate your patience.
Report: "Maxio Platform Outage: 503 Errors Observed"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Advanced Billing - Public Signup Page reCAPTCHA Error"
Last updateThe underlying problem has been identified and a fix was implemented. This issue has been resolved and reCAPTCHA for Public Signups is now functioning correctly.
We are continuing to investigate this issue.
We are currently investigating an error on Advanced Billing Public Signup Pages pertaining to reCAPTCHA that is preventing signups.
Report: "Advanced Billing: Service Disruption"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Unviewable invoices retrieved from the API or accessed via the Billing Portal."
Last updateThis incident has been resolved.
As of 16:40 UTC, invoice display functionality has been restored, and the root cause has been resolved. We are actively monitoring the situation and working to ensure that all invoices affected by this issue are viewable.
The issue has been identified and it is being fixed.
Report: "Pay Invoice Online Button Not Rendering on Invoice Public URL's"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We are currently investigating the "Pay Invoice Online" button not rendering properly on Advanced Billing Invoice Public URL's, preventing the invoices from being paid online.
Report: "Minor Customer Creation Degradation"
Last updateMaxio Advanced Billing experienced some performance degradation, which prevented new customer/subscription creation and returned 502 Gateway errors. Our engineers identified and resolved this issue quickly. If you experienced any issues or have any further questions, feel free to reach out to Maxio Support.
Report: "Advance Billing Webhook Degradation - Incorrect Timezone Format for Subset of Webhooks"
Last updateOn August 30th, the Advanced Billing system encountered a small period of degradation for a subset of webhooks that resulted in a temporary formatting change in webhook payloads that could have caused webhooks to fail parsing. The change was temporary, where the specific data changed was timezone format, for example, the format "Fri, 30 Aug 2024 11:03:04.000000000 EDT -04:00", changed to "2024-08-30T14:03:22Z". This change was quickly reverted, and we have proactively corrected formatting within impacted webhooks in order to allow webhooks that may have failed with the incorrect format to be replayed. No action is necessary unless you wish to re-send these events due to parsing errors occurring within your server. If your endpoints parse time and date for a specific format, then impacted webhooks may have initially failed with a 500 error. The now corrected webhooks are accessible via the Advance billing UI and may be replayed on your site if needed by: - Navigating to your site’s webhooks panel via Tools → Webhooks - Locate the missed event and select the box toggle associated with it - Select the “Resend Selected Webhooks” option to bulk resend selected webhook events These webhook events may be bulk resent via the API as outlined within our developer documentation below: https://developers.maxio.com/http/advanced-billing-api/api-endpoints/webhooks/replay-webhooks We recommend checking all webhook connections under Config→ Settings → Webhooks to ensure all endpoints are unpaused with the correct expected events selected. Additionally, we advise reviewing events for your site to see if any might have failed to properly replay the corresponding webhook or retrieve the data via API. We sincerely apologize for this inconvenience, and any disruption to your applications. Should you have any questions or concerns, please contact our Support Team for assistance.
Report: "Issues with Integration syncs"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
We are experiencing slowness with syncing with our Integrations to 3rd Party Services. The issue has been identified and we working on getting the syncs fully operational again.
Report: "Public Signup Page Regression Resulting in Failed Signups"
Last updateThis incident has been resolved. Public Signup pages are now fully functional.
We are continuing to monitor for any further issues.
We have reverted the release that introduced the regression and are monitoring results.
We are investigating a regression in the Classic (v1) Public Page Layouts introduced by a recent release. This regression is preventing signups from succeeding for Public Signup Pages using the Classic (v1) Layout, as well as card storage/updates via Self-Service Pages as it uses the Classic (v1) Layout. We currently are working to roll back the release that introduced this regression.
Report: "Advanced Billing Outage"
Last updateOn July 1, Maxio’s Advanced Billing module encountered a temporary issue due to an unexpected surge in load. This caused a small number of subscription renewals to fail. The disruption was traced to a specific part of our infrastructure. Upon identification, our team promptly resolved the issue, and full service has been restored. We have initiated a thorough post-incident review. This review will include enhancements to our monitoring systems, reevaluation of certain infrastructure configurations, and other internal process improvements to prevent future occurrences. We are currently addressing all affected subscriptions to ensure their renewals are processed correctly. You do not need to take any action unless our Support team reaches out to you for further assistance. We appreciate your patience and apologize for any inconvenience this incident may have caused. Thank you for your understanding as we work to fully resolve this matter.
We are happy to report that the application is back to operating at full capacity. This incident is resolved. During the full outage period, which occurred for a period of time between 12:30 PM EST and 1:01 PM EST, subscriptions attempting to process likely failed and will have encountered a processing error. Our team will remediate any processing errors that occurred for any affected subscriptions, and once the processing errors are cleared, the subscriptions will resume processing as normal and bill for their renewal. No action is necessary to resolve these errors on your end unless our Sr. Product Support Consultant team reaches out directly. API calls, and signups/subscription creation attempts for sites during the full outage period time frame may have failed, and will need to be retried. Our team is happy to provide API logs from this time period for your site(s) if needed. Thank you for your patience as our team worked to resolve this matter, and our sincere apologies for the disruption. Should you have any questions or concerns, please contact our Support Team for assistance.
We have re-enabled background jobs and processing for subscriptions should begin to resume, though delays are still expected. Exports have been paused for the time being, and our engineers are continuing to investigate some slowness within the application. We will provide more information as soon as we have another update to share.
We are continuing to monitor for any further issues.
While the application is back up, we are not yet fully operational. Background jobs are currently paused. Webhooks, emails, and processing are delayed. We are actively working to bring the application back to full operational status.
A fix has been implemented and the application is back up. We are closely monitoring and will provide another update as soon as possible.
We are currently investigating an outage with the Advanced Billing platform
Report: "Advanced Billing - Billing Portal Outage"
Last updateWe have confirmed that the Billing Portal is back to full functionality. This incident is resolved.
Our engineers have identified the underlying problem and corrected it. The Billing Portal is now loading correctly and we are currently monitoring to ensure no further problems arise.
We are currently investigating an outage with the Billing Portal and will provide another update as soon as we have more information to share.
Report: "Maxio UI is currently experiencing issues"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
We have implemented a solution and are currently monitoring the results.
We are currently investigating the issue.
Report: "App Icons Not Displaying"
Last updateThis incident has been resolved.
Icons appear to be displaying again. We are currently monitoring to ensure everything is displaying correctly.
We have confirmed that this matter is due to our third-party icon provider experiencing issues on their end, and that they are currently working on a resolution.
We are continuing to investigate this issue. Some user actions within the UI are considerably hampered by the icons missing. Automated processes and API requests are fully operational and unaffected.
We are investigating cases of icons not displaying within the application. This appears to be due to our third-party icon provider experiencing issues on their end. We will provide another update as soon as possible.
Report: "Email Sending Delays in Advanced Billing"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
Some Advanced Billing emails are experiencing delays in sending. While emails are sending correctly, there may be delays in sending and delivery. Our engineers are currently investigating.
Report: "500 Errors and potential slowness while using Advanced Billing UI"
Last updateThis incident has been resolved.
A fix has been implemented and we are now monitoring the results.
The issue is Identified and a fix is being implemented.
Report: "Advanced Billing - Period of Processing Degradation"
Last updateOn April 1, 2024, the Advanced Billing system encountered a period of degradation that affected a subset of subscriptions attempting to process during these periods of time. This period was caused by server connectivity problems, with the main result being that subscriptions encountered processing errors. These errors halt the processing of the subscription in order for our team to manually review and ensure the renewal is handled correctly when the processing error is cleared. The window of these degradation period is below for reference: - 04/01/2024 2:01 PM -0500 - 04/01/2024 2:06 PM -0500 We are happy to report that following this period of system degradation, the Advanced Billing system is fully operational. Our engineers are actively working to remediate any processing errors that occurred for subscriptions as a result of these incidents. Once the processing errors are cleared, the subscriptions will resume processing as normal and bill for their renewal. No action is necessary to resolve these errors on your end unless our Sr. Product Support Consultant team reaches out directly. Should you have any questions or concerns, please contact our Support Team for assistance.
Report: "Advanced Billing - Period of Processing Degradation"
Last updateOn March 28, 2024, the Advanced Billing system encountered a period of degradation that affected a subset of subscriptions attempting to process during this periods of time. This degradation was caused by server connectivity problems some of which were due to degradation on the part of third-party vendors. The main result of this period of degradation were that subscriptions encountered processing errors. These errors halt the processing of the subscription in order for our team to manually review and ensure the renewal is handled correctly when the processing error is cleared. This also resulted in webhooks not firing for some subscriptions. Our Support team will reach out directly should your site have been affected by webhooks not triggering and provide next steps where necessary. The window of the degradation period is below for reference: - 3/28/2024 6:05:32.030 AM -0500 - 3/28/2024 10:45:00.613 AM -0500 We are happy to report that following this period of degradation, the Advanced Billing system is fully operational. Our engineers are actively working to remediate any processing errors that occurred for subscriptions as a result of these incidents. Once the processing errors are cleared, the subscriptions will resume processing as normal and bill for their renewal. No action is necessary to resolve these errors on your end unless our Sr. Product Support Consultant team reaches out directly. Should you have any questions or concerns, please contact our Support Team for assistance.
Report: "Webhook Sending Slow"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented. Webhooks are sending but there may still be slight delays.
We are currently investigating slow sending of Webhooks within Advanced Billing.
Report: "Subscription Performance Degradation"
Last updateOn [3/09 from 18:00 UTC to 20:35 UTC](https://maxio.statuspage.io/incidents/gwwsv4t94ygz) and 3/10/2024 from 15:15 UTC to 17:08 UTC, the Advanced Billing system encountered a system-wide performance degradation that primarily affected signups, renewals, product migrations and reactivations. Component allocations and refunds may also have failed during these periods. This degradation was the result of a system background process that encountered a rare data condition that triggered an unindexed query to run. This very slow query caused the background process to consume system resources to the level that it adversely impacted other system processes, resulting in the system-wide performance degradation that occurred. During these periods of system degradation, subscription creation via Public Signup Pages, API, and UI channels may have failed. Product migrations attempted during this timeframe also may have encountered failures, and many subscription renewals encountered processing errors, pausing their renewal for review and remediation by our team. We recommend checking any internal logs for systems integrated with our platform via API for failed subscription creation and migration or other failed events in order to reattempt where appropriate. Our team is happy to enable audit log exports for your account to help pinpoint events affected by these incidents as well. Our engineers are currently working to remediate any processing errors that occurred for subscriptions as a result of these incidents. Once the processing errors are cleared, the subscriptions will resume processing as normal and bill for their renewal. No action is necessary to resolve these errors on your end unless our Sr. Product Support Consultant team reaches out directly requesting further insight on specific subscriptions. We are happy to report that the Advanced Billing system is now fully operational, and the underlying cause has been identified and addressed. Proper steps including improved code, testing, and monitoring are being taken by our developers to ensure this does not occur again. Thank you for your patience as our developers worked to identify and resolve this matter, and our sincere apologies for the disruption caused by these incidents.
This incident has been resolved.
The issue has been identified and addressed. Subscription signups and creation appear to be working correctly and we are monitoring to ensure continued success.
We are continuing to investigate this issue.
We are receiving reports of failed signups/subscription creation within the Advanced Billing platform. Our team is currently investigating and will provide more information as soon as possible.
Report: "Partial Platform Degradation"
Last updateThis Incident has been resolved.
Our team has identified the issue and has implemented a fix. We will now be monitoring the results.
We are currently investigating the issue
Report: "CSV Export Processing Delays"
Last updateOur engineers have confirmed that exports are now processing normally. This incident has been resolved.
We are continuing to investigate this issue.
We are currently investigating slow CSV Export processing within the Advanced Billing platform.
Report: "Partial Outage"
Last updateThis issue has been resolved. All Maxio instances are now operational.
The issue has been identified and a fix is being implemented.
We are urgently investigating a partial outage with the Maxio Platform. Some accounts may be experiencing 503 errors when trying to access the platform. We will provide more information as soon as possible.
Report: "Advanced Billing - Webhooks delayed"
Last updateThis incident has been resolved so that webhook ID's are no longer negative and webhook delivery has been re-enabled. The first appearance of the issue was on Sun, 07 Jan 2024, at 12:32:16 EST -05:00. After further review, our engineers confirmed that Webhook delivery was not affected by this matter, and all data in the webhook payloads are valid. However, some merchants who rely on webhook IDs being positive numbers may have been affected. Please reach out to our Support team should you need our engineers to re-send affected webhooks with positive webhook ID's.
We've noticed that the Advanced Billing application is sending out webhooks with negative ids. We've paused webhook delivery for now and we're investigating the issue.
Report: "Advanced Billing - Data Exports Not Completing"
Last updateThis incident has been resolved and CSV data exports are now functioning correctly. After further review, our engineers determined that in addition to affecting CSV Data exports, the underlying problem caused a partial degradation affecting webhooks and API calls. Between roughly 02:49 PM CDT and 05:34 PM CDT this degradation may have caused some webhooks to fail, as well as some API calls. Any webhooks that failed were automatically retried by our system, but any API calls that failed during this timeframe will need to be retried.
A fix has been implemented and we are monitoring results.
We have identified the underlying cause of this problem and are implementing a solution.
We are currently investigating a problem causing CSV data exports to fail within the Advanced Billing platform.
Report: "Advanced Billing - Degraded System Performance"
Last updateAll the background jobs are up and running and the system is back to full processing. During this outage, which occurred for a period of time between 1:00 PM EST and 1:40 PM EST, subscription processing for sites was temporarily paused, resulting in delays in subscription renewals around that time frame for sites. Any API calls, and signups/subscription creation attempts for sites during that time frame would have failed, and will need to be retried. As background jobs were paused during resolution of this issue, delays in export jobs and other background jobs are expected. This issue is now fully resolved.
A fix has been implemented and background jobs are being ramped back up to full capacity. We are currently monitoring to ensure full resolution implementation is successful.
Subscription processing, which was temporarily paused while identifying the issue, has resumed. Performance is still degraded and most background jobs system-wide have been temporarily paused. Our developers are still working on fully resolving this matter.
We have identified the underlying cause of this issue. The system is back up and operational, but performance is still degraded. Our developers are working to fully resolve this matter.
We are urgently investigating an outage with the Advanced Billing system. We will provide more information as soon as possible.
Report: "Maxio Payments - Failed Transactions"
Last updateThe incident has been resolved.
Transactions are now processing as expected and we will update affected customers with remediation steps if needed.
We are currently investigating reports of a potential service interruption with one of our Maxio Payments partners. We apologize for any inconvenience and will post another update as soon as we learn more.
Report: "Usage API endpoint outage"
Last updateThis issue has been resolved and all subscription processing for affected sites has resumed.
Fix has been deployed. Subscription processing is being resumed for affected sites.
A fix has been implemented and we are monitoring results. Affected sites will have their subscription processing resumed once the fix results are confirmed as being successful.
Our team is continuing to work on a fix for this issue. Currently, affected sites still have their subscription processing temporarily blocked to ensure usage is not missed on subscription renewals. Our Support team is notifying affected merchants directly with more information.
We are continuing to work on a fix for this issue. In the interim, we have temporarily blocked subscription processing for sites affected by this issue to ensure usage is not missed in subscription renewals. Our Support team will be notifying affected merchants directly with more information.
The usage endpoint is returning 500 and 406 Not Acceptable errors. Recording usage in the UI is also affected as are subscription creation attempts that include metered usage. Issue has been identified and we're working on a fix.
Report: "Billing Portal"
Last updateThis incident has been resolved.
We are currently investigating reports of a potential service interruption with the billing portal. We apologize for any inconvenience and will post another update as soon as we learn more.
Report: "Advanced Billing - 2FA Issues"
Last updateThis incident has been resolved.
We are currently investigating reports of a potential service interruption with two factor authentication (2FA). We apologize for any inconvenience and will post another update as soon as we learn more.
Report: "Maxio Mandrill (Mailchimp) Issue"
Last updateThe Incident has been resolved and all emails sent will be delivered as Maxio works through the e-invoice email queue
We are currently experiencing degraded performance issues with E-invoicing and have identified the issue. Our team is currently working to restore normal performance levels.
We are continuing to investigate this issue.
We are currently investigating reports of a potential service interruption with E-invoicing. Email delivery may be affected at this time. We apologize for any inconvenience and will post another update as soon as we learn more.