Historical record of incidents for Ticketmatic
Report: "serious disruption"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Problem with static assets"
Last updateThis incident has been resolved.
The cause for the problem is identified as a malfunctioning DNS connection, resulting in problems connecting to the interal filestore. We applied a fix and we are monitoring the situation.
We get reports of problems with downloading static assets like images and tickets. We are currently investigating this issue.
Report: "Payment problems due to incompatibility after upgrade"
Last updateWe apologize for the inconvenience caused by this disruption. We recognize the serious impact this may have had and we have addressed the issue with the utmost seriousness. The disruption was due to an upgrade by the payment provider, which led to an unexpected compatibility issue with the Ticketmatic payment handling system. As a result, certain payment methods became unavailable or failed, affecting approximately 20% of our customers. Our engineers responded immediately to the situation. The necessary modifications to the payment setups were successfully completed by 19:13.
Report: "Service problems due to high load"
Last updateOur apologies for the inconvenience that was caused by this disruption. We understand that this can have serious impact on your organisation and we take this very seriously. Due to an exceptional large peak as a consequence of 14 simultaneous peak sales at different Ticketmatic customers, an excessive load has occurred on the Ticketmatic systems. This overload has triggered safety measures which have in turn lead to Ticketmatic being temporarily unavailable or only after long response times. No data has been impacted. The Ticketmatic engineers have immediately taken action and have brought the situation back under control after 26 minutes. In order to avoid a similar scenario in the future, structural measures will be taken to improve defences against new occurrences of this scenario.
This incident has been resolved.
The load is under control and we are monitoring the situation.
We managed to bring the load on the system under control. We will now gradually increase the rate limiters to normal levels.
We are continuing to investigate this issue.
We are experiencing service problems due to very high load. We are investigating the issue.
Report: "Order crashes"
Last updateAfter deploy of a new version, we noticed crashes in the Order app. We quickly reverted to the previous version, everything should be fully functional again.
Report: "Issues with Mollie PSP payments"
Last updateMollie reports the incident as resolved
PSP Mollie currently has problems on its platform that causes problems with processing payments. More info and status updates on their status page: https://status.mollie.com/
Report: "Temporary issue during deployment"
Last updateThere was a small issue during a deployment around 13:50 CET that might have caused some crashes. It was resolved after a few minutes.
Report: "Web errors"
Last updateThis incident has been resolved.
We have resolved the issue, error levels are back to normal.
We see elevated error rates and are investigating
Report: "Ticket generation issues"
Last updateThe problems at AWS are resolved and ticket generation is fully functional again. Admins may still receive some error messages by e-mail that were delayed in the queue, these may be ignored.
AWS confirms that they are currently experiencing issues that are effecting EC2 and S3 in the eu-west-1 region. They are busy investigating this issue.
We are continuing to work on a fix for this issue.
Due to network problems at our hosting provider AWS, there are issues with generating tickets. We are looking for a solution.
Report: "Possible service disruption"
Last updateWe have identified the root cause of the outage. Our internal caching service experienced problems and an automatic failover was triggered to restore functionality. Failover occured quickly and without problems. But due to a bad endpoint configuration, our system did not use the new service and could not start up. The configuration is now fixed, future failovers will not cause this problem anymore.
The service seems fully operational again, we keep following up
An internal service failed over and was put in read-only mode, we are restoring the service now.
There is a possible service disruption causing part of the internal api calls to fail. Investigating.
Report: "Possible disruption of credit card payments"
Last updateThis incident has been resolved.
Credit card payments should be functional again. As displayed on Mollie status page: "We've identified the problem and are seeing no more failures on credit card payments. We are monitoring the situation and will resolve this incident once we're certain it will not reoccur. "
Payment processor Mollie indicates possible problems with credit card payment processing since 15:36. You can monitor https://status.mollie.com/ for up to date information
Report: "Server problem: ticketmatic backoffice unavailable"
Last updateThis incident has been resolved.
All services back online.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Printclient issues"
Last updateThe fix is rolled out and printing using the printclient should work again. Refresh the application if necessary.
We found the origin of the problem and have a fix ready. We will roll this out soon.
We are getting reports on printclient issues and are investigating. In the mean time, if you cannot print using the printclient, use order preview as a workaround to print tickets.
Report: "Database maintenance"
Last updateThis incident has been resolved.
We are performing scheduled database maintenance that may cause downtime.
Report: "Faulty deployment"
Last updateThis incident has been resolved.
Everything should be back to normal and all service should be restored.
We identified the cause of the problem and are working on a fix
Everything should be back to normal and all service should be restored. We're keeping a close eye on things.
A faulty deployment is causing requests to time out. We are actively investigating the problem.
Report: "Contact detail problem"
Last updateThis incident has been resolved.
The problem is fixed!
We are experiencing problems on the contact detail page. A fix is being rolled out now and will be available very soon.
Report: "Error when creating new event and selling seated tickets in backoffice"
Last updateThis incident has been resolved.
A fix is deployed that solves these problems. If necessary, refresh the application.
The problem with selling tickets from the Backoffice only occur when you use Dutch as interface language. While we are working on a final fix, you can use this workaround to be able to sell tickets through the Backoffice: - as user go to 'Mijn Ticketmatic ID' pagina (on top right, under your user name) - change your interface language to 'English' and Save - refresh the application (close browser and reopen application)
There are also related problems with selling tickets for seated events from the Backoffice. Websales is not affected. We are working on a fix.
We see errors when creating new events from the Backoffice. The reason is related to a problem with the latest deployment of the app this morning. We are now rolling out a new version that should fix this.
Report: "504 errors while using backoffice"
Last updateThis incident is resolved.
The problem seems to originate from a networking problem on one of the underlying worker nodes. We restarted the faulty node and this seems to resolve the problem. We keep monitoring for now.
We are investigating reports of users getting 504 errors while using the application.
Report: "Web sales down for some accounts"
Last updateThis incident has been resolved.
Rollback is completed, functionality is restored. We keep monitoring this.
Due to a failed deployment, web sales is down for some accounts. We are rolling back to the previous version.
Report: "[TM2 only] Backoffice login problem"
Last updateThis incident has been resolved.
We finished setting up new temporary infrastructure and backoffice login is again possible for all accounts. Make sure to use the launcher to login. We will keep on working to restore the original servers and bring these back in operation once this can be safely done.
7 Ticketmatic Windows servers are under heavy attack, making the Backoffice unusable. We are both mitigating the attack and setting up new infrastructure. Be sure to use Ticketmatic Launcher to login. Databases and webservers are unaffected. Online sales is unaffected.
There are problems logging in to the backoffice. We are currently investigating. Online ticket sales is unaffected.
Report: "Increased error rates"
Last updateEverything is back to normal. This disruption was caused by a recent change in our web socket connection handling code. A combination of factors led to a serious overload on one of our services, which in turn caused some connections to time out. We'll be updating our automated QA processes to ensure that this won't happen again in the future.
A fix has been deployed, service should no longer be disrupted. We're keeping a close eye on things but there shouldn't be any impact anymore.
We're observing increased error rates due to a disruption in one of our services. A small minority of our users will be impacted and encounter an error dialog. A simple reload of the web page should fix things and allow you to continue. A fix is being worked on, apologies for the inconvenience.
Report: "datacenter disruption: some accounts unavailable"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Service disruption"
Last updateThis incident has been resolved.
The issue is resolved for now, we keep monitoring
The cause for the disruption is identified and caused by excessive amount of database connections. The disruption is limited to a single account
We experience a service disruption at this moment. We are further investigating
Report: "temporary malfunction of one webinstance"
Last updatemight have impacted some ticketmatic accounts
Report: "possible login problems with iOS scanning app"
Last updatefixed version of iOS scanning app available in App Store
The latest release of the iOS scanning app can malfunction, preventing login. A fixed version is being published on the App Store. Previous versions work normally.
Report: "some servers unavailable"
Last updaterouter disruption was cause, problem solved
We are currently investigating this issue.
Report: "some servers unreachable"
Last updaterouter problem solved, all accounts reachable
several servers cannot be reached, affecting some customers
Report: "Accounts partially unavailable"
Last updateOur upstream provider confirms the cause was a problem with router infrastructure in the datacenter. All systems are operational for now but we keep monitoring the situation.
Everything looks ok now.
Some accounts were unavailable for a short time. The problem was situated at a network provider upstream. Everything seems back to normal now, we keep on monitoring the situation