BetterNow

Is BetterNow Down Right Now? Check if there is a current outage ongoing.

BetterNow is currently Operational

Last checked from BetterNow's official status page

Historical record of incidents for BetterNow

Report: "Increased error rate"

Last update
investigating

We are currently investigating this issue.

Report: "Errors in fundraisers API endpoint"

Last update
resolved

An error in a deploy caused an increased error rate for API requests to the fundraisers endpoint. The deploy has been rolled back as of 15:22 CEST

Report: "Increase error rate"

Last update
resolved

This incident has been resolved.

monitoring

Services are recovered, we continue to monitor traffic levels and error levels

identified

Autoscaling has caught up to the increased traffic and it looks like services are recovering

investigating

We are currently investigating this issue.

Report: "Increased error rate"

Last update
resolved

This incident has been resolved.

monitoring

Our autoscaling has caught up to a spike in requests and the error rate has normalized. We continue to monitor the health of the system.

investigating

We are currently investigating this issue.

Report: "Errors with Swish payments"

Last update
resolved

Swish has resolved their incident. We continue to monitor the error rate for Swish payments. https://status.swish.nu/incidents/1fwn9ny2lsh3

investigating

We see an elevated error rate for Swish payments for many merchants. More information can be found on the Swish status page at https://status.swish.nu/incidents/1fwn9ny2lsh3

investigating

We are currently investigating this issue.

identified

The issue has been identified and a fix is being implemented.

Report: "Downtime from 19:53-20:42 CET, October 28th"

Last update
resolved

Our monitoring shows a period of downtime between 19:53 and 20:42 CET yesterday, October 28th. This was due to an issue with our upstream provider, the details are here: https://status.heroku.com/incidents/2721

Report: "Increased response times"

Last update
resolved

This incident has been resolved.

monitoring

We are seeing response times improve in all regions.

identified

This looks to be a repeat of yesterday's incident: https://status.heroku.com/incidents/2685

investigating

For now it seems our monitoring from North America and South America are seeing the worst response times, while Europe and Asia-Pacific are only slightly elevated.

investigating

We're seeing longer response times than normal from some regions.

Report: "Increased response times"

Last update
resolved

This incident has been resolved.

identified

Our upstream provider Heroku is reporting problems. Details are at https://status.heroku.com/incidents/2684

investigating

We continue to see excessive response times and we're still trying to determine the cause.

investigating

We're seeing much longer response times than normal from some regions.

Report: "Increased error rate"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are currently investigating this issue.

Report: "Increased error rate"

Last update
resolved

This incident has been resolved.

monitoring

The error rate has returned to normal levels. We continue to monitor the system.

investigating

We are currently investigating this issue.

Report: "Clearhaus payment issues"

Last update
resolved

This incident has been resolved.

monitoring

An update from Clearhaus: "We have seen a significant increase in approval rates and the connectivity is now back to normal. Due to the severity of this issue and the previous sudden change in approval rate we are still actively monitoring this situation." We will continue to monitor this as well.

identified

From Clearhaus: "Our upstream provider has confirmed and is aware of a connectivity issue. Currently almost all authorizations, voids and Mastercard credits are impacted"

identified

Clearhaus is reporting issues with an upstream provider https://status.clearhaus.com/incidents/hp85pwr0m3yp

Report: "Increased error rate for Swish payments"

Last update
resolved

This incident has been resolved.

monitoring

We are seeing a reduction in error rates and will continue to monitor the system and Swish's incident until it is resolved.

investigating

Swish has an incident open for this at https://status.swish.nu/incidents/6xqpkqcsy8bb

investigating

We are currently investigating this issue.

Report: "Increased Error rate"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are currently investigating this issue.

Report: "Increased Error count"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are currently investigating this issue.

Report: "Increased Error Rate"

Last update
resolved

For a short period after 15:24 we saw an increase rate of errors on some pages. We have rolled back the deploy causing the issue and the error rate has returned to normal.

Report: "Dashboard/administration errors"

Last update
resolved

This incident has been resolved.

identified

The issue has been identified and a fix is being implemented.

investigating

We are currently investigating this issue.

Report: "Errors for logged-in charity users"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

Logged-in charity users are getting errors as they try to use the system. Non-charity users (fundraisers and donors) are not affected. A fix is being deployed.

identified

The issue has been identified and a fix is being implemented.

Report: "Timeout errors"

Last update
resolved

This incident has been resolved.

monitoring

We saw a brief period of network timeout errors with the BetterNow platform and api. For now it looks like a fix has been implemented by our upstream providers and the services have recovered. We will continue to monitor the services.

investigating

We are continuing to investigate this issue.

investigating

We are currently investigating this issue.

Report: "Errors loading images"

Last update
resolved

This incident has been resolved.

monitoring

Our upstream providers have applied a fix. We continue to monitor for errors.

identified

Our upstream providers have confirmed that there are intermittent problems loading the images from some European countries. We are working with them on a resolution to this issue.

investigating

We are currently investigating this issue. Our monitoring shows intermittent errors loading images. We are in contact with our upstream providers regarding this issue.

Report: "Donors received an error instead of being sent to pay their donations"

Last update
resolved

Between 16:04:05 and 18:35:39 on March 15th donors received an error instead of being sent to our payment partners to pay their donations. This was due to an error in code we deployed and was compounded by a missing alert when the specific error was triggered. At 18:35 we rolled back the bad deploy and added the missing alert to our monitoring system.

Report: "Connection errors"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are currently investigating this issue.

Report: "Problems with MobilePay online"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

Our payment provider is experiencing problems with MobilePay online transactions. Their status is here: https://status.quickpay.net/incidents/414 This does not affect card transactions or Swish or Vipps

Report: "Some transactions processed twice"

Last update
resolved

We have notified the affected customers by email. Any additional questions should be directed to orgsupport@betternow.org

monitoring

We now have a list of the 49 affected transactions. Clearhaus writes "Corrections will be sent starting Monday, and while you may see the effect already Tuesday, it could take several days also depending on your bank. You should see the correction no later than October 21st."

identified

One of our acquiring partners (Clearhaus) mistakenly processed transactions twice. All transactions (captures, refunds and credits) processed between 2021-10-06 19:00:00 UTC and 2021-10-07 18:59:59 UTC were affected for both Visa and Mastercard, with the exception of Visa credits. Clearhaus is working with Visa and MasterCard to automatically reverse the transactions during the night between Monday and Tuesday. Therefore, you do not need to manually refund any double donations. Clearhaus' incident report is here: https://status.clearhaus.com/incidents/t8z9l54g6tbw

Report: "Errors connecting to payment provider (QuickPay)"

Last update
resolved

This incident has been resolved.

monitoring

Quickpay's incident is here: https://status.quickpay.net/incidents/409 They are currently reporting that the issue is fixed, we will continue to monitor for errors

investigating

We are currently investigating this issue.

Report: "Intermittent Payment Errors"

Last update
resolved

This incident has been resolved.

monitoring

QuickPay reports that their systems are operational again. We'll continue to monitor for problems.

identified

Some QuickPay payment requests are refused with errors. https://status.quickpay.net/incidents/407 is their status issue for the problem

Report: "Connection errors"

Last update
resolved

This incident has been resolved.

monitoring

We are continuing to monitor for further issues. You can see more information from our upstream provider at https://status.heroku.com/incidents/2359

monitoring

A fix has been implemented and we are monitoring the results.

identified

Unfortunately we are continuing to see routing errors.

monitoring

A fix has been implemented and we are monitoring the results.

identified

Our upstream provider is not currently routing requests to our apps. We are working with them on a fix.

investigating

We are continuing to investigate this issue.

investigating

We are currently investigating this issue.

Report: "Problems with second-factor authentication"

Last update
resolved

This incident has been resolved.

investigating

We are currently investigating this issue. It seems as though some administrator accounts cannot provide 2 factor authentication. This affects usage of the dashboard, fundraisers, donations and payments are not affected.

Report: "SWISH payment failures"

Last update
resolved

This incident has been resolved.

monitoring

The error rate has returned to normal levels and we're monitoring the situation

identified

We see intermittent connection errors between our payment service provider and Swish for some Swish payments in Sweden. Card payments and payments in other countries are not affected.

investigating

We are currently investigating this issue.

Report: "Donation creation errors"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

Some donations via OnlineFundraising are refused with a system error. This is only for charities using OnlineFundraising for payments, not for those using QuickPay/Clearhaus.

Report: "connection errors"

Last update
resolved

This incident has been resolved.

monitoring

The connection errors seem to have subsided and we are monitoring the systems.

investigating

We are currently investigating this issue.

Report: "Slow image loading"

Last update
resolved

This incident has been resolved.

investigating

We are currently investigating this issue.

Report: "Increased donation error rate"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are currently investigating this issue.

Report: "Datastore Failure & Replacement"

Last update
resolved

Our secondary datastore failed and its standby replica was promoted to production. There were circa 6 minutes of downtime starting at 02:58 CEST.

Report: "Increased error rate"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We are currently investigating this issue.

Report: "Invalid SSL certificate on api.betternow.org"

Last update
resolved

Due to a configuration error our api.betternow.org endpoint had an invalid SSL certificate between 2020-03-17 16:00 CET and 2020-03-18 09:35 CET.

Report: "Partial outage"

Last update
resolved

Our provider has recovered and all outgoing emails have been delivered.

monitoring

We have implemented a change so that other background job processing can complete, and the queues are now clear of other jobs. Sending emails, such as for donation receipts and for notifying fundraisers of new donations, will still be delayed until our service provider recovers.

investigating

One of our service providers has an outage that is affecting transactional email sending along with other work processed by our background job queues.

investigating

We are currently investigating this issue.

Report: "Retroactive: Platform & API availability"

Last update
resolved

A bad deploy resulted in a 4 minute outage for the API and BetterNow Platform. The deploy has been rolled back and the sites are operational again.

Report: "Continuing Connection Errors"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We are currently investigating this issue.

Report: "Connection Errors"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been put in place and we're monitoring the results.

investigating

We are currently investigating this issue.

Report: "Elevated Errors"

Last update
resolved

A bad deploy caused a short period of elevated errors. We've rolled back and deployed a fix.

Report: "Connection errors"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are currently investigating this issue.

Report: "Connection errors"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

Our monitoring systems have alerted us to connection errors. We're investigating.

Report: "Increased error rate"

Last update
resolved

This incident has been resolved.

monitoring

We made some configuration changes and the error rate has now returned to normal levels. We'll continue closely monitoring the affected services.

investigating

Our monitoring systems have alerted due to an increased number of errors. We are investigating.

Report: "Slow Requests"

Last update
resolved

This incident has been resolved.

monitoring

We have identified the issue and put a fix in place. We'll continue monitoring the affected services.

investigating

Our monitoring systems have alerted us to an increase in the number of slow requests. We are investigating the issue.

Report: "Intermittent Server Errors"

Last update
resolved

This incident has been resolved.

monitoring

Error rates are returning to normal. We'll continue to monitor the heath of the system.

investigating

We're seeing an increased percentage of failed requests and are investigating

Report: "Increased failures due to upstream routing errors"

Last update
resolved

We're resolving this issue as error rates have been normal all day and the upstream incident has been resolved.

monitoring

Our last failure was at Apr 05 15:05:15 CEST. We will keep monitoring the failure rate as long the upstream Heroku incident is not resolved.

monitoring

We are seeing the error rate return to normal, and continue monitoring the progress on the Heroku issue.

identified

Unfortunately the error rate is increasing again. More information about this issue can be found on Heroku's incident page: https://status.heroku.com/incidents/1091

monitoring

We're seeing a much lower error rate, even though this incident is ongoing on Heroku's side. We will continue to monitor the error rate closely.

identified

We are unfortunately affected by the Heroku partial outage. Our monitoring shows up to 30% of inbound requests are failing.

Report: "Cloudbleed Response (Retroactive)"

Last update
resolved

Out of an abundance of caution, we have revoked all current user sessions in response to the "CloudBleed" security issue (https://en.wikipedia.org/wiki/Cloudbleed). This means you will need to log in again the next time you edit your Fundraiser or Charity. BetterNow has never used CloudFlare for any services, but as the impact of the issue is not yet fully known, and at least one of our vendors uses CloudFlare, we decided to revoke all current sessions and rotate the credentials we use to communicate with the affected vendors. It is important to note that our investigation shows no evidence that any breach of our service took place, but again we are acting out of an abundance of caution. "Better safe than sorry."

Report: "Errors connecting to payment processor"

Last update
resolved

This incident has been resolved.

identified

Some payment requests are failing. A fix has been deployed and will be live within the next few minutes.

Report: "Database failure"

Last update
resolved

Our monitoring systems are showing that all components are stable and online. The failover to the standby database succeeded and a new replica has been brought online.

monitoring

The platform and API are operational again, and we're keeping a close eye on the system.

identified

We are switching to our standby database replica and expect to bring the affected components online in a few minutes.

investigating

We are investigating problems due to a database failure

Report: "Connectivity problems"

Last update
resolved

We've seen normal levels of traffic for the last hour and are ready to enjoy the weekend sun. :)

monitoring

Connectivity seems to be restored by our service provider. We will continue to monitor the system closely before marking this incident as resolved.

identified

Our provider is continuing to work on resolving the issue. We will update this page with more information as soon as we have it, and latest 16:30 CEST

identified

Our upstream provider has confirmed the issue and is investigating. We will update this page with more information as soon as we know more, and latest 16:00 CEST.

investigating

Our monitoring systems have notified us of connection errors and we're investigating.

Report: "Connectivity errors"

Last update
resolved

The site is healthy again.

monitoring

We've deployed a fix and are continuing to monitor the health of the site.

investigating

Our monitoring has detected an increased number of errors. We're investigating.