Historical record of incidents for JustCall
Report: "Users Unable to Make and Receive Calls"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We've observed an increased rate of errors where users are experiencing difficulty in making & receiving calls.
Report: "JustCall app is loading slow"
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "JustCall app is loading slow"
Last updateWe experienced an unexpected increase in latency in one of our API services. This issue resulted in degraded performance for some of our users, causing inconvenience and disruptions. Users were temporarily re-routed from https://app.justcall.io to https://justcall.io After a thorough investigation, the root cause of the API latency was identified as increased database response time. The API latency issue has been successfully resolved, and normal operations have been restored. We sincerely apologize for any inconvenience this may have caused and appreciate your patience and understanding.
This incident has been resolved.
We are currently investigating this issue.
Report: "JustCall App loading slow for some customers"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
The web application is loading slow for some of the users. Our team is investigating it.
Report: "JustCall Dialer Not Loading for Some Users"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We've observed an increased rate of errors where users are trying to open the JustCall Dialer.
Report: "Users Unable to Receive Calls"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We've observed an increased rate of errors where users are experiencing difficulty in receiving incoming calls.
Report: "JustCall App loading slow for some customers"
Last updateThis incident has been resolved.
We are continuing to monitor to ensure a full recovery.
The team was able to address the issue. We will continue monitoring the service to ensure a full recovery.
The issues have been identified and the team is actively working to make the system stable and implement a fix.
We are continuing to investigate this issue.
High latency in loading the JustCall app was reported by our monitoring system.
Report: "Issue with Keap (fka Infusionsoft) integration"
Last updateOn April 26th at 7:00 PM (GMT), we discovered issues with contact syncing and logging of calls and SMS in Keap (formerly Infusionsoft) CRM. We resolved these problems by April 27th at 0:00 AM (GMT) and are currently working on syncing the affected calls and SMS. We anticipate that the syncing process will be finished by April 30th at 0:00 AM (GMT).
Report: "Security warning on JustCall Dashboard"
Last updateJustCall Dashboard was showing security warning for our customers. Timelines: 5:30 AM IST (Issue Reported) - Issues were reported in loading JustCall Dashboard 6:35 AM IST (Issue Fixed) - SSL certificate was renewed
Report: "JustCall Dialer briefly showed loading issue for some customers"
Last updateJustCall Dialer briefly showed loading issue for some customers. Timelines: Issue Reported: 02:48 AM PDT Issue Fixed: 02:53 AM PDT
Report: "Slowness in call logs dashboard and SMS section"
Last updateUsers reported difficulties in loading Call Logs dashboard and SMS section. Other sections of the JustCall Platform like Calling, Teams section, IVR, Sales Dialer were not affected. The issue was resolved in 37 mins
Report: "Issues in loading the JustCall Dialer"
Last updateOur team identified some issues with the JustCall dialer at 7:30 AM PST. Phone numbers were not loading on the dialer. At 7:30 AM the issue with JustCall phone numbers on the dialer was reported At 7:40 AM the dialer was operational. The incident was resolved Cause: One of the dependent services was giving CORS (https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) related error which led to the dialer page loading issues.
Report: "Intermittent issues in loading JustCall app. Start time : 10.27am GMT"
Last updateThis might be related to GCP's ongoing issues with https://status.cloud.google.com/incidents/pQohCqBLfFapHrkjY5Mh. The systems are back to normal.
The system should be up now, we are still monitoring the situation.
JustCall is right now suffering load-related issues, the app would be loading slow and in some cases, not loading at all. The team is looking into it.
Report: "Latency in Dialer and Calling Services"
Last updateOur monitoring systems detected increased latency in dialer & calling services at 10:00am PST. At 10:03 am PST, the issue causing increased latencies and error rates with calling services were identified. Load balancing services were deployed to distribute the traffic. At 10:05 am PST, we started seeing improvements with delays in dialer & calling services. We continued monitoring the service to ensure a full recovery. At 10:12 am PST, all services were operational. This incident has been resolved.
Report: "Elevated API Errors - 403 from Copper servers"
Last updateThis incident has been resolved.
We're experiencing an elevated level of API errors from Copper servers that has impacted our Copper integration. Our team is investigating this on priority and we have contacted Copper folks seeking a resolution
Report: "Elevated API Errors in Live Dashboard"
Last updateJustCall Live Dashboard is now operating normally. The issue impacting the Live Dashboard has been resolved, and all services are operating normally at this time.
JustCall Live Dashboard no longer experiencing API timeout. A fix has been implemented and we are currently monitoring the same.
We are currently experiencing elevated API timeout from JustCall Live Dashboard. We are monitoring the status and will update as soon as full functionality in our system is restored.
Report: "Elevated API Errors from Caching Instance"
Last updateCaching instance are now operating normally. The issue impacting the Caching instance has been resolved, and all services are operating normally at this time.
Our caching instance no longer experiencing connection issues across multiple products. We are currently monitoring the same.
We are experiencing elevated API errors across different platform. We are monitoring the status and will update as soon as full functionality in our system is restored.
Report: "Contacts not visible in AutoDialing Campaigns"
Last updateThis incident has been resolved from our end. If you find any discrepancies, please refresh the page once and if it still doesn't appear correctly, contact us on our chat support and we will surely help you out.
A fix has been implemented and we are monitoring accounts on our end to make sure 100% data is recovered for the contacts. More than 85% users will not see any difference. For the remaining 15%, our team is working hard to make sure the normalcy is returned. Please note that during the course of this event every other functionality including the campaigns were not affected and our users should be able to make calls, track campaign reports and listen to their conversations without any issues. If you are facing an issue or if you think your information is missing, please don't hesitate to contact us on our chat support or shoot us an email at help@justcall.io We would like to thank our awesome customers who helped us run some quick tests and confirmed our fixes, thus correcting the course of our action in restoring things.
We have recovered a lot of information for about 60% of the affected customers and are correctly working on to bring 100% normalcy.
We are continuing to work on a fix for this issue.
We have identified a potential issue where one of our disks hosting contacts for about 30% of our active customers for AutoDialer is running a bit sluggish and hence our customers may be seeing 0 or less than usual contacts in their AutoDialing campaigns. Our team is working hard to restore this disk to its maximum performance and work with our customers to restore contacts More updates will follow soon
Report: "Delays while running CRM integrations"
Last updateToday we had an incident where our CRM integrations got delayed due to unexpected call volumes that was way more than our estimated numbers. At around 11:30 AM EST, there were 200 calls added to our systems every second and this led to a delay in processing the CRM integrations from our end. We were notified about this by automated scripts at around 12:10 PM EST and an action was taken around 12:51 PM EST to clear the back log. By 01:40 PM EST, our backlog was cleared and as and when the new calls/texts arrived they were pushed almost immediately. This sudden bump in call volumes was due to the fact that we on-boarded a large number of enterprise clients today but it was totally our fault to not bump up resources allocated for integrations. The same has been taken care of and we promise that we will be extra vigilant moving forward.
Report: "Elevated API Errors across different platforms"
Last updateToday, at around 05:00 am PST, we had an issue where one of our instances was running a bit slow due to an enterprise client onboarding. Though our automated systems identified the problem, the failover protection never triggered in time. Our engineers were alerted of the situation and they manually deployed our zonal backups. During this time, our customers may have found that our website was not responsive, their calls were not going or they had issues accessing our REST APIs. We confirm that this lasted for about 5 to 10 minutes depending upon where in the world you were at that time. For the US and Europe, the downtime was approximately four minutes while for those who are connecting from the Asia Pacific region they may have experienced this running for about seven to eight minutes. We deeply regret the inconvenience and we make sure that such incidents do not happen in future. We have also taken steps on our end to continuously test our automatic failover protection every month once a day preferably on the weekend to make sure that this problem appears again. We want to assure you that we will continue to offer an uptime that no other cloud phone system can guarantee.
Report: "Intermittent delays in connecting calls and sending text messages for US region"
Last updateOn Friday, around 03:42 PM PST, there was an 18-minute incident where we had connection issues with US carriers. This may have caused our dialer on the web, mobile, and desktop apps to load slow and in some cases not load at all. Our engineering team was alerted within minutes (automated notification system) and we contacted our upstream providers to re-route traffic via different carriers. By 04:01 PM PST (18 minutes after the initial report), all calls and text messages were going through just fine and our throughput was restored to 100% capacity. Rest other parts of the service, integrations, and APIs were not affected. And, there was no issue with calls in other geographies.
Report: "Packet Loss in Multiple Regions Affecting Voice Calls"
Last updateAll systems are operational
Voice Calls are now operating normally. We will continue to monitor for system stability.
Starting at 7:52 am PST, Our service provider started detecting packet loss which caused dropped calls and call quality issues in multiple regions. We continue to actively investigate into this issue and expect to provide another update in 1 hour or as soon as we have more information.
Report: "Elevated API Errors from our notification service"
Last updateOneSignal confirms that issue has been resolved and we are seeing 100% successful posts while calling their are APIs
A fix has been implemented by OneSignal team and we are seeing that our customers have started receiving push notifications for incoming text messages and missed calls
The issue has been identified and a fix is being implemented.
Mobile app notifications for incoming text messages and missed calls are affected right now. We are contacting OneSignal for more updates
We're experiencing an elevated level of API errors and are currently looking into the issue. We have contacted our notification service provider OneSignal for more updates.
Report: "Elevated API Errors from our notification service"
Last updateThis incident has been resolved.
OneSignal team has identified the issue and they have confirmed that a fix has been rolled out
We are contacting OneSignal team for a response. Please note only mobile app notifications are affected now for text messages and missed calls.
OneSignal Team - We are experiencing an internal DNS outage and working to resolve it.
We're experiencing an elevated level of API errors and are currently looking into the issue. This is from our service provier that we use for sending notifications for SMS on desktop, mobile apps and web apps
Report: "Elevated API Errors from Twilio"
Last updateThis incident has been resolved.
Errors haven't appeared for last 4 minutes and we are monitoring this right now
We are continuing to investigate the issue.
We're experiencing an elevated level of API errors and are currently looking into the issue.
Report: "Elevated errors: Twilio APIs are delayed, causing multiple issues with calls and sms"
Last updateThis incident has been resolved.
We are currently investigating this issue and have contacted Twilio Account Manager for more information.
Report: "Login service for web portal is affected"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are currently investigating an issue with our login service for web portal. Our engineers have been notified and we are working on restoring that. Please note that those users who are logged in will not face any issues in making calls or sending text messages
Report: "Elevated API Errors | SMS"
Last updateThe issue impacting the SMS Service has been resolved, and services by our upstream provider are operating normally at this time.`
SMS Services are now operating normally, however processing of SMS queues was delayed during the incident. We expect to provide another update in 1 hour or as soon as more information becomes available
Our upstream no longer experiencing connection issues across multiple products. REST API, SMS Service are restored. The ability to process voice calls was unaffected. We expect to provide another update in 1 hour or as soon as more information becomes available
Our upstream provider are continuing to experience connection issues across multiple products included SMS Service. Their engineers are working on finding a solution. REST API access is being restored. The ability to process voice calls is unaffected but auxiliary services may be operating in a degraded mode. We expect to provide another update in 1 hour or as soon as more information becomes available.
Our upstream provider are continuing to experience connection issues across multiple products including SMS Service. Our engineers are working on finding a solution. The ability to process voice calls is unaffected but auxiliary services may be operating in a degraded mode. We expect to provide another update in 1 hour or as soon as more information becomes available.
Our upstream provider is continuing to experience connection issues across multiple products including SMS Service. Their engineers are working on finding a solution. The ability to process Voice Calls is unaffected but auxiliary services may be operating in a degraded mode. We expect to provide another update in 1 hour or as soon as more information becomes available
Our upstream provider is experiencing connection issues across multiple products including SMS service. Their engineers are working on finding a solution. We expect to provide another update in 30 minutes or as soon as more information becomes available.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
Our upstream provider have detected a potential issue while sending out the text messages. We are monitoring the status and will update as soon as full functionality in our system is restored.
Report: "Mobile notification(SMS, voicemail) temporarily unavailable due to provider incident"
Last updateAll systems have been returned to proper operation
Looks like the service is stable now. We're continuing to monitor the traffic.
OneSignal has identified the cause and is applying a fix. Some services are reachable now, but we're continuing to investigate further. (Incident: https://status.onesignal.com/incidents/qqj691b5j4yv)
We are continuing to investigate this issue.
A provider that we utilize for sending notifications for SMS & Voicemail on our mobile apps is experiencing an incident. We are monitoring the status and will update as soon as full functionality in our system is restored.
Report: "File upload service is down"
Last updateOur engineering team has confirmed that uploads are back. Though Filestack hasn't confirmed just yet (they had a super bad day of course), but we are confirming that the issue has been resolved. We apologize for the inconvenience caused and looks like it was something to do with cloud giant AWS.
Things are looking good from their end. We hope that this will be resolved soon. Meanwhile, the upload service is back and the FileStack status page shows they have implemented the fix.
Here's what the Filestack team has confirmed: US-EAST-1 uploads traffic was rerouted through our API servers in other regions. AWS confirmed that they have implemented fix and are deploying it to affected load balancers.
Here's what the Filestack team has confirmed: AWS confirmed the issue with their load balancers (https://status.aws.amazon.com/) that we are monitoring closely. In the meantime, we have provisioned emergency load balancers cluster and directed CDN traffic through it. Delivery and transformations should be back to normal, however uploads are still affected.
The issue has been identified, we are contacting Filestack for more details. We will wait for them to fix the issue they have been facing.
We rely on Filestack to facilitate file upload and that seems to be down right now. You can check their status here https://status.filestack.com/
Report: "Pipedrive API - error while refreshing tokens"
Last updateThis incident has been resolved from Pipedrive's end. We will reach out to customers if their integration is affected or if we feel there is a need to recover anything.
Things look good at the moment. The last error reported from their APIs was around 13 minutes back. We will continue to monitor the situation and mark this issue as resolved as soon as we get a green signal from our engineering team.
We can confirm that this is an issue from Pipedrive's end. We have reached out to our contacts in Pipedrive to gather an update from them.
We are investigating the issue. Meanwhile, looks like Pipedrive's web app is down as well. More info: https://status.pipedrive.com
We are receiving reports from our engineering team that there are issues while refreshing the access tokens for Pipedrive. These tokens help us access your account data and are required to keep your integration active with us. We are yet to confirm if this is an issue from PIpedrive's end or ours. So far things are not looking good and we will have an update for you in another 20 minutes.
Report: "Pipedrive API - delayed responses - Integration affected"
Last updatePipedrive APIs are responding just fine as per our recent reports. Our engineering team has confirmed that services are up and running from their end. More information around this: https://status.pipedrive.com
We are continuing to investigate this issue.
We are continuing to investigate this issue. Looks like their servers and APIs are getting back to normal.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
Pipedrive APIs are not responding. Our customers may face issues with integration. JustCall engineering team is following up with Pipedrive folks.
Report: "JustCall loading issues for US region | Elevated API errors"
Last updateOn January 21st, at 12:00pm EST JustCall was attacked by a giant group of bots from all over the world and overloaded our site all at once with the intention of taking it down. This type of attack isn't meant to steal or destroy anyone's data, it's just meant to cause the site to become inaccessible. At 12:01 pm EST, our internal notification system alarmed our engineering operations team. This internal notification system is designed to figure out our server load and start diverting traffic to our always-on copies of our codebase on different servers. By 12:12 pm EST, our Google Cloud servers were able to identify this fake traffic and take them down automatically. Our engineers on the other hand made sure that resource-intensive tasks or background jobs are immediately diverted to our other servers which are designed to handle these tasks. By this time, our latency was reduced manifolds and our customers were able to access the website, dialer, and mobile apps without any issues. Around 12:30 pm EST, we were able to bring down our server to our normal average, and at the time of this writing, our servers are performing just fine.
Today at 12pm EST, our engineers were alerted by our notification systems in place about latency in accessing our websites and load issues. We identified that we were attacked by a bot army which tries to pass through our firewalls thus increasing the traffic on our internal website (accessible by our customers). We immediately took steps to mitigate this and our automated load balancing systems transferred our internal traffic to multiple instances thus helping our customers able to make calls and send text messages. We agree that our customers may have seen blank pages, trouble accessing their dashboard and dialer during this time. The issue was severe for the first 12 minutes i.e till 12:12 pm EST and was completely under control by 12:32 pm EST. A detailed postmortem of this issue will be available in another 60 mins from here on our status page. If anything still seems not working properly, please clear your browser cache and cookies for the JustCall page and try again. If things still don’t seem right, contact us and we’ll investigate further.
Report: "REST API Outage affecting Zapier users as well"
Last updateThe issue started at 11:00 EST and lasted till 11:20 EST We have identified the cause and made sure that this doesn’t happen again. Our engineering team has evaluated the cause and sprung into action as soon as we got our first notice. We ensure that our REST APIs will continue to work fine without any issues and we apologize for the inconvenience caused today
We had an incident today where our REST APIs were affected due to an internal glitch. Our customers were not able to access their API credentials or they had issues accessing our APIs. This also affected our customers using Zapier since our APIs power our Zapier integration as well. Our engineers have fixed this now and the situation is under control. We sincerely apologize for the inconvenience caused and will make sure that such an incident doesn't happen again.
Report: "Elevated API Errors in Reports"
Last updateThis incident has been resolved.
We're experiencing an elevated level of API errors and are currently looking into the issue. Our teams will deploy a fix for this soon. Dashboard reporting will not be available for a while.
Report: "Elevated API Errors in Reports"
Last updateThis incident has been resolved.
We're experiencing an elevated level of API errors and are currently looking into the issue. Our customers will not be able to download reports in this case. We are actively investigating this concern.
Report: "Delay in analytics report"
Last updateThis incident has been resolved. We will still continue to monitor our servers to make sure we update our users if such an incident happens again
The issue has been identified and the team is working on resolving the same. The next update will be provided in 2 hours from here. We regret the inconvenience and will make sure that our dashboard reports are back up soon.
We are investigating an issue where there is a delay in our analytics reporting section. We are actively working to resolve the issue. We will provide another update in 1 hour or sooner once more information becomes available.
Report: "Elevated API Errors"
Last updateWe have experienced an elevated level of API errors. The issue lasted for 5 minutes 47 seconds , our engineering team have successfully resolved the same.
Report: "Some requests to our APIs are failing."
Last update* \[12:15 PM ET\] Delay in processing requests to our APIs were reported. * \[12:35 PM ET\] Our engineering team identified the issue and released a fix. They monitored the recovery. * \[01:11 PM ET\] This incident was resolved.
We're experiencing an elevated level of API errors and are currently looking into the issue.
Report: "Delay in initiating calls was reported"
Last update* \[11:00 AM EST\] Our monitoring system reported the delay and our engineering team started working on identifying the root cause. * \[11:20 AM EST\] They identified the root cause and started working on releasing a patch. * \[11:30 AM EST\] The patch was released by the engineering team to resolve the issue. They started monitoring the system status and re-initiated the monitoring system. * \[12:15 PM EST\] Engineering team confirmed all the systems are up and running and marked this as resolved.
Our monitoring system has reported a delay in connecting calls and our engineering team are working on finding the root cause.
Report: "Contact sync down due to delayed webhooks for Infusionsoft"
Last updateWe have re-instated our webhook services for Infusionsoft. Going forward, your recently created/modified contacts will be synced with JustCall address book almost instantly. We apologise for inconvenience caused due to this. Our support team has been notified to help you with your requests related to contact sync and any discrepancies that you are encountering.
We are working on to actively restore this service as soon as we can. In the meantime, you can contact us on chat support to have your contacts synced on priority. Please note - only your recently created and modified contacts on Infusionsoft are not passing over to your JustCall address book.
We have identified the cause of this issue and actively working to restore our webhook service. We have made sure that your contacts are synced on priority if you contact our support.
We are continuing to investigate this issue. Next update is awaited at 04:20 AM UTC on 2nd July.
We are receiving issues with our webhook service that keeps your contacts in sync with Infusionsoft. Any new contact created or modified on your Infusionsoft account will not be transferred over to JustCall during this time.
Report: "Duplicate messages for some MMS"
Last updateWe have resolved the issue causing duplicate MMS to be sent for messages with long message bodies (>1600 characters). The service is fully operational.
We are experiencing an issue that is causing duplicate MMS to be sent for messages with long message bodies (>1600 characters). Our engineers have identified the issue and are working on the fix. We will provide an update as quickly as possible.
Report: "Google Cloud Networking Issue"
Last updateWe've received a report of an issue with Google Cloud Networking as of Monday, 2018-11-12 14:16 US/Pacific. We have reports of Google Cloud IP addresses being erroneously advertised by internet service providers other than Google
Report: "Outage due to recovery disk corruption"
Last updateWe are really sorry for outage today. We were preparing for our upcoming weekend infrastructure upgrade and our boot disk got corrupted due to a faulty backup process. Hence, the web app was down for 30 minutes. We have resolved the issue and website is up. Some of the latest data got corrupted so you may not see some of the call activities for last 24 hours. But we are importing back the missing data from our backup so these logs will be up soon. Again sincere apologies for the outage and the inconvenience it caused.