CallTrackingMetrics

Is CallTrackingMetrics Down Right Now? Check if there is a current outage ongoing.

CallTrackingMetrics is currently Degraded

Last checked from CallTrackingMetrics's official status page

Historical record of incidents for CallTrackingMetrics

Report: "Intermittent issues with voice services."

Last update
resolved

The incident has been resolved. We will continue to monitor performance. Please reach out if you continue to experience issues with receiving or placing voice calls.

monitoring

A fix has been implemented and we are monitoring the results.

identified

Our upstream service provider has identified the problem causing the issues with CTM. They are working on a solution. Visit https://status.twilio.com/incidents/5mbfvb8mzhcn for updates.

investigating

Our technical partners are diligently working to identify and address the issue. Please visit https://status.twilio.com for more information. We will update you as we have more information.

investigating

We are continuing to investigate this issue.

investigating

We are aware of intermittent errors affecting your ability to answer calls. Our incident response team is investigating.

Report: "Investigating Potential Issue"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We've identified the cause, and have implemented a fix towards full recovery.

investigating

We are aware of intermittent errors occurring on the service. Our incident response team is investigating.

Report: "Difficulty connecting to app.calltrackingmetrics.com"

Last update
resolved

At approximately 1:19 pm EST on November 4, 2024 a brief period of very isolated issues connecting to https://app.calltrackingmetrics.com was experienced by a small subset of our users. Remediation actions were swift and the site recovered to normal operation at 1:26 pm EST.

Report: "Investigating Potential Issue"

Last update
resolved

This incident has been resolved.

monitoring

We have been working with our upstream provider and they have deployed a resolution to the issue. The issue is being tracked here by Twilio: https://status.twilio.com/ We will continue to post updates as we monitor the recovery.

identified

Our upstream provider has identified the issue and is working on a fix. We are seeing restoration.

identified

We have identified the issue is related to device authentication from our upstream voice provider. It is only affecting softphone and chat connections. We have found that the issue is intermittent and with jwt access tokens and that that refreshing and retrying can get through. We are actively working on a resolution with our provider.

investigating

We are currently investigating intermittent reports of customers having difficulty accessing our softphone. Some clients are able to authenticate but others are being affected by an issue with our upstream voice provider. We are actively working with them now

investigating

We are currently investigating intermittent reports of customers having difficulty accessing our softphone. This is not affecting all regions. We will report back shortly.

Report: "Investigating Potential Issue"

Last update
resolved

This incident has been resolved.

monitoring

All affected systems are operational.

monitoring

Session data is mostly restored. Session data is still delayed in the US-West areas.

monitoring

We are seeing recovery in most of the affected areas and continue to monitor our systems. Session data is still delayed.

identified

Our Salesforce integration has recovered. Delays in transcriptions, and email and text notifications are delayed but improving. Session data recovery continues.

identified

Our Salesforce integration is delayed but improving. Transcriptions are delayed Email and text notification are delayed but improving. Session Data recovery still continues.

identified

The issue has been identified and a fix is being implemented.

investigating

We are aware of issues currently affecting the indexing of tracking session data that might affect call routing that depends on the presence of that data. There will also be delays in this information showing in the call log and delayed indexing of the calls. Our technical parter is working on identifying the cause of the issue.

Report: "Investigating Potential Issue"

Last update
resolved

All issues affecting the normal operation of inbound and outbound calls have been identified and resolved.

investigating

Currently, the baseline call functionality has been restored and is operating normally. Users should experience no issues with standard voice calls. Our team is working diligently to resolve any residual issues to minimize any disruption to our users. We will provide another update in 30 minutes or as soon as more information becomes available.

investigating

Our technical partners continue investigating a service interruption affecting Twilio Voice services, impacting both inbound and outbound calls, including calls with Flex Services. Customers may experience degraded audio quality or difficulties connecting new calls. Their engineering team is actively working on identifying and resolving the cause. We aim to provide another update within 1 hour or as soon as more information becomes available. - We were seeing some recovery, but want to reflect the true status of the issue.

monitoring

We have observed increased normal operation of our the softphone and are monitoring performance.

investigating

Our technical partners continue to investigate the cause of the issue affecting inbound and outbound calls within CTM.

investigating

Our upstream provider has made us aware of issue that are affecting the proper operation of CTM. They have assured us that all available resources are working to resolve the issue in a priority manner. Currently, there is no timeframe for resolution.

Report: "Investigating Potential Issue"

Last update
resolved

This incident has been resolved.

monitoring

A small number of requests returned 500 errors. The cause for the errors has been identified and stopped. We're currently monitoring and working through a solution to prevent future errors.

investigating

We are aware of intermittent errors occurring on the service. Our incident response team is investigating.

Report: "Elevated 500's with Reports"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We are currently investigating this issue.

Report: "Investigating Potential Issue with Delayed Transcriptions"

Last update
resolved

This incident has been resolved.

investigating

We are aware of intermittent errors occurring with transcriptions. Your patience is appreciated while we look into the cause. We do not anticipate any loss of data; however the completion of the transcriptions might be delayed.

Report: "An issue with some CTM tracking numbers exists."

Last update
resolved

This incident has been resolved.

monitoring

Our technical partner has identified the cause and deployed a repair. Normal operation of voice calls is being reported. We will continue to monitor our systems.

investigating

We are aware of issues with a small subset of CTM tracking numbers in which a caller receives a busy signal and cannot connect. This cause of this issue is with systems that neither CTM nor our Direct Technical Partners control. We ask for your patience while this issue on the broader telecom network is being addressed. We will update via this channel as we receive new information.

Report: "Investigating Potential Issue affecting Logins and Access to Features"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We are aware of intermittent errors occurring that affect login and access to features. Our incident response team is investigating.

Report: "Issue with purchasing numbers"

Last update
resolved

Tracking numbers can be purchased using the inventory look up once again.

identified

We are aware of issues affecting the ability to purchase numbers within our platform. We are working on a resolution that will return the ability to purchase numbers to you. In the interim, you may request a number by visiting https://app.calltrackingmetrics.com/tracking_numbers/buy_new.

Report: "Investigating Issue with Transcriptions"

Last update
resolved

The issues with transcriptions have been resolved. Please report any issues to our Technical Support team at https://www.calltrackingmetrics.com/support.

monitoring

A fix has been put in place and we are monitoring performance. Transcriptions that did not happen are being retried and should be populated over the course of today.

investigating

We are aware of intermittent errors occurring on the service. Our incident response team is working with our transcription provider to solve the issue as quickly as possible. Additional updates will be posted as information is received.

Report: "Issue affecting AskAI"

Last update
resolved

Services have been restored and operations are back to normal. Thank you for your patience.

monitoring

Our technology partner has identified the issue affecting AskAI and has implemented a fix. Normal performance should be recovering if it has not already done so. Please visit https://status.openai.com/ for progress.

investigating

We are aware of an issue that may affect AskAI within CTM. Our technology partner is experiencing an outage and is investigating the cause. Please visit https://status.openai.com/ for progress.

Report: "Investigating Issue with our Transcription Service"

Last update
resolved

We observed elevated errors for approximately 1 hour . A process is running to transcribe all impacted recordings and workflows

investigating

We are aware of problems with our transcription service. Our incident response team is investigating.

Report: "Intermittent issues marking Smart Dialer calls as completed"

Last update
resolved

This incident has been resolved.

monitoring

Between 11:34 and 11:42 ET we experienced intermittent failures when marking Smart Dialer calls as completed. This was due to a networking issue with a caching server. The issue is resolved and we are monitoring.

identified

The issue has been identified and a fix has been implemented.

Report: "Delays with recording availability"

Last update
resolved

All Intelligence and Transcriptions that were recoverable have been recovered. Programmable Voice Recording and Intelligence services are operating normally at this time.

monitoring

New Voice recordings are now operating normally; our upstream carrier is still investigating impact to recordings & transcriptions during the impact window.

investigating

Our upstream provider has identified an issue causing some delays in recording availability. They are working to identify the cause.

Report: "CallTrackingMetrics Marketing Site Outage"

Last update
resolved

Our vendor has identified and resolved the network issue affecting our marketing site. Our CallTrackingMetrics platform was never affected.

identified

Our vendor is working on a networking issue. Our CallTrackingMetrics platform remains unaffected. To login you can go directly to https://app.calltrackingmetrics.com/

investigating

Our marketing site is experiencing an outage. Our CallTrackingMetrics platform is unaffected. To login you can go directly to https://app.calltrackingmetrics.com/

Report: "Investigating Potential Issue with Elevated 502 errors."

Last update
resolved

Services have been restored and operations are back to normal. Thank you for your patience.

monitoring

Root cause has been identified and mitigation steps have been implemented. We are still working on a full resolution and teams are monitoring error rates as they continue to improve.

investigating

We are aware of intermittent errors occurring on the service. Our incident response team is investigating.

Report: "Investigating Potential Issue - Elevated 500 errors"

Last update
resolved

Services have been restored and operations are back to normal. Thank you for your patience.

monitoring

Root cause has been identified and mitigation steps have been implemented. We are still working on a full resolution and teams are monitoring error rates as they continue to improve.

investigating

We are aware of intermittent errors occurring on the service. Our incident response team is investigating.

Report: "Issue with queued calls"

Last update
resolved

Our technical partner has resolved the issue with the queueing of calls.

investigating

Our Technical Partner has informed us of an issue which is causing calls to terminate abruptly within 10 seconds when routed through a queue. "Our monitoring systems have detected a potential issue with dequeue calls. Our engineering team has been alerted and is actively investigating. We will update as soon as we have more information."

Report: "Investigating Potential Issue"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring progress. We're currently processing jobs for reporting and events.

identified

The issue has been identified and a fix is being implemented.

investigating

We are aware of intermittent errors occurring on the service. Our incident response team is investigating.

Report: "Browser Softphone"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We are seeing partial recovery

identified

The issue has been identified and a fix is being implemented.

investigating

We are currently investigating this issue.

Report: "Investigating Potential Issue"

Last update
resolved

This incident has been resolved.

monitoring

We have identified and taken corrective action. We are monitoring performance.

investigating

We are aware of intermittent errors occurring on the service. Our incident response team is investigating.

Report: "Elevated 502 Errors"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We are continuing to investigate this issue.

investigating

We are continuing to investigate this issue.

investigating

We are currently investigating this issue.

Report: "Intermittent 502 errors occurring for white-labeled hosts"

Last update
resolved

Services have been restored and operations are back to normal. Thank you for your patience.

investigating

This is affecting a small subset of users. We are still working on the issue and you may connect to https://app.calltrackngmetrics.com.

investigating

You may connect to our main site https://app.calltrackingmetrics.com without errors.

investigating

We are continuing to investigate this issue.

investigating

We are aware of intermittent 502 errors occurring for white-labeled accounts. Our incident response team is investigating.

Report: "Intermittent issues searching for and purchasing phone numbers"

Last update
resolved

Services have been restored and operations are back to normal. Thank you for your patience.

monitoring

Root cause has been identified and mitigation steps have been implemented. We are still working on a full resolution and teams are monitoring error rates as they continue to improve.

identified

The issue has been identified and a fix is being implemented.

investigating

We are continuing to investigate this issue.

investigating

We are aware of intermittent issues searching for and purchasing phone numbers. Our incident response team is investigating.

Report: "Access to the CTM Service"

Last update
resolved

We experienced a brief period of elevated 500 errors between 10:04 and 10:12

Report: "Elevated 500/502 errors"

Last update
resolved

Starting at 11:36 and ending at 12:08 we experienced elevated 502 errors. This was due to a recovery process that was putting too much load on our databases related to the incident on Thursday. Our team is working on expanding our footprint to support increased load spikes like this now. The issue is resolved and we will not repeat this.

Report: "Issues with connecting to CTM"

Last update
resolved

This incident has been resolved.

monitoring

Our system is currently stable. Activities that were received during the incident will eventually appear in the log. New activities will appear in real time. We will continue to monitor performance.

monitoring

We are seeing recovery and are monitoring performance. We will update again at 5:00 pm EST.

identified

We are continuing to work on a fix for this issue.

identified

At the moment, our issues have returned. Our team is working on the resolution. The next scheduled update is at 4:30 pm EST.

monitoring

Site performance has improved, we continue to monitor. Our next update will be at 4:30 pm EST.

monitoring

We continue to monitor site performance. Thank you for your patience. Our next update will be at 3:30 pm EST.

monitoring

We are seeing recovery and continue to monitor performance. We will update again at 3:00 pm EST.

monitoring

Site availability has been restored. Our team continues to monitor as a regional outage in AWS infrastructure caused a database failure.

investigating

We appreciate your patience as we continue to investigate the cause of the issue. We will update again at 2:30 pm EST.

investigating

We are continuing to investigate the issue.

investigating

We are currently investigating the issue.

Report: "Google Call Extensions/Call Only Ads"

Last update
resolved

From 5:27pm EST to 8:04pm EST retrieval of Google Click data was limited. Our teams were alerted to the issue and worked on a solution. During this time call delivery was not impacted but retrieval of the additional Click data was not available. You may notice phone calls during this period missing attribution data.

Report: "Users experienced 500 errors when using CallTrackingMetrics"

Last update
resolved

We received reports of 500 errors when trying to perform various functions within CTM. The incident began at 12:11 p.m and was resolved at 12:39 pm. All times Eastern.

Report: "SMS Delays"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

Messages with a trust profile are not impacted by this issue. A work around is in progress to restore delivery

identified

The issue has been identified and a fix is being implemented.

investigating

We are currently investigating this issue.

Report: "Issue with Voice Calls - Inbound and Outbound"

Last update
resolved

All affected services have fully recovered.

monitoring

We are continuing to monitor for any further issues.

monitoring

We are continuing to monitor for any further issues.

monitoring

We have been informed of recovery and monitoring status by our upstream provider. Normal performance is being reported. We continue to monitor.

monitoring

We are seeing lag in handling of call event processing. This is primarily due to our upstream provider having a critical incident with webhook infrastructure.

monitoring

We continue to see improved performance with CallTrackingMetrics. We will continue to monitor and will update this page within the next 30 minutes.

monitoring

We have experience an issue with inbound and outbound calls and have taken corrective action. We are monitoring the performance and will update this page within 30 minutes.

Report: "Degraded performance with Voice Activities"

Last update
resolved

The issues with the ability to answer and place calls has been resolved. Please contact our Help Desk if you continue to experience issues.

monitoring

Voice activities are operating normally. We continue to monitor performance.

investigating

The issue causing the inability to answer and place calls continues to be investigated with the highest priority. We will update as soon as we have more information.

investigating

We are currently experiencing an incident that is negatively affecting the ability to answer and place calls within CTM. This issue lies with the upstream voice network and is being investigated with the highest priority. We will update as soon as we have more information.

Report: "API Logs"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We are continuing to work on a fix for this issue.

identified

We are continuing to work on a fix for this issue.

identified

We are looking into an issue impacting logging of API activity. Temporarily logs for third party API data will be suspended.

Report: "US Toll Free and Subset of US Long Code Numbers Experiencing Inbound and Outbound SMS and MMS Delivery Delays to/from US and Canada Networks"

Last update
resolved

This incident has been resolved.

identified

While we've observed noticeable recovery after the implementation of the workaround, we continue to see intermittent delivery delays sending and receiving Toll Free SMS and MMS messages in US and Canada, along with a small subset of T-mobile traffic on US long codes. The issue is also causing delivery receipts to be delayed, with delivery status stuck at "Sent". Our engineering team is continuing to work with our carrier partner to fully resolve the issue. We will provide another update in 1 hour or as soon as more information becomes available.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We are now observing some recovery in which some SMS and MMS to/from US and Canada networks from US Toll-Free and US long codes are being delivered. Our engineers have identified the issue and implemented a workaround for older messages. However, message DLRs will continue to be delayed even while messages are getting delivered. Our engineers are continuing to work with our carrier partner to resolve the issue. We will provide another update in 4 hours or as soon as more information becomes available. (from upstream)

identified

The issue has been identified and a fix is being implemented.

monitoring

We are continuing our investigation and have discovered inbound traffic to Twilio Toll-Free numbers from US and Canada networks is also affected and experiencing delivery delays.

identified

The issue has been identified and a fix is being implemented.

investigating

We are continuing to investigate this issue.

investigating

We are continuing our investigation and have discovered inbound traffic to Twilio Toll-Free numbers from US and Canada networks is also affected and experiencing delivery delays. In addition, we have discovered a small portion of US long code numbers are affected and also experiencing SMS and MMS delivery delays for inbound and outbound traffic to/from US and Canada networks. Our team continues working with our carrier partner to resolve the issue. We will provide another update in 2 hours or as soon as more information becomes available.

Report: "Elevated Error Rates"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are currently investigating this issue.

Report: "Intermittent Errors"

Last update
resolved

We encountered increase site issues starting at around 11:14pm EST until 11:32pm EST. We continue to monitor.

Report: "Elevated Error Rates"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We have managed to mostly isolate the impacts of the issue from a single availability zone in AWS however due to many APIs failing to respond and observing intermittent network availability from within the zone as instances lose connectivity we may experience elevated timeout errors.

investigating

The issue appears to be subsiding and possibly related to an outage in one region within AWS. We see we had multiple failover events from multiple systems at the same time all originating from a single region and availability zone within AWS.

investigating

We are currently investigating this issue.

Report: "Issues Sending SMS"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

This issue may also be impacting some call flows as well as degrading phone number availability for some phone numbers.

identified

We are working with our upstream providers on an issue related to sending text messages.

Report: "Issue affecting Call Log activities"

Last update
resolved

This incident has been resolved. Normal CTM call log functionality is restored.

monitoring

We are continue to see recovery of the issues and seeing increased normal performance. We are monitoring the situation.

identified

Beginning 3:15 PM EST, our upstream provider started experiencing API degradation causing latency and timeout errors for a myriad of services impacting our Calls, Texts and Chat communications. Their Engineers have worked on mitigating the issue and we are seeing recovery of product and services since 4PM EST. The next update will be in 30 minutes.

identified

Our carrier network has identified the issue causing the degraded performance and are taking steps to mitigate the issue.

monitoring

We are continuing to monitor for any further issues.

monitoring

We are monitoring the issue with our upstream and seeing nearly full recovery now

investigating

We are receiving notification from our upstream carrier network of issues that are affecting the normal operation of CTM. The issue is being investigated.

investigating

We are continuing to investigate this issue.

investigating

We are currently investigating this issue.

Report: "Elevated Error Rates From AWS Lambda"

Last update
resolved

This incident has been resolved.

monitoring

We are seeing fewer errors from AWS Lambda however AWS still reports recovering on some services so we will continue to monitor

identified

All delays are resolved, we are still seeing elevated errors from AWS

identified

Job processing delays related to AWS outage is causing some delays in delivery of text messages outbound and additionally we are still tracking some delays in updating the total active count of calls in the log. We are continuing to monitor status from AWS for a resolution.

identified

We are starting to see delays in some post processing tasks related to slow response times from aws dependent services. In some cases providers we work with that are currently impacted by AWS is impacting the time to process some events in the call log such as total active call counts etc.

identified

AWS has posted status on this issue. We are waiting on a resolution. All delayed lambda function invocations will be replayed once the issues are cleared.

investigating

The issues continue but they do appear limited to at least AWS Lambda, any customers not using Lambda do not appear to be impacted.

investigating

We are observing elevated error rates from AWS Lambda. This could cause issues with custom call scripts, triggers and routing rules that rely customers have created to run in AWS. We are looking into migrating some of these failing functions to a different region.

Report: "Issue with Voice Calls - Inbound and Outbound"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We have identified the cause of the issue and are implementing a solution.

Report: "Chat Issues Reported"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We are currently investigating issues with Chat.

Report: "Delays in processing Facebook conversions"

Last update
resolved

This incident has been resolved.

monitoring

We will continue to monitor https://status.fb.com/adsmanager

monitoring

We are holding conversions until Facebook's API's recover from their outage today. Once API access is fully restored we will begin the process of replaying failed conversion uploads.

Report: "Reports of outbound call errors"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are currently investigating this issue.

Report: "Saving Lambda Function Issue"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are investigating reports of issues saving Lambda functions. The issue appears to be related to a resource contention issue with AWS

Report: "Elevated errors sending push notifications via Google"

Last update
resolved

This incident has been resolved.

investigating

We are tracking the issue now with Firebase: https://status.firebase.google.com/

investigating

We are currently investigating elevated 502 errors when sending push notifications via Google push notification service. This would potentially impact notifications for agents using Chat and Native app Android users receiving incoming phone calls.

Report: "Temporary issues loading activity reports"

Last update
resolved

This incident has been resolved.

investigating

We are currently investigating this issue.