SharpenCX Outage History

Started Jun 03, 2025, 03:19 PM UTC· resolved Jun 03, 2025, 05:11 PM UTC

1. investigating Jun 03, 2025, 03:19 PM UTC

   We are aware of an issue which may be impacting some customers leveraging the IZ1 (app.iz1.sharpen.cx) environment. Some users may be experiencing disconnects, unexpected dead air, or failed outbound calls. We have all necessary resources engaged and are working to resolve the issue as soon as possible.

2. identified Jun 03, 2025, 03:26 PM UTC

   We have identified the issue and are in progress with executing corrective actions.

3. monitoring Jun 03, 2025, 03:45 PM UTC

   A fix has been implemented and we are monitoring the results. If your users are still experiencing Sharpen Q Phone connection issues, it is advised that they log out and back in to re-establish a successful connection.

4. identified Jun 03, 2025, 04:02 PM UTC

   We have observed some continued impact to Sharpen Q Phone connections for a subset of users. We have corrective actions identified and will be executing them shortly.

5. identified Jun 03, 2025, 04:07 PM UTC

   We are executing a fix for the remaining impacted users. Some users may experience a one time disconnect if they are on an active call. While this impact is not desired, it is necessary to fully restore the expected state of the supporting resources.

6. identified Jun 03, 2025, 04:23 PM UTC

   We are seeing additional disruption to outbound dialing from desk phones. We have isolated the fault and are working through corrective actions now.

7. identified Jun 03, 2025, 04:41 PM UTC

   We have identified all appropriate corrective actions and are in progress with executing them. We expect resolution shortly.

8. identified Jun 03, 2025, 04:48 PM UTC

   All Sharpen Q Phone connection issues have been resolved. We are investigating potential remaining issues based on other supporting infrastructure telemetry data.

9. monitoring Jun 03, 2025, 05:01 PM UTC

   We have observed stability in Sharpen Q Phone and call connectivity. We will continue to monitor the sustained recovery.

10. resolved Jun 03, 2025, 05:11 PM UTC

    This incident has been resolved.

Read the full incident report →

Started May 30, 2025, 07:25 PM UTC· resolved May 30, 2025, 08:03 PM UTC

1. investigating May 30, 2025, 07:25 PM UTC

   We are currently investigating an issue impacting inbound call handling in the IZ1 region. Callers may experience intermittent delays connecting or experience a busy signal.

2. identified May 30, 2025, 07:45 PM UTC

   Initial efforts have led to a recovery in busy signals for inbound calls in the IZ1 region. However, a smaller percentage of calls may still experience intermittent delays when connecting. Our team is still diligently working to correct the issue.

3. monitoring May 30, 2025, 07:48 PM UTC

   Inbound call handling in the IZ1 region has recovered, and normal service has resumed. We are continuing to monitor the platform to ensure continued stability. No further impact is expected at this time.

4. resolved May 30, 2025, 08:03 PM UTC

   The issue impacting inbound call handling in the IZ1 region has been fully resolved. Call traffic is operating normally, and no further disruptions are expected. Thank you for your patience while we worked to restore service.

Read the full incident report →

Started May 14, 2025, 03:42 PM UTC· resolved May 14, 2025, 10:43 PM UTC

1. investigating May 14, 2025, 03:42 PM UTC

   We are currently investigating an issue with inbound calls dropping or resulting in dead air.

2. investigating May 14, 2025, 04:28 PM UTC

   We have isolated the suspected fault point and are taking corrective action.

3. monitoring May 14, 2025, 04:51 PM UTC

   A fix has been implemented and we're monitoring recovery.

4. investigating May 14, 2025, 05:00 PM UTC

   We are seeing a resurgence of the symptoms at a low rate. We are resuming investigation to identify additional corrective actions.

5. investigating May 14, 2025, 05:01 PM UTC

   Current symptoms recognized are as follows. The investigation continues. 1. Dead air for a portion of inbound calls (variable rate of failure) 2. Unexpected disconnect from Sharpen Q phone 3. Unexpected drop of connected call at 2 minutes.

6. investigating May 14, 2025, 05:21 PM UTC

   We are continuing our investigation but have been able to identify that the current expected failure rate is ~20%. All necessary resources are engaged to resolve this issue as soon as possible.

7. investigating May 14, 2025, 05:45 PM UTC

   We have identified another corrective action and are coordinating steps to implement.

8. identified May 14, 2025, 06:08 PM UTC

   Current symptoms recognized are as follows. The investigation continues to correct the last symptom, unexpected drops from Sharpen Q phone. 1. Dead air for a portion of inbound calls (variable rate of failure) <- Corrected as of 1:54 PM ET 2. Unexpected disconnect from Sharpen Q phone 3. Unexpected drop of connected call at 2 minutes. <- Corrected as of 12:30 PM ET

9. identified May 14, 2025, 06:54 PM UTC

   Some WebRTC connections are still failing to connect and we're continuing our investigation. If a user has an active connection, it is recommended to not refresh to potentially lose the connection.

10. identified May 14, 2025, 07:38 PM UTC

    We are continuing our investigation. We've been able to identify and correct a portion of the resources supporting the Sharpen Q Phone WebRTC collective and are continuing efforts toward full restoration

11. identified May 14, 2025, 07:53 PM UTC

    We are observing disruptions to outbound calls and inbound call alerts for some agents. All necessary subject matter experts continue to investigate.

12. identified May 14, 2025, 08:15 PM UTC

    We are performing additional corrective actions on remaining WebRTC hosts to extend recovery. There may be a brief interruption during the change.

13. identified May 14, 2025, 08:27 PM UTC

    We are now observing a disruption to outbound Sharpen Q initiated calls.

14. identified May 14, 2025, 08:40 PM UTC

    Outbound calls are now working as expected.

15. identified May 14, 2025, 08:45 PM UTC

    The majority of WebRTC resources are now functioning as expected and most users should be seeing recovery. We are correcting a small portion of remaining infrastructure and will update when all necessary resources are fully restored.

16. monitoring May 14, 2025, 09:00 PM UTC

    All WebRTC resources are fully restored and confirmed to be working. All previously observed impacts have subsided and we're monitoring recovery.

17. resolved May 14, 2025, 10:43 PM UTC

    The primary service incident that began earlier today is resolved. We understand this has been an extended disruption and sincerely apologize for the impact this had on your operations throughout the day. We appreciate your patience and understanding. While the main issue is resolved, there is a remaining defect where some call recordings are not immediately accessible through the Insights user interface or the contact card. Our team is actively working on the final remediation steps for this. We anticipate that all call recordings will be fully accessible by the end of today. Should you require immediate access to a specific call recording before it becomes available in the UI, please open a request with our Care team. They will be able to retrieve the recording for you. Thank you again for your continued patience.

Read the full incident report →

Started May 14, 2025, 03:11 PM UTC· resolved May 14, 2025, 06:09 PM UTC

1. investigating May 14, 2025, 03:11 PM UTC

   We are currently investigating an issue that's causing between 30 and 45% of inbound calls to be dropped if they have been transferred to Sharpen Q from a Logic+ IVR. All necessary resources are engaged to identify and resolve this issue.

2. investigating May 14, 2025, 03:33 PM UTC

   This has been isolated to a very small set of customers. The majority of users are not impacted by this incident. We continue our investigation to identify and correct this issue.

3. monitoring May 14, 2025, 04:52 PM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved May 14, 2025, 06:09 PM UTC

   This incident has been resolved.

Read the full incident report →

Started May 08, 2025, 06:03 AM UTC· resolved May 08, 2025, 11:25 AM UTC

1. investigating May 08, 2025, 06:03 AM UTC

   We're currently investigating an issue affecting customers in IZ0 where the Insights page and dashboards are not loading as expected. Users may see partial content or a blank screen when accessing these features.

2. resolved May 08, 2025, 11:25 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Apr 10, 2025, 02:42 PM UTC· resolved Apr 11, 2025, 02:07 AM UTC

1. investigating Apr 10, 2025, 02:42 PM UTC

   We are currently investigating an issue causing intermittent dropped calls for inbound callers to the IZ1 (app.iz1.sharpen.cx) environment. If customers are leveraging Logic+ for their inbound IVR calls, subsequent transfers to Sharpen Q from the IVR, will result in an unexpected call drop at the 2 minute mark. We have all necessary resources engaged to correct the issue as soon as possible.

2. investigating Apr 10, 2025, 03:07 PM UTC

   We are continuing to investigate this issue.

3. investigating Apr 10, 2025, 03:29 PM UTC

   We have isolated the likely cause and are continuing our investigation. Thank you for your continued patience.

4. identified Apr 10, 2025, 03:40 PM UTC

   The issue has been identified and a fix is being implemented.

5. identified Apr 10, 2025, 04:04 PM UTC

   Our initial fix had positive impact, reducing the frequency of the event. However, additional measures were needed to establish full recovery. We have implemented additional changes and are analyzing our recovery.

6. monitoring Apr 10, 2025, 04:14 PM UTC

   A fix has been implemented and we are monitoring the results.

7. monitoring Apr 10, 2025, 04:14 PM UTC

   We are continuing to monitor for any further issues.

8. monitoring Apr 10, 2025, 04:34 PM UTC

   We are continuing to monitor resolution of the issue.

9. identified Apr 10, 2025, 05:20 PM UTC

   While our changes have introduced improvement in the frequency of failure, we are still observing some unexpected disconnected calls at the 2 minute mark for calls delivered to queue from a Logic+ IVR. We continue to have all necessary resources engaged to resolve this issue.

10. identified Apr 10, 2025, 06:08 PM UTC

    We are continuing to investigate this issue. At this stage we have isolated the fault zone and can see that, for customers leveraging Logic+ as the toolset for their IVR, between 30% and 50% of inbound calls may unexpectedly disconnect at the 2 minute mark. All necessary resources continue to be engaged to resolve this issue as soon as possible.

11. identified Apr 10, 2025, 06:37 PM UTC

    We are continuing our investigation and are working to restore service as soon as possible.

12. identified Apr 10, 2025, 07:15 PM UTC

    We are continuing to work on a fix for this issue.

13. identified Apr 10, 2025, 08:09 PM UTC

    We have identified a potential fix and are working on the hotfix now. We will work to release it as soon as it is ready.

14. identified Apr 10, 2025, 08:23 PM UTC

    Unfortunately the fix did not have the intended result and is being rolled back.

15. identified Apr 10, 2025, 08:46 PM UTC

    We are planning a more comprehensive fix planned to be completed by 9:00 PM EDT. We apologize for the continued disruption.

16. resolved Apr 11, 2025, 02:07 AM UTC

    This incident has been resolved.

Read the full incident report →

Started Apr 10, 2025, 01:34 PM UTC· resolved Apr 10, 2025, 02:05 PM UTC

1. identified Apr 10, 2025, 01:34 PM UTC

   Callers to the platform of IZ1 customers (those logging into app.iz1.sharpen.cx) may be experiencing dead air or disconnected calls when calling. This may be impacting up to 70% of calls, but is quite intermittent in its impact. We have all necessary resources engaged to resolve the issue and are working on a corrective action to deploy as soon as possible.

2. monitoring Apr 10, 2025, 01:57 PM UTC

   A fix has been implemented and we are monitoring the results.

3. resolved Apr 10, 2025, 02:05 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Apr 09, 2025, 03:59 PM UTC· resolved Apr 09, 2025, 04:40 PM UTC

1. identified Apr 09, 2025, 03:59 PM UTC

   Customer in IZ1 (app.iz1.sharpencx.com) may be experiencing intermittent disruption to inbound calls

2. identified Apr 09, 2025, 04:07 PM UTC

   We have identified the fault zone for this issue and are working through remediation options to restore service as soon as possible.

3. identified Apr 09, 2025, 04:19 PM UTC

   We have identified the source of the problem and are finalizing our corrective actions plans. We're targeting resolution within the next 15 minutes.

4. monitoring Apr 09, 2025, 04:27 PM UTC

   We are seeing initial signs of recovery, and are closely monitoring.

5. resolved Apr 09, 2025, 04:40 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Mar 03, 2025, 03:05 PM UTC· resolved Mar 03, 2025, 05:55 PM UTC

1. investigating Mar 03, 2025, 03:05 PM UTC

   We’ve detected a partial disruption to caller ID name lookups in Sharpen Q. Some users might see "Unknown Caller" displayed for the caller ID name on certain incoming voice calls. We’ve pinpointed the problem to our supporting callerID lookup provider and are actively working to fix it as quickly as possible. Though some caller ID names are missing, calls are still routing normally and displaying the correct caller ID number.

2. resolved Mar 03, 2025, 05:55 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Feb 28, 2025, 05:28 PM UTC· resolved Feb 28, 2025, 05:40 PM UTC

1. identified Feb 28, 2025, 05:28 PM UTC

   An issue has been identified that is blocking certain incoming IVR calls processed through Logic+ from successfully connecting to agents in Sharpen Q. We have all necessary resources engaged to restore service.

2. monitoring Feb 28, 2025, 05:36 PM UTC

   We have implemented a fix and are assessing recovery.

3. resolved Feb 28, 2025, 05:40 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Jan 06, 2025, 01:21 PM UTC· resolved Jan 06, 2025, 01:51 PM UTC

1. investigating Jan 06, 2025, 01:21 PM UTC

   We are aware of an issue impacting Sharpen Q operations in our IZ0 isolation zone. We have all resources engaged to resolve the issue as soon as possible.

2. identified Jan 06, 2025, 01:30 PM UTC

   We have identified the source of the issue and have implemented corrective actions. We're assessing the results of the change and will notify once resolution has been confirmed.

3. monitoring Jan 06, 2025, 01:34 PM UTC

   The fix has been confirmed. All systems look stable. We will continue to monitor for 15 minutes and then mark this item as resolved.

4. resolved Jan 06, 2025, 01:51 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Dec 19, 2024, 03:04 PM UTC· resolved Dec 19, 2024, 06:47 PM UTC

1. identified Dec 19, 2024, 03:04 PM UTC

   We are currently investigating an issue where Call Flows using SharpenDB2 for data dips are failing, resulting in inbound interaction failures or routing misbehavior. We have identified the issue and are working with our vendor to resolve.

2. identified Dec 19, 2024, 05:02 PM UTC

   Corrective actions/efforts are still underway with our vendor (AWS) -- where the underlying database communication issue has been identified. We'll proceed with validation efforts and post further updates to this status page, as soon as restoration has been achieved.

3. identified Dec 19, 2024, 05:53 PM UTC

   In collaboration with AWS we have identified a path to resolution. We expect resolution within 4 hours and will be providing hourly updates as we progress.

4. monitoring Dec 19, 2024, 06:32 PM UTC

   A fix has been implemented and we are monitoring the results.

5. resolved Dec 19, 2024, 06:47 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Dec 10, 2024, 07:54 PM UTC· resolved Dec 10, 2024, 09:20 PM UTC

1. investigating Dec 10, 2024, 07:54 PM UTC

   We're aware of an issue for customers hosted within our IZ0 infrastructure - where app behavior is significantly delayed/unresponsive. Symptoms range from seeing a "Connecting..." overlay appear in the Sharpen Q application, to being unable to answer interactions in a timely fashion. End users may also be experiencing problems with modifying their status or logging in/out of the application. We have all technical resources engaged at this time and are actively troubleshooting the issue.

2. monitoring Dec 10, 2024, 08:13 PM UTC

   We've identified an error trend within our infrastructure, and our Development Team has taken initial steps to correct the misbehavior. Our initial validation tests show a return to app responsiveness (being able to make/take calls, change statuses, etc). We're continuing to monitor our infrastructure for any indicators or error trends. Please reach out to the Care Team if you're seeing signs of sustained disruption.

3. resolved Dec 10, 2024, 09:20 PM UTC

   Recovery from this misbehavior was established at approximately 3:00PM Eastern, and we have not encountered any further responsiveness/delay issues across the application. Service has been fully restored, but we'll continue to monitor the situation. Please reach out to the Care Team in the event you have end users that are still encountering lingering symptoms.

Read the full incident report →

Started Nov 21, 2024, 02:14 PM UTC· resolved Nov 21, 2024, 03:03 PM UTC

1. identified Nov 21, 2024, 02:14 PM UTC

   We are aware of an issue in IZ1 (app.iz1.sharpen.cx) preventing users from being able to login. All resources are engaged and are working to restore service as soon as possible.

2. identified Nov 21, 2024, 02:20 PM UTC

   We have isolated the issue and can confirm that users are intermittently able to access app.iz1.sharpen.cx, but some are still experiencing failure. We have all necessary resources engaged and will continue to work for the earliest possible service recovery.

3. monitoring Nov 21, 2024, 02:34 PM UTC

   We are observing recovery as of 9:30 AM EST. We are continuing to monitor and identify the specific cause.

4. resolved Nov 21, 2024, 03:03 PM UTC

   We continue to see sustained recovery. This incident is resolved. Thank you for your patience.

Read the full incident report →

Started Nov 13, 2024, 06:54 AM UTC· resolved Nov 13, 2024, 08:30 AM UTC

1. investigating Nov 13, 2024, 06:54 AM UTC

   We're aware of an ongoing issue with the Sharpen Insights application suite - where navigating in-app, loading a saved Dashboard link, etc results in a blank white screen for all users. The issue has been reproduced by our Care Team in both IZ0 and IZ1 portions of our infrastructure. We're working to identify the underlying issue, and will share additional details as more information becomes available.

2. identified Nov 13, 2024, 07:45 AM UTC

   We've identified the underlying issue that is causing the reporting end-point to be unresponsive/only load a blank page in the application. We're in the process of getting things corrected and will share another update when there is full restoration to Insights.

3. resolved Nov 13, 2024, 08:30 AM UTC

   The underlying issue has been resolved for Insights, and the Care Team has verified all reports/dashboards/agent vues are loading at this time in IZ0 and IZ1.

Read the full incident report →

Started Nov 07, 2024, 05:56 PM UTC· resolved Nov 07, 2024, 06:27 PM UTC

1. investigating Nov 07, 2024, 05:56 PM UTC

   At approximately 12:45PM Eastern, our infrastructure team received an alert tied to Database availability. Services have automatically failed over to our backup infrastructure, and we're actively investigating the underlying cause - while performing additional process/app validations.

2. monitoring Nov 07, 2024, 06:08 PM UTC

   Our Dev Team verified that all services auto-recovered after the database failover completed - and that full service availability has returned to the application. Any users experiencing continued symptoms with inbound call handling/alerting or outbound calling difficulty should log out and back into the application. Please reach out to the Care Team if continued impact is encountered. We're actively monitoring our infrastructure at this time, and have engaged with Amazon Web Services to identify root-cause of the availability/restart issue.

3. resolved Nov 07, 2024, 06:27 PM UTC

   Transitioning to a resolved status, as all systems are back up and running - and we have full redundancy restored at the infrastructure level. A root-cause analysis report is being compiled at this time by our Development/Operations resources and the vendor (AWS), and will be sent out via cases that were opened with the Care Team when it is finalized.

Read the full incident report →

Started Nov 06, 2024, 01:52 PM UTC· resolved Nov 06, 2024, 02:00 PM UTC

1. investigating Nov 06, 2024, 01:52 PM UTC

   Customers running within our IZ1 infrastructure may notice delays with getting logged into the application. Upon navigating to app.iz1.sharpen.cx, it is taking upwards of 30 seconds for a login attempt to complete. Users may also experience delays with navigating between pages/toolsets in the Admin application.

2. resolved Nov 06, 2024, 02:00 PM UTC

   The underlying issue within our Infrastructure was identified and corrected. Login behavior is back to a fully operational status. If any users are still encountering problems at this time - have them fully log out of the application, then log back in (to force their browser session to pull an updated session configuration).

Read the full incident report →

Started Oct 29, 2024, 06:23 PM UTC· resolved Oct 29, 2024, 06:23 PM UTC

1. resolved Oct 29, 2024, 06:23 PM UTC

   Working with our Service Desk vendor (JIRA), the Care Team discovered that our "[email protected]" inbox monitoring entered a degraded state on 18OCT24. The underlying issue has been resolved as of 2:05PM Eastern on 29OCT24. If you had submitted a new service request or responded to an existing incident via to the "[email protected]" inbox between those dates, your message was unable to trigger the necessary automation for support case generation and updates to have been received. Please reach out to us in the event you are still experiencing any misbehavior associated to use of the Sharpen product suite or are having difficulty getting in contact with our Care Team!

Read the full incident report →