TOPdesk Outage History

TOPdesk is up right now

Pingoru has recorded 52 incidents for TOPdesk: 24 in the last 12 months and 3 in the last 90 days.

Source: status.topdesk.com

52incidents recorded
24last 12 months
3last 90 days
4 daystypical duration
Incidents per month
0510AprMayJunJulAugSepOct1
Share of hours with an open incident, any severity
0%50%100%AprMayJunJulAugSepOct25%
Pingoru began tracking TOPdesk in April 2026, so earlier months are not shown. Hover a bar for its value.
Show as a table
MonthIncidentsHours with an open incident
April 2026121 of 720 (3%)
May 2026120 of 744 (3%)
June 20269492 of 720 (68%)
July 20263267 of 744 (36%)
August 2026086 of 744 (12%)
September 2026260 of 720 (8%)
October 20261146 of 593 (25%)

Incidents, newest first

Minor September 11, 2025 RESOLVED: Email Delivery Delays – UK2 Datacenter 1d 4h UK2 SaaS hosting location

Started Sep 11, 2025, 09:31 AM UTC· resolved Sep 12, 2025, 01:32 PM UTC

  1. investigating Sep 11, 2025, 09:31 AM UTC

    We are currently experiencing a throttling of our email server in the UK2 datacenter, which may cause delays in sending outgoing emails. This is caused by multiple customers not configuring their SPF record correctly to include spf.topdesk.net Our support team is actively contacting these customers to resolve the matter as quickly as possible. In the meantime, but also in general, we recommend using your own mail server for outgoing emails, rather than the TOPdesk mail server, to help ensure timely delivery. Thank you for your understanding and cooperation. To inform TOPdesk you are affected by this issue, please visit My TOPdesk.

  2. investigating Sep 11, 2025, 09:32 AM UTC

    We are continuing to investigate this issue.

  3. resolved Sep 12, 2025, 01:32 PM UTC

    The email delivery issue in our UK2 datacenter has been resolved, and the mail queue is back to normal levels. The issue was related to outgoing emails sent from custom domains without the necessary SPF record configuration. To prevent similar issues in the future and ensure timely delivery, we recommend configuring your SPF records to include spf.topdesk.net if you use the TOPdesk mail server. Alternatively, using your own mail server for outgoing emails is also advised. Thank you for your patience and cooperation.

  4. postmortem Oct 13, 2025, 09:56 AM UTC

    On September 11, 2025, our UK2 datacenter experienced email delivery delays, due to multiple customers, including a high-volume one, missing the required SPF record. This misconfiguration led to our outgoing email server being throttled, resulting in minor delays for some customers. The issue was quickly resolved by our support team proactively reaching out to all customers without SPF records to recommend proper configuration and temporarily disabling mail functionality for the high-volume customer. To prevent recurrence, we advise all customers to use their own mail server for outgoing email or to ensure correct SPF configuration.

Read the full incident report →

Minor September 8, 2025 RESOLVED: Major - Mailimport problems 4h 28m Functionality malfunction

Started Sep 08, 2025, 10:33 AM UTC· resolved Sep 08, 2025, 03:02 PM UTC

  1. investigating Sep 08, 2025, 10:33 AM UTC

    We are currently experiencing disruptions with the mailimport . This is a problem we see on all our data centers. As result you may find that mail imports wil not work anymore. We are aware of the problem and are working on solving the problem. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when this issue is solved. The next update will be provided as events warrant until the issue has been resolved. E-mail updates will be sent right away when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR25 09 2081.

  2. monitoring Sep 08, 2025, 11:22 AM UTC

    We are pleased to let you know that the cause of the issue has been identified. A fix was directly made and rolled out. The imports should be working as normal again. We will monitor this major for a few hours more. If the issue remains resolved, we will close the Major around 17.00 hrs CEST. We apologize again for the inconvenience and thank you for your patience.

  3. resolved Sep 08, 2025, 03:02 PM UTC

    As communicated earlier this afternoon, we will now close the major. The disruptions that you have experienced today were caused by a dependency update of a library used in the mail import. This caused connections to Microsoft Exchange servers to fail. This was resolved by downgrading the version of the library. To prevent further issues, the problematic update will not be applied. We apologize again for the inconvenience and thank you for your patience.

Read the full incident report →

Major August 20, 2025 RESOLVED: Major: EU1 disruption 2h 25m Functionality malfunction

Started Aug 20, 2025, 06:49 AM UTC· resolved Aug 20, 2025, 09:15 AM UTC

  1. investigating Aug 20, 2025, 06:49 AM UTC

    We are currently experiencing disruptions with the EU1 hosting location. As a result your TOPdesk environment may not be available. We are aware of the problem and are working on a solution. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. The next update will be provided as events warrant until the issue has been resolved. E-mail updates will be sent right away when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR25 08 4429.

  2. investigating Aug 20, 2025, 07:41 AM UTC

    Our engineers are investigating networking issues in the EU1 hosting location. We have opened a support ticket with Microsoft. While some customers can access their environments again, our monitoring still indicates ongoing problems.

  3. investigating Aug 20, 2025, 08:29 AM UTC

    Mail import and file uploads may be affected by this major disruption. Some systems have been restarted, and overall system health is improving.

  4. monitoring Aug 20, 2025, 08:50 AM UTC

    All customer environments, including mail import and file uploads, are now operational. We will continue monitoring the situation and plan to close the incident after successful verification. A root cause analysis (RCA) will be completed and published on the status page. Customers still experiencing issues should contact support.

  5. resolved Aug 20, 2025, 09:15 AM UTC

    The EU1 hosting disruption has been resolved. All environments are operating normally, and no further issues have been detected. An RCA will be published once the investigation is complete.

  6. postmortem Sep 12, 2025, 11:50 AM UTC

    The major incident was caused by intermittent connection issues between the Kubernetes Service Cluster and the TOPdesk application. These connectivity problems led to disrupted access for customer environments, affecting critical services such as file uploads and mail import. Investigation revealed that the underlying issue originated from unstable communication with the application server, specifically related to the handling of temporary data. Although a restart of the affected components resolved the problem, we have identified the need to improve our monitoring. Going forward, we will enhance our monitoring to detect any issues related to temporary data sooner and minimize customer impact.

Read the full incident report →

Critical July 11, 2025 RESOLVED: EU1 problems slowness and availability 2h 21m EU1 SaaS hosting location

Started Jul 11, 2025, 08:37 AM UTC· resolved Jul 11, 2025, 10:58 AM UTC

  1. investigating Jul 11, 2025, 08:37 AM UTC

    We are currently experiencing disruptions with the EU1 hosting location. As a result your TOPdesk environment may not be available. We are aware of the problem and are working on a solution. To inform TOPdesk you are affected by this issue, please visit My TOPdesk.

  2. investigating Jul 11, 2025, 08:38 AM UTC

    We are currently experiencing disruptions with the EU1 hosting location. As a result, your TOPdesk environment may not be available. We are aware of the problem and are working on a solution. Thank you for your patience. We will update you as soon as there is more information.

  3. investigating Jul 11, 2025, 09:22 AM UTC

    We are aware of an ongoing issue affecting some customers in our EU1 hosting location, where certain environments are currently unreachable. Our SaaS team has investigated and identified a potential issue with the connection to the NetApp FileShare. At this time, the disruption appears to be affecting two containers: eu1c05 and eu1c06. We are actively working on resolving the problem and have contacted Azure Support for further assistance and troubleshooting. We will provide further updates as soon as we have more information. Thank you for your patience and understanding.

  4. resolved Jul 11, 2025, 10:58 AM UTC

    The issue affecting the NetApp FileShare has been resolved. All affected TOPdesk environments should now be fully accessible and functioning as expected. Azure Support is still investigating the root cause of this disruption. More detailed information, including a Root Cause Analysis, will be provided in a post-mortem report as soon as it becomes available. As the issue is now resolved, this major incident will be closed.

  5. postmortem Sep 05, 2025, 02:24 PM UTC

    On July 11, 2025, some customers experienced disruptions affecting their TOPdesk environments hosted in our EU1 location. Between approximately 10:00 AM and 1:00 PM \(CEST\), customers reported slow performance, inability to create tickets, issues downloading files, and in some cases, complete inaccessibility of their environments. Our SaaS team immediately began investigating and engaged with our cloud hosting partner, Microsoft Azure, to resolve the problem. All affected environments were fully restored and accessible again by 12:40 PM. Together with Microsoft Azure, we carried out a thorough investigation into the cause of the disruption. Unfortunately, neither our internal teams nor Microsoft’s specialists were able to identify a definitive root cause for the incident. No underlying issues were detected in the relevant systems or network infrastructure at the time of the disruption. We understand the importance of a stable and reliable TOPdesk environment for your organization, and we appreciate your patience as we worked to resolve the issue. Please rest assured that we are carefully monitoring our systems and continuing to work closely with our partners to help prevent similar incidents in the future. If you have any questions or concerns, our support team is available to assist you.

Read the full incident report →

Minor June 30, 2025 RESOLVED: - KI overview not loading 1h 48m Functionality malfunction

Started Jun 30, 2025, 09:21 AM UTC· resolved Jun 30, 2025, 11:10 AM UTC

  1. investigating Jun 30, 2025, 09:21 AM UTC

    We are currently experiencing disruptions with Knowledge Management. You might be unable to load the KI overview, work on existing KI's of create new ones. We are aware of the problem and are working on a solution. Please contact TOPdesk Support after submitting an incident, so we can store additional logfiles to investigate this. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. The next update will be provided as events warrant until the issue has been resolved. E-mail updates will be sent right away when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR25 06 8127.

  2. investigating Jun 30, 2025, 09:43 AM UTC

    Our development team has found the cause of the issues and is working on a fix. As soon as this has been found we will update you again. We thank you for your patience and apologize again for the inconvenience.

  3. resolved Jun 30, 2025, 11:10 AM UTC

    The cause of the problem loading Knowledge Management pages this morning was determined to have been caused by an issue within the newest version of the editor. Our development team has rolled back the version of the editor for Knowledge Management on all data centres. All environments should be able to use Knowledge Management again as normal. We thank you for your patience and apologize again for the inconvenience.

Read the full incident report →

Critical June 27, 2025 RESOLVED: SaaS disruption - Performance issues EU1 5h 14m EU1 SaaS hosting location

Started Jun 27, 2025, 06:53 AM UTC· resolved Jun 27, 2025, 12:07 PM UTC

  1. investigating Jun 27, 2025, 06:53 AM UTC

    We are currently experiencing disruptions with EU1 hosting location. As a result your TOPdesk environment may experience Performance issues. We are aware of the problem and are working on a solution. Please contact TOPdesk Support after submitting an incident, so we can store additional logfiles to investigate this. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. We aim to update this status page every 30 minutes until the issue has been resolved. E-mail updates will be sent when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. Internal major issues with EU1 To inform TOPdesk you are affected by this issue, please visit My TOPdesk.

  2. monitoring Jun 27, 2025, 07:37 AM UTC

    Our SaaS engineers have found a problem with our message queue running out of memory. We've taken steps to address the issue, and the queued jobs are being processed gradually. We are monitoring the situation and will update the status page as soon as we have more details.

  3. monitoring Jun 27, 2025, 08:50 AM UTC

    We wanted to update you on the current status of our message queue processing. The queue is still being processed at the moment. This should not have an immediate impact on the performance of TOPdesk. Our team will actively monitor the queue throughout the day to ensure it remains functional. If you encounter any urgent issues or require immediate assistance, please feel free to reach out to our support team.

  4. resolved Jun 27, 2025, 12:07 PM UTC

    The issues on EU1 have been resolved since 09:30. After monitoring the issue very closely, the build-up messaging queue decreased at the expected rate, and we are able to confirm that the issue has been resolved.

  5. postmortem Jul 25, 2025, 10:09 AM UTC

    On June 27th, the EU1 hosting location experienced a significant disruption that impacted the performance and availability of SaaS environments for many customers. Users reported issues such as endless loading when saving cards, and in some cases, the service was entirely unresponsive. We understand how vital our platform is to your daily operations and sincerely apologize for this inconvenience. The root cause of the disruption was a capacity issue with our messaging service at the EU1 hosting location. The messaging service experienced memory issues due to reaching its capacity limits, which resulted in a backlog of requests and contributed to degraded performance and responsiveness in customer environments. After our cloud engineers implemented a fix, the queued up tasks were processed gradually, resulting in customer environments returning to normal. We continued to closely monitor the situation to ensure stability and full recovery for all customers. We take this incident very seriously and have already identified several improvements to prevent similar issues in the future. In addition, we already have an ongoing project to resize the EU1 hosting location so that resources are properly managed and monitored. This will help prevent similar issues with the messaging service running out of memory and improve the overall resilience of our platform. We sincerely apologize for this service interruption and thank you for your patience and understanding. If you have any further questions or concerns, please reach out to our support team.

Read the full incident report →

Critical May 28, 2025 RESOLVED: Disruption EU1: Degraded performance 53m EU1 SaaS hosting location

Started May 28, 2025, 07:23 AM UTC· resolved May 28, 2025, 08:17 AM UTC

  1. investigating May 28, 2025, 07:23 AM UTC

    We are currently experiencing disruptions with the EU1 hosting location. Some functionalities of your TOPdesk environment might not be available. We are aware of the problem and are working on a solution. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. We aim to update this status page every 30 minutes until the issue has been resolved. E-mail updates will be sent when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR25 05 7706.

  2. monitoring May 28, 2025, 07:25 AM UTC

    We have identified the cause of the issue and implemented a fix. All functionalities will be fully operational within the next thirty minutes. Automated actions will proceed as planned, ensuring no data loss. We will continue to monitor the situation and will close the major incident once we confirm everything is functioning properly.

  3. resolved May 28, 2025, 08:17 AM UTC

    After a period of monitoring, we can confirm that all functionalities are fully operational. We apologize for any inconvenience this may have caused. If you have any questions, please don't hesitate to contact our support team. We will provide root cause analysis (RCA) as soon as possible.

  4. postmortem Jun 20, 2025, 11:42 AM UTC

    On May 28, 2025, a major incident affected the messaging system in the EU1 hosting location, disrupting functionalities such as imports and automated actions of multiple customer environments. The issue was identified when environments failed to connect to the messaging system, which was experiencing operational difficulties. Immediate actions were taken to diagnose and resolve the problem, starting with a restart of the system to restore normal functions. The restart action successfully reduced the message queue rapidly and roughly 15 minutes after it was cleared. The root cause was determined to be connectivity issues due to the degraded health of the messaging system, to which we have applied an upgrade to ensure future stability.

Read the full incident report →

Minor May 20, 2025 RESOLVED: Major: Asset + Supporting files imports not accessible 1d Functionality malfunction

Started May 20, 2025, 03:35 PM UTC· resolved May 21, 2025, 04:09 PM UTC

  1. investigating May 20, 2025, 03:35 PM UTC

    We are currently experiencing disruptions with our Asset and Supporting files imports. The Asset and Supporting files imports are not accessible, a error picture with the text 'You do not have permission to view this page.' appears We are aware of the problem and are working on a solution. Our apologies for the inconvenience. The next update will be provided as events warrant until the issue has been resolved. E-mail updates will be sent right away when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR25 05 5334.

  2. investigating May 20, 2025, 03:37 PM UTC

    The recommended solution is a restart to your environment, this should alleviate any lingering issues. If you cannot restart then there is a workaround related to the Supporting Files and Asset Management imports permission . Please go to the 'Supporting Files and Asset Management imports' section on the impacted permission group and ensure that all of the below sub permissions are checked. Please reach out via your ticket if you have any further questions.

  3. resolved May 21, 2025, 04:09 PM UTC

    The issue affecting the accessibility of Asset and Supporting files imports has been resolved. You should no longer encounter the error message stating "You do not have permission to view this page." To fully resolve any remaining issues, we recommend restarting your environment. This should alleviate any lingering problems. If a restart is not feasible, these will need to be manually checked. After this, the import will be accessible again. Please follow the workaround below: ⒈Navigate to Supporting Files and Asset Management imports Section: ◦Within the impacted permission group, ensure all sub-permissions in the Supporting Files and Asset Management imports section are checked. We appreciate your patience and understanding as we worked to resolve this matter. Should you encounter any further issues or have any questions, please do not hesitate to reach out.

Read the full incident report →

Minor May 20, 2025 RESOLVED: Service disruption my.topdesk 1d 4h Functionality malfunction

Started May 20, 2025, 08:31 AM UTC· resolved May 21, 2025, 12:56 PM UTC

  1. investigating May 20, 2025, 08:31 AM UTC

    We are currently experiencing disruptions with one of our services. As a result your my.topdesk.com may not be available. We are aware of the problem and are working on a solution. If you require support for your TOPdesk environment, please contact TOPdesk Support by phone or via email at [email protected]. Once we have more information, we will update this page. To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR25 05 4864.

  2. investigating May 21, 2025, 12:54 PM UTC

    The availability issues on My.TOPdesk are solved. Our teams have found the issue and implemented a fix. My.TOPdesk is available and operational for your requests. All tickets linked to the major will now be closed. If you experience any lingering problems please contact us through your known channels. Thank you for your patience, TOPdesk Support

  3. resolved May 21, 2025, 12:56 PM UTC

    The availability issues on My.TOPdesk are solved. Our teams have found the issue and implemented a fix. My.TOPdesk is available and operational for your requests. All tickets linked to the major will now be closed. If you experience any lingering problems please contact us through your known channels. Thank you for your patience, TOPdesk Support

Read the full incident report →

Minor May 19, 2025 RESOLVED: Service disruption My.TOPdesk 17h 59m Functionality malfunction

Started May 19, 2025, 01:26 PM UTC· resolved May 20, 2025, 07:25 AM UTC

  1. investigating May 19, 2025, 01:26 PM UTC

    We are currently experiencing disruptions with one of our services. As a result your My.TOPdesk.com may not be available. We are aware of the problem and are working on a solution. If you require support for your TOPdesk environment, please contact TOPdesk Support by phone or via email at [email protected]. Once we have more information, we will update this page. To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR25 05 4864.

  2. resolved May 20, 2025, 07:25 AM UTC

    This incident has been resolved.

  3. postmortem Jun 17, 2025, 01:19 PM UTC

    On May 15 and May 19, 2025, several customers were unable to log in to My TOPdesk. Due to a mistake in our system, several customer contracts were incorrectly synchronized, which in turn affected My TOPdesk availability. To prevent this from happening again, several fail-safes have been implemented. All data is now checked individually, and if any data is missing, the synchronization process of contract data is halted. We appreciate your patience and understanding as we worked to resolve this matter. Should you encounter any further issues or have any questions, please do not hesitate to reach out.

Read the full incident report →

Minor May 12, 2025 RESOLVED: Automated Actions don't trigger for some customers in production 1d 21h Functionality malfunction

Started May 12, 2025, 01:52 PM UTC· resolved May 14, 2025, 11:36 AM UTC

  1. investigating May 12, 2025, 01:52 PM UTC

    We would like to inform you about a recent issue related to our latest version upgrade, which has caused some disruptions in our automation processes. Specifically, certain events might not trigger as expected, leading to failures in automation workflows. No error messages will be displayed in the interface. The affected events and actions appear to involve special characters such as Æ, æ, Ø, ø, Å, å, ö, among others. Our development team has promptly addressed the issue and a fix has been created. The deployment of this fix will occur in two phases: * The first group will receive the update tonight. * The second group will receive the update tomorrow night. We understand the inconvenience this may cause and appreciate your patience as we work to restore normal operations. If you have any questions or require assistance, please do not hesitate to contact our support team. For the latest updates on this issue, please visit our TOPdesk Status Page: https://status.topdesk.com. Thank you for your understanding.

  2. identified May 12, 2025, 02:17 PM UTC

    The issue has been identified and a fix is being implemented.

  3. resolved May 14, 2025, 11:36 AM UTC

    The action service that was disrupted has been restored. We have had confirmation from several customers that sending email is working again. All tickets linked to the major will now be closed. If you experience any lingering problems please contact us through your known channels. What exactly caused this bug will be investigated and published in an RCA soon. Thank you for your patience, TOPdesk Support

Read the full incident report →

Minor May 8, 2025 RESOLVED: Slow or unavailable environments EU4 1h 9m Functionality malfunction

Started May 08, 2025, 10:11 AM UTC· resolved May 08, 2025, 11:20 AM UTC

  1. investigating May 08, 2025, 10:11 AM UTC

    We are currently experiencing disruptions with the EU4 hosting location. As a result your TOPdesk environment may not be available. We are aware of the problem and are working on a solution. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. The next update will be provided as events warrant until the issue has been resolved. E-mail updates will be sent right away when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. Multiple clients reporting problems with availability on EU4. To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR25 05 1932.

  2. investigating May 08, 2025, 10:45 AM UTC

    Our engineers have taken action to improve the performance of the affected servers. We are seeing the environments that were affected coming back online, and that the performance is nominal. We will proceed to evaluate this issue internally. Upon its completion, a Root Cause Analysis (RCA) will be posted on our status page for your reference. If you continue to experience any issues, kindly reach out to our support team for assistance. We appreciate your patience and understanding in this matter.

  3. resolved May 08, 2025, 11:20 AM UTC

    Our engineers have taken action to improve the performance of the affected servers. We are seeing the environments that were affected coming back online, and that the performance is nominal. We will proceed to evaluate this issue internally. Upon its completion, a Root Cause Analysis (RCA) will be posted on our status page for your reference. If you continue to experience any issues, kindly reach out to our support team for assistance. We appreciate your patience and understanding in this matter.

Read the full incident report →

Critical April 1, 2025 RESOLVED: SaaS disruption at EU2 hosting location 1h 23m EU2 SaaS hosting location

Started Apr 01, 2025, 08:56 AM UTC· resolved Apr 01, 2025, 10:19 AM UTC

  1. investigating Apr 01, 2025, 08:56 AM UTC

    We are currently experiencing problems on our EU2 hosting location As a result your TOPdesk environment may not be available. We are aware of the problem and are working on a solution. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. We aim to update this status page every 30 minutes until the issue has been resolved. E-mail updates will be sent when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. To inform TOPdesk you are affected by this issue, please visit My TOPdesk.

  2. investigating Apr 01, 2025, 09:20 AM UTC

    We identified an issue with the storage, resulting in servers losing access to their harddisk. This issue is not affecting all of EU2 but only a subset of servers. We reached out to our hosting provider Microsoft to investigate

  3. investigating Apr 01, 2025, 09:43 AM UTC

    Microsoft confirmed the issues and are actively investigating

  4. investigating Apr 01, 2025, 09:55 AM UTC

    The issue has been identified and is currently being fixed

  5. resolved Apr 01, 2025, 10:19 AM UTC

    The issue has been fixed by Microsoft and most environments are already back online. All remaining environments will become available again automatically within the next few minutes

Read the full incident report →

Minor March 11, 2025 RESOLVED: SaaS disruption - Login issues on CA1 1h 18m Functionality malfunction

Started Mar 11, 2025, 02:34 PM UTC· resolved Mar 11, 2025, 03:53 PM UTC

  1. investigating Mar 11, 2025, 02:34 PM UTC

    We are currently experiencing problems on the CA1 hosting location. As a result your TOPdesk environment may not be available, accessible or you will be stuck in a redirect loop. We are aware of the problem and are working on a solution. In a few reported cases a restart of your environment will fix the issue. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. The next update will be provided as events warrant until the issue has been resolved. E-mail updates will be sent right away when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR25 03 3244.

  2. resolved Mar 11, 2025, 03:53 PM UTC

    This issue has been resolved, and most customers confirmed they could reach/log in to their environment again after a successful restart. We will now proceed to evaluate this internally, and once the evaluation has been completed, a post-mortem will be published on our status page. If you still encounter issues with accessing your TOPdesk environment, please don't hesitate to reach out to your local Support department. We apologize for the inconvenience and appreciate your patience while we took care of the issue.

  3. postmortem Apr 01, 2025, 09:55 AM UTC

    During a scheduled maintenance, we transitioned from a self-managed cluster to Azure Kubernetes Service \(AKS\). As part of this process, all SaaS instances in the affected datacenter were scheduled for a restart. Unfortunately, several of these restarts did not complete successfully. Initial checks indicated that the login pages of these environments were visible, leading us to believe they were operational. However, customers soon reported that their environments were not functioning properly. Upon receiving these reports, our support team conducted a second round of restarts for the affected environments, which successfully resolved the issues and restored functionality. We apologize for the inconvenience this caused and appreciate your patience as we worked to resolve the problem.

Read the full incident report →

Minor March 3, 2025 RESOLVED: Major - CPU Issues 3d 15h Functionality malfunction

Started Mar 03, 2025, 08:31 PM UTC· resolved Mar 07, 2025, 11:41 AM UTC

  1. investigating Mar 03, 2025, 08:31 PM UTC

    We are currently experiencing disruptions on our datacenters. We are aware of the problem and are working on a solution. Please contact TOPdesk Support after submitting an incident, so we can store additional logfiles to investigate this. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. The next update will be provided as events warrant until the issue has been resolved. E-mail updates will be sent right away when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR25 02 6347.

  2. monitoring Mar 05, 2025, 01:37 PM UTC

    The new TOPdesk version-update including the fix has been rolled out over the last couple of days. This looks to have given us the intended result and the disruptions have disappeared. For now we will be closely monitoring this for the next coming days. We apologize for the inconvenience and thank you for your understanding. The current status can be found on our TOPdesk Status Page: https://status.topdesk.com

  3. resolved Mar 07, 2025, 11:41 AM UTC

    The new TOPdesk version-update including the fix has been rolled out over the last couple of days. Now every update group, which needed the fix, has been updated. Since the 5th of March 2025 no new calls have been added to the major and the monitoring shows the expected results. Therefore, this major incident has been resolved.

Read the full incident report →

Critical March 3, 2025 RESOLVED: Slowness at hosting location CA1 3h 4m CA1 SaaS hosting location

Started Mar 03, 2025, 05:13 PM UTC· resolved Mar 03, 2025, 08:18 PM UTC

  1. investigating Mar 03, 2025, 05:13 PM UTC

    We are currently experiencing disruptions with one of our services on CA1 . As a result your TOPdesk environment may not be available or respond slowly We are aware of the problem and are working on a solution. To help us troubleshoot this issue, please add the following information to your ticket when you are affected by this disruption: - The date and time of the last reproduction of this issue - The username of the user that reproduced this issue - A description of what the user was doing in TOPdesk - Browser activity and network traffic logs in a .HAR file, see KI 11015 on how to gather this information. Please contact TOPdesk Support after submitting an incident, so we can store additional logfiles to investigate this. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. We aim to update this status page every 30 minutes until the issue has been resolved. E-mail updates will be sent when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. To inform TOPdesk you are affected by this issue, please visit My TOPdesk.

  2. investigating Mar 03, 2025, 05:50 PM UTC

    Update: We are currently investigating these issues with multiple teams to identify the root of this issue.

  3. investigating Mar 03, 2025, 05:58 PM UTC

    Preliminary investigations indicate that the slowness is caused by external factors. Our services remain fully operational, and we are able to reach all internal environments without any issues. We will continue to monitor the situation closely.

  4. investigating Mar 03, 2025, 06:37 PM UTC

    After continued investigation our services remain fully operational and healthy, our cloud provider Microsoft Azure recently made us aware of issues they are experiencing... "Investigating reports of Azure issues in Canada and East US We're aware of reports of some issues accessing Azure resources in Canada and East US regions that we're currently investigating. We'll provide additional information as we become aware of it. This message was last updated at 18:22 UTC on 03 March 2025" You can keep up to date with their status page here... https://azure.status.microsoft/en-us/status

  5. investigating Mar 03, 2025, 07:05 PM UTC

    A new update from Microsoft indicates that they have identified the issue and have taken steps to mitigate the problem. Internally we have also seen improvements with the latency data coming from Microsoft, we will continue to monitor and ensure that performance returns to optimal levels. You can stay up to date with the issue on the Microsoft Azure status page... https://azure.status.microsoft/en-us/status

  6. resolved Mar 03, 2025, 08:18 PM UTC

    We are pleased to inform you that the issue has been resolved. All services are now operating normally, and access to resources in the CA1 region is now restored. This major will now be closed.

Read the full incident report →

Minor February 25, 2025 RESOLVED: Major - CPU Issues on EU1 6d 5h Functionality malfunction

Started Feb 25, 2025, 02:33 PM UTC· resolved Mar 03, 2025, 08:16 PM UTC

  1. investigating Feb 25, 2025, 02:33 PM UTC

    We are currently experiencing disruptions with EU1. We are aware of the problem and are working on a solution. Please contact TOPdesk Support after submitting an incident, so we can store additional logfiles to investigate this. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. The next update will be provided as events warrant until the issue has been resolved. E-mail updates will be sent right away when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR25 02 6347.

  2. investigating Feb 25, 2025, 05:29 PM UTC

    We are currently experiencing disruptions with EU1. The extra resources assigned yesterday appear to be effective; however, we are still noticing some occasional instability. Our developers are actively investigating and testing potential fixes for this issue with the highest priority. We are committed to resolving this problem as soon as possible. In the meantime, if you experience any interruptions or further issues while using TOPdesk, please do not hesitate to reach out to our support team. We apologize for any inconvenience and appreciate your patience as we work towards a solution. The next update will be provided as events warrant until the issue has been resolved.The current status can be found on our TOPdesk Status Page: https://status.topdesk.com

  3. investigating Feb 27, 2025, 01:51 PM UTC

    We are still investigating the disruption that is affecting customers in EU1. Our developers are still actively investigating and testing potential fixes for this issue with the highest priority. We are committed to resolving this problem as soon as possible. In the meantime, if you experience any interruptions or further issues while using TOPdesk, please do not hesitate to reach out to our support team. We apologize for any inconvenience and appreciate your patience as we work towards a solution. The next update will be provided as events warrant until the issue has been resolved. The current status can be found on our TOPdesk Status Page: https://status.topdesk.com

  4. investigating Feb 28, 2025, 02:02 PM UTC

    We will be testing a fix throughout this weekend, which will then be implemented as a new TOPdesk version-update. If this is succesfull, we will start rolling it out in the coming week. In the meantime, if you experience any interruptions or further issues while using TOPdesk, please do not hesitate to reach out to our support team. We apologize for any inconvenience and appreciate your patience as we work towards a solution. The next update will be provided as events warrant until the issue has been resolved. The current status can be found on our TOPdesk Status Page: https://status.topdesk.com

  5. investigating Mar 03, 2025, 01:30 PM UTC

    Last weekend the fix has been tested successfully. This means that the implementation of the new TOPdesk version-update including this fix, starts tonight. This will be rolled out in a staggered fashion over the next few days. In the meantime, if you experience any interruptions or further issues while using TOPdesk, please do not hesitate to reach out to our support team. We apologize for any inconvenience and appreciate your patience as we work towards a solution. The next update will be provided as events warrant until the issue has been resolved. The current status can be found on our TOPdesk Status Page: https://status.topdesk.com

  6. resolved Mar 03, 2025, 08:16 PM UTC

    We are pleased to inform you that the issue has been resolved. All services are now operating normally, and access to resources in the CA1 region is now restored. This major will now be closed.

Read the full incident report →

Minor February 21, 2025 RESOLVED: CPU issues on EU1 1h 55m Functionality malfunction

Started Feb 21, 2025, 09:20 AM UTC· resolved Feb 21, 2025, 11:15 AM UTC

  1. investigating Feb 21, 2025, 09:20 AM UTC

    We are currently experiencing disruptions with EU1 Container 08, 20, 22 and 24. We are aware of the problem and are working on a solution. To help us troubleshoot this issue, please add the following information to your ticket when you are affected by this disruption: - The date and time of the last reproduction of this issue - The username of the user that reproduced this issue - A description of what the user was doing in TOPdesk - Browser activity and network traffic logs in a .HAR file, see KI 11015 on how to gather this information. Please contact TOPdesk Support after submitting an incident, so we can store additional logfiles to investigate this. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. The current status can be found on our TOPdesk Status Page: https://status.topdesk.com To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR25 02 5945.

  2. resolved Feb 21, 2025, 11:15 AM UTC

    Our Saas Operations team has temporarily increased the SQL servers ressources on EU1. This fix has confirmed to have reduced performance issues for the affected customers, so the major will be closed. In the meantime, the root cause for the CPU issue is still being investigated.

Read the full incident report →

Minor February 11, 2025 RESOLVED: - Action Sequences on Hold 5h 8m Functionality malfunction

Started Feb 11, 2025, 02:08 PM UTC· resolved Feb 11, 2025, 07:16 PM UTC

  1. investigating Feb 11, 2025, 02:08 PM UTC

    We are currently experiencing disruptions with one of our services in the BR1 server. As a result your TOPdesk environment may be with the Action Sequences on Hold We are aware of the problem and are working on a solution. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. The next update will be provided as events warrant until the issue has been resolved. E-mail updates will be sent right away when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. No momento, estamos enfrentando interrupções em um de nossos serviços. Como resultado, seu ambiente TOPdesk pode estar com as sequências de ação em espera Estamos cientes do problema e estamos trabalhando em uma solução. Pedimos desculpas pela inconveniência. No momento em que escrevemos este texto, não podemos fornecer uma estimativa de quando seu ambiente estará disponível. O status atual pode ser encontrado em nossa página de status do TOPdesk: https://status.topdesk.com To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR25 02 2852.

  2. investigating Feb 11, 2025, 03:38 PM UTC

    Our team has identified the root cause of the problem and implemented the necessary solution. As a result, actions on our system are starting to gradually return to normal. Although the full restoration process may take some time, we are closely monitoring the system to ensure that everything works as expected again. We apologize for the inconvenience and thank you for your understanding. The current status can be found on our TOPdesk Status Page: https://status.topdesk.com Thank you Nossa equipe técnica identificou a causa raiz do problema e implementou a solução necessária. Como resultado, as ações em nosso sistema estão começando a retornar gradualmente ao normal. Embora o processo de restauração completa possa levar algum tempo, estamos monitorando atentamente o sistema para garantir que tudo volte a funcionar como esperado. Pedimos desculpas pelo inconveniente e agradecemos a compreensão. O status atual pode ser encontrado em nossa página de status do TOPdesk: https://status.topdesk.com Obrigado

  3. resolved Feb 11, 2025, 07:16 PM UTC

    Our team has identified the root cause of the problem and implemented the necessary solution. As a result, actions on our system are already full back to normal. We apologize for the inconvenience and thank you for your understanding. The current status can be found on our TOPdesk Status Page: https://status.topdesk.com Thank you Nossa equipe técnica identificou a causa raiz do problema e implementou a solução necessária. Como resultado, as ações em nosso sistema estão totalmente normalizadas. Pedimos desculpas pelo inconveniente e agradecemos a compreensão. O status atual pode ser encontrado em nossa página de status do TOPdesk: https://status.topdesk.com Obrigado

Read the full incident report →

Minor February 11, 2025 RESOLVED: Search index asset management 17/12 7m Functionality malfunction

Started Feb 11, 2025, 01:37 PM UTC· resolved Feb 11, 2025, 01:44 PM UTC

  1. investigating Feb 11, 2025, 01:37 PM UTC

    We are currently experiencing system wide disruptions with search indexes in the asset management module. As a result the searching function for the asset management module may not be functioning as expected. We are aware of the problem and are working on a solution. To help us troubleshoot this issue, please add the following information to your ticket when you are affected by this disruption: - The date and time of the last reproduction of this issue - The username of the user that reproduced this issue - A description of what the user was doing in TOPdesk - Browser activity and network traffic logs in a .HAR file, see KI 11015 on how to gather this information. Please contact TOPdesk Support after submitting an incident, so we can store additional logfiles to investigate this. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. The next update will be provided as events warrant until the issue has been resolved. E-mail updates will be sent right away when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR24 12 4748.

  2. resolved Feb 11, 2025, 01:44 PM UTC

    We are pleased to inform you that the problem has been solved completely. The problem emerged after a change to the implementation of the Asset Management search index creation. The environments of various customers with massive Asset Management databases were impacted by this. To solve the issue it was decided to reverse the change. In order to avoid this from happening again we are in the process of making the new implementation more robust, and capable of handling the load even with massive Asset Management databases. Furthermore, we are going to adjust our performance test database to better emulate such real-world databases. We apologize for any inconvenience that was caused and thank you for your patience.

Read the full incident report →

Critical November 25, 2024 RESOLVED: Microsoft graph gives a 503/504 error 1d 6h Functionality malfunction

Started Nov 25, 2024, 09:34 AM UTC· resolved Nov 26, 2024, 03:38 PM UTC

  1. investigating Nov 25, 2024, 09:34 AM UTC

    We are currently experiencing system wide disruptions with the outgoing (Email action) and incoming ( Mail import) mails using Microsoft Graph We are aware of the problem and are investigating whether this is related to a service at Microsoft. The current status can be found on our TOPdesk Status Page: https://status.topdesk.com To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR24 11 6886.

  2. investigating Nov 25, 2024, 09:58 AM UTC

    Our team has identified that these issues are likely related to the ongoing service disruptions reported by Microsoft. We are actively monitoring the situation and will provide updates as soon as we have more information.

  3. investigating Nov 25, 2024, 10:58 AM UTC

    Microsoft is actively investigating the issue and has identified the cause as a malfunctioning section of their infrastructure. We will continue to monitor the situation and will provide updates as soon as we have more information.

  4. investigating Nov 25, 2024, 12:30 PM UTC

    Microsoft is continuing to investigate the issue and different sources say that Microsoft is restarting different subsets of their infrastructure to help solving the issue. More information about the issue can be found in the Microsoft 365 Admin center under Microsoft Issue ID 'MO941162

  5. investigating Nov 25, 2024, 10:48 PM UTC

    Microsoft is continuing to redirect traffic and apply targeted remediation to impacted servers. We've observed initially positive results in some areas, and we're monitoring service health closely as our mitigation workstreams continue in parallel. We understand the significant impact of this event to your organization, we're treating this issue with the highest priority, and we're working to provide relief as soon as possible. This quick update is designed to give the latest information on this issue.

  6. resolved Nov 26, 2024, 03:38 PM UTC

    After reports from Microsoft and several positive reactions from customers the Microsoft Graph issue is resolved.

Read the full incident report →

Critical November 15, 2024 RESOLVED: Failure sending emails 56m AU1 SaaS hosting location, BR1 SaaS hosting location, CA1 SaaS hosting location, EU1 SaaS hosting location and 9 more

Started Nov 15, 2024, 03:02 PM UTC· resolved Nov 15, 2024, 03:59 PM UTC

  1. investigating Nov 15, 2024, 03:02 PM UTC

    We are currently experiencing system wide disruptions with one of our services. As a result you might be unable to send out mail when you use our mail server. We are aware of the problem and are working on a solution. To help us troubleshoot this issue, please add the following information to your ticket when you are affected by this disruption: - The date and time of the last reproduction of this issue - The username of the user that reproduced this issue - A description of what the user was doing in TOPdesk - Browser activity and network traffic logs in a .HAR file, see KI 11015 on how to gather this information. Please contact TOPdesk Support after submitting an incident, so we can store additional logfiles to investigate this. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. We aim to update this status page every 30 minutes until the issue has been resolved. E-mail updates will be sent when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR24 11 4509.

  2. investigating Nov 15, 2024, 03:12 PM UTC

    We are continuing to investigate this issue.

  3. resolved Nov 15, 2024, 03:59 PM UTC

    This incident has been resolved.

  4. postmortem Nov 19, 2024, 09:00 AM UTC

    On Friday 15th of November at around 14:45 CET we received notice of several customers not being able to send mail from TOPdesk. Due to the high number of incoming incidents, we treated this as a major incident and started research with high priority. Development was contacted and a fix was deployed on all servers at 16:16 CET. Further research showed that this particular error only occurs when you send a mail with an attachment. The root cause of this error was an inconsistency in a dependency that had been updated. Precautions will be implemented to avoid these discrepancies in the future.

Read the full incident report →

Critical November 13, 2024 RESOLVED: SaaS disruption at EU3 hosting location. 2 application servers unavailable 10m EU3 SaaS hosting location

Started Nov 13, 2024, 09:30 AM UTC· resolved Nov 13, 2024, 09:40 AM UTC

  1. investigating Nov 13, 2024, 09:30 AM UTC

    We are experiencing a problem with 2 application servers at our EU3 hosting location. This results in 23 environments being unavailable We are aware of the problem and are working on a solution. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. We aim to update this status page every 30 minutes until the issue has been resolved. E-mail updates will be sent when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR24 11 3429.

  2. investigating Nov 13, 2024, 09:33 AM UTC

    Both application servers have been restarted, all affected environments are available again.

  3. monitoring Nov 13, 2024, 09:34 AM UTC

    Both application servers have been restarted, all affected environments are available again.

  4. resolved Nov 13, 2024, 09:40 AM UTC

    We will evaluate this issue internally. Upon its completion, a Root Cause Analysis (RCA) will be posted on our status page for your reference. If you continue to experience any issues, kindly reach out to our support team for assistance. We appreciate your patience and understanding in this matter.

  5. postmortem Nov 19, 2024, 09:11 AM UTC

    The major incident on November 13, 2024, was caused by a hardware failure at the EU3 hosting location, which led to the temporary unavailability of two application servers. This disruption impacted 23 production environments, despite only 6 tickets being formally registered by users. According to the Windows event logs, the systems rebooted in a way that led us to conclude the servers either crashed or experienced an unexpected power loss. Our support team detected the issue at 10:12, and by 10:33, both affected application servers were successfully restarted, restoring all impacted environments.

Read the full incident report →

Notice November 4, 2024 RESOLVED: SaaS disruption NL4 environments unreachable 34m NL4 SaaS hosting location

Started Nov 04, 2024, 02:00 PM UTC· resolved Nov 04, 2024, 02:34 PM UTC

  1. investigating Nov 04, 2024, 02:00 PM UTC

    We are currently experiencing problems on NL4. As a result your TOPdesk environment may not be available. We are aware of the problem and are working on a solution. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. We aim to update this status page every 30 minutes until the issue has been resolved. E-mail updates will be sent when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR24 11 676.

  2. monitoring Nov 04, 2024, 02:18 PM UTC

    We have reached out to our service provider, Leaseweb, and they responded promptly. The issue with unreachable instances appears to be resolved now. We will continue to monitor the situation closely for a while to ensure everything remains stable. Thank you for your patience and understanding.

  3. resolved Nov 04, 2024, 02:34 PM UTC

    After closely monitoring the situation, we are happy to report that everything is stable, and our customers have confirmed that TOPdesk is working as expected once again. We will now close this major incident. An internal evaluation will be conducted soon, and we will share more details about the root cause in an upcoming post-mortem report. If you have any questions, please feel free to reach out to our support team. We sincerely appreciate your patience and understanding during this time.

  4. postmortem Nov 15, 2024, 02:22 PM UTC

    ### Incident Details The disruption began on November 4 with environments hosted in the NL4 data center being unreachable. Our team promptly responded by initiating an internal conference call and publishing the disruption on our status page to keep our customers informed. First investigation of the issue pointed towards the hosting location itself being unreachable and we contacted our hosting provider, who informed us that the primary edge firewall had been accidentally restarted during routine maintenance work on the secondary firewall. This restart caused all NL4 instances to become unreachable because none of the firewalls were online. Our hosting provider immediately began working on a resolution of the firewall issue and as soon as the hosting location was reachable again, a partial restoration of services was achieved with key servers and environments becoming reachable again. Our team monitored the situation closely and confirmed that the services were stable. We updated our status page to reflect the monitoring status and later confirmed the resolution of the issue. On the following day, additional issues related to the NL4 disruption were reported, including slowness in sending emails and certain functionalities within TOPdesk not working correctly. These issues were linked to the previous day's firewall incident, as a VPN configuration had been lost during the accidental restart. Our hosting provider re-applied the correct configuration, which resolved the remaining issues. Although we did not observe delays longer than two seconds in the logging of our SMTP servers, our engineers decided to reboot the SMTP servers to ensure they were not contributing to the slow email sending issue. By November 6, we received confirmation from our customers that everything was working as expected and all issues had been resolved. ### Root Cause The cause of the disruption was a combination of maintenance on the secondary firewall and the accidental restart of the primary firewall. Under normal circumstances, an accidental restart of the primary firewall would not have caused issues due to the presence of a secondary firewall. However, in this instance, the secondary firewall was also down due to maintenance purposes. As a result, there was no fallback available. Subsequent issues were related to a lost VPN configuration, which was corrected the following day. ### Actions Taken In response to the incident, we took immediate actions to restore services and communicate with our customers. Our team worked closely with our hosting provider to investigate and resolve the issue. We provided timely updates on our status page and confirmed the restoration of services with affected customers. Additionally, our engineers proactively rebooted the SMTP servers to eliminate any potential causes of the slow email sending issue. ### Next Steps The combination of both firewalls being offline is a rare case, as there will always be at least one firewall available under normal circumstances. While we cannot completely eliminate the potential for human error, we are taking several measures to mitigate its impact and enhance the robustness of our systems. To address this, we have created a backlog incident to explore ways to make our software more robust and less dependent on VPN configurations. Solving this issue will help ensure that functionalities continue to work properly even in the event of an accidental firewall reboot. We sincerely apologize for the inconvenience this disruption caused. We are dedicated to learning from this incident and improving our processes to ensure a more reliable service for our customers.

Read the full incident report →