tag:jira-service-management.status.atlassian.com,2005:/historyJira Service Management Status - Incident History2024-03-29T08:40:18ZJira Service Managementtag:jira-service-management.status.atlassian.com,2005:Incident/203943852024-03-29T08:40:18Z2024-03-29T08:40:18ZData residency migrations cannot be scheduled<p><small>Mar <var data-var='date'>29</var>, <var data-var='time'>08:40</var> UTC</small><br><strong>Resolved</strong> - Between 00:00 UTC to 06:30 UTC, we experienced issues with the admin screens used to schedule data residency migrations for Confluence, Jira Work Management, Jira Service Management, and Jira Software. The issue has been resolved and the service is operating normally.</p><p><small>Mar <var data-var='date'>29</var>, <var data-var='time'>07:35</var> UTC</small><br><strong>Investigating</strong> - We are investigating reports of errors when attempting to schedule a data residency migrations for Confluence, Jira Work Management, Jira Service Management, and Jira Software Cloud customers. We will provide more details once we identify the root cause.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/203570342024-03-26T03:17:55Z2024-03-26T03:17:55ZSome paginated queries in Forge hosted storage kept repeating the last page<p><small>Mar <var data-var='date'>26</var>, <var data-var='time'>03:17</var> UTC</small><br><strong>Resolved</strong> - We have identified and resolved a problem with Forge hosted storage, where some paginated queries kept repeating the last page. The incident was detected by our internal monitoring and was resolved quickly after detection by reverting the deployment. <br /><br />Activating changes recently made to the query cursors for paginated queries introduced a bug that impacted some apps. A small number of requests were impacted over a 16-minute window, while the incident lasted.<br /><br />Timeline:<br />- 25/Mar/24 10:08 p.m. UTC - Impact started, when the changes were deployed to production<br />- 25/Mar/24 10:09 p.m. UTC - Incident was detected<br />- 25/Mar/24 10:24 p.m. UTC - Incident was resolved and impact ended<br /><br />The impact of this incident has been completely mitigated and our monitoring tools confirm that query operations are back to the pre-incident behaviour. We have also resolved the underlying bug and deployed the fix to production, completely eliminating the cause of this incident.<br /><br />We apologise for any inconvenience this may have caused to our customers and our developer community.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/203143252024-03-20T22:41:04Z2024-03-20T22:41:04ZIssue with creating issues and transitions<p><small>Mar <var data-var='date'>20</var>, <var data-var='time'>22:41</var> UTC</small><br><strong>Resolved</strong> - Between 18:16 UTC and 22:XX UTC, March 20, 2024, we experienced creating issues and transitions for Jira Service Management, Jira Work Management, and Jira Software. The issue has been resolved and the service is operating normally.</p><p><small>Mar <var data-var='date'>20</var>, <var data-var='time'>22:05</var> UTC</small><br><strong>Identified</strong> - We are have identified an issue with creating issues and transitions that is impacting some Cloud customers. A fix is being rolled out now. Another update will be published in the next hour.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/201848392024-03-07T14:24:11Z2024-03-07T14:24:11ZPerformance issues in Jira<p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>14:24</var> UTC</small><br><strong>Resolved</strong> - Between 12:00 March 7th and 12-25 March 7th UTC, we faced performance problems with the Jira Issue View for some EU customers. The issue has been resolved and the service is operating normally.<br /><br />We are closing this incident for now.</p><p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>13:14</var> UTC</small><br><strong>Investigating</strong> - We have identified ongoing performance problems with the Jira Issue View for some EU customers. We are actively working to resolve this issue as quickly as possible.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/201712182024-03-06T05:50:00Z2024-03-06T05:50:00ZIssue with incoming email handler<p><small>Mar <var data-var='date'> 6</var>, <var data-var='time'>05:50</var> UTC</small><br><strong>Resolved</strong> - Between 20:25 March 5th and 04:49 March 6th UTC, we faced an issue with our incoming mail handlers, preventing the processing of new emails for some customers. The issue has been resolved and the service is operating normally.</p><p><small>Mar <var data-var='date'> 6</var>, <var data-var='time'>05:01</var> UTC</small><br><strong>Monitoring</strong> - We have deployed the fix and we are seeing recovery on affected cloud sites. We are now monitoring closely.</p><p><small>Mar <var data-var='date'> 6</var>, <var data-var='time'>04:06</var> UTC</small><br><strong>Identified</strong> - We have identified the root cause and we are currently working on deploying the fix. We'll update the status within the next hour.</p><p><small>Mar <var data-var='date'> 6</var>, <var data-var='time'>00:13</var> UTC</small><br><strong>Investigating</strong> - We have identified an incident that has caused the failure of the incoming email handler to a small number of customers, affecting issue creation and comments via email. We are investigating and will post updates within the next hour.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/201048252024-03-06T04:52:18Z2024-03-06T04:52:18ZConfiguration data discrepancies in automation rules<p><small>Mar <var data-var='date'> 6</var>, <var data-var='time'>04:52</var> UTC</small><br><strong>Resolved</strong> - All the impacted customers have been contacted and all the impacted rules have been fixed. We are now marking this issue as resolved.</p><p><small>Mar <var data-var='date'> 1</var>, <var data-var='time'>14:52</var> UTC</small><br><strong>Update</strong> - We have successfully restored the majority of automation rules that were affected, but we are currently in the process of manually fixing the remaining ones. Our support team will be contacting the customers impacted by this issue to proceed with resolving their cases.</p><p><small>Mar <var data-var='date'> 1</var>, <var data-var='time'>03:17</var> UTC</small><br><strong>Update</strong> - This issue has been mitigated, and we have restored the majority of automation rules that were affected. Those rules are working as normal, including the rules that had not been modified in the last 48 hours and the rules created after the incident. The remaining rules that haven't yet been restored are cases where customers have manually updated the rules after the incident happened.<br />We are proceeding with caution to avoid overwriting these changes.</p><p><small>Feb <var data-var='date'>29</var>, <var data-var='time'>17:30</var> UTC</small><br><strong>Update</strong> - We are continuously working on restoring the affected automation rules configuration data. At the moment the majority of the impact has already been mitigated and the team is currently focusing on a few specific outstanding scenarios. We will continue to update you as we progress.</p><p><small>Feb <var data-var='date'>29</var>, <var data-var='time'>15:20</var> UTC</small><br><strong>Update</strong> - We are currently working on restoring the lost automation rules configuration data. The email notifications related to the automation will be addressed upon restoring the lost data. Customers can update the configurations with the Automation rules themselves, and our fix will not override those changes. We will share further updates as we progress.</p><p><small>Feb <var data-var='date'>29</var>, <var data-var='time'>10:54</var> UTC</small><br><strong>Identified</strong> - We have identified the root cause and we are currently working on deploying fix to restore any lost configuration/rules. Next update will be provided in an hour.</p><p><small>Feb <var data-var='date'>29</var>, <var data-var='time'>09:23</var> UTC</small><br><strong>Update</strong> - We continue to investigate the issue with the Automation rules losing configuration data that is impacting certain cloud products. We are actively working to resolve this issue as quickly as possible.</p><p><small>Feb <var data-var='date'>29</var>, <var data-var='time'>08:22</var> UTC</small><br><strong>Investigating</strong> - We are investigating an issue with the Automation rules losing configuration data that is impacting some of the Cloud products. We are actively working to resolve this issue as quickly as possible.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/201024182024-02-29T02:53:17Z2024-02-29T04:00:18ZInvestigating new product purchasing<p><small>Feb <var data-var='date'>29</var>, <var data-var='time'>02:53</var> UTC</small><br><strong>Resolved</strong> - Between 28th Feb 2024 23:15 UTC to 29th Feb 2024 00:05 UTC, we experienced issue with new product purchasing for all products. All new sign up products have been successfully provision and confirmed issue has been resolved and the service is operating normally.</p><p><small>Feb <var data-var='date'>29</var>, <var data-var='time'>01:27</var> UTC</small><br><strong>Investigating</strong> - We are investigating an issue with new product purchasing that is impacting for all products. Customers adding new cloud products may have experienced a long waiting page or an error page after attempting to add a product. We have mitigated the root cause and are working to resolve impact for customers who attempted to add a product during the impact period. We will provide more details within the next hour.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/200974552024-02-28T17:16:59Z2024-03-07T04:54:32ZIssue with Automation and Connect Apps<p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>17:16</var> UTC</small><br><strong>Resolved</strong> - On 28th Feb between 12:15 UTC and 15:20 UTC Connect apps were unable to perform user-impersonation operations and failed to execute. This had a widespread impact on all product extensions based on the Connect platform resulting in failure when performing actions on behalf of a user.<br /><br />Jira Cloud Automation rules that triggered during the outage may have failed to run. Those rules will not be able to be replayed by customers or Atlassian. Our support engineers have details regarding which customers and automation rules that were impacted. However, support engineers will not be able to replay the failed automation rules.<br /><br />We have identified the root cause of the fault and have resolved it by scaling up the infrastructure. The incident is now resolved.<br /><br />If you have questions or require support - please reach out at https://support.atlassian.com/contact/.<br /><br />We apologise to all of our developers, customers and users who were impacted during this time.<br /><br />We will release a public incident review (PIR) here in the upcoming weeks. We will detail all that we can about what caused the issue, and what we are doing to prevent it from happening again.</p><p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>16:24</var> UTC</small><br><strong>Identified</strong> - Starting from 13:15 UTC, Automation and Connect Apps affecting certain cloud products<br /><br />We have scaled up the underlying services and we're seeing an improvement in response times and success rates. We continue to investigate the root cause and will provide the next update by 18:00 UTC.</p><p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>15:13</var> UTC</small><br><strong>Update</strong> - We continue to investigate the issue with the Automation and Connect Apps affecting certain cloud products. We are actively working to resolve this issue as quickly as possible.</p><p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>13:43</var> UTC</small><br><strong>Investigating</strong> - We are investigating an issue with Automation and Connect Apps that is impacting some Cloud products. We will provide more details within the next hour.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/200900202024-02-27T19:31:25Z2024-03-01T15:34:44ZSome email requests aren't being processed<p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>19:31</var> UTC</small><br><strong>Resolved</strong> - Between 14:41 UTC to 18:53 UTC, we experienced intermittent errors for email requests and logins for Jira Service Management. The affected email requests have been reprocessed and delivered. The issue has been resolved and the service is operating normally.</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>18:06</var> UTC</small><br><strong>Identified</strong> - We are investigating reports of intermittent errors for some Jira Service Management Cloud customers. We will provide more details once we identify the root cause.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/200370492024-02-21T11:11:19Z2024-02-21T11:11:19ZIntermittent 503 errors.<p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>11:11</var> UTC</small><br><strong>Resolved</strong> - Between 08:50 UTC to 09:04 UTC, we experienced intermittent 503 errors for Jira software and Jira service management. The issue has been resolved and the service is operating normally.</p><p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>10:23</var> UTC</small><br><strong>Monitoring</strong> - We have identified the root cause of the intermittent 503 errors and have mitigated the problem. We are now monitoring closely.</p><p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>10:11</var> UTC</small><br><strong>Investigating</strong> - We are investigating increased 503 error rates for Jira Software and Jira service management customers. We will provide more details once we identify the root cause.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/200347942024-02-21T04:57:17Z2024-03-01T08:24:59ZIncreased authentication errors across multiple products<p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>04:57</var> UTC</small><br><strong>Resolved</strong> - Between 2:30 UTC to 4:26 UTC, we experienced increased authentication errors for Confluence, Jira Work Management, Jira Service Management, Jira Software, and Atlassian Bitbucket. The issue has been resolved and the service is operating normally.</p><p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>04:22</var> UTC</small><br><strong>Monitoring</strong> - We have identified the root cause of the authentication errors and have mitigated the problem. We are now monitoring this closely and will provide further updates within the hour.</p><p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>04:04</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>03:41</var> UTC</small><br><strong>Investigating</strong> - We are investigating authentication issues impacting some Confluence, Jira Work Management, Jira Service Management, Jira Software, and Atlassian Bitbucket Cloud customers. We will provide more details within the next hour.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/200304552024-02-20T17:41:23Z2024-02-20T17:41:23ZSLAs not being calculated for Jira Service Management tickets<p><small>Feb <var data-var='date'>20</var>, <var data-var='time'>17:41</var> UTC</small><br><strong>Resolved</strong> - We have identified the issue and have also implemented a fix. The issue is now fixed, and you can reach out to our support portal in case of any further questions or issues related to the missing SLAs.</p><p><small>Feb <var data-var='date'>20</var>, <var data-var='time'>16:08</var> UTC</small><br><strong>Investigating</strong> - We are investigating reports of SLAs not being reported for some Jira Service Management cloud customers. Our team is working on identifying the root cause.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/200288982024-02-20T13:15:34Z2024-02-20T13:15:34ZIntermittent 503 errors<p><small>Feb <var data-var='date'>20</var>, <var data-var='time'>13:15</var> UTC</small><br><strong>Resolved</strong> - Between 04:33 UTC to 13:10 UTC, we experienced intermittent 503 errors for Jira Software, Jira service management and Jira work management. The issue has been resolved and the service is operating normally.</p><p><small>Feb <var data-var='date'>20</var>, <var data-var='time'>12:50</var> UTC</small><br><strong>Investigating</strong> - We are investigating reports of intermittent 503 errors for Jira Work Management, Jira Software and Jira Service Management Cloud customers. We will provide more details once we identify the root cause</p>tag:jira-service-management.status.atlassian.com,2005:Incident/199924642024-02-15T15:45:27Z2024-02-15T15:45:27ZAssets objects not loading<p><small>Feb <var data-var='date'>15</var>, <var data-var='time'>15:45</var> UTC</small><br><strong>Resolved</strong> - Between 12:30 PM UTC to 2 PM UTC, we experienced Asset object failures to load for Jira Service Management. The issue has been resolved and the service is operating normally.</p><p><small>Feb <var data-var='date'>15</var>, <var data-var='time'>15:03</var> UTC</small><br><strong>Investigating</strong> - We are investigating reports of intermittent errors for Assets objects not loading on Jira Service Management Cloud customers. We will provide more details once we identify the root cause.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/199871682024-02-14T23:32:18Z2024-02-27T05:44:51ZService Disruptions Affecting Atlassian Products<p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>23:32</var> UTC</small><br><strong>Resolved</strong> - We experienced increased errors on Confluence, Jira Work Management, Jira Service Management, Jira Software, Opsgenie, Trello, Atlassian Bitbucket, Atlassian Access, Jira Align, Jira Product Discovery, Atlas, Compass, and Atlassian Analytics. The issue has been resolved and the services are operating normally.</p><p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>22:55</var> UTC</small><br><strong>Monitoring</strong> - We have identified the root cause of the Service Disruptions affecting all Atlassian products and have mitigated the problem. We are now monitoring this closely.</p><p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>22:31</var> UTC</small><br><strong>Identified</strong> - We have identified the root cause of the increased errors and have mitigated the problem. We continue to work on resolving the issue and monitoring this closely.</p><p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>21:57</var> UTC</small><br><strong>Investigating</strong> - We are investigating reports of intermittent errors for all Cloud Customers across all Atlassian products. We will provide more details once we identify the root cause.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/199804202024-02-14T11:05:15Z2024-02-14T11:25:23ZJSM Customer portal users are unable to login through SSO<p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>11:05</var> UTC</small><br><strong>Resolved</strong> - Between 2024-02-13 10:30 UTC to 2024-02-14 4:30 UTC, we experienced JSM Customer portal users were not able to login through customer portal when they don't have a session that impacted some Jira Service Management Cloud customers. The issue has been resolved and the service is operating normally.</p><p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>02:42</var> UTC</small><br><strong>Identified</strong> - We've identified the root cause and are performing a fix to change the behaviour.</p><p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>02:41</var> UTC</small><br><strong>Investigating</strong> - JSM Customer portal users are not able to login through SSO when they don't have a session and try to open the Customer portal requests page. As a workaround, customers are able to access their tickets if they login via the help centre page.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/199715012024-02-13T06:27:20Z2024-02-13T06:27:20ZUser mention in issues' Description shown as @User instead of user name<p><small>Feb <var data-var='date'>13</var>, <var data-var='time'>06:27</var> UTC</small><br><strong>Resolved</strong> - Between 2024-02-12 21:00 UTC to 2024-02-13 05:52 UTC, we experienced mentioned user name showing as @User that impacted some Jira Cloud customers. The issue has been resolved and the service is operating normally.</p><p><small>Feb <var data-var='date'>13</var>, <var data-var='time'>03:56</var> UTC</small><br><strong>Identified</strong> - We continue to work on resolving the mention issue for Jira. We have identified the root cause and expect recovery shortly.</p><p><small>Feb <var data-var='date'>13</var>, <var data-var='time'>03:26</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Feb <var data-var='date'>13</var>, <var data-var='time'>02:12</var> UTC</small><br><strong>Investigating</strong> - We are investigating an issue with user name showing as @User for mention that is impacting some Jira Cloud customers. We will provide more details within the next hour.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/199320412024-02-07T17:44:38Z2024-02-07T17:44:38ZUser searches failing<p><small>Feb <var data-var='date'> 7</var>, <var data-var='time'>17:44</var> UTC</small><br><strong>Resolved</strong> - Between 15:40 UTC to 15:57 UTC customers experienced intermittent failures when searching for users in Atlassian cloud services: Confluence, Jira Work Management, Jira Service Management, Jira Software, Atlassian Bitbucket, Jira Product Discovery, and Compass. The issue has been resolved and the service is operating normally.</p><p><small>Feb <var data-var='date'> 7</var>, <var data-var='time'>17:05</var> UTC</small><br><strong>Monitoring</strong> - We have mitigated the problem. We are now monitoring closely.</p><p><small>Feb <var data-var='date'> 7</var>, <var data-var='time'>16:40</var> UTC</small><br><strong>Investigating</strong> - We are investigating an issue with our user search service that is impacting Atlassian cloud service: Confluence, Jira Work Management, Jira Service Management, Jira Software, Atlassian Bitbucket, Jira Product Discovery, and Compass. We will provide more details within the next hour.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/199179632024-02-06T02:40:54Z2024-02-06T02:40:54ZDrop in the success rate of Forge hosted storage API calls<p><small>Feb <var data-var='date'> 6</var>, <var data-var='time'>02:40</var> UTC</small><br><strong>Resolved</strong> - We identified a problem with the Forge hosted storage API calls, which resulted in a drop in invocation success rates in the developer console. The impact of this incident has been mitigated and our monitoring tools confirm that the success-rate is back to the pre-incident behaviour. It impacted 16 apps according to our logs, where these apps saw a reduced success rate of storage.get API calls, as listed in https://developer.atlassian.com/platform/forge/runtime-reference/storage-api-basic.<br /><br />As part of Forge's preparation to support Data Residency, Forge hosted storage has been undergoing a platform and data migration for storing app data. As part of this migration we do comparison checks for data consistency between the old and new platform. The previous incident earlier, https://developer.status.atlassian.com/incidents/9q71ytpjhbtl, had put the data on the new platform out of sync and so comparisons of the data from the old and new platform started showing failures and the migration logic retries on failures to test for consistency issues. This retry behaviour increased latency of these requests which led to 16 apps receiving an increased number of 504 timeout errors. <br /><br />Checking synchronously was identified by the team as a bug and should have been async. Once the root cause was identified we moved our backing platform rollout to a previous stage. The rollout is split into several stages. The issues we were having were on our blocking stage where we make calls to both the old and new platform and wait for both to complete so we can test any performance issues in the new platform before using it as our source of truth. It was in this blocking stage where we had a bug that included waiting on comparisons when it should've been async.<br /><br />To recover, we reverted back to our shadow mode stage. In this stage, all operations to the new platform are asynchronous, including comparisons that were blocking in the other stage and resulted in timeout issues and 504 errors being sent to apps. This is the state that Forge hosted storage has been in for several months without any problems.<br /><br />Here is the timeline of the impact:<br />- On 2024-02-05 at 06:42 PM UTC, impact started with comparisons start happening on out of sync data in blocking mode<br />- On 2024-02-05 at 08:57 PM UTC, impact was detected to API by our monitoring systems<br />- On 2024-02-05 at 11:34 PM UTC, rollout to new platform was reverted to known stable state and impact ended<br /><br />We will release a public incident review, PIR, here in the upcoming weeks for this and the incident that happened earlier, https://developer.status.atlassian.com/incidents/9q71ytpjhbtl. We will detail all that we can about what caused the issue, and what we are doing to prevent it from happening again.<br /><br />We apologise for any inconveniences this may have caused our customers and the developer community and committed to preventing further issues with our hosted storage capability.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/198915062024-02-02T08:53:24Z2024-02-02T08:53:24ZAutomation for Jira email address handling failures<p><small>Feb <var data-var='date'> 2</var>, <var data-var='time'>08:53</var> UTC</small><br><strong>Resolved</strong> - Between 02:00 AM UTC to 04:13 AM UTC, we experienced email address handling failures, for email fields and smart values, which resulted in rule executions failing for some customers using Automation for Jira - JSW, JSM, JWM, JPD. The issue has been resolved and the service is operating normally.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/198729862024-01-31T06:03:12Z2024-01-31T06:03:12ZPerformance degradation for Forge app invocations<p><small>Jan <var data-var='date'>31</var>, <var data-var='time'>06:03</var> UTC</small><br><strong>Resolved</strong> - At around 4am UTC, about 40 percent of Forge app invocations experienced high latency, with a portion of the requests failing, during a 15 minute time window.<br /><br />The scaling of the instances was misconfigured following a new deployment of the service, which needed manual intervention which took a few minutes to resolve the issue.<br /><br />Timeline:<br />- 2024-01-31 04:00 UTC: impact started<br />- 2024-01-31 04:03 UTC: incident detected<br />- 2024-01-31 04:15 UTC: the incident was resolved and the impact ended<br /><br />This issue is now resolved and Forge is fully operational. We apologize for any inconveniences this may have caused to our customers, partners, and our developer community.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/198064762024-01-24T07:29:27Z2024-01-24T07:29:27ZHigh number of sandboxes being suspended<p><small>Jan <var data-var='date'>24</var>, <var data-var='time'>07:29</var> UTC</small><br><strong>Resolved</strong> - All sandbox sites that were affected by this incident have been restored and are now accessible.</p><p><small>Jan <var data-var='date'>24</var>, <var data-var='time'>05:13</var> UTC</small><br><strong>Update</strong> - Majority of sandboxes have now been restored. We are in the final stages of restoring the remaining sandbox sites.</p><p><small>Jan <var data-var='date'>24</var>, <var data-var='time'>00:09</var> UTC</small><br><strong>Update</strong> - We've begun the bulk restoration of all affected sandboxes and will be progressively restoring access.</p><p><small>Jan <var data-var='date'>23</var>, <var data-var='time'>23:49</var> UTC</small><br><strong>Update</strong> - We've begun bulk restoration of all affected sandboxes and will be progressively restoring access.</p><p><small>Jan <var data-var='date'>23</var>, <var data-var='time'>17:54</var> UTC</small><br><strong>Update</strong> - We've identified a fix to restore affected sandboxes.<br />We're working to roll this out, and then we'll develop a process to apply to all affected sandboxes.</p><p><small>Jan <var data-var='date'>23</var>, <var data-var='time'>15:33</var> UTC</small><br><strong>Identified</strong> - We have identified the root cause and rolled back the code causing the issue. We're now exploring bulk restoring effected sandboxes.<br />No data was lost.<br />As a workaround, customers can 'Restore' their sandboxes and the app should be back online.</p><p><small>Jan <var data-var='date'>23</var>, <var data-var='time'>14:33</var> UTC</small><br><strong>Update</strong> - We continue to investigate this issue's root cause. More will be shared in the next updates.<br /><br />As a workaround, customers can 'Restore' their sandboxes and the app should be back online.</p><p><small>Jan <var data-var='date'>23</var>, <var data-var='time'>14:24</var> UTC</small><br><strong>Investigating</strong> - We are seeing sandboxes being suspended at an anomalously high rate. The team is working on finding the root cause and restore affected sandboxes.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/197969572024-01-23T00:07:35Z2024-01-23T00:07:35ZPerfomance degradation for Jira instances affecting small cohort of customers<p><small>Jan <var data-var='date'>23</var>, <var data-var='time'>00:07</var> UTC</small><br><strong>Resolved</strong> - Between 14:45 UTC to 18:25 UTC, we experienced degraded performance for Jira Work Management, Jira Service Management, Jira Software, and Jira Product Discovery for a small subset of instances. The situation has stabilized, and the service is operating normally.</p><p><small>Jan <var data-var='date'>22</var>, <var data-var='time'>18:13</var> UTC</small><br><strong>Investigating</strong> - We are investigating reports of intermittent degraded performance for Jira Service Management, Jira Software, Jira Work Management, and Jira Product Discovery Cloud, affecting a small cohort of customers in a US shard. We will provide more details shortly.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/197494152024-01-18T16:07:20Z2024-01-18T16:07:20ZJira Family is unavailable across shards<p><small>Jan <var data-var='date'>18</var>, <var data-var='time'>16:07</var> UTC</small><br><strong>Resolved</strong> - Between 06:52 UTC to 10:29 UTC, we experienced an outage for Jira Work Management, Jira Software, and Jira Product Discovery. The issue has been resolved and the service is operating normally.</p><p><small>Jan <var data-var='date'>18</var>, <var data-var='time'>10:52</var> UTC</small><br><strong>Monitoring</strong> - Our engineering team has implemented fixes for Jira and should be recovered again. We will continue to monitor all systems.</p><p><small>Jan <var data-var='date'>18</var>, <var data-var='time'>09:49</var> UTC</small><br><strong>Identified</strong> - We have identified the root cause of the issue and working on the fix.</p><p><small>Jan <var data-var='date'>18</var>, <var data-var='time'>08:35</var> UTC</small><br><strong>Investigating</strong> - We are investigating an issue with outage that is impacting Jira Work management, Jira Software, Jira Service Management and Jira Product Discovery. We will provide more details within the next hour.</p>tag:jira-service-management.status.atlassian.com,2005:Incident/197070372024-01-15T01:00:15Z2024-01-29T00:13:46ZOutage in Atlassian Intelligence functionality in multiple products<p><small>Jan <var data-var='date'>15</var>, <var data-var='time'>01:00</var> UTC</small><br><strong>Resolved</strong> - Between 23:45 UTC to 00:30 UTC, we experienced an outage in some Atlassian Intelligence features for Confluence, Jira Work Management, Jira Service Management, Jira Software, Atlassian Bitbucket, Jira Product Discovery, Atlas, and Compass. The issue has been resolved and the service is operating normally.</p><p><small>Jan <var data-var='date'>15</var>, <var data-var='time'>00:47</var> UTC</small><br><strong>Monitoring</strong> - We have identified the root cause of the increased errors and have mitigated the problem. We are now monitoring closely.</p><p><small>Jan <var data-var='date'>15</var>, <var data-var='time'>00:31</var> UTC</small><br><strong>Investigating</strong> - We are investigating an issue with Atlassian Intelligence that is impacting some Confluence, Jira Work Management, Jira Service Management, Jira Software, Atlassian Bitbucket, Jira Product Discovery, Atlas, and Compass Cloud customers. We will provide more details within the next hour.</p>