All Systems Operational
Uptime over the past 90 days. View historical uptime.
Simple Docker ? Operational
Windows Docker Host ? Operational
Linux Docker Host ? Operational
Central Proxy ? Operational
Kubernetes ? Operational
Central Kubernetes ? Operational
Legacy Kubernetes Operational
Devspaces Kubernetes Operational
Containeryard Kubernetes Operational
Central Registry ? Operational
Central Storage ? Operational
Databases ? Operational
SqlServerDB ? Operational
OracleDB ? Operational
AuroraDB ? Operational
PostgresDB ? Operational
DynamoDB ? Operational
Central VmWare Operational
Network ? Operational
Hypervisor ? Operational
Management Operational
AWS Interconnection ? Operational
CICD Operational
90 days ago
100.0 % uptime
Today
Jenkins ? Operational
Jervis ? Operational
Jive Operational
90 days ago
100.0 % uptime
Today
Electric Flow ? Operational
TeamCity ? Operational
SonarQube ? Operational
Nexus ? Operational
AWS eu-west-1 Operational
AWS EC2 eu-west-1 Operational
AWS RDS eu-west-1 Operational
AWS S3 eu-west-1 Operational
AWS us-east-1 Operational
AWS EC2 us-east-1 Operational
AWS RDS us-east-1 Operational
AWS S3 us-east-1 Operational
Central Generic ? Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
had a major outage
had a partial outage

No incidents or maintenance related to this downtime.

Scheduled Maintenance
SqlServerDB Maintenance Window - CENTRAL-101529 Aug 24, 02:00-05:00 UTC
Update - On date 2019-08-24T02:00:00.000+0000, the product will undergo scheduled maintenance. The maintenance window is as follows:
Scheduled Start Date: 2019-08-24T02:00:00.000+0000
Scheduled End Date: 2019-08-24T05:00:00.000+0000

What is being addressed:
1. Update SQL05/R to CU8 SQL 2016
* https://support.microsoft.com/en-us/help/4505830/cumulative-update-8-for-sql-server-2016-sp2

2. Update AWS drivers to latest (as recommended by AWS under case 6348224541)
* Upgrade to ENA 2.1.0 as this version allows an increase to RX queues depth to a maximum of 8192 which prevents drop of RX packets in stress scenarios.
* configure RSS to exclude CPU 0 for ENA (less load on CPU 0)

3. Enable 2 trace flags to reduce resource utilization / cpu spikes
* enable trace flag 174 (reduce CPU usage)
* enable trace flag 3427 (reduce tempdb sorts)
https://support.microsoft.com/en-us/help/4465518/recommended-updates-and-configurations-for-sql-server
Aug 20, 12:00 UTC
Scheduled - On date August 24, 2019 02:00 GMT, SqlServerDB will undergo scheduled maintenance. The maintenance window is as follows:
Scheduled Start Date: August 24, 2019 02:00 GMT
Scheduled End Date: August 24, 2019 04:00 GMT

What is being addressed: 1. Update SQL05/R to CU8 SQL 2016
* https://support.microsoft.com/en-us/help/4505830/cumulative-update-8-for-sql-server-2016-sp2

2. Update AWS drivers to latest (as recommended by AWS under case 6348224541)
* Upgrade to ENA 2.1.0 as this version allows an increase to RX queues depth to a maximum of 8192 which prevents drop of RX packets in stress scenarios.
* configure RSS to exclude CPU 0 for ENA (less load on CPU 0)

3. Enable 2 trace flags to reduce resource utilization / cpu spikes
* enable trace flag 174 (reduce CPU usage)
* enable trace flag 3427 (reduce tempdb sorts)
https://support.microsoft.com/en-us/help/4465518/recommended-updates-and-configurations-for-sql-server
Aug 19, 13:37 UTC
Past Incidents
Aug 23, 2019

No incidents reported today.

Aug 22, 2019
Windows Docker Host is unavailable - CENTRAL-101649
Resolved - Outage has been resolved, outage duration was 4:23(Hours:Minutes).
The docker host stability is back to normal.
All docker containers are running now and working.
Aug 22, 21:51 UTC
Identified - Starting August 22, 2019 17:26 GMT we are experiencing an incident: Windows Docker Host is unavailable. This is affecting restricted set of customers.
We are working on the resolution and expect to have it resolved within the next 1 minutes.

We will provide an update every 30m or as soon as incident is resolved.
Aug 22, 17:56 UTC
Windows Docker Host Maintenance Window - CENTRAL-101659
Completed - The scheduled maintenance has been completed.
Aug 22, 21:45 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Aug 22, 20:45 UTC
Scheduled - On date August 22, 2019 20:45 GMT, Windows Docker Host will undergo scheduled maintenance. The maintenance window is as follows:
Scheduled Start Date: August 22, 2019 20:45 GMT
Scheduled End Date: August 22, 2019 21:45 GMT

What is being addressed: The Emergency maintenance window planned for Thursday, August 22, 2019, is going to cover these changes:

Reboot the DWEU1 (10.69.11.122) host
Planned downtime: 1h
Aug 22, 20:43 UTC
Aug 21, 2019

No incidents reported.

Aug 20, 2019

No incidents reported.

Aug 19, 2019

No incidents reported.

Aug 18, 2019

No incidents reported.

Aug 17, 2019
Jenkins Maintenance Window - CICDOPS-4367
Completed - The scheduled maintenance has been completed.
Aug 17, 22:25 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Aug 17, 22:00 UTC
Scheduled - On date August 17, 2019 22:00 GMT, Jenkins will undergo scheduled maintenance. The maintenance window is as follows:
Scheduled Start Date: August 17, 2019 22:00 GMT
Scheduled End Date: August 17, 2019 22:25 GMT

What is being addressed: for CR https://jira.devfactory.com/browse/CICDOPS-4364
Aug 12, 23:26 UTC
Linux Docker Host Maintenance Window - CENTRAL-101147
Completed - The scheduled maintenance has been completed.
Aug 17, 04:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Aug 17, 02:00 UTC
Update - On date 2019-08-17T02:00:00.000+0000, the product will undergo scheduled maintenance. The maintenance window is as follows:
Scheduled Start Date: 2019-08-17T02:00:00.000+0000
Scheduled End Date: 2019-08-17T04:00:00.000+0000

What is being addressed:
Is required to increase the size of volume /srv/docker_volumes/kayako to 3TB to support the migration of new customer data and be able to fulfill the customer requirements.
Aug 9, 19:05 UTC
Update - On date 2019-08-13T02:00:00.000+0000, the product will undergo scheduled maintenance. The maintenance window is as follows:
Scheduled Start Date: 2019-08-13T02:00:00.000+0000
Scheduled End Date: 2019-08-13T04:00:00.000+0000

What is being addressed:
Is required to increase the size of volume /srv/docker_volumes/kayako to 3TB to support the migration of new customer data and be able to fulfill the customer requirements.
Aug 9, 15:14 UTC
Scheduled - On date August 10, 2019 02:00 GMT, Linux Docker Host will undergo scheduled maintenance. The maintenance window is as follows:
Scheduled Start Date: August 10, 2019 02:00 GMT
Scheduled End Date: August 10, 2019 04:00 GMT

What is being addressed: Is required to increase the size of volume /srv/docker_volumes/kayako to 3TB to support the migration of new customer data and be able to fulfill the customer requirements.
Aug 7, 23:21 UTC
SqlServerDB Maintenance Window - CENTRAL-101055
Completed - The scheduled maintenance has been completed.
Aug 17, 04:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Aug 17, 02:00 UTC
Update - On date 2019-08-17T02:00:00.000+0000, the product will undergo scheduled maintenance. The maintenance window is as follows:
Scheduled Start Date: 2019-08-17T02:00:00.000+0000
Scheduled End Date: 2019-08-17T04:00:00.000+0000

What is being addressed:
* upgrade SQL06EU from r3.8xlarge to r4.8xlarge and move SQL06EU to a Dedicated Host
* apply latest patch for MSSQL (CU 16 for SQL Server 2017), as it contains fixed for non-yielding scheduler conditions
Aug 7, 12:04 UTC
Scheduled - On date August 10, 2019 02:00 GMT, SqlServerDB will undergo scheduled maintenance. The maintenance window is as follows:
Scheduled Start Date: August 10, 2019 02:00 GMT
Scheduled End Date: August 10, 2019 04:00 GMT

What is being addressed: * upgrade SQL06EU from r3.8xlarge to r4.8xlarge and move SQL06EU to a Dedicated Host
* apply latest patch for MSSQL (CU 16 for SQL Server 2017), as it contains fixed for non-yielding scheduler conditions
Aug 5, 15:25 UTC
PostgresDB Maintenance Window - CENTRAL-101186
Completed - The scheduled maintenance has been completed.
Aug 17, 02:30 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Aug 17, 02:00 UTC
Scheduled - On date August 17, 2019 02:00 GMT, PostgresDB will undergo scheduled maintenance. The maintenance window is as follows:
Scheduled Start Date: August 17, 2019 02:00 GMT
Scheduled End Date: August 17, 2019 02:30 GMT

What is being addressed: We need to reboot central-postgres01 to take latest value for max_connections.
Total downtime would be < 2 minutes.
These customers will be impacted -
http://cdm-cdb.aureacentral.com/admin/products?connection=postgres01&engine=POSTGRE
Aug 12, 06:11 UTC
Aug 16, 2019
SqlServerDB Maintenance Window - CENTRAL-101413
Completed - The scheduled maintenance has been completed.
Aug 16, 04:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Aug 16, 03:00 UTC
Scheduled - On date August 16, 2019 03:00 GMT, SqlServerDB will undergo scheduled maintenance. The maintenance window is as follows:
Scheduled Start Date: August 16, 2019 03:00 GMT
Scheduled End Date: August 16, 2019 04:00 GMT

What is being addressed: Moving SQL05 to new dedicated host due to network issues on existing host.

Downtime: <1 minute for products
Aug 15, 16:07 UTC
Aug 15, 2019

No incidents reported.

Aug 14, 2019

No incidents reported.

Aug 13, 2019

No incidents reported.

Aug 12, 2019

No incidents reported.

Aug 11, 2019

No incidents reported.

Aug 10, 2019
Nexus Maintenance Window - CICDOPS-4360
Completed - The scheduled maintenance has been completed.
Aug 10, 23:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Aug 10, 22:00 UTC
Scheduled - On date August 10, 2019 22:00 GMT, Nexus will undergo scheduled maintenance. The maintenance window is as follows:
Scheduled Start Date: August 10, 2019 22:00 GMT
Scheduled End Date: August 10, 2019 23:00 GMT

What is being addressed: This mw is to execute CR https://jira.devfactory.com/browse/CICDOPS-4357
Aug 8, 22:29 UTC
Aug 9, 2019

No incidents reported.

← Incident History Powered by Statuspage