All systems are operational

About This Site

Maintenances and incidents regarding our services, networks and datacenters can be followed here.

Scheduled Maintenance
Refrigeration units preventive maintenance

We will undertake a preventive maintenance on our refrigeration units on DC3.

Impact: Minor loss of redundancy, no customer impact.

START: 10-Oct-2019 0630Z 0830LT

END: 11-Nov-2019 0330Z 0530LT

DC3 - Redundancy test of DC3 electrical generators

DC3 electrical generators - Redundancy test of DC3 electrical generators

Location : DC3 What is done : As part of the preventive maintenance of the Datacenter, we will proceed to a redundancy test of the Datacenter infrastructure. The operation consists of simulating a power failure on the MVLBs and validating the electrical load recovery and working on generator for 4 hours. Impact : no impact on client servers

Start : 11-Oct-2019 0700-1400Z 900-1600LT

Duration : 4 hours

RPN AMS1/DC3

Upgrade over the RPN network between AMS1 and DC3

Network supplier must perform maintenance on his Network. Maintenance window is from October 26th, 2019 at 10:00 pm UTC to October 27th, 2019 at 4:00 am UTC. The expected Impact length is 6 hours at the maximum, during the maintenance window

Impact : Downtime on the rpn network between AMS1 and DC3

Start : 26-Oct-2018 2200Z 0000LT

It will last : 6 hours

DC3 - Semi-annual maintenance of distribution pumps

DC3 - Maintenance of distribution pumps

Location : DC3 Substation-A, Substation-B, Substation-C

What is done : We will perform the semi-annual maintenance of the distribution pumps. Operations will be performed by our maintainer under our permanent supervision, according to the following previsional schedule:

Wednesday November 6th: Sub-station A

Thursday November 7th Sub-station B

Friday November 8th Sub-station C

Substations A and B supply the chilled water network of the computer halls. Substation C supplies the chilled water network of the operators halls and the air handing units.

Impact : no impact for customers

Start : 6-Nov-2019 0700Z 0900LT

Duration : 3 Days

DC3 - Preventive maintenance of air-handling units

DC3 - Preventive maintenance of air handling units

Location : DC3

What is done : preventive maintenance of air handling units will be performed on the whole site.

Thursday November 12th: Room 01 and 02

Wednesday November 13th: Room 03 and 04

Thursday November 14th: TGBT A and B

Friday November 15th: TGBT C and D

Monday November 18th: TGBT F1 and F2

Impact : loss of redundancy during the preventive maintenance of a TGBT air handling unit. No impact on TGBT cooling.

Start : 12-Nov-2019 0700Z 0900LT

Duration : 5 Days

Past Incidents

Sunday 22nd September 2019

No incidents reported

Saturday 21st September 2019

No incidents reported

Friday 20th September 2019

No incidents reported

Thursday 19th September 2019

DC3 [NETWORK] Backup fiber loss between DC5-A and DC3

We’ve lost our backup path between DC5-A and core Network in DC3. We’re currently working on backup recovery.

===================

09.19.19 0845 (1045LT)

Issue has been escalated to local team

09.19.19 0925 (1125LT)

issue escalated to our provider

09.19.19 1130 (1330LT)

Issue comes from public road works. The fibre was damaged. Our supplier is working on it.

09.19.19 1630Z (1830LT)

Two of the fiber trunks are impacted. Preparatory reconstruction work is in progress. Provider doesn't have an ETA yet due to the nature of the intervention. We're continuing our monitoring of the infrastructure.

09.20.19 0650Z (0850LT)

We are still working on the fiber link, the rpn will be unavailable between DC5-A and DC3 during 15 minutes this morning.

09.20.19 0900Z (1100LT)

20% of the fibres are fixed. Our supplier is working on it.

09.20.19 1220Z (1420LT)

25% of the fibres are fixed. Our supplier is working on it.

09.21.19 1000Z (1200LT)

Weldings are still ongoing but not over yet. There has been an escalation to prioritize the comeback of our links and redundancy to DC5-A

09.21.19 1130Z (1330LT)

Links are back online and running. We will keep on monitoring optic levels.

Mail Mail Hosting - Mail delivery issue

We are currently experiencing a general incident related to messaging : delay in receiving emails, non-accessible emails. Status closed with service back to normal.

===================

09.19.19 0800Z (1000LT)

Issue has been escalated to local team

09.19.19 0900Z (1120LT)

The issue has been identified and fixed by the engineering. The situation should be back to normal. There could be eventually some slowness.

09.19.19 1000Z (1200LT)

Closing the status. All is back to normal and this is confirmed also by our customers.

AMS1 Network ACL version update [ Network ]

We applied new versions of ACL on the router bb1.ams1 this morning at 7.00am UTC. This had an impact on traffic flowing to Hopus until we fixed the issue. Your traffic may be dropped for few minutes if going through this router and inbound from HOPUS

Thu Sep 19 07:03:56.654 UTC : new ACL applied
Thu Sep 19 07:05:39.851 UTC : end of issue due to rollback
Thu Sep 19 07:32:01.942 UTC : final fix, good configuration applied without any issue

Made to decrease outgoing flood and spoofing from our network to outside

Start : 19-Sept-2019 0500Z 0700LT

Duration : 30 minutes

Wednesday 18th September 2019

[DEDIBOX] Installations CentOS not functional

We detected an issue with installations of following systems: CentOS 7.6.

===================

18/09/2019 1215Z (1405LT)

Issue has been escalated to local team and necessary steps are being taken.

UPDATE: This issue is now fixed

RPN services [DC3] Room 3 - Zone 3-1 - Rack B10 - RPN switch outage

RPN switch s31-b10.rpn.dc3, in Room 3 - Zone 3-1 - Rack B10 is down, RPN connectivity for customers in this rack is down.

===================

09.18.19 1435Z (09.18.19 1635LT)

Issue is fixed

09.17.19 2235Z (09.18.19 0035LT)

Issue has been escalated to local team

Tuesday 17th September 2019

AMS1 - Preventive maintenance, scheduled 5 days ago

In order to upgrade one peering router in AMS1, we will isolate a router and reload it.

Impact : No impact on traffic from / to AS12876 as we have enough capacity on other peering routers.

Start: 09.16.2019: 07.00am UTC End: 09.16.2019: 09.00am UTC

UPDATE: Maintenance went without any issues. We are now done.

Monday 16th September 2019

DC3 DC3 - Battery Fire on a Power Feed

DC3 - A battery fire as been detected and is currently being contained. No impact on services.

===================

16/09/19 0930Z (1130LT)

Datacenter technicians are now controlling the fire.

16/09/19 0950Z (1150LT)

Fire under control. The smoke extraction of the room is in progress. No degraded operation for the moment

16/09/19 1010Z (1210LT)

The two failed battery chains will be isolated. All inverters are back in normal and nominal operation status.

Console Console / API instabilities

Our team noticed instabilities on both our Scaleway website, console and APIs.
We are currently investigating the issue and will update this status accordingly as soon as possible.

===================

09.16.19 0837Z (1037LT)

The product team has identified the root cause and working on a fix

09/16/19 0845Z (1045LT)

Issue is now closed, services are up and stable.
Thank you for your patience.

Dedibackup Dedibackup instability

Dedibackup dc3-7 connection issue. There's currently connection issues on the Dedibackup 7 of the DC3. Next update by 09.16.19 11:00:00 (13:00:00)

===================

09.16.19 0800Z (1000LT)

The product team has identified the root cause and working on a fix

09.16.19 1400Z (1600LT)

Issue has been fixed now, if you are still experiencing connection issues please contact our support team via ticket.