Some systems are experiencing issues

About This Site

Maintenances and incidents regarding our services, networks and datacenters can be followed here.

Scheduled Maintenance
[MAINTENANCE DC5-A] Racks H1;H2:H3;H4;H5;H6;H7;H8;H9;H10;H11;H12;H13;H14;H15;H16;H17

A scheduled maintenance is planned in DC5-A with a physical displacement of your servers listed above in order to optimize and upgrade our infrastructure. Servers will be unavailable for the duration of the maintenance detailed below.

This status will be updated as the operations are going through to keep you informed about the events.

Start : July 15th, 2020 - 2000UTC (2200LT)

End : July 16th, 2020 - 0300UTC (0500LT)

[DC5] Load test of generators

[DC5] Load tests of generators

As part of the preventive maintenance of the Datacenter, we will proceed to a redundancy test of the Datacenter infrastructure by carrying out generator tests.

The operation consists in simulating a cut of electrical TGBT and validate load recovery as well as function through generators. Running time on the generators will be reduced in view of the low load on the site.

Impact : None expected

Start: 16-Jul-2020 0600Z (0800LT)
End: 16-Jul-2020 1200Z (1400LT)
[MAINTENANCE DC5-A] Racks D1;D2;D3;D4;D5;D6;D7;D8;D9;D10;D11;D13

A scheduled maintenance is planned in DC5-A with a physical displacement of your servers listed above in order to optimize and upgrade our infrastructure. Servers will be unavailable for the duration of the maintenance detailed below.

This status will be updated as the operations are going through to keep you informed about the events.

Start : July 20th, 2020 - 2000UTC (2200LT)

End : July 21th, 2020 - 0300UTC (0500LT)

[DEDIBOX] API and Console session handler migration

[DEDIBOX] API and Console session handler migration

We'll migrate our Console and API session handler.

Console and API will be unavailable during the migration process.

Start : 25-Jun-2020 0300Z 0500LT

Duration : 30 min

[MAINTENANCE DC5-A] Racks A1;A2;A3;A4;A5;A6;A7;A8;A9;A10;A11;A12;A13;A14;A15;A16;A17;A18;A19

A scheduled maintenance is planned in DC5-A with a physical displacement of your servers listed above in order to optimize and upgrade our infrastructure. Servers will be unavailable for the duration of the maintenance detailed below.

This status will be updated as the operations are going through to keep you informed about the events.

Start : July 27th, 2020 - 2000UTC (2200LT)

End : July 28th, 2020 - 0300UTC (0500LT)

[DC3] Semi-annual maintenance of chillers

[DC3] Semi-annual maintenance of chillers

We will do the semi-annual maintenance of the chillers, according to the manufacturer’s maintenance range.
Operations will be performed by the manufacturer and supervised by our maintainer, according to the following previsional schedule:

  • Monday 17/08: Chiller n°1, Substation A
  • Tuesday 18/08: Chiller n°2, Substation A
  • Wednesday 19/08: Chiller n°3, Substation A
  • Thursday 20/08: Chiller n°4, Substation B
  • Friday 21/08: Chiller n°5, Substation B
  • Monday 24/08: Chiller n°6, Substation B
  • Monday 31/08: Chiller n°7, Substation C
  • Tuesday 01/09: Chiller n°8, Substation C

Substations A and B supply the chilled water network of the computer halls.
Substation C supplies the chilled water network of the operators halls and the air handling units.

Impact : None expected

Start: 17-Aug-2020 0630Z (0830LT)
End: 01-Sep-2020 0330Z (0530LT)
[DC3] Load tests of generators and PLC software updates

[DC3] Load tests of generators and PLC software updates

As part of the preventive maintenance of the Datacenter, we will proceed to a redundancy test of the Datacenter infrastructure by carrying out generator tests chronologically on the following electrical chains : A / B / C / D / F1 and F2.

During the load tests on the electric chains A / B / C and D, we will proceed to F1 line up backup test.
Description of operations :

  • 25/08/2020:
    • Morning: Load test on the electric chain A with F1 line up backup test.
    • Afternoon: Load test on the electric chain B with F1 line up backup test.
  • 26/08/2020:
    • Morning: Load test on the electric chain C with F1 line up backup test.
    • Afternoon: Load test on the electric chain D with F1 line up backup test.
  • 27/08/2020: As part of the mobile chillers installation, we will update PLC software on chains F1 and F2. After we will proceed to load test on the chains F1 andF2.

During the F1 line up backup test, we will simulate a failure of the generators in order to validate the F1 lineup backup scenario.Our teams and usual maintenance companies will be mobilized for the perfect workflow.

Impact : None expected

Start: 25-Aug-2020 0600Z (0800LT)
End: 27-Aug-2020 1600Z (1800LT)

Past Incidents

Sunday 12th July 2020

No incidents reported

Saturday 11th July 2020

No incidents reported

Friday 10th July 2020

PAR1 Compute API DEV1 boot/stop issues

We noticed that some DEV1 instances are getting stuck on boot and returning in OFF state.
Main reason is we currently have no available stock in this server range, and we are actively working to power up some additional hypervisor in order to allow booting instances.

We are also facing another issue with some DEV1 instances getting stuck stopping while it was planned to migrate them from a hypervisor to another (because of a failing RAID).
Our teams are working on it.

We will update this status as soon as we have additional details.

===================

10/07/20 1440Z (1640LT)

Some additional hypervisors have been deployed, which will help booting up some additional instances.
We are still working on adding more hypervisors in production.

10/07/20 1230Z (1430LT)

Issue identified by our product team.
We are working on deploying new hypervisors asap.
We are awaiting further details regarding next steps.

User Accounts API Random login issue - "Permission missing"

It has been brought to our attention that some users are unable to login on their Scaleway account, reporting a "Permission missing" issue.

Problem has been escalated to our dev team and is currently being investigated.
We will update this status as soon as we have additional details.

===================

10/07/20 1320Z (1520LT)

Issue has now been fixed, login is working correctly.

10/07/20 0900Z (1100LT)

Issue escalated to dev team, awaiting further details.

Thursday 9th July 2020

[AMS1] Maintenance on an agregation router, scheduled 3 days ago

We have scheduled a maintenance on aggregation router for n9k-h2-1.ams1 and n9k-h6-1.ams1, forwarding traffic in AMS1.Private network will remain unchanged, and no traffic disruption is expected for public network.

Start : July 9th, 2020 - 0800UTC (1000LT)

Wednesday 8th July 2020

[NETWORK] Fiber emergency maintenance between DC3/DC5-A, scheduled 3 days ago

Our fiber provider will carry out an emergency maintenance affecting the RPN connectivity between DC3 and DC5-A.

Impact : During the maintenance window, no loss of service is expected but some packet loss could be noticed

Starting: 8rd of July 11:00 PM CEST
Ending: 9th of July: 06:00 PM CEST
[DC2] Room 101 network maintenance, scheduled 4 days ago

We have scheduled a maintenance on aggregation router, forwarding traffic in room 101 for racks in row M,N,H,K

Private network will remain unchanged, and no traffic disruption is expected for public network.

Start : July 8th, 2020 - 0800UTC (1000LT)

[Database] Scheduled maintenance on all running instances, scheduled 4 days ago

Our Database team is scheduling an infrastructure upgrade on PostgreSQL RDB instances. We expect a short downtime on both classic and HA databases.

All instances will be updated progressively in a 3-hour window.

Impact :

A few seconds downtime on HA RDB instances. ~1min downtime on other RDB instances. Start : 08-July-2020 between 0600Z (0700LT) and 0800Z (0900LT)

[DC3] Scheduled maintenance on RPN SAN HA

Location : DC2 DC3 What will be done : Network equipment maintenance on :

  • San HA HDD 17-31
  • San HA SSD 4 What will be done : We are modifying the firmware of a switch at DC2. This will require a reboot of the equipment. Therefore the SANs master will be switched from DC3 instead of DC2 during the maintenance.

Impact : No service interruption. The keepalive will ensure the service remain available and does not timeout. You may notice a short lag but your SAN won't timeout. There redundancy will be lost during this maintenance.

Start : 08-july-2020 1700Z (1900LT) Duration : 30 minutes

[DC3] Scheduled maintenance on RPN SAN (HD / STD)

Location : DC3 What will be done : Network equipment maintenance on :

  • San std 25->27
    Details : A switch have to be rebooted as we are upgrading the firmware.

Impact : The service will be interrupted for 30 minutes. Once the maintenance is done you will have to reconnect your iSCSi target as the connection will have timeout during our migration. Note : If you have any questions regarding that maintenance feel free to contact our support team.

Start : 08-july-2020 1700Z (1900LT) Duration : 30 minutes

User Accounts API Account's modification issues

We are currently facing an issue regarding SSH key updates (creation, deletion, updates) and multi-users management.
You will not be able to apply new multi-users settings nor update SSH keys.

We will update this status as soon as we have additional details.

===================

08/07/20 1305Z (1505LT)

Issue has been fixed, accounts API is stable again.

08/07/20 1245Z (1445LT)

Issue escalated to our engineering team.
Issue has been identified, awaiting further updates.

Web [WEB HOSTING] Websites and FTP are not working on PF43

We are currently facing an issue with websites and FTP hosted on PF43.

Websites and FTP accounts are not working.

===================

2020-07-08 1200Z (1400LT)

The issue has been fixed.

2020-07-08 0912Z (1112LT)

Issue has been escalated to local team

Compute Nodes PAR1 - upgrade network control plane (CP)

Our teams are scheduling a Maintenance this Wednesday, July 8th.

Location : Paris

What is done : We'll upgrade our network control plane (CP) for instances.

Impact : only CP redundancy will be affected for p14's clusters 1 to 20.

Start : 08.07.2020 0700Z 0900LT End : 08.07.2020 1100Z 1300LT

Duration : 240 minutes

===================

2020-07-08 0847Z (1047LT)

Hardware upgrade done, CP recovered its redundancy

2020-07-08 0725Z (0925LT)

Migration started, overlay isolated, starting upgrade with 1st reboot

Database [Database] Scheduled maintenance on all running instances

Our Database team is scheduling an infrastructure upgrade on PostgreSQL RDB instances. We expect a short downtime on both classic and HA databases.

All instances will be updated progressively in a 3-hour window.

Impact :

A few seconds downtime on HA RDB instances. ~1min downtime on other RDB instances. Start : 08-July-2020 between 0600Z (0700LT) and 0800Z (0900LT)

DC3 Network [DC3] Switch room 4-6 bay E13 down

The public network switch room 4-6 bay E13 is no longer responding.

Our on-call network team has been notified of this issue.

===================

2020-07-08 0423Z (0623LT)

Issue has been escalated to local team

2020-07-08 0518Z (0718LT)

Issue has been fixed.

Tuesday 7th July 2020

No incidents reported

Monday 6th July 2020

No incidents reported