Some systems are experiencing issues

About This Site

Maintenances and incidents regarding our services, networks and datacenters can be followed here.

Scheduled Maintenance
[MAINTENANCE DC5-A] Racks D14;D15;D16;D17;D18;D19;C1;C2;C3;C4;C7;C8

A scheduled maintenance is planned in DC5-A with a physical displacement of your servers listed above in order to optimize and upgrade our infrastructure. Servers will be unavailable for the duration of the maintenance detailed below.

This status will be updated as the operations are going through to keep you informed about the events.

Start : July 6th, 2020 - 2000UTC (2200LT)

End : July 7th, 2020 - 0300UTC (0500LT)

[MAINTENANCE DC5-A] Racks H1;H2:H3;H4;H5;H6;H7;H8;H9;H10;H11;H12;H13;H14;H15;H16;H17

A scheduled maintenance is planned in DC5-A with a physical displacement of your servers listed above in order to optimize and upgrade our infrastructure. Servers will be unavailable for the duration of the maintenance detailed below.

This status will be updated as the operations are going through to keep you informed about the events.

Start : July 15th, 2020 - 2000UTC (2200LT)

End : July 16th, 2020 - 0300UTC (0500LT)

[DC5] Load test of generators

[DC5] Load tests of generators

As part of the preventive maintenance of the Datacenter, we will proceed to a redundancy test of the Datacenter infrastructure by carrying out generator tests.

The operation consists in simulating a cut of electrical TGBT and validate load recovery as well as function through generators. Running time on the generators will be reduced in view of the low load on the site.

Impact : None expected

Start: 16-Jul-2020 0600Z (0800LT)
End: 16-Jul-2020 1200Z (1400LT)
[DEDIBOX] API and Console session handler migration

[DEDIBOX] API and Console session handler migration

We'll migrate our Console and API session handler.

Console and API will be unavailable during the migration process.

Start : 25-Jun-2020 0300Z 0500LT

Duration : 30 min

[DC3] Semi-annual maintenance of chillers

[DC3] Semi-annual maintenance of chillers

We will do the semi-annual maintenance of the chillers, according to the manufacturer’s maintenance range.
Operations will be performed by the manufacturer and supervised by our maintainer, according to the following previsional schedule:

  • Monday 17/08: Chiller n°1, Substation A
  • Tuesday 18/08: Chiller n°2, Substation A
  • Wednesday 19/08: Chiller n°3, Substation A
  • Thursday 20/08: Chiller n°4, Substation B
  • Friday 21/08: Chiller n°5, Substation B
  • Monday 24/08: Chiller n°6, Substation B
  • Monday 31/08: Chiller n°7, Substation C
  • Tuesday 01/09: Chiller n°8, Substation C

Substations A and B supply the chilled water network of the computer halls.
Substation C supplies the chilled water network of the operators halls and the air handling units.

Impact : None expected

Start: 17-Aug-2020 0630Z (0830LT)
End: 01-Sep-2020 0330Z (0530LT)
[DC3] Load tests of generators and PLC software updates

[DC3] Load tests of generators and PLC software updates

As part of the preventive maintenance of the Datacenter, we will proceed to a redundancy test of the Datacenter infrastructure by carrying out generator tests chronologically on the following electrical chains : A / B / C / D / F1 and F2.

During the load tests on the electric chains A / B / C and D, we will proceed to F1 line up backup test.
Description of operations :

  • 25/08/2020:
    • Morning: Load test on the electric chain A with F1 line up backup test.
    • Afternoon: Load test on the electric chain B with F1 line up backup test.
  • 26/08/2020:
    • Morning: Load test on the electric chain C with F1 line up backup test.
    • Afternoon: Load test on the electric chain D with F1 line up backup test.
  • 27/08/2020: As part of the mobile chillers installation, we will update PLC software on chains F1 and F2. After we will proceed to load test on the chains F1 andF2.

During the F1 line up backup test, we will simulate a failure of the generators in order to validate the F1 lineup backup scenario.Our teams and usual maintenance companies will be mobilized for the perfect workflow.

Impact : None expected

Start: 25-Aug-2020 0600Z (0800LT)
End: 27-Aug-2020 1600Z (1800LT)

Past Incidents

Sunday 5th July 2020

No incidents reported

Saturday 4th July 2020

No incidents reported

Friday 3rd July 2020

[NETWORK] Fiber emergency maintenance between DC3/DC5, scheduled 2 days ago

Our fiber provider will carry out an emergency maintenance affecting the connectivity between DC3 and DC5-A. During the maintenance window, we will lose our network redundancy between the two sites. Start: 3-Jul 0900Z (1100LT) End: 4-Jul 0400Z (0600LT)

Console Console / Website random unavailability

Our web console is unfortunately not currently accessible. Investigations are ongoing in order to solve the issue as soon as possible. We will update you on the progress of this resolution when we have more details to share.

===================

07.03.2020 1018Z (1218LT)

Issue was due to an internal load-balancer issue. It has been solved by our engineering team.

07.03.2020 1000Z (1200LT)

Issue has been escalated to local team

Thursday 2nd July 2020

Mail [Web Hosting] Webmail access error

We are currently facing an issue with webmail access.

Webmail login attempts return to 504 error.

===================

07.02.2020 1610Z (1810LT)

Issue has been fixed, webmail is now available

07.02.2020 1545Z (1745LT)

Issue has been escalated to local team

Wednesday 1st July 2020

[AMS] IPv6 routing improvement, scheduled 4 days ago

[AMS] IPv6 routing improvement

We will start announcing 2001:bc8:1400::/38 and 2001:bc8:1800::/38 as more-specific prefixes in AMS. This will improve IPv6 external latency in AMS for hosts in those prefixes (for incoming traffic).

Start : 1-Jul-2020 0915Z 1115LT
Compute Nodes PAR1 - upgrade network control plane (CP)

Our teams are scheduling a Maintenance this Wednesday, July 1st.

Location : Paris

What is done : We'll upgrade our network control plane (CP) for instances.

Impact : only CP redudancy will be affected for p14's clusters 1 to 20.

===================

01.07.2020 0855Z (1055LT)

Hardware upgrade done, configuration updated, all services recovered their redundancy

01.07.2020 0745Z (0945LT)

Isolation done, hardware upgrade in progress

Start : 01.07.2020 0700Z 0900LT
End : 01.07.2020 1100Z 1300LT

Duration : 240 minutes

Tuesday 30th June 2020

[DC5] Power Unit Maintenance, scheduled 5 days ago

We will perform maintenance on the DC5 power units.

No impact is expected.

Location: DC5

Impact : None expected

Start: 30-Jun-2020 0600Z (0800LT)
End: 30-Jun-2020 1400Z (1600LT)
Mail [Web hosting] SMTP

We are currently encountering an issue regarding our SMTP servers. Your email may not be delivered properly. Our webhosting team is investigating the situation. Next update once we have more information

===================

07.01.20 0730Z (0930LT)

Issue has now been fixed, SMTP servers are now answering to webmail requests correctly.

06.30.20 1930Z (2130LT)

Issue has been escalated to local team

Object Storage [GLACIER] File restoration issues

We are currently encountering an issue with our restore system from GLACIER to STANDARD.
Our storage team is working on a fix and a patch should be deployed soon.
We will let you know once we have more information. No ETA for now.

You may retry to restore once in a while until the issue is properly fixed.

===================

06/30/2020 1230Z (1430LT)

The patch has been deployed, everything is now working properly.

06/30/2020 0920Z (1120LT)

The product team has identified the root cause and working on a fix

06.30.2020 0900Z (1100LT)

Issue has been escalated to local team

DC5-A Network Loss of RPN network redundancy

We are noticing an issue with one of our two RPN links in DC5-A.
The other link is still up but no redundancy is assured until it has been sorted out.

===================

02/07/20 1430Z (1630LT)

The issue has been fixed. Everything is back to normal.

1/07/20 1530Z (1730LT)

All teams are still working on the issue at this time. Cable pulling work has started. We will keep you updated about our progress.

30/06/20 1430Z (1630LT)

All our teams are still working to solve this collective outage.
The area where to search for the point of impact has been significantly reduced.

30/06/20 0800Z (1000LT)

Our Network team is aware of the issue and working on it.
Awaiting further updates.

Monday 29th June 2020

User Accounts API [USER] API Permission unreachable

[USER] API Permission unreachable

We received multiple reports regarding API permission calls which returns 503 errors. Issue is currently being investigated by our engineering team, and we will update this status as soon as we have additional details.

===================

29/06/29 15302 (1730LT)

A fix has been deployed by the product team

29/06/20 1530Z (1730LT)

Issue has been escalated to product team

Console TTY console not loading

We received multiple reports regarding random access issues on instances' TTY console.
Issue is currently being investigated by our engineering team, and we will update this status as soon as we have additional details.

===================

01/07/20 1700Z (1900LT)

Issue has now been fixed, you shouldn't face issues loading TTY console from your panel.
If you are still experiencing problems using the TTY console, please directly get in touch with our support team by ticket about it.

01/07/20 1500Z (1700LT)

The origin of the incident has been identified. The engineering team is currently working on a fix. It will be deployed soon.

29/06/20 0600Z (0800LT)

Issue has been escalated to product team.