Status impact color legend
- Black impact: None
- Yellow impact: Minor
- Orange impact: Major
- Blue impact: Maintenance
[fr-par-1] Latency Block Storage Issues
Incident
Report for Scaleway
Resolved
A BSSD volume cluster was put under heavy stress since 29/01/2025 - 12h30 UTC because of unusual number of volumes to purge. Issue has been diagnosed and patch applied at 29/01/2025 - 16h35 UTC to limit latencies impact.
Preventive measures have been taken to prevent another mishandling during this kind of maintenance.
Impact: high latencies on block storage in FR-PAR-1 since 29/01/2025 - 12h30 UTC until 29/01/2025 - 22h00 UTC.
Preventive measures have been taken to prevent another mishandling during this kind of maintenance.
Impact: high latencies on block storage in FR-PAR-1 since 29/01/2025 - 12h30 UTC until 29/01/2025 - 22h00 UTC.
Monitoring
99th latencies are back to acceptable levels. 99.95th may be still higher than usual for the next 10 hours until all queued operations are done.
We continue to monitor this issue.
We continue to monitor this issue.
Update
Some process and servers are still stuck in high load. Latencies have been limited but are still high.
We are still working on it.
We are still working on it.
Identified
A BSSD volume cluster was put under heavy stress since 12h30 UTC because of unusual number of volumes to purge. It provoked high latencies on block storage in FR-PAR-1
Issue has been diagnosed and patch applied at 16h35 UTC to limit latencies impact.
Low latencies volumes (class SBS) are not impacted.
Issue has been diagnosed and patch applied at 16h35 UTC to limit latencies impact.
Low latencies volumes (class SBS) are not impacted.
Investigating
It impacts block storage users on fr-par-1. They can experiment high latencies. RDB is as well impacted by this issue.
This incident affected: Elements - AZ (fr-par-1) and Elements - Products (Block Storage, Databases).