Disroot/Website
Disroot
/
Website
14
15
Fork 14
Code Pull Requests 1 Releases Activity

correction

This commit is contained in:
antilopa 2022-02-01 12:43:54 +01:00
parent e8811005be
commit 867ccb2814
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
pages/08.blog/outage_31jan/item.en.md
View File

@ -9,12 +9,12 @@ body_classes: 'single single-post'
---
Yesterday, on the 30th of January, we had performed some maintenance work in the datacenter. Main task was replacement of a faulty raid controller cache battery which broke few days before. Although the procedure went as planned, during the reboot process of the server, which was needed in order to perform part replacement, the file system got corrupt. This means that in order to bring the server back online we had to repair the filesystem. This operation took in total over 12 hours, and during that time, email, xmpp chat and nextcloud were not accessible. Additionally some dataloss has occured, meaning you might have lost a file or email you uploaded shortly before the server reboot.
Yesterday, on the 30th of January, we had performed some maintenance work in the datacenter. Main task was replacement of a faulty raid controller cache battery which broke few days before. Although the procedure went as planned, during the reboot process of the server, which was needed in order to perform part replacement, the file system got corrupt. This means that in order to bring the server back online we had to repair the filesystem. This operation took in total over 12 hours, and during that time, email, XMPP chat and Nextcloud were not accessible. Additionally some dataloss has occured, meaning you might have lost a file or email you uploaded shortly before the server reboot.
We would like to send you our deepest appologies and we hope we did not disturb your day with the outage. Although this was inavitable (looks like), we are going to work on better communication during the time of outage. We hope to minimize such long outages in the future with new hardware we are preparing to purchase, which will enable us to rebuild the current infrastructure adding some redundancy (as much as we can afford of course).
Although operational now, we still see few corrupt inodes on the data partition where we store emails and cloud files. This means we will need to run another check on that disk partition, which means we will have to put the service down again for few hours. This means we will continue maintenance work on saturday 5th Feb 2022 starting at 0:00 CET. We think it won't take much time but it is wise to be ready for few hours of downtime. Saturday night (european timezone) seems to be moment where traffic on the server is the lowest so we hope to impact as little people possible.
Although operational now, we still see few corrupt inodes on the data partition where we store emails and cloud files. This means we will need to run another check on that disk partition, which means we will have to put the service down again for few hours. This also means we will continue maintenance work on Saturday 5th Feb 2022 starting at 0:00 CET. We think it won't take much time but it is wise to be ready for few hours of downtime. Saturday night (european timezone) seems to be moment where traffic on the server is the lowest so we hope to impact as little people possible.
**Once again. Sorry for the outage and thank you for understanding and support during this time.**