Thanks for your patience. We've been back online for about 20 hours or so, and you folks have been extremely patient and supportive. We seem to have brought all our tools backs and things look stable. But if you notice anything not working, please report it.
Here's a summary of what happened. We've scheduled a replacement for a faulty disk and the time announced for that was shifted due to delays in availability. That disk drive is the primary boot drive. The procedure would have required a couple of hours only, as was announced, and scheduled for early morning to lessen the impact. We've also installed a new drive to have as primary, and planned to copy the old drive content to it. The screw up was two fold: 1. The old primary drive just become unusable and cannot be copied confidently. 2. The copy was not made onto the new drive, but to our database drive. Hence, we lost everything from the bad drive and the good drive, and the new drive remained empty. Don't want to blame any of the host/support staff that helped us. Ticket systems and shifts can cause such misunderstanding and it just happened.
This could have been fixed in less than 24 hours, by rebuilding the system from scratch, then restoring from our daily backups, but that would result in loss of any posts on Nov 6th (between daily backups), so we've tried a few tricks, resulting in the loss of a full day, and due to accidentally copying onto the database server with a reformat, there was no way of recovering Nov 6th data. So we then went to the building from scratch solution, which was difficult due to our heavily customized and outdated forum software (myplan, gallery, etc), so had to search for old OS/server software versions compatible with our software. Once the system was rebuilt and setup, we started restoring our data and testing it.
That's basically what happened
Thanks for your patience and understanding.
Wa'il