Emergency Maintenance 2019-04-07

I am taking all servers offline for emergency maintenance in a few minutes. I will update this thread with info as I am able. This is related to the lag we have been experiencing in all servers over the last few days.

 

It "feels" like a storage problem. I haven't troubleshot *nix io issues in a very long time, but let me know if you'd like some assistance... If you're not already monitoring disk queue length, that's where I would be watching.

 

Testing of the hardware has begun. The test will take a minimum of 30 minutes. I will update with my findings when the test is complete.

 

Thanks for all of your help! Your transparency on the situation as well is very helpful.

 

Hard drives appear to be in working order. Testing CPU and memory now.

 

Testing is 50% complete. Seems to be taking quite a while to test the memory.

 

CPU test is completed successfully. Memory test at 68% complete.

 

If all the hardware tests pass successfully, it means there is some kind of problem with the software running on the server, which is a much more complicated issue to resolve. We shall see...

 

I recently read that IT professionals don't use things like "smart" TVs, doorknobs, thermostats, refrigerators, anything. Too much software to upgrade, maintain, etc. Is this true with you too, c0w?

 

Yes! Haha!

Testing memory still at 68%...