Lightening and UPS units.
Last night we had pretty heavy Lightening storm, although I slept through it. My wife’s assistant called me this morning to tell me “The black box thingie on the floor of the server closet is foaming, and I can’t get to the internet”…
So I run out of my office at work, and arrive at my wifes office to find…sure enough, the UPS unit is foaming, and both production servers are down (powered off!). Wow, what happened here? It seems that the electrical storm hit the building somewhere and our dedicated server circuit carried the brunt of it. The surge hit the UPS, and blew the lid off the lead-acid (car battery style) battery inside, the electrolyte solution inside boiled out and reacted with the linseed oil and years of dirt on the hardwood floor. So in all, we’re down one UPS unit, 6 square feet of hardwood flooring, but both servers rebooted with out a problem. The UPS had actually kept it’s brain and told both servers to “shutdown -h now” and they dismounted all drives and shutdown. (Whew!).
Moral of the story - Get acid mats for under your UPS units! and keep baking soda on hand. We were pretty lucky.
I dragged the UPS down the hallway by it’s cord all the way to the dumpster (ouch $300 in the dumpster!).
wow… that’s something i’ve never seen before!.. what many people don’t realize about ups devices is that the battery must be replaced… and replacing the battery isn’t cheap… the last battery i had to buy was for a production environment ups that retailed for some $2000… nonetheless, i would still prefer to lose that than the data!.. glad to hear that your servers survived…
|nonetheless, i would still prefer to lose that than the data!.. glad to hear that your servers survived…|
Amen to that, I was concerned that some of the drives (18gb scsi2) wouldn’t remount, and I’d be left restoring from tapes.
what server environments are you running?.. the absolute worst of them all is having to go to tape for exchange 5.5… i would prefer to chew my arms off than to do that again!..
just curious, what is the spindle speed for the drives?.. when i built my daw, i used two cheetah 15k scsi drives in a raid 0 configuration… the i/o operations are incredibly fast… hopefully soon, i’ll be adding a couple of sata drives in a raid 1 config for storage… but, i’m concerned about the heat buildup in the case… it only has one 120mm fan in back and another 80mm fan in front… well, and the power supply fan…
okay, there you’ve gone and sparked up the nerd in me…
You are a lucky guy for sure.
UPS units are almost indispensable but they’re bombs also . I had a terrible experience with an UPS explosion a few years ago… Again, you are lucky.
That sounds like bad design to me.
Why wasn’t the UPS unit internally surge protected?
A little thyratron plus trigger and suitable current dump load will handle anything a lightning storm will throw at it.
I think you have a case against the manufacturers.
Mako, a UPS explosion is probably due to poorly vented hydrogen, an unavoidable byproduct of a lead-acid battery.
And the trouble is people stick UPS units in cupboards or basements or somewhere equally poor in air change, and as soon as the H2/O2 mixture reaches the critical level…one small spark and, you guessed it!