it's not that simple as just layman's terms of "overloading and over working." the system is designed to take the heavy load and work. what is difficult to troubleshoot are external and sometimes external converted to internal factors, especially when they are related to electrical power issues. for example, my company built a brand new data center facility with the latest and greatest equipment and building accessories including all kinds of plumbing for ventilation, heating, cooling, air-con, fire suppression, cable ducting, airflow, fiber conduits, etc. one morning into the first month, power started tripping and equipment racks started going down one by one. apparently, a coolant pipe condensed the air around it and water droplets concentrated at the lowest point in the plumbing, usually an elbow or vertical turn, and a few drips short the rack below, knocking it out, and in turn it tripped the entire row. other rows were spared because they were isolated by separate power feeds. each row is now on separate redundant feeds. because the condensation did not drip as much when techs typically spent much of their time looking for "root causes" (usually at power distribution panel on rack, cabling and equipment first), it took 2 more months for a sharp-eyed tech to see drops of water on the raised floor where rack had been removed. otherwise if rack had not been removed water drops would puddle on top of rack which is an equipment cabinet, and nobody would have guessed it was condensation from a pipe. the pipe was sealed tight. it just happened to be a very cold pipe collecting ambient moisture from hot air around it as equipment emits hot air and hot air rises, and moisture seeps in everytime door opens or tech walks in. the fix was to have fan trays on every row blowing hot air to a designated exhaust or egress chimney, and avoid rows to be placed at suspected condensation spots. it requires a bad experience to learn new lessons. and smrt cannot afford to let experienced techs go. they are the company's unsung heroes and extremely valuable asset.
