At apprx. 1:00pm Friday, A man, for reasons unknown, crashed his Yaris headlong into a switch that. *ahem* … switches only do one thing; turn ON and OFF. This particular switch does so with respect to real-time parameters, such as the voltage of sub-switches and transistors, the voltage of its parent switch and other factors.
Ideally it turns ON if “child” switches run low on power. OFF if child-switche aggregate excess power. You can think of power delivery as a cascading system, where the MAIN (source) Power Plants(s) create energy (usually via the rotation of turbines). This region formerly had a nuclear power plant nearby – the San Onofre Nuclear Generating System (SONGS). You know them as the “boobs” Leslie Nielsen refers to in “The Naked Gun” when tells his partner that he misses his ex and that “Everywhere I look, I see something that reminds me of her”
In 2015, SONGS is offline. (the radioactive rods, which must stay cooled and entombed within the 6-foot-thick steel-reinforced concrete “boobs” until the end of time – research SONGS and half-life for more info). AN ASIDE about SONGs and The Marines Corps: By shutting down SONGs, 100s of dedicated employees, many of them combat veterans of the Iraq War II “Bush’s Revenge” &c. Many of THEM were surrounded by the Fedayim in the surged-city of Ramadi and fought their way out. Heroes all; 2-4 and 2-5 is heavily decorated for bravery and sacrifice, The Return of The Feydayim being just one example – the Battlion and subsequent regiments were formed as part of the 1st Marine Division to support The Pacific Campaign (ca. 1940). Security-gigs were handed out to combat veterans of Iraq; SONGs-SEC was a cushy job they deserved. Cushy as it was, the security of SONGS can only be carried about soldiers with combat experience. The worst thing that ever happened on their watch was a helicopter crash in the patrol area. The pilot was an amateur and the crash a mistake. There was no intent to harm SONGS when this incident occurred. Note that SONGS was never harmed in any way shape or form with these men guarding it.
San Diego Gas & Electric laid them off as large companies do to ANYONE, including this writer. They closed, sealed and entombed one “boob” then another, all the while integrating a new energy supplier, The Hoover Dam, which is at the state line of CA & AZ. SONGS is IN San Diego County, close to Camp Pendleton. Where the 1st Marines train. Watch “The Pacific” on HBO GO if you haven’t. How importing energy from 500 miles away is better business than a nearby, stable and secure station within the community it serves is a question beyond my wisdom.
The system goes offline in planned and “unplanned” ways many times. Usually when too many ACs are running –SDG&E fails to implement ERP/MRP adequately. That much is evident.
THIS outage is unprecedented. After the crash circuit #0990 broke. Leaving about 500 people, including this writer, without power.
Conjecture based on available info– SDG&E initially reported that the system “failed due to a combination of factors” and the estimated restore time was 4:30pm. Later it was said to be “caused by an automobile accident” and the restore time kicked back another 30 minutes.
When the Customer-specific Action Items were not delivered on time, SDG&E kicked back the restore time to 5:00am — the following day. The assumption is that SDG&E has an SOP whereby IF the estimated restore time is unknown THEN add 12 hours to it and update the site.
- I conclude from that assumption a few likely scenarios
- SDG&E didnt’t know WHAT the problem WAS or HOW to fix it
- The on-site techs took a boondoggle and double-fucked it due to incompetence
- the design of C#0990 is shit
- the system itself has poor Exception Handling
- fault lies with management by deploying in adequately train field techs
- The Project Manager who designed C#0990 is predominantly at fault
I’ll get into the design and practice precepts that should be used in large systems. I’ll also provide tangible proof that the circuit design is not only faulty, but ugly and vulnerable. I can’t… for the life of me …picture an engineer leaving an Enterprise System with so many customers depending on it in it’s CURRENT STATE– Which merely to was to replace the switch. So far they have not even at least added guard rails to protect it from THE EXACT SAME ACCIDENT RECURRING. …. it *sigh* the techs who stood around all damn night and just stared into the whole could’ve grabbed a shovel and gotten started on a few pylons.
0990 was down for about 25 hours. Many reported lower times, this reporter’s studio was offline for just over one FULL DAY. The restore time, I discovered via a third party (I had no internet access and was conserving a small amount of juice on my smartphone) updated to 930am. To my assumptions, SOMEONE, likely an outside consultant, possibly from a competent Engineering Firm like Deloitte — was brought in. Power returned to the system at roughly 900am. It was done so in a way that minimized the risk of further damage. The presumably complete circuit was brought back on slowly and only for about 10 seconds. Then the system was brought back online in toto. Power was restored at 930am TO THE SECOND and has now been stable for 90 minutes. Bravo to the LEGIT engineer that got 0990 back up.
As for SDG&E – FUCK YOU. Self-Similarity Studios, The SoCal Picayune and affiliated media organizations missed out on an important opportunity I’d intended to implement on 9/25/15 and spent the previous 10 hours preparing to deploy when SDGE’s circuit failed, taking my ability to complete the task away. It was time-sensitive, so attempting to deploy an Organically-Optimized Marketing strategy is dead in the water, so fuck you very much.
All my cold-stored food, along with everyone else’s and local vendor Trader Joes. The less-fortunate got to raid a dumpster overflowing with TJ’s goodies, so there was at least one blessing in that. ALL businesses dependent on 0990 hemorrhaged $. Only Stater Brothers attempted to serve the public, but it was a tense situation. I saw three different people walk out with 12-packs of brew-dizz-ogs. Looting is an incomprehensible concept to me, especially with the OCSD in constant search of crime in a nearly crime-free locale. Overnite was tense as well. Some homeowners playing cowboy shit; flashin their POS 38 specials. Most residents evacuated. This writer stayed behind to watch the street, where SOME residents crouched low behind their windows in the dark.
SCARED WHITE PEOPLE IN THE DARK WITH FIREARMS AND OPPORTUNISTIC THIEVES ROAMING IS A FRIGHTENING COMBINATION. I kept picturing a father accidentally putting a slug the back his own kid’s head in a fit of paranoia, low light conditions, heat and exhaustion. I didn’t here any shots fired. For now we pray.
Time will tell if 0990 was at least PATCHED appropriately.
Preliminary report filed by the publisher FOR IMMEDIATE RELEASE. WIP. Updates to this post coming ASAP.
(secondary edits 9/26, 9/27 and 10/16….C#0990 has been up since it’s completion at 930am as reported above). The aforementioned replacedment switch i still stacked in the middle of A BUSY intersection with NO guard railings or pylons. A nearby fire plug is still covered by a GLAD-Hefty-Black-Sack-style bag and is presumably offline. [updated photo of accident site forthcoming, 10/2015].