Comments

You must log in or register to comment.

Vegan_Honk t1_j53ucgn wrote

If it takes only one working person to throw an entire system into chaos like this then the process their following needs to be updated. Quickly.

53

Gigglemind OP t1_j53w8tc wrote

Absolutely. Was there a pop up just saying "Are you absolutely fucking sure you want to do this"?

19

TheRealOneTwo t1_j53yh8z wrote

More than that. There needs to be authorization to do such an action in the program and the process

15

Gigglemind OP t1_j53z3jj wrote

Probably going to be a lot of pointing fingers up above. Would imagine this environment is pretty niche too.

7

axonxorz t1_j55tlnq wrote

You must not work with users often.

In the case of the accidental Hawaii missile alert, it was very poor UI to have basically hyperlinks almost beside each other for "drill" and "not a drill". But then

>Still, there is a second confirmation page as a safety measure, asking if the employee is sure they want to send the alert, which they also mistakenly pressed “yes” on.

I once had a user use their Outlook trash bin as their filing cabinet. Then she went to "file" some messages from her inbox without realizing she was already in the trash. Outlook pops up a warning saying that the messages will be irrecoverably deleted, she press "yes", then when she called me in a panic and I said "the computer did exactly what you told it to do", she told me "well what are you guys even here for"

9

Y-Cha t1_j560i4q wrote

>You must not work with users often.

I'll bungle this as my eyes are half open.

Yep, very important when writing the UAT scripts/procedure, at some points, to place yourself in the position of "brand new user, knows nothing." Also make sure your SME, if they're involved in the actual testing, remembers the same.

2

johnn48 t1_j54dn6b wrote

All it takes is a neo-Nazi with a gun to destroy a power station and plunge people into darkness.

3

insan3guy t1_j54kbvb wrote

Shooting a power distribution station and cutting power to a town is a little bit different than shutting down all united states air travel

4

johnn48 t1_j55856b wrote

>Two power substations in a North Carolina county were damaged by gunfire in what is being investigated as a criminal act, causing damage that could take days to repair and leaving tens of thousands of people without electricity, authorities said Sunday. Source

>About 35,000 people in North Carolina's Moore County remain without power on Wednesday after the substations were damaged in what authorities described as a "targeted" attack Source

> FBI warns of neo-Nazi plots as attacks on Northwest power grid spike Source

A simple Google search will show a disturbing pattern of attacks on power substations by suspected neo-Nazis. So these are not attacks by teenagers shooting up signs. These are attacks on the vulnerable infrastructure of the United States

−1

insan3guy t1_j56iou9 wrote

It would take many times the amount of time, people, and equipment that the NC attacks were perpetrated with in order to effect the kind of damage that the FAA problems created.

And they are not power stations, they’re distribution stations. Try shooting up a coal plant and see how far you get. Might even make it to within eyesight of it.

1

Vegan_Honk t1_j55i75u wrote

yes and now they know that.
Same with now people know that an individual person can bring a decades old system to its knees.

1

Draano t1_j56jfvo wrote

In the the most recent investment bank I worked at, no single person had the authority to make a change to a production system without multiple approvals, and a change would have to have a fellow technician verify the install and fallback process, there had to be documentation showing a test of the change as well as the implementation and fallback process on a staging system. If it was an emergency change that had implications to a live system with user impact, there would be 15 - 20 people dialed into a call and online to the system, observing every part of the change. Every user who would be impacted would have to be aware, and their C-level management would have to sign off on the change. And just getting the access to implement a one-time change required CIO approval.

But that's people's money. It's not just *actual lives at stake-*level stuff.

3

PMzyox t1_j53xngh wrote

I feel like this is a scapegoat answer. Someone high up made a bad decision that led to a critical outage, methinks

18

Professional-Can1385 t1_j544b39 wrote

I'm a contractor in a completely different part of the government. If something goes wrong, they blame a contractor, any contractor it doesn't matter. I've sat in meetings where people blamed things on "a contractor from XYZ." Never a name, just "a contractor." Makes a person not care if their work shuts down all of US airspace or not.

23

PMzyox t1_j545782 wrote

Yep that matches up with my experience too

8

Gigglemind OP t1_j53yrc6 wrote

Not a software engineer but has that ring to it. They said something in the article along making the system more resilient, aka how the fuck could this happen?

6

TooMad t1_j54odcp wrote

We added a resiliency power point slide.

6

msiekkinen t1_j53vxnn wrote

At least in my worst fat finger fuck up ive never crippled an entire countrys air travel

16

MasterpieceLive9604 t1_j53rzs2 wrote

Homer voice: 🎶Guess I forgot to put the fog lights in🎶

15

Gigglemind OP t1_j53q9sc wrote

Going to think of this when I have a bad day at work. Can't imagine what it's like when it dawn's on you what just happened.

14

SlinkyTail t1_j549yul wrote

"this is written in fortran, let me fix that and make it javascript" - some contractor.

8

carandfreedomgeek2 t1_j54gndn wrote

I worked on an FAA contract in the 90’s. The thing that amazes me most is that stuff like this doesn’t happen regularly. It was so bad I actually resigned. A year later I hear the entire contract had to be redone. Rampant incompetence.

7

msiekkinen t1_j53xpp5 wrote

Killall -9 contractors

4

dogsent t1_j5415bu wrote

Old guys rule. Some genius replaced an old guy with a contractor who was cheaper.

I think it's actually an accounting error that caused the problem. Too many damn accountants running the companies these days.

2