Slashdot Mirror
Ask Slashdot: How Best To Disconnect Remote Network Access?
An anonymous reader writes "Is there a device to automatically disconnect network or otherwise time limit a physical connection to a network? The why? We are dealing with a production outage of large industrial equipment. The cause? The supplier, with no notice, remotely connected to the process control system and completely botched an update to their system. We are down and the vendor is inept and not likely to have us back to 100% for a few days. Obviously the main issue is that they were able to do this at all, but reality is that IT gets overridden by the Process Control department in a manufacturing business. They were warned about this and told it was a horrible idea to allow remote access all the time. They were warned many times to leave the equipment disconnected from remote access except when they were actively working with the supplier. Either they forgot to disconnect it or they ignored our warnings. The question is, is there a device that will physically disconnect a network connection after a set time? Yes, we could use a Christmas tree light timer hooked up to a switch or something like that but I want something more elegant. Something with two network jacks on it that disconnects the port after a set time, or even something IT would have to login to and enable the connection and set a disconnect timer would be better than nothing. As we know, process control workers and vendors are woefully inept/uneducated about IT systems and risks and repeatedly make blunders like connecting process control systems directly to the internet, use stock passwords for everything, don't install antivirus on windows based control computers, etc. How do others deal with controlling remote access to industrial systems?"
How on Earth do YOU get to make fun of other employees at that company? I can think of at least a couple of filtering methods more elegant than a freaking christmas tree timer and I'm not even in IT. If all departments' staff quality is the same as IT I just hope that the "large industrial equipment" is not something that can affect other people.
Filtering access on a per-request basis is one thing, and I see how that's critical and can't think why you haven't implemented this already. Filtering access on a per-timer basis is the WORST WORST WORST idea ever. If I could make that any more caps locked I would. There are SO many things that can go wrong with a blind timer-based disconnection that I won't even bother to list them all, I will just paint the simplest of pictures in a newspaper title: "Incomplete update to a CNC machine leads to hands being sawn off".
Do yourself a favor and change jobs.
That's fine, until the process control guys unplug from your nice managed port, run a cable across the floor and plug into a port that you're not actively managing. And they will do that. If you don't think so then you haven't worked in that type of environment.
"Think about how stupid the average person is. Now, realise that half of them are dumber than that." - George Carlin
I am one of those registered engineers who really does understand both the IT and the Operations sides of the issue. Yes, I do process integration for a large utility and yes I live with my creations. Most of you in IT don't have a clue about the operational side of the fence, so please hold your snide comments until you understand the whole issue. Yes, we've seen what remote access follies can do. Allow me to point out that nobody in this business should be pushing patches to the plant floor. Remote firmware updates are reckless activities that deserve to be prosecuted for malpractice.
My employer has seen a few idiot project managers who, despite warnings from staff, contracted companies who demanded remote access. Suffice it to say that these people will not make such mistakes again.
In an office, there is usually a warm body at the other end of the keyboard. They can be instructed to do things. In any case, the product is data which can be backed up and restored if needed. If you chose to push patches in a situation like that, you could trust the end users to call you if something goes sideways. However firmware in a substation or in a controller is really not meant to be updated remotely. You should be standing there just in case you need to run things manually or need to shut down certain devices first. These places do not normally have people present to call if something doesn't work.
So when a vendor demands remote access to your substation or large asset, the answer should be an emphatic NO! and WTF? and "I'm taking my business elsewhere."
There is no good way to push a patch in to a control system. Those of you who think pushing patches is good need to come with me and clean up the messes that result from such behavior. You need to realize that software and data is not the end product here. There are no backups. There is only real product, real energy, and real messes when something fails. And if someone is hurt or killed, well, limbs and lives can not be backed up and replaced. If you're still throwing patches at the wall in the hope that nothing goes wrong, you are not welcome on the plant floor.