Slashdot Mirror
Ask Slashdot: How Best To Disconnect Remote Network Access?
An anonymous reader writes "Is there a device to automatically disconnect network or otherwise time limit a physical connection to a network? The why? We are dealing with a production outage of large industrial equipment. The cause? The supplier, with no notice, remotely connected to the process control system and completely botched an update to their system. We are down and the vendor is inept and not likely to have us back to 100% for a few days. Obviously the main issue is that they were able to do this at all, but reality is that IT gets overridden by the Process Control department in a manufacturing business. They were warned about this and told it was a horrible idea to allow remote access all the time. They were warned many times to leave the equipment disconnected from remote access except when they were actively working with the supplier. Either they forgot to disconnect it or they ignored our warnings. The question is, is there a device that will physically disconnect a network connection after a set time? Yes, we could use a Christmas tree light timer hooked up to a switch or something like that but I want something more elegant. Something with two network jacks on it that disconnects the port after a set time, or even something IT would have to login to and enable the connection and set a disconnect timer would be better than nothing. As we know, process control workers and vendors are woefully inept/uneducated about IT systems and risks and repeatedly make blunders like connecting process control systems directly to the internet, use stock passwords for everything, don't install antivirus on windows based control computers, etc. How do others deal with controlling remote access to industrial systems?"
Enable port security which ties each port to a mac address of the other device connected to it so that all ports on the network switch are locked down to just the devices white-listed to connect. Write down what port your gear is connected to which you want to limit access to the internet, and then simply disable or enable that port to allow it to connect.
We were all warned a long time ago that MS products sucked, remember the Magic 8 Ball said, "Outlook not so good"
iptables lets you specify times if you're using a linux box as the firewall, otherwise consult the fine manual that came with your equipment or consult a professional with said equipment. This is bog standard.
I'm a DCS system admin at an oil refinery. We keep the DCS and business lans totally separated, and that directive is driven from the top down. If anyone asks for remote access we just let them know that's NOT going to happen - end of story! It can be a pain getting files from one network to the other (patches, etc.) but certainly worth the effort.
How on Earth do YOU get to make fun of other employees at that company? I can think of at least a couple of filtering methods more elegant than a freaking christmas tree timer and I'm not even in IT. If all departments' staff quality is the same as IT I just hope that the "large industrial equipment" is not something that can affect other people.
Filtering access on a per-request basis is one thing, and I see how that's critical and can't think why you haven't implemented this already. Filtering access on a per-timer basis is the WORST WORST WORST idea ever. If I could make that any more caps locked I would. There are SO many things that can go wrong with a blind timer-based disconnection that I won't even bother to list them all, I will just paint the simplest of pictures in a newspaper title: "Incomplete update to a CNC machine leads to hands being sawn off".
Do yourself a favor and change jobs.
...is a post incident review with support people involved, and their management teams, along with directors and executive involvement to identify what the problem was that caused the business to be inoperative for the duration of the incident, what policies and procedures need to be followed going forwards, and so on. Once policies are established, solutions that support those policies can be implemented.
As an example for your situation, since a vendor was involved in an upgrade, that should have been part of a scheduled change. The change should be documented ahead of time as to what is being done, what systems are going to be touched, and who the responsible parties both within the company and external to the company are for that change. Included in the documentation should be the fallback plan for dealing with issues that crop up during and after the change, within an appropriate test window that is included in the change window, as well as clearly defined backout procedures. "fix and fall forward" or equivalent statements are not, and should not be, considered acceptable plans. Wherever possible you want to have documentation attached that the procedures involved have been tested in a suitable test environment. (This may not be possible in situations where a test environment would cost as much to prepare as the production environment.)
As far as limiting remote access, as others have pointed out, such limits are trivial based on what type of remote access is in place, and what policies are established. At the very least account authorizations required for performing changes on production devices should require someone in house approve that authentication, be specific to the time when those changes are scheduled to happen, and should not allow similar access to devices or types of devices not involved in the change.
You never know...
The supplier, with no notice, remotely connected to the process control system and completely botched an update to their system. We are down and the vendor is inept and not likely to have us back to 100% for a few days.
This isn't a technology problem.
Through their incompetence, they caused damages. Collect your evidence, hire a lawyer, and make demands. If they refuse to pay, sue them.
Watch how fast they start caring about doing remote upgrades more carefully, competently, and with customer involvement. The only thing companies collectively care about is making money. At the very least, you'll cause their liability insurance rates to go up.
Please help metamoderate.
One of the more amusing hardware hacks that I've seen in the physical security industry is when a customer hooked up the power lead of the remote access device (modem in this case, could have been a switch or something else) to a key card reader. The security staff would badge the reader, the output would turn on for 1 hour, and then shut off. The really nice thing about this is that now they could track who enabled the remote access. If the vendor wanted to connect from 3:00 to 6:00 for example they could create a time zone that would turn the output on for that time period, and only certain people had permissions to configure time zones. Worked pretty well.
Our salescritter had jokingly told this same customer, "This system can do everything except make your coffee in the morning." The customer took that as a challenge, and the next time we were there we found that he had set the system up so that when he badged in the front door for the first time in the morning it would fire a relay that would turn the coffee pot in his office on.
"Think about how stupid the average person is. Now, realise that half of them are dumber than that." - George Carlin
That's fine, until the process control guys unplug from your nice managed port, run a cable across the floor and plug into a port that you're not actively managing. And they will do that. If you don't think so then you haven't worked in that type of environment.
"Think about how stupid the average person is. Now, realise that half of them are dumber than that." - George Carlin
crontab:
"National Security is the chief cause of national insecurity." - Celine's First Law