Before you begin, revisit the mission statement for the utility. Decide now what kind of disruptions you're willing to tolerate and how much expense you're willing to fund before anyone brings you any kind of technology. Make sure the costs of disruptions and impact to the community is considered in making that decision. Check policy and local/state/provincial laws for potential impacts from outages, and include those impacts in your analysis.
When the analysis is complete, check with public relations to see what they think the response will be. you may need to revisit that decision. When you have a safe way to proceed, loudly and repeatedly publish your decision on how you intend to proceed and what your goals and tolerances are. Make sure the utility people know it and can repeat it in their sleep. Make sure the vendors know. Leave no stone unturned.
When you're about to start buying things and collecting bids, be very, very careful. If you expect the solutions you buy to do something in particular, make sure you word it that way in your requests for proposals. Don't say something like "must be able to log to our log servers" when you mean to say "must log to our log servers." Vendors will seize upon the former and, when you see they're not doing what you want they tend to respond, "well, it's *able* to do that but it'll cost an additional 27 bazillion dollars for a pro services engagement. It doesn't do that out of the box." Small utilities have a hard enough time with finding money without having to put up with that kind of silliness.
When things begin, you will hear from two types of people:
Control Systems people will repeat whatever the vendor tells them about what good security is (if they mention it at all), and the vendor will want to connect a lot of stuff to the Internet. They may also want to remotely access your control systems for support and maintenance. There are sometimes reasons for this, but it's almost always an extraordinarily bad idea. Vendor back doors and B2B connections into control systems usually lead to unauthorized changes and unexpected behavior. The next thing you know, you're dependent on the vendor for everything and they can start setting the contract renewal terms. This is bad and becomes costly.
Vendors and Control Systems folks also tend to focus entirely on encryption (and sometimes access controls) as the extent of security, totally ignoring data integrity and often hand-waving availability. Software vulnerabilities that lead to denials of service are an availability issue. Poor authentication and unencrypted protocols are a potential data integrity issue. Don't let the vendors or the Control Systems people slide on those- if they have integrity, encryption, and authentication controls, make the vendor implement them.
Make the vendor come on site to do maintenance and put your foot down with the Control Systems people who want remote access- if it's important enough for need their attention in the middle of the night or on Thanksgiving, it's important enough for them to drive to the site where the equipment is and deal with an issue. Utilities can usually pay for maintenance and overtime, but would rather not pay fines for an outage or a safety issue.
IT Security people will want to do all kinds of intrusive and disruptive scans on the systems, and they like to keep things patched. This is important, but sometimes patching can break the systems that the Control Systems people make. The reasonableness threshold for patching is usually tied to the most recent version of the software and patches that are available from the SCADA vendors. Absolutely keep the vendor's software and firmware patched to the most recent levels, and don't patch the OS beyond what's supported by the vendor. That sounds counter-intuitive to a lot of IT Security folks, but that's the way it is (hence the tedium about isolation.)
As for the collection of Control Systems folks and IT people, they'll need some way to
Slashdot Mirror
Before you begin, revisit the mission statement for the utility. Decide now what kind of disruptions you're willing to tolerate and how much expense you're willing to fund before anyone brings you any kind of technology. Make sure the costs of disruptions and impact to the community is considered in making that decision. Check policy and local/state/provincial laws for potential impacts from outages, and include those impacts in your analysis.
When the analysis is complete, check with public relations to see what they think the response will be. you may need to revisit that decision. When you have a safe way to proceed, loudly and repeatedly publish your decision on how you intend to proceed and what your goals and tolerances are. Make sure the utility people know it and can repeat it in their sleep. Make sure the vendors know. Leave no stone unturned.
When you're about to start buying things and collecting bids, be very, very careful. If you expect the solutions you buy to do something in particular, make sure you word it that way in your requests for proposals. Don't say something like "must be able to log to our log servers" when you mean to say "must log to our log servers." Vendors will seize upon the former and, when you see they're not doing what you want they tend to respond, "well, it's *able* to do that but it'll cost an additional 27 bazillion dollars for a pro services engagement. It doesn't do that out of the box." Small utilities have a hard enough time with finding money without having to put up with that kind of silliness.
When things begin, you will hear from two types of people: Control Systems people will repeat whatever the vendor tells them about what good security is (if they mention it at all), and the vendor will want to connect a lot of stuff to the Internet. They may also want to remotely access your control systems for support and maintenance. There are sometimes reasons for this, but it's almost always an extraordinarily bad idea. Vendor back doors and B2B connections into control systems usually lead to unauthorized changes and unexpected behavior. The next thing you know, you're dependent on the vendor for everything and they can start setting the contract renewal terms. This is bad and becomes costly.
Vendors and Control Systems folks also tend to focus entirely on encryption (and sometimes access controls) as the extent of security, totally ignoring data integrity and often hand-waving availability. Software vulnerabilities that lead to denials of service are an availability issue. Poor authentication and unencrypted protocols are a potential data integrity issue. Don't let the vendors or the Control Systems people slide on those- if they have integrity, encryption, and authentication controls, make the vendor implement them.
Make the vendor come on site to do maintenance and put your foot down with the Control Systems people who want remote access- if it's important enough for need their attention in the middle of the night or on Thanksgiving, it's important enough for them to drive to the site where the equipment is and deal with an issue. Utilities can usually pay for maintenance and overtime, but would rather not pay fines for an outage or a safety issue.
IT Security people will want to do all kinds of intrusive and disruptive scans on the systems, and they like to keep things patched. This is important, but sometimes patching can break the systems that the Control Systems people make. The reasonableness threshold for patching is usually tied to the most recent version of the software and patches that are available from the SCADA vendors. Absolutely keep the vendor's software and firmware patched to the most recent levels, and don't patch the OS beyond what's supported by the vendor. That sounds counter-intuitive to a lot of IT Security folks, but that's the way it is (hence the tedium about isolation.)
As for the collection of Control Systems folks and IT people, they'll need some way to