How to deal with a vulnerability in critical infrastructure

Rikard Bodforss

I don’t usually blog in English, since I try to keep my customers local to Sweden, but this time I believe the message might have an international audience, so I decided to use a more widespread language than Swedish.

Last week I gave a talk on the CS3STHLM summit which is the best ICS security conference in Europe, with speakers and delegates from all over the world. I talked about a vulnerability I found over five years ago in Power Generation Information Manager, which is an ABB Data Historian used in control systems. The talk was mainly focused on dealing with the aftermath of finding the vulnerability and the dilemma you are faced with when you know that decades from now there will be vulnerable systems out there. When do you disclose the vulnerability?

My reason for disclosing now was that I knew that ABB had a product ready that isn’t vulnerable, but they didn’t have any public security advisory on the vulnerable versions despite knowing about this for the last five years. I have been assured that ABB will release a public advisory any day now, so my talk had the desired effect. The other reason for waiting five years was that one of my customers had a vulnerable system until earlier this spring, and the information regarding my findings was classified secret and a matter of national security until they were able to upgrade. I also did inform the Swedish Civil Contingencies Agency (MSB) and the largest water and power companies in Sweden a few months back so they could have a head start before the information became public.

What was the vulnerability?

Well, since the video of the talk won’t be available before CS3 releases the YouTube version later this year, I guess I’ll have to fill you in on the details… To make a long story short, I discovered an authentication bypass that allowed me to access the user management interface of ABB PGIM. To add insult to injury, the user credentials were stored in clear text in the database and were transferred in clear text over the wire on request. The security model was like something from the 90’s where all trust was handed over to the client. To bypass this, I simply wrote my own client and didn’t bother with the hazzle of authentication before I asked the server for all its users and passwords. But, wait! It doesn’t end there. When I tested my credential miner on a production system, I noticed that the usernames and passwords I got back were (manually) synchronized with the Active Directory of the entire DCS system. I got high privilege accounts and Domain Administrator credentials to the 800xA domain. Game Over.

In my opinion, the only reason for having high privilege accounts in the Data Historian user database is poor understanding of how it works, poor implementation, and pure ignorance. They are not needed for any functional purpose, since authentication is a charade anyway.

What are the implications of this?

If an attacker were to get a foothold on a client/server network on an ABB 800xA system running PGIM as Data Historian, there is a tremendous risk that the attacker will be able to get valid credentials to the domain by exploiting this vulnerability unless this is mitigated. Access to the C/S network equals total ownage.

What versions are vulnerable?

All versions named PGIM are vulnerable, as are older versions of the software called Plant Connect. ABB IM, which is another data historian, is based on a completely different software and is not vulnerable to this attack. The latest version based on PGIM, Symphony Plus Historian, has a completely new authentication mechanism based on Microsoft AD and is not affected by this vulenarbility.

So, in short: If you are running ABB 800xA and have PGIM, you could be in trouble, especially if you are on version 4.x or 5.x, or if you have the older obsolete Plant Connect software installed.

What can I do? My next maintenance window is in six months!

No worries. My customer lived with this for five years. There are ways to mitigate the risk. First of all, a sound security model and proper network segmentation between the production networks and client networks is always a good start. You should also employ a layered security model with defense in depth. When this is done, make sure only authorized clients can access port 4242 on your PGIM server. Use firewall rules and/or a VPN gateway to authenticate clients before they are allowed to even address the PGIM server. You should also go through the user database of the PGIM server and remove any credentials that are the same as on your domain. The next step is detection. This attack is fairly easy to detect. Flag anything scanning for port 4242, and writing a Snort rule for detecting a Userlist command is easy. You want to detect those even if it is authorized since credentials are flying over the wire in clear text… Oh, and do plan for an upgrade during your next maintenance window.

Mitigations in short:

  • Segment production networks
  • Use a layered security architecture
  • Control access to PGIM using firewalls and VPN
  • Remove domain users from PGIM
  • Detect attempts to exploit the vulnerability
  • Upgrade to Symphony Plus Historian

How do I test for the vulnerability?

You are welcome to use my PoC code on github.com/rbodforss/pgpwner/. Please don’t use it to do anything bad. I release the code so the good guys can find their problems before the bad guys do. I am 100% certain that the bad guys already knew about this vulnerability and have been using it for years. It’s a lot less noisy than a brute force attempt and requires less effort that passing the hash with Mimikatz.

What now?

I received a lot of great feedback on my presentation and I will post a link to the presentation once it goes online. ABB has started to communicate with me again, and CERT-SE is reaching out to potentially vulnerable companies. Norwegian and German authorities were present at the presentation and are also informed. I’m sure I’m on someone’s shit list for blowing the whistle on a perfectly awesome backdoor, but that gives me great satisfaction.

If you need help sorting this out, don’t hesitate to give us a call. We’re here to help, and our business is securing society one critical system at the time.

Edit: I have updated vulnerable systems and cleared out a few details after working extensively with ABB on the public advisory.