Vulnerability vs. Penetration Testing

TOM: Welcome, everybody. This is our usual Tuesday afternoon 2 p.m. Central weekly Deeper Dive series. Today’s topic is vulnerability assessment versus a penetration test. In case you’ve never been here before, my name’s Tom Kirkham. I am Founder and CEO of IronTech Security, and IronTech’s Marketing Director, Kindsey Hanes, is here with me as well. Say hi, Kindsey.

KINDSEY: Hello.

TOM: Anyway, we’re real kickback on these and real casual. Sometimes these are topics that are part of the big webinar. What’s an EDR, or what does security awareness training look like, or let’s go on the Dark Web and see what the bad guys are buying and selling and doing. We did that one time. I’d have to do that again. We’ll come up with a really good topic to look for on the Dark Web – something that I won’t get a visit from the FBI on. But yeah, we actually went on the Dark Web, where you can see and do and buy all this stuff and where the bad guys hang out.

Lately, though, we’ve had a couple of requests for pen test, or penetration testing. That is a very specialized, custom test to see what your weaknesses are from hackers. It’s done by humans. I know no one is old enough, and it wasn’t a big hit at the box office, but if you’ve ever seen the Robert Redford and Sidney Poitier movie named Sneakers, that’s what Robert Redford’s company did.

The opening scene of the movie – not the opening, but beginning of the movie – he goes into a bank, withdraws $100,000. The teller says, “If you don’t mind me asking, why are you closing your account?” This movie is 20-something years old, so it was real money. He goes, “I just don’t feel like my money’s safe here anymore.” Puts it in a briefcase, walks upstairs to the boardroom of the bank, spins the case around, hands the list and says, “Here’s where all your vulnerabilities are. This is $100,000 I just stole from your bank. I need my check.” They paid him to professionally hack the bank and steal money. And he gave them a report so they could take care of those things.

There’s a lot of confusion around that. What is a security assessment? What’s a vulnerability assessment? And what is a penetration test? We’re going to take a deeper dive into those things, and maybe we’ll have bags of money like these divers have. Or there’s a bottle in that one, so I guess they’re not going to get rich.

IronTech does three things, and in case you didn’t know, we just added pen testing to our list of services – our security stack, as those in the know refer to it as. But in reality, there’s really only two things that are done, that are common amongst the security industry. Number one is a vulnerability assessment, and the second thing is a penetration test. I’m going to tell you what the differences are.

A vulnerability assessment. When we run a vulnerability assessment, we have tools that we install on the network and external to the network that are automated, and they just run through and see what’s vulnerable.

From the outside of a network, we scan your company’s IP address, see what ports on the firewall are open, and we detect what services are behind that port. In the non-geek speak terms, we know that Port 25 is part of email. So if you’ve got an email server in your building – which you probably shouldn’t, but you might – we’ll see Port 25 is open, and then we’ll discover the service behind it and we’ll get the version number, and then we’ll see if it’s been patched. We just report that in a vulnerability assessment. “Port 25 is open, this is the service that’s on it, and here’s what version it’s on.”

A vulnerability assessment doesn’t put things in context. It’s more likely to produce a false positive. For example, just because you have Port 25 open and there’s a mail server behind it, doesn’t mean there’s nothing wrong. It’s just itemized on the vulnerability assessment. It’s just “this port is open,” just like Port 80 is open on your network. No matter what, Port 80 is open. You’ve got to have Port 80 open on your network or you can’t get on the World Wide Web. There are other ports, too. 443. You might have Port 21 open. Things like that. There’s actually over 65,000 ports that could be open; I’m just talking about the 20 or 30 that are common. Remote desktop. The default port for it is 3389. If we discover 3389, we can log into it and see if, as a pen tester, we can remote access that device.

Vulnerability assessments do not consider security measures. In the case of remote desktop – in this day of COVID, we set up hundreds of remote desktops in about two weeks, three weeks tops. We had to open ports up to the outside world so they could remote access their desktop inside the office from their home. We don’t do 3389, although I don’t think it really matters anymore; it’s just what we do. We’ll do a different port number that’s not used by anything else.

The vulnerability assessment will show that Port 9000 is open and there’s RDP behind it. But it doesn’t take in the other security measures that we’ve put in place for that remote access, ad we do put very serious security measures in place. But nevertheless, it’s going to show up on a vulnerability assessment because it’s a nonstandard open port.

Consequently, these vulnerability assessments are really a technical report. It’s like something you may give to the manager or the president or the owner and it’s just a bunch of gobbledygook. He doesn’t have any context. He can’t take this technical analysis and determine a risk – unless he is an expert in infosec or at least has some sort of knowledge of IT. So it has no guidance, it has no context, and it’s really a technical report of what’s vulnerable.

Inside the network, it may show some things that he can action on, like password complexity or other password policies. A vulnerability assessment will pick those up. But generally speaking, it’s just a technical report, and it certainly doesn’t tell you what to do next.

And then finally, vulnerability assessments are generally not accepted by third parties. It does a third party no good to get a vulnerability assessment if they don’t have context. For example, if you’re required to be PCI complaint, you store credit cards, things like that, a vulnerability assessment is not going to get you there. You’ve got to be PCI DSS, which means you’ve got to have pen testing done. Pen testing is accepted by third parties because it’s got context and it allows for other security measures that you can’t see simply by doing a vulnerability assessment.

Now, the good news about it is a vulnerability assessment is the very first step of a pen test. What I mean by that is you’ve got to do the vulnerability assessment before you do a pen test. You have to. In fact, the right way to do it is you do your vulnerability assessment, your infosec guy, such as IronTech, looks at everything on there, and then they put it in context, and then they offer guidance, and then they put technical, administrative, and physical controls in place to address things like remote desktop access security that the vulnerability assessment reflects is a possible problem.

You do all of that stuff long before you do a pen test because pen tests are expensive. What’s our cheapest one? It’s like $3,000 or $4,000, I think. And vulnerability assessments are like $1,500, and you can do those more than once a year. You can do them every 6 months or every 3 months. They’re automated, so they just run and do their thing and spit out a few pages of a report. Pen tests are something you may only have to do once a year or just once, even, depending on what your third party compliance wants you to have.

If you’re subject to HIPAA, Sarbanes–Oxley – basically, if you’re in health, finance – law doesn’t have it, but if I was a law firm, I would do pen testing. Water utility, you’re not required to. I think that’s a tragedy. But it depends on what industry you’re in; you may or may not be required to do pen testing.

So what is a pen test? A pen test – remember the vulnerability assessment. If I were a pen tester, I will take that vulnerability assessment and look for things that I can directly attack using my skills, resources, and knowledge to directly attack and see how far I can get in. In the case of a remote desktop port, “Oh, this company’s got 20 ports open for remote desktop. They’ve got 20 people working in their homes, remoting into the office to do work during these COVID days. I’m going to launch an RDP session and go right into that port and get to the login screen.”

This varies depending on the scope of the pen test. How far and how much are you willing to pay to go as deep as we can? I may do a dictionary attack on the username and password. May even get a list of employees off the website or something like that. Sometimes RDPs store the username, so I’m already halfway there. All I need is a password. I just throw a dictionary attack on it and see if I can get in. If I get in to that remote desktop, it failed the pen test and now I’ve got access to the entire network.

But, however, if they have multi-factor authentication, like we put on our clients, they will not be able to get in.

So a human that is a white hat cyber attacker does his best to get inside that company to see what a bad guy might be able to do.

Then during the process, we’ve got this whole list of vulnerabilities, like really low security maturity, like they’re known to share passwords and they don’t use password managers and there’s sticky notes I can see just by looking in the front window of the accounting office that’s on the monitors, with a pair of binoculars. Or I walked in their office and just wrote them all down when I was in there when nobody was looking.

You put all these things in context, that you’re vulnerable. I know you’re vulnerable with having sticky notes on your monitor, but if the public can’t see it or even get to it and no one else can see it – no one else in your office can see your credentials – then it’s a lot more secure. It’s still a bad idea, but it is more secure in that environment.

So a good penetration test is going to determine the business impact of vulnerability. Is the remote desktop being open really something we’ve got to worry about? No, because we’ve put all these other security things in there.

Like I said earlier, before the pen test actually occurs, what is the scope of this pen test? That is based upon the tools, techniques, and resources of the most likely threat actor and the most likely type of attack to occur to this particular type of business. Then you work your way backwards. If you have intellectual property and you’re a Fortune 100 company, you’re going to be looking at a $100,000 pent set because you need to broaden the scope of what you’re searching. You’re going to put a team of pen testers on it that all have differing skills and bring something different to the table to really crack in there. Might be half a million dollars or a million dollars, whatever it is. The scope defines that.

If you’re a small company, four or five people in an accounting firm, you primarily do individual tax returns and very small businesses, no deep financial stuff and things like that, your attack surface is much smaller just simply because you’ve only got four or five computers. But you also are less likely to get hacked by a nation-state or a hacktivist. As a Fortune 100 company, they’ve got to assume nation-states, hacktivists, terrorists, and criminal syndicates are all gunning for them.

A penetration test will also recommend remediation, which I misspelled there. When they get done and they got in there, they’re going to say, “You need to do this, you need to do this, and you need to do this.” A vulnerability assessment will not do that because once again, it doesn’t have context or guidance.

And a pen test usually satisfies third party requirements, properly conducted.

So what does IronTech do? If you watch any of our webinars – and maybe at the end of this one, I have one – we talk about a security and risk assessment. Because we’ve done this for so long, and because of most of the types of clients we have, we understand that what you really need is a high-level security and risk assessment so the owner or the president understands the risk that he has and what he needs to do to dramatically improve his posture.

We’ve got clients with 75, 150 employees and things like that. We don’t do high level on those. We do the vulnerability assessments, and we will be doing pen tests now that we have that in our security stack. But for most of our clients that are under say – and it depends on how much money. You can have 10 people and you’re a $50 million company. In general, if you’re 15 or 20 employees or people, a vulnerability assessment is really about as far as you need to go unless you deal with Department of Defense or you’re a specialized law firm, you deal in investments, you’re a VC firm. Those are typically small, but they deal with billions of dollars, or hundreds of millions anyway. We’re going to do pen tests on that.

So our first step is a security and risk assessment, or we might skip that and go straight to a vulnerability assessment where we actually install software and do some scans and stuff. Then finally, like I said, the penetration test. That’s only after we do one or both of the ones before this. If we’re going to do a pen test, we’re going to do both of those before that. We will combine the security and risk assessment with the vulnerability assessment and then we’ll do a pen test.

So we’re going to do those first two bullet points, we’re going to put in all of our tools and techniques and defensive postures, technical/administrative controls around password complexity, password management, password managing tools, multi-factor authentication, cybersecurity awareness training. We’re going to be diligent on our backups and monitoring the backups and the firewalls and all of these different things we do to secure our clients from the bad guys.

Then once we implement all that stuff and we think we’re in really good shape, we’re going to go to the client and say, “You might want to do a pen test and see if we missed anything.” We outsource that. We don’t influence it or anything else like that. Pen testers are a unique breed. I think it’s a fun business. I mean, how else can you get paid for being a bad guy? You get to do all the bad guy stuff and you get paid for it, without the prison part. Anyway, I think it’s great.

All of this goes hand in hand with the NIST Cybersecurity Framework. Anyone that’s been to any of my webinars, you know I’m a big fan. Identify, protect, detect, respond, recover. That’s what these assessments and these testing things – the whole purpose of those is to have and implement a very, very good framework to make your defensive posture as good as it needs to be for you to have acceptable levels of risk. You don’t have to spend $1,000 a month if you’re a small firm. It can be very inexpensive and dramatically increase your defense posture. But we are going to be compliant with the NIST Cybersecurity Framework.

To reiterate, small firms, security and risk assessment. If you are running an active directory or a server that centralizes password management in Windows, we probably would rather do a vulnerability assessment because chances are your password administrative controls are bad if you’ve never had an infosec company. If you’ve only dealt with your IT company, yeah, I’ll bet they’re substandard and can be easily cracked. Probably just in the vulnerability assessment, or the results of a vulnerability assessment. And then finally, we’re only going to do a pen test after the NIST CSF is satisfied. That means we’ve got everything in place.

So if you want a security and risk assessment, just give us a call. I don’t have my chat window up. If anybody’s got any questions, throw it in chat. We can probably turn the mic on for you.

KINDSEY: We don’t have anything just yet.

TOM: There’s a link in the chat Kindsey just put up there to get your security assessment scheduled. I forgot to put the price on the slide. It’s $795. We frequently run big discounts on that, so keep your eyes peeled. I don’t think we’ve ever sent an email. It’s usually in the big webinars that we give that. But if you’re part of an organization that has us speak to your organization, there’s almost always a special on that security assessment.

And if you haven’t had us speak to your organization, let us know. We would love to do it. We do it all the time. I think we’re doing about three this week, aren’t we?

KINDSEY: Yep, we are. Looks like we do have a question from Daniel. “What do you recommend for local governments?”

TOM: It’s really the same, Daniel. That’s a big question. It depends on the size. First of all, we’ve got to go through city administrator or mayoral form of government, what departments you have, how many total employees are part of the same network. Basically, what it gets down to is chances are. We deal with communities that only have 300 people in it, and then we deal with others that have 100,000. So a local government is kind of a nebulous term.

But for the sake of argument, if you’ve got a server in your office that validates people when they log on in the morning and you can change their password or their credentials in the server when they forget it, then we would just do a vulnerability assessment first and see where you’re vulnerable, get a good idea of what your security maturity is like – we’ll do the risk assessment, too – and then we’ll propose the things that you need to put in place and how much it’s going to cost to get you where you need to be.

“What percentage of companies do you think have done vulnerability testing?” Not nearly as many as they should. “Pen testing is another one. For example, does a big company like Caterpillar do it regularly?” I would certainly hope so. It’s not something that is published, and as far as I know, I haven’t seen any research on it. But that is a good question.

It’s one of those things that I’m not sure I would want to tell everybody we ran a penetration test. And it’s not that I’m worried or embarrassed by something that we failed; it’s just the very fact that there is a report that exists in at least two organizations’ networks that has the results of the pen test. So if I was Caterpillar and I bragged to all the investors or the public that “Caterpillar just went through the most wicked, severe pen testing and we passed with flying colors and it was conducted by Dewey, Lynchman, & Howe,” I have just told the Chinese where they can hack and get our vulnerabilities in the results of the pen test.

“What industry sector is leading the parade on testing?” Well, health care, since they finally put teeth in HIPAA and they actually fine people and don’t give them a slap on the wrist. They realized that this could be serious money and practices can go out of business. Financial industry, they do pen testing. When you’re dealing with billions of dollars, you have to do everything you can to prevent being hacked.

Presumably national governments, state governments, all the departments of defense, military, all of those are going to do pen testing. And they may be a victim of pen testing – well, they are a victim of pen testing they didn’t pay for. The only difference between a pen tester and a hacker is what they do when they get in. The criminals and the nation-states are going to do some illegal stuff. They’re going to steal money or intellectual property or do military espionage. The pen tester is going to say, “Here’s how I got in. Here’s what you need to fix. I just want the money we agreed to from the get-go.”

That’s the only difference. They all use the same tools and techniques for the most part. There’s Kali Linux that you can download for free that’s got all these pen tests in it. There’s Nessus that’s a commercial software package that does pen testing. And then they use their own skills. They gather intelligence. They get a list of employees, and maybe they know their hours. They know their company’s public IP address. That’s easy to get. They do some manual good old-fashioned shoe leather, as they say in the private eye world.

But anyway, Daniel, getting back to your question, just start with our security and risk assessment. If you need a vulnerability assessment, we’d be happy to do that part of it. If you pay for the security and risk assessment and only that assessment, we will apply whatever you paid for that to any of our products and services going forward. The vulnerability assessment is straight-up whatever the price is, and same thing with pen test, regardless of whether you’re a client or not. But even the security and risk assessment, you have actionable things, you have a report, you get to understand what your risk is and what you need to do at a very basic level.

And if you’re a small local government and you’ve never done any of that, you’re going to dramatically improve your security. I mean dramatically. You’re going to go from a wing and a prayer, thinking you’re too small to be attacked – which no one is – or no one’s ever heard of your little town, like Toad Suck, Arkansas – the only people that have heard of Toad Suck are the people in Arkansas. There’s actually two Toad Sucks. Did you know that, Kindsey?

KINDSEY: I did not.

TOM: There’s Booger Hollow, too. Population: 7 and 1 coon dog. I’m perpetuating Arkansian hillbilly myths all of a sudden. [laughs] Anyway. We do have the world’s largest retailer. Very technologically sophisticated here. We don’t all have coon dogs. And a whole bunch of Fortune 500 companies are here.

Anyway, getting back to Daniel’s question, just get a security assessment scheduled. Click on that link and just get something scheduled. If you don’t move forward with this, it’s no obligation. We won’t charge you anything. But we can tell you a little bit more about it and feel out where you are, too, because we want to make sure that you’re getting what you need for where you’re at in cybersecurity defense.

Any other questions? Do we have a topic for next week, Kindsey?

KINDSEY: We do. It is how to lead a cybersecurity culture.

TOM: We’ve been postponing that.

KINDSEY: We have.

TOM: This is really good. It’s probably going to bore everybody to death, but it’s going to be very, very important. We’re going to talk about the difference between management and leadership when it comes to cybersecurity practices. So be sure to sign up for that. These are things that you can do without spending a penny. You can do it better by spending just a little bit that everybody on this webinar has the budget for, but you will have things – just like all of our webinars, you can walk away with things you can do to help your security without spending any money. I think that’s a bad idea; there’s a limit to how far you can take that.

If you do not have an EDR installed on your network, you’re a ticking time bomb. You guys have probably heard me preach about security awareness training and making sure your backups are monitored and checked to make sure they’re actually working when you need them – which most people don’t do. But if you do nothing more than replace your antivirus with an EDR, you’re going to dramatically improve your security posture. We’re talking $50 a month for five computers. $600 a year.

I want you to think about what you paid for cybersecurity insurance, or what you’re thinking about paying for cybersecurity insurance, because that’s easy. “I don’t need to know all this infosec abbreviations and know about nation-states and APTs and EDRs and SIM and DNS filtering. That’s all confusing to me. I’ll just get cybersecurity insurance and we’ll be good.” That’s the last thing you want to rely on. Just like homeowner’s insurance. I don’t go drive like a maniac because I’ve got car insurance. I don’t play Russian roulette because I have life insurance. Same thing with cybersecurity insurance.

Put the protections in place just like you’re going to fix your bad wiring in your house so it doesn’t burn down and become a threat to your lives and your family. Put the cybersecurity stuff in place so if you’re part of critical infrastructure, you don’t poison your customers, or you’re able to deliver water. You’re not damaging your equipment, your pipes. In any other business, you’re not letting your customers’ credit cards get out into the wild. That’s a big business, stealing credit cards, by the way. In fact, it’s got so many subspecialties I could probably do a Deeper Dive just on the credit card theft business or identify theft. We can add that to the list. That would be a good one to go on the Dark Web with.

KINDSEY: Oh yeah, for sure.

TOM: Anyway, show up for next week and we’ll talk about those things, what you can do to be a good security leader in your organization. See you next week. Thanks for attending.

KINDSEY: Thanks, everybody.