7MS #424: Cyber News - Everything is Pwned Edition

Brian Johnson -

This podcast is sponsored by Arctic Wolf, whose Concierge Security teams Monitor, Detect and Respond to Cyber threats 24/7 for thousands of customers around the world. Arctic Wolf. Redefining cybersecurity. Visit Arcticwolf.com/7MS to learn more.

Hello! We're back with our pal Joe "The Machine" Skeen (a.k.a. Gh0sthax) who has prepared some awesome and actionable news stories for us to digest. Today's stories include:

Transcript

Brian: Hi, everybody. Welcome to our July episode of the 7MS Cyber News, which is practical news for information security professionals; both aspiring and seasoned. And our goal as always is to hand-select news that is useful, meaning you can actually do something with what we share. And in management lingo, you’d call that actionable, good stuff.

My cohost for today’s episode is Joe Skeen, also known as Gh0stHax on our 7MS Slack and on Twitter. And to me, he’s sometimes known as Joe, the machine Skeen, which I don’t know if you even appreciate that nickname, but it just has that DJ buzz to it.

Thank you so much for being here, Joe, welcome. What kind of goodies do you have lined up for us today?

Joe: Yeah, thanks Brian. Hey, I’m hunkered down here in my bunker with face mask and hand sanitizer. I hope you’re the same, but I think we have some great stories lined up.

More importantly, we have some key takeaways for everybody. So get out your to-do list because we have some homework for you. So let’s get started.

So right on the heels of the last new segment where we talked about the Verizon DBIR and vulnerabilities as the entry point for hacking being on the downswing, we’re actually going to talk about a few recent vulnerabilities to kind of counter that narrative a little bit, but just a bit. These vulnerabilities have received a lot of press, so I feel kind of incumbent to talk about them today, and really even after the news died down.

So we won’t talk about all of them because they all have kind of a similar theme; Palo Alto, Citrix, F5, they’ve all received quite a bit of press.

Brian: Yeah, and then just as we’re getting prepped to record this, we had a big Tuesday. So we’re recording this on the 15th and yesterday was a big one. I think you would call it a level 10, right? Vulnerability for-

Joe: It’s a level 10.

Brian: … for Microsoft DNS server. And right now, at the time that we’re recording this, there’s a lot of information about it, kind of how the researchers ripped it apart and figured out how to leverage it. The only working proof of concepts I’ve seen have been of Rickroll. So if you run the exploits, you get Rick-Rolled, which I really appreciate.

But we will put an article in today’s show notes from checkpoint.com. They have a really nice take apart of the vulnerability. But in the meantime, I think the name of the game is just “Patch” for now. Is that as much as we want to say about it?

Joe: Yeah. And I think, again, that was kind of happy Tuesday vuln land before we recorded. We’ll have to wait for the exploits to get published. Like you said, there’s the Rickroll. So we’re not sure how bad it’s going to get, but man, the hits just keep on coming this summer. I mean, like you said, right on the heels of us saying vulnerabilities are down, they’re just back to back to back.

So there’s quite a few more that were just published this past week. One day after Patch Tuesday, there was a bunch more that dropped even outside of Microsoft. There’s Adobe, there’s Oracle. I see SAP had a critical vuln. My timeline is just full of vulnerabilities this week. And the SAP one was released again at a perfect 10. So we need to keep an eye on that.

And like minutes before we recorded Brian, Cisco has released a bunch of critical ones and it’s just all over the board. So there’s criticals, there’s highs, there’s just like, “Here we go again!” It’s like every week this week, it’s just been vuln after vuln after vuln with perfect 10s, so …

Brian: And I can’t imagine, I mean, if I wasn’t already kind of out of the sysadmin, network admin role and doing security, I think this would be about enough to make me want to quit my job. Because I can’t imagine the pressure you must feel whether you’re working for an internal organization or you’re an MSP and you got clients calling you, and it’s like, “Patch my stuff tonight!” And I imagine these people are just working around the clock.

And I don’t want to get too ahead of ourselves here, but I know some of these are yes, perfect 10, as you said, and they hit all the-

Joe: Not in a good way.

Brian: Right, all the juicy, exciting, but terrible things coming together where it’s got a score of 10, and they say the vulnerability is easy to exploit. You can automate it, you can do it over the internet, and best of all, there’s more — a lot of these don’t require valid credentials or any kind of coding skills to take advantage of. But let’s get it, what should we talk about first?

Joe: Yeah, so this first story, I know everybody talks about it. It’s been the talk of the news prior to the long holiday weekend. So I think most 7 Minute Security listeners probably already know about that. But we have some key takeaways at the end. So this vulnerability, it’s deemed so dangerous that it yet again, received a score of 10.

And one concerning thing about the timing of this release was the fact that it was announced on Wednesday just before the long US holiday weekend. So it actually took a couple of days for it to gain traction in the media. And I think people around Saturday started paying attention to it.

But when the patch was released, it indicated the vulnerability could allow attackers to take full control over unpatched systems. So by Sunday, we saw honeypots and people starting to tweet out and talk about honeypots were picking up the signatures from the active exploits.

For this particular one to be exploited, that TMUI, which is the Traffic Management User Interface has to be exposed to the internet. And most of you would say, “Well, there you go. That’s like Darwin worthy right there.”

Brian: You deserve it.

Joe: Yeah, but however, if you’ve never managed an F5, to me, they’re just ugly and complicated. So I don’t mean to offend F5, but interface has been around a while, it’s complicated, it’s a beast to configure. But you might ask yourself why someone would expose the managed interface to the internet.

Brian: That was exactly going to be my question. And I’ve never touched an F5, so I don’t want to kick somebody while they’re down. But is this just something that’s on port 443 or is it on like a weird port? I mean, would you have to intentionally put a risky config in place or is it pretty common? Like are we seeing this all over the internet?

Joe: Well, it turns out under some default configurations, the TMUI was automatically exposed as part of the initial configuration. You know a big iron like this is kind of set it and forget it. I mean, you may be changing other settings, but something like that, you may not have even noticed.

So a quick Shodan search, by lots of people, including me, because I was just curious about it — I saw varying numbers, but we’ll just land** around 8,000 of the TMUI interfaces exposed. It’s not a huge number, but what makes it so concerning is the types of companies that use F5. Because F5, like I said, it’s kind of considered the big iron for load balancers and traffic shapers. And they’re installed at those really large companies, big banks, government, ISPs. I mean, they’re just all over the place.

So to me, one of the key takeaways for us (and this is where your homework comes in) is start scanning your network with just simple things like Shodan, and see if there’s stuff that you didn’t even know was out there. Because you know, Nessus scans are great, and I know a lot of people do them, but take in some other sources.

So one comment I wanted to drop in there is that around the holidays, and I don’t know if you took advantage of this Brian — but it seems like almost every year, Shodan offers memberships that are like five bucks or something.

Brian: Yep, I scooped one up. Absolutely.

Joe: Yeah, and I did too. And so I think again, as a takeaway for your homework list, is that just get a membership, even expensive if your company will let you. They’re so darn cheap, and there’s a lot of other sources out there as well.

But getting back to my point, just like Google dorking, Shodan has a lot of searches that you can narrow down to your external IP space or your org name and so on. So just keep an eye on things that shouldn’t be exposed or don’t look right. I know personally, I’ve found things like RDP access and Azure that’s open that you wouldn’t even have thought because it’s outside of your IP space.

And next, you should make sure you have a way to allow people to report vulnerabilities to your organization. I really think as blue team operators and security teams, we need to get over that fear of saying, “If you find a vulnerability in our network, here is how you report it to us.”

I still see it really, really hard for reporting vulnerabilities to companies that don’t have like bug crowd or something else. So this should come right to the security team and not a person in customer service.

I’ve also seen other organizations that had vulnerabilities that were reported and it went to the customer service team and they thought it was phishing or something else. So they just didn’t even pass it on.

So you don’t have to have a bug bounty program to thank people for submissions either. I’ve seen a lot of companies that just give away swag or gift cards for stuff that you didn’t know about. There’s lots of different things that you can do on that front. So that’s kind of your homework.

The other thing I wanted to do is give a shout out to some of the bug hunters like Kin, he’s on the Slack channel conundrum. He was busy scanning and reporting those F5 bugs to proper bounty programs. And nice work and kudos to Kin for doing that because we need security researchers out there helping on your behalf. So when they report stuff, make sure that you thank them. Don’t be like, “Hey, we didn’t authorize you to do that security scan.” They’re doing a service for you, so be nice to them.

One last important point; frequently, attackers may only do things like scrape credentials. So in this F5 bug in particular, some of the earliest scripts were actually just dumping password files off their devices, so they could circle back around later and get access to them. So they may not have actually been dumping shells or any active attacks, it’s simply a credential scrape. So you have to assume that if you didn’t patch that you were probably compromised on that weekend before you got back.

Brian: Gosh, wow! There’s a lot to unpack there. And I think a couple of thoughts, first of all, the Shodan membership is yeah, a no brainer, very low cost, and provides a really nice service. And I’m really interested in your comment about having the vulnerabilities at company name dot com email or a form or something like that — because one of the last times I worked for a big MSP management was very much like, “Don’t even suggest on our public marketing website or anywhere else that we could have issues.” Like kind of just go the ignore route and if I’m watching-

Joe: Ignorance is bliss.

Brian: Right, and if I’m watching the chatter on Twitter I’ll see researchers like Troy Hunt and others go, “Hey, does anybody have a contact at Joe Skeen Inc because I’ve got a hot one here.” And then sometimes, a lot of times they just want to do the right thing. And to your point, they don’t even want a huge fat payout. I mean, they might, but if you give them a public thanks and maybe a t-shirt, they might be fine. But a lot of times it looks like the intentions are right.

But if they get frustrated, if they get shot down, if they get radio silence or just flat out denial from the organization, then they can be tempted to do things like, “Well, I’m just going to plop this on pastebin or I’m just going to write up a blog article and then it’s egg on your face,” and then the company back pedals later and tries to save face, which is just embarrassing for them.

On the Shodan and kind of vulnerability scanning note, you mentioned Nessus. And that made me think of I’ve got, and I will put in today’s show notes, a really nice Nmap script that I’ve used in a number of cases where we just wanted to keep an eye on a company’s external surface over the course of ISP changeover, or firewall changeover, where they wanted to make sure, for example, just 80 and 443 were open and that’s it, never change.

And we just ran this script every night through like a two-week cut-over, and then the Nmap script would do a diff from night to night. So it would only output a little blip simply to say everything’s the same or changed a little bit.

And that was super handy because then one evening we got that report. It had a little bit of text in it and it was just weird management port at 8445 is now open. And we were able to just quickly go, “Hey, let’s shut that down.”

So, I mean, even if it’s not something that costs you money like Shodan or Nessus, I mean, you can do a lot just with Nmap and hook it into some email scripts just to keep a better eye on your external perimeter.

Joe: Yeah, for sure.

We’d also kind of mentioned the Citrix and Palo Alto vulnerabilities and they have a similar narrative as the F5. But the key takeaways I want to point out about Citrix, they really downplayed the vulnerability. So I think maybe from a press perspective, people may not have taken them seriously.

But if we can learn anything from the F5 vulnerabilities, once the hackers are aware of issues, they expand upon the original attacks and find new ways or they look at what’s in the patch and they’re like, “Oh, well, here’s a new thing we can try.”

And we found, I think some of what was missing in the coverage, is that there was a lot of temporary suggestions that F5 gave as temporary workarounds that were actually bypassed. And I’m not sure if people picked up on that. So if you put in some of those temporary measures until you could actually get it patched, because it’s kind of hard to take some of that big iron down from a production site to patch them.

So all three vendors have actually continued to update their guidance on the vulnerabilities and they saw variations on the exploits as well. So make sure you follow up on the stories to see if there’s additional guidance, even after you have patched and continue following the stories.

And another point I wanted to make is like, when US Cyber Command is tweeting about those vulnerabilities, you should pay attention. They have threat Intel that most of us aren’t privy to. So you can add SAP and pay some attention to US Cyber Command Twitter as well.

Brian: I was going to ask about that. It’s been a while since I’ve been on the sysadmin, network admin side, like I was saying, so I’m not on these mailing lists anymore. But do you know from past experience at all — I mean, Palo, Citrix, all that stuff, do they have a general sign up for our show-stopping security alerts? So when there is stuff like this, you get it right to your inbox.

I mean, we’re in the security community, so we usually hear ramblings of it from various inputs. But if I’m again, just heads down just doing my job every day, I kind of want somebody to tap me on the shoulder and be like, “This is your weekend buddy, you’re doing this.”

Joe: Yeah, and I think one of my disappointments with a lot of the vendors is that they do have those, but it seems like it’s kind of exceptionally hard to find. And so there’s a website called vulners.com — they do a pretty good job. Because one of the things that I don’t like about the press now … like the Hacker News is a good site, but you tend to just get the headline and all of the details are missing. And you know, it’s only those headline grabbing things when there was 10 other vulnerabilities were almost as important that you may have missed.

So I think there’s a lot of different feeds. It depends on how you like to consume those, either via email or other areas, but there’s other ways to get those feeds. I think the vendor post themselves, you’re managing like 25 different feeds and it’s kind of hard to keep track.

Brian: Right. Yeah, probably at the end of the day, and maybe this is something we could poll listeners for some feedback. It seems like you would never want to just take, for example, a set of email mailing lists and be like, “I’m good. I’m signed up for all the key vendors and I’m tunnel visioned.” I mean, I think you really have to take that ownership upon yourself to really look at a wide variety of social media and the vendor’s website and yeah, places like the Hacker News. Otherwise, you can just easily miss so much.

Joe: Yeah, exactly.

So let’s move on to our next story. And our next story comes from ZDNet and we’ll link to it in the notes for the podcast. But they reported the US Secret Service sent out a security alert again, yet another source of security alert.

So this is, I guess, one of my gripes about the government too, is that they also have like six different sources. Sometimes the NSA will send stuff, sometimes the Secret Service, sometimes Cyber Command, and it seems like they all send different stuff. But that, again, is a gripe for another day.

But basically, what the story is saying is that they send out an alert last month to US private sector and government organizations warning about an increased risk of hacks against managed service providers.

So if you recall, in the last news segment, we talked about managed service providers being responsible for the decrease in dwell time and intrusion discovery. However, one of the downsides that can be overlooked is that MSPs are increasingly getting attention from the cyber-criminal gangs specifically.

The article goes on to say secret service officials said they’ve been seeing threat actors used hacked MSPs to carry out attacks against point of sale systems to perform business email compromise scams, and to deploy ransomware. So MSPs are juicy targets because they allow access to multiple customer sites from a single entry point.

So from a quote in a phone call today with ZDNet, Kyle Hanslovan, who is pretty popular (and I probably butchered his last name) — but he’s that CEO from Huntress Labs and his company provided support. Just his company provided support to 63 incidents of MSP hacks in 2019. And that resulted in ransomware on customer networks. But he suspects the total number of incidents were well over a hundred last year and probably more.

So what’s also not getting a lot of press is the fact that these MSPs are getting hacked and their customers are getting attacked as well. And I’ve only seen like two or three well-published reports on that.

Brian: Yeah, I’m sure it’s happening a lot more than we think. And that’s something that, again, back from my sysadmin, network admin days, I was nervous about as we used whatever it was; Zenith and Kaseya, some of those big — you log into one portal and then you got your fingers in 50 different clients. And just thinking about how one compromised cred … and multi-factor wasn’t a big deal back then. But it could give me one click access to pop a shell or pop a remote desktop session on a device.

And I think a lot of sysadmin, network admin types tend to do the pop in and out of boxes. Maybe they’re not good about doing the control-delete thing or having a timeout with a GPO. So just the ability to pop open a remote management session might land you on a desktop logged in as a domain admin and the world’s your oyster. So that just kind of gives me a heart attack.

And one of my old coworkers had this little jingle, he always used to laugh at how companies would have these very powerful, deep rooted remote agents on all their critical systems. And he would say, “This is basically spyware that you pay for.” These agents are on the machines, they’re telling some portal everything about what goes on in that system, has full rewrite access to all the data and then writes to do all the things. And while it’s awesome from a convenience standpoint, I don’t know, I still kind of break out in cold sweats every once in a while from a security standpoint.

Joe: Yeah, for sure.

Brian, I was going to have you mention — it kind of reminds me of something that you’re doing there at the 7 Minute HQ, been helping folks actually audit their MSPs, which I think is super cool. So, I mean, we strongly recommend that if you’re adding an MSP or during upcoming contract renewal, that you should put in a right to audit clause in the agreement. And then make sure you actually exercise that right to audit them.

So beyond just looking, if they have policies and they’re following them, are they’re eating their own security dog food? Like Brian says all the time, just your two other podcasts were about that. Are they having regular pintests by qualified third parties, following their patching parties? Like I think the onus is on you as the customer. Don’t rely upon what the MSP is saying, get that right to audit and exercise it. Any thoughts on that, Brian?

Brian: Yeah, we’ve worked on a few of those engagements together and I think the clients found it extremely valuable because maybe they’re going to work with a managed service provider that is taking over pretty much the entire network. I’ve seen some firms go to the point where almost everything is just a remote entry point. It doesn’t matter what device you’re on, kind of the traditional AD landside is dissolving for them. And it’s just whether I got my tablet or desktop, laptop, I just remote into remote.company.com, I sign in and off I go.

And really it’s a bit of a black hole because you don’t really know how the GPOs are configured, you don’t know what the backups look like, how encryption is used and none of that stuff. And so, we’ve worked together on some engagements to really sit down on a conference call with that MSP and with the client and ask the hard questions.

And a lot of times the answers we get are, “Yes, we’re in process getting policies and procedures together” or “We’re working towards aligning with a security framework.” And then we turn around to the customer and say, here’s your punch list of things that you really got to fight for yourself about, and you got to be a squeaky wheel about with this group to make sure — yeah, to your point — that they’re getting the regular third party pentesting, that they’re actually formalizing policies and procedures, that the gaps in there, maybe backup routines or encryption implementation get addressed.

Much like you can’t just assume I’m spinning something up in the cloud. Cloud equals secure by default. You can’t assume that putting all your eggs in an MSP basket is going to end up with unbroken eggs, I guess.

Joe: Yeah, exactly. And the fact that, I mean, imagine when you were managing all of that stuff, now multiply it times 25 customers or 50 customers. You’re just even more busy in that role. And so oftentimes just like for you, for them, some of those things get pushed to the back burner, but they have an obligation to set a higher standard than you would yourself. That’s why you outsource to them.

So MSPs can really be a help to an organization and increase their security posture in a lot of areas. We have to make sure just like you said, that they eat either on security dog food and that they’re practicing what they preach.

Brian: Right on.

Joe: So we have one final story and it kind of is a little bit of a tangent from some of our previous stories. But this story comes from Threatpost and the title is Most Popular Home Routers Have ‘Critical’ Flaws. Like everybody knows this.

But I think the link PDF is actually a good read on A, how they tested for the vulnerabilities, because it was a really exhaustive testing process. They extracted all the firmware from the routers and did some automated analysis of 117 routers. They all, I mean, every single one of them had some type of vulnerability.

Now there was a pretty far range on those vulnerabilities and I don’t want to dissect the entire story. But if you have time, the paper is a really good read. And if you want to see how the security is on your particular routers — what has me concerned, and this is the homework part again, is that so many people are working from home.

So we kind of used to joke about it, a little bit that the network edge is now in people’s home. Well now more than ever, the network edge is in people’s home. The amount of people working from home is just substantial.

So I think one of the things in the guidance that companies sent out, we should really think about sending out a reminder to all of those people working at home. Even if again, only a small percentage actually take us up on this, it’s better than doing nothing at all. We just send out a reminder to say, “Patch your router. Make sure that you have strong passwords on your router. Make sure that you don’t have default credentials on your router.” Just those general security things that we take care of in the office, we really have to remind our home users because of the explosion in the number of users working from home.

Brian: Makes me think that someone or multiple companies could be well to do for those that service more smaller businesses or home customers, to put some kind of package where you would come out and do some of that stuff for them. Do like a home network security health check where you check in like, “Do you use a strong Wi-Fi password?” And then actually getting in and patching the router.

Because man, some of these brands, I mean some of the N cap stuff you would pick up at Best Buy, these dealings and these NETGEARs, I know some of them are getting better. But even figuring out how to do the firmware update can be a pain. I mean, I’ve got a little higher end unify stuff, but I still need to be technical enough to go looking for the right device, looking for the right release.

And then sometimes, it’s kind of a convoluted web interface to upload an image and select it as the primary and then boot to it. And I go, man, if you’re just kind of Joe CFO at home and this is not your bag, they probably need a bit more step-by-step, a little more handholding. And I think somebody could really clean up if they were so motivated to make a home offering.

But yeah, there wasn’t a ton of great news from the summary of this report. They said one of the common issues is that a lot of these devices run on a Linux 2.6 kernel, which hasn’t been updated or maintained for a lot of years.

And then I was kind of scrolling through to see, well, do they kind of nudge one vendor as being the leader as far as security. But not really, they did just say Asus and NETGEAR in general seemed to prioritize device security more than others. But I didn’t really see anybody in there getting a gold star, did you?

Joe: No, and you would think — I mean, you see NETGEAR and Asus did all the security conferences in like the rooms and stuff like that. But I think the article just pretty well points out that all of them don’t have great security. So we need to at least be asking users to patch them. It’s not easy to say, “Buy one of these and you’ll be good.”

But the other thing to think about too is that home routers tend to be a set it and forget it item for folks at home as well. So they may have had the same router for 10 years and that’s time to kind of push them to get them upgraded and get newer equipment as well.

I think one of the concerns from IT departments is, “Well, we don’t want to end up supporting these devices.” And I think there’s a fine line that could be had there.

One company I worked for would actually encourage home users to call into the help desk and they would provide as much support as they could possibly provide. I’m sure the help desk, other help desks would be like, “Oh, we don’t want to do that. That’s just too much work.”

But again, with the security perimeter moved into everyone’s house, I think it’s more important than ever to make sure that we’re taking the security out there with them.

Brian: Yeah, I like that. One just small tangent on that note; I was asked during a presentation recently about Comcast ISP equipment, for example — and I’m sure other ISPs do this too, where they install the ISP connection, they get your router all set and going. And then there’s just a sticker on the back and says, “Your SSID is one, two, three, four, five. The password is blah.”

And they asked me at the end of this webinar, they said, “Is that fine? Is it fine to leave it at that password?” And I said, “You know what? It probably is because it was a strong password, really long, really strong password.” But the tinfoil hat guy in me a little bit says, well, what if there is a database of all these SSIDs matching with some algorithm to figure out what the pre shared key is. And so I encouraged them to change it to something they generated and controlled.

And I seem to be in the minority there. A lot of people thought I was wearing too big of a tinfoil sombrero. So I don’t know, where do you sit on that?

Joe: Yeah, I mean for me, personally, I would never take a default of anything. And I think there was another story a while back about a lot of them actually ended up being the Mac address of the wireless access point, which is drivable from the initial handshake. So just by doing your initial beaconing to it, it was the Mac address that they would use. And so they’re like, “Well, if that’s what it is, then that’s the default. All you have to do is just run your wireless probes and you’ll get the Mac address back as it’s trying to do the handshake to log in.”

Brian: That makes sense.

Joe: But there’s a lot of variety in that. But they have to be keeping a list of the default passwords because they need to know that if it was put on that router, that that went with that particular router, how did they know they’re not reusing them? So are they randomly … you just never know. I personally just wouldn’t trust anything, to get slapped on the back of a router like that.

Brian: And I think the more we want to try to help people, whether they be at work or at the home office, I think we, as security people, want to be able to give them kind of some tried and true rules. And I think like change passwords from the default is just one that shouldn’t really have any exceptions.

Joe: Exactly, yeah. It shouldn’t be, “At this situation, do this and this, do that.”

Well, Brian, that’s all of the stories I have for today. Is there anything else you wanted to mention before you take us out?

Brian: No, that was great stuff. Listen, Joe and I are praying for all you sysadmin, network admins out there who probably are getting no sleep because you’ve got your hands full. You’re up to your eyeballs in patching and config work to do. And the kind of kick to the stones a little bit could potentially be this Microsoft wormable DNS vulnerabilities. So we’ll kind of keep an eye on that, but we’ll get all these links in the show notes for you and wish and pray the best for you as you bang through this.

And Joe, as always, thanks. This is just fantastic stories and great conversation and always enjoy doing this. And we’ve been getting good feedback on it. So would love to keep it up.

Joe: Yeah, sounds good. We appreciate the feedback. And if there’s different directions you want us to take and different stories — like Brian said, at the very beginning, we really want to try and not just cover the 1525 stories like the other “news shows,” we want to kind of dig under the covers a little bit and actually provide some useful guidance for all of that news you’re getting.

Brian: Well, thanks a lot, and we’ll talk to you next week.

Joe: See you, Brian.

Brian: See you.