PODCASTS

BTS #82 - Firmware Analysis, Linux Malware, Future of AI

Below the Surface episode 82 was recorded on September 10, 2026, with host Paul Asadoorian joined by Vlad Babkin and Chase Snyder. The conversation moves across several current security stories, but its center of gravity is clear: modern infrastructure depends on trust mechanisms that are useful, fragile, and often difficult to verify from the outside.

The episode begins with signed UEFI shells and Secure Boot bypass risk, then expands into vendor trust, firmware transparency, recovery operating systems, Windows Defender bypasses, Linux process masquerading, firmware analysis tooling, and attacks against network-device management systems. Across those topics, the recurring question is not whether vendors are acting maliciously, but whether defenders can independently verify the firmware, binaries, configurations, and management layers their organizations depend on.

Paul, Vlad, and Chase also spend time on the practical reality of defending Linux-based infrastructure. Attackers do not always need advanced kernel implants when simple persistence, process naming, /tmp, cron, systemd, vendor-specific startup mechanisms, and weak visibility into network appliances are enough.

Key Topics Covered

  • Signed UEFI shells and Secure Boot bypasses: Paul opens with newly discussed vulnerable UEFI shell behavior affecting vendors including AMI and Cisco UCS. The group compares it to prior Framework UEFI shell issues and the broader risk of signed administrative tools that can be abused to disable or bypass Secure Boot. See also Eclypsium’s related research on BombShell and signed UEFI shells.
  • Vendor trust versus independent verification: Chase frames the UEFI discussion as a vendor-trust problem. Vlad argues that vendors can run strong security programs and still sign something unsafe, because complex signing, release, and firmware workflows inevitably produce mistakes.
  • Why firmware transparency is hard: Paul and Vlad discuss whether the industry needs something like certificate transparency for signed UEFI binaries. Vlad points out that publishing hashes alone is not enough, binaries may need to be public, and the problem becomes much larger when extended to firmware and device images beyond UEFI.
  • Network-device firmware scale: The conversation broadens from UEFI binaries to network-device firmware, where the number of device-specific builds, embedded components, and vendor variations makes transparency and verification significantly harder. Eclypsium’s Network Devices work is directly relevant to this part of the discussion.
  • Research access and vendor opacity: Vlad argues that vendors should stop hiding firmware behind paywalls and encryption. His point is that researchers finding vulnerabilities and reporting them is less damaging than attackers finding and exploiting them quietly.
  • AI pressure on vulnerability discovery: Vlad notes that open and specialized security models are improving, making large-scale firmware and binary analysis more accessible to both defenders and attackers. The group treats this as another reason vendors should assume more scrutiny is coming.
  • Citrix Unicon OS and recovery environments: Paul discusses Citrix’s Linux-powered recovery environment for broken Windows PCs. The idea is practical—users can keep working while Windows is repaired—but the group debates the new management and attack surfaces created by adding another trusted bootable environment.
  • Windows Defender fake AV registration: Paul covers research showing how Windows Defender can be disabled by registering a fake antivirus provider. The group compares this to broader endpoint-security impersonation and tampering techniques.
  • Fire Ant, Cisco devices, and credential theft: Chase brings up the Fire Ant campaign, where binaries on Linux management hosts impersonated SentinelOne and Cybereason agents. Paul connects the campaign to Cisco IOS XE targeting, TACACS abuse, packet sniffing, and the value attackers place on network-device credentials.
  • Linux freedom as attacker opportunity: The group contrasts Windows restrictions with Linux flexibility. Paul’s point is that Linux gives administrators broad control, but once attackers gain access, they inherit much of that same flexibility.
  • AI-first Linux workflows: Chase introduces Omarchy, an AI-forward Linux setup promoted by David Heinemeier Hansson. Paul connects that idea to his own use of AI agents to deploy and troubleshoot virtual machine images using console output and accumulated context.
  • OpenAI, Codex logs, and trust in AI systems: Vlad describes a controversy involving claims around AI-assisted mathematical work, private Codex sessions, and the possibility of training on de-identified session data. The group uses the story to discuss data locality, local models, closed model providers, and why sensitive research may move into more controlled compute environments.
  • Dependency-free firmware analysis: Paul highlights Matt Brown’s Moria and Mithril tools. Moria extracts file systems from firmware images without relying on legacy dependency chains, while Mithril analyzes extracted contents for secrets, SBOM components, licenses, and CVEs. Vlad sees the strongest value in Mithril’s structured analysis, while cautioning that any firmware extractor should still be containerized.
  • Linux malware hiding as kworker: Paul explains that recent Linux malware cases have masqueraded as kworker processes, relying on process-name confusion rather than advanced kernel rootkits. He discusses ways defenders can compare process listings, /proc, and kernel-thread attributes to distinguish real kernel workers from userland impostors.
  • Persistence on Linux and network appliances: Paul and Vlad close with common persistence mechanisms: systemd, SysV init scripts, cron, vendor-specific startup systems, MOTD scripts, /tmp, and eBPF. They emphasize that network appliances often lack the visibility defenders would expect on general-purpose Linux servers.

Timestamps

00:00 – Opening setup and episode preview
00:51 – Paul introduces episode 82 with Vlad Babkin and Chase Snyder
01:10 – Eclypsium listener resources and episode setup
02:05 – Signed UEFI shells, AMI, Cisco UCS, and Secure Boot bypass risk
05:02 – Vendor trust, signed tools, and third-party verification
07:13 – Why vendors cannot fully self-verify every signed component
08:46 – Operational ideas for using UEFI shells without trusting them by default
09:48 – Murphy’s Law, TPM, Secure Boot, signatures, and chain-of-trust limits
11:52 – Attestation APIs, transparency logs, and why signed-binary transparency is difficult
14:16 – Firmware access, researcher visibility, and pressure from AI-assisted discovery
15:46 – Citrix Unicon OS as a Linux recovery environment for Windows PCs
18:33 – Recovery environments as management surface, not necessarily new attack surface
19:36 – Network boot, work-from-home constraints, and VPN appliance trust problems
21:28 – Windows Defender bypass through fake AV provider registration
23:13 – Fire Ant campaign, endpoint-agent impersonation, and Cisco device targeting
24:23 – TACACS, credential capture, packet sniffing, and network-device authentication risk
26:22 – Linux control, attacker freedom, and endpoint tampering
27:25 – Omarchy and AI-first Linux environments
28:45 – AI agents, terminal output, and virtual lab deployment workflows
30:43 – Arch Linux derivatives, Manjaro, CachyOS, and SteamOS
32:10 – AI math controversy, Codex logs, and closed-model trust
35:56 – Local models, cloud hardware, data locality, and compute cost
38:49 – Moria and Mithril for firmware extraction and analysis
41:25 – SBOM, secrets, licenses, CVEs, and structured firmware-analysis output
42:47 – Why firmware extraction tools should still be sandboxed or containerized
44:58 – Style Smuggler, Magento/Adobe Commerce, and Linux backdoors
46:51 – Malware masquerading as kworker on Linux systems
48:52 – Why network devices make process visibility harder
49:36 – /proc, sysfs, and sources of truth for Linux detection
51:36 – Persistence through startup services, scheduled tasks, and vendor mechanisms
54:00 – MOTD scripts, /tmp, and common attacker working directories
55:14 – Defender-tooling exceptions, privilege separation, root execution, and eBPF
56:18 – Cisco Firepower Management Center exploitation and configuration theft
57:52 – Cisco Type 7, weak password hashes, FortiBleed, and one-way upgrade gates
59:41 – Closing

Core Eclypsium References

Vulnerabilities, Campaigns & Security Topics

  • CERT/CC vulnerability note — Referenced in connection with signed UEFI shells affecting multiple vendors
  • AMI — Mentioned as one of the vendors affected by the UEFI shell issue
  • Cisco UCS / Cisco Unified Computing System — Mentioned in relation to the signed UEFI shell issue
  • Framework UEFI shells — Prior comparison point for signed shell behavior and Secure Boot bypass risk
  • Fire Ant campaign — Referenced for Cisco device compromise, credential theft, log blinding, and Linux management-host impersonation
  • TACACS / TACACS+ — Discussed as an authentication and authorization system for network gear, and as a target for credential theft
  • Cisco IOS XE — Mentioned in connection with Fire Ant and Cisco network-device targeting
  • Cisco Firepower Management Center / FMC — Discussed as a management system where compromise can expose configurations across many managed devices
  • FortiBleed — Referenced in the discussion of Fortinet password hashes, upgrades, downgrade support, and lingering credential risk
  • Cisco Type 7 password encoding — Mentioned as an example of weak legacy password protection
  • eBPF — Discussed as a possible place for powerful Linux backdoors and detection complexity

Tools, Platforms & Technical References

  • UEFI Shell — Central to the Secure Boot bypass discussion
  • EDK II — Mentioned as the UEFI reference codebase
  • Secure Boot — Discussed throughout the signed-shell conversation
  • TPM — Mentioned as part of the broader trust stack
  • Linux /proc — Discussed as a source of process and system truth
  • sysfs — Mentioned by Vlad as another useful Linux visibility source
  • systemd — Discussed as a persistence and scheduled-task surface
  • SysV init — Discussed as another startup mechanism
  • cron — Mentioned as a scheduled-task persistence mechanism
  • /tmp — Discussed as a common writable location abused by malware
  • Unicon OS — Citrix recovery environment discussed by Paul and Chase
  • Windows Defender — Discussed in relation to fake AV provider registration
  • SentinelOne and Cybereason — Referenced as endpoint-security agents impersonated in the Fire Ant discussion
  • Omarchy — AI-forward Linux environment mentioned by Chase
  • Arch Linux, Manjaro, CachyOS, SteamOS — Mentioned in the Linux desktop discussion
  • Claude, Codex, and local models — Discussed in relation to AI workflows, privacy, and data locality
  • Moria — Matt Brown’s firmware file-system extraction tool discussed by Paul
  • Mithril — Matt Brown’s firmware-analysis tool for secrets, SBOM components, licenses, and CVEs
  • unblob — Existing firmware extraction tool compared against Moria
  • FACT and EMBA — Firmware-analysis tools mentioned by Vlad
  • SquashFS, JFFS2, YAFFS2, UBIFS, CramFS, RomFS, UBI, UImage — Firmware file systems and image formats Paul lists in the Moria discussion

Transcript

Paul Asadoorian (00:51.788): Welcome to Below the Surface. It’s episode number eighty-two, being recorded on September tenth, twenty twenty-six. I’m Paul Asadoorian, joined by Mr. Vlad Babkin. Vlad, welcome.

Vlad Babkin (01:02.950): Hello.

Paul Asadoorian (01:04.798): Mr. Chase Snyder is here. Chase, welcome.

Chase Snyder (01:07.795): Hey Paul, hey Vlad.

Paul Asadoorian (01:10.490): Just a quick announcement before we dig into the topics for this week. Below the Surface listeners can learn more about Eclypsium and how we help organizations assure and secure the infrastructure their operations depend on by visiting eclypsium.com/go. You’ll find our guide to AI data center security, our white paper on eradicating hidden threats in network edge devices, and a case study showing how a major cloud provider used Eclypsium to protect its infrastructure hardware during a merger and IT consolidation. You can also request a demo to see how Eclypsium verifies device integrity, hardens against risk, and detects active exploits across enterprise infrastructure — from the AI data center to the network edge to user endpoints. All that and more at eclypsium.com/go.

Alrighty, I guess we’ll start with the relevant thing, because it came from one of our own researchers. I actually talked about this last night on my Paul’s Security Weekly podcast, but it bears repeating because it’s getting picked up in the news a lot. There was a CERT/CC vulnerability note issued for this. This issue impacts multiple vendors, including AMI, Cisco, and others. It was found by Stas — we really should have just brought Stas on the show, I don’t know what we were thinking. We’re going to have to loop Stas into this podcast eventually; I’ll have to bribe him with something.

But this is similar to previous disclosures we’ve had — Framework’s UEFI shells included — in that these are, for all intents and purposes, debugging and administrative tools that give you a shell within UEFI to perform maintenance and administrative tasks. They often come from various OEMs, and they’re often signed to be trusted by the secure boot root of trust — which can vary just how many systems trust them, as was the case with Framework. Framework signed it with their own key, so only systems that had their key on them would trust their UEFI shell. And as we’ve talked about before, these UEFI shells contain functionality that —

Paul Asadoorian (03:33.518): — could allow a malicious actor to do things such as disable secure boot. This functionality isn’t necessarily limited to memory map commands, as an example — which I think is one of Mickey’s favorites; I always have to ask Mickey how to do that, I’m like, “what do I need to do again?” I think his GitHub is what I refer to every time I have to do it. But you can essentially flip some bits, at a high level, to disable secure boot from a trusted shell. And Stas found, I believe, more of these shells, with some other nuances that impact AMI as well as Cisco’s Unified — UCS, is it? It’s their platform, a chassis you can put a lot of different blades of hardware and software into from Cisco, and they call it their Unified Computing System, or something like that. But it runs somewhere in that mix — it runs UEFI and does UEFI Secure Boot, and they were impacted by this as well. Vlad, Chase, anything you want to add?

Chase Snyder (04:40.177): Unified Computing System. Sorry — system.

Paul Asadoorian (04:58.484): On this one —

Chase Snyder (05:02.965): I mean, something we talk a lot about is vendor trust, right? And this is a situation where the vendors are doing a lot of the right things to try to establish the possibility of trusting them, and yet the very things that are supposed to secure that — they’re signing with their own keys — are still able to be routed around, or used against them, to the point where — I don’t know, what could a vendor do to make you really trust them in a way that you didn’t feel it necessary to have some sort of third-party measurement or accountability of that trustworthiness, at this point, when these exact systems that are supposed to be securing these things are —

Paul Asadoorian (05:47.672): You can’t. Yeah — once these tools are out there and once they’re signed, from what I’ve read — sorry to cut you off there, Vlad — but we were having this discussion about what we can do inside of UEFI. Even some of the folks who maintain the UEFI reference code, EDK II, have had this debate with other researchers you’d recognize who publish tools in this space as well. And when you read that discussion, it’s the same thing we talk about internally: these tools, like UEFI shells, are super dangerous. You can try to make them as safe as possible, but there could be other environmental factors that allow them to be abused. The general sense is manufacturers just shouldn’t release these tools, and they shouldn’t be used. However, some OEMs, from what I understand — as was the case with Framework — use them to let you apply capsule updates to your UEFI BIOS. So there’s that — they’re all legitimate use cases. Can we find better ways? Probably. I think Framework was doing it to support Linux users, ironically enough — I say “ironically” because I run Linux, have Framework laptops, and that’s how I was applying the updates. So, sorry, Vlad, go ahead.

Vlad Babkin (07:13.712): So yeah, in this case, the question of trust in the vendor — no matter what vendors do, you cannot really trust them. Not because the vendor is necessarily bad, but because humans mess up. It’s really hard to actually account for everything. Eventually you’re going to slip up — you’re going to sign a UEFI shell, you’re going to sign something vulnerable, and you’re not going to know. The only people who will reveal that to you are third parties. That’s it. There is just nothing you can do. You can run your security program really, really well, be very careful about what you sign — but the moment some signed tool from you turns out not to be safe, it’s really hard for the vendor to build a tool chain that can actually detect all of this stuff. And if each vendor builds their own, you now have to integrate with twenty-five different tools.

It’s not going to be one size that fits all of them at once. So as a vendor, even if you build out the full stack for verification, customers aren’t going to be inclined to use your stack, for the simple reason that customers potentially have twenty-five vendors like you. Your stack now has to support pretty much every other vendor the customer has, and that becomes a much bigger problem — a product of its own. So this is where there’s a market for companies like us. And there’s literally nothing that can be done about it. So it’s —

Paul Asadoorian (08:46.093): Well, one thing I was suggesting — and it doesn’t solve all the problems — is that if you do need to use a UEFI shell to perform some kind of debugging or maintenance outside of the OEM, the customer shouldn’t sign those UEFI shells so they don’t work with secure boot. So you’d have to go into the BIOS, temporarily disable secure boot — hopefully your —

Vlad Babkin (09:08.220): Mm-hmm.

Paul Asadoorian (09:13.373): — UEFI BIOS is protected with a password, so you have to enter that password to get into the BIOS to disable secure boot, requiring physical access. Then I can do my maintenance, use the shell, and re-enable secure boot. That does help — it’s somewhat inconvenient, and it’s also error-prone. Back to your point, Vlad — if I’m the administrator and I forget to re-enable secure boot, well, our product is going to tell you that you don’t have secure boot enabled anymore, but you could still make that mistake.

Vlad Babkin (09:48.593): Yeah, and the major point about all of this is Murphy’s Law — what can happen will happen. There’s no way a vendor can protect against and detect all of this without building a whole product just for this purpose. And there’s no other way than to verify — you cannot really trust. You can add some trust mechanism, you have TPM, you have secure boot, you have signatures, and yet here we go — somebody signed the UEFI shell, and it’s done. It’s just bad. Not because the vendor is evil or wants to compromise security — it’s just that somewhere down the line, a product team made a decision that didn’t involve the security team. And given how scarce really good security people are in the UEFI space, it’s understandable this will keep happening. There’s just nothing you can do about it — you can’t hire more people, because there aren’t enough people who can solve this problem for every single vendor on the planet.

It’s going to be a perpetual problem. There’s just nothing you can do about it. And even then — if you signed the UEFI shell, did you sign it just for yourself, or for multiple other vendors, if you used your enterprise key? Will a computer from vendor B trust a UEFI shell signed by vendor A? You might want to know that you have this UEFI shell from another vendor on your system. Again, this is pretty much a chain-of-trust problem. TLS has the same exact problem, and it has an incredibly complicated solution called certificate transparency, where every certificate comes with a transparency log, so whenever a certificate is created for a domain, everyone knows the certificate got created. You cannot solve this for UEFI — UEFI just doesn’t work that way.

Vlad Babkin (11:37.380): Even if you did, and everybody suddenly found out, somebody would have to be watching for new UEFI binaries with a UEFI shell inside — and a lot of vendors would become quite a lot more stingy about pushing every single binary they create into a public log.

Paul Asadoorian (11:52.652): Yeah, that was my suggestion — create an attestation API you can query. That’s yet another standard all vendors would have to implement, where you’d make available the list of things you signed and the hashes of those, so we’d have a public record we could query: “What did you sign?” And when you sign something new, it goes into the API and it’s queryable.

Vlad Babkin (12:18.437): Yeah.

Paul Asadoorian (12:19.413): That would be amazing, but it would require so much effort and coordination between the vendors.

Vlad Babkin (12:24.048): Yeah, this would require coordination and agreements that all UEFI binaries are now public, because just publishing the binary name and hash isn’t enough to understand what the hash actually is — you’d have to publish the binary itself. You don’t have to publish the source code, but you’d still have to publish the binary. Vendors are going to be stingy about this, as we know.

Paul Asadoorian (12:31.638): Mm-hmm.

Paul Asadoorian (12:35.519): Right. Right.

Vlad Babkin (12:44.558): And this only solves the problem for UEFI. If you want to solve it for everything, you have to add everything to the list — network device firmware would now need its own attestation API. And while UEFI doesn’t have that many unique binaries, network devices will. It would literally be a database of millions and millions of combined compiled binaries. It’s going to take a huge amount of space. So this problem is much harder to solve —

Paul Asadoorian (12:45.280): Right.

Paul Asadoorian (13:09.493): Mm.

Vlad Babkin (13:14.462): — for binaries than it is for certificates, which are relatively small. And even relatively small certificates cause problems — just go to crt.sh and look at the performance when you search for something like Google, and watch it start choking, because Google has so many certificates. This is not a simple problem to solve. And the only way you can solve it is to verify — you cannot really trust. Even if you build this transparency log, somebody has to be scraping it for UEFI shells and vulnerable binaries that share a common vulnerability. It’s not so simple.

Paul Asadoorian (13:53.201): We’re certainly not going to solve that problem on this show, but those are great suggestions and things to think about as an industry, to combat this problem — because this is not the last time we’re going to be talking about some piece of software that was signed into a root of trust for secure boot that has a vulnerability or an access method that allows for a bypass.

Vlad Babkin (14:16.731): Yep. My suggestion, at the very least, to start solving this problem: stop hiding your firmware behind paywalls and encryption. Let researchers in. It’s going to be cheaper for you as a vendor if researchers find vulnerabilities and report them to you, compared to bad actors finding those vulnerabilities and exploiting them for years until somebody else finds it.

Paul Asadoorian (14:27.124): Yeah.

Paul Asadoorian (14:40.842): Right, right. Because when a threat actor finds a piece of signed software that lets them do something malicious, like bypassing a root of trust, they’re not going to tell anyone — they’re going to use it in their campaigns until it’s discovered. So you’re right, Vlad, I agree it’s better to let researchers do this analysis.

Vlad Babkin (14:58.929): And —

Vlad Babkin (15:03.481): And pressure is mounting, guys. GLM-5.3 Cyber is out on Hugging Face. Open source models, fine-tuned for security, are going to catch up. We’re maybe half a year behind some model at Mythos level appearing in open source, or in closed-source environments for major intelligence agencies — or even cybercriminal groups, who will be hunting for this too. How much do you need to spend? You need to buy two or three rigs with a lot of GPUs — a couple million dollars. A cybercriminal enterprise probably has that kind of money to run the model and analyze everything. So pressure is mounting.

Paul Asadoorian (15:42.132): Mm-hmm.

Vlad Babkin (15:46.506): It’s not because of us.

Paul Asadoorian (15:46.601): Yeah, changing gears — Citrix has added what they call a Linux-powered escape patch for broken Windows PCs, because that never happens — it happens on every operating system. There’s a dual-boot option for Unicorn OS. It installs a separate, Linux-based recovery environment alongside Windows, so if, in their example, ransomware or a bad update breaks Windows, you can reboot your system into Unicorn OS, access Citrix-specific virtual apps and desktops, and approved internal apps while IT fixes your Windows box — that’s how they’re —

Chase Snyder (16:35.189): I think it’s actually Unicon OS — which I wouldn’t bother being pedantic about, except that Unicorn OS is also a thing, and it’s different. But the Citrix one is Unicon OS.

Paul Asadoorian (16:39.430): Thank you.

Paul Asadoorian (16:43.956): It is also a thing — my brain was just doing that thing where it automatically fills in letters based on what it expects to read.

Vlad Babkin (16:50.693): Yeah, extra letters.

Chase Snyder (16:51.133): I feel like they really set you up for that, because why would you name something Unicon and not just go with Unicorn? Come on.

Paul Asadoorian (16:55.936): Yes, thank you for that. So it doesn’t repair Windows, but it sounds like a way for the user to keep working in a different operating system while their Windows install is being repaired — which is super weird, but I think the premise isn’t bad.

As much as I hate dual boot, because it causes so many issues, using it to have a recovery environment you can boot into — Windows has its own recovery environment too, but it’s not intended to give you a space to work while your Windows partition is being fixed. That’s kind of an interesting idea, but I think it opens up an additional attack surface. If I were a threat actor — and Unicon OS, thank you, Chase — it would be signed and valid in the root of trust. So if I gained access to the Windows system and wanted another place to hide, and I was lazy and didn’t want to build my own backdoors, I’d just put my malicious scripts and backdoors on the Linux OS, and maybe force some condition that gets the user to boot into that Linux OS. Now I can compromise what the user is doing there — and I can also infect the Windows system, so that when they boot back into Windows, they’re already infected.

Vlad Babkin (18:33.839): So, two comments on this. First — if you’re privileged enough to do all of that from Windows, nothing’s stopping you from just making another partition or hiding in the Windows UEFI partition. So this doesn’t really add new attack surface in that regard — it does add new management surface, which you now have to protect and update, so you might end up with a really outdated Unicon OS once something bad happens. Second point: it’s absolutely wild that there’s a market for this feature in the first place. How unstable did Windows updates have to become for there to be a company specifically capitalizing on that?

Paul Asadoorian (18:55.105): Yes.

Vlad Babkin (19:15.351): Think about how often this has to happen for there to be a market for it — it’s not a once-a-year event, probably not even a once-every-few-months event. It has to happen monthly for anyone to be willing to sell this. That’s not a very good flag for Microsoft.

Paul Asadoorian (19:36.137): No, certainly not. I also think having the OS local on the system might not be the right answer — maybe something that’s network-bootable, where you maintain the image once. However, as we know — Vlad’s making faces — the network-booting process itself introduces more attack surface for an attacker to potentially exploit.

Vlad Babkin (20:00.604): Yep.

Vlad Babkin (20:04.119): It’s not just that — I’ll kill your idea with two words: working from home.

Paul Asadoorian (20:08.852): Yeah. Mm-hmm.

Vlad Babkin (20:10.908): I don’t have to add anything else to network boot to kill it.

Paul Asadoorian (20:15.956): Although bootloaders can load an operating system over the internet — I don’t think I’d recommend that, but it’s possible, which is crazy to think about. There’s too much in between you and wherever, going across the internet, that can go wrong.

Vlad Babkin (20:31.858): Yeah.

Vlad Babkin (20:37.746): Yeah — either way, you can’t rely on the normal VPN solutions enterprises use for this, because we know how broken VPN appliances are. Unless you have incredibly strong protection for your VPN, you can’t reliably boot from it.

Paul Asadoorian (20:39.136): You don’t want to give the attacker control of your OS — that’s really what it boils down to.

Paul Asadoorian (20:58.923): Yeah.

Vlad Babkin (21:01.072): And even then, you have to get a VPN appliance into every home your employee is in, or the feature isn’t usable. It’s really murky ground — you have to solve the VPN trust problem first, which is actually a hard problem, as we know. Just look at the weekly CVEs for yet another VPN appliance.

Paul Asadoorian (21:28.778): Yeah, for sure. Also, while we’re on Windows — there was an interesting bypass for Windows Defender. As many of us know, Windows Defender lets you install another EDR or AV product in its place. Windows Defender has this facility where you can register another antivirus or EDR solution, and it’ll say, “I’ll stay out of the way — you have something that replaces me, so I won’t activate, and I’ll rely on whatever AV or EDR product you’re installing.” It turns out the way it does that is deep in Windows subsystems, registry keys, and things like that.

This particular research from Improsec talks about a way to register a fake AV provider. I don’t know if this is necessarily new, but it’s a new article I saw — you basically register a fake AV that does nothing, that’s completely benign, kind of like a no-op instruction, and Windows says, “I’ll turn off my AV because you have AV” — except your AV is completely fake. So this is one way to sidestep EDR products, if they work the same way — though I guess it’s more of a Windows Defender bypass than another product’s bypass. I’m curious whether there’s another product you could register on top to disable that too.

Chase Snyder (23:13.545): This is reminding me — I’m trying to remember if we already talked about this on the show — the Fire Ant campaign hijacking Cisco devices to steal credentials and blind security logs. One of the little details noted in there was that they renamed and timestamped binaries to impersonate the SentinelOne and Cybereason endpoint security agents.

Paul Asadoorian (23:41.344): Yes.

Chase Snyder (23:42.098): Made it look like — I don’t know if that would be enough to trigger this, like “looks like there’s SentinelOne on here, I’ll just back away slowly from Microsoft Defender.” But it seems like there’s been a long-running narrative that attackers can disable or fiddle with an endpoint agent on the device, because it operates on the device — if the attacker has access, they possibly have access to tamper with the endpoint agent itself. But the idea of just putting a random file in there that says, “This is the SentinelOne agent,” and another —

Paul Asadoorian (24:23.380): And did they do that in Fire Ant on Windows? Because I read that Fire Ant was specifically looking for IOCs — I believe it was Cisco IOS XE they were targeting in that campaign. We’re actually actively researching and working on detections for some of the techniques used in that campaign. What I also found interesting inside that campaign was that threat actors went after the TACACS server — your AAA provider, your credential authority, basically single sign-on for all of your network gear. They went after the TACACS server and just implanted backdoors in it, so anytime someone authenticated, they could collect those credentials — essentially backdooring it. Also — I’m not sure if this was in the article — but I know that many deployments of TACACS don’t use the encrypted version of the protocol. Cisco added that capability at some point, but originally TACACS would send credentials in the clear, and later they added the ability to use encryption. Attackers — I don’t know if it was this campaign or another one — would compromise a router, firewall, or some kind of gateway, put it into packet-sniffing mode, and just sniff for TACACS credentials going across the wire, then reuse those credentials. That’s an interesting campaign — probably more interesting than the fake AV on Windows systems, in my opinion. But I’m more of a networking and Linux nerd than a Windows nerd.

Chase Snyder (25:58.346): Yeah, really interesting campaign. The binaries impersonating the SentinelOne and Cybereason security agents were specifically on Linux management hosts in the Fire Ant campaign.

Paul Asadoorian (26:15.915): Mm.

Paul Asadoorian (26:22.719): Yeah, that’s interesting. Linux gives you a lot more access, and it’s a double-edged sword — it gives you the freedom to do whatever you want with your computer, and even in the enterprise, when you’re running Linux, that operating system has those same capabilities. One of the most famous memes I ever posted to social media — got almost a million views — was of someone asking Windows, “Hey, can I uninstall Microsoft Edge?” and Windows says, “No, I can’t let you do that.” Then a Linux user asks, “Hey, can I delete the bootloader?” and Linux says, “Yeah, sure, go for it.” You have the freedom to do that. The other side of that sword is that once attackers get on a Linux system, they have those same capabilities to manipulate it.

Chase Snyder (27:25.427): Yeah, this could be a whole other topic, but I’ve been tracking DHH — David Heinemeier Hansson, the 37signals guy, invented Ruby on Rails, I think. He’s been working on and promoting — I think it’s an Arch Linux fork or something, I barely know what I’m talking about here — called Omarchy.

Paul Asadoorian (27:50.634): Mm-hmm.

Chase Snyder (27:55.774): He’s describing it as an AI-first, AI-forward Linux deployment, where basically to do all the setup you plug in your AI API keys and get going. I saw a great image where someone had deployed Omarchy on their machine — they’re advertising it as being really capable even on super old machines, like “bust out your ten-year-old laptop and put Omarchy on it and join the future.” They had set it up —

Paul Asadoorian (28:07.709): Yeah, I gotcha.

Chase Snyder (28:25.589): — put it in front of a mirror, so the webcam could see what was happening on the screen, and were like, “go ahead and work on it this way” — so it’s watching itself on the screen while doing, I forget what kind of task, some sort of optimization for that particular device. But yeah, imagine —

Paul Asadoorian (28:33.386): That’s awesome.

Paul Asadoorian (28:37.382): In the mirror, yeah.

Paul Asadoorian (28:45.769): I mean, even better — give it the terminal output. A lot of times I’ve tuned AI agents to monitor console output — for example, in some of our internal projects, it can monitor the serial console output from a virtual machine, make changes, boot the machine, see if it’s booting okay, see what the error is, troubleshoot it, and try again. I kind of need to make that into a skill.

Chase Snyder (28:55.722): Yeah.

Chase Snyder (29:03.753): Mm-hmm.

Paul Asadoorian (29:15.121): When we deploy new virtual images to our virtual lab, that’s the process my Claude instance goes through to deploy new things, and it saves me so much time troubleshooting — very similar to what you’re describing. I’ll say, “Here’s a new VM image, go research the web, figure out how to deploy it, then use all of your memories and context and go deploy it in our virtual lab.” It sets up the networking, sets up the hostnames, does all the things. I had a request from a team this week — “we need this new image” — and I said, “Yep, I’m on it,” and it churned for about an hour while I did other work, and then it said, “yeah, it’s deployed now, it’s all set — here are the credentials, here’s everything.” So it can definitely do that. But it’s important that you’re prompting it and giving it the appropriate context. If you just took that cold and said, “go set up a hypervisor and —”

Chase Snyder (30:00.670): That’s incredible. What a great feedback loop. But imagine —

Paul Asadoorian (30:13.972): — deploy this image, it’s not going to have all that context. It has probably almost a year’s worth of context that I’ve been curating so it can get to that point.

Chase Snyder (30:22.995): Yeah, I think that’s kind of the premise of the Omarchy thing — that it’s set up so it’s hard to shoot yourself in the foot with it, but still easy to use AI to get it to do what you want. It seems extremely popular, or at least there’s a lot of noise about it in the forums I watch. I haven’t tried it out yet — I do have an old MacBook lying around, maybe I’ll try it.

Paul Asadoorian (30:32.597): Mm.

Paul Asadoorian (30:43.668): There’s a lot of popularity — I use an Arch derivative for my desktops. CachyOS is also growing a lot in popularity — it’s relatively new. Actually, it’s funny you mention that, Chase — I read an article recommending which Arch Linux derivative to run, listing pros and cons of each, and I was reading it thinking, “should I stay on Manjaro, or should I make the jump?” because I’ve heard about some of these other ones. The newer ones just don’t have the track record yet — Manjaro has that longevity and history. Though they’ve had some internal disputes and issues within the team, largely from just being around so long and people not agreeing — I think they’ve worked it out since then, but it’s interesting. SteamOS is based on Arch Linux too.

Speaking of AI — there’s, what I understand, Vlad, a dispute about math? People stole math from each other? I haven’t read up on this a whole ton, other than knowing it sounds like people stole math from each other.

Vlad Babkin (32:02.875): Yep.

Vlad Babkin (32:10.077): So yeah, the events, as I saw them unfold, looked like this: OpenAI drops a massive bomb that AI just solved a Millennium Prize Problem, worth a million dollars, from this famous institute. Now, I’m not going to pretend I’m a mathematician and understand this problem all that well, but from what I understood, it’s a physics-slash-math problem, and they solved a version of it that other physicists and mathematicians don’t quite appreciate as much as the original — while it’s technically allowed by the conditions of the prize, they used external forces, which mathematicians usually don’t want to use; they want to solve it without external forces. That’s point one.

Point two is that OpenAI apparently had access to Codex logs — which they deny, but also confirm, but then still deny — that mathematicians who’d been working on this problem for a year had used Codex sessions. And then OpenAI went ahead and tried to bully one of the mathematicians into pulling another co-author off the author list, because that co-author worked at Anthropic. When the mathematician disagreed, they started threatening him — that his career was going to end because of this one decision. What the hell, OpenAI — are you a big bully now? And apparently they also deny that they used the —

Paul Asadoorian (33:34.514): Right.

Vlad Babkin (33:39.997): — logs, but then they also claim they might have used de-identified data from the sessions, which might have slipped into their training and helped their AI reach the conclusions. So they don’t fully deny it — it’s a very slimy situation where they say, “we didn’t use it, but we might have.” The best comment I’ve seen on this is: “Hey, we made an omelet, but we might have used the de-identified eggs from another chicken, which might have made the omelet bigger.”

Paul Asadoorian (33:49.963): Yes, yes.

Paul Asadoorian (34:06.636): Mm.

Vlad Babkin (34:10.642): They might have had the actual solution in the training data, which obviously they’d find out later down the line. And they used it after hearing rumors that these mathematicians were getting close to a solution — so they had a tip that they might have something in their system with a solution, and just jumped on it. It’s a very bad look for OpenAI, to be honest. Moreover, their solution isn’t even to the harder version of the problem — the harder part is the one without external forces, and they still didn’t solve that. So the achievement that AI can do this is real, but it’s always going to have this little shadow — that it potentially wasn’t really AI, that they might have just grabbed the solution from a private Codex session, which casts a shadow on everybody using Codex.

Paul Asadoorian (34:55.276): Mm.

Paul Asadoorian (34:59.060): Yeah, true. And I think they’re highly motivated — OpenAI, Anthropic — to keep up, and to use that private data to keep pace with the open models. They have to maintain that edge in the market, because free and more open models are catching up.

Vlad Babkin (35:06.526): To use your private data.

Vlad Babkin (35:13.362): Yep.

Vlad Babkin (35:19.198): Mm-hmm.

Vlad Babkin (35:24.452): Open models are at worst six months behind — don’t overestimate the gap. Chinese models aren’t as far behind frontier as you’d like to believe; they don’t have as much of a lead as they want you to think. And judging by this, AGI is still not here, no matter what OpenAI says to us. And by the way, considering their behavior, and that their model weights are actually closed, “OpenAI” is a bit of a misnomer — let’s agree on that much, at least.

Paul Asadoorian (35:33.398): Mm-hmm.

Vlad Babkin (35:52.070): They’re behaving like a big bully in this situation, which is not very deniable.

Paul Asadoorian (35:56.896): Well, there’s also the debate we constantly have with my nerd friends — yeah, we want to use local models, but how do we get the hardware to run them when everyone’s bought up all the storage and RAM for who knows how many years, which makes it much harder to run these local models, because the price of hardware is now exorbitant.

Vlad Babkin (36:23.634): Yep, and outside of the exorbitant price, even buying and renting cloud hardware is going to be better. You can’t buy your full hardware chain, but you can rent hardware and install whatever software you want. It’s going to be a lot harder to sneak your data out of a setup like that. So while it won’t truly be a local model, it’ll still have a level of data locality you won’t get from Codex or Claude — and it’s still going to be pricey as hell.

Paul Asadoorian (36:53.525): Yeah, yeah.

Vlad Babkin (36:53.766): Your budget is going to be a lot higher than what you’d pay for a monthly plan from Codex — though we don’t know for how long, because the prices for those plans keep getting higher, since companies are strapped for compute.

Paul Asadoorian (36:59.369): Yeah. Yeah.

Vlad Babkin (37:10.526): And again, at some point open source models are going to catch up, whether you like it or not. At that point your choice will be giving all of your data away, or using a local model and buying a million dollars’ worth of hardware. Not doable for a private person, obviously, but very doable for a company — imagine a large enough company spending a million bucks on hardware to run some of its tasks locally, especially the more sensitive ones. Your regular coding you can outsource, but more sensitive research will probably start moving into silos, especially because of what OpenAI just did.

Paul Asadoorian (37:55.955): Yeah, a hundred percent.

Vlad Babkin (37:58.534): And OpenAI didn’t even fully deny it — I’ve read their statement on it — they claim they didn’t use the data, but they might have used de-identified pieces of it in training.

Paul Asadoorian (38:12.039): Mm. Shady, for sure.

Vlad Babkin (38:12.561): Yeah — shady, shady, shady. They’re preparing their defense for once it comes out that they actually used the data, so they can fall back on “we didn’t intentionally use it.” They’re preparing that now because they probably know something — if they knew for sure they hadn’t used the data, they’d have come out with a full, confident explanation. But they couldn’t. What they can’t say says a lot more than whatever they claim publicly.

Paul Asadoorian (38:45.503): Right, right.

Paul Asadoorian (38:49.467): I’m super excited about two tools released by our friend Matt Brown at Brown Fine Security. For those of you who’ve done firmware reverse engineering — I’ve done more than my fair share over the years — one of the challenges with that work is that firmware often has multiple different file systems inside it. A lot of these are older, esoteric file systems, and the code and libraries to interpret and read them are very old and antiquated, so you end up in dependency hell trying to get the tools to work, because the tools that analyze firmware rely on these older, poorly maintained libraries.

That puts you in dependency hell with unblob, for example — I’ll run unblob inside a container so it can figure out what dependencies it needs, and someone else has already made that container work, so I just use their image and get much better results. But it’s still a pain — you might need a different tool outside of unblob. So what Matt Brown did was code, from scratch, tools that don’t have the file system dependencies and can unpack file systems within firmware that are SquashFS, JFFS2, YAFFS2, UBIFS, CramFS, RomFS, UBI, and UImage. I think I’ve seen almost all of those inside firmware at some point. That tool is called —

Vlad Babkin (40:45.810): Yeah.

Paul Asadoorian (40:47.859): — Moria. And Matt also introduced Mithril, which analyzes those extracted file systems in a standard manner — I believe it outputs JSON. Both tools emit structured JSON for automation and agentic workflows, but Mithril analyzes those extracted files for secrets, SBOM components, licenses, and applicable CVEs. I haven’t tested these tools yet, but they sound awesome — I need this in my life.

Vlad Babkin (41:25.105): Yeah, it sounds — sounds really awesome. Sadly we don’t have Matt Brown here today — we did have him on the show once, by the way. What I will question is the value of the first tool. Mithril obviously has a lot of value, because there aren’t a lot of really good tools that hunt for SBOM data, secrets, and licenses — there are just no good tools, hands down. You can find some, but they’re usually slow and very non-standard —

Paul Asadoorian (41:31.627): Mm.

Paul Asadoorian (41:31.627): Mm. We did — yep.

Paul Asadoorian (41:48.064): Mm-hmm.

Vlad Babkin (41:56.738): — you don’t get really good, clean output from them. Think of tools like FACT, or EMBA — all of those produce output in a very non-standard format. But what I will question is the value of Moria itself as a tool, because in a real deployment, you’ll want to containerize Moria anyway, since you can’t guarantee Matt Brown’s code doesn’t have vulnerabilities when extracting all of these file systems. So you still want to put it in a container — but if you’re going to be containerizing it anyway —

Paul Asadoorian (42:01.526): Mm-hmm. unblob, yep.

Paul Asadoorian (42:22.239): Right.

Vlad Babkin (42:26.631): — is it easier to maintain compatibility with all those file systems in a single new tool, or just use a lot of tried-and-tested dependencies in the container you have to build anyway? So in real production, I wouldn’t say Moria adds a lot of value compared to unblob. But Mithril, hands down, is just more or less unique.

Paul Asadoorian (42:47.787): Does Moria have any intelligence to handle anything malicious that might be in the image, or —

Vlad Babkin (42:58.511): unblob already tries to handle that — so it’s not new. And even if Moria does have that intelligence, are you going to trust it? I wouldn’t run this tool without a container anyway, because extracting file systems is incredibly hard. It’s not that I don’t trust Brown — he probably did a lot of security work on this — but chances are he slipped up on at least one of these ancient formats, which are sometimes poorly documented and might have a lot of unexpected inputs. It’s not so easy to code all of this — that’s my main point. It’s not that the tool is bad, I’m just questioning its value over unblob. But then again —

Paul Asadoorian (43:03.967): Yeah, yeah.

Paul Asadoorian (43:07.988): Right.

Vlad Babkin (43:16.731): — Mithril is definitely the much more valuable piece here, because being able to go over everything Moria just extracted is insanely strong.

Paul Asadoorian (43:55.135): Yeah — my AI assistant wrote this quote for me, and it nailed it, which it doesn’t always do: “Firmware analysis should not begin with rebuilding an archaeological museum of abandoned Python scripts.” Yep, a hundred percent — it summarized it nicely.

Vlad Babkin (44:15.592): Yep. What I’m saying is, somebody has to maintain unblob’s image — you have to maintain it, because it is, again, a museum. So it does solve at least a real level of the problem.

Paul Asadoorian (44:21.908): Yes.

Paul Asadoorian (44:25.801): Right. Yes.

Paul Asadoorian (44:30.687): Mm-hmm.

Vlad Babkin (44:31.058): What I’d question is making a whole new tool versus coming up with specific libraries for each file system that we can audit and validate separately, and then integrate them into a new tool, or into unblob, or other known tools. If you make a standalone library for these, we could put them into 7-Zip, for example, and into all of the other known and private tools.

Paul Asadoorian (44:53.843): Mm-hmm.

Paul Asadoorian (44:58.659): My last story here is interesting — the Style Smuggler campaign. It’s a Linux-based campaign, but the initial access vector is through Magento and Adobe Commerce versions, which wasn’t so interesting to me. What was interesting is where they got to once they deployed backdoors on the Linux systems.

Overall — and I don’t want to give too much away here — I look at these every single week, it’s a huge part of my role at Eclypsium, understanding the threat landscape, primarily on Linux, since most network edge devices are Linux-based. When I look at what attackers are doing, it’s often not very advanced. The way they persist and hide inside Linux-based systems isn’t very advanced — I don’t see a lot of kernel-level rootkits being deployed. One can theorize why: it could be that they don’t need it, if our visibility into a lot of these Linux-based systems is already low, or that people just aren’t scrutinizing and monitoring their Linux systems the way they do Windows systems — I think that’s part of it too. Also, a kernel-level implant relies on the kernel, and how the kernel is built can differ across every device and distribution — sure, you might run Debian, but have you modified, extended, or customized your kernel in a way that makes it unique, and harder to implant a kernel-level rootkit into? Perhaps.

Vlad Babkin (46:50.118): Yeah.

Paul Asadoorian (46:51.911): So what I find are very basic techniques. One I’ve now seen in three different cases of Linux malware: one of the processes they run for persistence, they’ll call it “kworker.” Now, if you’ve ever done a process listing on a Linux system, you’ll see multiple kworker processes, and they’re in square brackets, typically with certain monikers or text after the word “kworker” that indicates something specific — these are kernel processes, not userland or other services. They’re specific to the kernel and, in fact, contain — and this is my area of research — other attributes you can search for that indicate they’re genuinely a kernel worker thread. Largely they hang off process ID 2, which is the kthreadd process. But attackers are just spinning up services in a couple of different ways, calling them “kworker,” masquerading them as kernel processes to evade detection from someone doing a process listing. So where my research is right now is: where can I look on Linux systems — and this gets into kernel internals, though not even that deep, maybe about midway in — to identify the real kernel worker threads versus something masquerading as one? If the attacker is operating from inside the kernel, with a kernel-level rootkit, they can make those look much more believable and harder to detect. But again, in these three recent cases, they’re just spinning up processes that look like kworker, and that’s how they’re hiding. It’s all in how you look.

Vlad Babkin (48:52.210): Yeah, it also depends a lot on what kind of Linux device you’re targeting. If you’re running on a network device, it’s very likely you only have a very basic process list, where you can’t find any of the parameters you just described at all — because, again, network device vendors running Linux aren’t giving us visibility into any of that.

Paul Asadoorian (49:06.346): Mm.

Vlad Babkin (49:12.306): We only get a process list with just names — sometimes not even process IDs. So suddenly we can’t see any of this. Why would you even need to hide, if at best you’re only going to see the name, and see that it’s “kworker,” and there’s no chance you get anything else? Sometimes you have to exploit the device itself just to be able to monitor it, which is bad.

Paul Asadoorian (49:12.533): Right.

Paul Asadoorian (49:29.436): Mm-hmm.

Paul Asadoorian (49:36.819): You can also explore /proc as another potential source of truth. Again, attackers can manipulate a lot of different things in various ways on Linux systems, but if you use /proc as one source of truth and your process listing as another, when you start to see discrepancies, that’s where you can start to identify suspicious behavior.

Vlad Babkin (50:06.003): Yep. Yep. You have to look at /proc — and by the way, that’s part of what our product does. I’m not going to reveal much more than that, for obvious reasons, but that’s definitely a thing to look at. And /proc has a lot of tasty data besides just process listings.

Paul Asadoorian (50:06.567): Suspicious behavior.

Paul Asadoorian (50:15.206): Mm. Sure.

Paul Asadoorian (50:26.117): So much. Yes. Yes.

Vlad Babkin (50:28.477): And another spot to look at for tasty data — as a small spoiler — is sysfs. You can find a lot in sysfs as well. That’s the stuff you can do pretty easily; I’m not going to reveal some of the deeper stuff, but let’s just say there are quite a few more interesting files on Linux systems that you can read and get a lot of really useful data from.

Paul Asadoorian (50:36.234): Mm.

Paul Asadoorian (50:44.393): Yeah, yeah, yeah.

Vlad Babkin (50:51.427): Outside of that, nobody’s really looking, because nobody has to. A lot of the Linux systems under attack nowadays are network devices, and you can’t even run an agent on those network devices to do any kind of series detection.

Paul Asadoorian (51:07.848): Yeah, it makes detection hard — that’s my life right now.

Vlad Babkin (51:09.361): In this case, yeah, we’re talking about attacks on Adobe Commerce and Magento, so those aren’t network appliances, for those reasons — this is an outlier. Attacks on commerce software aren’t as popular as attacks on network devices right now. Your network edge device gets attacked constantly; your commerce platform gets a notable exploit maybe every half year or so. Draw your own conclusions from that.

Paul Asadoorian (51:36.457): Yeah, and I see sometimes many similarities between attackers going after an application running on a Linux server and attackers going after a network device — they’ll sometimes use the same or similar techniques. One huge area of abuse, outside of trying to hide processes with cute names like that, is scheduled tasks and startup services. I see that a lot, both on the Linux server side and the device side. If you want to persist as an attacker using things like systemd, SysV init scripts, cron — I don’t want to give them more ideas, but there are other ways to invoke processes regularly on Linux systems that, if you throw an AI at it and ask, “tell me all about systemd and all the different ways” — you’ve got user directories that can do that, other ways within systemd, systemd’s own cron-like facility for running a scheduled task. Are you looking there, to see if someone’s invoked some program that isn’t part of the regular system, that could be malware? Those are the categories the IOCs I’m seeing tend to fall into.

Vlad Babkin (52:47.622): I —

Vlad Babkin (52:56.028): At service —

Vlad Babkin (53:03.655): At the risk of giving attackers extra ideas, I have to throw in another thing: vendor devices usually have their own mechanism to run stuff, and that mechanism is usually unique to the device.

Paul Asadoorian (53:08.180): Mm.

Paul Asadoorian (53:13.035): Mm-hmm.

Paul Asadoorian (53:17.450): Right.

Vlad Babkin (53:18.001): So, yeah, that gets bad very, very fast. Network devices almost universally don’t rely on systemd, and don’t rely on SysV init, to run their periodic tasks — that’s something we observe in the wild. So, again, because attackers probably already know all of this — at least attackers with some sense — I’m mentioning it so vendors have a chance to actually react, because you also have to take into account that the device you’re looking at doesn’t just have systemd or SysV init — it often has the vendor’s own systems for running periodic scripts and startup tasks too.

Paul Asadoorian (53:55.401): Right.

Paul Asadoorian (54:00.158): And, you know, even the MOTD — Message of the Day — is actually a script that executes every time someone logs in. That’s yet another way to do that.

Vlad Babkin (54:09.811): Yup. Yup. Yup. Because, again, I want to give defenders at least a fair chance to find all of these.

Paul Asadoorian (54:12.859): Right. And some of these techniques date back fifty years, to Unix — I mean, some of these you could probably still do today, but you could also have done them fifty years ago on a Unix system, which is frightening. Also, the last thing there is /tmp — that’s another directory you have to monitor.

Vlad Babkin (54:32.157): Yeah, this is not new.

Paul Asadoorian (54:40.363): A lot of the IOCs I’m seeing are processes running out of /tmp. Typically, on a Linux system, you wouldn’t have a regular process running out of /tmp, but attackers use that directory because you can almost always write files to /tmp — it’s almost always there. So if you’re developing malware meant to run ubiquitously across different systems, /tmp is great. From a defender’s perspective, monitor that very, very closely.

Vlad Babkin (54:55.390): Yeah.

Vlad Babkin (55:14.899): Yep. I’ll give you another hint — some devices have /tmp mapped to a different folder. And another hint: some cybersecurity tooling might be forced to run from /tmp as well, because there’s literally no other way. So potentially your defender tooling might actually run from there, and another piece of defender tooling might not be aware of it — so if you run two or three defender tools, they may trigger one another. You need to not be trigger-happy if you run periodic scans, and you have to make some exceptions for those — review that the same scan triggers the same contents each time, because defender tooling might run periodically, and you need to make it as hard as possible for attackers to capitalize on your exceptions. And because privilege separation on those devices is often so bad, everything runs as root — so an attacker who gets code execution is immediately root; they don’t have to hunt for a local privilege escalation. eBPF modules are the same.

Paul Asadoorian (56:18.591): Yeah, a hundred percent. That’s a huge area for backdoors — I probably should have called that out; it’s similar to a kernel-level rootkit, since you’re actually running inside the kernel with eBPF. That’s certainly been in play for a few different campaigns I’ve seen as well. In addition to Firepower Management Center — when we talk about remote code execution as root, I believe that was one that came out this week, two vulnerabilities in FMC being exploited to give you root, which I probably should have put in the show notes. It begs the question — similar to attackers going after TACACS — that going after your management system means I don’t just get one router’s config, I can get all your router configs.

In fact, in that campaign, what I read was that attackers had customized scripts that, once they compromised FMC, would run and collect all of the configuration files for every piece of network gear you’re managing with FMC, package it up, and send it over the C2 channel to the attackers. And remember — your configs contain credentials, a map of your network, and the services running on all of your other devices, among other things.

Vlad Babkin (57:41.716): Mm-hmm.

Paul Asadoorian (57:45.203): — among other things.

Vlad Babkin (57:52.725): Yep. And not every vendor does password hashing all that well. I’ll mention the infamous Cisco Type 7 hash, which you can decode straight into a password without any brute force.

Paul Asadoorian (57:56.745): Yeah. Mm-hmm.

Paul Asadoorian (58:04.275): Right.

Vlad Babkin (58:04.948): But to give Cisco credit, they deprecated it and are public about it being really bad. They also deprecated the next hash, which was based on something like MD5 or SHA-1, or something not very secure. They deprecated that too — Cisco is one of the better vendors in this regard. For other vendors, you might have surprises.

Paul Asadoorian (58:19.092): Yep. Yeah, Fortinet did the same thing and deprecated theirs — we talked about it with FortiBleed, right? So running the later versions of firmware on these devices puts you in a much better position, and can sometimes automatically deprecate those older, weaker hashes on your systems.

Vlad Babkin (58:39.710): You still have to make sure those weaker hashes aren’t stored somewhere in backups. With FortiBleed, I believe the bug was that the old hash was still stored to allow for downgrades — which, to be honest, is a mistake. You shouldn’t do that.

Paul Asadoorian (58:44.284): Yep, that was FortiBleed, right? Yep.

Paul Asadoorian (58:59.784): Right.

Vlad Babkin (59:00.104): If you make a hard gate with a hash update as a vendor, you should say, “if you upgrade to this version, I’m going to kill all of your old hashes, and if you downgrade, your users will lose their passwords.” That’s how you’d handle it. Yeah, it’s a risk, but in this case it’s a reasonable one — leaving the old, exposed hash around forever just to allow downgrades turned out not to be a good trade-off. Certain updates have to be one-way gates — if you go through the upgrade, downgrading will cause problems, and that’s something vendors have to take into account.

Paul Asadoorian (59:41.436): Awesome. Well, Vlad, Chase, thank you so much for appearing on today’s show. Thank you everyone for listening and watching this edition of Below the Surface. We’ll see you next time.

Back to all Podcasts