BTS #56 - Vulnerabilities & Backdoors In IT Infrastructure
In this episode, the hosts discuss various cybersecurity topics, focusing on Nvidia vulnerabilities, the implications of backdoors in technology, and the importance of secure boot and certificate management. They also delve into SonicWall’s security challenges and the ongoing debate of building versus buying security solutions, particularly in the context of AI infrastructure and cloud services.
Subscribe
Below the Surface – Episode 56
Recording Date: Wednesday, August 6th, 2025
Host: Paul Asadoorian
Guests: Vlad Babkin, Chase Snyder
NVIDIA Vulnerabilities in AI Infrastructure
NVIDIA Triton Framework Vulnerabilities
Paul: Let’s talk about Nvidia first up. Nvidia Triton, which is an open source framework for interfacing with Nvidia hardware to do AI workloads. Did I get that right?
Chase: That’s sounds about right. Inferent server. Yeah. Good enough. That was the container toolkit one. Yeah, that was Nvidia scape, which was in their container toolkit. And this is in the Triton inference engine, but that goes to show that the Nvidia vulnerability landscape, no shade on Nvidia, but the more that they become sort of foundational to the AI infrastructure, the more of this kind of thing is going to be discovered and also be newsworthy.
Paul: Yeah, it’s super interesting because NVIDIA makes a lot of hardware and software to support AI workloads in various capacities. I’m still trying to learn what software and hardware they have and why people use it. That’s what I get hung up on. I’m like, but what does it do? Why do people use it?
Trail of Bits Discovery Using Semgrep
Chase: Yeah, high risk, high risk vuln in widely deployed Nvidia stuff. Certainly a newsworthy, newsworthy event. And shout out Semgrep. One of, I don’t know, Vlad, I know you’re a grep enthusiast. It’s somewhat part of your, Semgrep. What do feel about Semgrep? The semantic, semantic.
Vlad: So, SMGrab is kind of similar idea to grab, but it’s based on rules to detect different bad source code patterns. So, he did some grabbing around there and found interesting stuff as far as I understand. So, in this case, if I think it was, yeah, public C source code, I guess. So, it seems to be open source stuff.
Paul: Yeah, because Triton, sorry, and we should have said that Nvidia Triton is open source software, hence having access to the source code to use Samgrep.
Vlad: Yeah, and some grab has a whole bunch of public rules which can detect interesting stuff. So if you use it on the code base that didn’t previously use some grab, you will find thousands of things it can find for different languages. The point is it has some level of false positives. So don’t expect all of your findings to be true positives, but it tries to detect code smells. And in this case, it actually succeeded.
HTTP Protocol Complexity and Vulnerabilities
Paul: It’s interesting how many vulnerabilities still trace back to handling of the HTTP protocol in addition to the parameters that are passed along to the application via HTTP. Vlad, we talked about your discovery of the bypass authentication or command injection. The vulnerability we talked about was also manipulation of HTTP headers to bypass things.
Vlad: Yeah, the problem with HTTP is how incredibly complex it is for how simple it looks like. What can be simpler? Just send a verb, a path, HTTP version, a bunch of headers in the body, right? Well, not so simple. There is also chunk encoding. There is like all of the different upgrades to HTTP2 or to web sockets. There is HTTPS handling and a lot of fun with that. There are special headers which can arrive for which you can send, which somehow alter processing of HTTP. There is also stuff like how proxies interpret these headers. So if you stack enough proxies, you might have a lot of fun vulnerabilities just based on proxies handling different order of headers differently.
Vlad: I honestly question the sanity of even trying to write your own HTTP parser, because it looks like they actually did write their own, at least if I’m correct. But my question would be, why would you write your own HTTP parser? What do you gain? I mean, especially in this case, like the most part of performance impact here is obviously inference. You aren’t going to win a lot from ultra-fast parser.
NVIDIA Export Control and Backdoor Concerns
Paul: NVIDIA was also in the news. It says the headline was kind of, it was a little dizzying to read, but I think I get the gist of it now. It says NVIDIA is defiant over backdoors and kill switches in GPUs as US mall’s tracking requirement and calls them permanent flaws that are a gift to hackers. So what I read from this, like you read the headline, you’re like, why is NVIDIA like so eminent about putting back doors in their firmware and GPUs. That’s not the case at all. It’s completely, it’s the opposite.
Vlad: It’s about, it’s, yeah, they’re saying they didn’t put backdoors like all of the normal companies do, but we all know the truth, right? So let’s put it this way. Like, if you’re pressured by a lawmaker to put stuff in your hardware that you produce, you will probably put this stuff in whatever you produce.
Historical Context and Security Implications
Vlad: So, there was a close historic comparison for me that comes to mind would be when US was doing export ciphers for TLS and for SSL, when specifically NSA required that SSL vendors to put special export ciphers for the open source version of open SSL. And those ciphers specifically were weak for NSA to be able to spy on everything. And what that resulted in was not what exactly we wanted, like sure, NSA got their wish to spy on those, but what happened is that everybody else also got to spy on all of the communications.
Vlad: Moreover, because it’s a GPU, it’s not exactly connected to the Internet directly. So even if you put such controls, let’s say, okay, let’s say government wins, let’s say Nvidia puts controls into their GPUs to be able to kill switch them remotely, well, China is just not going to connect them to the Internet. So we ain’t turning off Chinese data centers, but this makes all of the data centers weak if it is on all of the cards. Imagine someone actually managing to find how to trigger this backdoor on their own and just shutting down your data center permanently.
UEFI Secure Boot Certificate Expiration
Microsoft Certificate Rollover Timeline
Paul: I want to navigate into the UEFI waters because that’s something that we know and we love and we focus on here at Eclipseum and there’s some really cool things that have been posted. I want to start with Matthew J. Garrett’s blog post titled, Secure Boot certificate rollover is real but probably won’t hurt you. So the high level is Microsoft like certificates can’t live forever. have to have, well, usually you configure a certificate to have an expiration date. You can configure them without an expiration date. But Microsoft has issued certificates that are part of the secure boot process that are expiring in October of 2026 is when the production PCA 2011 certificate expires.
Paul: And there’s a couple of other certificates June. So the UEFI CA 2011 in June of 2026 and that I believe is that’s a key exchange key a CAC certificate and that’s the CAC certificate that is the third party CA one that will sign third party software like Shim.
Certificate Chain Complexity
Paul: Also Matthew J. Garret says, if you grab a copy of Shim that shipped in Fedora, it’s not directly signed with the Microsoft UEFI CA 2011 certificate. Instead, it’s signed with the Microsoft Windows UEFI Driver Publisher certificate that chains to the Microsoft UEFI CA 2011 certificate. So now we’re the certificate chaining, which makes things wonderfully complex to talk about. He says, it’s not unusual, intermediates are commonly used and rotated, but we look more closely. It was issued in 2023 and expires in 2024. Older versions of Shim were signed with older intermediates. A large number of Linux systems are already booting certificates that had expired, yet things keep working.
Why Expiration Date Enforcement is Problematic
Vlad: So the problem with checking the expiration date is A, your computer does not have magic clock. We just think with the universe and know the exact time and date. Like this is point one. So bypassing this defense is very trivial. Just set the system clock to whatever date you want. So in this case, this is way too early in the boot process to actually be able to have a reliable date like TLS certificates that you are used to.
Vlad: Point two is that expiration date here is not exactly the perfect protection against malware. So this is why we have DBX. So if you sign malware, we want to revoke the signature immediately and not wait for a year to revoke it, obviously. And point three is the only reason to rotate the certificates is to make sure that we don’t use the same key forever because it can be brute-forced, getting faster, faster, faster, faster and faster.
Vlad: But expiration dates there is a very murky thing to actually touch. okay, let’s say we decide to rotate them. Let’s say after 2026, all of the computers are using the new certificate without the old one. Then what? What happens if you want to install some older hardware and it doesn’t have, it needs a driver which is just not signed? What are you gonna do next?
Gaming Anti-Cheat and Secure Boot Requirements
Kernel-Level Anti-Cheat Concerns
Paul: Well, what’s going to happen is the potential here is people disable Secure Boot altogether. However, if you’re I don’t know if you guys do any gaming. I do not. But I happen to see that it was at Battlefield six, I think was the game. And it was all mentions of UEFI and Secure Boot. And I’m like, why? That’s because the game in order to run requires that Secure Boot be enabled.
Chase: Yeah, lots of Reddit discussion of like Valorant is another, you know, multiplayer online shooter that requires a secure boot and UFI to be enabled. There’s so much instruction about how to bypass it.
Vlad: Yeah, in this case, the reason for the games is that at one point somebody decided it would be great to put anti-cheat into the kernel as a kernel driver and whatnot, because like, and make it permanently turned on, which is like as concerning as we just discussed about in video backdoor, because technically now you have a Valorant developer who has a kernel driver in your system which is constantly turned on and monitoring all of your actions. That’s as friggin’ concerning.
Vlad: So at that point I decided I’m never gonna play any games which put kernel anti-cheats for this reason. And by the way, to shout out to all of the players of those games, rethink your life choices.
Tencent Ownership Revelation
Chase: I just read that Riot Games, which owns or makes Valorant, is owned by Tencent, which is a Chinese multinational company. And now it all ties back to the earliest discussion of…
Vlad: So guys, congratulations! All of the Valorant players out there, congrats! You have a Chinese spyware on your computer, which is legal. Well, legal. At corner level, which bypasses your antivirus probably as well.
Vlad: Yeah, just a disclaimer, we are not accusing Valorant or Riot Games of anything, just in case, but the possibility here is very obvious. Yes, it’s pure speculation. So that like, we don’t want to defame anything, but it’s a very questionable thing to have.
ShadeBIOS: Advanced UEFI Malware Research
Paul: But while we’re on the topic, I did want to talk about shade bios. So I went through all of the besides Las Vegas, Black Hat and Def Con talks. And I pulled out like what’s interesting to Paul, but then I also highlighted like what’s most interesting to Eclipseum in terms of research that’s being presented at these conferences. And perhaps the most interesting one and most concerning one for me is security researcher Kazuki Matsuo who’s a Japanese security researcher at the firm FFRI Security.
Paul: Now, if that sounds familiar, you may have read Kazuki’s paper that was published last year on SMM in malware. That paper, SMM Pack, Obfuscation for SMM Modules with TPM Sealed Keys, was also authored by the presenter. And then if you go back to Blackhat last year, Kazuki also presented the option ROM talk at Blackhat last year. We just described what option ROMs are. Kazuki presented on how to put malicious code in an option ROM to get code to run inside of UEFI at pretty early stages in the process.
ShadeBIOS Technical Details
Paul: But Kazuki’s like, hold my beer. This year at Black Hat, he’s like, I’m going to write malware that just purely lives in UEFI. the gist of this research, and it’s being presented pretty much like right now as we’re talking. So if you’re at BlackHat, I hope you attended this talk. But the current research called ShadeBios is when UEFI does things like exit boot services, it’s supposed to remove memory regions that were used in the boot process. And this research looks like they’re preserving some of these regions in order to run malware in those regions.
Chase: Yeah, that’s what, that’s what he says. The researcher says in the article is, I’m deceiving the OS loader by changing the memory map. so basically saying like, by the way, keep running this memory region where the BIOS is located while the OS is running.
Vlad: So I don’t think I dig into it, into this research specifically. But it sounds really interesting, let’s put it this way. If you get malware to run this early on, a lot of antivirus software is just gonna die. Like you cannot do anything with it.
Detection Capabilities and Challenges
Paul: Well, that’s what the researchers claiming is that this evades all EDR antivirus software, which I believe. I don’t, and even just based on little conversations we had earlier in the show, I believe we’re well positioned to detect this. And if you read the talk notes, Kazuki is going to present how to detect it using memory forensic techniques in a specific tool. But I’ve sent that already to our research and engineering teams. I think we’re already aware of this tool that lets you look at the memory regions. I think we’re pretty well poised to detect this based flat on what you said. We’re looking at what’s on the spy flash. We’re looking at certain regions of memory.
Vlad: Yep, we are also collecting SPI and MMI dumps. I’m not sure how well that will detect malware that’s implanted in the side SPI. So it might work. I’m not sure about the shadow bias, shade bias, whatever it’s called, research in this case. And if he actually kills how they’re presented to us, so can he present something else than what we see? that would be… Yeah, so this is a cat and mouse game. Our project might already be positioned to detect it, but advanced enough attacker can do a lot of interesting stuff.
