What a Terrible Day to Have a Computer

Well.

Today* was fueled by caffeine and adrenaline.

If you're any sort of system administrator, or know any system administrators, you know what I'm talking about.

If you aren't and don't, the short version is that multiple pieces of software that form the foundation of large parts of the web were announced to have security vulnerabilities, and there was a mad dash to get things patched before (more) hell broke loose. It started yesterday, escalated several times, and is probably not actually over.

The unavoidable question is: so, is this was our lives are just like now? A life of running fire drills?

To start at the ending, my answers are (and please, realize that these are coming at the end of a long, over-stimulated day, and I'm talking about stuff that's still ongoing, for which we definitely don't have the complete story):

Now: I'm not a professional system administrator: I have been in the past, but now I'm a professor. I know quite a fair bit about computer and network security, including doing research in the area, but I'm not an infosec professional. I still play a small but significant role in administering my research group's computing infrastructure, as well as the testbed infrastructures we run for computer scientists, which have tens of thousands of users. So, take this piece with these grains of salt.

Fundamentally, I like administering computer systems - it's what drew me to computing research in the first place, and it's why I administer a bunch of stuff at home that I really would be, objectively speaking, better off not administering.

I also like working on teams with smart, competent, resourceful people, which is why I don't just computer at home by myself, I work with other faculty, with research staff, with students.

Administering computer systems is a social activity.

Here's the first thing that has gone horribly wrong in computer security recently.

There are certain secret numbers that, if you know them, you can use to impersonate secure connections to just about any server on the Internet. It may (or will absolutely not, if you are familiar with web security) surprise you to learn that this is actually the normal state of affairs, and it's by design: we, collectively, trust a certain set of organizations, called Certificate Authorities (CAs), to create and safeguard these numbers - you can prove that you know the number without revealing what it is, and if you do this, we trust you to attest to the identity of things like websites. This is called Public Key Cryptography, and it's extremely cool.

This is not the problem (well, it kind of is, but that's a whole different story).

The problem is that yesterday, we learned that one of these secret number is, and has been for about a year: 4.

Yes, technically 4 is just as random as any other integer, but this does seem bad, right?

The thing is that these numbers are supposed to be very very long, such that guessing them is infeasible. You can guess 4. I can guess 4. Anyone can guess 4.

I don't think we know the full story yet, but what it looks like is that one of the CAs that almost all of us trust generated some random secret numbers and threw away almost all of the digits, leaving them with secrets like 4, 56, and 155. They've been using them since early 2025. Why didn't someone notice this until now? I think it mostly boils down to (a) the fact that it's not as obvious to notice as my simplistic presentation here makes it sound, and (b) generally speaking, this is not something people are usually looking for (it probably will be now!)

The scramble yesterday was to stop trusting those numbers to secure websites, and to reduce our trust in the CA that "created" them, eMudhra. Did you know you trusted eMudhra? Well, you did.

This kind of situation has been planned for, and there systems in place that help mitigate it in some settings (primarily web browsers). For example, there are public lists maintained of numbers you shouldn't trust anymore: you can "revoke" the number 4, and some (but by no means all) clients will learn about it eventually. It's bad when these secrets are compromised, it's a legitimate major security problem, but things like this have happened before, and they're going to happen again, and there is an (imperfect) system in place to help make it not quite so disastrous.

Thus, for most of us, yesterday afternoon was largely a matter of making sure that software was generally up to date, and having a good laugh at memes about the number four.

Later in the day, the PHP foundation, which is responsible for the PHP language that runs a large part of the web (estimates on how much vary) announced that they were going to drop a patch for a security vulnerability that affected every single version of PHP going back decades.

Under a common way of scoring the severity of vulnerabilities, this one got a 9.2. Out of 10.

So.

If you don't work in security, you might wonder: why tell us that they were going to release a patch, instead of just releasing it?

Well, once you release the patch, people can start trying to figure out what it fixed. And once they do that, well, they can start developing exploits, and using them.

And this is where, however many words I am into this post, I finally say: "AI"

Based on what we think we know now about the vulnerability (-ies, plural, as it turns out), they are somewhat complex to exploit. A few years ago, we would have probably assumed that it was going to take a while to make an exploit. In fact, that's exactly what happened: a few years ago, apparently, the potential for this vulnerability was noticed, but people didn't, at the time, figure out how to exploit it, so they considered it hypothetical and didn't prioritize fixing it. Recently, it was re-discovered, and an exploit found, with the assistance of an LLM.

But today, the assumption is: once the patch was out there, people would use LLMs to develop exploits, and this would happen fast.

The current state of the world is that bots scraping web pages to collect training data for LLMs are ubiquitous and relentless. My research group has had to battle them many times in the last few months, and we haven't "won," we've just kept the load on our servers to a level where they're still usable for their actual purpose. It seems very likely that some of these bots are run by people malicious enough to turn them into botnets that start exploiting this new vulnerability.

So, the current best practice is: let the OS vendors, big companies, clouds, etc. know, so they can start making packages for people to update to. Announce a time, give everyone a little while to get ready.

So, my research group got ready. We know which of our services use PHP - we wrote them! We divvied up responsibility for the major services between us. I threw together a quick scanner to check the rest of our IP space in case there was stuff we had forgotten (in fact, there was.)

Some of the machines we had to update were in datacenters. On campus. In our university's off-campus datacenter. On rooftops around our campus. In boxes attached the sides of buildings. In buses (yes, literally) - these, and the rooftop, and building-side nodes are part of our POWDER wireless testbed. Of course, it's 2026, and we visited none of those places - I was in my office, everyone else was at home.

Four of us were ready at 8 AM our time, including someone who was supposed to be leaving on vacation. Shortly before 8, we shut down many of the webservers, and firewalled off others.

As it turns out, the patch was huge. (It also came out something like 30 minutes early.) Looks like a re-write of the entire memory allocator, but I haven't dug too far yet. The vulnerability was, as expected, a Remote Code Execution (RCE) vuln - send the right traffic to a vulnerable server, it doesn't matter what application is running inside of PHP, you get to run something else, with the permissions of the webserver, of your choosing.

These are people I've worked with for 10 to 25 years, and it's great to work as a team. We co-ordinated over slack. We all know what each other know. When something goes wrong, we know who to ask. When something needs doing, we know who can do it. Everyone has a common goal, and overlapping, but diverse, expertise.

We build packages for our most of our FreeBSD servers ourselves, and we couldn't start until the patch was out. So a fair amount of time was just waiting for builds to complete. Some of the Linux distributions we use were "late" getting packages into their own repositories. More waiting. We installed new PHP packages, did very light testing, brought webservers back up. In one case, we had been planning to move a website to a new server anyway, but this forced our hand - there was some last minute databases syncing and whatnot to do. We had another server that simply cannot be updated, so we carefully cordoned it off so that it can only be reached by the hosts that actually need to contact it. While we were at it, we made sure to revoke the number four from all of our servers.

As the day went on, more of the group came online. Offered encouragement. Lent a hand. Like I said, this is a social process.

A lot of stuff was back up within 30-60 minutes. Some took a few hours.

While waiting for packages to build and stuff to restart, I Posted Through It on social media. It lightened my mood.

We let CloudLab and POWDER users know that everything was clear.

One of the problems we had is that we were on a version of PHP that reached end-of-life (EOL) a few months ago. So it didn't get patched, and we had to do a version update in addition to applying the patch. We knew this in advance, one of the reasons why it's good to give folks some warning.

Our code is, let us say, "vintage." It was originally written for PHP 3 (the world is on 8 now). Over the years, we've modernized it and ported it forward, and we were already on an earlier release in the 8.x series, so it wasn't a huge jump. But the code has some history, is what I'm saying.

Knowing this, I had done a quick check the night before, and it seemed that we were going to get a bunch of warnings about deprecated syntax, but nothing would break.

Luckily, that turned out to be true.

But I spent most of the rest of the day tracking down and fixing deprecation warnings. And installing the fixed code very carefully, hoping no users would notice - seems that they didn't.

Oh but wait: we are NOT done.

In the late morning, GitLab released a patch for an RCE of their own. Completely unrelated (it's not written in PHP). This one didn't have a severity of 9.2 - it had a severity of 10.

We run a sizeable GitLab server, it has 700 users and 2.5k repositories - this is the main thing we have to defend against bots, so we know that they know it's there. (The bots mostly spend their time downloading every diff between every revision of several copies of ... the Linux kernel. Super rare stuff, guys. Never before has so much dumb shit been done in service of "smart" technologies.)

So, we had it down within minutes. Updated and back up 15 minutes later.

Somewhere in this mess, I don't know exactly when, NodeJS releases patches for their own high-severity RCE. All supported versions (probably old unsupported ones too.) Again, completely unrelated.

Luckily, we only run one NodeJS thing, and could just firewall it off from everything but the one machine that needs to use it.

(I do, however, run some stuff in my homelab that uses NodeJS. I just pulled it all down, only brought back up the bits that had updated their container images.)

I had a chat with a potential collaborator, went to class, talked with a friend, went home, and fell asleep.

I didn't manage to get my Real Job, giving my feedback on one of my student's papers, done. I'll work on that when I'm done getting this out of my system.

So what do I take away from this?

First, we have had days like this before. Not exactly like this, but kind of like this. The first one I personally remember is when the Code Red worm hit in 2001. We didn't run any vulnerable servers, but the huge increase in traffic all over the world just wrecked everything all day. In 2003 there was a remote root exploit in OpenSSH, get root on any machine in the world without a password. In 2008, it was discovered that most Debian machines had the same SSH keypair for the root user, due to a fixed seed when generating them "randomly." Again, free root on a ton of machines. In 2017, Spectre came out kicked off an era of side channel attacks. There are lots of others. So, the world being on fire, all at once, is a thing that has happened before, and a thing that will definitely happen again.

Does AI change this? Yes and no. It is being used to find new vulnerabilities. And, as we saw above, the act of fixing a vulnerability, itself, makes the exploit public knowledge too. But you have to do it: you have no idea who else might have already known about it, and even have been exploiting it, quietly. We've had other rounds of tools used to do mass scans of software looking for bug and vulnerabilities: program analysis, fuzzing, etc. Each have kicked off waves of bug reports, then eventually become part of the standard bug-hunting toolkit. That said, LLMs seem to be causing a much bigger wave than earlier tools.

But.

I believe that only part of LLMs' effectiveness in finding and exploiting vulnerabilities comes from the technology itself. Yes, they are surprisingly good on both fronts. They do seem to be getting better at it. But you cannot ignore the economic factor: there have been orders of magnitude more investment, over a much shorter time, than any other bug-finding tool. How much of their effectiveness comes from the fact that, right now, a whole lot of people - especially the frontier AI labs - are willing and able to spend tens of billions of dollars training new models and millions of dollars (probably more? a lot more? I haven't seen any estimates so I'll stick with a low number) pointing them at open-source software?

I really don't know the answer to this question, and I don't get the sense that anyone else really does, either - we're just going to have to wait and watch and learn. While the system is stuffed full of investors' cash, I expect we're going to get a lot more AI-assisted vulnerability and exploit finds. Does the system transition to being stuffed full of consumers' cash? Does it level off or contract? People have clearly placed their bets, but we have to wait to find out.

And this brings me to another point, which, I must say, is going a bit out on a limb, but which I think is very important - I'd really like to hear people's feedback on this.

Right now, finding and exploiting security vulnerabilities in software is good PR for the AI labs.

Right now, while they are spending money that was invested, not earned, and are desperately pushing for more investment, it makes a lot of sense for them to put special attention into training models for this purpose, and to spend inference money, themselves and with security research partners, on the task.

Look: today, hundreds of thousands? millions? more? of administrators of all kinds spent their day on defense. And they know it was because of AI. Hundreds of millions of other people have heard us complaining, or being awed, or being smug that we don't use NodeJS, or whatever.

It's a very, very good demonstration of the very real effectiveness of the technology.

At some point, the labs need to actually make money, though.

Is this going to be a good way for them to do it, long-term?

Is overall model improvement just going to be so effective that it doesn't matter if they spend money on this specific use case, it'll just keep being great?

It's worth noting that every other bug finding technology has generally followed a trajectory where it's very effective at first, opening up new classes of vulnerabilities. Once the kinds of bugs it's good at finding have mostly been found, it's a long, slow path of incremental improvement. Still useful, becomes part of the permanent toolkit, but the shine wears off. If the PR value of finding bugs with LLMs fades, and it becomes just another use case, will it start looking like the other tools in our toolbox? Or is this fundamentally different? I don't trust anyone who thinks they have a definitive answer to this question. My hunch is that we have a bunch of turbulence, then reach a new equilibrium, but I could be wrong.

This phrase seems to have somewhat gone out of style, but when I was first administering networks, we called people capable of using offensive tools that others built - but unable to build their own - "script kiddies." The idea is that the main resources that a script kiddie has are time and a certain moral vacuity. Not, really, much in the way of expertise.

I think that LLMs already are - and will continue to - enhance security experts' abilities to find exploits. However, these experts tend to be fairly rare, and many of them have specific goals. Think, for example, state intelligence services. Militaries. Corporate espionage. That kind of thing. The limiting factors here are expertise, and the "market" for these services. Seems like at least the latter will probably grow. But my money is not on it growing in a way that fundamentally re-shapes our network environment.

Where LLMs for offense really shine is with the script kiddies, who neither have, not care about, gaining expertise. They want to steal. To extort. They are bored. They have grudges. They want to watch the world burn. Time used to be an important resource here, since it took time to find and use the tools that others made, and to turn that into whatever you were seeking. I suspect time becomes less important with LLMs, since you can (to some extent now, and likely more in the future) turn them loose against targets. This leaves a lack of morality as the main "feature" needed to become a script kiddie.

To me, then, the question as to whether LLMs wreak havoc on the internet as offensive weapons comes down to whether our society maintains, or degrades (or even improves) its moral character. It's not, directly, the technology. It's the people, and whether we let the technology change us and morally de-skill ourselves.

There is another kind of defense that LLMs can potentially be useful for that I skipped over before. Largely, because my thinking on it is less developed at this point. Yes, LLMs can help us find - and patch - security vulnerabilities. But what about the "LLMs as cops" or "good guy with an LLM" argument, where they are used to provide active defense against the threats from other LLM-discovered vulnerabilities or agent-driven attacks?

This sounds like a dystopia to me. What's the fundamental difference between a cop-agent taking action against a "bad guy" agent, and a cop-agent taking (digital! I'm not talking about RoboCop here - I hope!) action against a person on the network? How can it even know? In the physical world, we have processes for allocating resources to, and limiting, policing. I'm not going to call these processes "good." In the US, for instance, I think we have gone much too far in militarizing our police (along with many other ways we do policing wrong). But now imagine the agentic version of this. SWAT teams, at scale? Whoever has the biggest datacenter gets to enforce their own version of "safety"? It's hard to see how this ends up anywhere other than extreme digital authoritarianism.

This does not rule out LLMs for defense, but positions them as tools for defenders, not agents with a cop badge and a gun.

I want to finish with some thoughts about software quality.

The vulnerabilities above were, as far as I know, in human-written code. Some or all were found with the aid of LLMs.

If, as some believe, and, yeah, I think there is mounting evidence for this, a lot of future systems are built mostly by LLMs, what do vulnerabilities look like in that world? Is this software more secure? Or less?

I see five reasons to despair and two causes for hope.

The first reason to despair is that the large LLM-built systems that I've spent any time looking at, and which I've seen others' analysis of, are security nightmares. They are full of duplicated, but inconsistent, security checks. They have what you might call a security architecture, but then data flows through them in ways that means that just about anything can happen anyway. They have ad-hoc conditions and arbitrary limits. They are awful.

But wait: won't better models write more secure code? Well, here are my next two reasons to despair.

In order to produce secure code, you have to care about producing secure code, and you have to be able to tell how secure the code is. Yeah, people screw this up. In fact, they usually screw it up. But: I think that both of these factors are often (not always!) lacking in LLM-generated projects. A major use of LLMs is for people to do things that they are not, themselves, very good at. Makes sense! But if you (not you personally, of course, some other "you") don't have expertise in understanding the security of web applications, well, you're not really going to be able to tell whether the thing the LLM generates for you is very secure, are you? Additionally, our current moment has shifted the balance towards caring about "just building cool new stuff", and away from caring about its security. Again: not new, that's a pretty normal instinct. But, as the value of the code itself gets driven to zero in favor of the value of the product, the value of security is getting driven in the same direction.

Okay, maybe you don't care about the security of the code, and wouldn't be able to tell how secure it was anyway, but surely the LLM "cares" and can tell, right? I don't think so. A major strategy used to improve the quality of code that comes from LLMs is extensive testing. People hate writing tests, but matrix math doesn't hate anything, so you can get it to churn out huge amounts of test cases. Maybe some of them are even good! Then you can follow an iterative improvement process, where the LLM produces code, it gets tested against various things, if it fails, it gets improved until it passes. But security is not so testable. Knowing that something is secure means proving that some undesirable thing cannot happen. You can't "prove" something like that through testing, without testing every possible execution (as well as being able to precisely define and identify "what shouldn't happen"!) (I know! I know! Formal methods! I'll get to that in a minute!) Plus, most security vulnerabilities these days involve things like complex orderings in concurrent code, resource exhaustion of various types, complex chains that corrupt values in one place and make use of that corruption much later in other code paths, re-use of the application's own code as "gadgets" to execute attack logic - the kind of stuff that is just a nightmare to test. We're not talking your basic unit test here where you check to see if the code URL-encode values in a string. Tools like fuzz testing, or LLM-based vulnerability search are great auditing tools. But they are slow, expensive, and not really part of iterative development loops. So, no. I don't think testing by agentic loops or semi-antagonistic LLM "teams" is going to make up for lack of careful human oversight.

I don't dispute that LLM-written code is getting, and will continue to get, better. I don't dispute that a person who cares about producing secure code and is capable of understanding its security properties can use LLMs to generate code that is as, or maybe even more, secure than human-written code. But I do not believe that the major things driving improvements in LLM code are going to drive us to more secure code.

Fourth, products that, themselves, contain an LLM that is hooked up to "tools" that can interact with something other than just text (coding agents, personal assistants, etc.) have an extremely strange security posture. Most software products are built to protect themselves, and hopefully you, from "external" threats: network attacks, malicious files, that kind of things. But products with an LLM in them have to be built to protect you and the external world from the product. If the product has access to your medial history, and email, then you have to consider the fact that the product might email your medical history to Mark Zuckerberg. Bizarre. Based on what I've seen of coding harnesses and personal agents, we are exceptionally bad, so far, at adapting to this threat model.

Fifth, LLMs generate a lot of code. A LOT of code. They are not limited by exhaustion, boredom, or frustration. They are, to a degree, limited by time, but this is why people use multiple agents to ramp up the code output. The more code, the more it tends to contain vulnerabilities. We've known this forever. Yeah, of course, you can take precautions against it. But that doesn't change the basic math. Vulnerabilities in human code are limited, in part, by the fact that humans only write so much code. If you believe in a world (which, to an extent, we are already in) where LLMs churn out far more code than humans ever did, then you've removed one of the caps we had on how many vulnerabilities were getting created.

Two reasons to be optimistic, though:

First, there's formal methods. These are ways that you do get to prove properties - including security properties - about code. Far more powerful than testing it to death. Think mathematical proof of additive identity, instead of exhaustively testing that every integer, when you add zero to it, is still the same. This has historically been a hell of a lot of work, but some parts of the proofs have gotten more mechanized, and people are trying to get LLMs to do some of the other tedious work that has previously made this infeasible for large complex systems. This may work out well. I'll rain on this parade a little though, because you still have the problem that you need to be sure that the formal, math-like properties you're proving about the software (a) correspond to the informal things you care about ("does not send my medical history to Mark Zuckerberg"), and (b) are comprehensive. I don't see these being particularly compatible with most vibe coding.

Finally, there's a significant chance that the current euphoria (in some quarters, anyway) for churning out huge quantities of code, just because we can, might cool off. This would reduce the "too much code" problem, and may reduce the amount of code generated by people who do not and cannot care about its security. There are, I would propose, limits to how much software we can realistically use - right now, there is plenty of novelty to be grabbed, but eventually we'll reach a sort of saturation again where, yeah, we do still produce a lot of new software, but it's limited by how much software anyone actually wants. Perhaps, once Candy Crush does start emailing your medical history to Mark Zuckerberg because an ad asked nicely, we might start caring a bit about security, too.

(Okay, I realize, that's pretty much what was going on BEFORE LLMs, and we didn't care then, either, sorry for failing to end on an optimistic note.)