You may never hear me say this again because I am a gruff cynic who is not sure that hanging out here is good for his psyche, but thank goodness for HN.
While I have been chasing around today, trying to resolve this on different servers, it turns out a three line nginx mitigation (blocking multipart uploads) I wrote in the wee small hours GMT has been quietly protecting a vulnerable machine I could not shut down (from attacks powered by every single mainstream agentic AI tool — Anthropic, OpenAI, Qwen cloud, Baidu, seemingly also AWS, Meta)
Including an hour before the PHP patch actually shipped.
So without the HN discussion I wouldn't have found enough people pointing at possible mitigations and I would have lost a server to this, and I thank everyone who pointed out that second mitigation in particular.
> from attacks powered by every single mainstream agentic AI
Is it so bad? Have you measured the number of attacks?
> So without the HN discussion I wouldn't have found enough people pointing at possible mitigations and I would have lost a server to this
I run a tiny server. I like this server a lot. I only patched an hour after the Debian package was ready and didn't think about it again. Do you mean to imply there's a real chance my little server has been compromised?
[Edit:] My server is quite public too. It hosts four SLDs and many subdomains and receives both human and bot traffic. Only one or two subdomains use PHP, but a reasonable attacker surely tries all of them.
> Is it so bad? Have you measured the number of attacks?
It is not a torrent, no. Dozens up to hundreds, rather than hundreds to thousands.
But they would very likely have succeeded, right? It's a fairly turnkey vulnerability.
> I run a tiny server. I like this server a lot. I only patched an hour after the Debian package was ready and didn't think about it again. Do you mean to imply there's a real chance my little server has been compromised?
Can't say one way or another. Have a look at your log file for POST requests in quick succession for things like
> [Edit:] My server is quite public too. It hosts four SLDs and many subdomains and receives both human and bot traffic. Only one or two subdomains use PHP, but a reasonable attacker surely tries all of them.
All my "obviously PHP app" domains got visited. None of my bespoke app domains (which are not obviously PHP). Though over time, the bespoke domains do get a whole bunch of boring probes for WP vulnerabilities etc., like all websites get.
Depends how your logging works. If you ingest logs to a remote server it's quite hard for an attacker to clean those up.
Also I'd guess that most of the PHP exploits tried today weren't sophisticated but rather brute force and then either cryptomining or botnetting or something like this.
I think it doesn't, but I have to say, I have never seen an automated attack on a PHP application that has bothered to hide its traces.
If your attacker is someone known to you, some kind of persistent threat, then maybe they would? But most of this stuff is typically kind of worm-like; attack one box, use it to attack another, move on.
With this particular server I am in two minds about whether it is likely I have any persistent type hackers, but I must say what it looks like is automated attacks. AI script kiddies, almost. These same boxes attract all the typical PHP app probes (WP2shell etc.)
And FWIW what it actually mostly looked like was hits from some sort of cloud-based AI search API; they suddenly stopped at the same time as if someone else had pulled a plug. So maybe someone running an agentic search API decided not to be part of the probes.
Agreed. It made my ancient old man face tear pores by smiling when I saw people organizing in the comments to throw sandbags against the wall while waiting to be rescued.
I'd like to express my dissatisfaction that the "big guys" are basically getting free security over getting earlier disclosure.
Just saying "hey blocking multipart uploads can mitigate most of attacks early would save people so much trouble. Of course not every CVE will have workaround but many do and whole secrecy about releasing it is only to benefit of few big tech companies and nobody else
Is it just me, or is it the truly brave that decide to deploy and protect PHP instances? Especially if WordPress or Drupal are involve?
I finished moving the remainder of my stuff off of PHP and WordPress last year and haven't missed it at all. Backwards compatibility was typically pretty bad.
It seemed like maintaining existing working PHP code was always a cycle of: upgrade to newer PHP, that version has breaking changes, fix that code or update to a new WordPress (plus all of the plugins that would inevitably need updating too).
I don't really want to be in the WordPress world anymore (I still maintain a couple alas).
But this isn't about PHP code, really — this is about C code (with less-than-ideal exception handler patterns). This problem is more Heartbleed than WP vulnerability.
I quite like modern PHP, I don't really think it's necessarily any less secure (exception handler weirdness notwithstanding), and more to the point, I don't think not using it is any kind of automatic improvement in web app security. It's not like server-side JavaScript is intrinsically better.
I am certainly doing some thinking today; I switched off a couple of web servers so I didn't have to think about them while I was working on other stuff, and actually I think they might not get switched back on. But it's not so much that they are running PHP as that they were kind of neglected and it's always worth reconsidering what you need to keep around.
I think I have reached a point where low value stuff is not worth subsidising just for fun.
In nearly every org I've been in or contracted to, the public landing page of the company gets a special place outside of any ops of general technical team, and is instead the work of a marketing group or a "creative" outsourcer who will default to Wordpress on PHP.
Unless your position that any statement whatsoever made in the second person that isn't a compliment is then an attack?
San Francisco is a magical place where "you are doing X wrong" without slathering on six paragraphs of fake kindness constitutes some kind of personal attack
In no sane culture is an engineering assessment made in the second person a personal attack. You'd call "you are holding the screwdriver wrong" a personal attack too.
He is, in fact, not mitigating the attack, and he is, in fact, by calling his non-workaround a mitigation, encouraging people to do ineffective things instead of patching. These facts remain true in the real world regardless of what you call a personal attack.
And yes, San Francisco has become top of the world for indirect fake kindness that transforms plain factual statements about the world into social offenses.
"The sun is bright. Don't look at it " -- RUDE!
"You might want to consider that the sun might be considered bright to some people and that you might be more comfortable looking at this other cool thing instead. But no pressure" -- gruff, barely tolerable
Location isn't the issue. HN users all over the world have no trouble making their substantive points thoughtfully, and SF has its fair share of commenters breaking the rules.
2) actually, this particular mitigation is very clearly stopping all the unusual patterns of POST request attempts I can see (it's not usually a busy server)
3) I know it is not the only mitigation that is necessary
4) … so it isn't the only mitigation I have applied
5) I am not working under a false sense of security, and I know it is temporary
6) I am not an idiot, and I am working on patching this machine, but this particular server has a unique and complicated configuration and needs a bespoke build
7) I have been prioritising much more serious risks for other people, all day, for no pay
Sorry dang but I had a wider point to make there and I'm not really going to apologise this time. The rudeness is only part of what needed to be responded to.
Also I have survived on too much coffee and probably too many chocolate biscuits. So actually, sorry, on reflection :-) What a day.
We should really move to a different security model on the internet.
For example, when I buy something from a webshop, why does that webshop need my personal information? Why can't I give it a token, which it can then use to send my package, because only the postal system knows the address coupled to the token, etc.
And we should also ban user tracking for advertising purposes, obviously.
We are working on this better model in the Ethereum ecosystem via technologies like Zero Knowledge Proofs and private attestations. The Ethereum Attestation Service has been using the example of proving to the bartender that you are over 21 without disclosing your birth date. https://docs.attest.org/docs/core--concepts/privacy
This is all new and rapidly evolving technology but giving apps / agents / others just the information they need to know while having a credibly neutral, private, and trustless database mediating is a true cypherpunk vision of how the internet should work and it's being built by people with the right ethos and principles to be as reliable, well engineered, and long lasting as the internet itself.
We'll probably never convince end-users to "vote with their feet" and use Privacy-focused products. But we can absolutely, through legislation or just precedent, raise the Risk profile until it's better to leave the ad revenue on the table and run a private service.
Shift liability to those who store and sell data. They get full economic responsibility for fallout of breaches of data that is retained past the time necessary to complete the transactions. If they sell the data then they hold liability for all breaches of downstream parties as well. Require them to carry insurance against breaches.
It would completely destroy the data brokers and targeted ad industry but the world would be a better place as a result. Also not going to happen but I can dream.
We've already moved away from sites storing credit card information, because the risk of storing it yourself is simply too high. We need to make the stakes higher for other personal information.
> As a result of the leak and pre-announcement around 20:00 UTC, Calif, one of the security firms that had reported the issues mistakenly published a video.
This is the most surprising part to me. It sounds like Calif saw the leaks, interpreted that as embargo being broken, and then raced to publish their [anime dancing] marketing video? I'd have expected much better.
This reads like some guy tripped and fell on top of his mouse and in so doing, accidentally turned on screen recording, performed the exploit a few times, got the anime girl asset and soundtrack, dropped all of them into Premiere, added the weird effects, rendered the video, and published the video to Twitter. "Ow. What did I fall on? Oh noooo, I broke the embargo!"
Your comment is funny. But obviously, "mistakenly" refers to the mistake of believing the embargo being broken, and then, very intentionally and not mistakenly, publishing a stupid video.
The video was posted hours early, quickly deleted (but not before being reshared), and then posted again at the proper time. The link in your comment is not the first release of the video
Given how security moguls are handling that, we should go back to "just release it, consequences be damned". Coz that gets normal people better chance of actually fixing it
That sounds oddly threatening, especially given that chainguard is in the right here. Embargoes are stupid in 2026 and make no sense. The rumor of this issue was enough for it to be independently discovered by several white hats and almost certainly many threat actors.
> Embargoes are stupid in 2026 and make no sense. The rumor of this issue was enough for it to be independently discovered by several white hats and almost certainly many threat actors.
The point of the embargo was to prevent all rumors? But it did not hold.
Releasing a statement that a critical security bug exists with no more information, does the opposite of preventing rumors. I think one of the most surefire ways of generating rumors is to say "something big will happen tomorrow, please don't speculate about it".
Anyway, GPs point is that even if nobody leaked anything, and the only thing publicly known was the existence of the bugs, people would anyway have been able to find them independently. So what is the purpose of waiting 24h before releasing the patches?
What are these "rules" you speak of? There's usually nothing binding in these agreements. It's just groups either trying to do the right thing by end users or trying not to look like assholes, get a better payout, or some combination thereof.
Finding an exploit and publishing is is not against the law. In some cases, publishing immediately is the responsible thing to do. For example, when it's actively being exploited in the wild, and waiting just puts people at risk that don't have any recourse to protect themselves, since they don't know they're at risk. There can be nuance to that as well, but in general, as soon as it's being exploited, anything that makes it harder for end users to a) know there is an exploit, or b) tell if they're susceptible to it (e.g. through a POC) is just taking away people's ability to take the action they deem appropriate.
I think the PHP security team and the whole PHP team could have been more communicative and keep releasing information, adapting to the news, suspected exploits and mitigations.
If a video is released showcasing the exploit and following mitigation recommendation, PHP should clarify what it is and if the mitigations are recommended or not. From the pre-announcement:
> We reserve the right to release earlier should details of either vulnerability become public, or should evidence of active exploitation appear in the wild.
If no info is released by them, no other party should have released anything. If info is released or leaked from partners or involved parties, you should notify and explain what's going on.
But no, complete silence from PHP news or any PHP community member.
I get that it was a complicated situation, and I think PHP is the "less wrong" party involved, but I think this constructive and external criticism could help in the future. It has been an interesting day (GMT+2 here) and a rollercoaster of monitoring and mitigating without any reliable source.
> But no, complete silence from PHP news or any PHP community member.
Not entirely correct as they clearly announced that they were going to release a fix at a specific date and time; the mitigation doing the rounds was not adequate and if the PHP people republished it, it would lull people into a false sense of security. Overcommunication is also bad, especially if the official platform or the individual members are being watched by tens of thousands of people hungry for news or updates.
They knew (since august, arguably since 2024), they had a fix ready, everything was in place. Except for the untimely / accidental early leaks.
The complete silence I refer is after the incidents and leaks. They stated in their preannouncement "We reserve the right to release earlier should details of either vulnerability become public". That sounded to me that if something is leaked, they'll release earlier or clarify. Neither was done and only after the release, clarified things.
But yes, the preannounce was the right move and the leaks were unfortunate, complicating things. But if cloudflare, bunny and other actors start mitigating in their WAF, I think something has gone wrong and, either you act by releasing earlier or calm the waters saying "the exploits are wrong" as per stated in their post, they thought it was the case.
No meaningful info was released. A lot of incorrect info was speculated on.
I have a lot of sympathy for php team - it would be a lot of work to respond to every conspiracy theory on twitter, while trying to get everything ready, in the middle of the night no less.
I have a lot of sympathy for the PHP team as well, I hope it doesn't sound like it.
A lot of speculated info was right: the mitigations were mostly right, the video showing exploitation was real and the release of a fixed image was real.
I think if you do a pre-announce, and you say that you could release it earlier or release new information and you don't, and there are reasons to inform your users, then even if it's the middle of the night for you, you should.
As they stated, they are a small team and they have changed their structure, and they are learning and improving. I think if something like this happens, info is vital for us, the administrators of PHP servers
> Are there structurally much safer ecosystems out there?
Several attributes make a safer ecosystem. An important one is diversity - technical diversity.
For an interpreted language, the runtime is important. Python has several runtimes, as does Ruby. Javascript also has several, from the few browsers, to node, deno and bun. PHP has for all practical purposes, only one. So if there's a security incident in PHP, it affects almost all users. But when e.g. JRuby has a vulnerability, it will affect only a fraction of the installed ruby applications.
For compiled languages, the tooling, libraries and runtimes is where diversity is important. With e.g. rust, the vast majority of CVEs come from libraries (crates). But since there often are several crates for a single concept, diversity helps mitigate here too.
But again with all the ones you mentioned, they have a "core" runtime that the vast majority of production apps use, so if NodeJS has a vulnerability it would be just as big.
The interesting one will be whether "boring" ports - like recently Typescript's TSC from Typescript to Go and now to Rust - will be safer for being the same code and logic, converted to another "safer" language (by transpilers or LLMs).
if there's a use case for re/decompiling, it's this one. with at least one strong alternative implementation, there would be less incentive on the attacking side to go all out on either PHP implementation.
Garbage collection is also an option. Lots of backends on the JVM!
(Of course there's Log4shell, but that could've happened to a Rust-based stack too, and might be more likely to, considering the proliferation of deeply-nested dependency trees full of tiny libraries in that ecosystem.)
While the main Java and .NET virtual machine implementations are much more secure than PHP, they were still written at least partially in unsafe C++. I won't be surprised if frontier LLMs eventually find more vulnerabilities in those platforms.
> On October 21st at 07:41 UTC, more than six hours before the coordinated release, Chainguard unilaterally broke the embargo and published built images containing the fix.
This is incredibly damming of Chainguard. I struggle to see justification for this, even with the chaos approaching the coordinated release.
Let's hope so. Shall we create a new norm? A 365 day timeout, for companies like Chainguard that deliberately break embargoes.
Very irresponsible behaviour in my opinion; and if it's a decision made unilaterally by their CEO, probably a breach of their SOC 2 certification. What's next? "$X isn't patching this CVE fast enough, here's a binary blob of our fix, don't worry, we're making this free!"
Nah, Chainguard should be praised for this. It was a different company that leaked the POC early, and then PHP, Wiz, OpenAI, and Calif sat on their hands doing nothing while the vuln was being actively exploited in the wild. Chainguard did the right thing by getting the fix out as quick as possible to stop the bleeding (or gushing flow of blood, really) rather than sitting on it so they could make a coordinated marketing post, like the others did.
That's a reasonably take. Embargos only work until the information's leaked, and once that's happened, however it happened, the bad guys have the exploit information and the clock starts ticking.
PHP’s view is that Chaingiard started the clock because the video was insufficiently detailed. Not sure if that is the case, but just sharing I had a different reading of the statement.
They released some binaries, built for their own docker image based on their own Linux distro. Other people couldn't use Chainguard's "fix" to patch their own systems. The only people who seem to have benefited from the stunt are the hackers who used AI to reverse-engineer the binaries and write exploits.
That was a giant fuck-you to the rest of the PHP ecosystem, nothing more.
Chainguard effectively held everyone hostage, and forced everyone hosting container images and wanting to be online, to trust their builds and then saying they did it for "free" to get free advertisement. Incredibly irresponsible behavior. They used this as a big fat advertisement board.
I dunno, the post is awfully critical of Chainguard for leaking a fixed binary 6 hours early, but it was the research firm "partner" that "mistakenly" posted the dancing anime POC video over 24 hours early that created the mass panic (not to mention spawning millions of agents crawling the php codebase and trying to brute force a POC and probably using more energy than a lot of countries in the process).
wow, okay. yes, that really does look like a "my first PoC" video made by an early teenager. hard to believe that this was produced by a company that wants to be taken seriously.
There do seem to be the perpetual problem with the field of computer security that only insufferable and immature kids matter, due to discrete math adjacent nature of computer science, and yet they also legitimately can't be trusted, or does it not... unless LLMs do change everything.
I wish people would name this idiotic company, I can't see Twitter here and all people are talking about is a childish video they made. I don't ever want to do business with said company, ever.
They're apparently not trustworthy, and behave as children.
>As a result of the leak and pre-announcement around 20:00 UTC, Calif, one of the security firms that had reported the issues mistakenly published a video. It showed their proof of concept working against a number of targets and revealed a few additional, though limited, details about the vulnerabilities.
Calif more than Chainguard, but still Chainguard too.
Calif was obviously twitching to claim credit for their findings, and what I find troubling is this eagerness was more important to them than giving users a chance to update their systems.
Even if they ended up waiting only a day post release I would still say it seems a bit too soon. Maybe a week after. Not everyone’s system have aggressive patching schedules and PHP is soo heavily used.
This is probably the future, tho. Being first to disclose a vuln is a Big Deal in some quarters. And now, the velocity of AI driven pentest swarms mean every researcher who finds something worthwhile has to think "how close is the next droog's swarm to finding what I found" and the temptation to jump the gun only gets stronger.
I don't know, it's showing most sites being pwnd in less than 20s with some taking just under 30s. Of course, that's on homeboy's local Mac. Not sure how long it takes to gain control with PHP2Shell to run the exploit though. Just started looking at that part myself
Everything i saw on twitter prior to the release turned out to be incorrect. At most people figured out it had something to do with session upload handler and UAF, but nobody knew what.
I want to go back to talks by bored defcon guys or gals calmly stating how they obtained private keys of a globally trusted CA with just a bit of math. Which, mind you, ate the other part of the teams time.
Now I know how much I hustled today likely due to that atrocity of a video. I think I will take a vacation day and sit in a forest near a river on friday.
dopamine junkies crave stimuli, and realistic details of humans are just noise for the data. anime humans are simplified to extremis which alone makes some people think it's about kids
I kinda get Chainguard's position here. They were part of the embargo, saw exploits hitting in the wild already and decided to release rather than hold things back.
Embargos make sense if no one is exploiting it yet, but if its out in the wild, the best path forward is releaseing as soon as possible.
The person who wrote the widely-referred-to tweet last night (claiming to "have recreated & lab/field tested CVE-2026-109618 / CVE-2026-109619") followed up a couple hours later and said that their POC had not been proven to work "in the wild"
"Cloudflare successfully deployed multiple mitigations for all its customers while the vulnerability was leaked before the official release date and we observed attack traffic targeting our customers."
If that's not an in-the-wild exploit, it literally couldn't have been further away than one or two more iterations.
It sounds like there was no "co-ordination" behind any of this co-ordinated disclosure. You'd have expected there to be an IRC channel, a mailing list, a Slack chat, or something for something of this scale.
Was there even an incident co-ordinator here? Was any of the parties talking to each other?
Many people actively following would have 403'ed multiparts so people seeing those blocked might have assumed it started being exploited early instead of probes / background noise.
There were signs that others broke the embargo too. Public (this one video everyone is talking about), but maybe also non-public. There were some rumors going around which part of PHP was affected. That was enough for some people to figure out the vulnerability.
But it's also possible, that some attackers got non-public inside information.
If the embargo is already broken in a meaningful way, it suddenly becomes much less clear if it is still in place after all. Emphasis on >less clear<.
if their comment in true (linked in sibling comments), then it's really not damning. typically an embargo is ended when someone leaks early or it's being exploited in the wild.
so, if they saw evidence of exploits in the wild, it is not unusual at all to release the fix.
If they're gonna unilaterally release an exploit patch 6 hours before an agreed upon embargo for a CVSS 9.2 that affects like ~70% of the internet then they better have reams of evidence showing this was being exploited in a widespread manner and not just "we saw a dancing anime girl edit of a theoretical exploit on twitter, so we had to release it to the world"
And evidence they at least gave a heads up to everyone in on the embargo, eg "I have to release you all have 1 hour to get on board".
There is a MASSIVE difference between a small number of people exploiting in the wild, and a few hours after a release a bazillion script kiddies doing so. The move to release and break an embargo was childish, absurd, self-serving, and lacking in all common sense. Not to mention, breaking trust.
I agree with those in this thread who say, never going to do business with them. Ever.
As a customer, I'd immediately assume no one in the security community would trust them again, and seek someone else to give my money to. Why pay a) an untrustworthy company who b) now is not trusted by anyone and c) won't ever be able to early release again anyhow? And if anything, are left out of important conversations.
Way to burn bridges. You only get to be an asshat once.
It's the age of LLMs. Exploits go from theoretical to real fairly quickly. PHP's open source, it's not impossible to conceive of someone seeing that video, getting the general idea of what to do, and working the rest out with a decent model.
Because if there are already publicly available exploits seen, it makes no more sense to keep holding patches back just so that OpenAI etc. can get a nice press release
2. Calif sees leak, goes and releases dancing anime girl video before pulling it, goes viral
3. Chainguard sees dancing anime video, doesn't recognize Calif, and thinks the exploit is in the wild
[Edit 3a]: Or alternatively, Calif's leaked video causes the PoC to be found by others by providing enough of a lead
4. Chainguard releases updated binaries thinking it doesn't matter, causing the real exploit to then be found early
5. Which then causes PHP Foundation to basically have a meltdown
Personally I blame Calif for the unprofessional video which absolutely completely looks like some script kiddie and for being too detailed [edit 2: assuming Calif produced it and the anime wasn't a prank edit]. In this case it wasn't just unprofessional fun but may have literally messed up the responsible disclosure.
> 3. Chainguard sees dancing anime video, doesn't recognize Calif, and thinks the exploit is in the wild
I'm not sure about this part. I saw https://jackalsyn.com/posts/php-up-2026.html going around a bit over 12 hours ago, before the chainguard images went out, and plenty of other chatter of exploitation happening in the wild.
Not really sure if 3 is correct. Might have been Calif leaks the video -> it goes viral -> people make PoCs based on the video -> Chainguard releases the binary because there seem to be credible PoCs out there (that are not just the Shigure Ui video).
I was asking myself today; how to ensure that you don't miss a cve? I mean you'll sooner or later stumble across it if you're monitoring your communities but I thought that maybe there exists some kind of central solution, like a service were you could subscribe for your tech stack and get notified?
There are vendors who supply this service as a product -- Wiz, who were one of the groups to independently discover the vulnerability, is one such vendor, there are plenty of others.
I'm not overly keen on the tooling: as far as I can tell the whole industry is prone to flagging false positives as "vulnerabilities" rather than "potential vulnerabilities" and it's a bit too easy to turn those reports into a tick-box "you must update now because the tool says so". But they also do flag true vulnerabilities. And they pay for teams of security researchers to find and fix issues, not least because that's good marketing.
Big companies usually use some sort of vulnerability response management software which is connected to your CMDB, ideally containing SBOMs of everything you use, and subscribed to various providers of CVEs.
Those tools then map the CVEs to your SBOMs and you'll get alerts, security incidents and so on.
Several tech stacks have this built in or available.
For my rust applications, I use "cargo audit" which checks all dependencies against the CVEs. For my typescript and javascript applications, deno audit and/or yarn npm audit.
For the rest of the stack `debsecan` does exactly what you want on Ubuntu: report CVEs for installed software.
I was thinking about this as well today. And while I'm sure there are many cybersec companies that send out alerts, I trust this forum the most.
Maybe it would be worth it to have an agent that knows your stack, reads the top threads on hn all day long and messages you when it thinks there is something serious going on that could impact you.
I don't think there's any robust method. I lurk HN, subscribe to security lists, run our code through scanning tools regularly. All options suck, all options have gaps, and all options mean you have to stay on the hamster wheel.
Lots of licensed solutions have this (typically SAST or DAST tooling like Snyk, etc...). Given your build manifest/SBOM they cross-reference CVEs for vulnerabilities during/after build time.
I may be misunderstanding this but think the hard part about building a threat intel feed like this is just trusting that the underlying integration would surface something you care about... especially if you want to DIY it. You can scrape HN, GitHub advisories, NIST, whatever you want... but you may give yourself a false sense of security after seeing the "all clear" checkbox each morning
I hope that someone has some sort of a reasonable approach here / I would love to help work on something open source if not
Any tool to scan dependencies takes various inputs like repos, container stores or already built SBOM and gives you warnings. It doesn't much matter if you have actual code and repos as long as you know how to point it at some. The fremium ones like snyk would love to send you an email report on growing CVE counts on your last run but only until hell freezes over..
> In 2024, PHP underwent its first external security audit in years. The PHP Foundation secured funding from the Sovereign Tech Agency to organize this audit ...
Well that's your problem right there! There are trillion dollar companies profiting off the back of open source projects without supporting the projects adequately. Why should they be scraping for funding for basic table stakes like audits?
I can empathize with them, this is actually quite difficult when you're an open source project.
You have your source code public, so you can't commit/push because then you've leaked info.
Your build pipeline is also likely public, so you can't just build from another source code... source.
You can create a private repo, but now you also need to build a private build system.
You don't get the same generous build resources for private repos, so you either:
a) simplify the build to just the essentials
b) pay
Unless you have significant funding, you'll probably go for a).
Now you have to maintain two systems, also with some differences between them.
Plus you need a way to get the build artifacts from the private system to the public one, typically at a fixed point in time.
Any failure/delay (e.g. less powerful build on private repo, errors because you exercise it rarely) causes people to get upset that you didn't respect the announced time.
Maybe you're also using things like image attestation, another can of worms.
Those are all challenges that surely need to be overcome!
But they had _two months_ to figure out how to overcome them. For them to not have figured out a solution and have pre-built images ready to push as soon as the embargo lifted is ridiculous.
I don't know about PHP, but at work our artifacts are signed, and you certainly cannot have the signing certificate on your laptop.
It's stored in some secure server, and only accessible from some allowlisted build nodes, for specific builds (release branch, ...).
Now of course, there are ways around that if required, but they are, by design, manual, cumbersome and slow. I could not create a release build locally in a few minutes. A few hours maybe if the whole organisation dedicated itself to it.
Setting up all that for single man project might be a hassle but for any sofware with actual dev team I wouldn't call it hard to do.
I'd worry more about the fact that essentially publishing binary-first means that it ever gets compromised that makes it easier for attacker to smuggle exploit using "this is security update" pretext.
I'd say that for someone like PHP, in this era of AI-driven vulnerability/patch race, having a private CI/CD pipeline is a necessity.
PHP is written in C, I'd be surprised if similar problems didn't lurk in other places.
Maybe they will construct the private pipeline after this crisis. Millions of websites are probably now waiting for all the downstream Docker images to be built and pushed.
The images everyone is waiting for are built by Docker though? So now if you're Docker you have to have a private CI/CD pipeline with all library projects?
It would be great, but it's also something to manage. Just keeping it working with all the changes that happen across all projects would be something.
Making a cloud CI/build/release system whose behavior can't be reproduced locally requires some kind of brain damage. I can't believe people tolerate this.
The problems you describe in your first post are only problems when you can't reproduce your build process locally. With a local build you can always do it in private for free.
The problems you bring up here are eminently solvable too. Yes there are many solutions for building for multiple architectures and versions. Cross builds, containers, VMs, or just having more than one computer locally. You can address building the wrong thing with reproducible builds, but for a one off patch like this it's not the most likely problem and relying on a trusted maintainer temporarily during an emergency is better than waiting for a glacially slow cloud build while exploits spread. Pushing builds to the cloud is inherently a cloud step, so it's fine to leave keys for that in the cloud, but there should be a process for publishing local builds and it should be essentially instant when the build is already done.
You should always have a local build process that replicates CI. There is no excuse.
The PHP Foundation should manage it, the Docker ""Official" Images" approach clearly just doesn't work properly for responding to vulnerability patches.
There's over 3 million downloads per month for the official PHP image. That may certainly not be "most PHP users". But still is a a lot of users.
I'm quite certain that your exact sentiment is why the images weren't updated immediately: I'm afraid PHP foundation doesn't see docker images as a particularly important distribution system.
> I'm quite certain that your exact sentiment is why the images weren't updated immediately:
I wouldn't be that certain.
I think it's likely that the vast majority of PHP Docker image downloads are for local development, for one thing.
But nevertheless, it would not come close to the number of websites running PHP from the Ubuntu Server packages. It is absolutely right that the effort was focussed on the distribution route. Because PHP is not "expert" hosting territory; it is still more often whole cloud VMs running the distro packages.
I find it incredibly naive that they’d hope it would not leak after weeks/months of embargo.
Being given advance notice of an upcoming security fix is an immense privilege, organisations should turn around quickly when given such information, they shouldn’t be given more than a couple of days to update IMHO, it’s not like the rest of the world can wait.
Keep in mind that some of the organizations are presumably patching ancient versions of PHP like 5.6. I am sure this is a process which takes considerable time, especially as upstream PHP has washed their hands of such old versions and the downstreams are effectively on their own.
(It is often much cheaper for an enterprise to just get a long term support contract than to properly upgrade to a modern PHP stack.)
> (It is often much cheaper for an enterprise to just get a long term support contract than to properly upgrade to a modern PHP stack.)
I've experienced similar to this. I know I shouldn't be, but I still get amazed every time I hear that some small company is still using an old DOS machine running core software based on dBase that nobody has any clue on how works and the developer has since moved on to a higher plane. One person told me that they sweat every time the power goes out longer than the attached UPS can run holding their breath that the machine powers back up.
PHP foundation also doesn't "Run" the apt or rpm packages, yet they certainly coordinated with debian, and other big distros to push the updates out timely.
Now I don't know what happened here. Maybe PHP foundation reached out to docker but got no help, maybe they simply forgot about docker as distributor of their software.
> The PHP security team closely monitored and inspected appearing PoCs after the leak but did not identify any functional exploits at the time of release/public disclosure
Okay, but how close did they get? Were they close enough that a determined actor could've (perhaps using an LLM) managed to get to a functioning exploit?
I know of a php exploited about 4 hours before their release time 1400 UTC, using the session bug in an exploit that was passed around. I'm really not very much in the loop about cyber security things.
So instead of using a memory safe language or harden the C code they hope LLMs will find bugs in their code? Neither do they mention how their way to patch Exceptions into C makes It very difficult for Fuzzers to find those bugs? Good luck this statement should tell everyone to migrate away from that horrible language and runtime.
It's one of the biggest runtimes on the internet; the C code will be some of the safest out there. Rewriting to a memory safe(r) language is a huge bet, even today. Nobody's stopping anyone from forking and creating alternative runtimes.
This is what I thought as well. How is this still so popular if it has such fundemental flaws? With the increasing power of LLM, more vulnerabilities will definitely surface sooner or later.
As long as programmers trust 3rd-party-provided content-length input, it will be insecure. At this point, I'm not sure why we even use content-length anymore since you're going to need to size your buffers based on the actual content.
There should be more interpreters or runtimes. Regardless of the language they are written in. Indeed a rustPHP, maybe a JPHP, cPHP, etc etc. There is zend, but AFAIK that's still using the "official" PHP under the hood. And there is or was hiphop. Not sure how that is going, though.
Because now a vulnerability in PHP affects practically 100% of the users. But with more diversity, it would affect only a part. And if one runtime proves insecure more often than another one, organizations have a choice to move to an interpreter that better fits their security needs.
I'm generally bullish on migrating C projects to Rust, but that will only reduce a particular class of vulnerabilities. There will still be other vulnerabilities which remain discoverable by LLMs, and for those, the problem of accelerated exploit creation will remain.
I'm really not sure how you mitigate this issue, other than serving static HTML and reducing the attack surface to cut out dependencies like PHP.
> The PHP security team, operating on consensus, struggled to choose between these and feared the repercussions of shipping the wrong choice.
This is the real story. You ship one of them (the breaking change) with —fuck-my-shit-up or something as a compile time option to switch to the other fix.
>On Tuesday, October 20th, at around 15:00 UTC, the two CVE IDs began circulating on online forums and social media. They came with a wrong release time and a wrong CVSS score of 10.0 (the actual score is 9.2). No technical details leaked.
This was partly me. I take responsibility for posting it on HN. I am truly sorry for the disruption this has likely caused.
My contact had informed me there was no embargo, and told me her contact had told her the same. It transpired this morning that was a lie.
>On October 21st at 07:41 UTC, more than six hours before the coordinated release, Chainguard unilaterally broke the embargo and published built images containing the fix. Like other distributors, Chainguard had received the patch in advance so its users could update the moment the vulnerabilities were disclosed. Any attacker could now study the fix in binary form while every other PHP user was still waiting for the release. This disregarded weeks of work by many parties toward a coordinated disclosure.
On the plus side, at least I'm not Chainguard. Yikes.
>Chainguard unilaterally broke the embargo and published built images containing the fix.
>This disregarded weeks of work by many parties toward a coordinated disclosure.
Good. Coordinated disclosure meant that people couldn't protect themselves. That's not a good thing. It meant sites were forced to shut down instead of simply updating.
No... Uncoordinated disclosure meant that attackers were suddenly armed with detailed information about an already-hyped severe vuln hours before the majority of downstream consumers had the means to patch their systems.
Maybe your calculus could hold if Chainguard's registry were the primary distribution mechanism for PHP in use worldwide (such that the benefit of an updated image would at least be widespread compared to the sitting ducks relying on the other repositories), but it isn't.
I will nitpick on that, if it was just released as a patch, there wouldn't be a hype to begin with, just point update to apply.
I'm just gonna assume patch was ready for a while and the wait time was just letting big players to update their own stuff before it is available for the public, and at that point it is just making the rest of the internet second class citizens not "worthy" enough to get the update quicker.
If the sites are taking no action they will be vulnerable no matter when the patch is released. For the people who do take action before chainguard's release the safe option was to just turn off your server (and let any big competitor site stay up and take your traffic since they got the patch ahead of time). People taking this option will be offline until a fix is available for them to use. They are not in danger of being hacked.
Trying to say that uncordinated disclosure is bad because people might get hacked yesterday instead of today just is not a convincing argument to me. I prefer the solution that results in the minimum amount of downtime for sites and sites provide value to society.
That is incredibly damaging to the trust that coordinated vulnerability disclosure depends upon. The obvious consequence is that Chainguard should never again receive embargoed fixes in advance. You don’t get invited back into the room after demonstrating that you can’t be trusted with the information.
They didn’t get to unilaterally decide that the embargo was inconvenient and publish early.
Why do you place a higher priority on maintaining coordinated vulnerability disclosure than of the safety of actual users of the software being protected?
You may never hear me say this again because I am a gruff cynic who is not sure that hanging out here is good for his psyche, but thank goodness for HN.
While I have been chasing around today, trying to resolve this on different servers, it turns out a three line nginx mitigation (blocking multipart uploads) I wrote in the wee small hours GMT has been quietly protecting a vulnerable machine I could not shut down (from attacks powered by every single mainstream agentic AI tool — Anthropic, OpenAI, Qwen cloud, Baidu, seemingly also AWS, Meta)
Including an hour before the PHP patch actually shipped.
So without the HN discussion I wouldn't have found enough people pointing at possible mitigations and I would have lost a server to this, and I thank everyone who pointed out that second mitigation in particular.
YES!!!!
I absolutely loved being here today.
So many brilliant minds screaming and hating together, trying to survive the tide.
There aren’t many places like this left on the commercial internet, especially in this brave new AI-driven world.
I felt a great disturbance in the Force. But in a good way.
Together, we are strong. May the Force be with us all!
> from attacks powered by every single mainstream agentic AI
Is it so bad? Have you measured the number of attacks?
> So without the HN discussion I wouldn't have found enough people pointing at possible mitigations and I would have lost a server to this
I run a tiny server. I like this server a lot. I only patched an hour after the Debian package was ready and didn't think about it again. Do you mean to imply there's a real chance my little server has been compromised?
[Edit:] My server is quite public too. It hosts four SLDs and many subdomains and receives both human and bot traffic. Only one or two subdomains use PHP, but a reasonable attacker surely tries all of them.
> Is it so bad? Have you measured the number of attacks?
It is not a torrent, no. Dozens up to hundreds, rather than hundreds to thousands.
But they would very likely have succeeded, right? It's a fairly turnkey vulnerability.
> I run a tiny server. I like this server a lot. I only patched an hour after the Debian package was ready and didn't think about it again. Do you mean to imply there's a real chance my little server has been compromised?
Can't say one way or another. Have a look at your log file for POST requests in quick succession for things like
> [Edit:] My server is quite public too. It hosts four SLDs and many subdomains and receives both human and bot traffic. Only one or two subdomains use PHP, but a reasonable attacker surely tries all of them.All my "obviously PHP app" domains got visited. None of my bespoke app domains (which are not obviously PHP). Though over time, the bespoke domains do get a whole bunch of boring probes for WP vulnerabilities etc., like all websites get.
> Have a look at your log file
Surely it doesn't take a very sophisticated attacker to clean up after themselves at least a little bit?
Depends how your logging works. If you ingest logs to a remote server it's quite hard for an attacker to clean those up. Also I'd guess that most of the PHP exploits tried today weren't sophisticated but rather brute force and then either cryptomining or botnetting or something like this.
I think it doesn't, but I have to say, I have never seen an automated attack on a PHP application that has bothered to hide its traces.
If your attacker is someone known to you, some kind of persistent threat, then maybe they would? But most of this stuff is typically kind of worm-like; attack one box, use it to attack another, move on.
With this particular server I am in two minds about whether it is likely I have any persistent type hackers, but I must say what it looks like is automated attacks. AI script kiddies, almost. These same boxes attract all the typical PHP app probes (WP2shell etc.)
And FWIW what it actually mostly looked like was hits from some sort of cloud-based AI search API; they suddenly stopped at the same time as if someone else had pulled a plug. So maybe someone running an agentic search API decided not to be part of the probes.
It was fun being here today. Validated a lot of the live frustrations and quirks and discovery.
HN was my source of truth for mitigating this
You would have been fine without HN, just use AI
Agreed. It made my ancient old man face tear pores by smiling when I saw people organizing in the comments to throw sandbags against the wall while waiting to be rescued.
It’s scary that people haven’t been able to move uphill though.
I mean, this isn't a things-made-in-PHP-are-bad problem.
It's a bit more a "certain C programming techniques are bad" problem.
This is more like log4j, heartbleed etc.
Won't be the last of these.
It is bizarre to me that servers are considered something to be “lost” in this day and age. They should as borderline ephemeral as us.
By "lost" I mean, it might have been broken, commandeered, exfiltrated.
This is PHP world; it may be a share nothing architecture, but it's full of long-running VMs and very long-running apps.
I think calling private data "lost" if it is accessed is not a misnomer, even if you have backups.
You lost its privacy
I'd like to express my dissatisfaction that the "big guys" are basically getting free security over getting earlier disclosure.
Just saying "hey blocking multipart uploads can mitigate most of attacks early would save people so much trouble. Of course not every CVE will have workaround but many do and whole secrecy about releasing it is only to benefit of few big tech companies and nobody else
Is it just me, or is it the truly brave that decide to deploy and protect PHP instances? Especially if WordPress or Drupal are involve?
I finished moving the remainder of my stuff off of PHP and WordPress last year and haven't missed it at all. Backwards compatibility was typically pretty bad.
It seemed like maintaining existing working PHP code was always a cycle of: upgrade to newer PHP, that version has breaking changes, fix that code or update to a new WordPress (plus all of the plugins that would inevitably need updating too).
I don't really want to be in the WordPress world anymore (I still maintain a couple alas).
But this isn't about PHP code, really — this is about C code (with less-than-ideal exception handler patterns). This problem is more Heartbleed than WP vulnerability.
I quite like modern PHP, I don't really think it's necessarily any less secure (exception handler weirdness notwithstanding), and more to the point, I don't think not using it is any kind of automatic improvement in web app security. It's not like server-side JavaScript is intrinsically better.
I am certainly doing some thinking today; I switched off a couple of web servers so I didn't have to think about them while I was working on other stuff, and actually I think they might not get switched back on. But it's not so much that they are running PHP as that they were kind of neglected and it's always worth reconsidering what you need to keep around.
I think I have reached a point where low value stuff is not worth subsidising just for fun.
In nearly every org I've been in or contracted to, the public landing page of the company gets a special place outside of any ops of general technical team, and is instead the work of a marketing group or a "creative" outsourcer who will default to Wordpress on PHP.
I’ve been to a few places like this and managed to get rid of the Wordpress infestation.
[flagged]
Please don't cross into personal attack.
https://news.ycombinator.com/newsguidelines.html
There's no personal attack here.
Unless your position that any statement whatsoever made in the second person that isn't a compliment is then an attack?
San Francisco is a magical place where "you are doing X wrong" without slathering on six paragraphs of fake kindness constitutes some kind of personal attack
"You're working on a false sense of security and encouraging others to do the same" was needlessly personal.
You can make your substantive points without that, both in San Francisco and elsewhere.
In no sane culture is an engineering assessment made in the second person a personal attack. You'd call "you are holding the screwdriver wrong" a personal attack too.
He is, in fact, not mitigating the attack, and he is, in fact, by calling his non-workaround a mitigation, encouraging people to do ineffective things instead of patching. These facts remain true in the real world regardless of what you call a personal attack.
And yes, San Francisco has become top of the world for indirect fake kindness that transforms plain factual statements about the world into social offenses.
"The sun is bright. Don't look at it " -- RUDE!
"You might want to consider that the sun might be considered bright to some people and that you might be more comfortable looking at this other cool thing instead. But no pressure" -- gruff, barely tolerable
Location isn't the issue. HN users all over the world have no trouble making their substantive points thoughtfully, and SF has its fair share of commenters breaking the rules.
You have a long history of throwing sharp elbows around on this site. You did it just yesterday on an unrelated topic: https://news.ycombinator.com/item?id=50167498.
We've asked you many times to stop. If you would please review the rules at and fix this, we'd appreciate it.
https://news.ycombinator.com/item?id=45089286 (Sept 2025)
https://news.ycombinator.com/item?id=44729387 (July 2025)
https://news.ycombinator.com/item?id=24254142 (Aug 2020)
https://news.ycombinator.com/item?id=22029640 (Jan 2020)
https://news.ycombinator.com/item?id=21104133 (Sept 2019)
https://news.ycombinator.com/item?id=17976632 (Sept 2018)
https://news.ycombinator.com/item?id=17976625 (Sept 2018)
https://news.ycombinator.com/item?id=17791892 (Aug 2018)
https://news.ycombinator.com/item?id=17492560 (July 2018)
Well, OK then:
1) you're being rude, just stop it and get a grip
2) actually, this particular mitigation is very clearly stopping all the unusual patterns of POST request attempts I can see (it's not usually a busy server)
3) I know it is not the only mitigation that is necessary
4) … so it isn't the only mitigation I have applied
5) I am not working under a false sense of security, and I know it is temporary
6) I am not an idiot, and I am working on patching this machine, but this particular server has a unique and complicated configuration and needs a bespoke build
7) I have been prioritising much more serious risks for other people, all day, for no pay
8) again, to reiterate, you're rude.
Please don't respond to a bad comment by breaking the site guidelines yourself. That only makes things worse.
https://news.ycombinator.com/newsguidelines.html
Sorry dang but I had a wider point to make there and I'm not really going to apologise this time. The rudeness is only part of what needed to be responded to.
Also I have survived on too much coffee and probably too many chocolate biscuits. So actually, sorry, on reflection :-) What a day.
It happens! Good luck with the rest of it.
I appreciate how your excessive-biscuit status has yet to be determined :)
We should really move to a different security model on the internet.
For example, when I buy something from a webshop, why does that webshop need my personal information? Why can't I give it a token, which it can then use to send my package, because only the postal system knows the address coupled to the token, etc.
And we should also ban user tracking for advertising purposes, obviously.
We are working on this better model in the Ethereum ecosystem via technologies like Zero Knowledge Proofs and private attestations. The Ethereum Attestation Service has been using the example of proving to the bartender that you are over 21 without disclosing your birth date. https://docs.attest.org/docs/core--concepts/privacy
This is all new and rapidly evolving technology but giving apps / agents / others just the information they need to know while having a credibly neutral, private, and trustless database mediating is a true cypherpunk vision of how the internet should work and it's being built by people with the right ethos and principles to be as reliable, well engineered, and long lasting as the internet itself.
What kind of political / economic system would make user privacy a priority? I'm guessing not finance capital neoliberalism...
Finance follows the risks and the users.
We'll probably never convince end-users to "vote with their feet" and use Privacy-focused products. But we can absolutely, through legislation or just precedent, raise the Risk profile until it's better to leave the ad revenue on the table and run a private service.
Shift liability to those who store and sell data. They get full economic responsibility for fallout of breaches of data that is retained past the time necessary to complete the transactions. If they sell the data then they hold liability for all breaches of downstream parties as well. Require them to carry insurance against breaches.
It would completely destroy the data brokers and targeted ad industry but the world would be a better place as a result. Also not going to happen but I can dream.
"As compensation for leaking your personal data, best I can do is...one year of Equifax monitoring."
We've already moved away from sites storing credit card information, because the risk of storing it yourself is simply too high. We need to make the stakes higher for other personal information.
Very fitting LL definition:
https://bsky.app/profile/luciferslexicon.bsky.social/post/3m...
> As a result of the leak and pre-announcement around 20:00 UTC, Calif, one of the security firms that had reported the issues mistakenly published a video.
This is the most surprising part to me. It sounds like Calif saw the leaks, interpreted that as embargo being broken, and then raced to publish their [anime dancing] marketing video? I'd have expected much better.
mistakenly?
This reads like some guy tripped and fell on top of his mouse and in so doing, accidentally turned on screen recording, performed the exploit a few times, got the anime girl asset and soundtrack, dropped all of them into Premiere, added the weird effects, rendered the video, and published the video to Twitter. "Ow. What did I fall on? Oh noooo, I broke the embargo!"
Heck of a mistake.
Your comment is funny. But obviously, "mistakenly" refers to the mistake of believing the embargo being broken, and then, very intentionally and not mistakenly, publishing a stupid video.
I bet they had the video already made, the mistake was publishing it too early.
That's literally what they just said??
reminds me of the guy that dials in to the mullah, asking "hey, I fell on plate of shwarma and ate it, what should I do now?"
some people are just lacking coordination
Definite candidate for the next "What Did We Get Stuck in a Twitter Video this Year?" list.
That video was published 4 hours ago by lcfr_hax, the embargo ended four hours ago. Was it posted earlier and then deleted or something?
That video ended the embargo. If someone releases a video telling you how to run an embargoed exploit, it's not an embargoed exploit any longer.
That's not how embargos work.
de jure de facto
The embargo was announced for Oct 21 14:00 UTC: https://www.php.net/archive/2026.php#2026-10-20-1
The video was posted 44 seconds before that time: https://x.com/lcfr_hax/status/2112906567662399830
This can't possibly be interpreted as some major violation of the embargo, so what are people talking about?
The video was posted hours early, quickly deleted (but not before being reshared), and then posted again at the proper time. The link in your comment is not the first release of the video
It was posted yesterday 24h before and then deleted.
This is EXACTLY how marketing/pr firms actually coordinate no matter what they SAY.
This is not the dancing anime girl video, is it?
That's the one
This is your brain on anime waifus.
Related:
PHP 8.2.35 - https://news.ycombinator.com/item?id=50180873 - Oct 2026 (439 comments)
PHP2Shell: AI uncovers critical RCE in PHP - https://news.ycombinator.com/item?id=50180836 - Oct 2026 (146 comments)
PHP session.upload_progress heap UAF (CVE-2026-109618 / -109619) - https://news.ycombinator.com/item?id=50176077 - Oct 2026 (61 comments)
Security advisory: Critical remote code execution vulnerabilities, update now - https://news.ycombinator.com/item?id=50172041 - Oct 2026 (631 comments)
Tell HN: 2x PHP RCE CVEs with CVSS 10.0 Incoming - https://news.ycombinator.com/item?id=50167754 - Oct 2026 (60 comments)
> Calif, one of the security firms that had reported the issues mistakenly published a video.
> Chainguard unilaterally broke the embargo and published built images ...
Won't be the last time this happens. If there is no consequence of breaking the rules, people will keep doing that.
Perhaps Chainguard will be the last to receive the next bug report.
Given how security moguls are handling that, we should go back to "just release it, consequences be damned". Coz that gets normal people better chance of actually fixing it
I’d say getting called out like this is a consequence.
So people will keep doing that
That sounds oddly threatening, especially given that chainguard is in the right here. Embargoes are stupid in 2026 and make no sense. The rumor of this issue was enough for it to be independently discovered by several white hats and almost certainly many threat actors.
> Embargoes are stupid in 2026 and make no sense. The rumor of this issue was enough for it to be independently discovered by several white hats and almost certainly many threat actors.
The point of the embargo was to prevent all rumors? But it did not hold.
Releasing a statement that a critical security bug exists with no more information, does the opposite of preventing rumors. I think one of the most surefire ways of generating rumors is to say "something big will happen tomorrow, please don't speculate about it".
Anyway, GPs point is that even if nobody leaked anything, and the only thing publicly known was the existence of the bugs, people would anyway have been able to find them independently. So what is the purpose of waiting 24h before releasing the patches?
What are these "rules" you speak of? There's usually nothing binding in these agreements. It's just groups either trying to do the right thing by end users or trying not to look like assholes, get a better payout, or some combination thereof.
Finding an exploit and publishing is is not against the law. In some cases, publishing immediately is the responsible thing to do. For example, when it's actively being exploited in the wild, and waiting just puts people at risk that don't have any recourse to protect themselves, since they don't know they're at risk. There can be nuance to that as well, but in general, as soon as it's being exploited, anything that makes it harder for end users to a) know there is an exploit, or b) tell if they're susceptible to it (e.g. through a POC) is just taking away people's ability to take the action they deem appropriate.
I think the PHP security team and the whole PHP team could have been more communicative and keep releasing information, adapting to the news, suspected exploits and mitigations.
If a video is released showcasing the exploit and following mitigation recommendation, PHP should clarify what it is and if the mitigations are recommended or not. From the pre-announcement: > We reserve the right to release earlier should details of either vulnerability become public, or should evidence of active exploitation appear in the wild.
If no info is released by them, no other party should have released anything. If info is released or leaked from partners or involved parties, you should notify and explain what's going on.
But no, complete silence from PHP news or any PHP community member.
I get that it was a complicated situation, and I think PHP is the "less wrong" party involved, but I think this constructive and external criticism could help in the future. It has been an interesting day (GMT+2 here) and a rollercoaster of monitoring and mitigating without any reliable source.
> But no, complete silence from PHP news or any PHP community member.
Not entirely correct as they clearly announced that they were going to release a fix at a specific date and time; the mitigation doing the rounds was not adequate and if the PHP people republished it, it would lull people into a false sense of security. Overcommunication is also bad, especially if the official platform or the individual members are being watched by tens of thousands of people hungry for news or updates.
They knew (since august, arguably since 2024), they had a fix ready, everything was in place. Except for the untimely / accidental early leaks.
The complete silence I refer is after the incidents and leaks. They stated in their preannouncement "We reserve the right to release earlier should details of either vulnerability become public". That sounded to me that if something is leaked, they'll release earlier or clarify. Neither was done and only after the release, clarified things.
But yes, the preannounce was the right move and the leaks were unfortunate, complicating things. But if cloudflare, bunny and other actors start mitigating in their WAF, I think something has gone wrong and, either you act by releasing earlier or calm the waters saying "the exploits are wrong" as per stated in their post, they thought it was the case.
“PHP security team”
Everyone in such a ragtag team would have an overlapping day job where the same crisis management needs to happen.
No meaningful info was released. A lot of incorrect info was speculated on.
I have a lot of sympathy for php team - it would be a lot of work to respond to every conspiracy theory on twitter, while trying to get everything ready, in the middle of the night no less.
I have a lot of sympathy for the PHP team as well, I hope it doesn't sound like it.
A lot of speculated info was right: the mitigations were mostly right, the video showing exploitation was real and the release of a fixed image was real.
I think if you do a pre-announce, and you say that you could release it earlier or release new information and you don't, and there are reasons to inform your users, then even if it's the middle of the night for you, you should.
As they stated, they are a small team and they have changed their structure, and they are learning and improving. I think if something like this happens, info is vital for us, the administrators of PHP servers
If mitigation is known, it should be posted as soon as it is found. Not "wait patiently for packages to update and hope to not get hacked".
I agree. That's what I hoped they'd have released between preannounce and version release. I think it's the minimum and doesn't hurt anyone.
But posting the mitigation helps guide attackers on how to exploit the vulnerability, and do so sooner.
> 2024, PHP underwent its first external security audit
> PHP security team, operating on consensus, struggled to choose
What an unfortunate waste of a security audit. (and consensus is an obvious flaw in decision making)
> PHP runs a large share of the web... Yet security work on PHP itself still depends on a small group ...
Another unfortunate big structural issue that's unlikely to change.
Are there structurally much safer ecosystems out there?
I wonder if that isn't a side effect of how its structured...
When a corporation sponsors the language, such as Java, Go, .Net, there's an inherent economic component which acts as a forcing function.
PHP being a foundation sounds nice. But consensus wasn't strong enough to affect the key outcomes.
why nobody in foundation management was not intereated in resulta of rhe security audit?
making php is the core of (checks notes) php foundation
or is the management full of completely not technical people who use it to boost their CVs?
> Are there structurally much safer ecosystems out there?
Several attributes make a safer ecosystem. An important one is diversity - technical diversity.
For an interpreted language, the runtime is important. Python has several runtimes, as does Ruby. Javascript also has several, from the few browsers, to node, deno and bun. PHP has for all practical purposes, only one. So if there's a security incident in PHP, it affects almost all users. But when e.g. JRuby has a vulnerability, it will affect only a fraction of the installed ruby applications.
For compiled languages, the tooling, libraries and runtimes is where diversity is important. With e.g. rust, the vast majority of CVEs come from libraries (crates). But since there often are several crates for a single concept, diversity helps mitigate here too.
> PHP has for all practical purposes, only one.
I know you added a caveat and all, but no, PHP has a number of alternative runtimes just like the other languages you mentioned: https://github.com/gakowalski/alternative-interpreters#alter...
But again with all the ones you mentioned, they have a "core" runtime that the vast majority of production apps use, so if NodeJS has a vulnerability it would be just as big.
The interesting one will be whether "boring" ports - like recently Typescript's TSC from Typescript to Go and now to Rust - will be safer for being the same code and logic, converted to another "safer" language (by transpilers or LLMs).
if there's a use case for re/decompiling, it's this one. with at least one strong alternative implementation, there would be less incentive on the attacking side to go all out on either PHP implementation.
...given amount of CVEs for Ruby that appears to not matter much
Rust and go don't have a C backbone like PHP does. So they are inherently safer to these kinds of vulnerabilities.
Garbage collection is also an option. Lots of backends on the JVM!
(Of course there's Log4shell, but that could've happened to a Rust-based stack too, and might be more likely to, considering the proliferation of deeply-nested dependency trees full of tiny libraries in that ecosystem.)
it's a good point. I wonder if we'll see a PHP fork and rewrite in rust. like tcrs. given the recently issues they would seem to benefit the most...
There are plenty of alternative interpreters or compilers out there already, no need to start a new one again: https://github.com/gakowalski/alternative-interpreters#alter...
Yes. They're operated by big tech and called C# or Java.
While the main Java and .NET virtual machine implementations are much more secure than PHP, they were still written at least partially in unsafe C++. I won't be surprised if frontier LLMs eventually find more vulnerabilities in those platforms.
Some code example for the exploits https://github.com/david-alvarez-rosa/php-rce
> On October 21st at 07:41 UTC, more than six hours before the coordinated release, Chainguard unilaterally broke the embargo and published built images containing the fix.
This is incredibly damming of Chainguard. I struggle to see justification for this, even with the chaos approaching the coordinated release.
They (is it the CEO?) commented on it earlier here: https://news.ycombinator.com/item?id=50176735
In their place I would worry that this was the last time anyone shared an embargoed build with them.
Let's hope so. Shall we create a new norm? A 365 day timeout, for companies like Chainguard that deliberately break embargoes.
Very irresponsible behaviour in my opinion; and if it's a decision made unilaterally by their CEO, probably a breach of their SOC 2 certification. What's next? "$X isn't patching this CVE fast enough, here's a binary blob of our fix, don't worry, we're making this free!"
Nah, Chainguard should be praised for this. It was a different company that leaked the POC early, and then PHP, Wiz, OpenAI, and Calif sat on their hands doing nothing while the vuln was being actively exploited in the wild. Chainguard did the right thing by getting the fix out as quick as possible to stop the bleeding (or gushing flow of blood, really) rather than sitting on it so they could make a coordinated marketing post, like the others did.
That's a reasonably take. Embargos only work until the information's leaked, and once that's happened, however it happened, the bad guys have the exploit information and the clock starts ticking.
PHP’s view is that Chaingiard started the clock because the video was insufficiently detailed. Not sure if that is the case, but just sharing I had a different reading of the statement.
It was being discussed on here hours prior to the Chainguard release.
>... Wiz, OpenAI, and Calif sat on their hands doing nothing while the vuln was being actively exploited in the wild.
What was Wiz/OpenAI supposed to do? PHP was already aware of the issue.
But did they "get the fix out"?
They released some binaries, built for their own docker image based on their own Linux distro. Other people couldn't use Chainguard's "fix" to patch their own systems. The only people who seem to have benefited from the stunt are the hackers who used AI to reverse-engineer the binaries and write exploits.
That was a giant fuck-you to the rest of the PHP ecosystem, nothing more.
> It was a different company that leaked the POC early,
[citation needed]
By which i mean, this seems untrue. If you mean the anime video, that is not a leak of a PoC.
I think multiple companies didn't respect the embargo, in different ways.
That can be for marketing purposes or by mistake. Not sure how to deal with that.
Chainguard effectively held everyone hostage, and forced everyone hosting container images and wanting to be online, to trust their builds and then saying they did it for "free" to get free advertisement. Incredibly irresponsible behavior. They used this as a big fat advertisement board.
Dont trust us with information lol.
A security vendor entrusted with advanced patches breaking embargo, is much worse than a security researcher breaking embargo.
> What's next? "$X isn't patching this CVE fast enough, here's a binary blob of our fix [...]"
https://www.chainguard.dev/athena
https://www.chainguard.dev/unchained/this-shit-is-hard-build...
I dunno, the post is awfully critical of Chainguard for leaking a fixed binary 6 hours early, but it was the research firm "partner" that "mistakenly" posted the dancing anime POC video over 24 hours early that created the mass panic (not to mention spawning millions of agents crawling the php codebase and trying to brute force a POC and probably using more energy than a lot of countries in the process).
i keep seeing mentions of the video, but is it still available anywhere?
https://x.com/k_firsov/status/2112641845926989944
Song is Petite Meller - High Forever (Marlon Hoffstadt Remix). Which is annoyingly catchy
wow, okay. yes, that really does look like a "my first PoC" video made by an early teenager. hard to believe that this was produced by a company that wants to be taken seriously.
thanks.
If anything this is further justification for why these patches need to be kept under wraps.
This industry is full of immature children who cannot keep their mouths shut for 24 hours. They have no clue what massive coordination this requires.
These 'immature children' have discovered the exploit, so good luck keeping it under wraps from themselves.
There do seem to be the perpetual problem with the field of computer security that only insufferable and immature kids matter, due to discrete math adjacent nature of computer science, and yet they also legitimately can't be trusted, or does it not... unless LLMs do change everything.
The management class is certainly itching to ditch the remaining nerds.
Any idiot can create their own company. For example, I have one :)
they claim 60 full time employees
Any idiot can work for any idiot that starts a company!
I wish people would name this idiotic company, I can't see Twitter here and all people are talking about is a childish video they made. I don't ever want to do business with said company, ever.
They're apparently not trustworthy, and behave as children.
It's Calif. They're named in the post.
>As a result of the leak and pre-announcement around 20:00 UTC, Calif, one of the security firms that had reported the issues mistakenly published a video. It showed their proof of concept working against a number of targets and revealed a few additional, though limited, details about the vulnerabilities.
You can see the same demo video on their site: https://calif.io/php2shell/#php-5-4-0-rce
calif.io
That gif is a segment from Shukusei!! Loli Kami Requiem by Shigure Ui, the illustrator-VTuber-singer, FWIW...
1: https://youtu.be/Ci_zad39Uhw?t=15
https://media.infosec.exchange/infosec.exchange/cache/media_...
Definitely archiving this, lol.
lmao, these are funny af
https://x.com/k_firsov/status/2112641845926989944
Agree. If I'm following along correctly, it seems Calif is really the one to blame not Chaingaurd.
Both are to blame.
Calif more than Chainguard, but still Chainguard too.
Calif was obviously twitching to claim credit for their findings, and what I find troubling is this eagerness was more important to them than giving users a chance to update their systems.
Even if they ended up waiting only a day post release I would still say it seems a bit too soon. Maybe a week after. Not everyone’s system have aggressive patching schedules and PHP is soo heavily used.
This is probably the future, tho. Being first to disclose a vuln is a Big Deal in some quarters. And now, the velocity of AI driven pentest swarms mean every researcher who finds something worthwhile has to think "how close is the next droog's swarm to finding what I found" and the temptation to jump the gun only gets stronger.
The video has no usable details in it.
It's maybe not professional to release an omg panic video, but not even remotely the same thing as what chainguard did.
I don't know, it's showing most sites being pwnd in less than 20s with some taking just under 30s. Of course, that's on homeboy's local Mac. Not sure how long it takes to gain control with PHP2Shell to run the exploit though. Just started looking at that part myself
So, just because they showed they can doesn't mean they showed how.
I dont know why the timing of the bug is surprising to you.
there were security guys publishing inferred details from [INFO] messages, it did contain more than zero usable details
Everything i saw on twitter prior to the release turned out to be incorrect. At most people figured out it had something to do with session upload handler and UAF, but nobody knew what.
> but it was the research firm "partner" that "mistakenly" posted the dancing anime POC video over 24 hours early that created the mass panic
They took it down but someone (Kirill Firsov: https://x.com/k_firsov/status/2112641845926989944) added the anime stuff and reposted it. Totally irresponsible.
Edit: okay he didn't add the anime, but he reposted it.
The company, Calif, has it with the music, effects and Ui Shigure animation on their own website and youtube channel: https://calif.io/php2shell/#php-5-4-0-rce
I think I'm too old for this industry
I want to go back to talks by bored defcon guys or gals calmly stating how they obtained private keys of a globally trusted CA with just a bit of math. Which, mind you, ate the other part of the teams time.
Now I know how much I hustled today likely due to that atrocity of a video. I think I will take a vacation day and sit in a forest near a river on friday.
what's up with the obsession of these people with anime children?
dopamine junkies crave stimuli, and realistic details of humans are just noise for the data. anime humans are simplified to extremis which alone makes some people think it's about kids
good god that video is absolutely horrible. why is stuff blurred then the same blurred stuff is suddenly all visible? nothing changed.
it's called creative licensing. you can't account for taste though.
I kinda get Chainguard's position here. They were part of the embargo, saw exploits hitting in the wild already and decided to release rather than hold things back.
Embargos make sense if no one is exploiting it yet, but if its out in the wild, the best path forward is releaseing as soon as possible.
Did they actually see exploits in the wild? The PHP Foundation team disputes this, and say they didn't observe any working PoC capable of an exploit.
The person who wrote the widely-referred-to tweet last night (claiming to "have recreated & lab/field tested CVE-2026-109618 / CVE-2026-109619") followed up a couple hours later and said that their POC had not been proven to work "in the wild"
[0] https://x.com/J4ck3LSyN/status/2112703096443097263
[1] https://x.com/J4ck3LSyN/status/2112753673235501137
The whole post looked to be vibecoded and the code did not even match the claims
As far as i know they lied and are now retracting their claims. What they posted was not consistent with the details of the real bug.
Cloudflare's blog post announcing its updated WAF rules directly implies the existence of in-the-wild exploits: https://blog.cloudflare.com/php-vulnerabilities-waf-protecti...
"Cloudflare successfully deployed multiple mitigations for all its customers while the vulnerability was leaked before the official release date and we observed attack traffic targeting our customers."
If that's not an in-the-wild exploit, it literally couldn't have been further away than one or two more iterations.
Attack traffic doesn’t necessarily mean the attacks were, or would have, succeeded.
From Cloudflare’s perspective, there is no downside to saying what they observed, and deploying defenses.
We may never know if this was actually being successfully exploited prior to patch availability.
You shouldn't wait until the attacks succeed to start deploying defenses.
That sounds like cloudflare trying to reassure paranoid customers. I think its a reach to go from that statement to actual in the wild exploits.
Chainguard most likely interpreted Calif's dancing anime girl video as a script kiddie who had found a POC.
Can you blame them? Calif's unbelievably unprofessional video's leak definitely spikes the cortisol of any actual professional.
It sounds like there was no "co-ordination" behind any of this co-ordinated disclosure. You'd have expected there to be an IRC channel, a mailing list, a Slack chat, or something for something of this scale.
Was there even an incident co-ordinator here? Was any of the parties talking to each other?
And then decided to not ask the PHP foundation about it?
Many people actively following would have 403'ed multiparts so people seeing those blocked might have assumed it started being exploited early instead of probes / background noise.
This single-handedly assures that I will never, ever do business with Chainguard or any member of its C-suite executive team.
There were signs that others broke the embargo too. Public (this one video everyone is talking about), but maybe also non-public. There were some rumors going around which part of PHP was affected. That was enough for some people to figure out the vulnerability.
But it's also possible, that some attackers got non-public inside information.
If the embargo is already broken in a meaningful way, it suddenly becomes much less clear if it is still in place after all. Emphasis on >less clear<.
if their comment in true (linked in sibling comments), then it's really not damning. typically an embargo is ended when someone leaks early or it's being exploited in the wild.
so, if they saw evidence of exploits in the wild, it is not unusual at all to release the fix.
If they're gonna unilaterally release an exploit patch 6 hours before an agreed upon embargo for a CVSS 9.2 that affects like ~70% of the internet then they better have reams of evidence showing this was being exploited in a widespread manner and not just "we saw a dancing anime girl edit of a theoretical exploit on twitter, so we had to release it to the world"
And evidence they at least gave a heads up to everyone in on the embargo, eg "I have to release you all have 1 hour to get on board".
There is a MASSIVE difference between a small number of people exploiting in the wild, and a few hours after a release a bazillion script kiddies doing so. The move to release and break an embargo was childish, absurd, self-serving, and lacking in all common sense. Not to mention, breaking trust.
I agree with those in this thread who say, never going to do business with them. Ever.
> Not to mention, breaking trust.
I highly doubt their paying customers feel the same way
Maybe, but if I was maintaining PHP or another critical software, I’d think twice about letting Chainguard have inside information.
I would let Chainguard deal with the patches post official release like all other unreliable parties.
A weird take.
As a customer, I'd immediately assume no one in the security community would trust them again, and seek someone else to give my money to. Why pay a) an untrustworthy company who b) now is not trusted by anyone and c) won't ever be able to early release again anyhow? And if anything, are left out of important conversations.
Way to burn bridges. You only get to be an asshat once.
> You only get to be an asshat once.
I have seen no evidence to back up this claim. Quite the contrary actually.
> You only get to be an asshat once.
TIL observing an exploit in the wild, after the details leaked during the embargo and pushing a fix for your customer is "being an asshat".
It's the age of LLMs. Exploits go from theoretical to real fairly quickly. PHP's open source, it's not impossible to conceive of someone seeing that video, getting the general idea of what to do, and working the rest out with a decent model.
Brain rot is making us incompetent, dangerously impatient, or a mixture of both.
Not sure if this [1] comment is from them?
Because if there are already publicly available exploits seen, it makes no more sense to keep holding patches back just so that OpenAI etc. can get a nice press release
[1] https://news.ycombinator.com/item?id=50176735
So if I'm understanding correctly:
1. There was a leak of the pre-announcement
2. Calif sees leak, goes and releases dancing anime girl video before pulling it, goes viral
3. Chainguard sees dancing anime video, doesn't recognize Calif, and thinks the exploit is in the wild
[Edit 3a]: Or alternatively, Calif's leaked video causes the PoC to be found by others by providing enough of a lead
4. Chainguard releases updated binaries thinking it doesn't matter, causing the real exploit to then be found early
5. Which then causes PHP Foundation to basically have a meltdown
Personally I blame Calif for the unprofessional video which absolutely completely looks like some script kiddie and for being too detailed [edit 2: assuming Calif produced it and the anime wasn't a prank edit]. In this case it wasn't just unprofessional fun but may have literally messed up the responsible disclosure.
> 3. Chainguard sees dancing anime video, doesn't recognize Calif, and thinks the exploit is in the wild
I'm not sure about this part. I saw https://jackalsyn.com/posts/php-up-2026.html going around a bit over 12 hours ago, before the chainguard images went out, and plenty of other chatter of exploitation happening in the wild.
Which seems to be someone bullshitting with AI and had nothing to do with the exploit in question.
That same user wrote a followup tweet last night asserting that their POC wasn't ready to do an exploit in the wild https://x.com/J4ck3LSyN/status/2112753673235501137
Or maybe the anime girl video was an edit someone else made on top of an otherwise normal Calif video.
Still not great though.
Unbelievably the anime girl video was the official version from Calif.
https://calif.io/php2shell/#php-5-4-0-rce
The video got taken down by a copyright claim from Sony Music Entertainment. Unthinkable levels of unprofessionalism by Calif.
Not really sure if 3 is correct. Might have been Calif leaks the video -> it goes viral -> people make PoCs based on the video -> Chainguard releases the binary because there seem to be credible PoCs out there (that are not just the Shigure Ui video).
I was asking myself today; how to ensure that you don't miss a cve? I mean you'll sooner or later stumble across it if you're monitoring your communities but I thought that maybe there exists some kind of central solution, like a service were you could subscribe for your tech stack and get notified?
There are vendors who supply this service as a product -- Wiz, who were one of the groups to independently discover the vulnerability, is one such vendor, there are plenty of others.
I'm not overly keen on the tooling: as far as I can tell the whole industry is prone to flagging false positives as "vulnerabilities" rather than "potential vulnerabilities" and it's a bit too easy to turn those reports into a tick-box "you must update now because the tool says so". But they also do flag true vulnerabilities. And they pay for teams of security researchers to find and fix issues, not least because that's good marketing.
Big companies usually use some sort of vulnerability response management software which is connected to your CMDB, ideally containing SBOMs of everything you use, and subscribed to various providers of CVEs. Those tools then map the CVEs to your SBOMs and you'll get alerts, security incidents and so on.
Several tech stacks have this built in or available.
For my rust applications, I use "cargo audit" which checks all dependencies against the CVEs. For my typescript and javascript applications, deno audit and/or yarn npm audit.
For the rest of the stack `debsecan` does exactly what you want on Ubuntu: report CVEs for installed software.
I was thinking about this as well today. And while I'm sure there are many cybersec companies that send out alerts, I trust this forum the most.
Maybe it would be worth it to have an agent that knows your stack, reads the top threads on hn all day long and messages you when it thinks there is something serious going on that could impact you.
I don't think there's any robust method. I lurk HN, subscribe to security lists, run our code through scanning tools regularly. All options suck, all options have gaps, and all options mean you have to stay on the hamster wheel.
Lots of licensed solutions have this (typically SAST or DAST tooling like Snyk, etc...). Given your build manifest/SBOM they cross-reference CVEs for vulnerabilities during/after build time.
Some have free licenses, many don't.
There are also better tools like Sysdig, Twistlock, etc. which scan containers at runtime for CVEs.
Sometimes build time is too slow (e.g., a repo that doesn't receive updates every day or so).
I’m curious about this too, although I would somewhat hesitate to rely on that, except as a contributing point of information.
PHP2SHELL, in particular, made me expand my weekly security review to include my external network resources.
The natural conclusion for this seems to be realtime monitoring CVE and pre-CVE vulnerability indications in exposed stacks.
GH Dependabot.
I may be misunderstanding this but think the hard part about building a threat intel feed like this is just trusting that the underlying integration would surface something you care about... especially if you want to DIY it. You can scrape HN, GitHub advisories, NIST, whatever you want... but you may give yourself a false sense of security after seeing the "all clear" checkbox each morning
I hope that someone has some sort of a reasonable approach here / I would love to help work on something open source if not
Any tool to scan dependencies takes various inputs like repos, container stores or already built SBOM and gives you warnings. It doesn't much matter if you have actual code and repos as long as you know how to point it at some. The fremium ones like snyk would love to send you an email report on growing CVE counts on your last run but only until hell freezes over..
Turn on automatic updates.
That's great until an attacker compromises the automatic update system and uses it as a vector.
I would wager that far more systems get exploited because they are out of date (sometimes woefully) than by poisoned package repos.
But, certainly do your own threat assessment and develop and implement whatever plan works for you.
That's what cooldowns are for
That's great until you have a public zero-day exploit.
If it's big enough like this one you can always make an exception.
but the embargo delayed the updates
This is how you get wrecked by malicious packages being published or other breaking bugs. Pin your versions.
It's interesting that even 4chan was taken down for over 24hours now. Responsible people there.
And nothing of value was lost.
SomethingAwful too
DeviantArt as well
> In 2024, PHP underwent its first external security audit in years. The PHP Foundation secured funding from the Sovereign Tech Agency to organize this audit ...
Well that's your problem right there! There are trillion dollar companies profiting off the back of open source projects without supporting the projects adequately. Why should they be scraping for funding for basic table stakes like audits?
Maybe a new one? https://fedi.katze.sh/@elise/117479383739450605
https://www.openwall.com/lists/oss-security/2026/10/21/23
Step 1: Ensure that docker images are available when the embargo is released. We're 2 hours in, and they still aren't available.
I can empathize with them, this is actually quite difficult when you're an open source project.
You have your source code public, so you can't commit/push because then you've leaked info.
Your build pipeline is also likely public, so you can't just build from another source code... source.
You can create a private repo, but now you also need to build a private build system.
You don't get the same generous build resources for private repos, so you either:
a) simplify the build to just the essentials
b) pay
Unless you have significant funding, you'll probably go for a).
Now you have to maintain two systems, also with some differences between them.
Plus you need a way to get the build artifacts from the private system to the public one, typically at a fixed point in time.
Any failure/delay (e.g. less powerful build on private repo, errors because you exercise it rarely) causes people to get upset that you didn't respect the announced time.
Maybe you're also using things like image attestation, another can of worms.
Linux Kernel approach of "just publish fix without fanfare" seems to be least compromising way to do it. Every bug can be security bug
Those are all challenges that surely need to be overcome!
But they had _two months_ to figure out how to overcome them. For them to not have figured out a solution and have pre-built images ready to push as soon as the embargo lifted is ridiculous.
Did they have the money and the people? PHP is chronically underfunded.
A private build followed by a manual push would take literally two commands and could be done from a maintainer’s laptop… it is not rocket science.
I don't know about PHP, but at work our artifacts are signed, and you certainly cannot have the signing certificate on your laptop. It's stored in some secure server, and only accessible from some allowlisted build nodes, for specific builds (release branch, ...).
Now of course, there are ways around that if required, but they are, by design, manual, cumbersome and slow. I could not create a release build locally in a few minutes. A few hours maybe if the whole organisation dedicated itself to it.
Setting up all that for single man project might be a hassle but for any sofware with actual dev team I wouldn't call it hard to do.
I'd worry more about the fact that essentially publishing binary-first means that it ever gets compromised that makes it easier for attacker to smuggle exploit using "this is security update" pretext.
I'd say that for someone like PHP, in this era of AI-driven vulnerability/patch race, having a private CI/CD pipeline is a necessity.
PHP is written in C, I'd be surprised if similar problems didn't lurk in other places.
Maybe they will construct the private pipeline after this crisis. Millions of websites are probably now waiting for all the downstream Docker images to be built and pushed.
The images everyone is waiting for are built by Docker though? So now if you're Docker you have to have a private CI/CD pipeline with all library projects?
It would be great, but it's also something to manage. Just keeping it working with all the changes that happen across all projects would be something.
All library projects, certainly not.
But the top 1 percent by usage/downloads, I can imagine, and PHP probably qualifies into that top slice.
Making a cloud CI/build/release system whose behavior can't be reproduced locally requires some kind of brain damage. I can't believe people tolerate this.
That's... not what I said at all?
But let's go with that scenario, now you have all sorts of problems, from practical to security, like:
- accidentally/maliciously building the wrong thing? there's a patch, people agreed to it, but the wrong commit is built
- building for multiple architectures/versions?
- signing keys on someone's laptop?
- worse, image push/write keys on someone's laptop?
And so on.
The problems you describe in your first post are only problems when you can't reproduce your build process locally. With a local build you can always do it in private for free.
The problems you bring up here are eminently solvable too. Yes there are many solutions for building for multiple architectures and versions. Cross builds, containers, VMs, or just having more than one computer locally. You can address building the wrong thing with reproducible builds, but for a one off patch like this it's not the most likely problem and relying on a trusted maintainer temporarily during an emergency is better than waiting for a glacially slow cloud build while exploits spread. Pushing builds to the cloud is inherently a cloud step, so it's fine to leave keys for that in the cloud, but there should be a process for publishing local builds and it should be essentially instant when the build is already done.
You should always have a local build process that replicates CI. There is no excuse.
The PHP Foundation should manage it, the Docker ""Official" Images" approach clearly just doesn't work properly for responding to vulnerability patches.
FWIW, Docker images are a nice-to-have, but OCI containers are not how most PHP users get their PHP.
For all the long-tail users, PHP is a distro package. That route needed the most work instead, and it got it.
There's over 3 million downloads per month for the official PHP image. That may certainly not be "most PHP users". But still is a a lot of users.
I'm quite certain that your exact sentiment is why the images weren't updated immediately: I'm afraid PHP foundation doesn't see docker images as a particularly important distribution system.
> I'm quite certain that your exact sentiment is why the images weren't updated immediately:
I wouldn't be that certain.
I think it's likely that the vast majority of PHP Docker image downloads are for local development, for one thing.
But nevertheless, it would not come close to the number of websites running PHP from the Ubuntu Server packages. It is absolutely right that the effort was focussed on the distribution route. Because PHP is not "expert" hosting territory; it is still more often whole cloud VMs running the distro packages.
I find it incredibly naive that they’d hope it would not leak after weeks/months of embargo.
Being given advance notice of an upcoming security fix is an immense privilege, organisations should turn around quickly when given such information, they shouldn’t be given more than a couple of days to update IMHO, it’s not like the rest of the world can wait.
Keep in mind that some of the organizations are presumably patching ancient versions of PHP like 5.6. I am sure this is a process which takes considerable time, especially as upstream PHP has washed their hands of such old versions and the downstreams are effectively on their own.
(It is often much cheaper for an enterprise to just get a long term support contract than to properly upgrade to a modern PHP stack.)
That... just makes it so their tech debt turns into entire world's security problems.
5.6 is unsupported. There should be not getting special treatment just because they have massive tech debt
> (It is often much cheaper for an enterprise to just get a long term support contract than to properly upgrade to a modern PHP stack.)
I've experienced similar to this. I know I shouldn't be, but I still get amazed every time I hear that some small company is still using an old DOS machine running core software based on dBase that nobody has any clue on how works and the developer has since moved on to a higher plane. One person told me that they sweat every time the power goes out longer than the attached UPS can run holding their breath that the machine powers back up.
today I learned that building your image from scratch is fundamental in cases like this. Luckily I came prepared thanks to Claude Code suggestion.
Aren't those run by docker? php foundation cannot be responsible
PHP foundation also doesn't "Run" the apt or rpm packages, yet they certainly coordinated with debian, and other big distros to push the updates out timely.
Now I don't know what happened here. Maybe PHP foundation reached out to docker but got no help, maybe they simply forgot about docker as distributor of their software.
Yep, all our efforts are now focused on working around that. It's pretty neat!
Never using docker hub ever again
> The PHP security team closely monitored and inspected appearing PoCs after the leak but did not identify any functional exploits at the time of release/public disclosure
Okay, but how close did they get? Were they close enough that a determined actor could've (perhaps using an LLM) managed to get to a functioning exploit?
I know of a php exploited about 4 hours before their release time 1400 UTC, using the session bug in an exploit that was passed around. I'm really not very much in the loop about cyber security things.
I saw two on twitter, they were both pretty far away.
What is the penalty of breaking the embargo? Not shipping patches anymore beforehand?
It might make other projects reconsider sharing advanced patches with Chainguard.
Nothing's gonna happen, just as with previous embargo breaks, people complain yet nobody involved gets punished
Off topic, but TIL that .foundation is a TLD!
Related:
PHP2Shell: AI uncovers critical RCE in PHP
https://news.ycombinator.com/item?id=50180836
Patch the Planet: global response coordination as agents uncover critical vulnerabilities
https://news.ycombinator.com/item?id=50181175
Upcoming critical PHP security release (CVE-2026-109618 / -109619)
https://news.ycombinator.com/item?id=50172041
Very disappointed by Hetzner Ubuntu mirrors not getting updated in time. Had to manually override to get the patch.
Same here, I suppose they didn't bother triggering a cron job early? They did patch managed servers and the like according to https://status.hetzner.com/incident/f4144d55-38c3-48d1-8664-....
So instead of using a memory safe language or harden the C code they hope LLMs will find bugs in their code? Neither do they mention how their way to patch Exceptions into C makes It very difficult for Fuzzers to find those bugs? Good luck this statement should tell everyone to migrate away from that horrible language and runtime.
It's one of the biggest runtimes on the internet; the C code will be some of the safest out there. Rewriting to a memory safe(r) language is a huge bet, even today. Nobody's stopping anyone from forking and creating alternative runtimes.
This is what I thought as well. How is this still so popular if it has such fundemental flaws? With the increasing power of LLM, more vulnerabilities will definitely surface sooner or later.
Please identify a non-toy language that has never had a vulnerability.
Sounds like a lot went wrong, the 2024 issue not being patched and then multiple parties leaking stuff before the scheduled release date.
As long as PHP is written in C, it will be insecure.
It sounds cliche but yes, they should rewrite PHP in rust or other memory safe language.
It's pretty popular in Go: https://frankenphp.dev/
As long as programmers trust 3rd-party-provided content-length input, it will be insecure. At this point, I'm not sure why we even use content-length anymore since you're going to need to size your buffers based on the actual content.
Because when file finishes you want to verify whether the size matches.
And for non-malicious content, you can drop file above the upload max limit before user wastes time sending the file.
There should be more interpreters or runtimes. Regardless of the language they are written in. Indeed a rustPHP, maybe a JPHP, cPHP, etc etc. There is zend, but AFAIK that's still using the "official" PHP under the hood. And there is or was hiphop. Not sure how that is going, though.
Because now a vulnerability in PHP affects practically 100% of the users. But with more diversity, it would affect only a part. And if one runtime proves insecure more often than another one, organizations have a choice to move to an interpreter that better fits their security needs.
And, i don't understand why we are still using PHP. Agents can use Rust or go directly and we get benefit of performance...
Because like 30% of the websites in the world are on WordPress, and countless others in custom PHP? Not exactly a trivial task.
Claude please rewrite my entire WordPress site as an OCaml Mirage Unikernel that runs directly on bare metal, no Linux.
I bet that takes a week at most.
While at it, create a custom firmware for the BIOS to utilize the CPU more.
In fact, design an FPGA that runs my wordpress blog about my stamp collection in pure hardware. Make no mistakes.
and the increased costs of maintenance of a code base that humans will be reluctant to read.
Because its way easier to verify LLM generated code for Laravel over some Rust project. Or do you just vibe it and dont look at code anymore?
I'm generally bullish on migrating C projects to Rust, but that will only reduce a particular class of vulnerabilities. There will still be other vulnerabilities which remain discoverable by LLMs, and for those, the problem of accelerated exploit creation will remain.
I'm really not sure how you mitigate this issue, other than serving static HTML and reducing the attack surface to cut out dependencies like PHP.
> The PHP security team, operating on consensus, struggled to choose between these and feared the repercussions of shipping the wrong choice.
This is the real story. You ship one of them (the breaking change) with —fuck-my-shit-up or something as a compile time option to switch to the other fix.
The guy who posted the CVE IDs early: https://news.ycombinator.com/item?id=50183326 (dead comment)
Hah. Wiz Calif-oAI.
serious question
for legacy systems, how difficult would it be for an attacker assisted by "AI" to stage the conditions
on a scale of 1 (easiest) to 10 (hardest)
Given the exploits are literally out there, 0.
You forgot to say "Asking for a friend"
What do you mean by stage these conditions?
0. Seriously.
>On Tuesday, October 20th, at around 15:00 UTC, the two CVE IDs began circulating on online forums and social media. They came with a wrong release time and a wrong CVSS score of 10.0 (the actual score is 9.2). No technical details leaked.
This was partly me. I take responsibility for posting it on HN. I am truly sorry for the disruption this has likely caused.
My contact had informed me there was no embargo, and told me her contact had told her the same. It transpired this morning that was a lie.
>On October 21st at 07:41 UTC, more than six hours before the coordinated release, Chainguard unilaterally broke the embargo and published built images containing the fix. Like other distributors, Chainguard had received the patch in advance so its users could update the moment the vulnerabilities were disclosed. Any attacker could now study the fix in binary form while every other PHP user was still waiting for the release. This disregarded weeks of work by many parties toward a coordinated disclosure.
On the plus side, at least I'm not Chainguard. Yikes.
The old joke:
The guide to PHP security:
Use something else.
The end.
>Chainguard unilaterally broke the embargo and published built images containing the fix.
>This disregarded weeks of work by many parties toward a coordinated disclosure.
Good. Coordinated disclosure meant that people couldn't protect themselves. That's not a good thing. It meant sites were forced to shut down instead of simply updating.
No... Uncoordinated disclosure meant that attackers were suddenly armed with detailed information about an already-hyped severe vuln hours before the majority of downstream consumers had the means to patch their systems.
Maybe your calculus could hold if Chainguard's registry were the primary distribution mechanism for PHP in use worldwide (such that the benefit of an updated image would at least be widespread compared to the sitting ducks relying on the other repositories), but it isn't.
I will nitpick on that, if it was just released as a patch, there wouldn't be a hype to begin with, just point update to apply.
I'm just gonna assume patch was ready for a while and the wait time was just letting big players to update their own stuff before it is available for the public, and at that point it is just making the rest of the internet second class citizens not "worthy" enough to get the update quicker.
If the sites are taking no action they will be vulnerable no matter when the patch is released. For the people who do take action before chainguard's release the safe option was to just turn off your server (and let any big competitor site stay up and take your traffic since they got the patch ahead of time). People taking this option will be offline until a fix is available for them to use. They are not in danger of being hacked.
Trying to say that uncordinated disclosure is bad because people might get hacked yesterday instead of today just is not a convincing argument to me. I prefer the solution that results in the minimum amount of downtime for sites and sites provide value to society.
You need to pay to get access to chainguard images.
That is incredibly damaging to the trust that coordinated vulnerability disclosure depends upon. The obvious consequence is that Chainguard should never again receive embargoed fixes in advance. You don’t get invited back into the room after demonstrating that you can’t be trusted with the information.
They didn’t get to unilaterally decide that the embargo was inconvenient and publish early.
Why do you place a higher priority on maintaining coordinated vulnerability disclosure than of the safety of actual users of the software being protected?
They made them available for everyone and free in this case.
https://news.ycombinator.com/item?id=50176735
No, you don't, you just don't get older patched versions unless you pay.
not the ones related to this