The article focuses on OSS, but closed-source software is at major risk too. Perhaps more.
It's gotten much easier to reverse engineer binaries in general, and security patches in particular. Basically, an LLM can turn binaries into 'readable' code, and then reason about said code.
That happens a lot though, even OpenAI is attempting to lock functionality (like computer-use, 2 weeks ago) behind a binary -- Mac only they said, no EU. I saw a guy crack it the same day, ported to Windows*. There are many many things like Rive that use binaries, obfuscation and uglification has been the name of the proprietary game for a long ass time guys like "surely nobody would go through that trouble", yeah an LLM would ralph loop through it all day long, and make what you paid good money for pretty much free for anyone to use whenever they feel like it, we're back to "you wouldn't download a car would you?" levels
*that was two entire weeks ago, what I'm seeing now makes that guy's binary crack look like a toy, it's becoming systematized now
I had done a fair bit of reverse-engineering-jar-files in the pre-LLM era for various reasons. The biggest problem with decompiled java files was naming. The original variable names, class names etc were not retained and the decompiler would use some alphanumeric series. That'd make reading code very hard. Curious how the current LLMs are able to address this. Maybe it's able to figure out how the class, variable etc is used and name it accordingly. (All this is assuming the original code itself was readable because there are enough bad programmers)
I do not think author understands how opensource works. You have a problem on your computer, in __your__ software, and somehow some random dude is responsible for fixing it? Sure if you gimme a few kilo USDs I will drop everything and come to rescue you. But for free it is a volunteer gig I do once a month....
Say I had $1000, how do I get the best value for money to discover vulnerabilities? Are there any worthwhile LLM powered services that are turnkey and ready to go?
From what I've heard, every LLM before Mythos (which you can't get, they'll call you if you're big enough) will have far too many false positives to be helpful, so I guess the best option would be to use an agent to help you (not lights-off vibe coding!*) take advantage of all the older tools like valgrind and closing all the compiler warnings?
* I presume I'm not the only one to find the agents tasked with adding unit tests will sometimes try to sneak through "open source code and apply regex to confirm presence or absence of specific string literal".
They can speed you up significantly, but you absolutely do need to pay attention to what they produce.
With all respect to the Anthropic folks, that's just marketing. (If they're reading this: let us into the program so I can be proven wrong here.)
I'm sure what they have is awesome, but it's clear that there are people out there with some decent prompts that are getting results out of widely available models as well.
The big thing we're sharing is: bulk scanning by random people in random geographies got a _lot_ better around January, it's widely distributed, and it's going to get a lot better regardless of whether that specific version of Mythos becomes widely available or not.
> prompts that are getting results out of widely available models as well.
Absolutely, and the "false-positive" issue people keep citing as why Mythos is so good is easily solved in the harness, simplest solution is starting fresh context with another prompt to evaluate if it's a false-positive or not, just adding that drastically cuts down the rate.
That is false. A year ago every LLM generated report was slop - more likely a false positive than correct. However in the past few months nearly every LLM generated report is real.
Not sure about turnkey solutions for finding vulnerabilities that doesn't involve having to hand over a bunch of identity proofs for them to store on their insecure infra and also enrolling in programs.
Besides that, hiring a beefy GPU instance at Vast.ai or similar places then running your own uncensored models on it, I've had great success with AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-NVFP4, smart + uncensored, but there are lots of options, probably some are already tailored for security research.
Clearly for commercial oriented opensource software, security through obscurity is one way to keep the pace in the short term. Not an option for proper open source software.
Will this be the case that people who use open source software that is easily detectable will also start to shy away from using them for the fear of zero-days?
One of the benefits of Open source has been that there are more eye balls on the source, leading to more secure code/better quality. I think given enough time the bug reports will plateau and we will be back to a normal cadence - once the tsunami is over, hopefully things will settle at a more manageable cadence .
I'm not sure that the benefit of many eyes helps here. So much of this bulk scanning is low-effort, and if you're a smart person developing closed source software you get the benefits of bulk scanning, but _at the time of your choosing_ .
OSS has always had tradeoffs and I sadly think this one is going straight to the "Cons" column. We still think the Pros outweigh the Cons, but this is NotGreat.
The problem on the side of closed source software is that if there had been leaks of source code, the vulnerabilities and exploits may remain unknown for long time.
I would go to say that most closed source software code gets leaked. Most companies hold that info close and don't disclose it, even if legally required unless it's made public.
Nobody really knows of course. However it is safe to assume they are not so stupid as to ignore what is happening in the other areas (at least some of them), and so they are running their own targeted scans and then trying to figure out how to make money (or whatever their goal is) by exploiting them. They are also using LLMs to try things on closed source that are more than a brute force attack, though I have no idea what those would be.
> Did you have other plans for the weekend? Or a long term project you’re prioritizing? That’s nice, you have a new plan — fix every vulnerability that comes in NOW.
Umm... no? It's called OPEN source. Expecting people to cancel their plans to make your free software more secure is pretty audacious. Luckily, many WILL, but the expectation is just foolish.
Good luck getting anyone who values their time to even triage the results.
I would rather lick the bottom of a NYC dumpster that a rat had just died in.
Software will eventually become "unmaintainable due to lack of interest", because of this very thing. People not invested in this are not "in peril" in any way.
A lot of people are invested without realizing it. I'm typing this on a computer running linux, with all the standard services/software. I maintain one OSS project (icecc - we have always said only run on trusted networks. I'm sure there are a lot of issues in our code but nobody has bothered run a scan yet to my knowledge), but I don't pay attention to everything. I'm sure there are known easy to exploit (with a LLM) issues on this computer just because my distro hasn't updated yet. (I need a better distro, but even the most up to date will constantly have these issues)
What you just described may be accurate. But it also is the essence of a "trap".
My comment about investment was more to that point.
If software "is a trap", even my ever-computing loving wrote first programs on an Apple II in the 80s will only be as you sort of describe invested in by reference (minimal usage).
But no-one will sign up for a "trap" as a career, and only those who do will deal with its problems. The first thing that comes to mind is "Johns", "Hotels", and the trappings of the sex trade.
Apparently the AI company Metabase has a very poor code base. Like so many others, instead of questioning their own (or AI) output, they help their AI overlords by promoting security scans.
Fact is that Mythos found only one issue in curl and nothing at all in most code bases. It is getting quiet around Mythos, and the AI companies will move on to the next scam.
Mythos found only one issue in curl - but it didn't start until many other LLMs had been run and found a lot of issues that were fixed. If Mythos was run a year ago it would have found over 100 issues (of course it didn't exist a year ago, nor did the other tools).
Curl had many old protocols and code from the 1990s that no one used. Besides, Mythos was claimed to be better than existing tools.
In most open source projects, Mythos or similar tools have found nothing. The AI people only contact the projects where they find something, because it would be bad for marketing otherwise.
It's gotten much easier to reverse engineer binaries in general, and security patches in particular. Basically, an LLM can turn binaries into 'readable' code, and then reason about said code.
But yeah, if you're distributing binaries publicly, then you're going to have very similar problems.
*that was two entire weeks ago, what I'm seeing now makes that guy's binary crack look like a toy, it's becoming systematized now
* I presume I'm not the only one to find the agents tasked with adding unit tests will sometimes try to sneak through "open source code and apply regex to confirm presence or absence of specific string literal".
They can speed you up significantly, but you absolutely do need to pay attention to what they produce.
I'm sure what they have is awesome, but it's clear that there are people out there with some decent prompts that are getting results out of widely available models as well.
The big thing we're sharing is: bulk scanning by random people in random geographies got a _lot_ better around January, it's widely distributed, and it's going to get a lot better regardless of whether that specific version of Mythos becomes widely available or not.
Absolutely, and the "false-positive" issue people keep citing as why Mythos is so good is easily solved in the harness, simplest solution is starting fresh context with another prompt to evaluate if it's a false-positive or not, just adding that drastically cuts down the rate.
Besides that, hiring a beefy GPU instance at Vast.ai or similar places then running your own uncensored models on it, I've had great success with AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-NVFP4, smart + uncensored, but there are lots of options, probably some are already tailored for security research.
One of the benefits of Open source has been that there are more eye balls on the source, leading to more secure code/better quality. I think given enough time the bug reports will plateau and we will be back to a normal cadence - once the tsunami is over, hopefully things will settle at a more manageable cadence .
OSS has always had tradeoffs and I sadly think this one is going straight to the "Cons" column. We still think the Pros outweigh the Cons, but this is NotGreat.
Source that is unmaintained is dead. Nobody is looking at it, even the maintainer has something better to do.
Do you know whats even more powerful than "eyeballs"? Money.
Won't matter if is closed source, signed, and or obfuscated. =3
Umm... no? It's called OPEN source. Expecting people to cancel their plans to make your free software more secure is pretty audacious. Luckily, many WILL, but the expectation is just foolish.
These alerts are absolutely not being shared publicly before we have a fix for them.
Ignore (admittedly low-effort LLM generated) reports at your own peril.
If software "is a trap", even my ever-computing loving wrote first programs on an Apple II in the 80s will only be as you sort of describe invested in by reference (minimal usage).
But no-one will sign up for a "trap" as a career, and only those who do will deal with its problems. The first thing that comes to mind is "Johns", "Hotels", and the trappings of the sex trade.
Fact is that Mythos found only one issue in curl and nothing at all in most code bases. It is getting quiet around Mythos, and the AI companies will move on to the next scam.
In most open source projects, Mythos or similar tools have found nothing. The AI people only contact the projects where they find something, because it would be bad for marketing otherwise.