There have been a bunch. Did any auditor lose a license, credibility, or even a night's sleep? Even accountants aren't held to their standards, and they are supposed to guard the holiest of holiest: shareholder money.
It does seem to be a word game, because "it's not stopping any bullets" either isn't honest (it does stop bullets from hitting you when the enemy doesn't know where to shoot) or it's limited, just like obscurity is ("it may stop a few bullets, but it won't stop all, and there will be other weapons it can't stop either"). I think public key exchange is considered security, but it still requires to obscure your private keys.
Perhaps a better word would be resistance (to intrusion), which is a dimension orthogonal to visibility.
There is no need for age verification: ban all mobile devices and access to anything vaguely "social" (and possibly porn) for everyone under 18. Place heavy fines. But it requires more guts than governments have. They get lobbied, coerced by corruptible friends, and threatened, and we end up with the worst of all outcomes: addicted children without attention and lack of privacy.
In a sense, but it is a bit more devious. It basically invalidates all past fMRI studies. Not that anyone should have taken those seriously, but it looks like another nail in the coffin. fMRI analysis is (was?) basically: squeeze each brain scan into a standard box, then average the BOLD responses (that's roughly oxygen usage between 3s and 9s after activity). This abstract says that --at least in some cases-- those averages are wrong. Not just hiding information through aggregation, but flat-out lying.
Just from reading the link, I do see an objection: they studied repetitions, which are known to be different from the initial response, so this may not be the fMRI's eulogy.
The averages a "standard" fMRI analysis produces highlights brain areas which may not even have been involved in the majority of subjects, because the pattern is so wide spread. That is in contrast with your usual average or median over e.g. height. It's a bit like averaging squares on a chess board and concluding that the opening is played in the middle two columns.
This may be objectively scored, but it is not an indication of anyone's coding capabilities. This test measures which model almost accidentally came up with the best strategy (against other bots). This is not representative of coding. You would need to test 100 or more of such puzzles, widely spread across the puzzle spectrum, to get an idea which model is best at finding strategies involving an English dictionary.
I don't think that is entirely fair.. I don't see them stating anywhere they are measuring coding capabilities... "Using complex games to probe real intelligence."
And this seems very much in line with the methodology in ARC-AGI-3.
The results here, in the OP article and in https://www.designarena.ai all tell a similar story: Kimi K2.6 is up and in the SOTA mix.
The task was writing a "bot" to play the game. The title is "Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge." How does that not imply measuring coding capabilities?
> You would need to test 100 or more of such puzzles, widely spread across the puzzle spectrum
Would you? I am not very knowledgable on LLMs, but my understanding was that each query was essentially a stateless inference with previous input/output as context. In such a case, a single puzzle, yielding hundreds of queries, is essentially hundreds of paths dependent but individual tests?
If you are referring to the parent post, yes, hard to draw conclusions from such a small sample size.
For our testing, we use hundreds of different environments across disciplines, and it seems to line up with subjective experience better than other benchmarks. We test coding, agentic coding, and non-coding reasoning in the environments.
> dismissing the Manhattan Project as hopelessly stalled in 1944
Then again, there are enough examples of failed projects. Why should this be comparable to the Manhattan project? In 1944, it was only two years underway, whereas Shor's algorithm is over 30. Tons of articles have been published on quantum computing, while the A bomb was kept as secret as possible, making learning from other countries, sometimes even from colleagues, impossible. In 1942, an atomic explosion was still hypothetical, whereas quantum computing had its first commercial service 7 years ago. Etc.
So, while in principle lack of progress doesn't guarantee failure, a comparison to the Manhattan Project is stylistic bullshit.
> Then again, there are enough examples of failed projects. Why should this be comparable to the Manhattan project? In 1944, it was only two years underway, whereas Shor's algorithm is over 30.
1944 is a bit arbitrary. Szilard for one was thinking about it earlier:
> […] He conceived the nuclear chain reaction in 1933, and patented the idea in 1936. In late 1939 he wrote the letter for Albert Einstein's signature that resulted in the Manhattan Project that built the atomic bomb….
How long was humanity thinking about flying before the Wright brothers and 1903? We had Babbage's analytical engine (and Lovelace) in 1837, with Zuse's Z2 and the British bombes both in 1940; Zuse's Z3 in 1941.
The main point is that just as you can't ask for tiny nuclear explosion because nuclear physics just doesn't work that way, you also can't ask for factoring of 21 with Shor's algorithm. Quantum computing just doesn't work that way, sorry.
The analogy between nuclear fission and quantum computing doesn’t really work. Fission was a relatively new physical phenomenon the Manhattan Project scientists were studying to turn it into a weapon of mass destruction on a scale that too had no precedent except in natural disasters. Quantum computing is a new technology that is supposed to make already effectively computable problems computable faster; it is ideally supposed to provide an increase in capacity, not capability. It should definitely be able to make tiny computations work before going for the bigger problems. That’s how all computing works, if it can’t solve simple problems, it’s never going to solve bigger ones. What you’re saying here essentially sounds like “there will be a magical event one day when quantum computing solves the biggest computing problems and we’ll all realize it works.”
I am not particularly invested either which way about the likelihood of quantum computing being a major breakthrough or not but this is seeming like yet one more area of computing research like crypto and LLMs which in recent years is increasingly being flooded by people on a hype train.
>The main point is that just as you can't ask for tiny nuclear explosion because nuclear physics just doesn't work that way
You absolutely can, which is why Fermi did just that as part of the Manhattan project with the Chicago Pile 1, demonstrating the first self sustaining nuclear chain reaction.
No you really can't. Being able to factor 15 but not 21 with Shor's algorithm is normal. I know it sounds absurd, but it really is that way. Because factoring 21 is about 100x times harder than factoring 15.
My point was that the comparison with nuclear explosions is wonky, since we (in the world of that analogy) already have seen a tiny nuclear explosion 15 years ago. And we kept being told that explosions 100 times larger are just around the corner, but explosions 25% larger are way too hard to expect.
I get that there's a lot of R&D going on to make larger quantum computers a thing and that there's been very definite progress, but factoring 21 is just too hard to expect for now. But that also pushes the date where pre-quantum cryptography is broken further into the future. If we still struggle to factor one of the smaller 5 bit numbers, factoring the 128 bit numbers necessary to break elliptic curve cryptography seems quite far away.
reply