"I need you to turn your key and enable the missile silo's MCP server, sir".
~ the opening scene from a reboot of War Games, probably.
A few years ago there was consternation over the US's missile launch system using 8" floppy disks, that it was needless archaic and had never been updated. Can't say that if the launch is mediated by the latest hotness LLM.
Reducing buffer size puts back pressure on the whole system, which can be valuable to manage load (but often throttles faster stages and that throttling makes people uncomfortable). A meaningful metric is how much of the buffer is used at any given time and the throughout. If the buffer is backed up, that says there's a bottle neck on the consumption side of the buffer and more bandwidth is needed there. For whatever reason, adjusting buffer sizes is the more common action taken. A buffer provides throughput management but it also provides info/metrics about the operation of the system.
I remember a book I read as a pre-teen, 40 or so years ago, about a kid who wanted to be "perfect". A wear a tree of broccoli on a string around your neck to learn how to overcome embarrassment. A perfect person never makes mistakes, and the best way to not make mistakes is to not do anything. Similar "requirements" of perfection and their expression are presented. The kid eventually finds himself in an empty room, by himself, doing nothing, wearing broccoli. Perfection was achieved, but at the cost of an extremely boring life.
I'm trusting this is a throwaway example and that you used a real IPv6 address literal in this test, without the "IPv6" and with only colons and no dots (unless you mean to use v4 mapped address with dots)? Because this IPv6 literal is so malformed that I'm hardly expecting it to do something sane and changing that to "@2001" is nasal-demons quality undefined behavior. I tried with this exact literal and it let me send it but then there was a tiny red pop-up at the top of the gmail interface that said "could not be delivered, check your network connection" (which is odd; the same kind of pop-up that appears in gray when you legitimately are not connected to the internet) and it ended up in my drafts with the To: field empty.
I just tried to send a message to a "test@[" my current IPv6 address "]", and gmail told me
Error
The address "test@[«redacted»]" in the "To" field was not recognized.
Please make sure that all addresses are properly formed.
This address doesn't have an MDA listening on it, but it didn't accept it enough to give me a non-delivery notification, it didn't even let me send it. gmail did accept an IPv4 address literal in brackets, although it hasn't given me back a non-delivery notification. What it stuffed into my Sent folder for this message has the square brackets stripped and the IPv4 address appears right after the @.
> There are very principled reasons why LLMs do not know how many letters are in words, and it says nothing about their facility for understanding meaning. … Tokens are the most basic input unit of an LLM. But tokens don't generally correspond to words or letters, rather sub-word sequences. So Strawberry might be broken up into two tokens 'straw' and 'berry'.
This sounds like a description of a child who has not learned to read yet. You ask a child who is not aware of the alphabet and of "words" how many r's are in strawberry you'd get a non-sense answer too. So what you're really pointing out is that the LLMs have not been trained on "the english language" and how words are constructed and what they are composed of. That they operate by tokens that don't correspond to words or letters is irrelevant as an answer to why they can't count the letters in a word. It's not that I know how many r's are in strawberry because of how I'm understanding the word "strawberry", I know how many r's are in strawberry because I know how to spell strawberry. The LLM needs to be trained on this the same way someone who is learning to read would be trained on it. No one should be surprised that an LLM can't "read" in the same way no one should be surprised that a child can't "read".
>That they operate by tokens that don't correspond to words or letters is irrelevant as an answer to why they can't count the letters in a word.
This interpretation takes things too far away from how LLMs are constituted and so misses important explanatory power. The issue of counting letters in a word isn't about an ability to spell, it's about the nature of one's perception. We perceive words as sequences of individual letters. LLMs do not. I can ask you to tell me how many r's are in some nonsense word sequence and you're fully capable of doing that. LLMs do not see sequences of letters so they are intrinsically at a disadvantage for this kind of question. But this says nothing about its capacity for intelligence anymore than not naturally being able to distinguish frequencies of photons hitting your retina has anything to say about human intelligence.
> But this says nothing about its capacity for intelligence anymore than not naturally being able to distinguish frequencies of photons hitting your retina has anything to say about human intelligence.
I disagree with this pretty strongly, because I don't think you're correct that I don't have the ability to distinguish frequencies of photons hitting my retina. We have a lot of tools that can determine the frequency of light and I can use those on any source of light that I wish to measure that may hit my retinas.
If you ask an LLM how many Rs are in strawberry, it wouldn't think like this. It would confidently state that there are two Rs. Even though it "knows" that it can write a python script to count the number of Rs in strawberry, it doesn't do that. Why not? Is it maybe because it isn't intelligent? Yeah, you can prompt an LLM to write a script to count the number of Rs in strawberry, but that's a use of your intelligence, not the LLM's.
>We have a lot of tools that can determine the frequency of light and I can use those on any source of light that I wish to measure that may hit my retinas.
Yes, which is why I said naturally distinguish. Have you asked a frontier model how many r's are in strawberry recently? They get it right now. Either through RHLF to ensure they spell out the word letter by letter or some other means. Humans and LLMs both use tools or alternative means to overcome perceptual limitations. I don't see an in principle difference here.
This "common sense" you refer to, is it the same common sense Babbage was subject to?
"On two occasions, I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able to rightly apprehend the kind of confusion of ideas that could provoke such a question."
If you freely follow a recipe telling you to put glue on your food, I also don't trust you cooking anything and I definitively don't trust you coming up with your own recipes.
> it turned out that predicting elections is still very hard
So maybe we shouldn't be doing it. The value of predicting an election in the large out in public seems kind of dubious, and it's more like gambling than actually being useful. A candidate only runs, and continues running, if they think they can win. All predictions like these do is confuse voters leading up to election day and while they are voting. It keep candidates from making strong cases for their platform, keeps the voters from listening to the candidates' platforms, and encourages team-based partisan politics.
In watching the demo, I didn't come away with the impression that they were removing search results. Yes, they are pushing AI hard, but users can still opt to use Google in the more traditional way. Unless I misunderstood the demo, it's definitely possible to choose.
An interesting aspect of this is the decrease in quality feedback on th organic links. If most people never get down to the actual links there is very little to tell which ones were good or if they had any relevance.
There is also that less incentive to properly maintain the search algorithms to fight SEO and spam.
For all intents and purpose, organic search results have been given a death sentence and are just waiting for the last moment.
Sometimes a given presentation is called biased but it's not that the reporting is biased but that the actual event is or the source material is biased or lacking. "Dog bites man" is not biased against the dog if they don't or can't get the dog's take. And if they do get the dog's take and the dog wastes everyone's time by ranting/barking about cats and they don't print it, it's not the reporting that's biased, the dog squandered its chance to offer its perspective on the topic at hand.
Sometimes there is no "other side" or the other side offers meaningless contribution. The trend to present oneself as unbiased have often given platforms to voices that are not worthy of having a platform, for whatever reason, and someone needs to make a call, and they should be transparent about it. Are they not giving time because they choose to ignore legitimate and useful information or because giving someone a platform to rehash all the bogus reasons that the moon landing was faked again isn't worth it (to use an extreme example). Moon landing deniers can set up their own web site to push their faked moon landing agenda, they don't need to clutter up everyone else's content with their nonsense in the name of "unbiased reporting".
I'm talking about things like say "rent control" (just an example). NPR's and related programs, the core creators arguably believe in it. If they have a segment on it they'll gloss over any evidence that it's a net negative. They'll present it as a solution, talk up how awesome it is through a 20 minute segment with positive words and excited attitudes. "IF" they ask someone about possible negatives they'll kept it short and surround it with questions and attitudes of "don't trust this person" and "dismiss this counter evidence". Then they'll end with some sob story about the people rent control helps.
It's the same for 100s of other topics on which there is strong arguments to be made that the proposal will have the opposite of the intended outcome.
~ the opening scene from a reboot of War Games, probably.
A few years ago there was consternation over the US's missile launch system using 8" floppy disks, that it was needless archaic and had never been updated. Can't say that if the launch is mediated by the latest hotness LLM.
reply