It's going to happen and at some level I'd rather war casualties were measured in robots rather than people.
My concern is the cottage industry of integrating guns with half baked AI at the lowest cost. And probably vibe coded too.
The companies don't care - a sale is a sale. MoD maybe doesn't care - 90% accuracy and less human casualties on their own side are a win. Governments want to save money and by the time they find out the robots go rogue, it will be too late to do anything about it.
The problem is always the same. It's not just MoD (is it MoW now?) that will have access to this.
YoloV8 + optical flow works fine on an esp32. You want to give a drone rough coordinates for a refinery and hit something in it, like a storage tank? That'll work. This means, give it 5 years, relatively small groups will have access to it. This cannot be stopped.
The only real answer is to work to have groups that you can trust to have access to this first.
Sadly, building an AI that analyses camera imagery and aims at humans, from scratch, is these days almost an intern project. It's not really something you can control or ban, the way you can control, dunno, uranium enrichment.
Integrating it with a robot and sticking a gun on it, thankfully, requires a bit more know-how.
3) they can see if any countermeasures would be effective
4) they can figure out what to look for and find those weapons before they're fired
cfr. nuclear deterrence, right. There is "nothing" the US can do about other nations enriching uranium and making bombs, other than bombing those countries. The US can't change the laws of physics, follow the right formula and it'll work. However the US can figure out exactly what to look for to either prevent it from happening through intervention or at the very least get some warning before it's used ...
And then it will be just another war crime committed daily conflicts, and nothing will happen because there is no world police ?
Ask Ukrainians, Lebanese, Gazaoui, Somalilanders, or even Iranians for that matters - that may not make a big difference to today...
What I would love to see is a local government suing an arms producer for the efficacy of their weapons. (Or even funnier, the owner of a home destroyed by a drone, suiving the GPS company.)
We all know that the only things people in suits are really afraid of, more than hell, is a bad Q4 report and an expensive lawsuit.
My concern is the cottage industry of integrating guns with half baked AI at the lowest cost. And probably vibe coded too.
The companies don't care - a sale is a sale. MoD maybe doesn't care - 90% accuracy and less human casualties on their own side are a win. Governments want to save money and by the time they find out the robots go rogue, it will be too late to do anything about it.