Why would anybody do that you would simply get terrible results compared to dozens of other more capable models. It's for converting to text not answering questions. Just seems like you need some sort of weird angle to bring out an anti AI stance
But anyways just a side note one way to help reduce these errors is if you pass in both the original image and the OCR'd text to the models that make the decisions