Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The User-Agent was not random; that's how I was able to block them this time.


But they were far from honest, if, as you say, crawlers impersonate real users and their browsers.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: