Here's a good blog post on the subject:

http://www.ansible.com/blog/ansible-performance-tuning

If you want to use paramiko (like Fabric does), you can -c paramiko too. This mode came first, but -c ssh offers more features and can be just as quick.