Published on December 22nd, 2015 | by MaxiSoler0
Crawlic Beta – Web Recon Tool
Crawlic is a Web Recon Tool (find temporary files, parse robots.txt, search folders, google dorks and search domains hosted on same server)
Automatic GIT/SVN clone (using dvcs-ripper) if .git or .svn folder if found.
Licensed under: “THE BEER-WARE LICENSE” (Revision 42): Ganapati wrote this file. As long as you retain this notice you can do whatever you want with this stuff. If we meet some day, and you think this stuff is worth it, you can buy me a beer in return.
./crawlic.py -u http://site.tld/ -t rtf
-t : techniques to use for scanning (default rtfgd):
r : robots.txt t : temporary files (~, .bak, etc) f : folders g : google dorks d : reverse dns search (via bing)
- Python 2.x
- git/svn ripper needs LWP.pm library (Original dvcs ripper: https://github.com/kost/dvcs-ripper)