LaNMaSteR53.blog

Recently, I have been listening to Kevin Johnson’s (SecureIdeas) SANS SEC542 mp3s on my trips back and forth from the office just to refresh some of the basics of web app pentesting. Day 2 includes a block of instruction where Kevin talks about using advanced search engine queries to discover subdomains, replicating a dns zone transfer when zone transfers are disabled on the dns server. Basically, the technique involves making search engine requests which restrict the url and site to the target domain. Then, based on the results of the search, excluding the subdomains that are returned. Repeat until the search engine returns 0 results. The final search query excludes all of the public facing subdomains that the search engine is aware of. Conduct a dns look-up of each of the identified subdomains, and you’ve got yourself a dns zone transfer of all the subdomains with public facing web servers.

Kevin discusses a couple of tools that automate this process. Some of the tools he discusses use paid-for API keys (therefore, not openly available) while others do a lot of different things, but are not designed for this specifically. Bottom line, he doesn’t provide a tool which does this. Thus, the idea for GXFR was born.

GXFR shows its strength in requiring only a few requests and having the ability to be tuned to avoid shunning. I was able to resolve 33 subdomains for SANS.org (at the time) with only 8 requests and avoid shunning by employing a 5 second delay (default) between requests. GXFR also has the ability to issue requests via http or https based upon whether the ‘–no-encrypt’ option is used. If all you want to do is discover subdomains, but aren’t necessarily interested in the ip information, then exclude the ‘–dns-lookup’ option. If you want to be extra careful, or you are already shunned, you can use the ‘–proxy’ option and feed the script a list of open http/socks proxies or an ip and port of a single proxy. When using the list, the script will “proxy spray” all of your search engine queries by sending each request through a proxy randomly selected from the list. The script does some proxy validation at run-time, letting you know if the proxy is valid, and exempting it from further use if it is not. This feature provides the user will all kinds of flexibility. i.e. send requests through netcat/ssh pivots or tor, monitor requests via interception proxy, use script from behind transparent proxy, etc. The latest version of the script comes with the ability to set a custom user agent string. Your welcome bitform! :D

There are a couple of issues. The “search engine of choice” enforces a maximum words per search limit of 32 words and a maximum search string length of 2074 characters, so GXFR is limited to identifying ~32 sub domains or less depending in which threshold you hit first. It sucks, I know. I’m currently looking for a way around these limitations without using the API. However, in the mean time, the tool still does the job of automating the task for you, as the word and character limits would still apply if the technique was done manually.

I have shared GXFR with Kevin and you can expect to see it in the SEC542 curriculum in the near future and upcoming releases of SamuraiWTF. It may also pop up in the web sections of Ed Skoudis‘ SEC504 and SEC560. As always, if we don’t share, we all fail, so please let me know if you identify any bugs while using the tool. Thanks, and enjoy!

Sample output in verbose mode: (3 proxies listed in the file ‘proxies’, 2 of which were invalid)

root@bt:~# ./gxfr.py *domain-omit* --proxy proxies --dns-lookup -v
[-] domain: *domain-omit*
[-] querying search engine, please wait...
[+] sending query to 196.201.211.145:80
[!] subdomain found: www.
[!] subdomain found: educations.
[!] subdomain found: promotions.
[!] subdomain found: wanfest.
[+] sleeping to avoid lock-out...
[+] sending query to 222.215.230.56:80
[!] 222.215.230.56:80 failed: <urlopen error Tunnel connection failed: 403 Forbidden>
[+] sending query to 127.0.0.1:8080
[!] subdomain found: secure.
[!] subdomain found: images10.
[!] subdomain found: espanol.
[+] sleeping to avoid lock-out...
[+] sending query to 202.171.253.70:80
[!] 202.171.253.70:80 failed: <urlopen error Tunnel connection failed: 404 Not Found>
[+] sending query to 196.201.211.145:80
[!] subdomain found: twmts.
[!] subdomain found: e.
[!] subdomain found: e3wwwtest.
[!] subdomain found: m.
[!] subdomain found: biz.
[!] subdomain found: mts.
[!] subdomain found: blog.
[!] subdomain found: partner.
[!] subdomain found: www.biz.
[+] sleeping to avoid lock-out...
[+] sending query to 127.0.0.1:8080
[!] subdomain found: e4wwwtest.
[!] subdomain found: e4ssltest.
[!] subdomain found: e3ssltest.
[!] subdomain found: images17.
[+] sleeping to avoid lock-out...
[+] sending query to 196.201.211.145:80
[-] all available subdomains found...
[-] successful queries made: 5
[+] final query string: https://encrypted.google.com/search?num=100&q=inurl%3A*domain-omit*+site%3A*domain-
omit*+-site:www.*domain-omit*+-site:educations.*domain-omit*+-site:promotions.*domain-
omit*+-site:wanfest.*domain-omit*+-site:secure.*domain-omit*+-site:images10.*domain-omit*+-site:espanol.*domain-
omit*+-site:twmts.*domain-omit*+-site:e.*domain-omit*+-site:e3wwwtest.*domain-omit*+-site:m.*domain-
omit*+-site:biz.*domain-omit*+-site:mts.*domain-omit*+-site:blog.*domain-omit*+-site:partner.*domain-
omit*+-site:www.biz.*domain-omit*+-site:e4wwwtest.*domain-omit*+-site:e4ssltest.*domain-
omit*+-site:e3ssltest.*domain-omit*+-site:images17.*domain-omit*

[subdomains] - 20
www.*domain-omit*
educations.*domain-omit*
promotions.*domain-omit*
wanfest.*domain-omit*
secure.*domain-omit*
images10.*domain-omit*
espanol.*domain-omit*
twmts.*domain-omit*
e.*domain-omit*
e3wwwtest.*domain-omit*
m.*domain-omit*
biz.*domain-omit*
mts.*domain-omit*
blog.*domain-omit*
partner.*domain-omit*
www.biz.*domain-omit*
e4wwwtest.*domain-omit*
e4ssltest.*domain-omit*
e3ssltest.*domain-omit*
images17.*domain-omit*

[-] querying dns, please wait...
[+] querying dns for www.*domain-omit*...
[+] querying dns for educations.*domain-omit*...
[+] querying dns for promotions.*domain-omit*...
[+] querying dns for wanfest.*domain-omit*...
[+] querying dns for secure.*domain-omit*...
[+] querying dns for images10.*domain-omit*...
[+] querying dns for espanol.*domain-omit*...
[+] querying dns for twmts.*domain-omit*...
[+] querying dns for e.*domain-omit*...
[+] querying dns for e3wwwtest.*domain-omit*...
[+] querying dns for m.*domain-omit*...
[+] querying dns for biz.*domain-omit*...
[+] querying dns for mts.*domain-omit*...
[+] querying dns for blog.*domain-omit*...
[+] querying dns for partner.*domain-omit*...
[+] querying dns for www.biz.*domain-omit*...
[+] querying dns for e4wwwtest.*domain-omit*...
[+] querying dns for e4ssltest.*domain-omit*...
[+] querying dns for e3ssltest.*domain-omit*...
[+] querying dns for images17.*domain-omit*...

[ip]            [subdomain]
216.52.208.185  espanol.*domain-omit*
216.52.208.188  secure.*domain-omit*
216.52.208.185  www.*domain-omit*
12.130.131.124  e.*domain-omit*
210.14.67.182   mts.*domain-omit*
205.186.163.206 blog.*domain-omit*
216.52.208.162  e3ssltest.*domain-omit*
204.14.213.161  e4wwwtest.*domain-omit*
208.44.23.113   images10.*domain-omit*
208.44.23.106   images10.*domain-omit*
204.14.213.162  e4ssltest.*domain-omit*
69.93.50.155    wanfest.*domain-omit*
216.52.208.154  biz.*domain-omit*
204.14.213.154  biz.*domain-omit*
202.167.248.98  twmts.*domain-omit*
208.44.23.130   promotions.*domain-omit*
208.44.23.104   promotions.*domain-omit*
208.44.23.123   images17.*domain-omit*
208.44.23.104   images17.*domain-omit*
216.52.208.154  www.biz.*domain-omit*
216.52.208.161  e3wwwtest.*domain-omit*
184.84.247.59   educations.*domain-omit*
184.84.247.26   educations.*domain-omit*
204.14.213.151  m.*domain-omit*
75.140.128.155  partner.*domain-omit*