OSINT: Search Engines

Using Google or BING to search the Internet is not really difficult in general. Nevertheless I would like to mention a few techniques that makes my investigation life a bit easier. There are a few good search operator you should know which will make your investigation more efficient. I will focus my searches on the Google Search Engine. Most of them will also work on BING or other search engines.

Site Operator

If you would like to only search within a specific domain, you can use the site operator function:

site:cnn.com "Olympic Games"

The search result will only show pages hosted under cnn.com.

File Type Operator

If you are only interested in certain file types you can use the file type operator. Below a few examples:

"Digital Forensics" filetype:pdf
"OSINT" filetype:pptx

Bear in mind that media files such as MP3, MP4, etc. are normally not indexed and therefore can not be used as filter options.

Hyphen

You might face a tremendous amount of search hits. Obviously it will take too much time to crawl through them all. A useful search operator to cull down your search result is the “hyphen”. The hyphen operator is useful if you would like to cull down your search result by removing all hits where a certain search term is present. You can also combine more than one hyphen operator:

"Michael Jordan" -basketball
"Michael Jordan" -basketball -nba
InURL Operator for FTP searches

Interesting information does not only lie on regular http/https sites but also on other services such as FTP. Therefore you need to have a process in place to search FTP sites as well. There is a way to only search on FTP sites (if they are indexed by the search engine) using a search operator. For that you need to make use of the hyphen and the InURL operators:

inurl:ftp -inurl:(http|https) "Hacking Manual"

In addition, there are a few search engines out there that claim to have indexed many public FTP servers:

Google Custom Search Engines

Google actually allows you to create your own custom search engine where you can set your default search operators in advance. For example, you can create a custom search that only searches in social network sites or for example only on your local news pages. Here is how to create your custom search:
1. Go to: http://www.google.com/cse
2. Type in the websites you would like to include in your custom search
3. Give your search a name
4. Save your custom settings
=> You will now get an URL to your custom search engine.

Search Monitoring

Checking your prefered search engine every week for new results can be really annoying and time consuming. There are a few “search monitoring” services out there, which do the job for you:

Web Archives

If you would like to go way back in time and you are more interested in the past than in the present, then you should take a look at web archives. There are some handy services for you where you can check how a specific website had looked in the past. My favourite resource for that is http://archive.org which provides a very easy user interface. Other services are:

Image Searching

If you are rather interested in images than in text searches, you should check out one of the following image search services:

TOR Search Engines

It is also possible to search within the darknet even without being connected to the tor browser. Most of the search engines are using tor proxies that basically works as a bridge between your regular internet connection and the darknet.

I don’t have any information how reliable search results are within the darknet as this part of the internet is very dynamic and is normally not indexed at all.

Social Network Searches

I cover social media seraches in particuliar in a few other posts on my blog. Nevertheless I would like to mention two websites where you can quickly search through many social media sites at once:

Internet Of Things

Shodan is the search engine for the Internet Of Things. It will let you search for devices connected over the internet.

Resources
  • A lot of useful information can be found in the book “Open Source Intelligence Techniques” by “Michael Bazzell“.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> 

I am real - CAPTCHA verification * Time limit is exhausted. Please reload CAPTCHA.

*