AI, the sidekick for your OSINT Knowledge base

Generative AI is here to stay in our current society. The past year has been a race between several major players to convince the public of the utility and necessity of models such as GPT-4 and the ChatGPT built on it.

AI is also being experimented with a lot within OSINT as well in order to make tasks easier for investigators. The results of this are variable and tend to improve (with exceptions) as the models process more data and are trained further.

Building searches

For example, it is possible to build search queries with DorkGPT.com, so-called Google Dorks. Don't know how to build a search around CSV files behind a website? Enter this and DorkGPT generates (with ChatGPT in the background) for the query: site:website.com filetype:csv.

Of course, it is important to keep thinking like an investigator; the model learns from the input and the input is also reviewed by (third) parties to make the model better. Uploading evidence to an online platform is therefore very unwise, as is having a report of findings written with personal details, for example.

AI as sidekick

Nevertheless, it is possible to use AI precisely as a sidekick. Think of Watson as the backup to Sherlock Holmes or K.I.T.T., the helpful car from the TV series Knight Rider. Below, we describe an example of how you can use AI to further build your OSINT knowledge base.

Many investigators recognise the problem that many links are shared within the OSINT world, but that even just adding all those links as bookmarks in the browser takes a lot of time. For this, we can use our AI sidekick. In this example, we will use the free version of ChatGPT, version 3.5. If you have the paid version 4, the advantage is access to the internet via Bing, which enables searching for input or having the data extracted by uploading a file.

How does it work?

In version 3.5, we can supply ChatGPT with data from the internet via a Chromium or Firefox Extension called WebChatGPT.

After this, we can build a prompt to ask for a list of certain sites. For example: please create a list of 10 VPN providers with URLS to their website.

Should you already have a site with several useful links for your OSINT knowledge base, use a Chromium or Firefox extension like Link Gopher or Instant Data Scraper to extract the links data from the web page and then paste it as input into ChatGPT.

After pasting the links into the prompt window, apply the step below. Should there be a lot of links without being categorised, you can also have ChatGPT categorise this.

To import this as bookmarks, we need to turn it into an HTML file. This can be done manually, but why should we? After all, we have our sidekick for this. So the next prompt we build reads as follows: can you provide the html code for that list of 10 vpn providers so I can save it as an HTML file and then import it into Chrome?

This results in the image HTML code Top 10 VPN providers.

I know from experience that this is not quite the correct format yet, so I ask that ChatGPT take out all list functions. The prompt for this is: could you please remove all li tags from the code. This results in the image HTML code Top 10 VPN providers minus li tags.

You can then save this code in Notepad as bookmarks.html.

The final step is to go to the bookmarks manager in the browser, choose import and then choose the file bookmarks.html. This imports the links and makes them directly accessible to visit in the browser.

This is one example of how AI can be used as a sidekick to support an investigator. Of course, there is much more to the above example, but we hope this article shows a way to safely apply AI within OSINT investigations.

Henri Beek is OSINT Trainer, Investigator and Certified Maltego Trainer at DataExpert and is happy to share his knowledge and experience in one of the OSINT training courses. View the training offer here.