Thursday, November 29, 2012

Searching Browser Client-Side Storage

Is there anything you don't know? Google it! Bing it! Ask Jeeves! We sometimes forget how ubiquitous internet search is in our lives - Google in particular. We use Google Maps to find our way around, we use Google's search engine to find a good restaurant, we use Google for almost everything. That's why Google is great for finding information about a wayward spouse.

Web Browser Client-Side Storage

A lot of what internet browsers retain for data is kept in "client-side" storage. Most browsers retain a SQLite database in the user's directory. From our previous discussions we know a few tricks to get information from these SQLite sources already. Grab a SQL browser and we'll dive into the rabbit hole of what's on a hard drive.

File Location - Where's it at?

On Win7:  C:\Users\[UserName]\AppData\Roaming\Mozilla\Firefox\Profiles\[Profile]
On WinXP:  C:\Documents and Settings\[UserName]\Local Settings\Application Data\Mozilla\Firefox\Profiles\[ProfileName]

On Win7:  C:\Users\[UserName]\AppData\Local\Google\Chrome\User Data\[Profile]
On WinXP:  C:\Documents and Settings\[UserName]\Local Settings\Application Data\Google\Chrome\User Data\[Profile]

On Win7:  C:\Users\[UserName]\AppData\Local\RockMelt\User Data\[Profile]

Internet Explorer (IE):
This is a different kind of beast. We'll discuss this later but it is important to note they do not have a SQLite db structure.
On Win7:  C:\Users\[UserName]\AppData\Local\Microsoft\Windows\Temporary Internet Files
On WinXP:  C:\Documents and Settings\[UserName]\Local Settings\Temporary Internet Files

Types of Storage

The following table is sourced from a Chrome Browser extension called Click&Clean. If their product is anywhere near as good as their chart on client-side storage - it should be excellent!

Below you can see Cookies, Local Storage, Web Databases (SQL), IndexedDB, File System, Application Cache, Flash Cookies, and Silverlight Cookies...

From Click&Clean

I know what you're saying - "But everyone blocks cookies!" True. Everyone blocks 3rd Party Cookies. These are the cookies advertisers put on your system to track your behavior. What people usually don't block are 1st Party Cookies. These are cookies that are directly linked to the domain you choose to visit with your browser. They remember what's in your shopping cart, your login info (if you authorize autofill), your browser session state, etc. These cookies are delicious.

Local Storage,Web Databases (SQL)
The real big deal is in the local storage databases on your computer. These tables track downloads, search terms, archived history of URLs visited and more. See below for the schema and descriptions of the data present.

Application Cache
Nirsoft has a cache reader for you! Chrome, Firefox, Internet Explorer, Opera included. Even (gasp) Safari.

The cache is a set of already downloaded images and media from webpages that is stored on your hard drive. The browser uses these cached elements to speed the loading of webpages. The cache readers will recover images and the URLs that reference them. You will see a lot of elements like the graphic files for navigation buttons for a site, images for headers and/or other elements of a web page. If you are lucky... You may be able to retrieve images from online dating sites, Facebook images, and thumbnails of other incriminating evidence.

I strongly recommend reviewing the cache of every browser on the computer and recovering/saving files you find. The cache is fluid and can be flushed at any time or the relevant data may be overwritten by newer cached data. 

Flash Cookies & Silverlight Cookies
I haven't found anything worth noting in Flash or Silverlight Cookies. There may be data present that is interesting to an investigator but I haven't found it yet.

Local Storage, Web Databases (SQL)

Data Schema and Relevant Files
Here's a list of databases in the User's directory and a list of the data files contained within. There are many SQLite databases as well as flat files and cache files. There may be different tables present depending on the browser (Chrome, Firefox) and the browser version.

For a complete list of data files I've encountered, go here or click on the database you want to investigate below.
  • Archived History
    • Meta
    • URLs
    • Visits
    • Visit Source
    • Keyword Search Terms
  • Cookies
    • Meta
    • Cookies
  • Extension Cookies
    • Meta
    • Cookies
  • Favicons
    • Meta
    • Favicons
    • Icon Mapping
  • History
    • Meta
    • Downloads
    • Presentation
    • URLs
    • Visits
    • Visit Source
    • Keyword Search Terms
    • Segments
    • Segment Usage
  • History Index YYYY-MM
    • Meta
    • Pages
    • Pages Content
    • Pages Segments
    • Pages SegDir
    • Info
  • Login Data
    • Meta
    • Logins
  • Shortcuts
    • Omni Box Shortcuts
  • Top Sites
    •  Meta
    • Thumbnails
  • Web Data
    • Meta
    • Keywords
    • Autofill Profile
    • Autofill Profile Emails
    • Autofill Profile Phones
    • Autofill Profile Trash
    • Credit Cards
    • Web Intents
    • Keywords Backup
    • Logins
    • IE7 Logins
The following are flat files (non-database files) in the User's directory. These open with Notepad and display readable contents. 
  • Bookmarks
  • Current Session
  • Current Tabs
  • Last Session
  • Last Tabs
  • Preferences
  • Visited Links

So.... Where do I start?

I've presented a lot of information. So naturally you want to know where's the best information for web behaviors found.
  • Look into the History Index YYYY-MM, History, Archived History and Web Data databases first.
  • Get a cache reader and review what is in the cache for each browser on the computer.
Analyzing this data should help you confirm or deny your suspicions.


What's in a URL?

You are very likely to recover URLs. Sometimes these are neat and clean to read but a lot of times the URL is a jumble of variables and escaped characters.

Google Search URL
Here are some common variables you will find in a Google search string:
  • q = search term
  • as_sitesearch = searches specified domain only
  • sort = sorting parameter for results
The authoritative source for variables would be Google. They've posted a helpful, lengthy page on search parameters here.

URL Encoding - Escaped Characters
URLs often have characters in them (:, /, ., etc) which would cause problems for other parsing engines like Java or Flash. The characters are often replaced by a '%' and the two digit hexadecimal code for the character. For example ':' = %3A and '?' = %3F.

Here's a list of characters that are often replaced in a URL with a more script-friendly value. I generally use the substitute function in Excel to replace the hex value with the ASCII character.


TL;DR - Summary

1) Find the local storage on your machine
2) Open a SQLite files with a database browser
3) Save file as HTML, close the file, re-open with a spreadsheet program (Excel)
4) Repeat until you've opened all files and saved them all into one workbook
5) Utilize the pivot table function to investigate the data
6) Have a beer

V/r - DNS