Introduction to Character Encoding

Understanding how Character Encoding works is an essential part of understanding digital evidence. It is part of the common core of skills and knowledge.

A character set is a collection of letters and symbols used in a writing system. For example, the ASCII character set covers letters and symbols for English text, ISO-8859-6 covers letters and symbols needed for many languages based on the Arabic script, and the Unicode character set contains characters for most of the living languages and scripts in the world.

Characters in a character set are stored as one or more bytes. Each byte or sequence of bytes represents a given character. A character encoding is the key that maps a particular byte or sequence of bytes to particular characters that the font renders as text.

There are many different character encodings. If the wrong encoding is applied to a sequence of bytes, the result will be unintelligible text.

ASCII

The American Standard Code for Information Interchange, or ASCII code, was created in 1963 by the American Standards Association Committee. This code was developed from the reorder and expansion of a set of symbols and characters already used in telegraphy at that time by the Bell Company.

At first, it only included capital letters and numbers, however, in 1967 lowercase letters and some control characters were added forming what is known as US-ASCII. This encoding used the characters 0 through to 127.

7-bit ASCII is sufficient for encoding characters, number and punctuation used in English, but is insufficient for other languages.

Extended ASCII

Extended ASCII uses the full 8-bit character encoding and adds a further 128 characters for non-English characters and symbols.

 

Hex viewer showing extended ASCII character encoding

Unicode

Fundamentally, computers just deal with numbers. They store letters and other characters by assigning a number for each one. Before Unicode was invented, there were hundreds of different encoding systems for assigning these numbers. No single encoding could contain enough characters: for example, Europe alone requires several different encodings to cover all its languages. Even for a single language like English no single encoding was adequate for all the letters, punctuation, and technical symbols in common use.

These encoding systems also conflict with one another. That is, two encodings can use the same number for two different characters, or use different numbers for the same character. Any given computer (especially servers) needs to support many different encodings; yet whenever data is passed between different encodings or platforms, that data always runs the risk of corruption. Unicode provides a unique number for every character, no matter what the platform, no matter what the program, no matter what the language.

The Unicode Standard is a character coding system designed to support the worldwide interchange, processing, and display of the written texts of the diverse languages and technical disciplines of the modern world. In addition, it supports classical and historical texts of many written languages. Unicode 10.0 adds 8,518 characters, for a total of 136,690 characters.

Unicode can be implemented by different character encodings; the Unicode standard defines UTF-8, UTF-16, and UTF-32 (Unicode Transformation Format).

Codepoint

The number assigned to a character is called a codepoint. An encoding defines how many codepoints there are, and which abstract letters they represent e.g. “Latin Capital Letter A”. Furthermore, an encoding defines how the codepoint can be represented as one or more bytes.

The following image shows the encoding of an uppercase letter A using standard ASCII.

 

Image showing character encoding and the transition from Character A to binary and codepoints

 

UTF-8, UTF-16 and UTF-32

UTF-8 is the most widely used encoding and is variable in length. It is capable of encoding all valid Unicode code points and can use between 1 and 4 bytes for each code point. The first 128 code points require 1 byte and match ASCII.

UTF-16 is also a variable-length and is capable of encoding all valid Unicode code points. Characters are encoded with one or two 16-bit code units. UTF-16 was developed from an earlier fixed-width 16-bit encoding known as UCS-2 (for 2-byte Universal Character Set).

UTF-32 is a fixed length encoding that requires 4 bytes for every Unicode code point.

Browser Data Analysis

It is important to understand character encoding when examining Internet and browser data. Browser applications use a variety of different encoding methods for storing data. For example, some browsers use UTF-16 for storing page titles and the default Windows encoding for storing URL data (e.g. Windows 1252). Windows 1252 is a 1-byte character encoding of the Latin alphabet, used by default in the legacy components of Microsoft Windows in English and some other Western languages.

Selecting a Code Page in NetAnalysis®

An appropriate Code Page can be selected when creating a New Case in NetAnalysis®.

Digital Detective NetAnalysis® new case screen and option to set character encoding

Clicking the button next to the code page shows the following window. This allows the user to select the appropriate code page (if required).

 

Digital Detective NetAnalysis® code page screen to select character encoding

References

Introduction

This version of NetAnalysis® introduces support for two new browsers as well as adding support for the latest release versions of existing browsers which are already supported.

Some notable new features include support for decrypting the logins and passwords from the latest Mozilla based browsers as well as processing Mozilla session and search engine files. We have also added support for Microsoft Edge backups and Apple Safari recently closed tabs, last session files, user notification permissions and search descriptions.

Some improvements to the software include DirectX hardware acceleration support for the data grid which increases performance. We have also added the ability to save data stored in encoded data URLs.

New Browser Support

AOL Desktop Browser v9

AOL Desktop was an Internet suite produced by AOL which contained an integrated web browser. Prior to version 9.8, the browser was based on the Trident layout engine as used by Internet Explorer. From v9.8 onward, Trident was replaced with CEF (Chromium Embedded Framework) to provide users with a more modern browsing experience. Despite AOL Desktop being discontinued in 2018, it is still encountered during investigations.

Blisk Browser v0 – 8

Blisk is a Chromium based web browser which has been designed to be used by web developers. It provides an array of tools for web development and testing across a number of different devices. It contains a pre-installed set of emulation tools for testing phones, tablets, laptop and desktop devices. This makes it a simple task for web developers to test how their code renders across multiple devices, browsers and screen resolutions.

Updated Support for Existing Supported Browsers

NetAnalysis® currently supports a wide variety of desktop and mobile browsers. There have been a number of changes to the currently supported browsers. Here are some of these changes:

Login and Password Decryption

A recent change to the encryption/decryption methodology for Firefox Desktop browsers resulted in the process requiring access to a new file called key4.db; using this file matches the behaviour of some mobile versions of the browser. NetAnalysis® supports the decryption of login information and passwords using both key store files.

New Support for Existing Browsers

To enhance our support for existing web browsers, we have added the following:

Mozilla Session Stores

Mozilla Firefox and many of the Mozilla based browsers store session information relating to the state of a user’s browsing session so that the windows and tabs that were open when the browser was last closed, terminated unexpectedly or a software update applied can be restored.

There are usually multiple versions of a user’s session store file located in the user profile folder with backup copies saved to the sessionstore-backups folder.  Session store files have different file names depending on how the browser uses them during the session restore process:

  • sessionstore,
  • recovery,
  • previous,
  • upgrade.

As well as information on the currently open windows and tabs, a session store file also stores information on recently closed windows and tabs and cookies relating to the saved session. In the more recent versions of Firefox these session store files are now saved in a compressed format.

NetAnalysis® now recovers all versions of Mozilla based session store files.

NetAnalysis® showing Mozilla Session Data

NetAnalysis® showing Mozilla Session Data

Mozilla Search Engine Data

Mozilla Firefox and many of the Mozilla based browsers store their search engine data in a JSON format search file.  This includes the default search engines that come preinstalled with the browser and user installed search engines and search engine add-ons.  The user can then choose to search with one of these alternative search engine rather than the default. In the most recent versions of Firefox the search engine file is now saved in a compressed format.

We have added support for the import of all versions of this file to NetAnalysis®.

Microsoft Edge Backups

Microsoft Edge recently added a feature to create an automatic backup of the user’s ‘favourite’ entries using the Netscape bookmark file format. NetAnalysis® can identify and import these files.

Apple Safari Search Descriptions

Quick Website Search was a feature added to Safari v8.  If a website includes an OpenSearch description document, the site can be identified by the browser as having searchable content.  The first time a user visits such a website, Safari will add it to the Manage Websites panel of Safari’s Search Preferences.  The user can then access content from this website directly from Safari’s Smart Search field thus bypassing their normal search engine. Safari stores this Quick Website Search information in a SearchDescriptions.plist file.

NetAnalysis® now recovers Safari Quick Website Search information.

Apple Safari User Notification Permissions

Safari allows the user to manage website push notifications.  The list of websites that have asked for permission to display alerts can be viewed in Safari’s Notifications Preferences. Each website has an option to allow or deny the push notifications.

NetAnalysis® now recovers this information and details the notification permission setting to the Information panel.

Apple Safari Last Session

All versions of Safari v3+ on both Mac OS X and Windows contain a LastSession.plist file which records the current state of the browser.  Safari can use this file to reopen all the windows and tabs which were open the last time the browser closed or terminated unexpectedly.  The Safari menu item Reopen All Windows from Last Session allows the user to do this manually.

Apple Safari Recently Closed Tabs

Apple Safari v10+ keeps track of recently closed tabs in a RecentlyClosedTabs.plist file.  This allows the user to reopen closed tabs using the Recently Closed Safari menu item.

We have added support for the import of Last Session and Recently Closed Tabs into NetAnalysis®.

New Features

We have added some new features to NetAnalysis®:

Saving Data from Encoded Data URLs

Data URLs are prefixed with the data: scheme and allow content creators to embed small files inline in documents. They are composed of four parts: a prefix (data:), a MIME type indicating the type of data stored, an optional base64 token if the data is non-text, and the data itself:

data:[<mediatype>][;base64],<data>

Right clicking on the data URL allows the user to select Save Data from URL, this will show a Save File window prompting the user to select a location and file name. The decoding engine will automatically identify the correct file extension based on the source data.

NetAnalysis® Saving Base64 Encoded Data URLs

NetAnalysis® Saving Base64 Encoded Data URLs

DirectX Hardware Acceleration Support

In this release of NetAnalysis®, we have added support for DirectX hardware acceleration. This allows us to employ the client machine’s video card (integrated or dedicated) to render the data grid. DirectX acceleration provides us with an incredible speed boost. If the source system is unable to provide the resources for DirectX painting, the application will revert to GDI+ rendering.

Introduction

This version of NetAnalysis® introduces support for a number of new browsers as well as adding support for the latest release versions of existing browsers which are already supported. The major features for this version includes support for the changes to the latest Mozilla Firefox cache and favicons as well as adding support for processing Mozilla based cache with a missing index file. We have also considerably enhanced our support for Sleipnir.

New Browser Support

We have added support for the following browsers:

Cyberfox

Cyberfox is a Mozilla based browser designed by 8pecxstudios™. They claim they take over where Mozilla left off by working to make a fast, stable and reliable 64bit web browser that is accessible to all. It is available for Windows in two processor-specific builds, one optimized for Intel based CPU’s, and one optimized for AMD based CPU’s. It is also available in x86 versions. Cyberfox is also available for 64bit Linux.

Cyberfox ships with many customizable options allowing the user to personalize their web browsing experience. It has advertising features and components removed that collect information. It also has the ability to turn off the automatic loading of images on the web.

IceCat

GNU IceCat, formerly known as GNU IceWeasel, is a free web browser distributed by the GNU Project. It is based on the Mozilla platform and is available for installation of GNU/Linux, Windows, macOS and Android.

IceCat includes additional security features such as the option to block third party zero-length image files resulting in third party cookies, also known as web bugs. The software also provides warnings for URL redirection and has functionality to set a different user agent string for different domains.

Waterfox

Waterfox is an open-source web browser based on Mozilla which is available for 64bit Windows, macOS and Linux systems. It has been designed to take advantage of 64bit system architecture and claims to provide speed improvements over Firefox.

Updated Support for Existing Supported Browsers

All of the mainstream browsers have updated their file formats and added new features. In addition to adding new browser support, we have enhanced the support provided for existing browsers:

Mozilla Firefox Cache v2 Missing Index

In the situation where the Index file is not present in the Mozilla cache v2 folder, we have added support for NetAnalysis® v2.7 to process these orphaned entries.

Sleipnir SE

We have considerably enhanced our support for Sleipnir. With added support for the Sleipnir.sqlite database, NetAnalysis® v2.7 now extracts History, Downloads, Bookmarks, Tab Groups, Tab Information and Tab History. We also extract Favicons, History Thumbnails and Tab Previews. The screen below shows a Tab entry with the Preview image displayed.


 

HstEx v4.7

This release of HstEx® adds the ability to recover a number of new artefacts as well as adding support for three new browsers. We have also made a number of changes to support the updates released by the browsers already supported.

New Features

HstEx® v4.7 now supports the following:

Microsoft Edge Top Sites

Microsoft Edge provides the user with three content options for a new tab page; Top Sites and my feed, Top Sites, or a blank page. Top Sites were added to Microsoft Edge v25 and are initially pre-populated with examples; however, it is relatively easy for the user to modify these sites. HstEx® v4.7 now has the ability to recover individual Top Sites entries.

 

Mozilla Firefox Form History and Bookmarks

We have enhanced the recovery of Mozilla Firefox and Mozilla based browsers by adding support for Form History and Bookmark  entries. HstEx® v4.7 can recover SQLite records from moz_formhistory and moz_bookmarks tables.

 

Introduction

This version of NetAnalysis® introduces support for a number of new browsers as well as adding support for Chromium Simple Cache format used by a number of the mobile browsers. We have also added support for Microsoft Internet Explorer and Edge Recovery Store, Tab Session, Travel Log, Roaming Tab Sessions and the detection of InPrivate browsing.

New Browser Support

We have added support for the following browsers:

Opera Neon

Opera Neon LogoOpera Neon is a new concept browser: “a vision of what browsers could become”. It was first released in January 2017 and is available for Mac and Windows.  The browser is Chromium based but with some additional unique features.  Opera Neon gives the user new ways to interact with web content, including the ability to drag, push and pop the tab icons.

NetAnalysis® will recover the standard Chromium based artefacts as well as the top sites, tab page icons and the gallery snapshots.  The tab page icons and the gallery snapshots are written to the case export folder and loaded into the Viewer window.

Brave

Brave Browser LogoBrave is another new, open-source, multi-platform web browser developed by Brave Software; it is based on the Chromium web browser and its Blink engine. It claims to block website trackers and remove intrusive Internet advertisements. The browser also claims to improve online privacy by sharing less data with advertising customers.

NetAnalysis® will recover the standard Chromium based artefacts.

Updated Support for New Versions of Existing Browsers

All of the mainstream browsers have updated their file formats and added new features. In addition to adding new browser support, we have enhanced the support provided for existing browsers:

Google Chrome/Chromium Based Simple Cache for HTTP

This disk cache is used by default in Google Chrome on Mac OS X, Linux and Android mobile devices.  It can also be enabled on Chrome and most Chromium based browsers running on Windows desktop. It was initially designed as a simple cache back-end to deal with the IO bottlenecks which impaired mobile browsing performance on some platforms.

NetAnalysis® supports processing Google Chrome and Chromium based Simple disk cache and well as exporting and rebuilding web pages.

Firefox and Mozilla based permissions.sqlite

This database holds preferences about which sites are allowed or prohibited to set cookies, to display images, to open popup windows and to initiate extensions installation.

NetAnalysis® can read this information and display the permission settings in the Information panel.

 Mozilla Firefox Permissions in NetAnalysis

Vaivaldi Notes

Vivaldi browser allows the user to save notes while they browse.  A note can be linked to a specific web page and the user can attach full page or selected area screenshots as well as files from their computer.

NetAnalysis® now recovers Vivaldi Notes.  The note content is written to the case export folder and indexed.  Any attachments are written to the case export folder.

Mozilla Firefox v2 Cache

The Disk Cache format v2 for Mozilla Firefox has evolved and changed. NetAnalysis® supports all versions of this disk cache format and allows cache objects to be exported as well as rebuilding web pages.

Microsoft Recovery Store, Tab Sessions, Roaming Tab Sessions and Travel Logs

Microsoft Internet Explorer and Edge browsers keep track of browsing history in two main ways; History and Travel Log. The active tab’s list of back/forward navigations is called the Travel Log. Within Internet Explorer, you can see this list with a click-and-hold on the back or forward arrow. This data can also be used for recovering sessions in the event of the browser crashing, or by starting a new session with tabs from the last session when set as an option by the user. The browsers store this data in recovery store and tab session files.

 

Microsoft Edge v38 Recovery Store

Detection of InPrivate Browsing

InPrivate Browsing LogoIf a user activates InPrivate browsing, the browser continues to write Travel Log data to the Recovery Store and Tab Session files. At the end of the InPrivate session, the browser deletes these files. NetAnalysis® has the ability to genuinely identify InPrivate browsing sessions and will flag them by placing an icon at the start of the URL (as shown below). HstEx® also has the ability to recover deleted InPrivate Recovery Store and Tab Session files.

Some forensic tools claim to recover InPrivate browsing, but in fact are only searching for URLs in the Travel Log stream and have no idea whether they relate to InPrivate browsing or not.

 

Microsoft Internet Explorer - Edge Detected InPrivate Browsing Session

Improved Reporting

Reporting has been completely overhauled to allow reports to be generated on records filtered with a Find Panel active search as well as an active filter.  Previously, reports could be generated on all rows in the grid or on the rows visible when a filter is active.

There are some additional report templates.  A template based on the original NetAnalysis® v1 “Print – Current to PDF” report has been added named “Simple History”.  There is a new template based on the original v1 “Group By Host” named “History By Host” and a new template based on the original v1 “Group by Index Type” named “History By EntryType”.

Improved Cache Exporting and Page Rebuilding

The cache exporting engine has been revisited and considerably improved. We have increased processing speed, as well as enhancing the capability of the process. The following bullet points highlight some of the enhancements we have made.

  • Cache extraction and page rebuilding has been improved to speed up processing and is able to handle much larger volumes of cached page data.
  • Improved content detection.
  • Added support for Brotli decompression.
  • Google Chrome / Chromium Based cache v2 Sparse data entries are now extracted and used in cache export and page rebuilding.  Chrome uses this method to store large cache data in its disk cache.  Internally the cache stores the data as sparse chunks among a set of child cache entries that are linked together from a main parent entry.
  • Processing “srcset” attribute has been added.
  • Processing “data-thumb” attribute has been added.
  • Processing “data-src” attribute has been added.
  • Added support for Chrome Dictionary files during export.

Improved Exporting

Exporting functionality has been improved to include records filtered with a Find Panel active search as well as an active filter. Previously, the exported rows would be dependent upon the active filter or all rows in the grid would be included.

User Interface Improvements

We have made some changes to the user interface to enhance usability:

Save and Load Column Layout

It is now possible to save and reuse grid column layouts. We have provided a number of sample layouts to demonstrate the feature. This is particularly useful if you like to arrange the columns in a certain order, or if you like to remove some of the columns altogether. To save a column layout, select Column » Save Column Layout. To load a column layout select Column » Load Column Layout. There is also an option to save data grouping if you select save with Data Settings when saving the layout.

Right Click Grid Filter By

We have added two new dynamic filters which can be accessed by right clicking a target record. By selecting Filter By, a sub-menu will appear showing the Host Name and Browser Version strings for this record. Clicking either entry will result in a filter being applied relating to the clicked item.

 

NetAnalysis Right Click Filter

Clear All Active Filters and Searches

Following user feedback, we have added a simple, one-click, option to remove all active filters and searches thereby restoring the full record count to the grid. This can be activated by selecting Tools » Show All Records (Shift + F5) or Right Click » Show All Records.

HstEx® v4.6

This release of HstEx® adds the ability to recover a number of new artefacts as well as adding support for two new browsers. We have also made a number of changes to support the modifications introduced by all of the main stream browsers.

New Features

HstEx® v4.6 now supports the following:

Microsoft Internet Explorer/Edge

Microsoft Internet Explorer and Edge browsers keep track of the visits for each tab; these visits are stored in what is known as a Travel Log. The Travel Log allows the user to navigate backwards and forwards through the log of visits. This information is saved into a Tab Session file. HstEx® can recovery individual Travel Log entries for Internet Explorer v8 to 11 and Microsoft Edge v20 to 38. HstEx® can also search for, and recover, Recovery Store, Tab Session and Roaming Tab Session data (including page thumbnails and previews).

Recovery of Data Relating to InPrivate Browsing

InPrivate Browsing LogoWhen Recovery Store, Tab Session and Roaming Tab Session files are targetted for recovery and the resulting data was from an InPrivate browsing session, NetAnalysis® has the ability to identify and flag such sessions.

Recovery of Google Chrome/Chromium Based Simple Cache for HTTP

This disk cache is used by default in Google Chrome on Mac OS X and Linux and also Android mobile devices.  It can also be enabled on Chrome and most Chromium based browsers running on Windows desktop. It was initially designed as a simple back-end to deal with the IO bottlenecks which impaired mobile browsing performance on some platforms. HstEx® can now recover Simple Cache entries.

New Browser Support

We have added support for the following browsers:

Opera Neon

Opera Neon is a new concept browser: “a vision of what browsers could become”. It was first released in January 2017 and is available for Mac and Windows.  The browser is Chromium based but with some additional unique features.  Opera Neon gives the user new ways to interact with web content, including the ability to drag, push and pop the tab icons.

HstEx® can recover the following:

  • History Entries
  • Download Entries
  • Cookie Entries
  • Cache Entries
  • Simple Cache Entries
  • Keyword Search Terms
  • Form History
  • Login Data

Brave

Brave is another new, open-source, multi-platform web browser developed by Brave Software; it is based on the Chromium web browser and its Blink engine. It claims to block website trackers and remove intrusive Internet advertisements. The browser also claims to improve online privacy by sharing less data with advertising customers.

HstEx® can recover the following:

  • History Entries
  • Download Entries
  • Cookie Entries
  • Cache Entries
  • Simple Cache Entries
  • Keyword Search Terms
  • Form History
  • Login Data

Introduction

This release of NetAnalysis® brings support for some new browsers and new artefacts as well as adding support for the modified cache format in Mozilla Firefox. We have also added support for the new versions of the Microsoft Edge download object.

New Browser Support

We have added support for the following browsers:

360 Security Browser

360 Secure/Security Browser (360安全浏览器) is a web browser developed by the Qihu company of Beijing, China. It offers page layout using either the Trident engine, as used in Internet Explorer, or the WebKit engine that was adapted for Google Chrome. It was first released in September 2008.

We have added support for the import of bookmarks which are stored in a format specific to 360 Security Browser. NetAnalysis also now supports history and downloads from the earlier versions (v3-5) as well as all the standard artefacts from v6+. We also support the import of the UnClosed Pages SQLite database which contains information on pages saved by the user when the Browser was shut down.

360 Speed (Extreme) Browser

360 Speed (or 360 Extreme) Browser (360极速浏览器) is another freeware Chromium-based browser by the Qihu 360 Software Company. It offers a cloud synchronisation account and claims protection against phishing.

NetAnalysis now supports the import of all the standard artefacts from 360 Speed Browser including the cross-domain Cookies found in v7.

UC Browser

UC Browser is a mobile browser developed by Chinese mobile Internet company UCWeb. Originally launched in April 2004 as a J2ME-only application, it is available on platforms including Android, iOS, Windows Phone, Symbian, Java ME, and BlackBerry.

With a huge user base in China, India, Indonesia, Pakistan and continued growth in emerging regional markets, UC Browser reached 100 million global users in March 2014. According to StatCounter, UC browser is the second most used smartphone/mobile web browser worldwide, passing Apple Safari in October 2015.

We have added support for the import of all the standard artefacts from UC Browser. NetAnalysis will also import URL shortcuts from the UC Browser Omnibox SQLite database.

Updated Support for New Versions of Existing Browsers

Some of the mainstream browsers have made modifications to their file formats to add new features. NetAnalysis® has been updated to support these new file formats. We have also added support for the following files and databases:

Microsoft Edge v25 – 38 (EdgeHTML v14) Downloads

Microsoft has released new iterations of the download object stored in the iedownload container. We now support these latest versions.

Apple Safari v10

The latest version of Safari updated the Downloads.plist and the History.db database schema. NetAnalysis® v2.5 has been updated to support Apple Safari v10 history and downloads.

Additional Support for Existing Browsers

We have also added support for the following artefacts:

Mozilla Firefox Backup Bookmarks

Mozilla Firefox and many Mozilla Based Browsers backup their bookmark data to JSON format and more recently LZ4 compressed JSON format files. We have added support for the import of these file types into NetAnalysis®.

Opera Session Database

Opera v15-29 stored its tab and session data in a session.db SQLite database. We have now added support to NetAnalysis® for the import of this database.

Mozilla Firefox Cache

In the recent versions of Mozilla Firefox, the cache version 2 format has been updated. We have added support to NetAnalysis® (and HstEx®) for this new structure.

Google Chrome Segment Usage

Google Chrome and many Chromium-based browsers store URL segment and segment usage information in the History SQLite database. The segment usage information contains details on the number of visits per day to a particular segment. A segment is a generic and simplified version of a URL which means similar URLs may be grouped together as a single segment. This usage information allows the browser to calculate the highest ranked segments which can then be used for the most visited view. We have now added support for the import of these tables to NetAnalysis®.

Chromium Form History and Login Data Recovered from HstEx®

We have added a number of new artefacts in HstEx® v4.5. With Chromium-based browsers, you can now recover individual entries from the “logins” table located in the Login Data SQLite database. You can also recover individual entries from the “autofill” table located in the Web Data SQLite database. All of these artefacts can be recovered and loaded into NetAnalysis® for review and analysis.

Torch Browser Accelerated Downloads Recovered from HstEx®

Torch browser stores its downloads in the History SQLite database in a table called “accelerated_downloads”. We have added the ability to recover these entries in HstEx® v4.5 and import them into NetAnalysis® for review.

New Features

We have added some new features to NetAnalysis® to make the software easier to use and to assist with productivity. We have also added some new analytical tools which can be used to drill down into the various artefacts of stored URL data and cookie values.

Check for Software Update

In previous versions of NetAnalysis®, we had a feature to allow the user to check whether a new version of the software was available for download. We have had numerous requests to add this feature back, so from this release, you can check for new versions and get direct access to the latest download. This feature can be accessed from the Help menu by selecting Help » Check for software update.

 

NetAnalysis Check For Software Update

New Decoding/Analysis Options

To enhance the data analysis capabilities built-in to NetAnalysis®, we have added some new timestamp decoding support. In the data examination/analysis window, the user can now select Mac Absolute, HFS+ (Mac OS) and OLE Automation timestamps.

Introduction

This release brings support for Google Chrome’s History Provider Cache and Network Action Predictors, Microsoft’s Internet Explorer and Edge Typed URLs and Bookmarking across the various supported Browsers.

History Provider Cache

The History Provider Cache is a binary file which contains the data used by Google’s HistoryQuickProvider (HQP). The HQP serves up autocomplete candidates from the profile’s history database. As the user starts typing into the omnibox, the HQP performs a search in its index of significant historical visits for the term or terms which have been typed. The resulting candidates are scored and a limited number of only the most relevant matching URLs visited are presented to the user.

 

Digital Detective NetAnalysis Chrome History Provider Cache

The image above shows the History Provider entries from a Google Chrome History Provider Cache file loaded into NetAnalysis. The History Provider Cache contains WordListItem and WordMapItem objects. These objects store the list of words used to search against. When the file is processed, they are written out to an external text file (located in the Export Folder) and are included in the list of files added to the search index.

Microsoft Internet Explorer and Edge Typed URLs

Microsoft Internet Explorer and Edge browsers also have a similar feature to Google Chrome’s History Quick Provider. As entries are typed into and/or selected from the Address Bar, the browser saves the entry to a location in the Registry under the sub-key TypedURLs. Over different Operating Systems and browsers, the number of entries stored has varied. In later releases, Microsoft has also added corresponding TypedURLsTime and TypedURLsVisitCount sub-keys. In NetAnalysis v2.4, we have added support for reading registry hive files and can extract the typed URL information. We can also read the corresponding time and visit count information. The information panel in the screen shot below shows the corresponding registry sub-keys for the data.

 

Digital Detective NetAnalysis TypedURLs

Network Action Predictor

We have added support for the import of Network Action Predictor data for Google Chrome and Chromium Based Browsers. This data can be either autocomplete predictor, resource prefetch predictor or logged in predictor entries.

If the autocomplete prediction feature is enabled, Chrome will use a prediction service to help complete searches and URLs typed into the omnibox. If the Chrome prerendering feature is enabled, the Browser will attempt to speed up navigation for a user by prerendering pages that it predicts the user is likely to navigate to.

The stored prediction data can be viewed live in the Browser by typing: chrome://predictors in the Chrome omnibox. Chrome will display tabs for both the Autocomplete Action Predictor and the Resource Prefetch Predictor entries. The Logged In Predictor entries were made obsolete as of Chrome v44.

The Autocomplete Action Predictor entries show a history of the characters the user typed into the omnibox and the URL that was then selected.

The Resource Prefetch Predictor entries list the resources that were predicted to be needed for a given URL. The Browser determines which resources to fetch based on prior browsing history.

Digital Detective NetAnalysis Network Action Predictors

In the screen capture above, the user text entered by the user is shown in the information panel against the associated Autocomplete Predictor entry.

Bookmarks

We have added support for the import of bookmark data as well as extraction of associated Bookmark images to the export folder for the following browsers:

  • Mozilla Firefox and Mozilla Based Browsers
  • Google Chrome and Chromium Based Browsers
  • Apple Safari (including Reading List)
  • Opera Presto v3-12
  • Opera Presto v7-12 Notes
  • Opera v15-16
  • Opera v25+
  • Netscape HTML Bookmarks

Apple Safari bookmarks are stored in the Bookmarks.plist file. On Mac OS X, Safari also stores the user Reading List entries in this file whereas under Windows, these were stored in a separate ReadingList.plist file. When Reading List entries are extracted, any preview text is copied to the export folder. We support importing data from both Bookmarks.plist and ReadingList.plist files.

Opera Presto stored its bookmarks in a Hotlist format file. This format was also used to store Opera notes. NetAnalysis can now extract bookmarks for Opera v3-12 and notes for Opera v7-12.

Opera v15-16 stored its bookmarks in a bookmarks.db database. Opera v17+ then reverted to using the Chromium based file format. Opera added their own extra structure on top of the Chromium format from Opera v25+. NetAnalysis now supports all of these format variations. Any bookmark web page preview image files are also extracted to the export folder. These previews can be displayed using the Viewer panel.

The Netscape HTML file format is still widely used as a data exchange format by the current Browsers. The latest versions of Chrome, Firefox and Safari allow the user to import and export bookmarks in this format; while Opera allows the user to import Netscape HTML format bookmarks. Any Netscape HTML file format bookmark favicons are therefore copied to the export folder under folder name “Unidentified Browser”.

Digital Detective NetAnalysis Apple Safari Reading List and Bookmarks

The screen capture above shows bookmark and reading list data from Apple Safari v9. The screen capture below shows bookmark data from Opera v36.

Digital Detective NetAnalysis Opera Bookmarks with Page Preview

Change Log

 

We are pleased to announce the next major release for NetAnalysis® and HstEx® has just been published. For an overview of the new features we are shipping inside NetAnalysis® v2.2 and HstEx® v4.2, please take a moment to review our release notes and change log:

NetAnalysis® v2.2

This release brings a number of new features and improvements. We have added support for six new browsers as well as making the necessary updates required to support the changes in the mainstream browsers. We have also added support for some new artefacts.

New NetAnalysis® Browser Support

We have added new support in NetAnalysis® for the forensic analysis of the following browsers:

New Artefacts

Favicons

We have added support for the import of Favicon data as well as extraction of icons and associated Favicon images to the export folder for the following browsers:

The following screen shows some filtered Favicon entries from Safari.

Digital Detective NetAnalysis showing Apple Safari Favicons

During the import process, the actual icons/image files are extracted to the export folder. Open the export folder by selecting Tools » Open Case Export Folder and select the Favicons folder for the corresponding browser. This will show you all of the extracted images. You can match the unique reference number for the image (URN) to the unique reference number of the record entry. The image below shows a typical Favicons folder.

Extracted Favicons

Any History record which has an associated Favicon entry will have the Favicon URL displayed in the Favicon URL column for that entry.

Chromium Session / Tab Restore

Google Chrome and many of the Chromium based browsers store session and tab information in four files:

  • Current Session
  • Current Tabs
  • Last Session
  • Last Tabs

These files store information relating to the current and last browsing session and can be very helpful in a forensic investigation. We have now added support to import the tab navigation information. The screen below shows opening a new session with the default new tab selected and then directly navigating to a test page on the Digital Detective web site.

Digital Detective NetAnalysis Chrome Session and Tab Restore

Base58 Decoding

Base58 is a group of binary-to-text encoding schemes used to represent large integers as alphanumeric text. It is similar to Base64 but has been modified to avoid both non-alphanumeric characters and letters which might look ambiguous when printed. It is therefore designed for human users who manually enter the data, copying from some visual source, but also allows easy copy and paste because a double-click will usually select the whole string.

Compared to Base64, the following letters have been omitted from the alphabet: 0 (zero), O (capital o), I (capital i) and l (lower case L) as well as the non-alphanumeric characters + (plus) and / (slash). In contrast to Base64, the digits of the encoding don’t line up well with byte boundaries of the original data. For this reason, the method is well-suited to encode large integers, but not designed to encode longer portions of binary data. The actual order of letters in the alphabet depends on the application, which is the reason why the term “Base58” alone is not enough to fully describe the format.

Base58 is used for:

We have added Base58 decoding to the decoding/examination window. The following shows an example Bitcoin address being decoded:

Digital Detective NetAnalysis Base58 Decoding

HstEx® v4.2

This release brings support for an additional six new browsers, updated support for all the existing supported browsers and some user interface enhancements.

New HstEx® Browser Support

We have added new support for the following browsers:

Updates for Existing Browsers

Google Chrome has updated the SQLite database schema format number for new databases which has resulted in a significant change to the on disk structure of individual SQLite records. To take into account this change, we have updated the recovery engine for Chrome Cookies, Downloads and History entries.

To review the current supported browsers, please see: Supported Browsers

User Interface Enhancements

To assist with selecting the most appropriate recovery modules, we have added a new toolbar to the Recovery Job window. It is now possible to select the following recovery profile scenarios:

  • Common: This option selects the most common recovery profiles
  • Windows: This option selects the recovery profiles for browsers that can be installed on Windows
  • OS X: This option selects the recovery profiles for browsers that can be installed on OS X
  • Linux: This option selects the recovery profiles for browsers that can be installed on Linux
  • Xbox: This option selects the recovery profiles for browsers that can be installed on Xbox
  • Select All: This options selects all recovery profiles
  • Clear All: This option deselects any currently selected recovery profiles

 

Digital Detective HstEx Recovery Job

We are pleased to announce the next major release for NetAnalysis® and HstEx® has just been published. For an overview of the new features we are shipping inside NetAnalysis® v2.1 and HstEx® v4.1, please take a moment to review our release notes:

Here is an example of some the updates:

Username and Password Decryption

Firefox and other Mozilla based browsers include a Password Manager that can save the passwords provided by the user as they log in to websites. The Password Manager securely stores the usernames and passwords used to access websites and then automatically fills them in for the user when they next visit the site. For additional security, the user can also set a Master Password to protect the Password Manager. The user is then prompted to enter the Master Password when the browser needs to access the stored passwords. Usernames and passwords are encrypted and stored within the Mozilla profile.

NetAnalysis® v2.1 is now able to decrypt and display the usernames and passwords stored for each web site. The following image shows the NetAnalysis® Information Panel with some decrypted Username and Password values. Also, the entry on line number 1 shows that the Master Password has not been set in this case.

 

NetAnalysis v2 Mozilla Firefox Username and Password Decryption

Mozilla Firefox Username and Password Decryption

 

New Browser Support

In addition to extending support for the existing browsers and their recent changes, we have now added support for two new browsers:

  • SRWare Iron v1 – 38
  • K-Meleon v1 – 74

Apple Safari 8

Apple Safari v8 was released with OS X Yosemite and brought with it a change to its history storage. As a result, HstEx® v4.1 has been updated to support the recovery of individual entries from Safari v8 history records. History records are split across History Items and Visits. We offer an option to recover both types.

 

HstEx v4 Recovery of Apple Safari v8 History Visits and Entries

Recovery of Apple Safari v8 History

 

Improvements

We have been working hard to increase the performance, accuracy and stability of HstEx® v4. As a result, we have updated all of our SQLite recovery engines to ensure they are accurate and fast. We have improved the handling and reporting of corrupt entries (partially recovered records are flagged in NetAnalysis® v2). We have also made some improvements to the recovery of Binary Plist data.

Firefox v32+ Cache v2

Mozilla Firefox officially released their new caching backend with the release of Firefox v32 back in September 2014. The structure is completely different from that used previously. HstEx® v4.0 was the first forensic tool to support the recovery of deleted Mozilla Firefox Cache v2 records. After Firefox v33 was released, Mozilla made some further changes to the file format. HstEx® v4 supports all the currently released formats of Mozilla’s Cache v2 structure. We have also made some further improvements to the recovery of Cache v2 records, in particular the identification of corrupt data.

Keyword Search Terms

We have extended support for the recovery of individual keyword search terms for all Chromium based browsers and have improved the recovery of very large keyword strings.

New Artefacts

We have added support for the extraction of over a dozen new artefacts and data types. For a detailed list of each artefact, please see the following:

Here are a few examples:

Google Search EI/SEI Parameter Decoding

Google search URLs will sometimes contain an EI or SEI parameter. We have added support to the URL/Cookie Examination and Analysis window to allow automatic decoding of these parameters. The window below shows the automatic decoding of a Google URL which contains an EI parameter. The EI parameter is a Base64 encoded 16 byte value. The first 4 bytes contain a timestamp which can be seen in the example below.

 

NetAnalysis v2 Decoding Google EI Parameter

Decoding Google EI Parameter

 

Google Chrome Autofill Profiles

Autofill forms is a feature of Google Chrome and other Chromium based browsers. It allows for the user to store information such as name, address, phone number and email address as an Autofill entry so that forms can be automatically populated. In NetAnalysis® v2.1, we extract the data from the Autofill Profiles and display them in the main grid. We also extract the corresponding form data and save it to the export folder for indexing and searching.

 

NetAnalysis v2 Google Chrome Autofill Profiles

Google Chrome Autofill Profiles

Google Chrome Credit Card Autofill

The window below shows the extraction of Google Chrome Credit Card Autofill data. The text relating to the autofill fields are extracted to the export folder so that the data can be indexed and searched.

 

NetAnalysis v2 Google Credit Card Autofill

Google Chrome Credit Card Autofill

Apple Safari Reading Lists

The window below shows a number of Apple Safari Reading List entries. These represent sites the user has selected to view at a later date. Once the user visits a site from the Reading List, the Date Visited is updated to reflect the date and time of the visit.

 

NetAnalysis v2 Apple Safari Reading List

Apple Safari Reading List

 

Opera Blink Favorite Entries

The window below shows a number of Opera Favorite entries.

NetAnalysis v2 Opera Favorite Entries

Opera Favorite Entries

 

Download Version

Kent, UK – September 2014

Digital Detective Group Ltd., a provider of leading digital forensic software, announced today the release of their new ground breaking software NetAnalysis® v2.

NetAnalysis® v2 is a state-of-the-art application for the extraction, analysis and presentation of forensic evidence relating to Internet browser and user activity on computer systems and mobile devices.

Their NetAnalysis® suite also includes HstEx® v4, an advanced data recovery solution designed to recover deleted browser artefacts which can be imported into, and analysed in NetAnalysis®. Utilising powerful parallel processing and Intelli-Carve® technology, HstEx® offers a considerable speed increase and accuracy in forensic data recovery.

Craig Wilson, the Managing Director of Digital Detective Group said:

NetAnalysis® v2 is a software product that offers significant improvements over existing applications and methodologies. We have invested a significant amount in research and development and have authored a completely new ground-breaking product, engineered through innovation and fresh thinking.

Our primary goal is to develop innovative new technologies and focus our efforts on areas where science presents new opportunities most likely to lead to significant forensic advances. We aim to build upon our reputation as a pioneer in the field of digital forensic science and are committed to developing leading products that will advance our mission to make the world a safer place through digital forensic expertise.

Research and development is a key element to developing advanced, cutting-edge technology and our team works to solve the challenges that exist in the highly dynamic, hi-tech world of digital forensics. We are extremely pleased with this new product and hope that it will give our law enforcement and corporate customers that extra edge.

About Digital Detective

Digital Detective was founded in 2001 and develops forensic software for the extraction and analysis of digital data from a variety of digital devices.

Guided by the vision of Managing Director Craig Wilson, Digital Detective Group is built on the principles of innovation, creativity and commitment to excellence.

In 2002, Digital Detective released their flagship product NetAnalysis®, a software application designed specifically for the extraction and analysis of Internet trace evidence. The software is trusted and utilised by the leading law enforcement agencies, commercial companies and government entities worldwide.

Introduction

We are pleased to announce the release of NetAnalysis® v2. This new version is a state-of-the-art application for the extraction, analysis and presentation of forensic evidence relating to Internet browser and user activity on computer systems and mobile devices.

The NetAnalysis® suite also includes HstEx® v4, an advanced data recovery solution designed to recover deleted browser artefacts which can be imported into, and analysed in NetAnalysis®. Utilising powerful parallel processing and Intelli-Carve® technology, HstEx® offers a considerable speed increase and accuracy in forensic data recovery.

NetAnalysis® v2 is a software product that offers significant improvements over existing applications and methodologies. We have invested a significant amount in research and development and have authored a completely new ground-breaking product, engineered through innovation and fresh thinking.

Our primary goal is to develop innovative new technologies and focus our efforts on areas where science presents new opportunities most likely to lead to significant forensic advances. We aim to build upon our reputation as a pioneer in the field of digital forensic science and are committed to developing leading products that will advance our mission to make the world a safer place through digital forensic expertise.

Research and development is a key element to developing advanced, cutting-edge technology and our team works to solve the challenges that exist in the highly dynamic, hi-tech world of digital forensics. We are extremely pleased with this new product and hope that it will give our law enforcement and corporate customers that extra edge.