Saturday, June 24, 2017

MetaDiver 3.1.2 released

MetaDiver 3.1.2 has been released! The latest is available for download.

http://ift.tt/1BFPceC

v3.1.2 (build 1635)
-bugfix to button for mapping gps for pictures
-update settings mapping db default templates
-remove old “Type” column mapping from shortcuts
-cleanup tika logging for no metadata. Moved from error to debug log and removed error on individual null fields causing false positives.

 

Enjoy!

Dave


by Dave via EasyMetaData

Friday, June 16, 2017

MetaDiver 3.1.1 is released

The latest version of MetaDiver is available for download.

Download: Metadiver 3.1.1

Numerous improvements from previous release. Using the latest version is highly recommended!

Changelog

v3.1.1 (build 1623)
-bugfixes to paging in Review window
-fix to keyword search not pulling back hits in some cases
-prevent empty line in keywords on save
-performance optimizations
-resized evidence path window on process form
-bugfixes

v3.1.0 (build 1620)
-check for update ui change
-package update for ElastSearch.Net 5.4, DocumentFormat.OpenXML, AlphaFS 2.1.3
-fix case info form fields not saving

v3.1.0 (build 1602)
-upgrade outlook redemption from 5.10.0.4312 to 5.14.0.4798
-fixes minor issues with filtering
-preview email fixed when column profile changed from summary
-fixed temp directory issue when cleaning it up.
-fix keyword search issue with switch to generics
-fix sorting of dropdown on review window
-review window – metadata fields now uses 3rd party control for better look and feel
-review window – metadata fields now show all fields every time you select a row
-udpated filterlist for filtering by specific extensions on intake
-renamed about to ? and reated submenu’s from menu bar
-added feedback option from ? menu item
-change parser mapping to x-parsed-by for openofficexml docs and email msg via redemption
-updated SQLite Core to 1.0.105.1
-bug fixes

v3.0.1 (build 1588)
-improved tika handling with task timeout
-fixed handling for tika hang/bug on rotten officexml docs
-added back the fallback to ms openxml document mapping when tika hangs on faulty officexml format
-added back the fallback to ms openxml document mapping when tika fails to parse officexml format
-memory leak fixed
-change lnk file parser info field mapping to “x-parsed-by” from “parsed”
-improved tika metadata parsing
-added new metadata field mappings
-keywords run save to caseinfo.json file
-intake path’s save to caseinfo.json file
-bugfixes

 


by Dave via EasyMetaData

Thursday, May 4, 2017

Upgraded hosting hardware

In the past week I moved the websites to vps from shared hosting for www.easymetadata.com and www.redrocktx.com. I’m noticing a huge difference for $4/m more. I know I’m stubborn for not ditching the whole website thing and moving to medium.. I’m just not that hipster. I like having a shell and control.

Anyway, hopefully you notice the improved responsiveness as well!

Enjoy


by Dave via EasyMetaData

MetaDiver 3.0 beta is released #dfir #infosec #metadata

I’m happy to announce the first beta release of MetaDiver 3.0!

About: MetaDiver is a utility to slice and dice files and recover metadata from various types of files such as emails, documents, pictures, videos and music among many files. With MetaDiver you will find detailed metadata that many tools either do not find.

It has been a year since the last version release! That’s a long time but, a lot of great things have happened in my life over the past year… and I had a lot of things i wanted to improve in MetaDiver before I put out a new release. It has been the classic scope creep. The look and feel may not be very different at first look but you will quickly notice the difference. Hopefully all of those late nights coding after work was worth it and you find MetaDiver as useful as we do in our lab.

Feedback: If you have thoughts and suggestions I’ve put together a feedback form. Please give me helpful feedback!

Change: The new version 3 has some great new features including:

  • Keyword searching added – now you can load keywords then process the metadata. Hits will display in a column “Search hits”
  • 3rd party UI controls for better user experience when reviewing metadata
  • Rewrite of the processing engine and much of the codebase with two focuses.
    • #1 SPEED – I’ve clocked it on software raid1 at 380 MB/s
    • #2 MEMORY – Memory utilization is way down
  • Replaced homebrew logging with apache log4net
  • Removed dependency on win32 shell for item type detection
  • Tika known document types (globs) are processed by default and other unknown file types are handled by the Tika engine unless user checks the box for “process unknown file types”.
  • Added picture review in review window
  • Added GPS exif review in the new picture tab in review window
  • Added the ability to click a buttom to bring GPS coordinates up in online map
  • All known document types and media are now fed into tika (custom doc types will be handled by metadiver parsers)
  • Forensic artifact Windows shortcuts and Jumplists are handled by shellify for now. ~Possibly switching to Lecmd codebase in 3.1
  • Various bug fixes and other enhancements

Please fill out the feedback form so I can get an idea of what you like, how you use MetaDiver, and what you’d like to have added

Download: You can download metadiver 3.0 (beta 1) here

Enjoy!

Dave


by Dave via EasyMetaData

Thursday, February 2, 2017

Funny Mac behavior with fat32 volume label mod dates #dfir

Recently I had to do some testing to see what causes the modified date for a  fat32 volume label to get changed. It has been understood for as long as i can remember that the modified date for a volume name is set when you format your thumb-drive or hard disk partition.
So I did some testing and my testing shows that MacOS doesn’t follow the rules! In fact any time I plugged a fat32 thumb-drive in to a Mac running 8.5 or later the modification date for the Volume Label was modified. You say what? Yup, I was able to reproduce this behavior all the way up to the latest iteration of MacOS, Sierra I right now I think. It’s important to state that date only changes if the FAT32 volume has a volume name set. If it’s the default fat32 name the date will not change!

[UPDATED]
Testing has shown that each time the FAT32 thumb-drive with a volume label set is plugged in to the Mac the value gets changed by MacOS to the current datetime.

*For background on fat32 volume serial numbers and date time verification Digital Discovery has this paper “Volume Serial Numbers and Format Date/Time Verification” last updated in 2005.

Why is this happening? Well, from my investigations in to the log file it appears the fat32 driver may be parsing the modification date incorrectly and causing the kernel driver to set a new date that it thinks is valid.
This can have some fairly significant implications for you investigations related to fat32 formatted devices if they have MacOS artifacts. Once again, the important caveat to what I just told you is that this only happens when there is a volume label set. So if fat32 is NO NAME then you shouldn’t see the date change. Please test and let me know if you have any additional findings!
Happy hunting!
-Dave

by Dave via EasyMetaData

Friday, December 30, 2016

Added source for simple console app to dump metadata and content using #TIKA using .NET.

I decided I needed to put out a simple command line program for dumping metadata. It’s been sitting on my todo list for too long.  I’ve been using Tika for a long time now and it’s amazing how many file format’s it supports. The file formats it supports keeps grows with every new release. This is bare bones compared to MetaDiver and is strictly TIKA based.

TIKA supported formats: http://ift.tt/2imwF1I

There are so many supported format’s I can’t list them all.

I know we already have a lot of programs out there to for parsing metadata from files but most are commercial. Phil Harvey’s Exiftool is a free program that does an amazing job at metadata but you should always have another option. More importantly, each tool has limits to formats. Tika supports constuming exiftool  as of 1.9 to supplement metadata using the Java version! Pretty amazing.

I decided to keep it simple with the 1.0 release. You’ll get the key value pairs from the file metadata and you can also dump the text from the file to the console.

Sample output:

T:\MD_DumpCLI>MD_DumpCLI.exe -f "T:\Test_data\exif\IMG_0581.JPG"
Author:  David Dym
License: Apache 2.0
 http://ift.tt/1r8ST99

Filename: N:\Test_data\exif\IMG_0581.JPG

Aperture Value: f/2.8
Brightness Value: 5067/1265
Color Space: sRGB
Component 1: Y component: Quantization table 0, Sampling factors 2 horiz/2 vert
Component 2: Cb component: Quantization table 1, Sampling factors 1 horiz/1 vert
Component 3: Cr component: Quantization table 1, Sampling factors 1 horiz/1 vert
Components Configuration: YCbCr
Compression: JPEG (old-style)
Compression Type: Baseline
Content-Type: image/jpeg
Creation-Date: 2011-10-23T13:55:09

.... (cutoff the other 200+ fields)

Github Page: http://ift.tt/2imFVTo

Enjoy!

Dave


by Dave via EasyMetaData

Tuesday, July 26, 2016

Extract document #metadata – #Tika and #exiftool

Intro

Metadata is critical to any investigation. So much knowledge can be gleamed from the review of metadata from pictures and documents that it’s a big topic in the news. Look at the DNC hack last month. But for those of us in the digital forensics and the field of information security metadata has always been critical to our investigations.

If metadata is a new/confusing term for you then go read about it: http://ift.tt/12SKG6a

When using commercial products to some investigators trust the information from the commercial tools they paid licenses to use without validating the results using a secondary tool or reading the ‘release notes’ for caveats. Always read the release notes! It’s best practice to always test your tools to ensure you are not only getting accurate results but also as many results as possible!

Scenario

I have been reminded once again that commercial tools can miss document metadata. Sometimes its because you are running an old version, other times because you did not select the proper check box when processing, and other times the tool just does not support the document format. Some obscure document types have spotty records with even the most popular commercial software.

In this case the software uses file extensions to determine file type on a standard pass and if you didn’t run an extra processing option to have it use the magic header/document signature to determine document type the forensic tool would not parsed the metadata completely.

In this case a colleague was reviewing an office .xlsx document that showed only ‘content created’ and ‘source modified’ as the document properties. This looked fishy to me so I suggested that more data could be present by inspecting the file manually. In this case renaming the xlsx to gzip and unzipping it to read the .xml files manually.

Sure enough multiple fields including ‘author’, ‘last modified by’, and more were present! In this instance file properties under Windows only showed the ‘content created’ and ‘date last saved’. The same exercise could have been conducted using Tika or ExifTool. So, if I hadn’t suggested digging deeper the metadata might have been missed…

Metadata tools

I’ll list some tools I recommend for validating metadata results from documents and images below.  I’m also showing a way to run the Apache Tika tool that reads metadata from files under Windows without having to use Java! The reason this is fun is Tika reads a TON of file formats but is written in Java and I don’t like to install Java unless required and this will let you use it on any machine running .NET.

Free tools that get comprehensive metadata are:

Sorry for the shameless plug of my own metadata extraction tool called ‘MetaDiver‘ that uses Tika heavily for metadata extraction.

Tika in Windows

You can run the latest Tika on windows to inspect files individually.

To use Tika on Windows you will need to do a few things.

  • Download the Tika jar file here.
  • Download iKVM here.

Use the syntax from a command prompt “ikvm.exe -jar tika.jar” from the ikvm directory. It’s that simple.

start_tika_with_ikvm

tika_gui_with_ikvmtika_file_metadata_with_ikvm

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

ExifTool

To use exiftool just download exiftool and run it from a command prompt.

exiftool1

ExifTool with Tika

According to the Tika documentation you can wrap Tika around exiftool to add even more information extraction. Tika already supports an insane number of document formats.

http://ift.tt/2auHpYp

I haven’t tested this configuration.

Wrap-Up

Exiftool and Tika are both free, well maintained, cross platform and regularly updated with the latest changes to document formats. Having the latest version because file structures change as software changes and testing the latest version are really important when it comes to metadata to ensure you can read everything from the file you are inspecting!

Beware of how the different tools handle dates regarding time zones, UTC vs local time and daylight savings time.

I hope you find this post informative and actionable.

Enjoy!

 

 

 


by Dave via EasyMetaData