Thursday, May 4, 2017

Upgraded hosting hardware

In the past week I moved the websites to vps from shared hosting for www.easymetadata.com and www.redrocktx.com. I’m noticing a huge difference for $4/m more. I know I’m stubborn for not ditching the whole website thing and moving to medium.. I’m just not that hipster. I like having a shell and control.

Anyway, hopefully you notice the improved responsiveness as well!

Enjoy


by Dave via EasyMetaData

MetaDiver 3.0 beta is released #dfir #infosec #metadata

I’m happy to announce the first beta release of MetaDiver 3.0!

About: MetaDiver is a utility to slice and dice files and recover metadata from various types of files such as emails, documents, pictures, videos and music among many files. With MetaDiver you will find detailed metadata that many tools either do not find.

It has been a year since the last version release! That’s a long time but, a lot of great things have happened in my life over the past year… and I had a lot of things i wanted to improve in MetaDiver before I put out a new release. It has been the classic scope creep. The look and feel may not be very different at first look but you will quickly notice the difference. Hopefully all of those late nights coding after work was worth it and you find MetaDiver as useful as we do in our lab.

Feedback: If you have thoughts and suggestions I’ve put together a feedback form. Please give me helpful feedback!

Change: The new version 3 has some great new features including:

  • Keyword searching added – now you can load keywords then process the metadata. Hits will display in a column “Search hits”
  • 3rd party UI controls for better user experience when reviewing metadata
  • Rewrite of the processing engine and much of the codebase with two focuses.
    • #1 SPEED – I’ve clocked it on software raid1 at 380 MB/s
    • #2 MEMORY – Memory utilization is way down
  • Replaced homebrew logging with apache log4net
  • Removed dependency on win32 shell for item type detection
  • Tika known document types (globs) are processed by default and other unknown file types are handled by the Tika engine unless user checks the box for “process unknown file types”.
  • Added picture review in review window
  • Added GPS exif review in the new picture tab in review window
  • Added the ability to click a buttom to bring GPS coordinates up in online map
  • All known document types and media are now fed into tika (custom doc types will be handled by metadiver parsers)
  • Forensic artifact Windows shortcuts and Jumplists are handled by shellify for now. ~Possibly switching to Lecmd codebase in 3.1
  • Various bug fixes and other enhancements

Please fill out the feedback form so I can get an idea of what you like, how you use MetaDiver, and what you’d like to have added

Download: You can download metadiver 3.0 (beta 1) here

Enjoy!

Dave


by Dave via EasyMetaData

Thursday, February 2, 2017

Funny Mac behavior with fat32 volume label mod dates #dfir

Recently I had to do some testing to see what causes the modified date for a  fat32 volume label to get changed. It has been understood for as long as i can remember that the modified date for a volume name is set when you format your thumb-drive or hard disk partition.
So I did some testing and my testing shows that MacOS doesn’t follow the rules! In fact any time I plugged a fat32 thumb-drive in to a Mac running 8.5 or later the modification date for the Volume Label was modified. You say what? Yup, I was able to reproduce this behavior all the way up to the latest iteration of MacOS, Sierra I right now I think. It’s important to state that date only changes if the FAT32 volume has a volume name set. If it’s the default fat32 name the date will not change!

[UPDATED]
Testing has shown that each time the FAT32 thumb-drive with a volume label set is plugged in to the Mac the value gets changed by MacOS to the current datetime.

*For background on fat32 volume serial numbers and date time verification Digital Discovery has this paper “Volume Serial Numbers and Format Date/Time Verification” last updated in 2005.

Why is this happening? Well, from my investigations in to the log file it appears the fat32 driver may be parsing the modification date incorrectly and causing the kernel driver to set a new date that it thinks is valid.
This can have some fairly significant implications for you investigations related to fat32 formatted devices if they have MacOS artifacts. Once again, the important caveat to what I just told you is that this only happens when there is a volume label set. So if fat32 is NO NAME then you shouldn’t see the date change. Please test and let me know if you have any additional findings!
Happy hunting!
-Dave

by Dave via EasyMetaData

Friday, December 30, 2016

Added source for simple console app to dump metadata and content using #TIKA using .NET.

I decided I needed to put out a simple command line program for dumping metadata. It’s been sitting on my todo list for too long.  I’ve been using Tika for a long time now and it’s amazing how many file format’s it supports. The file formats it supports keeps grows with every new release. This is bare bones compared to MetaDiver and is strictly TIKA based.

TIKA supported formats: http://ift.tt/2imwF1I

There are so many supported format’s I can’t list them all.

I know we already have a lot of programs out there to for parsing metadata from files but most are commercial. Phil Harvey’s Exiftool is a free program that does an amazing job at metadata but you should always have another option. More importantly, each tool has limits to formats. Tika supports constuming exiftool  as of 1.9 to supplement metadata using the Java version! Pretty amazing.

I decided to keep it simple with the 1.0 release. You’ll get the key value pairs from the file metadata and you can also dump the text from the file to the console.

Sample output:

T:\MD_DumpCLI>MD_DumpCLI.exe -f "T:\Test_data\exif\IMG_0581.JPG"
Author:  David Dym
License: Apache 2.0
 http://ift.tt/1r8ST99

Filename: N:\Test_data\exif\IMG_0581.JPG

Aperture Value: f/2.8
Brightness Value: 5067/1265
Color Space: sRGB
Component 1: Y component: Quantization table 0, Sampling factors 2 horiz/2 vert
Component 2: Cb component: Quantization table 1, Sampling factors 1 horiz/1 vert
Component 3: Cr component: Quantization table 1, Sampling factors 1 horiz/1 vert
Components Configuration: YCbCr
Compression: JPEG (old-style)
Compression Type: Baseline
Content-Type: image/jpeg
Creation-Date: 2011-10-23T13:55:09

.... (cutoff the other 200+ fields)

Github Page: http://ift.tt/2imFVTo

Enjoy!

Dave


by Dave via EasyMetaData

Tuesday, July 26, 2016

Extract document #metadata – #Tika and #exiftool

Intro

Metadata is critical to any investigation. So much knowledge can be gleamed from the review of metadata from pictures and documents that it’s a big topic in the news. Look at the DNC hack last month. But for those of us in the digital forensics and the field of information security metadata has always been critical to our investigations.

If metadata is a new/confusing term for you then go read about it: http://ift.tt/12SKG6a

When using commercial products to some investigators trust the information from the commercial tools they paid licenses to use without validating the results using a secondary tool or reading the ‘release notes’ for caveats. Always read the release notes! It’s best practice to always test your tools to ensure you are not only getting accurate results but also as many results as possible!

Scenario

I have been reminded once again that commercial tools can miss document metadata. Sometimes its because you are running an old version, other times because you did not select the proper check box when processing, and other times the tool just does not support the document format. Some obscure document types have spotty records with even the most popular commercial software.

In this case the software uses file extensions to determine file type on a standard pass and if you didn’t run an extra processing option to have it use the magic header/document signature to determine document type the forensic tool would not parsed the metadata completely.

In this case a colleague was reviewing an office .xlsx document that showed only ‘content created’ and ‘source modified’ as the document properties. This looked fishy to me so I suggested that more data could be present by inspecting the file manually. In this case renaming the xlsx to gzip and unzipping it to read the .xml files manually.

Sure enough multiple fields including ‘author’, ‘last modified by’, and more were present! In this instance file properties under Windows only showed the ‘content created’ and ‘date last saved’. The same exercise could have been conducted using Tika or ExifTool. So, if I hadn’t suggested digging deeper the metadata might have been missed…

Metadata tools

I’ll list some tools I recommend for validating metadata results from documents and images below.  I’m also showing a way to run the Apache Tika tool that reads metadata from files under Windows without having to use Java! The reason this is fun is Tika reads a TON of file formats but is written in Java and I don’t like to install Java unless required and this will let you use it on any machine running .NET.

Free tools that get comprehensive metadata are:

Sorry for the shameless plug of my own metadata extraction tool called ‘MetaDiver‘ that uses Tika heavily for metadata extraction.

Tika in Windows

You can run the latest Tika on windows to inspect files individually.

To use Tika on Windows you will need to do a few things.

  • Download the Tika jar file here.
  • Download iKVM here.

Use the syntax from a command prompt “ikvm.exe -jar tika.jar” from the ikvm directory. It’s that simple.

start_tika_with_ikvm

tika_gui_with_ikvmtika_file_metadata_with_ikvm

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

ExifTool

To use exiftool just download exiftool and run it from a command prompt.

exiftool1

ExifTool with Tika

According to the Tika documentation you can wrap Tika around exiftool to add even more information extraction. Tika already supports an insane number of document formats.

http://ift.tt/2auHpYp

I haven’t tested this configuration.

Wrap-Up

Exiftool and Tika are both free, well maintained, cross platform and regularly updated with the latest changes to document formats. Having the latest version because file structures change as software changes and testing the latest version are really important when it comes to metadata to ensure you can read everything from the file you are inspecting!

Beware of how the different tools handle dates regarding time zones, UTC vs local time and daylight savings time.

I hope you find this post informative and actionable.

Enjoy!

 

 

 


by Dave via EasyMetaData

Wednesday, June 29, 2016

New Forensic Image viewer under dev #dfir #infosec

There is a new forensic viewer in town called “Pancake Viewer”. It’s being developed by our good friend Forensicmatt known for the Triforce journal parser

Pancake viewer is there to review forensic artifacts interactively in a simple interface, for free. Its familiar and based fully on opensource libraries. If your curious about the backend it’s powered by python, libtsk and dfvfs and for the gui it uses wxpython.

What makes it different you ask? It’s the ability to explore the image and the volume shadow copies! Because its open source lgpl and python you can make changes to meet your needs as long as you share your changes.

Currently it’s an alpha project but moving quickly towards a working release.

Take a look: http://ift.tt/29cX5SL

Enjoy!

Dave


by Dave via EasyMetaData

Monday, May 16, 2016

MetaDiver 2.5 released!

I’m happy to announce that MetaDiver 2.5.0 is available for download. This is a big release with some fun new stuff.

In this release there are a number of important and exciting new features

You can now build your own column profiles for different document formats when reviewing meta-data. It’s a simple builder but you can build profile for any set of columns.

ColumnProfileBuilder1

Also in this release I’ve introduced 3rd party controls to improve the user experience. You will notice you can ctrl-f to search for text in Tika Strings and Binary Strings! It even supports regex.

Stringsctrlf1

You can now filter on a cell value in the grid. This is a nice time saver I have found especially when going through email. An example is when you find a sender or subject or conversationid and then filter on it. You then have just the results for that value showing in the grid!

FilterOnField1

When selecting a column to search from the drop down it will now auto search to find your text as you type. This makes finding the column you want much faster!

ColumnAutoComplete1

You now select the Columns you want to be exported in the Export window. I know this is a long time coming.

ExportProfile1

Other stuff includes:

-fix status bar on hex and binary strings screens.
-culling columns is now optional
-fixed email mapping for message last modified time
-added check for path's longer than 255 when selecting case path
-"select all" added to grid
-misc bug fixe

There is more but I’m tired of screenshots. I hope you enjoy this release and use the product. Look forward to feedback!

You can Download MetaDiver 2.5.0.

Enjoy!


by Dave via EasyMetaData