All posts by Kris Wasserman

eDiscovery Problem Solver

#MetadataMatters – Don’t Be That Guy

I seldom get behind viral videos, but quite frankly, I’m shocked no one has thought of this yet. Thanks to my good friend Kris Taylor for passing it along. 

 

Check out www.Metadata-Matters.com for news aggregated from around the web geared towards the Legal Technology Industry.

#MetadataMatters – Doing It Right: Combining Technology and Cooperation in eDiscovery

Fantastic session yesterday at LegalTech.  Thanks to David Leone, Ariana Tadler, Maura GrossmanRonni Solomon, and Conor Crowley.

LegalTech® New York 2014  February 4 Timeline   the most important legal technology events of the yearDoing It Right:  Combining Technology and Cooperation in eDiscovery

“Cooperation” and “transparency” became the buzzwords in 2008, when The Sedona Conference issued its Cooperation Proclamation.  Since then, the world of e-discovery has evolved at warp speed.  Volume has exploded.  Complexity has increased.  Judicial expectations have evolved.  At the center of it all, advancing technology has changed everything.

Ariana Tadler, Partner at Milberg LLP and Chair Emeritus, as well as an active member of The Sedona Conference® Steering Committee for Working Group I, leads a panel of eDiscovery thought leaders and practice experts in an overview of eDiscovery in 2014:

Attend this session to learn more about how to proactively combine technology and cooperation to “do e-discovery right,” including:

  • When and how to use cutting edge technologies – and when to avoid them
  • Updates on current “best practices” and judicial expectations
  • How to cooperate in the world of technology-assisted review
  • How to deal with the “pretend cooperator”

Gain from the experience and expertise of seasoned eDiscovery professionals in this mixture of discourse and dialogue.

For a look at today’s LTNY agenda go to: http://www.legaltechshow.com/r5/cob_page.asp?category_id=76596&initial_file=cob_page-ltech_agenda-timeline.asp

#MetadataMatters – What to do (and not do) when collecting ESI

Forensics in February

There are countless examples of recent cases wherein sanctions and adverse inference instructions were handed down to litigants due to improper collection procedures.

These have entailed everything from misguided self-collections, inadequate or missing chain-of-custody documentation, and even employees making material alterations to information they assumed would be damaging.

For a great article written by our friends over at Stroock, Stroock & Lavan that provides a primer for what you and your team should consider in advance of collecting data in the face of litigation click here

FREE WEBINAR – The Dangers of Self-Collecting! 
Register today to learn about pitfalls to be avoided in the early stages of the discovery process. Topics covered will consist of the latest best practices, pre-collection preparation, and vendor vetting.

Collin Bentley, Director of Digital Forensics at Superior Discovery, will be be on-hand to field questions regarding the latest industry trends and what to consider when dealing with social media or mobile device acquisitions. Click the Events button below for the agenda and more in-depth information.

Eventbrite - Dangers of Self Collecting - Forensics in February Webinar Series

CommandLineFu: File/Byte Count of Folder List

 while read -r dir; do echo -n `du -hsb "$dir"` ; echo "|"`find "$dir" -type f | wc -l` ; done < UD_input.txt | tee UserData.log

 

Uses text file with one directory per line as input and prints:

BYTES [space] DIRECTORY NAME [pipe] FILE COUNT

1842531456 ./FD99_UserShare/z/Bond/|213

CommandLineFu: FileType Report w/ Dates

More find magic — this little beast takes a while to run on large directories but is worth it’s weight in gold.  Now I just need a way to convert Epoch time to YYYYMMDD format inline. (New project)

find ./foo/ -type f -printf '%f|%h|%s|%AY%Am%Ad|%TY%Tm%Td|' -exec stat --printf "%W|" '{}' \; -exec file -bp '{}' \; > bar.log

%f == file name without leading directories

%h == leading directories without file name

%s == size in bytes

%A = Last access time (Y,m,D = YYYYMMDD format)

%T == Modification time (Y,m,D = YYYYMMDD format)

(using printf keeps everything on the same line)

stat %W == file birth date in Epoch time

file -bp == checks file type, b==brief, p==preserve date

CommandLineFu: FOR LOOP – report file count in pwd/* && print disk usage

#!/bin/bash
for i in */ ;
    do
        echo -n "$i:" >> "/path/to/some/file/already/created.txt" ;
        find "$i" -type f | wc -l >> "/path/to/same/file/already/created.txt" ;
        du -hs "$i" ;
    done
exit 0

Quick and dirty…
Actually this is quite slow when dealing with directories containing thousands of little files.
But it gets the job done.  I’ll play around with it and see if forking helps.

CommandLineFu: Read list of filenames – test if they exist

while read -r file; do if [[ ! -e $file ]]; then echo "$file|error" ; fi ; done < input.txt

I’ll usually pipe the output to tee so I can watch what’s going on:

while read -r file; do if [[ ! -e $file ]]; then echo "$file|error" ; fi ; done < input.txt | tee error_report.txt

CommandLineFu: Split a Text file based on line numbers

Assumptions: I have a text file that contains 25+ Million lines, I want to split them into 100,000 line text files.

#!/bin/bash
x=1
y=100000
z=1
while [ $z -le 26 ]
do
    sed -n "$x,${y}p;${y}q;" tbl_001.txt > "t$z.txt"
    x=$(( $x + 100000 ))
    y=$(( $y + 100000 ))
    z=$(( $z + 1 ))
done

If you want to split by a different amount change the “y” variable, and the + whatever number to the number of lines you want.

The “z” variable is used as the filename, and the cutoff point.  If my original file only had one million lines I would change the “while” condition to 10 instead of 26.

I’m sure there’s a way to have the machine do the math for me.  But I don’t have the patience to hunt down how to do this right now.  I imagine it would have something to do with storing the line count (wc -l) in a variable, prompting the end user for the max line count (read $maxcount), and looping until the file is completely done.  (Not sure how to do this last part).  A project for another day.

Lotus Notes NSF Repair [8.5.2]

Error: “Database is corrupt; cannot allocate space” when opening database after server upgrade.

Problem

You have recently upgraded your Domino server from a prior Domino release. When users attempt to open their mailfile on the server, however, the following error displays:

“Database is corrupt; cannot allocate space”.

Resolving the problem

This error often indicates database corruption. In this case, you may be able to work around the issue by running Fixup, Compact and Updall as follows:

  1. fixup -f (This causes Fixup to check all documents in the database.)
  2. compact -i -c -d -K (ignore errors, copy-style, delete view indexes, set large UNK table)
  3. updall -R

 

MS Access: Get Extension from File Name

 IIf(InStrRev([name],".")>0,Mid([name],InStrRev([name],".")+1),"")

Assumes a single field labeled [name] contains a filename without the leading path. Should a file not have an extension, or a period contained within the resulting value will be blank.

CommandLineFu: Basic Filetype Report

A little more find magic:

find . -type f -printf '%d|%k|%f|' -exec file -F"|" -pi '{}' \;

Explanation:

find .

Start searching within the working directory.

-type f

Return only files, not directories

-printf '%d|%k|%f|'

Output tree depth, then a pipe, file size in bytes, then a pipe, then the filename without the leading directories, and a pipe.

-exec file -F"|" -pi

Execute “file” command on file found using a pipe delimiter (-F”|”)instead of the default “:” colon, additionally do not change last access times (-p), and output the file type in mime format (-i).

for dir in */; do a=${dir%%,*} ; find "$dir" -type f -printf '%d|%k|%f|' -exec file -F'|' -pi '{}' \; > "a$"_LIST.TXT & done ; wait

I use the above FOR loop when I have many folders that I need to generate a report for.

In this case all of my top level folders are named after a person “Last, First”, so I create a variable cutting everything after the first comma. I use that variable (the last name) to create an empty text file.  The results of the find command are appended to this text file.

The only obvious pitfall is if two folders have the same value before the comma.  Or two different people have the same last name.

CommandLineFu: Create Allegro Export Report

for i in {001..999} ; do a=`find ./$i/Native/ -type f 2>>/dev/null | wc -l` 
&& b=`find ./$i/Text/ -type f 2>>/dev/null/ | wc -l` 
&& c=$(($( wc -l ./Data/loadfile.txt 2>>/dev/null | awk '{ print $1 }')-1)) 
&& echo -n "$i," && echo "$a,$b,$c" ; done

Assumes all Allegro exports are located in the same top level directory, and follow a simple alpha-numeric sequence.

In the future I plan on building a simple script around this that prompts for a top-level directory so it can be run from anywhere on the network.  Additionally, storing the report to a text file and running subsequent calculations to determine any time a line item doesn’t match perfectly between Native, Text and Record count.

Allegro Export FOR loop

CommandLineFu: AWK – comma quote delimited file

awk -F'^"|","|"$' '{ OFS="|" ; print $2,$3,$4 }' somefile.txt

Use awk to parse a comma delimited file.

OFS=”|” outputs to a pipe delimited file.

Add one to all column references, $1 = $2, $2 = $3, etc.

CommandLineFu: Bash Netcat Copy

Have you ever had to copy millions upon millions of little files across your network very, very quickly? Have you exhausted all of your other command line hacks yet? Of course you have, or you wouldn’t be reading this. (Or you’re my mom.)

Ok… I get that the audience for this type of thing is rather limited. But this is one of those posts that will get more hits by me, then anybody else. This is strictly for demonstrating how to send thousands or millions of little files across a network using bash, tar, and netcat. (Mom you can stop reading now.. I don’t make a cameo in the video… you can skip this one. Thanks for the click though.)

The Code: One-liners are a beautiful thing

##Talking Box

tar -cz [source_dir] | nc [destination_ip] [destination_port]

##Listening Box

nc -l -p [local port] | tar -C [destination_dir] -xzf -

The setup:
Running cygwin-X from one of my XP boxes I tunneled into two different linux boxes (krispc7 == ubuntu 11.04 && kris@bt == backtrack 5).

ssh -X kris@krispc7
ssh -X kris@bt

I did this to work entirely within a native linux environment mainly because I’ve only ever done this with cygwin in the past. Also, so I can demonstrate everything on the same screen using terminator (my favorite GUI shell) and not have to run multiple desktop recorders. I don’t actually need the X forwarding, and I’m sure that my performance was lacking because of this. Additionally the files being copied were on a separate windows file server (we’ll call him e5). So that throws the whole speed thing out the window. Combine that with my extremlely verbose switches and you could probably print the files out of one machine, and physically scan them into the other machine quicker than the actual copy process took place.

Like I said… for demonstration purposes only. Running the compression and netcat instance on a third party machine is just plain stupid in this situation if you’re trying to move stuff really fast (not to mention that this particular hack box has no legs at all). The ideal environment would be to run the talkie box command on the actual talking box.

Moving on…

I ssh into bt (i know everyone roots into their bt boxes… but I don’t allow root to ssh anything), go to the network shared directory on e5 that contains the subfolders with the millions of little files and initiate the talkie side of the command. I then ssh into krispc7 and initiate the listening side of the command.

…Actually it’s the other way around… but you get the idea (“YOU”, is me talking to myself in my own post. Now I’m omnisciently referring to myself in the third person twice removed… and you thought you had problems.)

So listening box is listening, and talking box is waiting for me to hit enter. In the bottom right of the video is a simple while loop I used to count the number of new files in the destination directory.

I let the copy/nc job run for about ten minutes before I killed the video. But I cut a lot out while editing… so 10 minutes happens in less than two. (Who really wants to watch a video of files being copied?).

What’s happening here you ask?
Each file is being compressed on (what is supposed to be) the local machine and instead of being output to an individual zip or tarball file I’m simply redirecting the compressed data into netcat which sends the information over a tcp connection pointed at a specific port. The listening box in turn is monitoring the port defined (9998 in my video) for any and all incoming data and redirects it to be decompressed in the output location of choice.

Maybe tomorrow I’ll run a test that involves copying a bunch of stuff back and forth between two high-end machines (without any man in the middle), and compare the speeds when using different types of compression. Then compare those to a standard scp, windows drag and drop file copy, and my favorite… xxcopy.

Until then, enjoy the show. (Always launch the videos in full screen to watch in HD).

HOW TO: SKYPE for Dummies (and loved ones)

You can do it!

So I went to the doctor the other day and he said that I’m developing a form of ‘Repetitive Stress Injury’ in my neck.  My neck of all places!  I can only attribute this to how many times a day I shake my head back and forth followed by a self administered forehead slap. Why you ask?  Because no one in my tiny little sphere of influence knows how to use Skype?

Why am I such a hardcore proponent of Skype?  Because anything your app can do, Skype can do better!

  • Instant messaging (for all you keyboard junkies)
  • Group chat and Group video conferencing (few can compete with the group video conferencing feature)
  • A real phone number (you choose the area code)
  • Free calls to other Skype users (HELLO!)
  • Call forwarding (forward incoming skype calls to your cell phone)
  • Visual Voicemail (first to come up with this one, everyone else has been copying since)
  • File Transfers (not the fastest, but extremely easy to use)
  • Desktop/Screen Sharing (great for one on one How-To sessions)
  • Platform independant (OSX, Windows, Linux, mobile devices)
  • Integrates seamlessly with Facebook (see your news feed from the Skype home window)
  • Send and receive SMS messages (Can’t say I’ve ever used this one, but I did use a service once that converted my voicemails to text messages and sent them to my cell phone — kind of cool, although wrought with inaccuracies )
  • Unbelievably cheap international calling rates (for calling those people in countries where the internet doesn’t exist yet)
  • Outbound calling to anywhere in the US for $30 a year (untouchable)
  • An unbelievable amount of gadgets and specialized devices built just for Skype (my personal favorite –> Bluetooth Retro Handset [shameless plug])
  • A countless number of add-on apps (you name it — call recorders, faxing capabilities, lie detectors, translators, games, etc. etc.)
  • Integrates directly into web browsers (recognizes phone numbers on websites and converts them to links == single click to call)
  • To Go – Fake Forwarding (amazing feature that can be re-purposed for a number of clever uses)

The list goes on kids.  While some features cost a buck or two, most features the casual user will ever need are completely FREE.  Skype is a text book example of how to find success using the Freemium business model. Below is the first in what will likely turn into a series of Skype How-To videos.

This one is aimed at all those loved ones out there still using tin cans and string to reach out and touch when you’ve got a brand new, shiny, ultra-expensive mac laptop sitting in front of you. I can do separate videos for Windows and Linux if need be… just ask.

CommadLineFu: TIME STAMP HISTORY

A little gem I came across today in the ##linux channel on freenode.

$ export HISTTIMEFORMAT="%F %R "

Inserts a time stamp into the history command.

RTFM Coffee Mug

I’m a night person.  There’s no doubt about it. And, according to very reliable sources, I have been this way since birth.  I’ve sent memos out, but the rest of the world has yet to modify their schedule to coincide with the hours in which I am most productive.  My clone, while good to have around when an extra liver is needed, would be useless in this regard as he would prefer to sleep the day away while I do all the work.  Furthermore, my attempts at post-natal gene modification of my peer group (tried to inject bat DNA into co-workers) didn’t yield the results I was hoping for, and now the girl in HR just looks at me funny.  Alas, I’m forced to modify my schedule to coincide with the rest of you day-walkers.  (I’m a giver… I know).

Unfortunately for me… I’m up till all hours of the night because my brain decides to finally join the rest of my body around midnight.  This, in turn, results in me losing about 2 to 5 hours of sleep each night.  By Friday morning I often find myself getting kicked off the subway on my way to work because I resemble the living dead.

To date, the best solution I have found to this problem (that doesn’t include using my friends as guinea pigs) has been caffeine.  The only downside to this solution is the logistics and ramp up time it takes to get said chemical into my bloodstream.  Walking around the office dragging an intravenous caffeine drip once again drew the ire of human resources.  To make matters worse, this ramp up time is wrought with all sorts of external stimulus that requires the use of my higher brain functions, namely speech.

In the spirit of single-handedly killing flying mammals I found an interactive portable caffeine container that also wards off those early morning requests without having to strain anything other than my basic motor functions.  It’s called a mug.  But not just any mug.  Usually a mug in my hand combined with an eyeball dangling from my skull is enough to repel the early morning onslaught.  But, some still require that I spell things out for them.

In beautiful eggshell white helvetica font is my canned response for all the ‘how-to’ requests that are asked of me prior to moonrise…. RTFM.

Shameless plug disclaimer:

From time to time I will use my webspace to touch on geek-ware or services that I find interesting.  While I’m not getting paid to hock these wears at the moment, steps are being taken to allow for referral links that would generate a small commission for me on certain products.  In light of this, and to assure everyone I’m not just a greedy capitalist pig, I will only provide these referral links for widgets and gadgets that I actually own and have purchased with my own hard earned money, or if it is something I really want (Christmas list page coming soon).  I will always distinguish between the stuff I own and endorse by providing a video or still shot showing me with said product.  Feel free to flame at will.

PROTECT YOURSELF: SAFE BROWSING THE EASY WAY

Browsing securely with firefox extensions from kris wasserman on Vimeo.

When you park your car at the mall do you leave all of the windows down, keys in the ignition, and your wallet sitting on the passenger seat?  Of course not, then why do you do it when you’re browsing the web?  The next time I get a call because you clicked on something you shouldn’t have, or you decided to run the latest and greatest in fake anti-virus software I’m just going to format your hard drive and install linux on your machine with the edubuntu kidsafe packages installed to protect you from yourself.

Help me, help you.  Take 5 minutes to protect yourself from the evil doers out there on the interwebs and install these firefox extensions.

1. Adblock plus — https://addons.mozilla.org/en-US/firefox/addon/adblock-plus/?src=hp-dl-mostpopular
2. Adblock plus pop-up blocker — https://addons.mozilla.org/en-US/firefox/addon/adblock-plus-pop-up-addon/
3. Redirect Remover — https://addons.mozilla.org/en-US/firefox/addon/redirect-remover/
4. Browser protect — https://addons.mozilla.org/en-US/firefox/addon/browserprotect/
5. WOT — https://addons.mozilla.org/en-US/firefox/addon/wot-safe-browsing-tool/
6. No Script — https://addons.mozilla.org/en-US/firefox/addon/noscript/

They’re all free, and for the most part, easy to use.  The only add on that requires the use of anything north of your brain stem is the ‘No Script’ add on.  If you do not plan on reading the instructions for this one prior to installing it, please watch the video for a demonstration on how to use it.

Also covered in this video, tips about secure http transmission, and setting up temporary fake email addresses.

This video broke the record for my longest tutorial to date, but I felt that this is one of the more important topics that I will ever discuss on this site.  There are no safe neighborhoods on the internet.  All of the bad guys live right next door to you.



IPHONE QUICKTIP: TAKE A SCREENSHOT

One of the best oversights in clever programming ever has got to be Apple’s auto-correct feature.  It’s almost as if the spell check software was replaced by Freud.  So you’ve been texting back and forth with someone and your phone decided it was smarter than you and inserted the text you meant to say.   Now you need to take a screenshot of your conversation and share it with world.  To do so, hold down the home button (the only round button on the face of the phone).  While holding down the home button, tap the ‘wake/sleep’ button on the top of the phone.  You’ll see the screen flash to let you know that the screen contents have been captured.  Jump back to your home screen and launch the camera or photos app, and you’ll see your photo.  Now upload that thing to facebook and make mom proud!

CommandLineFu — COPY FILES WITHOUT CHANGING ATTRIBUTES

xxcopy “k:\foo” “k:\bar” /e /tc /h /bb /k /pb /yy /oN..\barfoo.log

/e           ==           copy all files and folders recursively
/tc          ==           do not modify date and time stamps (create, mod, and access)
/h           ==           include hidden files
/bb         ==           skip file if it already exists
/k            ==           preserve source permissions
/pb         ==           display progress bar
/yy         ==           automatically reply “YES” to all prompts
/oN..\barfoo.log    ==           create a logfile in the directory above working directory named barfoo.log

( go to http://www.xxcopy.com/xcpydnld.htm to download the latest version of xxcopy )

When running my own copy jobs I typically disregard the ‘Progress Bar’ switch because it has to sit there and calculate the number of files about to be copied (useless to us geeks). Eliminating the /PB switch allows the copy process to begin immediately.

If you want to get real clever create a batch file containing a single xxcopy command for each folder that you want to copy, and create a script to launch each one individually.

Copy Script 1:
xxcopy k:\copytest1 k:\outputtest1 /e /tc /h /bb /k /yy /oNcopy00.log

Copy Script 2:
xxcopy k:\copytest2 k:\outputtest2 /e /tc /h /bb /k /yy /oNcopy02.log

Copy Script 3:
xxcopy k:\copytest3 k:\outputtest3 /e /tc /h /bb /k /yy /oNcopy03.log

Launch Copy Scripts Script:
@echo off
##create destination folder structure
xxcopy k:\copytest k:\outputtest /t
##call on individual copy scripts
start call k:\bat0.bat
start call k:\bat1.bat
start call k:\bat2.bat
start call k:\bat3.bat
start call k:\bat4.bat
start call k:\bat5.bat
start call k:\bat6.bat
start call k:\bat7.bat
start call k:\bat8.bat
##kill unnecessary shells
taskkill /im cmd.exe /f
exit

( Change video quality to HD/720p to watch in full screen )
___________________________________________________________________________________

CommandLineFu – REMOVE BATES NUMBER REFERENCE FROM OCR TEXT FILES

find . -type f -iname “*.TXT” -exec sed -i ‘s/<< …-.-…….. >>//g’ ‘{}’ \;
*Assuming original Bates number uses the following format ABC-X-01234567

Depending on how cygwin is configured be sure to convert files back to ‘DOS’ format after using sed
find . -type f -iname “*.TXT” -exec unix2dos ‘{}’ \;

The find command is good if your files are not organized in a cleanly numbered subfolder structure, or are mixed in with other file types.  Usually simply globbing will do the trick a bit faster:

sed -i ‘s/foo/bar/g’ IMAGES/00/0[0-9]/*.TXT ; unix2dos IMAGES/00/0[0-9]/*.TXT

sed == http://sed.sourceforge.net/sedfaq.html

_____________________________________________________________________________________

WIN7 QUICK TIP: WHERE’D MY MENU GO?

Windows Explorer Win7 menus missing
What happened to the menus in Windows Explorer?

 

Being that Microsoft decided to wait something just short of an eon to release a new version of it’s operating system many people have become extremely comfortable with windows XP.  Even though Windows 7 has been around for some time now, it’s finally starting to penetrate the marketplace on a large enough scale to mess with the Mom’s and Dad’s of the world.

To view the old “File – Edit – View – Tools” pull down menus in Windows Explorer simply press the ALT key on your keyboard.

Windows Explorer Pull Down Menus
Press the ALT key on your keyboard to view XP style pull down menus in explorer

 

 

 

 

 

HOW TO: ROTATE TEXT (PowerPoint 2007)

_____________________________________________________________________________________

(Change the video size to fullscreen and the quality to HD in order to view the video in actual size)

During one of my union required coffee breaks I was shooting the breeze with a friend in the same office building as myself.  I have often commiserated with her over the many long hours that I have spent working on various PowerPoint presentations.  It wasn’t long before the Q&A started.

Microsoft has made a number of changes to the user interface in recent Office releases which has left the people who don’t use the application on a daily basis almost entirely in the dark.  The menu bar has been replaced entirely by what is now called ‘The Ribbon”.  The trick to the ribbon is knowing that the menu options and features that are displayed can and will change depending on what you select.

This short video will show how to grab and rotate text in a couple of different ways.  First by simply dragging the green rotate ball that is displayed at the top of a selected text box. If you were to hold down the shift key while rotating you’ll notice that the rotation will snap into place as opposed to freely turning on point.  This option is great if you quickly need a perfect right angle rotation.

If you really want to be precise, the other option is to select the text box and use the rotate feature from the “FORMAT” menu on the ribbon.

_____________________________________________________________________________________

Day One… a new project is born.

I often find that a good way to get started, in terms of getting anything down is to simply start writing.  Just let the words fall on the page.  This will usually end up being the equivalent of rubbing dirt all over a clean little league uniform after sitting on the bench the entire game… but hey at least I did something, right?

So this is me making my permanent mark on the interwebs.  What little I have to offer will hopefully fill these pages in the years to come to create a library of useful little snapshots of Kris’ geek side.

Perhaps grandchildren will one day mock the need for me to physically pound keys hunched over a desk, or maybe the hive-mind created by our iTunes brain chips will kill off all need for entertaining reading or technical research.

But, in order for anything to meet it’s end, it must first begin.  So I am asking for input.  Simply reply to make a request for a how-to video.

If you had an all purpose geek at your disposal what would you ask of him?  _____________________________________________________________________________________