While working on large project manipulating a huge set of data, I thought it would be encouraging to see my progress. The manipulation entailed dumping a record set into a CSV file, editing it in spreadsheet form, and then importing it back into the database.
Since the CSV file has one line per record, I figured all I had to do was count the lines in all the CSV files and I’d have the number of records completed. Luckily, my MacBook operating system is based on openBSD, a version of unix. Unix and all the variants have a powerful toolset of commands that one can link together without having to write a program or script for the simple tasks. The word count command, “wc”, will output the number of lines, words and characters in a file, so constraining it to the lines only gives me the desired output. Using a wildcard for the files I want it to evaluate will cause it to count all the files I’ve manipulated.
wc -l *.csv
444 280-0001 to 280-9999.csv
27 290-0001 to 290-9999.csv
53 295-0001 to 295-9999.csv
Now I only want the total so I just need the last line. There’s a command for getting that too! It’s called, logically enough, “tail”. tail is a very useful command for quickly retrieving the last bit of a file, such as the last few entries of a log. By default it returns the last 10 lines. Since I only want the last line, I used the “-n” switch to specify the number of lines to return. But how do we get the output from wc, which by default goes to the screen, to the input of tail? That magic of redirection is handled by the pipe operator. The pipe is represented by the vertical line character “|”.
wc -l *.csv | tail -n1
So anytime I want to see how many records I’ve got done I can navigate to the directory that my files are in, type that command and find out. What I’m looking for is status at a glance. I don’t go to my mail application all the time to see if I have new mail, the mail icon shows me the number of new mails. It’s the same with text messages, overdue tasks, etc. Here’s where GeekTool (http://projects.tynsoe.org/en/geektool/) comes in. What GeekTool does is allow you to embed dynamic information right into your desktop. I set GeekTool to draw a box on my desktop and put the output of my shell command in it. I also set it to refresh every minute. You can set the colors of the background and text as well as adjust the transparency. I decided to put the box in the lower left of my desktop where I’ll be able to see it.
Now I have the pleasure of watching that counter increase shortly after I’ve completed another bunch of records.