Using home directory as GTD inbox - version 2

The GTD software I described a few weeks ago evolved quite significantly since then.

Fortunately my inbox is still empty:

$ inbox_size
Your inbox is empty.

It can be used in two modes - either single-shot report of inbox contents with inbox_size, or continuous screening mode plus UI notification with inbox_size_notify. inbox_size.rb is a library (symlinked from /home/taw/local/bin/inbox_size) which finds all items in all my inboxes. It also handles special items:

• Unread emails in Gmail inbox
• Uncommitted changes to one of the repositories
• Music log not committed to last.fm
• Passwords file chanced since last encrypted copy
  
• Last backup older than 3 days
• Any things I wanted to be informed about

The code

The main code is in inbox_size.rb:

require 'time'
require 'magic_xml'

$offline = false

def inbox_ls
  items_whitelist = %w[
    /home/taw/Desktop
    /home/taw/ebooks
    /home/taw/everything
    /home/taw/img
    /home/taw/ipoddb
    /home/taw/local
    /home/taw/movies
    /home/taw/music
    /home/taw/ref
    /home/taw/website
    /home/taw/website_snapshot
  ]

  files = (Dir["/home/taw/*"] +
           Dir["/home/taw/Desktop/*"] +
           Dir["/home/taw/movies/complete/*"] -
           items_whitelist)
  items = files.map{|x|x.sub(%r[\A/home/taw/],"")}

  # Code for handling special inbox items goes here
  # ...

  return items.sort.map{|item| "* #{item}"}
end

if $0 == __FILE__
  if ARGV[0] == '--offline'
    ARGV.shift
    $offline = true
  end
  items = inbox_ls
  if items.empty?
    puts "Your inbox is empty."
  else
    puts "#{items.size} items in your inbox:", *items
  end
end

inbox_size_notify which scans the inbox continuouly and displays UI notifications if it's not empty is:

require 'inbox_size'

max_displayed = 30

big_timer = 5
old_items = []

while true
  items = inbox_ls
  next if items == []

  if items == old_items
    big_timer -= 1
    sleep 60
    next unless big_timer == 0
  end
  big_timer = 5

  if items.size > max_displayed
    displayed_items = items.sort_by{rand}[0, max_displayed].sort + ["* ..."]
  else
    displayed_items = items
  end
  system "notify", "Inbox is not processed", "#{items.size} items in your inbox:", *displayed_items

  sleep 60
  old_items = items
end

Script which displays KDE notifications is:

header = "Notification"
msg = ARGV.join("\n") # "All your base\nAre belong to us"

system 'dcop', 'knotify', 'Notify', 'notify', 'notify', header, msg, 'nosound', 'nofile', '16', '0'

Backup reminder

Since my disk died I became more serious about backups. I indent to have at least regular rsync of my SVK repository and some important files. Here's a script which rsyncs these files from shanti (my main box) to ishida (an old laptop).

t0 = Time.now

rv = system 'rsync -rL ~/.mirrorme/ taw@ishida:/home/taw/shanti_mirror/'

unless rv
  STDERR.puts "Error trying to rsync"
  exit 1
end

t1 = Time.now

File.open('/home/taw/.last_backup', 'w') {|fh|
  fh.puts t1
}

puts "Started: #{t0}"
puts "Started: #{t1}"
puts "Time: #{t1-t0}s"

If backup was successful a time stamp is saved to /home/taw/.last_backup. inbox_size.rb reminds me if I didn't backup for more than 3 days:

  # Time since last rsync
  time_since_last_rsync = Time.now - Time.parse(File.read("/home/taw/.last_backup").chomp)
  if time_since_last_rsync > 3 * 24 * 60 * 60
    items << "Over 3 days since the last backup"
  end

Tickler file

The "tickler file" (/home/taw/.tickler) contains all things I want to be reminded about. Appointments, deadlines, new episodes of The Colbert Report, whatever. Of course usually I want to be reminded before the deadline, not on the deadline, so the date must be some time before the event of interest. Entries in the tickler file look something like that:

Sat Jul 21 05:49:14 +0200 2007
15 days to Wikimedia Foundation validation deadline

It can be edited as a text file, but it's more convenient to add new entries with add_tickler script:

$ add_tickler 24h "New TCR episode will be available"

unless ARGV.size == 2
  STDERR.puts "Usage: #{$0} 'due' 'msg'"
  exit 1
end

due = ARGV.shift
msg = ARGV.shift

due_sec = case due
when /\A(\d+)s\Z/
  $1.to_i
when /\A(\d+)m\Z/
  $1.to_i * 60
when /\A(\d+)h\Z/
  $1.to_i * 60 * 60
when /\A(\d+)d\Z/
  $1.to_i * 60 * 60 * 24
else
  STDERR.puts <<EOF
Usage: #{$0} 'due' 'msg'
Due can be:
* 15s
* 15m
* 15h
* 15d
EOF
  exit 1
end

due_time = Time.now + due_sec

File.open("/home/taw/.tickler", "a") {|fh|
  fh.puts due_time
  fh.puts msg
}

The tickler file is checked by the following code in inbox_size.rb:

  # Tickler items
  tickler = File.readlines("/home/taw/.tickler")
  while not tickler.empty?
    deadline = Time.parse(tickler.shift.chomp)
    msg = tickler.shift
    if Time.now > deadline
      items << msg
    end
  end

The passwords file

Pretty much every website requires an account nowadays. I don't want to reuse password on multiple website, so I generate them randomly (cat /dev/urandom | perl -ple 's/[^a-zA-Z0-9]//g' | head) and keep them in unencrypted file /home/taw/.passwords which I simply grep if I want to login to some weird website again (normally Firefox remembers these passwords anyway, but sometimes it's necessary).

As it would suck to lose all accounts, I AES-256-CBC encrypt this file and keep encrypted copies in /home/taw/ref/skrt/, which is mirrored to multiple servers. As I need to enter my password to encrypt the file, it cannot be done automatically. The most inbox_size.rb can do is reminding me if there's no up-to-date skrt file:

  # skrt up to date ?
  pwtm = File.mtime("/home/taw/.passwords")
  last_skrt_tm = Dir["/home/taw/ref/skrt/*"].map{|fn| File.mtime(fn)}.max
  if pwtm > last_skrt_tm
    items << "No up-to-date skrt available"
  end

In which case I run the following skrt_new script:

t = Time.now
fn = sprintf "skrt-%04d-%02d-%02d", t.year, t.month, t.day
system "openssl aes-256-cbc /home/taw/ref/skrt/#{fn}

Music log

The iPod-last.fm bridge consists of two parts - one which extracts the log from an iPod, and one which submits the data to last.fm. They communicate using very simple format, with lines like that (time is local):

Sumptuastic ; Cisza (Radio Edit) ; Cisza (Single) ; 185 ; 2007-07-11 17:51:27

Nothing in the format is iPod-specific, so I wrote a wrapper around mplayer which logs music it plays to /home/taw/.music_log. It can also randomize songs and search for them recursively in directories. It uses a few extra programs - id3v2 to get song title, artist and album (from either ID3v2 or ID3v1 tags), and mp3info to get playing time.

def mp3_get_metadata(file_name)
  song_info = `id3v2 -l "#{file_name}"`
  artist    = nil
  title     = nil
  album     = nil

  if song_info =~ /^TPE1 \(Lead performer\(s\)\/Soloist\(s\)\): (.*)$/
    artist = $1
  elsif song_info =~ /^Title  : .{31} Artist: (.*?)\s*$/
    artist = $1
  end

  if song_info =~ /^TIT2 \(Title\/songname\/content description\): (.*)$/
    title = $1
  elsif song_info =~ /^Title  : (.{0,31}?)\s+ Artist: .*$/
    title = $1
  end

  if song_info =~ /^TALB \(Album\/Movie\/Show title\): (.*)$/
    album = $1
  elsif song_info =~ /^Album  : (.{0,31}?)\s+ Year:/
    album = $1
  end

  return [artist, title, album]
end

def mp3_get_length(file_name)
  `mp3info -F -p "%S" "#{file_name}"`.to_i
end

def with_timer
  time_start = Time.now
  yield
  return [time_start, Time.now - time_start]
end

randomize = true
if ARGV[0] == "-s" # --sequential
  randomize = false
  ARGV.shift
end

songs = ARGV.map{|fn| if File.directory?(fn) then Dir["#{fn}/**/*.mp3"] else fn end}.flatten
songs = songs.sort_by{rand} if randomize

songs.each{|song|
  time_start, time_elapsed = with_timer do
    rv = system "mplayer", song
    exit unless rv
  end
  artist, title, album = *mp3_get_metadata(song)
  length = mp3_get_length(song)

  next unless length >= 90 and (time_elapsed >= 240 or time_elapsed >= 0.5 * length)

  date = time_start.strftime("%Y-%m-%d %H:%M:%S")

  File.open("/home/taw/.music_log", "a") {|fh|
    fh.puts "#{artist} ; #{title} ; #{album} ; #{length} ; #{date}"
  }
}

It's a good idea to commit the log to last.fm often, but I'm not doing it automatically yet, as network problems with last.fm are too frequent. Instead inbox_size.rb reminds me if there are old uncommitted entries in the log:

  # .music_log not empty and older than one hour
  if File.size("/home/taw/.music_log") > 0 and File.mtime("/home/taw/.music_log") < Time.now - 60*60
    items << "Music log not clean"
  end

Uncommitted stuff in repositories

I sometimes get distracted by some interruption and forget to commit things to repositories.
I wrote uncommitted_changes script which checks local checkouts of all repositories I use (currently 1 SVK and 2 SVN repositories) if there are any uncommitted changes. I use svn/svk diff instead of svn/svk status as the latter finds all kinds of temporary files, and I always svn/svk add all new files when I start coding anyway.

Dir.chdir("/home/taw/everything/") { system "svk diff" }
Dir.chdir("/home/taw/everything/rf-rlisp/") { system "svn diff" }
Dir.chdir("/home/taw/everything/gna_tawbot/") { system "svn diff" }

inbox_size.rb simply checks that output of this script is empty:

  # Uncommitted changes
  uc = `uncommitted_changes`
  unless uc == ""
    items << "There are uncommitted changes in the repository"
  end

Unread Gmail emails

The last kind of inbox items tracked by inbox_size.rb are email inbox items. Google APIs are almost invariably ugly Java-centric blobs of suckiness, so instead of using Gmail API I simply get the list from RSS, parsed using magic/xml.

  # Unread Gmail messages
  unless $offline
    gmail_passwd = File.read("/home/taw/.gmail_passwd").chomp
    url = "https://Tomasz.Wegrzanowski:#{gmail_passwd}@mail.google.com/mail/feed/atom"
    XML.load(url).children(:entry, :title).each{|title|
      items << "Email: #{title.text}"
    }
  end