autoreporter gem

Many times I wanted to display status of something in terminal tab. The obvious way to do so would be something like:

while true
  system "clear" # Clear the screen
  system "rake status" # run command reporting status
  sleep 60
end

Which is nearly ruby -e one-liner with some golfing.

This is good enough if rake status is close to instantaneous. Unfortunately if it does anything nontrivial - like checking with external network service, searching filesystem and so on - then most of the time when you switch to status tab you'll be seeing empty or incomplete screen instead of probably much more useful previous minute's report.

And so I created autoreporter script, which I now packaged into a gem of the same name. To use it open terminal tab and run autoreporter rake status or whatever you want it to run to get status. It will manage screen refreshes, STDERR redirection and such, and has command line switches for things like delay and verbosity.

I used it for over a year now. Initially I thought it will need fancy features like multiple independent report sources, watching file changes, forcing refresh by some keyboard shortcut, and so on, but in reality I never needed any of that.

Before I wrote autoreporter I tried to use rerun but lack of output buffering really kills it.

Test-Driven Development for Optimists

If you do proper Test-Driven Development, writing your tests first, then your machine will call your code - and by extension you as well -a  failure. Over and over, every day you code, and use nasty red color for it just to drive the message:

Failures:

  1) HelloWorld says hello
     Failure/Error: expect(hello).to eq("Hello, world!")

     NameError:
       undefined local variable or method `hello' for #<RSpec::ExampleGroups::HelloWorld>
     # ./spec/hello_spec.rb:3:in `block (2 levels) in '

Finished in 0.00054 seconds (files took 0.13016 seconds to load)
1 example, 1 failure

Failed examples:

rspec ./spec/hello_spec.rb:2 # HelloWorld says hello

This kind of endless negativity cannot possibly be good for anyone - and you'll get it not just for minor mistakes you make, you'll get it for every single test. No wonder people burn out, switch to test-last development, or start making excuses for not writing tests:

Fortunately it's mostly just a matter of phrasing. Thanks to gem install rspec-half_full and setting up proper .rspec:

--color
--require rspec/half_full
--format RSpec::HalfFull

You can enjoy much more positive attitude:

Future successes:

  1) HelloWorld says hello
     Future Success: expect(hello).to eq("Hello, world!")
     NameError:
       undefined local variable or method `hello' for #<RSpec::ExampleGroups::HelloWorld>
     # ./spec/hello_spec.rb:3:in `block (2 levels) in '

Finished in 0.00084 seconds
1 example, 1 future success

Rerun on next step to success:

rspec ./spec/hello_spec.rb:2 # HelloWorld says hello

And not only is this psychologically healthier, it's also more accurate - specs you're writing are not failures, they're just description of future successful code you're planning to write.

Hopefully it will make you at least a little bit happier.

Continuous Integration for personal projects

This seems really obvious in retrospect (so I blame you guys for not telling me earlier), but somehow I never connected the dots before yesterday.

CI is really useful, but it's a massive pain in the ass to set it up for personal projects on own servers. Fortunately it turns out pretty much every CI service website offers CI for free for any software that's open source.

Somehow I have 51 public repositories on github, not counting forks (as well as some private code, but much less of that). Some of them are fairly seriously maintained, a lot is just random stuff I wrote once and I probably will never use again, but I had no reason not to post online.

So I opened free account on Semaphore and setup CI for 4 of them. It mostly required specifying dependencies in Gemfiles (I'm not committing Gemfile.lock files to repositories, this causes more problems than it solves) in a bit more detail, but wasn't too difficult.

The only nontrivial problem was with z3 gem, which depends on z3 library which isn't available on two year old Ubuntu 14.04 LTS which semaphore uses as its only option, so I had to use build instructions like:

echo "deb http://archive.ubuntu.com/ubuntu/ xenial main" | sudo tee -a /etc/apt/sources.list
echo "deb http://archive.ubuntu.com/ubuntu/ xenial universe" | sudo tee -a /etc/apt/sources.list
sudo apt-get update

Followed by apt-get install libz3-dev; bundle install; bundle exec rake spec which do the actual job.

I plan to go through 51 public repositories and decide one by one which are worth getting CI and which probably aren't.

Sensible git diff for json files

JSON is rarely great for anything, but it's very often good enough, so one thing you'll often run into is JSON in git repositories.

Human-edited JSON (for which you should probably use something like JSON5 instead) is reasonable to work with.

Unfortunately typical machine-generated JSON is completely undiffable - it's all one big line, so trying to run git diff or git log -p will produce full content on before and after side, and you'll have no idea what actually changed without copying both before and after to external files, formatting them, and diffing the result - fairly slow and messy process.

Fortunately this is solvable. First, get latest version of my unix-utilities repositories for json_pp script for pretty-printing JSON (or any other json pretty-printer) and put it somewhere in your $PATH.

Then tell git to treat json as special file for purpose of diffing.

To do it globally:

echo "*.json diff=json" >> ~/.gitattributes

git config --global core.attributesfile ~/.gitattributes

git config --global diff.json.textconv json_pp

Or instead you can do it for just one project:

echo "*.json diff=json" >> .gitattributes

git config diff.json.textconv json_pp

You can use similar technique for other sort-of-text-but-not-really files like machine-generated XML.

textconv is only applied for human readable command output, so it doesn't affect any internal workings of git, and if you want to see raw diffs for any reasons you can always use --no-textconv argument to git diff.