Arduino microcontroller and Ruby to display song lyrics

I got myself an Arduino, Seeeduino version to be more exact, and I'll build a robot based on it. I'm using the word "robot" very vaguely, I just want to make a bunch of fun projects taking advantage of the microcontroller.

The first non-trivial project I did was displaying lyrics for music played from a computer. I'll get to the interesting bits in due time, let's start with the basics.

Lyrics

The first problem was finding lyrics with timing information. I really wanted it to play Still Alive from Portal, but I couldn't find any timed lyrics. And I was far too lazy to find accurate timing myself. Then I remembered - many Stepmania songs already come with timed lyrics. So I wrote a parser to their LRC format. It is much more complicated than you'd expect - we only have two lines of 16 characters each, and even silly songs are much longer than that. So there's quite a bit of logic necessary to break longer lines into pairs of 16-character fragments, and extrapolate their timing. API for this library is very simple: LRC.new(file_handle).each_line{|top_line, bottom_line, time| ...}.

class Line
  attr_reader :text, :start_time, :end_time
  def initialize(text, start_time, end_time)
    @text, @start_time, @end_time = text, start_time, end_time
  end
  def to_s
    "#{@text} [#{@start_time}..#{@end_time}]"
  end
  def text_fragments
    unless @text_fragments
      @text_fragments = []
      text = @text.sub(/\|/, ' ')
      while text != ""
        text.sub!(/\A\s*/, "")
        if text.sub!(/\A(.{1,16}(\b|\Z))/, "")
          @text_fragments << $1
        else text.sub!(/\A(.{1,16})/, "")
          @text_fragments << $1
        end
      end
    end
    @text_fragments
  end
  def text_fragment_pairs
    pairs = ((text_fragments.size+1) / 2)
    pairs = 1 if pairs == 0
    pairs
  end
  def duration
    end_time ? end_time - start_time : 2.0
  end
  def each
    fragments = text_fragments
    time_shift = duration / text_fragment_pairs
    time = start_time
    while fragments.size > 0
      yield(fragments.shift||"", fragments.shift||"", time)
      time += time_shift
    end
  end
end

class Lrc
  def initialize(fh)
    entries = []
    fh.each{|line|
      line.sub!(/\s*\Z/, "")
      line =~ /\A\[(.*?)\](.*?)\Z/ or raise "Cannot parse: #{$1}"
      time, txt = $1, $2
      next unless time =~ /\A(\d+):(\d+\.\d+)\Z/
      entries << [$1.to_i*60+$2.to_f, txt]
    }
    @lines = []
    entries.size.times{|i|
      cur, nxt = entries[i], entries[i+1]||[nil]
      @lines << Line.new(cur[1], cur[0], nxt[0])
    }
  end
  def print!
    @lines.each{|x| puts x}
  end
  def each_line(&blk)
    @lines.each{|line|
      line.each(&blk)
    }
  end
end

Seeeduino board

I'll get back to the computer-side software, let's go to the hardware for a moment. Seeeduino board is awesomely easy to use, it uses mini-USB connection for power (it can take battery power too), for uploading programs, and for emulated serial communication. No extra cabling necessary.



Top pins are digital I/O 0 to 13 (some also work as PWM / analog output, but we're not using this feature here), bottom pins are analog input and power pins. On the left there's USB mini-B, external power in, and three switches - manual/auto reset (set to auto), 5V/3.3V (set to 5V, as we need 5V on output), and USB/external power (there is no external power so it works as a power switch).

There are a few interesting diodes, that you can see blinking on the movie. Red RX and TX are for USB communication - RX is blinking every time we receive another lyrics line, we're not sending anything back here. The green diode is connected to digital pin 13, you can conveniently use it a pulse to verify that your program is actually running and not hanging up somewhere.

LCD display

This is surprisingly complicated piece of equipment. It turns out almost all one and two line character LCD displays in the world follow the same HD44780 standard. Which is quite complex. Fortunately there's a library for Arduino for handling them, whih solves most of our problems.

The first big complication is that it needs 3 different voltages - 5V for electronics, 4V for LCD backlight, and unspecified voltage for liquid crystals which regulated contrast.

We already have 5V of course, that's what USB uses, and what all the TTL uses. Backlight requires 4V plus minus 0.5V according to datasheets, but that's a lie, it's barely visible around 3.5V. I tried to cheat and put some resistor between its positive pole and 5V, but that didn't really work, I would need a very small resistor as its actual power not just reference voltage. I finally figured out how to get 4V with a proper mix of batteries - two 1.2V rechargables, and one 1.5V alkaline, giving it a grand total of, well 3.84V at the moment, I probably need to recharge them. Surprisingly AAA batteries work well enough in AA battery holder without any extra hacks.

I might be wrong, but the third voltage, needed to drive liquid crystals, seems to be just reference voltage, and not actually used to power anything. Setting it too low makes liquid crystals all black, setting it too high makes them all transparent. The idea is to have on ones black, and off ones transparent. It turns out putting 5kohm between ground and contrast pin (achieved by a pair of 10kohm resistors in parallel) works well enough. It seems to drive it to about 0.96V. I'm not sure what's the optimal level, but 1kohm and 10kohm are too extreme for convenient viewing.

The LCD circuit has 8 data pins, only 4 of which are used, R/W pin, which is grounded, so we only ever write there, and two extra control pins.

Data pins are connected to Arduino pins 7-10, control pins to Arduino pins 2 and 12, all for no particular reason, these are just library defaults.

Timing

I first wanted to put timing and lyrics into program driving Arduino, but there are two big timing issues. First, song on computer and lyrics need to start at the same time - what could be achieved either by some Arduino-computer communication, or by me pressing two buttons at the same time. Ugly but possible. The second problem is that Arduino doesn't seem to have any real-time clock, or instruction timer. I might be wrong about that, there wasn't any obvious one in documentation. It has good delay function, but driving LCD requires waiting for it, and we would need to do some nasty guessing how long it took. There's also the third problem that every new song would require uploading new Arduino program.

So I decided instead to send lyrics from computer to Arduino as they are played. It's quite nice because the same program and circuit can be repurposed as a Twitter client or anything else I want.

Arduino program

The program is unbelievably easy. We initialize USB serial line at 9600 baud, initialize LCD, and set pin 13 (driving that green diode) to output mode.

Then if any character is available on serial line - if it's NUL/LF/CR we treat it as clear screen, otherwise we display it. The display thinks it has 40 characters per line, so we can fill it with spaces and we don't need a command to move cursor to the second line. Clearing the screen resets the cursor.

If there's no serial data available we blink the diode and wait 100ms. All the heavy lifting is provided by LCD4Bit and Serial libraries.

And yes - this is C++. I forgot to mention. It doesn't have main() function. As Arduino can run only one program, and it never exits, it has two functions. setup() to start it up, and loop() which is run in a loop automatically. There are some complications here, somehow resetting the serial line causes setup() to rerun, I'm not sure why. Maybe it is a segfault even, it's C++ after all. And LCD doesn't become operational immediately, it takes a second or so and if we write in this time data is going to be ignored. But let's just skip that for now.

#include <LCD4Bit.h>

LCD4Bit lcd = LCD4Bit(2); 

void setup() {
  Serial.begin(9600);
  pinMode(13, OUTPUT);
  lcd.init();
  lcd.printIn("Ready to play");
}

int diode = HIGH;

void loop() {
  int val;
  
  if(Serial.available()) {
    val = Serial.read();
    if(val == 0 || val == 10 || val == 13)
      lcd.clear();
    else
      lcd.print(val);
  } else {
    digitalWrite(13, diode);
    diode = (diode == HIGH) ? LOW : HIGH;
    delay(100);
  }
}

Ruby lyrics driver

Ruby gem serialport provides all our serial communication needs. Just create an object using SerialPort.new("/dev/cu.usbserial-A9009rh4", 9600) (name of serial over USB device will differ), and use #write(data) to write there.

There are just a few more complications, to get good camera shots we need to wait a few seconds between initializing serial communication and starting to send data. 3s would be enough, but it defaults to 15s so I can position the camera etc.

And we want mplayer to start without any delay and without any garbage on the output, so -really-quiet -hardframedrop -nocache and redirecting everything to /dev/null

require 'rubygems'
require 'serialport'
require 'lrc_extract'

class Arduino
  def initialize
    @sp = SerialPort.new(Dir["/dev/cu.usb*"][0], 9600)
  end
  def print(text0, text1)
    @sp.write("\n"+text0+(" " * (40-text0.size))+text1)
  end
  def countdown(i)
    i.times{|j|
      @sp.write("\n#{i-j}")
      sleep 1
    }
    @sp.write("\nGO!")
  end
end

class Song
  def initialize(song_dir)
    @mp3_file = Dir["#{song_dir}/*.mp3"][0]
    @lrc_file = Dir["#{song_dir}/*.lrc"][0]
    raise "No mp3 file in #{song_dir}" unless @mp3_file
    raise "No lrc file in #{song_dir}" unless @lrc_file
    @lrc = Lrc.new(open(@lrc_file))
  end
  def each_line(&blk)
    @lrc.each_line(&blk)
  end
  def fork_mp3_player!
    unless pid = fork
      exec "mplayer -really-quiet -hardframedrop -nocache '#{@mp3_file}' >/dev/null </dev/null 2>/dev/null"
    end
    pid
  end
end

class Timer
  def initialize
    @start_time = Time.now
  end
  def sleep_until(time)
    cur_time = Time.now - @start_time
    to_sleep = time - cur_time
    sleep(to_sleep) if to_sleep > 0
  end
end

a = Arduino.new
song = Song.new(ARGV[0] || "songs/Dance Dance Revolution 6th Mix -Max-/WWW.BLONDE GIRL (MOMO MIX)")
a.countdown((ARGV[1] || 15).to_i)

begin
  pid = song.fork_mp3_player!
  timer = Timer.new
  song.each_line{|text0, text1, line_time|
    timer.sleep_until(line_time)
    puts "#{text0} #{text1}"
    a.print(text0, text1)
  }
  Process.waitpid pid
rescue
  system "kill", "-9", "#{pid}"
  raise
end

What next

If someone has Still Alive lyrics with timing, I'll upload a video of that to Youtube. I thought about Twitter client or something like that, but 32 characters make even 140 seem very very long. The display is also quite slow, so smoothly scrolling letters probably won't work too well. It might be because we use it in write only mode, so instead of waiting for busy flag to become false we simply wait maximum necessary delay. That's just a guess.

Anyway, I have plenty of things to connect to Arduino, if anything interesting ever comes out of it I'll let you know.