The best kittens, technology, and video games blog in the world.

Wednesday, January 03, 2007

iPod hack for better statistics

2007-03-01 overall top artist chart for taw
I'm in a love-hate relationship with Apple products. They are usually cool. But most of the time, they have really stupid design issues that are hardcoded and cannot be fixed by me. One simple example - I cannot tell my iPod to simply forget about the concept of "albums" (it's friggin' trance, not some 1960s music for which people actually cared which album it was on).

One such design screwup is recording, for every song, number of times it was played, and the last time it was played, but not the full log.

And no, the full log wouldn't be too big. By recording song id and time stamp (4+4 bytes) on each play, having iPod running continuously for 10 years, with one song every 3 minutes, the log would still be a mere 14MB. It's hard to get an iPod smaller than 8GB nowadays.

Lack of full log really sucks when it comes to software that coordinates iPods with They have two choices:

  • Ignore play count. Sumbit just the last time something was played. Every song is sumbitted just once, even if it was played 1000 times on a loop.
  • Add fake entries, so that every song is sumbitted exactly the right number of times.

I modified my bridge to give user a choice which way they prefer. The default is still going to be the former option - exact timing, bad statistics. But personally, I like the latter one - bad timing, exact statistics, way more.

I think it's not exactly how clients are supposed to behave, but it needs explicit user switch, and it results in better charts. And what's really about if not accurate pretty charts ? ;-)

Just grab the new version of bridge, and set $add_fake_songs_to_make_playcount_match in config.rb whichever way you feel.


Anonymous said...

Got any instructions on how to use it?

I tried amending the config file to suit and then ./ bot get's (play count first) and the ./as_submit (in terminal obviously). get_play count seemed to work, it displayed a list of tracks that are definitely on my iPod and have been played. get rating did nothing, but then again I have no ratings on any of my tracks so maybe that's right

as for as_sub I am not getting an error messages but it has been 'busy' for about 20 minutes now and has not released the cursor

Anonymous said...

not to worry.. I found this

Unpack it (tar xvzf tarball.tar.gz).

Edit config.rb and put your user name and password there (and any other settings you want to change)

Make sure your iPod is mounted somewhere. Usually it's something like /media/sda2/ or /media/sdb2/. If it's somewhere else, just edit config.rb.

Run ./get_play_counts.rb. It will read list of songs you played from your iPod and print it.

Does it work ? Great.

Send the songs to
./get_play_counts.rb | ./as_submit.rb"

on your site in another topic and I now have it working.


taw said...

Great :-)
I'll add some README to the next release.

Anonymous said...

Hi does this only work on linux or Mac, I cant get it to work on Windows, sorry im really confused. Also I do not know what you mean by Make sure your iPod is mounted somewhere. Usually it's something like /media/sda2/ or /media/sdb2/. If it's somewhere else, just edit config.rb.

I dont know what mounting your ipod is, I think mine is mounted on the I drive but im really not sure. Please help me out

taw said...

Anonymous: Yes, this only works on Linux/Mac. You could get it working under Windows (with Cygwin or sth), but it's really too much work. I think there are other programs for Windows that do the same thing, just try one of them.

Anonymous said...

Thanks alot for getting back to me so quickly, I was a bit skeptical about commenting coz I wasnt sure if you still looked at this. If you know any of the Windows programs that can also do this, id really like to know.

Thanks again

taw said...

Anonymous: Sorry, but I don't use Windows, so I don't really know.

Anonymous said...

Damnit I really wanna do this as ive only recently signed up for and my stats suck but ive listened to so much music on my ipod and wanted all my stats updated. is there any file I could send you with all my play counts and then I could give you my login and pass and you could do it for me? Pretty pleassseee, im dying for this

taw said...

Anonymous: If you really really want that, you can mail me files "Play Counts" and "iTunesDB" in iPod_Control\iTunes directory on your iPod.

But that wouldn't even be very useful - won't let you submit songs that played before the ones you submitted most recently. That's for spam protection. And play counts on iPods are cleaned by iTunes and other iPod management software every for no particular reason all the time.

It really only works well if you sync iPod to often.

alkos333 said...

I changed the option to submit the proper number of play counts, instead of the time stamps. I then ran ./get_play_counts.rb | as_submit.rb and it reported success for all posts. However, when I went to, I showed me the following message:

Some tracks you submitted have not been added to your profile for the following reason:
Spam protection triggered: You submitted a track dated earlier than your last submission.

taw said...

alkos333: has very annoying "spam protection" feature that you can only submit songs in order they were played. It also doesn't inform the submitting program that anything went wrong :-/

So if you want to submit songs you played on your iPod to you must do so before you use any program that submits there. This is very annoying but I cannot really do anything about it.

Well, I could check date of the last song you submitted on and then fake timestamps maybe. Or lobby them to make exception in their spam filter for my script. I'm not sure what's the right thing to do.

Unknown said...

Anon.: Yes, this only entireness on Linux/Mac. You could get it excavation low Windows (with Cygwin or sth), but it's truly too much learning. I conceive there are otherwise programs for Windows that do the equal action, right try one of them.
Cheap Ipod Shuffle