Tricking iTunes into having a HTTP/mt-daapd based library

Posted on by
Reply

I have a massive problem- a massive library. It’s great to listen to at home, but what if I wanted to listen to it at work? The nature of my job as a web developer is to get given a task (usually large and complex) and then getting my head down in the code, with some music to accomodate.

As far as solutions go,  I could bring my iPod Nano in, but that’s only 8 GBs worth of music .. and I like a bit of variety when I work. I have also looked into alternative streaming sources to mirror most of the library, but I don’t think the variety of music or the price of services like Spotify are justifiable enough and radio services like we7 are just a bit tedius and annoying to use.

As I have a small, linux based server at home, I tried a number of streaming solutions. Some where I control a continual, singular music stream (like SHOUTcast) and some where I could stream tracks individually from a web interface, like Subsonic.. But it just wasn’t the same experience as having a nice, native interface to use.

For a while, I had been using a combination that sort of worked. Firefox-based music app Songbird has a number of plugins available, one of which connects to a DAAP server and presents it as a library. This solution was OK, but I had to manually update some of the plugin, as it was so out of date and Songbird wouldn’t install it for compatibility reasons. Songbird was also very slow to use and often used a substancial amount of CPU just playing music. It really wasn’t working out.

Times were getting desperate. I had even started learning Objective-C and Cocoa, so I could make my own app, which would have been a long and lengthy solution (but a nice skill to have, I must admit). The files are available on my DAAP server over standard HTTP and I could type in a URL of a file and it’ll play like a standard MP3, but nothing I couldn’t find would catalog and play them together.

It wasn’t till today whilst playing around with an iTunes XML file that I realised that it can play files over HTTP. I simply exported a library which had 1 song to XML (File – Library – Export Library), opened the XML file in a text editor and changed the location of the file to a URL on my home server. Import the library file into iTunes (File – Library – Import Playlist) and hey presto, you’re streaming HTTP files from your iTunes library!

So, If I can trick iTunes into play a song from my library, can I trick it into importing everything? Yes! You’ll need to have the following:

  • A Linux server/workstation, running mt-daapd that is accessible from the internet
  • A Mac which will act as the client (this may work on PCs, but I don’t have one to hand to test this on)
  • Somewhere that runs PHP for you to parse XML (for this tutorial, anyway)

Some of you may be aware that DAAP transmits all of it’s library data over XML-RPC (although a lot of you won’t, as DAAP is pretty much dead now), which is just standard XML with the right parameters. I hit my library with the following command, which gets every single song and saves it to my local machine:

curl -O http://{mt-daapd username}:{mt-daapd password}@{mt-daapd location}:{mt-daapd port}/databases/1/items?output=xml

With that file, I created script in PHP, that’ll parse through all of the song elements and then save it in the iTunes library format:

<?php
libxml_use_internal_errors(true);

$xml = new DOMDocument();
$xml->load("{location of exported XML}");

$xpath			= new DOMXPath($xml);
$q				= "//dmap.listingitem";
$songs			= $xpath->query($q);
$libraryFile	= fopen("{location for new XML}", "w+");

fwrite($libraryFile, "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n
<!DOCTYPE plist PUBLIC \"-//Apple Computer//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n
<plist version=\"1.0\">\n
<dict>\n
	<key>Major Version</key><integer>1</integer>\n
	<key>Minor Version</key><integer>1</integer>\n
	<key>Application Version</key><string>10.4</string>\n
	<key>Features</key><integer>5</integer>\n
	<key>Show Content Ratings</key><true/>\n
	<key>Library Persistent ID</key><string>A5414C8423107D2B</string>\n
	<key>Tracks</key>\n
	<dict>\n");

foreach($songs as $song) {
	$discNum		= (isset($song->getElementsByTagName("daap.songdiscnumber")->item(0)->nodeValue)) ? $song->getElementsByTagName("daap.songdiscnumber")->item(0)->nodeValue : 0;
	$genre			= (isset($song->getElementsByTagName("daap.songgenre")->item(0)->nodeValue)) ? $song->getElementsByTagName("daap.songgenre")->item(0)->nodeValue : "";
	$discCount		= (isset($song->getElementsByTagName("daap.songdisccount")->item(0)->nodeValue)) ? $song->getElementsByTagName("daap.songdisccount")->item(0)->nodeValue : 0;
	$trackNum		= (isset($song->getElementsByTagName("daap.songtracknumber")->item(0)->nodeValue)) ? $song->getElementsByTagName("daap.songtracknumber")->item(0)->nodeValue : 0;
	$trackCount		= (isset($song->getElementsByTagName("daap.songtrackcount")->item(0)->nodeValue)) ? $song->getElementsByTagName("daap.songtrackcount")->item(0)->nodeValue : "";
	$album			= (isset($song->getElementsByTagName("daap.songalbum")->item(0)->nodeValue)) ? $song->getElementsByTagName("daap.songalbum")->item(0)->nodeValue : "";
	$year			= (isset($song->getElementsByTagName("daap.songyear")->item(0)->nodeValue)) ? $song->getElementsByTagName("daap.songyear")->item(0)->nodeValue : "";
	$compilation	= (isset($song->getElementsByTagName("daap.songcompilation")->item(0)->nodeValue) == "1") ? "<true/>" : "<false/>";

	fwrite($libraryFile, "
		<key>".htmlentities($song->getElementsByTagName("dmap.itemid")->item(0)->nodeValue)."</key>
		<dict>
			<key>Track ID</key><integer>".htmlentities($song->getElementsByTagName("dmap.itemid")->item(0)->nodeValue)."</integer>
			<key>Name</key><string><![CDATA[".$song->getElementsByTagName("dmap.itemname")->item(0)->nodeValue."]]></string>
			<key>Artist</key><string><![CDATA[".$song->getElementsByTagName("daap.songartist")->item(0)->nodeValue."]]></string>
			<key>Album</key><string><![CDATA[".$album."]]></string>
			<key>Genre</key><string>".htmlentities($genre)."</string>
			<key>Kind</key><string>MPEG audio file</string>
			<key>Size</key><integer>".htmlentities($song->getElementsByTagName("daap.songsize")->item(0)->nodeValue)."</integer>
			<key>Total Time</key><integer>".htmlentities($song->getElementsByTagName("daap.songtime")->item(0)->nodeValue)."</integer>
			<key>Track Number</key><integer>".htmlentities($trackNum)."</integer>
			<key>Year</key><integer>".htmlentities($year)."</integer>
			<key>Date Modified</key><date>".date("Y-m-d",$song->getElementsByTagName("daap.songdatemodified")->item(0)->nodeValue)."T".date("h:i:s",$song->getElementsByTagName("daap.songdatemodified")->item(0)->nodeValue)."Z</date>
			<key>Date Added</key><date>".date("Y-m-d",$song->getElementsByTagName("daap.songdateadded")->item(0)->nodeValue)."T".date("h:i:s",$song->getElementsByTagName("daap.songdateadded")->item(0)->nodeValue)."Z</date>
			<key>Bit Rate</key><integer>320</integer>
			<key>Sample Rate</key><integer>".htmlentities($song->getElementsByTagName("daap.songsamplerate")->item(0)->nodeValue)."</integer>
			<key>Comments</key><string>".htmlentities($song->getElementsByTagName("dmap.itemid")->item(0)->nodeValue)."</string>
			<key>Normalization</key><integer>4715</integer>
			<key>Disc Number</key><integer>".htmlentities($discNum)."</integer>
			<key>Disc Count</key><integer>".htmlentities($discCount)."</integer>
			<key>Track Count</key><integer>".htmlentities($trackCount)."</integer>
			<key>Compilation</key>".$compilation."
			<key>Artwork Count</key><integer>1</integer>
			<key>Persistent ID</key><string>2D88E2A07E1DB1A6</string>
			<key>Track Type</key><string>File</string>
			<key>Location</key><string>http://{mt-daapd username}:{mt-daapd password}@{mt-daapd location}:{mt-daapd port}/databases/1/items/{$song->getElementsByTagName("dmap.itemid")->item(0)->nodeValue}.mp3</string>
			<key>File Folder Count</key><integer>5</integer>
			<key>Library Folder Count</key><integer>1</integer>
		</dict>");

}
fwrite($libraryFile, "</dict>
</dict>
</plist>");

As above, if you import the file created in the above script (File – Library – Import Playlist), you’ll find all of your songs from your mt-daapd server in iTunes!

Limitations:

  • iTunes doesn’t seem to include any files from your mt-daapd server iTunes DJ
  • There is a slight delay between songs, which can be fixed by putting a small crossfade on
  • As the files aren’t being transcoded, your upstream bandwidth would need to cope with at least the bitrate of your MP3. I’m fortunate enough to have a 1000 kbit upstream, so I find it very rare to have breakups in my streaming.
  • I can’t guarantee this’ll work for file formats other than MP3

Next internal project: A web-based personal finance app

Posted on by
Reply

Since I moved over to Mac from Windows, I found it quite difficult to find a personal finance application that meets all my requirements. There are a few that have pretty interfaces, but when It comes to managing goals and dealing with day to day tasks efficiently (spending by category, net worth etc..) they just didn’t cut it.

So, I’m doing what most developers would do when they can’t find an application that meets their requirements.. Developing my own! I’m planning to start off with a simple personal finance app that deals with day to day transactions with a running total. The transactions will be edited on-the-fly through JSON and the transactions pages will be totally keyboard friendly. I will then move on to friendly visualisations of financial information, goals, virtual accounts and then reporting with custom filtering.

The basics of the app are done. The user can log into their account securely and edit transactions on the fly through the JSON interface. They can also navigate the table using the keyboard pretty easily too, although it needs some tweaking.

A large portion of my development time was spent on the import routine for OFX files. I have only tested this with my current bank export format but OFX is meant to be a standard, so hopefully I shouldn’t see much variation on it. The site will loop through all accounts defined in the OFX file and create any that do not exist already. It will also import all transactions that do not exist against those accounts, skipping those that have been already imported based on a unique hash created by using a number of data points in each transaction entry. Once the file has been processed, a balance function is executed for each account which will then work out the current account balance based on all transactions. I’m really happy with this particular part of the site and I plan on expanding this functionality to other import file formats.

Hopefully by the next update on this project, I will have designed a proper UI (It’s just standard HTML styling at the moment) and I will have also perfected transaction editing, so there should be screenshots for you all to have a look at, if you’re interested.

New things coming soon..

Posted on by
Reply

Hey everyone! Through some luck and a fistful of cash, I managed to nab this great domain! On here, I will be posting my views on certain topics and updates on some internal projects that I’m working on.

I coded the last site on a custom unfinished framework that I was creating at the time. Since then, I decided to use familiar technologies to base my projects on, such as WordPress and Zend Framework, as it was taking too much of my time and I could just use something that has been coded well and thoroughly tested. The site was also hosted on a VPS in the US- which is fine for most. I, however found it fairly sluggish and I wanted something a little more local. After a bit of sourcing, I managed to find a super fast VPS host with a datacenter in Kent- so you should see a significant speed increase now.

I hope you enjoy what I have to offer on here in the future. I’ve left discussions enabled, so I look forward to your comments on my views.

Thanks,
Oli