Picked up at the comic shop yesterday:
- Criminal #7 - part two of Brubaker and Phillips' second arc in this pitch-perfect crime series. Also features an essay by Steven Grant on Robert Altman's film adaptation of Chandler's The Long Goodbye. If you haven't read the first arc, "Coward", the collection is in stores now.
- Knights of the Dinner Table #128 - stories for gamers about gamers. If you've ever played D&D, this is a must-read.
- Jack of Fables #12 - begins a new arc entitled "The Bad Prince". I sense that Pris will be at least half-naked before this one's done.
Following up on my last post: my buddy Todd (Mr. Foxmarks) pointed out that to connect the FireWave to my receiver I just need three mini-jack-to-RCA cables. I didn't realize that back before the digital days, these inputs were how you got surround sound into the receiver. But when I looked, there they were, lonely, forlorn and a little dusty. Now I'll have something to plug into them. Whoopee! I ordered the FireWave via next-day air, so I should be watching movies off my Mini in full 6-channel glory before the week is done.
Saturday morning I was digging through the stack of Netflix DVDs on my stereo looking for something to put on while grinding hunting quests in Nagrand and found this great Coheed & Cambria concert DVD, Live at Starland Ballroom. It was sort of scratched and beat up, and the thought occurred to me that if I ripped the DVD, I could store it in my iTunes library and play it through Front Row. That would be handy.
I recalled using MacTheRipper once before, so I grabbed that again and ripped the DVD to my desktop. Super easy. Then I did a little research and found HandBrake, a tool for encoding DVDs. I used it to convert the ripped DVD to a Quicktime ready mp4, anamorphic, with AAC 5.1 surround sound. This took a few tries and some experimenting with settings based on a few threads in the HandBrake forums. The presets in 0.8.5b1 helped a lot. Now I have the movie in iTunes and can play it anytime I want without having to walk to the other end of the house where my CDs are stored (the DVD was part of a CD/DVD set) or leave the DVD lying somewhere around my stereo as I had been, letting it get abused.
I'm pretty stoked about this and am considering ripping all of my DVDs - I have a couple hundred. The biggest issue with that is adding storage capacity. My iTunes library is using about half of a 200GB hard drive, and I'd need at least that much more space for all of my DVDs, not to mention room to grow. I'm not sure how well iTunes handles spreading library media across drives - when i add a track or movie to iTunes, how does it choose which disk to copy to? Something to look into. Then again I need to back up all that media, and the theater Mini's disk, and my laptop's disk, so it's starting to look like I need 700GB or more in total. I wonder if single disks are approaching 400GB yet.
Then of course, there's the question of whether or not to replace the Mini with an Apple TV. Right now I can download audio, movies, YouTube videos and rip CDs and DVDs directly on the Mini and write them to locally attached storage with several hundred gigs of space. If i used an Apple TV, I'd have to download and rip on my laptop, copy them to network storage (USB 2.0 drives hanging off my the Airport Extreme), and then have the Apple TV copy the 160GB worth that I deem most likely to watch back across the network (or I could have the Apple TV stream from network storage, but I'm skeptical of the WoW latency that would introduce). So I think I'll stick with the Mini for now.
And of course there's the fact that I don't get true 6-channel surround sound from either the Mini or an Apple TV. There's a relatively cheap ($99) analog decoder called the Griffin FireWave that claims to overcome Quicktime's (or is it Core Audio's?) downmixing of AAC 5.1 audio to stereo (read more here, here and here), but the FireWave outputs 6 channels over 3 mini-jack outputs, for which my 2001 Marantz receiver doesn't appear to have the appropriate inputs, so it looks like I'll be out receiver shopping if I want to hear movies or concerts in surround off of any Apple product. Luckily I have an XBox 360 to tide me over until then, but it makes so much noise, and then I have to get off my ass and put the DVD into it, which is what I was trying to avoid in the first place...
When I originally implemented support for subscriptions and preferences in Cosmo's Atom subsystem, I represented their info in sort of strange ways. Subscription entries had a couple of extension elements that specified the sub's collection UUID and ticket key, the entry's title was the sub's display name, and the entry had no content (which isn't technically legal). Preference entries used the title element to specify the preference key and included the preference value as text content. The representation styles weren't symmetrical with each other nor with that for items (which are completely represented by an external format like JSON or EIMML and included wholly in the entry's content element). The client programmer had to learn all these different styles, which is confusing and sucky, and write different framework code for each. Even worse, by not treating subs and prefs as media, we couldn't do things like POSTing and PUTting subs and prefs directly into their collections or GETting them directly out of their collections without wrapping them in Atom entries.
Let's take a step back and think about why we're actually using APP in Cosmo. APP is basically a set of rules layered on top of basic REST principles for allowing the server to create, update and delete resources. When the client wants to create or update a resource with APP, it encloses a representation of the resource inside an Atom entry document. The idea with most APP implementations is that the entry meta-data will actually be useful in some way. But for Cosmo items, subscriptions and preferences, it tends not to be. These resources don't have client-modifiable attributes that correspond to Atom entry data like id, owner, updated. So, modeling them as media resources makes a lot of sense. This is almost like not using APP at all, except for the fact that Cosmo follows the general rules for control and publishing (sections 4.3 and most of 5) and provides service documents (section 5.1), which I think are the two most important parts of the document. This means that Cosmo clients can be written with standard Atom and APP libraries rather than built on top of lower level HTTP libraries.
We do care about Atom entry (and feed) documents when the server is sending data to the client (because the documents include the links to related resources that provide the property of the Resource-Oriented Architecture that RESTful Web Services calls "interconnectedness") but not so much when the client sends data to the server. This means we should probably come up with self-contained representations for subscriptions and preferences that don't piggyback on top of Atom entries (but can be reasonably included as entry content for clients that, say, don't support sending media resources).
XHTML + microformats is an increasingly popular way to represent resources these days. It's easy to parse and generate XHTML in basically any client and server software, and the grammar is already known, so we don't have to come up with a new one and write validators for it. Microformats.org lists a number of best practices for defining microformats, including describing them with XMDP (which itself is just an XHTML microformat).
This week I checked in code that uses XHTML and custom microformats to represents subscriptions and preferences. Profiles and examples can be found at http://chandlerproject.org/Projects/CosmoMicroformats (the profile documents aren't available yet at their permanent non-wiki locations, but they will be shortly). I'd like to come up with examples that display more pleasantly in browsers, but the ones that are there now are perfectly useful. This is my first attempt at defining microformats, and I can't claim any mastery of semantic HTML, so it may well be that there are better element choices than span and div. Feedback is welcome.
Over the last several months I've designed and implemented two HTTP-based protocols for accessing Cosmo: an end user-oriented "feed service" for managing your collections of content items, subscriptions to other people's collections, and your user preferences; and Morse Code, a batch protocol used by Chandler desktop to synchronize entire collections across multiple clients.
When I started working on these protocols eight months ago, I wasn't very familiar with the concept of REST. I'd heard the term kicked around, and I'd read enough about it to know that it was a different architectural style than RPC, but I couldn't articulate the differences intelligibly and wouldn't have been able to tell you why I'd prefer a REST architecture for Cosmo. Still, enough had creeped into my subconscious that I was able to sense some of the deficiencies of RPC that we'd found and drive myself towards REST in the design of the new protocols.
For example, the calendar part of our web UI used JSON-RPC to talk the server, and that presented some clear architectural challenges. The name of the RPC operation and its parameters (including the object to call the remote method on), and sometimes the authentication credentials, were hidden inside the JSON payload in the HTTP request, so the servlet filter responsible for authenticating protocol requests and authorizing them to operate on their targets could not find the information it needed to do its job without processing the request body, a responsibility of the JSON-RPC servlet which isn't executed until after the filter has been invoked. In some circumstances, we had to pass credentials as parameters of the RPC operation and process them manually, bypassing our standard filter-based security choke point and adding explicit awareness of security concerns to additional layers.
By contrast, the resource-oriented Cosmo Management Protocol (an older, HTTP+POX version of the feed service designed for user management and other system administration tasks) uses Basic authentication, maps the HTTP method to the operation to perform on the server, and addresses the resource on which the operation is performed by URI only. This allows the security filter to step in and make decisions about authentication (extracting the credentials from the Authorization header and validating them) and authorization (determining the operation and the resource being operated on and evaluating access control rules) without ever having to process or even peek into the request body. A different software layer (the CMP servlet) is responsible for translating the POX request body into a representation of the resource and updating the resource's state. Acegi Security is usable right out of the box with just some declarative configuration; there are no compilation dependencies between the security and processing layers, whereas our JSON-RPC-based stuff had to use Acegi Security APIs explicitly in order to manually authenticate and authorize the RPC operations.
Another benefit is the individual addressing of CMP resources. This lets the server apply security rules based on a resource's URI - for example, requests to /admin/user/** are only accessible by administrative users, but any old authenticated user can access /account/view to see a web page with is account details, and any unauthenticated user can pull up /account/signup to get a web page that lets them sign up for an account. With the JSON-RPC setup, all requests were POSTed to a single URI, so there was no opportunity to use it to discriminate between security rules. Had we used JSON-RPC for any part of our web Ui other than the personal calendar interface, the RPC implementation layer would have had to manually select and evaluate security rules based on the RPC method call.
I didn't realize any of this at the time that I designed CMP a couple of years ago; like a naive teenager, I just did what felt natural. We wanted a network protocol that administrators could easily script, and I figured using HTTP's built-in verb set would make it easy to script tasks with Perl+LWP or curl. When Bobby proposed JSON-RPC for the Ajax calendar UI, I didn't see anything wrong with it. Neither of us knew any better. We figured hey, there's a Javascript library for it, so let's get something working and move on from there. The problems only became obvious over time, but of course by then we were fairly well invested in the existing JSON-RPC implementation and had to wait for a suitable point in our development schedule before we could consider a wholesale replacement. And that's where the feed service I mentioned above comes in.
I'll talk more about the feed service in subsequent posts and discuss topics that will have you on the edge of your seat like: the difference between the feed as a general purpose data representation and the feed meant to be consumed by a feed reader, the joys of synchronization over HTTP, the folly of inventing your own XML dialect when an existing one will do, how to build an Atom-based web service with Abdera, how I beergoggled GData but saved myself from having to gnaw my arm off the next morning, and how reading Leonard Richardson and Sam Ruby's RESTful Web Services could have saved me a big chunk of the 8 months I put into Morse Code and the feed service.
