Blogging Roller

APP draft #10 is available. I'm still reading it over, but the major changes appear to be:

• Categories can be specified at the workspace and collection level. Multiple category schemes are allowed and both fixed and free-form categories (e.g. tags) are allowed.
• Collection titles are now specified by an <atom:title> instead of an attribute on the <collection> element.
• A new "slug" header has been added for media posts so that clients can specify the file-name to be used for the uploaded file.

I'm especially happy about the category support -- now Atom protocol can do everything that MetaWeblog API can do, and much more. I'll be updating my client and server implementations during the next week.

Here's the abstract of the talk I gave this morning at Tri-XML 2006:

Beyond blogging: Atom format and protocol. Like XML-RPC and SOAP before, feeds and publishing protocols were born in the blogopshere and quickly moved beyond blogging. Nowadays, web service providers are using RSS/Atom feeds and REST-based publishing protocols as lightweight alternatives to SOAP. And developers are finding new ways to combine web services from different sites into new applications, known as "mash-ups" in the lingo of Web 2.0. If you'd like to do the same, then attend this talk to learn about the new IETF Atom feed format (RFC-4287) and the soon-to-be-finalized Atom protocol, which together form a strong foundation for REST-based web services development.

Here's a rough outline of the talk:

• Introduction
• Beyond blogging
• Blogs hit the hit time
• The web is bloggy
• Atom as an alternative to WS-*
  
• Understanding feeds
• Birth of RSS
• RSS 1.0: the RDF fork
• The simple fork and RSS 2.0
  
• Atom: the standard
• Parsing feeds
• Fetching and parsing feeds
• Universal Feed Parser
• ROME utilities
• Windows RSS platform
• Serving feeds
• Approaches for generating and serving feeds
  
• Feed autodiscovery
• Styled feeds
  
• Atom protocol
• Compared to MetaWeblog
• REST based approach
• Introspection
• Collections
• Extending Atom
• Atom protocol in action
  
• Getting a service doc
• Getting collections
• Posting an entry
• Posting an image
• Demo: interacting with an Atom server via command-line
  

And here are the slides: TriXML2006-BeyondBlogging.pdf

Tags: topic:[Atom Publishing Protocol], topic:[Atom], topic:[APP], topic:[RSS], topic:[feeds]

Over the weekend, I put my finishing touches on the (last) two new chapters for RSS and Atom in Action. Tomorrow they'll both be off to copy-editing, typesetting and then to the printers for publication in mid-June.

I really lucked out in the reviewer category. Thanks to Walter VonKoch of Microsoft's Windows RSS Platform team, who not only answered my questions but kindly offered to review the Windows RSS chapter. And thanks also to former co-workers Pat Chanezon and Alejandro Abdelnur, who reviewed the ROME chapter.

By the way Alejandro is back from Asia, blogging again and already coming up with cool new APIs for ROME. Checkout ROME.Mano, a pipeline framework for RSS and Atom feeds.

Via The Aquarium I see that Mark Hadley's work on Web Application Description Language (WADL) is now a Sun Technical Report. WADL provides a way to describe a REST based web application or service so that tools can discover services, generate proxies, etc. As I understand it, WADL is to REST as WSDL is to SOAP.

There's also something new since the last time I looked at WADL. Mark has added a section on the Atom protocol and examples that show how to use a WADL file to replace an Atom introspection document. Looks like good stuff to me. If you need an introspection doc for your REST based web service, why not use WADL?

Via Google, I found that there's also a WADL presentation on-line.

I'd like to thank the folks who developed and run the FeedValidator, a valuable service that let's you know if your feeds validate against the RSS, Atom and commonly used extension specs. The warnings that it issues may be irritating and some can be safely ignored, but they're valuable just the same. I don't particularly like the warnings about <content:encoded> (which Roller now uses, by the way) and the style attribute, but I understand why they're necessary. If you want to whine about something, whine about the crappy RSS specs that we're all stuck with not the folks that are trying to help you understand them.

Update: In Roller RSS 2.0 feeds, we now use <atom:summary> for entry.summary (which is new) and <description> for entry.text (as we always have).

I'm seeing lots of interest in my MS Feeds API post yesterday, sparked by links from Sam Ruby, Dave Winer and Randy Morin. Some people might have gotten the impression that I was criticizing the decisions Microsoft made in mapping RSS elements and extension elements to the Feeds API object model. I wasn't.

I think Microsoft made pretty good choices, given the simplified object model that they're working with. If somebody is using funky RSS, then they mean it. For example, if somebody declares the Content Module namespace and uses the <content:encoded> namespace in their feed, then that's probably the content that they want folks to use. I think that's the philosophy Microsoft used in making those decisions, except for prefering <pubDate> over <dc:date>, which I don't understand.

The problem is, the Feeds API object model is a little too simple. Like RSS 2.0, it doesn't model the common things that bloggers do like having both a summary and content for each item, or having  name and/or e-mail address for each author. That's why people use extensions like the <content:encoded> and <dc:creator> (or prefer Atom, which does a better job of modeling those common things). I hope Microsoft will fix this by improving the object model and if they do, they won't have to make as many choices about which elements to use.

The Windows RSS platform includes a Feeds API that parses all forms of RSS and Atom to a simplified  object model.

For example, an Item object has an Author property and not an author name, author e-mail and author URI which are all possible in Atom. And, an Item object has a Description field and not description and content (as in Wordpress feeds) or summary and content (as in Atom feeds).

So, how does the Feeds API decide how to map elements to this  simplified object model? I did some C# experiments and here are some of my findings. Note that the Feeds API is beta software and will certainly change for the better (I hope) by the time it is released in IE7 and Windows Vista.

Item contains	Feeds API returns
<dc:creator>dave</dc:creator>	item.Author = "dave"
<author>dave@example.com</author>	item.Author = "dave@example.com"
<author>dave@example.com</author> <dc:creator>dave</dc:creator>	item.Author = "dave"    (prefers funky RSS)
<description>my desc</description> <content:encoded>my content</content:encoded>	item.Description = "my content"    (prefers funky RSS)
<pubDate>    Thu, 9 Mar 2006 23:13:04 -0500 </pubdate>	item.Date =    "3/10/2006 4:13:04 AM"    (uses GMT)
<pubDate>    Thu, 9 Mar 2006 23:13:04 -0500 </pubdate> <dc:date>    2004-08-19T11:54:37-08:00 </dc:date>	item.Date =    "3/10/2006 4:13:04 AM"    (prefers core RSS element)
<atom:summary>my summary</atom:summary> <atom:content>my content</atom:content>	item.Description = "my content"

First, it's interesting that those funky RSS elements that Winer dislikes are preferred over the core RSS elements in important places. And second, what if you're not happy with Microsoft's mapping choices in this area?

For example, how do you get both description and content from those Wordpress feeds? Wordpress (and Typepad) uses the <description> element as a summary and the funky <content:encoded> element for the full content (see Winer's own Wordpress.com feed for example). You've got to parse the XML yourself. The Feeds API tries to makes that easy by providing both the XML for the entire feed and the XML fragment for each item, but I think most developers would prefer to have a more complete object model.

See also: What's up with the Windows RSS Platform

Tags: topic:[rss], topic:[atom], topic:[feeds], topic:[ie7], topic:[vista]

You know last week, when I said the book was ready to go to the printers and would be available this week as an e-book? We'll, I was wrong.

While we waited for Atom protocol to stabilize, things changed in the world of C# and Java feed APIs. Microsoft introduced the Windows RSS platform and a pre-release of the Windows Feeds API is available in the IE7 beta. And ROME has come along way too; now with Atom format 1.0 support and a growing list of extension modules. We decided that we just couldn't publish a book on RSS and Atom without covering the Windows RSS platform and ROME in-depth. So now I'm under the gun again, writing away into the wee hours of the night. I should be done by April 14th and, with luck, the book will be out in late May, just in time for JavaOne. That explains my sudden interest in the Windows RSS platform.

The kids hate it, but I think it's for the best. Manning will have the very first book that covers the Atom protocol (with a working client and server), the Windows RSS platform and ROME in-depth. It'll definitely be worth the wait.

Tags: topic:[atom], topic:[rss], topic:[ie7], topic:[atom protocol]

The Windows RSS Platform (or Feeds API) is the feed handling engine that powers the new RSS features in IE7. It will also be included in Windows Vista for use by other applications. Note that here, RSS is a generic term meant to include both RSS and Atom -- the Feeds API supports both. The Feeds API is packaged in a DLL called msfeeds.dll and available to programmers as a set of dual-interface COM objects. Here are the features exposed via the Feeds API.

• Common feed list: list of feeds for current user, organized as folder hierarchy.
  
• Feed store: local cache of feeds, feeds available via abstract object model
  
• Download engine: for managing and monitoring large enclosure downloads
  
• RSS sharing extensions: new XML elements to support bi-directional sync via RSS

The Feeds API gives you access to the current user's feed subscription list, a feed parser that can handle any form of RSS and Atom as well as the IE7 podcast download engine. The parser parses feeds to an abstract object model designed to represent any sort of feed. It handles funky RSS and in some cases prefers the funky elements (e.g. <content:escape> over <description>).

I'd like to learn more about how the Feeds API decides which elements to use, how sync works, and how the whole package compares to the premier Java Feeds API ROME. So, I've downloaded IE7 and started experimenting with the API from C#. I'll be posting more on this topic in the next week or two.

Here are some of the references I've been using to understand the API:

Feeds API docs, specs and whitepapers from Microsoft 

• The Feeds API documentation at MSDN Library
  
• The Simple Sharing Extensions (SSE) for RSS and OPML specification
• The Simple List Extentions for RSS specification
  
• Simple Sharing Extensions for RSS tutorial (in Vista only?)
• RSS Support in Longhorn - good overview
• PDC05 presentation about RSS in Windows Vista

Microsoft employee blogs about the RSS platform

• The Windows RSS Platform described on the Microsoft RSS Team Blog
• The Windows RSS Platform ala carte from the  Microsoft RSS Team Blog
• A blog about the RSS Simple Sharing Extensions from Microsoft's Jack Ozzie
  
• Related: Ray Ozzie's blog post Wiring the Web
  
• FeedBandit developer Dare Obasanjo's feedback on Windows RSS Platform
  
• Windows Live Mail Desktop Beta (RSS feeds and "blog-it" feature)
  

Other blogs about it

• Newsgator's Nick Bradbury's feedback on Windows RSS Platform
  
• PDC'05 blog about the above Windows RSS Platform presentation
  
• Jim Mathies built an aggregator in a weekend with the RSS platform
  
• A simple windows RSS store viewer and exploration of msfeeds.dll implementation
  
• John Udell interviews Microsoft's Amar Ghandi about the RSS platform

Update1: added a couple of new links suggested by Mark Woodman
Update2: added reference to Simple List Extensions
Update3: added link to RSS in Windows Vista presentaton

Tags: topic:[atom], topic:[rss], topic:[ie7], topic:[atom protocol]

I've used code from the excellent Pebble and Blojsom blog servers in the past (and given credit in the Roller CREDITS file). I'd love to be able to contribute back and now there's an opportunity to do that. So to Simon and David (or anybody else hacking those servers), if you want to get Atom protocol working in your server, the easiest way might be for you to bring in some code from Roller. I specifically designed our Atom protocol implementation to allow for sharing and to be free of Roller dependencies.

For example, here's how you'd do it for Pebble:

• Bring the classes from the package org.roller.presentation.atomapi into Pebble (except for RollerAtomHandler, you won't need that one).
  
• You'll also need to bring in the ROME and JDOM jars if you're not aleady using them.
  
• Implement the interface AtomHandler with calls to the Pebble backend, call it PebbleAtomHandler or something similar.
  
• Change one line of code in the AtomServlet method createAtomRequestHandler() to create your new PebbleAtomHandler instead of the Roller one.

And feel free to pepper me with questions along the way. I'd be happy to help and happy to make changes to make this sharing easier. I'm also considering the idea of an Atom Server Kit package in my Blogapps project (on second thought, ROME might be a better home).

When you're done, head over to the #atom channel on irc.freenode.net so we can do some interop testing with MatisseBlogger and other Atom protocol clients.

Joe Gregorio: APP, OpenSearch and Microformats. Get used to seeing them; those small pieces loosely joined are the future of web services.

Joe's talking about the new Lucene Web Services API, which is based on Atom protocol (APP), OpenSearch and Microformats. It's very cool to see the APP already applied outside of the realm of blogs.

I was planning on submitting Chapter 8 of RSS and Atom in Action to Manning today, but Atom protocol draft 7 has appeared. The changes look good and the only really significant one for me is the move from list templates, which allowed indexing into a collection, to next/previous paging as we had in draft 4. I'm going to revise my implementation, Chapter 8 and turn it in on Wednesday. Once that's done, I'll release Blogapps v0.1.

There's a new draft of the Atom Protocol available and I've already started working on updating my client (the BlogClient example from my upcoming book RSS and Atom in Action) and server (in the Roller sandbox) implementations. Surprisingly, the new spec doesn't look all that different from the previous one, so perhaps a weekend of work will do the trick.

My second JavaOne was a great experience, but it was a little stressful because up until last night I couldn't find any of my co-speakers. I spent most of Wednesday preparing to give the whole talk by myself, but luckily for me (and the attendees), Pat and Kevin showed up just in time. Unfortunately, Pat showed up with some very bad news for us at Sun: he's leaving to work at Google.

In the end, I think the talk went pretty well. Kevin did most of Pat and my slides on syndication because we had split the talk 50-50 when we couldn't locate Pat on Wednesday night (and assumed he was still in Paris). He did a good job with the material and added in some interesting points from his experience at Rojo.com where they parse millions of feeds per hour with the Java-based Apache Commons (sandbox) FeedParser.

We were a little disappointed with the turnout. I'd be surprised if the 700+ seat Yerba Buena theater was more than 30% full. The fact that were in a lunchtime timeslot on the last day of the show certainly didn't help. Anyhow, I'm relieved that it's over and ready for a nice long week off.

$entry.displayContent($url.entry($entry.anchor))

Time to test the power of positive thinking: this is the month that Atom Protocol settles down and becomes stable enough for me to finish up the book.