Blogging Roller

WIP #6: Feed-based Integration

Dave Johnson — Mon, 8 Oct 2012 10:38:43 +0000

This is the sixth in my series of Web Integration Patterns. Check out the intro at this URL http://rollerweblogger.org/roller/entry/web_integration_patterns

Synopsys

This pattern is about integrating web sites and applications by using standard feed formats to convey timely information, updates, status messages, events and other things from one web application to another.

Motivations

Take advantage of the many tools and services that can produce, consume, aggregate and build work-flows around feeds as a way to integrate web applications.
Use a standards-based approach to ensure that that widest range of feed producers and consumers can be supported. Make it so everybody who needs can read your feeds.

Related Patterns

Search Integration
Web APIs

A feed is a list of entries, each with an timestamp, ID, title, content and metadata like categories and tags. Entries are arranged in reverse chronological order. The entries in a feed can represent just about anything from blog entries, Flickr photos, YouTube videos, source-code change sets or tasks in a change management system. A feed is an XML resource that is available at a URL. If you want updates, then you poll that URL, ideally using HTTP Conditional GET so that you only pull down the feed when it has been updated.

Generally speaking there are two standard feed formats in use on the web: RSS and Atom, both are based on XML. Both use different element names, for example: what Atom calls "entries" RSS calls "items." Because these standard formats are so widely supported, providing a feed is an effective way to share updates from your web site or application.

Elements of Atom (from my 2006 presentation on Atom)

Another flavor of feeds is ActivityStrea.ms, which is essentially a feed format with a schema for representing about 70 different types of activities. These activities can be social network activities like share or friend and they can also be business activities like assign, resolve or schedule. One advantage of using the ActivityStrea.ms standard is that it has both an Atom and a JSON mapping.

Producing Feeds

Feed-based Integration is listed as a basic pattern because it can be very easy to implement. The ability to produce feeds is built-in to many different types of web applications from blog and wikis to continuous integration servers. If you are writing your own web application, you can choose to use XML tools to produce your feeds, a templating engine or a dedicated feed toolkit like ROME. You’ll find plenty of XML tools and templating engines no matter what language you are using.

Which type of feed should you produce? That depends. Atom is the most complete specification and is a true IETF standard, so often it is the right choice. To make the right decision, you have to consider who is going to be consuming your feeds. If your consumers prefer RSS, them give them that. If your consumers prefer JSON over XML, then consider ActivityStrea.ms in JSON flavor.

Consuming Feeds

You can use a wide variety of tools to parse and process feed data. For example, there are many web sites and services that can digest feeds and trigger other events and processing. Services such as Yahoo Pipes and If This Then That can read feeds, process each item and perform other actions based on item values.

Processing a feed with Yahoo Pipes (from Marsh Gardiner's post)

You’ll also find a variety of JavaScript widgets that can parse and display feeds in a website. Most web portal and software with a web dashboard features will include a component that can display feeds. And, of course, you can use a reader that supports RSS/Atom feeds, such as Google Reader or Feedly or FlipBoard, to monitor and read feeds.

If you need to add RSS/Atom reading features to your own software, you can use standard XML parsing tools and, for most language, you’ll find that there are open source libraries specifically designed for parsing feeds.

For more information in producing and consuming feeds, RSS and Atom in Action. Also, you might find my presentation on Beyond Blogging: Atom Format and Protocol useful.

Beyond the basics

Feeds are a great way to do simple integrations, but there are limitations and there will be times you’ll need to go beyond the basics with RSS and Atom feeds. Here's an example. Normally, with feeds, clients have to repeatedly poll the feed URL for updates. This is annoying, and inefficient, even with HTTP Conditional GET. To address this problem, you can setup a PubSubHubub server that will subscribe to feeds and will then notify other subscribers instantly when updates are available, so that those other subscribers don’t have to poll.

Another problem is that, if you don’t poll often enough, you might miss some updates and they may “scroll” off the bottom of the feed before you see them. Feed providers can address this problem by supporting Feed Paging and Archiving, which allows clients to use next and previous links to “page” back to feed items that are no longer in the first page of the feed.

One more beyond-the-basics item to mention is the related pattern Web APIs, which we'll cover later. Web APIs are listed as a related pattern to Feed-based Integration. That's because feeds have been used as the basis for several "Web APIs" or protocols. These protocols specify how to use HTTP POST, GET, PUT and DELETE and create, retrieve, update and delete web resources that are represented as feed entries. Examples are the IETF's Atom Protocol, Microsoft's OData and Google's GData APIs.

That’s it for Feed-based Integration. In my next posts, we'll move in to the Advanced Patterns.

More than bloggy

Dave Johnson — Mon, 7 Mar 2011 06:00:00 +0000

I’ve been meaning to write on this topic for a some time and to explain how I’ve gone from being an advocate of RSS/Atom feeds, Atom Publishing Protocol and things bloggy to being a proponent of Linked Data (video), Semantic Web, RDF and other things that I previously considered to be nuisances. I've also got a new topic and blog series to announce, so here goes.

Once upon a time, I actually said “the web is bloggy” in conference presentations and without shame. It sounds silly and, for a blog server developer self-serving, but it’s true and it’s still true to this day. So much of what we do on the web can be made more useful via feeds. That’s because so many things can be usefully represented as a well formed log entry, i.e. a URL, title, date, description and a chunk of content. No matter whether your web application is Flickr, Twitter or IBM/Rational Team Concert, you can enable all sorts of interesting integrations and mashups by providing feeds.

Atom Publishing Protocol (APP) applied the feed concept to web APIs, using a feed to represent a collection of entries and allowing API callers to use HTTP to create, retrieve, update and delete entries. Entries could be anything of course. In the case of Google’s APP-based GData Protocol, entries are Calendar Events, Picasa Photo uploads or YouTube videos. In the case of the IBM/Lotus Connections API, entries are Forum Posts, Social Network Profiles, File Uploads and more.

Feeds only get you so far

I’m not going to argue that RSS is dead (or Atom) because I don't believe that, but the feeds approach only gets you so far. With feeds, we agree on a small set of common properties, e.g. URL, title, date, description and content. Not all resources on the web have those properties and many resources have a lot more. So in some cases applying Atom means forcing things to fit into the Atom model and in some cases it means inventing new properties for Atom, something that is supported by Atom format.

APIs only get you so far

I’m not going to argue that APP is dead or a failure either, but APP is an API approach and, for web integration, APIs only get you so far. It’s definitely a good thing for developers when a web site or application provides an API, but if you have to integrate a set of N web sites and each has it’s own API and (even if they’re all based on APP) you’re stuck doing N x N point-to-point integrations. APIs are not necessarily the best way, and certainly not the only way, to enable integrations on the web.

Linked Data

Linked Data (W3C page) is a different approach. Instead of forcing everything to fit the blog model, Linked Data gives us a way to define common vocabularies of properties about resources on the web and, most importantly, the links between them. Linked Data enables integration by allowing sites to weave their resources into the interlinked web of data that is the web.

Next: Web Integration Patterns

These days, what's important to me at work is integration and in my mind, Feeds, APIs and Linked Data are different “patterns” for web-based integrations. They can be used in complementary ways. Just as we have defined and cataloged Enterprise Integration Patterns, I think it will be useful to do the same for Web Integration Patterns. I know I'll learn something in the process. In my next posts, I'll expand on this concept and after that, I'll start cataloging, in an informal and bloggy way, the patterns that I'm familiar with. More later...

RSS and Atom part of the stimulus plan

Dave Johnson — Sat, 21 Feb 2009 08:32:13 +0000

Aaron Swartz: As chaunceyt pointed out, the new stimulus bill's implementation instructions require that each government agency report the money it gives out in RSS:

For each of the near term reporting requirements (major communications, formula block grant allocations, weekly reports) agencies are required to provide a feed (preferred: Atom 1.0, acceptable: RSS) of the information so that content can be delivered via subscription.

Pretty amazing to see a government so tech-savvy.

Uncle Sam should contact Manning, who, I'm pretty sure, has stacks of RSS and Atom In Action available for a very good price.

Media Blogging for Roller

Dave Johnson — Thu, 19 Feb 2009 12:57:37 +0000

For the past five months I've had the pleasure of mentoring two San Jose State Univ. graduate students, Ganesh Mathrubootham and Tanuja Varkanthe, who are working on a project for classes CMP 295A and B. They picked one of the projects that I first proposed for Google Summer of Code and then for Glassfish's student outreach program, Media Blogging for Apache Roller. It's turned out to be a major project and the central new feature in the upcoming Roller 5.0 release.

The plan

The basic idea is to make it really easy to upload images, audio and video files to Roller, and really easy to include them in Roller blog posts and RSS/Atom feeds. Of course, the devil is in the details and Ganesh and Tanuja really have those covered. They have put together the most detailed and well thought-out plan and design ever for a new Roller feature. You can find the proposal page and the full Media Blogging for Roller Project Plan (PDF, 2mb) on the Roller wiki. Here's a key excerpt from the project summary:

Roller currently lacks support for the latest blogging features. Roller does allow users to upload any type of content to their blogs and include that content on blog entries as images or podcasts, but lacks tools to make media blogging a seamless experience for bloggers. Interface to manage uploaded files is not sortable and not page-able. Once the user has uploaded a file, which could be an image or a podcast, he needs to explicitly cut and paste the URL into his blog post.

Overall, Rollerâ€™s support for media blogs is currently limited to basic file upload functionality, which is cumbersome to use for creating media blogs. This project will revamp the existing file upload interface to incorporate powerful media blogging features into Roller. Also, successful media management websites such as Flickr and YouTube are driven by a public media library, offering different ways for users to search and locate the content of their like. This feature incorporated into a blog server can make it very powerful and we intend to do that as part of this project.

To get an idea of what this is all about, let's take a look at some screenshots/wireframes taken directly from the project plan.

New media file upload dialog

First up is the new media file upload dialog. There are a couple of interesting things here. We'll have metadata for each upload, including description, tags and copyright message. We'll also have the option of including the file in the gallery. We'll support a media gallery for each blog, and new files added to the gallery are included in a special media feed for the blog.

File Uploads browser

To make it easy to manage your media files, the proposal includes a new upload browser interface, with a tabular and a hierarchical view. Thumbnails will be automatically generated on upload. Search and filtering controls will make it easy to find and operate on the files you are looking for, based on file metadata.

Select media dialog

When you are writing a blog post, you'll be able to browse for and include media files without leaving the blog editor interface. You'll be able to choose the size and orientation of the image or video in the blog post, as you can see below.

Needless to say, I'm pretty excited about these new features, and to looking forward to a major new Roller release, and one where I don't have to do much of the work. If you want more information on 5.0 then check the Apache Roller 5.0 proposal page. And if you want to help out then join the Roller development mailing list and introduce yourself.

ROME 1.0 RC2 on the way

Dave Johnson — Wed, 7 Jan 2009 08:48:11 +0000

Good news for ROME fans. Nick Lothian picked up the puck and is galloping towards the finish line (sorry, I'm terrible at sports analogies).

Nick Lothian on ROME dev:

I've gone and built some preview jars for the upcoming ROME 1.0RC2, ROME Fetcher 1.0RC2 and Modules 0.3 release.

Those jars can be found here: https://rome.dev.java.net/servlets/ProjectDoc...

I've created source and javadoc jars as well as the normal jars - the idea being that I'll get them uploaded to some maven repository.

If you have some spare time, please take a look at these and test them and let me know of any problems. Assuming there are no big issues found I'd like to do a proper release in a couple of days.

Guess that means I should test Propono with RC2.

re: delicious.com Blog Posting tool

Dave Johnson — Mon, 22 Sep 2008 17:56:32 +0000

Oh and by the way, I have not given up on that Delicious.com Blog Posting tool that I wrote about in August. I've got the feed fetching, entry composing parts done now and I'm currently working on the blog posting code. I hope to have something ready to blog about this week.

Updating RSS and Atom in Action?

Dave Johnson — Wed, 9 Jul 2008 16:32:28 +0000

Check out Tim Bray's Atomic Monday post for a summary of recent developments in the world of AtomPub protocol and Atom format. With the uptick in interest in Atom and especially AtomPub protocol's usage in mainstream APIs from Google and Microsoft, I wonder if if would make sense to update RSS and Atom in Action.

The bulk of the book is up-to-date, for what it covers. RSS and Atom haven't really changed and, though the book came out before the spec was finalized, the AtomPub material is basically sound. The ROME material is still current too.

What's missing are all of the new developments: a dozen or more new AtomPub based APIs from Google (GData and OpenSocial) and Microsoft (Live and Astoria) and new tools like Apache Abdera (incubating). Not to mention Windows RSS Platform 2.0 and ROME 1.0 (which is coming soon). Updating the book to cover all of those topics would be quite an endeavor and I'm not sure it's the right approach to covering AtomPub. What do you think?

Yahoo Weather RSS module for ROME

Dave Johnson — Wed, 23 Jan 2008 15:40:22 +0000

Apparently, I spoke to soon about ROME being in maintenance mode. There's an all-new Yahoo Weather module for ROME from Robert "kerbernet" Cooper.

ROME vs. Abdera

Dave Johnson — Tue, 22 Jan 2008 18:18:40 +0000

For Java developers starting out with RSS and Atom, here are some notes to help you figure out the differences between the Java.net ROME and Apache Abdera (incubating) projects.

ROME is a set of Java tools for parsing, fetching and generating all forms of RSS and Atom feeds. The core ROME library is relatively small and depends only on the somewhat creaky old JDOM XML parser. Available separately are modules to support various feed extensions such as OpenSearch, iTunes, GeoRSS, etc. ROME was originally developed and open sourced by Sun Portal dev team members in 2004.

ROME Propono is a subproject of ROME that supports publishing/editing entries and files to blog servers and AtomPub servers. Propono is made up of three parts: 1) a Blog Client library can publish via either the old lagacy MetaWeblog API or the shiny new AtomPub protocol, 2) an AtomPub client that publishes only via AtomPub and 3) a framework for creating AtomPub servers. Propono was developed by Ramesh Mandava and Dave Johnson, based on code from RSS and Atom in Action and open sourced as part of the Sun Web Developer Pack in 2007.

Abdera is a set of Java tools for working with Atom feeds and AtomPub protocol. This includes a parser, writers, an AtomPub client and a framework for creating AtomPub servers. Abdera's Atom feed parser uses STAX, so it uses less memory and is faster than ROME. Abdera's Atom feed support is more comprehensive than ROME's and it supports signatures, encryption, Atom to JSON, extensions for Threading, Paging, GeoRSS, OpenSearch, GoogleLogin, etc. etc. Abdera was developed by IBM and contribued to Apache in 2006.

Now let's compare frameworks. The pros and cons of ROME are:

Pro: complete RSS support, all of the dozen various flavors
Pro: it's generally simple and small, depending only one jar (JDOM)
Pro: easy to understand and use the AtomPub server framework
Pro: MetaWeblog API support
Con: Atom feed support not as comprehensive as Abdera
Con: parser uses lots of memory, slower, JDOM based
Con: community not as active, seems to be in maintenance mode (See also Ohloh stats)

The pros and cons of Abdera are:

Pro: comprehensive Atom feed support, lots more Atom extensions
Pro: faster more efficient parser
Pro: In the Apache Incubator with active and growing community (See also Ohloh stats)
Con: lots of dependencies
Con: AtomPub server framework poorly documented, overly complex (rewrite coming soon)
Con: no RSS support (there is something in Abdera contrib, but it's incomplete).

There you have it. ROME and Abdera folks: think that's a fair comparison? Are you a ROME or Abdera user? How would you like to see these frameworks move forward?

The "all Dave" feed

Dave Johnson — Wed, 9 Jan 2008 15:19:15 +0000

Another thing I worked on over the winter break was an all-Dave feed, a single page and feed that combines all of the various things that I post to the web: blog entries, Flickr photos, del.icio.us links and Twitter activity. So far, this is what I've got: http://rollerweblogger.org/roller/page/feeds.

Happy New Year 2008

Dave Johnson — Tue, 1 Jan 2008 23:54:24 +0000

Happy New Year to all the readers of this blog and all the folks who happen to have arrived here via one of the various planets, spam blogs and Google search result pages that include it. I hope you all have a prosperous and pleasant year ahead. I'm hoping for the same. It's nice to start the new year with some good news and I've got some; check this out:

RSS and Atom in Action has been out for over a year now and just as the phenomenal sales are starting to drop off, something wonderful has happened. The book has been nominated for the prestigious Stephen T. Colbert Award for Literary Excellence. As you can see in the photo above, at least one copy of the book is already carrying the Colbert nominee seal. Avoid the rush and buy your copy of RSS and Atom in Action now.

Those who have read Colbert's I Am America (And So Can You!) will understand the significance of this important award and understand the impact on future sales of the book. To learn more about the nomination process and criteria, buy yourself a copy of Colbert's book and then flip to page 214½. Cheers!

RSS best practices doc published

Dave Johnson — Tue, 16 Oct 2007 22:16:42 +0000

RSS advistory board: The RSS Profile makes it easier for feed publishers and programmers to implement RSS 2.0, offering advice on issues that arise as you develop software that employs the format. For 18 months, the board worked with the RSS community on interoperability issues, receiving help from representatives at Bloglines, FeedBurner, Google, Microsoft, Netscape, Six Apart and Yahoo. The profile tackles the most frequently asked questions posed by developers:

How many enclosures can an item contain?

Are relative URLs OK in item descriptions?

Is it OK to use HTML in elements other than an item's description?

For the answers, read the sections on enclosures, item descriptions and character data, respectively. Sam Ruby announced this morning that the Feed Validator now tests for conformance to the profile, offering 11 new checks for improving interoperability.

Kudos to the RSS Advistory Board. The RSS Profile is an excellent guide to publishing RSS. Plus, it's very nice to see the confusing parts of RSS clarified and to see those clarifications linked to from the RSS spec.

Windows Live Writer + AtomPub

Dave Johnson — Tue, 16 Oct 2007 09:32:34 +0000

Joe Cheng: Iâ€™m starting a (hopefully short) series of blog posts documenting the specifics of how the upcoming AtomPub-enabled release of Windows Live Writer will behave, and what AtomPub-enabled blog* servers can do to ensure the best interop with us. I also hope other blog client implementers can learn from our experience and consider using the same rules and behaviors we do when implementing their own AtomPub support.

Very nice. I'll be watching this one closely, looking for ways to improve Roller's and ROME's AtomPub support.

Creating custom modules with ROME

Dave Johnson — Mon, 15 Oct 2007 11:26:25 +0000

Google Blog Search turned up this nice article on creating custom modules with ROME by Sujit Pal, and a nice mini-review of my book:

Sujit Pal: "However, Dave Johnson's book provides a lot of background information and a lot of nice examples in Java and C#. I would highly recommend it to anyone who needs to get up to speed quickly with ROME and RSS/Atom."

It's good to see people are still buying and finding the book useful. It's been out for over a year now, but I don't think it's really out of date at all. I've been keeping the examples up-to-date as part of the Blogapps project and even enhancing them: the example Atom protocol client code from the book is now part of the ROME Propono project.

Blogapps 2.1 released

Dave Johnson — Tue, 2 Oct 2007 21:49:40 +0000

<a href= "http://www.amazon.com/dp/1932394494?tag=bloggingrolle-20&link_code=as3&creativeASIN=1932394494&creative=373489&camp=211189"> The next releases that I'd like to announce are the Blogapps 2.1 Examples and the Blogapps 2.1 Server.

If you'd like to learn more about the Blogapps examples and server then read The Blogapps Project article at Java.net. Here's a quick summary:

The Blogapps project hosts a collection of useful RSS and Atom utilities and examples from RSS and Atom In Action by Dave Johnson. They're designed to be useful even if you haven't read the book and they're available under the Apache License 2.0 so you can use the code in your applications and you can modify and redistribute them as you wish.

What's changed since 2.0? The examples have been updated to include the latest version of ROME Propono, which means that most of them now support the final Atom protcol spec. The server has been updated to include Roller 4.0 RC5, which also includes Atom protocol support and JSPWiki 2.4. And of course, various bugs have been fixed. Here are the release files, installation instructions and release notes.

This blog entry was posted via Atom protocol and the MatisseBlogger blog-client, which you can see in the screen-shot below (which was also posted via Atom.

What's next? Not sure at this point, but I will do another Blogapps release once ROME 1.0 is released.

ROME Propono 0.6 released

Dave Johnson — Mon, 1 Oct 2007 11:20:19 +0000

The first release to discuss is ROME Propono, which includes a ROME based Atom protocol client library, Atom protocol server framework and an Blog Client library abstraction that supports both Atom protocol and the MetaWeblog API.

I've been working on Propono 0.6 off-and-on since May, keeping it in sync with the latest version of the Atom protocol, testing it against Tim Bray's APE and adding various improvements needed in my other projects. Over the weekend I finally had enough time to get a release out. You can find the full-details at the link below but basically this release adds support for the final Atom Publishing Protocol specification and better support for relative URIs.

What's next? Once ROME 1.0 is released Real Soon Now, I'll get a another release out and I'll probably call it ROME Propono 1.0.

What exactly is RSS anyway?

Dave Johnson — Wed, 16 May 2007 10:48:56 +0000

Is it this http://cyber.law.harvard.edu/rss/rss.html, promoted by Dave Winer?

This http://www.rssboard.org/rss-specification, from the RSS advisory board?

Or one of the other dozen or so incompatible formats.

It doesn't really matter. There's only one Atom: RFC-4287.

Hat tip to Sam Ruby.

TS-6029 - Beyond Blogging: Feeds in Action

Dave Johnson — Thu, 10 May 2007 16:27:29 +0000

As usual, I'm enjoying JavaOne and meeting lots of new people and putting faces to names. I've also been practicing my talk daily so I can give you my best performance, so eat lunch a little early or a little late and come see how it goes. Here are the details:

TS-6029 - Beyond Blogging: Feeds in Action
Friday 05/11/2007 12:10 PM -1:10 PM
Moscone Center Esplanade 307 / 310

Propono 0.5 released

Dave Johnson — Mon, 23 Apr 2007 15:32:10 +0000

ROME Propono 0.5 is a minor bug fix release of Propono. You can get the release files and updated Javadocs from the Propono 0.5 release page.

ROME Propono 0.4 released

Dave Johnson — Tue, 10 Apr 2007 15:20:26 +0000

I'm happy to announce the first release of the ROME subproject Propono. Propono is a ROME-based Java class library that supports publishing protocols, specifically the Atom Publishing Protocol and the legacy MetaWeblog API. Propono includes an Atom client library, an Atom server framework and a Blog client that supports both Atom protocol and the MetaWeblog API.

Here's the project page
http://wiki.java.net/bin/view/Javawsxml/RomePropono

Here's the Propono 0.4 release page:
http://wiki.java.net/bin/view/Javawsxml/RomeProponoRelease04

And here's a link to the API docs, which include details, diagrams and code examples:
https://rome.dev.java.net/apidocs/subprojects/propono/0.4/overview-summary.html

I'll be testing Propono this week and next (at the Google-hosted APP interop meeting) so now is a great time to provide feedback and bug reports. I plan on releasing Propono 0.5 in the *very* near future.

planet.sun.com

Dave Johnson — Sat, 7 Apr 2007 18:28:39 +0000

We haven't released the standalone Roller-Planet application yet, but the .Sun Engineering team quietly deployed the latest bits at planet.sun.com a couple of weeks ago in response to requests from the Glassfish, SWDP and other teams for planet-style web sites. You can follow the links on the main page to find planets for Glassfish, SWDP, Sun India, Sun Alumni, Sun Java System Web Server, web services and globalization bloggers.

What's Roller-Planet? It's a community aggregation server, similar to Planet-Planet but with some key differences: it's got a web UI that enables groups of users to run their own planet sites, it's based on Java and it uses the ROME feed parser and fetcher. I've written about it before. We don't have a release plan yet for Roller-Planet so if you really want to try it you'll have to fetch and build it from the Apache Roller SVN repo.

@JavaOne: Beyond Blogging: Feeds in Action

Dave Johnson — Wed, 4 Apr 2007 17:32:34 +0000

As promised here's some more info on my JavaOne 2007 session. It's based in part on the Beyond Blogging presentation that I presented to a tiny audience at the local Tri-XML conference last year. Tim Bray didn't attend my talk, but he read the slides and called them "the single best introduction and overview I've ever seen about feeds and syndication and RSS and Atom and all that stuff." I shouldn't brag. Had he attended the talk he might have had an entirely different opinion, who knows. Anyhow, the presentation has been updated, stream-lined and large portions rewritten to cover ROME and ROME's new Propono sub-project. Here's the outline:

The web is bloggy
    Blogging, XML and web services
    Feeds on the web today
    Feeds as an integration technology
    WS Death Star
Understanding RSS and Atom
    The birth of the RSS
    The RDF fork: RSS 1.0
    Elements of RSS 1.0 (abridged)
    Feed Extension Modules
    The simple fork: RSS 0.92 â€“ RSS 2.0
    Elements of RSS 2.0 (abridged)
    RSS limitations
    Atom Publishing Format
    Elements of Atom (abridged)
    RSS and Atom feed family tree
Consuming feeds with ROME
    It's just XML
    ROME RSS/Atom feed utilities
    How does ROME work?
    ROME SyndFeed model
    How to fetch feeds
    Fetching a feed with ROME Fetcher
Producing feeds with ROME
    Serving feeds: generate XML
    Use your favorite XML tools or...
    Generating Atom with ROME
    Serving feeds
    Cache cache cache
    Serving Atom with ROME
    Feed auto-discovery
    Serving valid feeds
Publishing with ROME Propono
    Atom Publishing Protocol (APP)
    What does APP do?
    How does it do all that? The REST way
    APP Service Document
    An Atom collection <feed>
    Getting an APP collection - with paging
    ROME Propono
    ROME Propono Atom Client API
RSS/Atom trends
For More Information
Summary
Q & A

And for the folks who've written to me about Propono, the new ROME subproject devoted to publishing via Atom protocol (and the MetaWeblog API): I hope to commit the code to ROME and starting blogging about it this week.

Newsgator launches Java ME powered feed reader

Dave Johnson — Thu, 15 Mar 2007 12:06:36 +0000

Newsgator, makers of FeedDemon and NetNewsWire, have launched a Java ME powered RSS/Atom feed reader. Here are some details from the press release:

NewsGator Go! for BlackBerry and NewsGator Go! for Java were developed in a technology partnership with FreeRange Communications, the leading developers of mobile publishing and mobile RSS solutions, and is available for BlackBerry, Sony Ericsson, Nokia, and most other MID-P 2.0 mobile phones, including Samsung, Sanyo and LG. NewsGator Go! is available for $29.95 or $14.95 if bought with any of NewsGatorâ€™s desktop readers, including FeedDemon, NetNewsWire and Inbox.

Via Randy Morin.

Sun Web Developer Pack R1 with RSS and Atom goodies

Dave Johnson — Mon, 12 Mar 2007 21:20:29 +0000

The Sun Web Developer Pack (SWDP) finally uncloaked today, so I can talk a little more openly about what I and my Java EE co-workers have been working on. You can get the full scoop at the SWDP site, but basically SWDP is a bundle of technologies to help developers build "Web 2.0" or next-generation web applications on the Java platform. Ajax, scripting languages, REST and of course RSS/Atom are all part of that. The RSS/Atom bits are ROME 0.9 (Beta), Blogapps 2.0 (Early Access) and a ROME-based Atom Server kit based on code from Roller. There are also example Atom server implementations in the REST API and Phobos components of the pack.

I'm pretty excited that we're putting some resources behind ROME and that both ROME and Blogapps are part of SWDP R1, but I'm even more excited about the next release. In R2 we'll drop the Atom Server Kit and Blogapps BlogClient and we'll replace them with ROME Propono a brand new Atom protocol client and server library that we're getting ready to contribute to the ROME project.

I haven't had a whole lot of time to experiment with the various components in the pack, but I have played with Phobos and I think it's pretty compelling. Phobos is a "lightweight, scripting-friendly, web application environment." It's not just for creating server-side JavaScript applications, but that's the angle the I find interesting. So many developers are creating JavaScript/Ajax applications these days that working in JavaScript on both client and server-sides makes sense -- especially when you can debug into JavaScript code in your IDE as you can with the Netbeans Phobos module. Also note that the jMaki Ajax components work with JSP, PHP and Phobos -- jMaki and Phobos look like a winning combination.

JavaOne here I come!

Dave Johnson — Fri, 2 Mar 2007 21:28:09 +0000

I'm very happy to report that my talk was accepted for JavaOne 2007. I'll be giving a thoroughly revamped and updated version of my Beyond Blogging talk (aka Johnson on Feeds), which adds in-depth coverage of ROME and ROME Propono. I'm excited about going to JavaOne again, but I'm also feeling a bit frazzled. I'm swamped with work right now and I've got a stack of deadlines clustered around the March 20 due date for the slides.

I helped out with the session selection for the web-tier track this year so I can say with some confidence that at least one track is going to rock -- we've got a wonderful set of talks lined up, so don't miss it. The full session schedule isn't online yet, but registration is open.

Placeblogger

Dave Johnson — Wed, 3 Jan 2007 08:14:59 +0000

Placeblogger is a new blog and aggregation site that's all about local blogging from Lisa Williams and friends. It's powered by Bryte, which is based on the Drupal content management system and offers blogs, feed aggregations, photo galleries and polls.

You can help build the database by submitting your favorite place blogs. The database supports a number of different "blog types." You can add aggregations, so Joe's local planets would be suitable, and you can add community sites so Orange Politics would fit right in too. I submitted Raleighing.

Here's some more reading on the topic:

Rich Burridge's blog-to-book blogapp

Dave Johnson — Tue, 2 Jan 2007 14:55:15 +0000

Rich has put together a interesting blogapp that pulls all entries from a blog and turns them into a book, using either cups2pdf or OpenOffice.org Writer. I had the same idea when I was writing RSS and Atom in Action, but I was going to go the DocBook route and eventually dropped the idea because DocBook seemed a bit too complex.

I don't think Rich's work is Roller-specific. Rich used Grabber to get the entries out of Roller and into simple HTML files, so the approach should work with other blog servers that support the MetaWeblog API.

Farewell to 2006

Dave Johnson — Sun, 31 Dec 2006 23:11:21 +0000

I've been too busy with year-end projects to blog over the past couple of days and now suddenly, it's time to say farewell to 2006. So I'll do that with a quick summary of the year.

2006 was a pretty good year for me. I published my first book: RSS and Atom in Action. Roller is still growing, reached 3.0 status and is now very close to becoming a top level Apache project. IBM started contributing to and announced a Web 2.0 product suite that will include Roller. I did my first solo JavaOne presentation and spoke at both ApacheCon EU and ApacheCon US. And, I haven't mentioned it yet, but I also landed a new job inside Sun, which starts on January 8th (more about that later).

On the home-front: the boys (now 4, 8 and 10) are all healthy, happy and doing well in school. We celebrated my dad's 70th birthday and Alex's 10th birthday. We took family trips to Ocracoke, Atlanta, Austin, Northern Virginia and made numerous visits to the in-laws beach house near Topsail Island. Plus, Andi and I escaped from the kids for a week in Ireland to celebrate our 15th wedding anniversary -- our first kidless vacation in about ten years.

I hope you had a good year too and will have an even better 2007. Happy new years!

Roller-Planet mind map

Dave Johnson — Fri, 22 Dec 2006 22:16:15 +0000

I'm glad I was able to help Simon get his personal planet back online yesterday. And I'm glad the task was fairly easy. All Simon needed as a new version of Blogapps PlanetTool updated to use ROME 0.9 and I was planning on doing that anyway.

What's PlanetTool you wonder? PlanetTool is a command-line program which reads a set of RSS/Atom newsfeeds and then uses a set of templates to generate a planet site with HTML, RSS, Atom, OPML and other representations. Simon uses it to bring together his personal blog, Sun blog, del.icio.us links and Flickr.com photos into a single webpage and a single feed. If you subscribe to that feed, you'll get just about everything that Simon publishes to the web.

If you're interested in learning more about PlanetTool, here are some of my previous posts on the topic:

Try PlanetTool, it's easy!
ROME + Texen = PlanetTool
PlanetTool internals
It's also covered in Chapter 11 of RSS and Atom in Action.

The above title Try PlanetTool, it's easy! is a little misleading, but it brings me to my point. PlanetTool is only easy if you're a developer or a power-user; somebody who can handle running Java on a server, editing an XML config file and setting up a cron job. Simon could handle it, but I'd like to make planets easier.

In fact, I'd like to make it as easy to create a planet as it is to create a blog. This past week, I've been thinking about how to do that by taking the simple ROME powered Roller-Planet code, which is found in both Roller and PlanetTool, and build it into a multi-user planet server -- kinda like Roller, but for planets instead of blogs. To get my thoughts into digital form I worked up a little FreeMind mind-map on the topic, dumped it to text, added some wiki syntax and some screen-shots. The result is this: a RollerPlanetMindMap that outlines ideas for the future development of Roller-Planet.

JavaOne 2007 call for papers closes today

Dave Johnson — Fri, 15 Dec 2006 08:52:44 +0000

There's still time to get those proposals in. I ended up submitting three proposals for technical sessions related to RSS/Atom and one for a Roller birds-of-a-feather (BOF) session.

Here's the link to submit proposals: http://www.cplan.com/sun/javaone07/cfp.

ROME 0.9 (beta) is available

Dave Johnson — Mon, 11 Dec 2006 18:52:49 +0000

A new release of the RSS and Atom Utilities (ROME) project ROME 0.9 (beta) is now available on the project's Java.net website. This new release includes fixes to Atom relative URI resolution, easier parsing for RSS feeds that use <content:encoded>, better support for mapping of RSS to and from Atom and numerous small fixes. Here are some quick links to the relevant release docs and files.

Let's review a couple of the changes that might affect the ways you use ROME, starting with the RSS parsing changes.

Easier parsing of RSS feeds that use <content:encoded>

Now that ROME includes built-in support for <content:encoded>, most user will no longer need to download and configure the separate and optional Content Module plugin. That's important because many RSS feeds use <description> as item summary and <content:encoded> as item content. It's also important because Atom supports both entry <summary> and <content>. So, whether you are parsing RSS or Atom the code to get an RSS item or Atom entry summary and content is the same. To this this, we made a number of small changes:

Added a new com.sun.syndication.feed.rss.Content object to the RSS model
Added code to converters to convert RSS Content to SyndEntry Content and vice-versa
Added code to generate <content:encoded> if content is present

Here's a simple example that shows how to parse and print out parts of an RSS feed with entries that have both summary and content (using ROME's SyndFeed model):

    String urlstring = "http://scripting.wordpress.com/feed/";
    InputStream is = new URL(urlstring).openConnection().getInputStream();
    SyndFeedInput input = new SyndFeedInput();
    SyndFeed feed = (SyndFeed)input.build(
        new InputStreamReader(is, Charset.forName("UTF-8")));

    Iterator entries = feed.getEntries().iterator();
    while (entries.hasNext()) {
        SyndEntry entry = (SyndEntry)entries.next();
        System.out.println("-------------");
        System.out.println("Title: " + entry.getTitle());
        System.out.println("Published: " + entry.getPublishedDate());
        if (entry.getDescription() != null) {
            System.out.println("Description: "
               + entry.getDescription().getValue());
        }
       if (entry.getContents().size() > 0) {
           SyndContent content = (SyndContent)entry.getContents().get(0);
           System.out.println("Content type=" + content.getType());
           System.out.println("Content value=" + content.getValue());
       }
    }

That certainly makes things easier in most cases, but if you need full support for all of the features defined by the Content Module specification you can still download and use ROME's Content Module plugin -- it works exactly as it did before.

Better mapping for mapping Atom and RSS

As ROME users already know, ROME supports parsing feeds to three different object models the RSS model, the Atom model and "SyndFeed" -- an abstract model that you can use when you do not care about the source format of the feed. ROME includes converters to convert to and from the different models. We made two changes to improve the mapping done by the converters. First, as you saw above, we added support for Content to our RSS model. That allows us to map Atom summary/content nicely to and from RSS description/content, like so:

RSS <description> <--> SyndEntry.description <--> Atom <summary>
RSS <content:encoded> <--> SyndEntry.contents[0] <--> Atom <content>

Second, we added better support for Atom title and subtitles. In the old ROME, there was some potential for information loss because the SyndFeed model did not completely support the Atom model. Specifically, titles and subtitles did not support a type attribute. To fix that without breaking the API (i.e. ensuring that old ROME code still compiles and works), we added a number of new methods to make the type information available by returning title and description as SyndContent objects (which have type and value properties).

SyndFeed additions:

public SyndContent getTitleEx()
public void setTitleEx(SyndContent title)
public SyndContent getDescriptionEx()
public void setDescriptionEx(SyndContent desc)

SyndEntry

public SyndContent getTitleEx()
public void setTitleEx(SyndContent title)

We left the existing getters and setters in place, so your existing code will continue to compile and work properly. And that's it for the quick review...

What's next?

Download it, try it out, report bugs and give the ROME project some feedback and support. And what's next for ROME? I think ROME is just about ready for a 1.0 release, but there's one last feature we need to add: some form of feed security to strip potentially malicious JavaScript from feeds. Then onto ROME2.

For even more information on ROME...

There's a complete chapter on ROME in my recently published book RSS and Atom in Action.

Latest links

Dave Johnson — Tue, 28 Nov 2006 08:14:20 +0000

Sam Ruby: Atom AutoDiscovery
Comments seem to indicate we may see a shared RSS and Atom spec, which would be good
Atom Autodiscovery spec
M. Pilgrim and J.Snell, Ed.
RSS Autodiscovery spec 1.0
Rogers Cadenhead, James Holderness and Randy Charles Morin
The Cafes: RELAX Wins
"Among the XML cognoscenti, the debate is effectively over"
Time for ROME 0.9?
New release coming soon with fixes to content/summary handing, relative URI resolving and date parsing

ROME progress

Dave Johnson — Wed, 15 Nov 2006 17:58:04 +0000

The ROME mailing list has been a little quiet lately. I'm hoping to change that. Roller's built-in planet aggregator uses ROME, Roller's Atom protocol implementation does too and I recommended ROME in my book, so I'd really like to see ROME continue to improve and grow. Now that I'm focusing on a standalone version of Roller-Planet, I've got some time to devote to those goals. Last week I cleared the bug list, this week I committed some improvements to ROME's summary/content handling and next I'd like to start pushing for a ROME 1.0 release. If you'd like to see ROME thrive, please join the fun.

Blogapps 2 progress

Dave Johnson — Wed, 15 Nov 2006 17:57:15 +0000

In my off-hours, I've started work on Blogapps 2. Blogapps is a collection of RSS/Atom utilities and applications based on the code from RSS and Atom in Action. You can read more about the project in my recent Blogapps article on on Java.net. Up until now, I've been working alone, but now the project now has a couple of committers. Ramesh Mandava (of the Java WSDP team) joined to help with the Blogapps 2 effort.

We're starting with some renaming. Instead of using chapters-oriented directories and package names, we're more logical and intuitive application names. We're also switching from package name com.manning.blogapps to org.blogapps. Later, I hope to update some dependencies (e.g. Apache XML-RPC 3.0), consolidate/streamline some of the utilities and explore alternatives to Tomcat/HSQLDB for the Blogapps server.

Couple more RSS/Atom articles online

Dave Johnson — Sun, 12 Nov 2006 09:46:20 +0000

The Server Side posted an excerpt from RSS and Atom in Action last week. Chapter 2: Development kick-start explains how to setup the Blogapps Server and how to post to just about any blog server via MetaWeblog API from Java and C#. And if you're interested in that, then you'll also be interested in The Blogapps Project, which was published on Java.net last month.

On the O'Reilly site, Mark Woodman's How to Build an RSS 2.0 Feed is now available as an O'Reilly Short Cut, a 56-page PDF for $7.99. Marks says that he covered RSS 2.0 from the perspective of the RSS Advisory Board Profile (aka RSS 2.0.8), which seems like a good idea. And he covered ROME too.

New RSS and Atom articles online

Dave Johnson — Wed, 8 Nov 2006 17:54:10 +0000

The second installment of James Snell's developerWorks article on Atom Publishing Protocol (APP) is online. In part 2 he shows how to post to an Atom server and one of his examples is Roller. If you want to try Snell's example code with Roller, but you don't want to go through the trouble of installing full-on Roller/Tomcat/MySQL, try the super easy-to-install Blogapps Server bundle.

Here are links to parts 1 and 2 of Snell's article:

Getting to know the Atom Publishing Protocol, Part 1
Create and edit Web resources with the Atom Publishing Protocol
Getting to know the Atom Publishing Protocol, Part 2
Put the Atom Publishing Protocol (APP) to work

And now, both parts of RSS and Atom in Action Chapter 4: Newsfeed Formats are online at WebReference.com. The chapter includes a history of RSS and Atom newsfeed formats and diagrams that illustrate the elements each format.

Newsfeed Formats, Part 1
From the Birth of RSS to the Nine incompatible versions of RSS
Newsfeed Formats, Part 2
the new standard: Atom

How to add a RSS 1.0 feed to your Roller blog

Dave Johnson — Mon, 6 Nov 2006 21:00:12 +0000

By default Roller 3.0 provides provides Atom 1.0 and RSS 2.0 format feeds for each blog, but you can easily add other formats. In his latest blogs.sun.com post, Henry Story explains how to add an RSS 1.0 feed to your Roller blog using the new Roller 3.0 macros.

Pundit's Monitor

Dave Johnson — Tue, 31 Oct 2006 23:53:55 +0000

Looks like Elias had a fun weekend creating Pundit's Monitor, a political blog monitoring tool using a heap of Java tech: the Nutch search engine/web crawler, Burton's TailRank FeedParser for auto-discovery and ROME for feed parsing (though he doesn't mention that in the post).

RSS and Atom in Action in action

Dave Johnson — Fri, 20 Oct 2006 16:34:10 +0000

Nick Lothian wrote to tell me about the Education.au blog, an aggregated site that uses the PlanetTool example from Chapter 11 of RSS and Atom in Action.

The Blogapps Project

Dave Johnson — Tue, 17 Oct 2006 16:57:51 +0000

My Java.net article on the Blogapps Project just went live today.

The Blogapps project provides what is essentially a complete RSS and Atom development kit, which includes feed parsers, generators, blog client libraries, an Atom protocol implementation, a set of ten useful blogapps, and an easy-to-install blog and wiki server. This article explains the project's purpose and how to install and use the project's products, the Blogapps Examples and Blogapps Server, to jump-start your RSS and Atom development.

RSS and Atom in Action on Slashdot

Dave Johnson — Wed, 11 Oct 2006 19:19:45 +0000

Simon P. Chappell writes "We've all seen them, those icons that decorate blogs and websites; sometimes they're just little orange squares with white stripes, while others say RSS or Atom. Many of us have heard of feeds and podcasts and aggregators. What are these things and where did they come from? Well, Dave Johnson, the author of the open source Roller blogging software, is glad you asked and by way of an answer, he's written RSS and Atom in Action." Read the rest of Simon's review.

Woohoo! Simon likes the book and gives it an 8/10 rating.

RSS and Atom in Action at the JavaRanch

Dave Johnson — Tue, 19 Sep 2006 11:27:46 +0000

I'm doing a RSS and Atom in Action book promotion at JavaRanch this week in the XML and Related Technologies forum. That means that I'll be checking the forum several times a day and answering all of your questions about the book's topics. At the end of the week, we'll be giving away copies of the book to four randomly selected posters.

So far, I've been answering basic questions like what is RSS, what's the difference between RSS and Atom and what's the book about?

Tags: topic:[RSS], topic:[Atom], topic:[feeds], topic:[Java], topic:[Web 2.0]

Tri-XML 2006 presentation

Dave Johnson — Sat, 29 Jul 2006 12:12:20 +0000

Here's the abstract of the talk I gave this morning at Tri-XML 2006:

Beyond blogging: Atom format and protocol. Like XML-RPC and SOAP before, feeds and publishing protocols were born in the blogopshere and quickly moved beyond blogging. Nowadays, web service providers are using RSS/Atom feeds and REST-based publishing protocols as lightweight alternatives to SOAP. And developers are finding new ways to combine web services from different sites into new applications, known as "mash-ups" in the lingo of Web 2.0. If you'd like to do the same, then attend this talk to learn about the new IETF Atom feed format (RFC-4287) and the soon-to-be-finalized Atom protocol, which together form a strong foundation for REST-based web services development.

Here's a rough outline of the talk:

Introduction

Beyond blogging
Blogs hit the hit time
The web is bloggy
Atom as an alternative to WS-*

Understanding feeds

Birth of RSS
RSS 1.0: the RDF fork
The simple fork and RSS 2.0
Atom: the standard

Parsing feeds

Fetching and parsing feeds
Universal Feed Parser
ROME utilities
Windows RSS platform

Serving feeds

Approaches for generating and serving feeds
Feed autodiscovery
Styled feeds

Atom protocol

Compared to MetaWeblog
REST based approach
Introspection
Collections
Extending Atom

Atom protocol in action

Getting a service doc
Getting collections
Posting an entry
Posting an image

Demo: interacting with an Atom server via command-line

And here are the slides: TriXML2006-BeyondBlogging.pdf

Tags: topic:[Atom Publishing Protocol], topic:[Atom], topic:[APP], topic:[RSS], topic:[feeds]

Last two chapters to production

Dave Johnson — Mon, 1 May 2006 20:00:40 +0000

Over the weekend, I put my finishing touches on the (last) two new chapters for RSS and Atom in Action. Tomorrow they'll both be off to copy-editing, typesetting and then to the printers for publication in mid-June.

I really lucked out in the reviewer category. Thanks to Walter VonKoch of Microsoft's Windows RSS Platform team, who not only answered my questions but kindly offered to review the Windows RSS chapter. And thanks also to former co-workers Pat Chanezon and Alejandro Abdelnur, who reviewed the ROME chapter.

By the way Alejandro is back from Asia, blogging again and already coming up with cool new APIs for ROME. Checkout ROME.Mano, a pipeline framework for RSS and Atom feeds.

Tip'o'the hat to the FeedValidator and crew

Dave Johnson — Tue, 18 Apr 2006 11:34:01 +0000

I'd like to thank the folks who developed and run the FeedValidator, a valuable service that let's you know if your feeds validate against the RSS, Atom and commonly used extension specs. The warnings that it issues may be irritating and some can be safely ignored, but they're valuable just the same. I don't particularly like the warnings about <content:encoded> (~~which Roller now uses, by the way~~) and the style attribute, but I understand why they're necessary. If you want to whine about something, whine about the crappy RSS specs that we're all stuck with not the folks that are trying to help you understand them.

Update: In Roller RSS 2.0 feeds, we now use <atom:summary> for entry.summary (which is new) and <description> for entry.text (as we always have).

Re: Experimenting with the MS Feeds API

Dave Johnson — Fri, 17 Mar 2006 14:02:37 +0000

I'm seeing lots of interest in my MS Feeds API post yesterday, sparked by links from Sam Ruby, Dave Winer and Randy Morin. Some people might have gotten the impression that I was criticizing the decisions Microsoft made in mapping RSS elements and extension elements to the Feeds API object model. I wasn't.

I think Microsoft made pretty good choices, given the simplified object model that they're working with. If somebody is using funky RSS, then they mean it. For example, if somebody declares the Content Module namespace and uses the <content:encoded> namespace in their feed, then that's probably the content that they want folks to use. I think that's the philosophy Microsoft used in making those decisions, except for prefering <pubDate> over <dc:date>, which I don't understand.

The problem is, the Feeds API object model is a little too simple. Like RSS 2.0, it doesn't model the common things that bloggers do like having both a summary and content for each item, or having name and/or e-mail address for each author. That's why people use extensions like the <content:encoded> and <dc:creator> (or prefer Atom, which does a better job of modeling those common things). I hope Microsoft will fix this by improving the object model and if they do, they won't have to make as many choices about which elements to use.

Experimenting with the MS Feeds API

Dave Johnson — Thu, 16 Mar 2006 08:46:57 +0000

The Windows RSS platform includes a Feeds API that parses all forms of RSS and Atom to a simplified object model.

For example, an Item object has an Author property and not an author name, author e-mail and author URI which are all possible in Atom. And, an Item object has a Description field and not description and content (as in Wordpress feeds) or summary and content (as in Atom feeds).

So, how does the Feeds API decide how to map elements to this simplified object model? I did some C# experiments and here are some of my findings. Note that the Feeds API is beta software and will certainly change for the better (I hope) by the time it is released in IE7 and Windows Vista.

Item contains	Feeds API returns
<dc:creator>dave</dc:creator>	item.Author = "dave"
<author>dave@example.com</author>	item.Author = "dave@example.com"
<author>dave@example.com</author> <dc:creator>dave</dc:creator>	item.Author = "dave" (prefers funky RSS)
<description>my desc</description> <content:encoded>my content</content:encoded>	item.Description = "my content" (prefers funky RSS)
<pubDate> Thu, 9 Mar 2006 23:13:04 -0500 </pubdate>	item.Date = "3/10/2006 4:13:04 AM" (uses GMT)
<pubDate> Thu, 9 Mar 2006 23:13:04 -0500 </pubdate> <dc:date> 2004-08-19T11:54:37-08:00 </dc:date>	item.Date = "3/10/2006 4:13:04 AM" (prefers core RSS element)
<atom:summary>my summary</atom:summary> <atom:content>my content</atom:content>	item.Description = "my content"

First, it's interesting that those funky RSS elements that Winer dislikes are preferred over the core RSS elements in important places. And second, what if you're not happy with Microsoft's mapping choices in this area?

For example, how do you get both description and content from those Wordpress feeds? Wordpress (and Typepad) uses the <description> element as a summary and the funky <content:encoded> element for the full content (see Winer's own Wordpress.com feed for example). You've got to parse the XML yourself. The Feeds API tries to makes that easy by providing both the XML for the entire feed and the XML fragment for each item, but I think most developers would prefer to have a more complete object model.

See also: What's up with the Windows RSS Platform

Tags: topic:[rss], topic:[atom], topic:[feeds], topic:[ie7], topic:[vista]

The never ending story of RSS and Atom in Action

Dave Johnson — Wed, 15 Mar 2006 21:38:04 +0000

You know last week, when I said the book was ready to go to the printers and would be available this week as an e-book? We'll, I was wrong.

While we waited for Atom protocol to stabilize, things changed in the world of C# and Java feed APIs. Microsoft introduced the Windows RSS platform and a pre-release of the Windows Feeds API is available in the IE7 beta. And ROME has come along way too; now with Atom format 1.0 support and a growing list of extension modules. We decided that we just couldn't publish a book on RSS and Atom without covering the Windows RSS platform and ROME in-depth. So now I'm under the gun again, writing away into the wee hours of the night. I should be done by April 14th and, with luck, the book will be out in late May, just in time for JavaOne. That explains my sudden interest in the Windows RSS platform.

The kids hate it, but I think it's for the best. Manning will have the very first book that covers the Atom protocol (with a working client and server), the Windows RSS platform and ROME in-depth. It'll definitely be worth the wait.

Tags: topic:[atom], topic:[rss], topic:[ie7], topic:[atom protocol]

What's up with the Windows RSS platform?

Dave Johnson — Sun, 12 Mar 2006 12:03:17 +0000

The Windows RSS Platform (or Feeds API) is the feed handling engine that powers the new RSS features in IE7. It will also be included in Windows Vista for use by other applications. Note that here, RSS is a generic term meant to include both RSS and Atom -- the Feeds API supports both. The Feeds API is packaged in a DLL called msfeeds.dll and available to programmers as a set of dual-interface COM objects. Here are the features exposed via the Feeds API.

Common feed list: list of feeds for current user, organized as folder hierarchy.
Feed store: local cache of feeds, feeds available via abstract object model
Download engine: for managing and monitoring large enclosure downloads
RSS sharing extensions: new XML elements to support bi-directional sync via RSS

The Feeds API gives you access to the current user's feed subscription list, a feed parser that can handle any form of RSS and Atom as well as the IE7 podcast download engine. The parser parses feeds to an abstract object model designed to represent any sort of feed. It handles funky RSS and in some cases prefers the funky elements (e.g. <content:escape> over <description>).

I'd like to learn more about how the Feeds API decides which elements to use, how sync works, and how the whole package compares to the premier Java Feeds API ROME. So, I've downloaded IE7 and started experimenting with the API from C#. I'll be posting more on this topic in the next week or two.

Here are some of the references I've been using to understand the API:

Feeds API docs, specs and whitepapers from Microsoft

The Feeds API documentation at MSDN Library
The Simple Sharing Extensions (SSE) for RSS and OPML specification
The Simple List Extentions for RSS specification
Simple Sharing Extensions for RSS tutorial (in Vista only?)
RSS Support in Longhorn - good overview
PDC05 presentation about RSS in Windows Vista

Microsoft employee blogs about the RSS platform

The Windows RSS Platform described on the Microsoft RSS Team Blog
The Windows RSS Platform ala carte from the Microsoft RSS Team Blog
A blog about the RSS Simple Sharing Extensions from Microsoft's Jack Ozzie
Related: Ray Ozzie's blog post Wiring the Web
FeedBandit developer Dare Obasanjo's feedback on Windows RSS Platform
Windows Live Mail Desktop Beta (RSS feeds and "blog-it" feature)

Other blogs about it

Newsgator's Nick Bradbury's feedback on Windows RSS Platform
PDC'05 blog about the above Windows RSS Platform presentation
Jim Mathies built an aggregator in a weekend with the RSS platform
A simple windows RSS store viewer and exploration of msfeeds.dll implementation
John Udell interviews Microsoft's Amar Ghandi about the RSS platform

Update1: added a couple of new links suggested by Mark Woodman
Update2: added reference to Simple List Extensions
Update3: added link to RSS in Windows Vista presentaton

Tags: topic:[atom], topic:[rss], topic:[ie7], topic:[atom protocol]

The talk went well

Dave Johnson — Thu, 30 Jun 2005 23:15:20 +0000

My second JavaOne was a great experience, but it was a little stressful because up until last night I couldn't find any of my co-speakers. I spent most of Wednesday preparing to give the whole talk by myself, but luckily for me (and the attendees), Pat and Kevin showed up just in time. Unfortunately, Pat showed up with some very bad news for us at Sun: he's leaving to work at Google.

In the end, I think the talk went pretty well. Kevin did most of Pat and my slides on syndication because we had split the talk 50-50 when we couldn't locate Pat on Wednesday night (and assumed he was still in Paris). He did a good job with the material and added in some interesting points from his experience at Rojo.com where they parse millions of feeds per hour with the Java-based Apache Commons (sandbox) FeedParser.

We were a little disappointed with the turnout. I'd be surprised if the 700+ seat Yerba Buena theater was more than 30% full. The fact that were in a lunchtime timeslot on the last day of the show certainly didn't help. Anyhow, I'm relieved that it's over and ready for a nice long week off.