Blogging Roller

WIP #6: Feed-based Integration

2012-10-08T10:38:43+00:00

This is the sixth in my series of Web Integration Patterns. Check out the intro at this URL http://rollerweblogger.org/roller/entry/web_integration_patterns

Synopsys

This pattern is about integrating web sites and applications by using standard feed formats to convey timely information, updates, status messages, events and other things from one web application to another.

Motivations

Take advantage of the many tools and services that can produce, consume, aggregate and build work-flows around feeds as a way to integrate web applications.
Use a standards-based approach to ensure that that widest range of feed producers and consumers can be supported. Make it so everybody who needs can read your feeds.

Related Patterns

Search Integration
Web APIs

A feed is a list of entries, each with an timestamp, ID, title, content and metadata like categories and tags. Entries are arranged in reverse chronological order. The entries in a feed can represent just about anything from blog entries, Flickr photos, YouTube videos, source-code change sets or tasks in a change management system. A feed is an XML resource that is available at a URL. If you want updates, then you poll that URL, ideally using HTTP Conditional GET so that you only pull down the feed when it has been updated.

Generally speaking there are two standard feed formats in use on the web: RSS and Atom, both are based on XML. Both use different element names, for example: what Atom calls "entries" RSS calls "items." Because these standard formats are so widely supported, providing a feed is an effective way to share updates from your web site or application.

Elements of Atom (from my 2006 presentation on Atom)

Another flavor of feeds is ActivityStrea.ms, which is essentially a feed format with a schema for representing about 70 different types of activities. These activities can be social network activities like share or friend and they can also be business activities like assign, resolve or schedule. One advantage of using the ActivityStrea.ms standard is that it has both an Atom and a JSON mapping.

Producing Feeds

Feed-based Integration is listed as a basic pattern because it can be very easy to implement. The ability to produce feeds is built-in to many different types of web applications from blog and wikis to continuous integration servers. If you are writing your own web application, you can choose to use XML tools to produce your feeds, a templating engine or a dedicated feed toolkit like ROME. You’ll find plenty of XML tools and templating engines no matter what language you are using.

Which type of feed should you produce? That depends. Atom is the most complete specification and is a true IETF standard, so often it is the right choice. To make the right decision, you have to consider who is going to be consuming your feeds. If your consumers prefer RSS, them give them that. If your consumers prefer JSON over XML, then consider ActivityStrea.ms in JSON flavor.

Consuming Feeds

You can use a wide variety of tools to parse and process feed data. For example, there are many web sites and services that can digest feeds and trigger other events and processing. Services such as Yahoo Pipes and If This Then That can read feeds, process each item and perform other actions based on item values.

Processing a feed with Yahoo Pipes (from Marsh Gardiner's post)

You’ll also find a variety of JavaScript widgets that can parse and display feeds in a website. Most web portal and software with a web dashboard features will include a component that can display feeds. And, of course, you can use a reader that supports RSS/Atom feeds, such as Google Reader or Feedly or FlipBoard, to monitor and read feeds.

If you need to add RSS/Atom reading features to your own software, you can use standard XML parsing tools and, for most language, you’ll find that there are open source libraries specifically designed for parsing feeds.

For more information in producing and consuming feeds, RSS and Atom in Action. Also, you might find my presentation on Beyond Blogging: Atom Format and Protocol useful.

Beyond the basics

Feeds are a great way to do simple integrations, but there are limitations and there will be times you’ll need to go beyond the basics with RSS and Atom feeds. Here's an example. Normally, with feeds, clients have to repeatedly poll the feed URL for updates. This is annoying, and inefficient, even with HTTP Conditional GET. To address this problem, you can setup a PubSubHubub server that will subscribe to feeds and will then notify other subscribers instantly when updates are available, so that those other subscribers don’t have to poll.

Another problem is that, if you don’t poll often enough, you might miss some updates and they may “scroll” off the bottom of the feed before you see them. Feed providers can address this problem by supporting Feed Paging and Archiving, which allows clients to use next and previous links to “page” back to feed items that are no longer in the first page of the feed.

One more beyond-the-basics item to mention is the related pattern Web APIs, which we'll cover later. Web APIs are listed as a related pattern to Feed-based Integration. That's because feeds have been used as the basis for several "Web APIs" or protocols. These protocols specify how to use HTTP POST, GET, PUT and DELETE and create, retrieve, update and delete web resources that are represented as feed entries. Examples are the IETF's Atom Protocol, Microsoft's OData and Google's GData APIs.

That’s it for Feed-based Integration. In my next posts, we'll move in to the Advanced Patterns.

More than bloggy

2011-03-07T06:00:00+00:00

I’ve been meaning to write on this topic for a some time and to explain how I’ve gone from being an advocate of RSS/Atom feeds, Atom Publishing Protocol and things bloggy to being a proponent of Linked Data (video), Semantic Web, RDF and other things that I previously considered to be nuisances. I've also got a new topic and blog series to announce, so here goes.

Once upon a time, I actually said “the web is bloggy” in conference presentations and without shame. It sounds silly and, for a blog server developer self-serving, but it’s true and it’s still true to this day. So much of what we do on the web can be made more useful via feeds. That’s because so many things can be usefully represented as a well formed log entry, i.e. a URL, title, date, description and a chunk of content. No matter whether your web application is Flickr, Twitter or IBM/Rational Team Concert, you can enable all sorts of interesting integrations and mashups by providing feeds.

Atom Publishing Protocol (APP) applied the feed concept to web APIs, using a feed to represent a collection of entries and allowing API callers to use HTTP to create, retrieve, update and delete entries. Entries could be anything of course. In the case of Google’s APP-based GData Protocol, entries are Calendar Events, Picasa Photo uploads or YouTube videos. In the case of the IBM/Lotus Connections API, entries are Forum Posts, Social Network Profiles, File Uploads and more.

Feeds only get you so far

I’m not going to argue that RSS is dead (or Atom) because I don't believe that, but the feeds approach only gets you so far. With feeds, we agree on a small set of common properties, e.g. URL, title, date, description and content. Not all resources on the web have those properties and many resources have a lot more. So in some cases applying Atom means forcing things to fit into the Atom model and in some cases it means inventing new properties for Atom, something that is supported by Atom format.

APIs only get you so far

I’m not going to argue that APP is dead or a failure either, but APP is an API approach and, for web integration, APIs only get you so far. It’s definitely a good thing for developers when a web site or application provides an API, but if you have to integrate a set of N web sites and each has it’s own API and (even if they’re all based on APP) you’re stuck doing N x N point-to-point integrations. APIs are not necessarily the best way, and certainly not the only way, to enable integrations on the web.

Linked Data

Linked Data (W3C page) is a different approach. Instead of forcing everything to fit the blog model, Linked Data gives us a way to define common vocabularies of properties about resources on the web and, most importantly, the links between them. Linked Data enables integration by allowing sites to weave their resources into the interlinked web of data that is the web.

Next: Web Integration Patterns

These days, what's important to me at work is integration and in my mind, Feeds, APIs and Linked Data are different “patterns” for web-based integrations. They can be used in complementary ways. Just as we have defined and cataloged Enterprise Integration Patterns, I think it will be useful to do the same for Web Integration Patterns. I know I'll learn something in the process. In my next posts, I'll expand on this concept and after that, I'll start cataloging, in an informal and bloggy way, the patterns that I'm familiar with. More later...

RSS and Atom part of the stimulus plan

2009-02-21T08:32:13+00:00

Aaron Swartz: As chaunceyt pointed out, the new stimulus bill's implementation instructions require that each government agency report the money it gives out in RSS:

For each of the near term reporting requirements (major communications, formula block grant allocations, weekly reports) agencies are required to provide a feed (preferred: Atom 1.0, acceptable: RSS) of the information so that content can be delivered via subscription.

Pretty amazing to see a government so tech-savvy.

Uncle Sam should contact Manning, who, I'm pretty sure, has stacks of RSS and Atom In Action available for a very good price.

Media Blogging for Roller

2009-02-19T12:57:37+00:00

For the past five months I've had the pleasure of mentoring two San Jose State Univ. graduate students, Ganesh Mathrubootham and Tanuja Varkanthe, who are working on a project for classes CMP 295A and B. They picked one of the projects that I first proposed for Google Summer of Code and then for Glassfish's student outreach program, Media Blogging for Apache Roller. It's turned out to be a major project and the central new feature in the upcoming Roller 5.0 release.

The plan

The basic idea is to make it really easy to upload images, audio and video files to Roller, and really easy to include them in Roller blog posts and RSS/Atom feeds. Of course, the devil is in the details and Ganesh and Tanuja really have those covered. They have put together the most detailed and well thought-out plan and design ever for a new Roller feature. You can find the proposal page and the full Media Blogging for Roller Project Plan (PDF, 2mb) on the Roller wiki. Here's a key excerpt from the project summary:

Roller currently lacks support for the latest blogging features. Roller does allow users to upload any type of content to their blogs and include that content on blog entries as images or podcasts, but lacks tools to make media blogging a seamless experience for bloggers. Interface to manage uploaded files is not sortable and not page-able. Once the user has uploaded a file, which could be an image or a podcast, he needs to explicitly cut and paste the URL into his blog post.

Overall, Rollerâ€™s support for media blogs is currently limited to basic file upload functionality, which is cumbersome to use for creating media blogs. This project will revamp the existing file upload interface to incorporate powerful media blogging features into Roller. Also, successful media management websites such as Flickr and YouTube are driven by a public media library, offering different ways for users to search and locate the content of their like. This feature incorporated into a blog server can make it very powerful and we intend to do that as part of this project.

To get an idea of what this is all about, let's take a look at some screenshots/wireframes taken directly from the project plan.

New media file upload dialog

First up is the new media file upload dialog. There are a couple of interesting things here. We'll have metadata for each upload, including description, tags and copyright message. We'll also have the option of including the file in the gallery. We'll support a media gallery for each blog, and new files added to the gallery are included in a special media feed for the blog.

File Uploads browser

To make it easy to manage your media files, the proposal includes a new upload browser interface, with a tabular and a hierarchical view. Thumbnails will be automatically generated on upload. Search and filtering controls will make it easy to find and operate on the files you are looking for, based on file metadata.

Select media dialog

When you are writing a blog post, you'll be able to browse for and include media files without leaving the blog editor interface. You'll be able to choose the size and orientation of the image or video in the blog post, as you can see below.

Needless to say, I'm pretty excited about these new features, and to looking forward to a major new Roller release, and one where I don't have to do much of the work. If you want more information on 5.0 then check the Apache Roller 5.0 proposal page. And if you want to help out then join the Roller development mailing list and introduce yourself.

ROME 1.0 RC2 on the way

2009-01-07T08:48:11+00:00

Good news for ROME fans. Nick Lothian picked up the puck and is galloping towards the finish line (sorry, I'm terrible at sports analogies).

Nick Lothian on ROME dev:

I've gone and built some preview jars for the upcoming ROME 1.0RC2, ROME Fetcher 1.0RC2 and Modules 0.3 release.

Those jars can be found here: https://rome.dev.java.net/servlets/ProjectDoc...

I've created source and javadoc jars as well as the normal jars - the idea being that I'll get them uploaded to some maven repository.

If you have some spare time, please take a look at these and test them and let me know of any problems. Assuming there are no big issues found I'd like to do a proper release in a couple of days.

Guess that means I should test Propono with RC2.

Atom news: Apache Abdera graduates

2008-11-21T15:54:48+00:00

Congratulations to the Apache Abdera team, who've just graduated to full Apache top level project status. The don't have the new site at abdera.apache.org up yet and they're still not quite at 1.0 yet, but this is a major milestone. They've got the best Atom format and protocol toolkit around, in my opinion.

via Garett and James.

Updating RSS and Atom in Action?

2008-07-09T16:32:28+00:00

Check out Tim Bray's Atomic Monday post for a summary of recent developments in the world of AtomPub protocol and Atom format. With the uptick in interest in Atom and especially AtomPub protocol's usage in mainstream APIs from Google and Microsoft, I wonder if if would make sense to update RSS and Atom in Action.

The bulk of the book is up-to-date, for what it covers. RSS and Atom haven't really changed and, though the book came out before the spec was finalized, the AtomPub material is basically sound. The ROME material is still current too.

What's missing are all of the new developments: a dozen or more new AtomPub based APIs from Google (GData and OpenSocial) and Microsoft (Live and Astoria) and new tools like Apache Abdera (incubating). Not to mention Windows RSS Platform 2.0 and ROME 1.0 (which is coming soon). Updating the book to cover all of those topics would be quite an endeavor and I'm not sure it's the right approach to covering AtomPub. What do you think?

ROME vs. Abdera

2008-01-22T18:18:40+00:00

For Java developers starting out with RSS and Atom, here are some notes to help you figure out the differences between the Java.net ROME and Apache Abdera (incubating) projects.

ROME is a set of Java tools for parsing, fetching and generating all forms of RSS and Atom feeds. The core ROME library is relatively small and depends only on the somewhat creaky old JDOM XML parser. Available separately are modules to support various feed extensions such as OpenSearch, iTunes, GeoRSS, etc. ROME was originally developed and open sourced by Sun Portal dev team members in 2004.

ROME Propono is a subproject of ROME that supports publishing/editing entries and files to blog servers and AtomPub servers. Propono is made up of three parts: 1) a Blog Client library can publish via either the old lagacy MetaWeblog API or the shiny new AtomPub protocol, 2) an AtomPub client that publishes only via AtomPub and 3) a framework for creating AtomPub servers. Propono was developed by Ramesh Mandava and Dave Johnson, based on code from RSS and Atom in Action and open sourced as part of the Sun Web Developer Pack in 2007.

Abdera is a set of Java tools for working with Atom feeds and AtomPub protocol. This includes a parser, writers, an AtomPub client and a framework for creating AtomPub servers. Abdera's Atom feed parser uses STAX, so it uses less memory and is faster than ROME. Abdera's Atom feed support is more comprehensive than ROME's and it supports signatures, encryption, Atom to JSON, extensions for Threading, Paging, GeoRSS, OpenSearch, GoogleLogin, etc. etc. Abdera was developed by IBM and contribued to Apache in 2006.

Now let's compare frameworks. The pros and cons of ROME are:

Pro: complete RSS support, all of the dozen various flavors
Pro: it's generally simple and small, depending only one jar (JDOM)
Pro: easy to understand and use the AtomPub server framework
Pro: MetaWeblog API support
Con: Atom feed support not as comprehensive as Abdera
Con: parser uses lots of memory, slower, JDOM based
Con: community not as active, seems to be in maintenance mode (See also Ohloh stats)

The pros and cons of Abdera are:

Pro: comprehensive Atom feed support, lots more Atom extensions
Pro: faster more efficient parser
Pro: In the Apache Incubator with active and growing community (See also Ohloh stats)
Con: lots of dependencies
Con: AtomPub server framework poorly documented, overly complex (rewrite coming soon)
Con: no RSS support (there is something in Abdera contrib, but it's incomplete).

There you have it. ROME and Abdera folks: think that's a fair comparison? Are you a ROME or Abdera user? How would you like to see these frameworks move forward?

Abdera rocks on

2008-01-11T18:18:08+00:00

There have been some interetsing Atom items recently on James Snell's blog. There was a link to a nice IBM DeveloperWorks article on using Abdera to write Atom feeds to JSON. (I was kinda hoping for an XSL/T, but not matter how you do it, it's a definitely feature that belongs in Abdera.) And there's a good quote from Adrian Sutton's blog post about the Atom features in IBM's Roller and Abdera based Lotus Connections product. And most recently, James linked to the new Google Feed Server project: an Abdera based AtomPub server from Google. All good stuff.

Gotta say, Abdera's looking better and better. It's got the features (IRI support, encrytion, pluggable auth, STAX parsing, etc.) and it's got Dan and appears to have some good momentum going. At this rate, I'm not sure how much more work I'll be putting into ROME Propono. If somebody were to, just for example, contribute an Abdera based AtomPub implementation to Roller, I might have to stop entirely.

The "all Dave" feed

2008-01-09T15:19:15+00:00

Another thing I worked on over the winter break was an all-Dave feed, a single page and feed that combines all of the various things that I post to the web: blog entries, Flickr photos, del.icio.us links and Twitter activity. So far, this is what I've got: http://rollerweblogger.org/roller/page/feeds.

Happy New Year 2008

2008-01-01T23:54:24+00:00

Happy New Year to all the readers of this blog and all the folks who happen to have arrived here via one of the various planets, spam blogs and Google search result pages that include it. I hope you all have a prosperous and pleasant year ahead. I'm hoping for the same. It's nice to start the new year with some good news and I've got some; check this out:

RSS and Atom in Action has been out for over a year now and just as the phenomenal sales are starting to drop off, something wonderful has happened. The book has been nominated for the prestigious Stephen T. Colbert Award for Literary Excellence. As you can see in the photo above, at least one copy of the book is already carrying the Colbert nominee seal. Avoid the rush and buy your copy of RSS and Atom in Action now.

Those who have read Colbert's I Am America (And So Can You!) will understand the significance of this important award and understand the impact on future sales of the book. To learn more about the nomination process and criteria, buy yourself a copy of Colbert's book and then flip to page 214½. Cheers!

Latest Links: Feedsync, AtomPub for SOA, OpenSocial and more

2007-12-06T14:00:10+00:00

snellspace.com: Sync!
"Within the course of implementing several Atompub servers, the issue of â€œfeed synchronizationâ€ has come up repeatedly"
Paul Fremantle's Blog: A new kind of (SOA) Registry
"as we looked at the REST space, we kept noticing how close the [AtomPub] is to our needs"

OpenSocial - Berlin Web 2.0 Expo
Google API evangelist Pat Chanezon's OpenSocial presentation
opensocket - Run OpenSocial Gadgets in Facebook
An OpenSocial container written as a Facebook application.
Ruby on Rails: OpenSocial container plugin 0.0.1
"This is a very early version, but it is under very active development "

IBM's Carol Jones on Web 2.0 Research and Collaborative Software
Redmonk video discussion of Lotus Connections, Dogear, internal/private vs. external bookmarking
Wayne Horkan's weblog eclectic: Weblog language translator
With Roller specific functionality. Based on JavaScript and Google translation API
ZDNet.com: Social nets and identity fragmentation
FaberNovel Consulting has mapped out some trends in social networking and digital identity"
Postgres at Hi5: June 2007 user group preso
How unique features of PostgreSQL helped Hi5 scale (PDF presentation)
Scaling Java and PostgreSQL with Hyperic
Another perspective on PostgreSQL scalability at Hi5 (PDF presentation)
Skrentablog: Scaling Facebook, Hi5 with memcached
"discussion of high volume [sites] using memcached as a critical scaling tool"

Latest links Nov. 12, 2007: Glassfish, OpenSocial and more

2007-11-12T07:23:08+00:00

Sound Opinions from Chicago Public Radio and American Public Media
Best podcast ever

The Aquarium: GlassFish Interim Governance Board - Now Complete
"The complete roster is Tony, Greg Luck (Wotif.com) and Pierre Delisle (Google), and Simon and myself (Sun)"
Bistro!: GlassFish/SJS AS in production - which bundle, which profile, ...?
Explains differences between developer, cluster and enterprise profiles
GlassFish Podcast: The GlassFish Podcast
Finally, a Glassfish podcast! Props to Alexis Moussine-Pouchkine
Apple - Downloads - UNIX & Open Source - GlassFish
Download Glassfish V2 directly from Apple
Red Hat and Sun Collaborate to Advance Open Source Java Technology
"Red Hat has signed Sun's broad contributor agreement that covers participation in all Sun-led open source projects."

Megginson: First looks at OpenSocial: part 1 (URLs)
Lists the URIs for the AtomPub collections available in OpenSocial
Megginson: First looks at OpenSocial: part 2 (members and friends)
How OpenSocial members and friends are represented in Atom format
Megginson: First looks at OpenSocial: part 3 (activities)
How OpenSocial activities are represented in Atom format and manipulated via AtomPub protocol
Megginson: First looks at OpenSocial: part 4 (persistence data)
How OpenSocial persistence is implemented via AtomPub protocol
Google: OpenSocial Container Sample
Shows "basic demonstration-level OpenSocial container can be implemented"
Dare Obasanjo: OpenSocial Tech. Overview and Critique
"Despite these misgivings, I think this is a step in the right direction. Web widget and social graph APIs need to be standardized across the Web."

snellspace.com: Notes, Part 2
"lots and lots of things that can be modeled as collections of web resources"
netzooid: Building Services with AtomPub
"While APP is not the one true protocol, I think I?m hooked"

Windows Live Writer + AtomPub

2007-10-16T09:32:34+00:00

Joe Cheng: Iâ€™m starting a (hopefully short) series of blog posts documenting the specifics of how the upcoming AtomPub-enabled release of Windows Live Writer will behave, and what AtomPub-enabled blog* servers can do to ensure the best interop with us. I also hope other blog client implementers can learn from our experience and consider using the same rules and behaviors we do when implementing their own AtomPub support.

Very nice. I'll be watching this one closely, looking for ways to improve Roller's and ROME's AtomPub support.

Creating custom modules with ROME

2007-10-15T11:26:25+00:00

Google Blog Search turned up this nice article on creating custom modules with ROME by Sujit Pal, and a nice mini-review of my book:

Sujit Pal: "However, Dave Johnson's book provides a lot of background information and a lot of nice examples in Java and C#. I would highly recommend it to anyone who needs to get up to speed quickly with ROME and RSS/Atom."

It's good to see people are still buying and finding the book useful. It's been out for over a year now, but I don't think it's really out of date at all. I've been keeping the examples up-to-date as part of the Blogapps project and even enhancing them: the example Atom protocol client code from the book is now part of the ROME Propono project.

Latest links: AtomPub news round-up edition

2007-10-12T12:31:13+00:00

I've been in crunch-mode working on a new project, but I've been trying to keep an eye on what's going on the world of Atom Publishing Protocol or AtomPub as the cool kids call it. Here's a wrap-up of some of the AtomPub news I've picked up on the past month or so.

Specifications

The big news is, of course, that AtomPub has been granted an RFC number RFC-5023 and is now an IETF "proposed standard" like Atom format (RFC-4287). The folks at Ricoh have provided a Japanese translation.

Extensions to AtomPub are under development and you can find a good list of them at the XML Cover Pages page for Atom. One I find interesting is James Snell's AtomPub Feature Discovery spec, which is designed to help AtomPub clients determine what features are supported and required by servers. The features spec is controversial and has sparked a lot of good discussion including, Brian Smith's email Atom Weblog Publishers Guide and Sergey Beryozkin and Dan Diephouse's discussion of AtomPub and WS-Policy

Server implementations

More big news to report here: the new Wordpress 2.3 release provides full support for AtomPub. Sam Ruby blogged about it in Wordpress 2.3 loves AtomPub and the Wordpress 2.3 AtomPub docs provide a good summary and cover some limitations. Yesterday, Sam posted Wordpress vigilance and plans. He wants to get Wordpress to score a perfect 0 errors and 0 warnings on the APE tests, which means among other things preserving foreign markup.

New releases of Apache Roller have also been updated for the final AtomPub spec. The upcoming Roller 3.1.1 and Roller 4.0 releases include AtomPub support. Both releases are up for vote now, so they should be out Real Soon Now. And I'm working on adding AtomPub support for comments for Roller 4.1, building on top of the Atom Threading Extension (RFC-4685).

I've been googling and browsing the SixApart site, but I can't figure out if Movable Type 4.0 supports the final AtomPub spec. Anybody know?

And Google has added another AtomPub based API. Read about it on Bill de hÃ“ra's blog YouTube via GData and on the YouTube Developer API blog: New YouTube API Released into the Wild.

Client implementations

In Stick a fork in it Joe Cheng writes "The Windows Live Writer team is still on track to deliver AtomPub support in the next version, which I am looking forward to immensely. Itâ€™s definitely an exciting time to be in the blogging tools space!" Based on what I saw at the last face-to-face AtomPub interop, it looks like Microsoft may have the best AtomPub-based blog client around. I wonder when MarsEdit and Ecto will get on board with AtomPub.

Toolkits and programming libraries

I already blogged about the Propono 0.6, a Java-based client and server library for AtomPub, which has been updated for final spec. Since then the Apace Abdera project has released Apache Abdera (incubating) 0.3 release is available with a long list of improvements and new features. There are also a couple of usage notes from James Snell. Also on the topic of Abdera, Dan Diephouse has done some work to make Spring and Abdera work well together.

The Perl AtomPub::Client at CPAN has updated for final AtomPub too.

That's all I've got for now. Did I miss any important AtomPub news?

Blogapps 2.1 released

2007-10-02T21:49:40+00:00

<a href= "http://www.amazon.com/dp/1932394494?tag=bloggingrolle-20&link_code=as3&creativeASIN=1932394494&creative=373489&camp=211189"> The next releases that I'd like to announce are the Blogapps 2.1 Examples and the Blogapps 2.1 Server.

If you'd like to learn more about the Blogapps examples and server then read The Blogapps Project article at Java.net. Here's a quick summary:

The Blogapps project hosts a collection of useful RSS and Atom utilities and examples from RSS and Atom In Action by Dave Johnson. They're designed to be useful even if you haven't read the book and they're available under the Apache License 2.0 so you can use the code in your applications and you can modify and redistribute them as you wish.

What's changed since 2.0? The examples have been updated to include the latest version of ROME Propono, which means that most of them now support the final Atom protcol spec. The server has been updated to include Roller 4.0 RC5, which also includes Atom protocol support and JSPWiki 2.4. And of course, various bugs have been fixed. Here are the release files, installation instructions and release notes.

This blog entry was posted via Atom protocol and the MatisseBlogger blog-client, which you can see in the screen-shot below (which was also posted via Atom.

What's next? Not sure at this point, but I will do another Blogapps release once ROME 1.0 is released.

ROME Propono 0.6 released

2007-10-01T11:20:19+00:00

The first release to discuss is ROME Propono, which includes a ROME based Atom protocol client library, Atom protocol server framework and an Blog Client library abstraction that supports both Atom protocol and the MetaWeblog API.

I've been working on Propono 0.6 off-and-on since May, keeping it in sync with the latest version of the Atom protocol, testing it against Tim Bray's APE and adding various improvements needed in my other projects. Over the weekend I finally had enough time to get a release out. You can find the full-details at the link below but basically this release adds support for the final Atom Publishing Protocol specification and better support for relative URIs.

What's next? Once ROME 1.0 is released Real Soon Now, I'll get a another release out and I'll probably call it ROME Propono 1.0.

A logo for the Ape

2007-08-23T11:47:53+00:00

Just learned from Tim's blog that Greg Borenstein just came up with a cute logo for the Atom Protocol Exerciser (Ape), which, in case you don't know, is the closest thing we have to a compatibility test for Atom servers. The logo is featured on the Ape's home page.

Atom Protocol Exerciser (Ape) setup notes

2007-06-28T20:23:06+00:00

UPDATE: things have changed quite a bit since I published this blog entry. There is now a Ruby Gem for installing Ape and the process is much easier than the one I describe here. See Sam Ruby's post Migratory Ape for details - Dave (March 1, 2008)

There's some very cool Atom news from Tim Bray this week. First, there's mod_atom, a plugin for the Apache HTTPD web server that provides a stripped down Atom server (written in C) that stores Atom entries as files. So now any HTTPD server can serve as a simple Atom store.

Second, Tim mentioned that he's updated his Atom Protocol Exerciser (Ape) to support the new Atom protocol namespace and to add a couple of new tests. That's perfect timing for me because I'm starting to work on Roller's APP implementation and some other APP based things.

I need my own local copy of the Ape, so I decided to download the code an get it up and running. To make things easy, I decided to do this work on my Ubuntu system because it's already setup with gcc, cvs, Apache HTTPD and Ruby. For my future reference and to help other Atom fans to get started, here are my notes from my Ape setup experience.

Setting up the latest Ruby#

Tim told me that the Ape needs Ruby 1.8.6, but Ubuntu has only 1.8.4 and apt-get upgrade ruby didn't help so I downloaded and built 1.8.6 myself. Here's what I did:

% wget ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.6.tar.gz
% tar xzvf ruby-1.8.6.tar.gz
% cd ruby-1.8.6
% ./configure
% make
% sudo make install

Once the make install finished, I found that I had a fresh new copy of Ruby in /usr/local. Next, I set up my environment for running Ruby by adding the following three lines to my .bashrc file.

export RUBY_HOME=/usr/local
export RUBYLIB=$RUBY_HOME/lib/ruby:$PREFIX/lib/site_ruby/1.8
PATH=$RUBY_HOME/bin:$PATH

Setting up the latest RubyGems and Builder#

But that's not all the Ruby you need for the Ape, you also need some RubyGems so I downloaded and built the latest version of that too. I followed nearly the same steps as with Ruby:

% wget http://rubyforge.org/frs/download.php/20989/rubygems-0.9.4.tgz
% tar xzvf rubygems-0.9.4.tgz
% cd rubygems-0.9.4
% ./configure
% make
% sudo make install

Next, I setup environment for running RubyGems by adding GEM_HOME to my .bashrc. I did this by adding the following line after the definition of RUBY_HOME.

export GEM_HOME=$RUBY_HOME/lib/ruby/gems/1.8

Ape needs the RubyGems Builder, so I ran this command to get the latest version of the Builder from the RubyGems repository:

% sudo gem install builder

Downloading Ape#

Finally, I've got enough Ruby in place to get going. So next I got the latest source for the Ape. Unfortunately, I think you need a Java.net account for this part. Here's what I did:

% cd /home/dave/src.downloads
% cvs -d :pserver:snoopdave@cvs.dev.java.net:/cvs login
% cvs -d :pserver:snoopdave@cvs.dev.java.net:/cvs checkout ape

Setting up an HTTPD ScriptAlias for running the Ape #

Next, I created a ScriptAlias so I can run the Ape via CGI. I did this by adding the following to the end of my Apache config file (/etc/apache2/apache2.conf on Ubuntu):

ScriptAlias /apecgi/ /home/dave/src.downloads/ape/src/
<Directory /home/dave/src.downloads/ape/src/>
   Options +ExecCGI
</Directory>

The Ape scripts don't the have the #!/path/to/ruby line that you'd expect to find in a CGI script, so initially I couldn't figure out how to get them to run. I pinged Tim and he told me that he uses a bash CGI script to run the Ape's entry-point go.rb script. Following Tim's example, I created a file named "go" in the Ape src directory with the following contents:

#!/bin/bash
export RUBY_HOME=/usr/local
export GEM_HOME=$RUBY_HOME/lib/ruby/gems/1.8 
export RUBYLIB=$RUBY_HOME/lib/ruby:$PREFIX/lib/site_ruby/1.8
PATH=$RUBY_HOME/bin:$PATH
export PATH
ruby go.rb

Putting the Ape HTML form in place#

So the Ape code is in place and ready to run, but I still need to serve Ape's HTML form. I decided to serve the form from HTTPD docroot, so I created a directory for it mkdir /var/www/ape. Then I went looking for the HTML form. Turns out, it's not in CVS. So I browsed to http://tbray.org/ape and used Firefox's "Save Page As..." and "Web Page, complete" to save index.html. Then I copied that index.html file into /var/www/ape and modified the action to point to /apecgi/go.

And off we go...#

Finally, I restarted Apache via sudo apache2ctl restart, browsed to http://localhost/ape and found Ape there, ready to roll.

That was a lot more complicated that I had expected; makes me wonder if I'm the first person (other than Tim, of course) to deploy the Ape.

Latest links: do you Dare criticize the APP?

2007-06-11T09:47:42+00:00

There was quite a flurry of blogging about the Atom Publishing Protocol (APP) over the weekend, all kicked off by Dare Obasanjo's criticisms of the protocol. Some of the posts were critical of Dare and his motives, but I'm thankful he started the conversation. Here's the dump from my del.icio.us account:

Dare Obasanjo: Why GData/APP Fails as a General Purpose Editing Protocol for the Web - Good only for microcontent, no support for partial updates, poor support for hierarchy and "we [Microsoft] will likely standardize on a different RESTful protocol"

Bill de hÃ“ra: APP on the Web has failed: miserably, utterly, and completely - APP editor Bill de hÃ“ra responds to Dare's APP critique
snellspace.com: Silly: James Snell responds to Dare's APP critique. Cites usage of APP in Lotus Connections.
Joe Gregorio: In which we narrowly save Dare from inventing his own publishing protocol: APP editor Joe Gregorio's point-by-point response to Dare's APP criticisms
David Megginson: REST, the lost update and the sneakernet test: - "Without exclusive locks, thereâ€™s no way to avoid this problem, but it is possible to detect it."
Tim Bray: Iâ€™ve Seen This Movie: Tim Bray responds to Dare's post: "Microsoft has tried to swim against the current of the Internet a few times before"
snellspace.com: Beyond APP - Partial updates: Dare's post inspires James to post a strawman proposal for using HTTP PATCH to implement partial updates w/Atom.
Keith Fahlgren: Partial Updates, a simpler strawman: "The basic inspiration is thinking about each operation as an XSLT template"
snellspace.com: Partial Updates in APP, Part 2: James responds to Keith's simpler strawman.
Dare Obasanjo: GData isn't a Best Practice Implementation of the Atom Publishing Protocol: Dare responds to the flurry of APP posts he kicked off

Friday Atom and REST links

2007-05-25T22:30:40+00:00

A bunch of Atom and REST related links that I came across while catching up with my blog reading today:

Atom and LDAP sitting in a tree. Trey Drake has released his OpenDS based Atom store as an open source project on Java.net at http://atom.dev.java.net. It's a directory server distributed as a Java web application that supports both Atom Publishing Protocol (APP) and Lightweight Directory Access Protocol (LDAP).

Signing, encrypting and decrypting Atom. On IBM developerWorks, Nicolas Chase explains how "digital signatures and encryption can easily mesh with Atom data using the Apache Abdera API."

Google GData: A Uniform Web API for All Google Services. Dare Obasanjo praises Google for creating a single uniform and RESTful web services API for eight of its key services, the APP based GData API. He writes "not only is it now possible to create a single library that knows how to talk to all of Google's existing and future Web services since they all use GData. It is also a lot easier to provide 'tooling' for these services than it would be for Yahoo's family of Web services given that they use a simple and uniform interface."

RESTful web services support in Netbeans. Geertjan links to blog entries and a screen-cast that explain Netbeans 6.0 support for RESTful web services, including the early access JSR-311 REST API.

Generate code from your WADL REST API. Eduardo at The Aquarium links to Thomas Steiner who is making progress on a WADL editor and a generator, bringing WSDL-like code generation to RESTful web services.

What exactly is RSS anyway?

2007-05-16T10:48:56+00:00

Is it this http://cyber.law.harvard.edu/rss/rss.html, promoted by Dave Winer?

This http://www.rssboard.org/rss-specification, from the RSS advisory board?

Or one of the other dozen or so incompatible formats.

It doesn't really matter. There's only one Atom: RFC-4287.

Hat tip to Sam Ruby.

The Apache Lucene GData server project

2007-05-15T22:18:21+00:00

German Viscuso: Generally spoken GData provides a general interface to make information available even beyond a browser context by providing a single API that could be used to query, update, and index structured data anywhere on the web. Could GData become a simple and open replacement for all the proprietary communications protocols currently in use by database vendors?

Interesting thoughts. I've heard about the Atom Publishing Protocol (APP) based Lucene Web Services API, but I hadn't heard about the Apache Lucene GData server project. Atom protocol is moving fast, especially considering the fact that it's not finished.

TS-6029 - Beyond Blogging: Feeds in Action

2007-05-10T16:27:29+00:00

As usual, I'm enjoying JavaOne and meeting lots of new people and putting faces to names. I've also been practicing my talk daily so I can give you my best performance, so eat lunch a little early or a little late and come see how it goes. Here are the details:

TS-6029 - Beyond Blogging: Feeds in Action
Friday 05/11/2007 12:10 PM -1:10 PM
Moscone Center Esplanade 307 / 310

APP and OpenID at JavaOne

2007-05-07T14:43:56+00:00

Trey Drake: How do you demo a directory server? Build cool apps around it. To that end, we've built an Atom/APP server, a lightweight OpenID server, a blogging and "twitter" like app - all powered by OpenDS. Drop by our booth (Glassfish alley at CommunityONE and .org section of the pavilion during JavaONE). Ludo and I will introduce OpenDS and show off the demos in two talks; today at CommunityONE at 5PM and Wednesday at 1:30 in the CommunityCorner.

Very cool. I'm not going to be the only one talking about Atom protocol at JavaOne. I'll have to stop by the CommunityCorner, that sounds too good to miss.

Propono 0.5 released

2007-04-23T15:32:10+00:00

ROME Propono 0.5 is a minor bug fix release of Propono. You can get the release files and updated Javadocs from the Propono 0.5 release page.

APP interop follow-up

2007-04-22T19:59:19+00:00

Following up on the APP interop event last week, here are the ROME Propono APP client issues we found:

And here are the Roller APP server issues:

I plan on fixing the Roller issues for this summer's Roller 4.0 release. I won't have time to fix the Propono issues for the upcoming Propono 0.5 release (due this week), so they'll be in 0.6 sometime after JavaOne.

WSO2 Web Services Mashup Server

2007-04-18T01:17:17+00:00

I was wondering what web services vendor WSO2 was doing at the APP interop event. Turns out, they've got a "Web Services Mashup Server" in the works.

APP interop event day #2

2007-04-18T01:16:33+00:00

The Atom Publishing Protocol interop event is over and now I'm catching up on blogs and email in my hotel room in Mountain View, CA. In the end, I was able to run ROME Propono successfully against Blogger/GData, AOL Journals and Wordpress. I also found a dozen small problems in Propono and in the Roller APP server.

For more information on the event, check O'Reilly's Keith Fahlgren's summary of the event titled Atom Publishing Protocol a Success. Keith mentions that "big industry players like AOL, Google, IBM, Microsoft, Oracle, and Sun are working on APP clients and servers and sent people to the interop event with interesting code" and I agree that's definitely a good sign for the protocol.

I thought it was particularly interesting that database vendors IBM and Oracle showed up with DB2 and Oracle-backed Atom stores. If Google's half-dozen or so Atom protocol based services aren't enough make you stand up and take notice, surely that should get your attention.

APP interop event day #1

2007-04-17T10:24:47+00:00

Day one of the Atom Publishing Protocol (APP) interop event was a success, at least from my point-of-view. I was able to test the Propono client against Blogger.com/GData, AOL Journals and an implementation from Oracle. I found and fixed problems in Roller's APP implementation. Plus, it was great to meet all of the good folks implementing Atom and to get a close look at the Google campus. I'm getting ready to drive back to Google now so... gotta go.

Check out Tim Bray's blog for some photos of the event.

ROME Propono 0.4 released

2007-04-10T15:20:26+00:00

I'm happy to announce the first release of the ROME subproject Propono. Propono is a ROME-based Java class library that supports publishing protocols, specifically the Atom Publishing Protocol and the legacy MetaWeblog API. Propono includes an Atom client library, an Atom server framework and a Blog client that supports both Atom protocol and the MetaWeblog API.

Here's the project page
http://wiki.java.net/bin/view/Javawsxml/RomePropono

Here's the Propono 0.4 release page:
http://wiki.java.net/bin/view/Javawsxml/RomeProponoRelease04

And here's a link to the API docs, which include details, diagrams and code examples:
https://rome.dev.java.net/apidocs/subprojects/propono/0.4/overview-summary.html

I'll be testing Propono this week and next (at the Google-hosted APP interop meeting) so now is a great time to provide feedback and bug reports. I plan on releasing Propono 0.5 in the *very* near future.

planet.sun.com

2007-04-07T18:28:39+00:00

We haven't released the standalone Roller-Planet application yet, but the .Sun Engineering team quietly deployed the latest bits at planet.sun.com a couple of weeks ago in response to requests from the Glassfish, SWDP and other teams for planet-style web sites. You can follow the links on the main page to find planets for Glassfish, SWDP, Sun India, Sun Alumni, Sun Java System Web Server, web services and globalization bloggers.

What's Roller-Planet? It's a community aggregation server, similar to Planet-Planet but with some key differences: it's got a web UI that enables groups of users to run their own planet sites, it's based on Java and it uses the ROME feed parser and fetcher. I've written about it before. We don't have a release plan yet for Roller-Planet so if you really want to try it you'll have to fetch and build it from the Apache Roller SVN repo.

Atom protocol interop event

2007-04-05T17:16:06+00:00

I just got approval to attend the Atom protocol interop event at Google April 16 and 17. I'll be bringing at least three Atom protocol implementations: Roller's Atom server, Propono's simple file-based Atom server and Propono's Atom client.

Atom protocol as the substrate for reliable messaging

2007-04-05T16:48:03+00:00

I've heard the argument before that the REST approach to web services doesn't give you reliable messaging and that's the reason you need to stick with WS-*. Today Bill de hÃ“ra disputes that notion with an interesting and somewhat provocative post that mentions a couple of specs for messaging via HTTP (HTTPLR and BTF) and argues that Atom protocol can serve as the basis for web-scale reliable messaging.

Bill de hÃ“ra: There are a number of reasons to choose Atom Protocol as the substrate for web-scale reliable messaging. First, a ton of software will be written to target APP in the next few years, and there is plenty of scope for extending the protocol; this suggests openly available and flexible software stacks. Second, since all document collections in Atom Protocol are served as Atom Feeds, it has inherent support for systems management and end to end reconciliation. Third, Atom entries have identity and are natural envelopes, unlike SOAP, where identity and true enveloping requires further specification (essentially raw Atom presents a better basis for interoperation than raw SOAP). Fourth, Atom Protocol can support binary content transmission not just XML, and thus can transmit arbitrary payloads. Finally, because Atom Protocol respects media types and deployed HTTP infrastructure, independent proxy inspection and security check-pointing can be installed cleanly, also eliminating the need to rewrite 2 stack layers and buy XML appliances to support and secure SOAP backed web services. It seems to be a question of when, rather than if, this will get built out.

I would have blogged about this earlier today, but Bill's blog looked foobar and I didn't realize that today is CSS Naked Day. My blog doesn't look half bad naked.

@JavaOne: Beyond Blogging: Feeds in Action

2007-04-04T17:32:34+00:00

As promised here's some more info on my JavaOne 2007 session. It's based in part on the Beyond Blogging presentation that I presented to a tiny audience at the local Tri-XML conference last year. Tim Bray didn't attend my talk, but he read the slides and called them "the single best introduction and overview I've ever seen about feeds and syndication and RSS and Atom and all that stuff." I shouldn't brag. Had he attended the talk he might have had an entirely different opinion, who knows. Anyhow, the presentation has been updated, stream-lined and large portions rewritten to cover ROME and ROME's new Propono sub-project. Here's the outline:

The web is bloggy
    Blogging, XML and web services
    Feeds on the web today
    Feeds as an integration technology
    WS Death Star
Understanding RSS and Atom
    The birth of the RSS
    The RDF fork: RSS 1.0
    Elements of RSS 1.0 (abridged)
    Feed Extension Modules
    The simple fork: RSS 0.92 â€“ RSS 2.0
    Elements of RSS 2.0 (abridged)
    RSS limitations
    Atom Publishing Format
    Elements of Atom (abridged)
    RSS and Atom feed family tree
Consuming feeds with ROME
    It's just XML
    ROME RSS/Atom feed utilities
    How does ROME work?
    ROME SyndFeed model
    How to fetch feeds
    Fetching a feed with ROME Fetcher
Producing feeds with ROME
    Serving feeds: generate XML
    Use your favorite XML tools or...
    Generating Atom with ROME
    Serving feeds
    Cache cache cache
    Serving Atom with ROME
    Feed auto-discovery
    Serving valid feeds
Publishing with ROME Propono
    Atom Publishing Protocol (APP)
    What does APP do?
    How does it do all that? The REST way
    APP Service Document
    An Atom collection <feed>
    Getting an APP collection - with paging
    ROME Propono
    ROME Propono Atom Client API
RSS/Atom trends
For More Information
Summary
Q & A

And for the folks who've written to me about Propono, the new ROME subproject devoted to publishing via Atom protocol (and the MetaWeblog API): I hope to commit the code to ROME and starting blogging about it this week.

Latest links: March 21, 2007

2007-03-21T22:42:13+00:00

David Van Couvering: Why use Atom Publishing Protocol for REST?
"Why not just use HTTP and JSON and have done with it?"
Max Ross: Ode to Hibernate (and Shards)
"Shards, a framework that adds support for horizontal partitioning (or in Google parlance, "sharding") to Hibernate Core"
James Snell: Don't Panic
"it is possible to get very far with nothing but the base APP protocol."
Elias Torres: Is AtomPP for or not for blogs?
"we'll probably have to do some good amount of extra work in defining how blogging clients will interact with a blogging service"
Rob Yates: Atom Publishing Protocol - not enough for blogs?
Rob wants a guide that "allows blogging clients to be interoperable with blogging servers"
Re: I-D ACTION:draft-ietf-atompub-protocol-14.txt
Roy Fielding: "the sky is not falling, and AtomPP implementations will interoperate just fine"

Newsgator launches Java ME powered feed reader

2007-03-15T12:06:36+00:00

Newsgator, makers of FeedDemon and NetNewsWire, have launched a Java ME powered RSS/Atom feed reader. Here are some details from the press release:

NewsGator Go! for BlackBerry and NewsGator Go! for Java were developed in a technology partnership with FreeRange Communications, the leading developers of mobile publishing and mobile RSS solutions, and is available for BlackBerry, Sony Ericsson, Nokia, and most other MID-P 2.0 mobile phones, including Samsung, Sanyo and LG. NewsGator Go! is available for $29.95 or $14.95 if bought with any of NewsGatorâ€™s desktop readers, including FeedDemon, NetNewsWire and Inbox.

Via Randy Morin.

Sun Web Developer Pack R1 with RSS and Atom goodies

2007-03-12T21:20:29+00:00

The Sun Web Developer Pack (SWDP) finally uncloaked today, so I can talk a little more openly about what I and my Java EE co-workers have been working on. You can get the full scoop at the SWDP site, but basically SWDP is a bundle of technologies to help developers build "Web 2.0" or next-generation web applications on the Java platform. Ajax, scripting languages, REST and of course RSS/Atom are all part of that. The RSS/Atom bits are ROME 0.9 (Beta), Blogapps 2.0 (Early Access) and a ROME-based Atom Server kit based on code from Roller. There are also example Atom server implementations in the REST API and Phobos components of the pack.

I'm pretty excited that we're putting some resources behind ROME and that both ROME and Blogapps are part of SWDP R1, but I'm even more excited about the next release. In R2 we'll drop the Atom Server Kit and Blogapps BlogClient and we'll replace them with ROME Propono a brand new Atom protocol client and server library that we're getting ready to contribute to the ROME project.

I haven't had a whole lot of time to experiment with the various components in the pack, but I have played with Phobos and I think it's pretty compelling. Phobos is a "lightweight, scripting-friendly, web application environment." It's not just for creating server-side JavaScript applications, but that's the angle the I find interesting. So many developers are creating JavaScript/Ajax applications these days that working in JavaScript on both client and server-sides makes sense -- especially when you can debug into JavaScript code in your IDE as you can with the Netbeans Phobos module. Also note that the jMaki Ajax components work with JSP, PHP and Phobos -- jMaki and Phobos look like a winning combination.

JavaOne here I come!

2007-03-02T21:28:09+00:00

I'm very happy to report that my talk was accepted for JavaOne 2007. I'll be giving a thoroughly revamped and updated version of my Beyond Blogging talk (aka Johnson on Feeds), which adds in-depth coverage of ROME and ROME Propono. I'm excited about going to JavaOne again, but I'm also feeling a bit frazzled. I'm swamped with work right now and I've got a stack of deadlines clustered around the March 20 due date for the slides.

I helped out with the session selection for the web-tier track this year so I can say with some confidence that at least one track is going to rock -- we've got a wonderful set of talks lined up, so don't miss it. The full session schedule isn't online yet, but registration is open.

Atom protocol powered Blogging Portlet

2007-02-12T23:17:17+00:00

Jeffrey Blattman has put together a new Portlet that makes it possible to blog via Atom Publishing Protocol (APP) from a portal page. He's added it to the Open Source Portlet Repository on Java.net so you can try it out against your favorite APP server.

APP interop at E-Tech

2007-01-29T23:42:10+00:00

James Snell brings news of an Atom Publishing Protocol (APP) interop event to occur in March at the O'Reilly E-Tech conference. I'd love to use that as an excuse to go to to E-Tech, since my current work involves APP in ROME and Roller, but it sounds like I can participate remotely.

Placeblogger

2007-01-03T08:14:59+00:00

Placeblogger is a new blog and aggregation site that's all about local blogging from Lisa Williams and friends. It's powered by Bryte, which is based on the Drupal content management system and offers blogs, feed aggregations, photo galleries and polls.

You can help build the database by submitting your favorite place blogs. The database supports a number of different "blog types." You can add aggregations, so Joe's local planets would be suitable, and you can add community sites so Orange Politics would fit right in too. I submitted Raleighing.

Here's some more reading on the topic:

Rich Burridge's blog-to-book blogapp

2007-01-02T14:55:15+00:00

Rich has put together a interesting blogapp that pulls all entries from a blog and turns them into a book, using either cups2pdf or OpenOffice.org Writer. I had the same idea when I was writing RSS and Atom in Action, but I was going to go the DocBook route and eventually dropped the idea because DocBook seemed a bit too complex.

I don't think Rich's work is Roller-specific. Rich used Grabber to get the entries out of Roller and into simple HTML files, so the approach should work with other blog servers that support the MetaWeblog API.

Farewell to 2006

2006-12-31T23:11:21+00:00

I've been too busy with year-end projects to blog over the past couple of days and now suddenly, it's time to say farewell to 2006. So I'll do that with a quick summary of the year.

2006 was a pretty good year for me. I published my first book: RSS and Atom in Action. Roller is still growing, reached 3.0 status and is now very close to becoming a top level Apache project. IBM started contributing to and announced a Web 2.0 product suite that will include Roller. I did my first solo JavaOne presentation and spoke at both ApacheCon EU and ApacheCon US. And, I haven't mentioned it yet, but I also landed a new job inside Sun, which starts on January 8th (more about that later).

On the home-front: the boys (now 4, 8 and 10) are all healthy, happy and doing well in school. We celebrated my dad's 70th birthday and Alex's 10th birthday. We took family trips to Ocracoke, Atlanta, Austin, Northern Virginia and made numerous visits to the in-laws beach house near Topsail Island. Plus, Andi and I escaped from the kids for a week in Ireland to celebrate our 15th wedding anniversary -- our first kidless vacation in about ten years.

I hope you had a good year too and will have an even better 2007. Happy new years!

New Atom protocol spec draft and Queen City planets

2006-12-27T16:50:19+00:00

Joe Gregorio announces a new Atom Publishing Protocol Spec (draft #12) and he says it might end up being the final. I guess it's time for a new Blogapps release with APP draft #12 and ROME 0.9 support.

Plus, Joe has put together a set of new planet sites for towns in the Charlotte, NC area; all based on feeds from Google Base, Google Blogs, Google News, Craigs List, Flickr and the Weather Service. The sites look useful, but the ads combined with the minimalist design make them look a little spammy on first glance. Perhaps a short "about this site" paragraph is in order.

Roller-Planet mind map

2006-12-22T22:16:15+00:00

I'm glad I was able to help Simon get his personal planet back online yesterday. And I'm glad the task was fairly easy. All Simon needed as a new version of Blogapps PlanetTool updated to use ROME 0.9 and I was planning on doing that anyway.

What's PlanetTool you wonder? PlanetTool is a command-line program which reads a set of RSS/Atom newsfeeds and then uses a set of templates to generate a planet site with HTML, RSS, Atom, OPML and other representations. Simon uses it to bring together his personal blog, Sun blog, del.icio.us links and Flickr.com photos into a single webpage and a single feed. If you subscribe to that feed, you'll get just about everything that Simon publishes to the web.

If you're interested in learning more about PlanetTool, here are some of my previous posts on the topic:

Try PlanetTool, it's easy!
ROME + Texen = PlanetTool
PlanetTool internals
It's also covered in Chapter 11 of RSS and Atom in Action.

The above title Try PlanetTool, it's easy! is a little misleading, but it brings me to my point. PlanetTool is only easy if you're a developer or a power-user; somebody who can handle running Java on a server, editing an XML config file and setting up a cron job. Simon could handle it, but I'd like to make planets easier.

In fact, I'd like to make it as easy to create a planet as it is to create a blog. This past week, I've been thinking about how to do that by taking the simple ROME powered Roller-Planet code, which is found in both Roller and PlanetTool, and build it into a multi-user planet server -- kinda like Roller, but for planets instead of blogs. To get my thoughts into digital form I worked up a little FreeMind mind-map on the topic, dumped it to text, added some wiki syntax and some screen-shots. The result is this: a RollerPlanetMindMap that outlines ideas for the future development of Roller-Planet.

JavaOne 2007 call for papers closes today

2006-12-15T08:52:44+00:00

There's still time to get those proposals in. I ended up submitting three proposals for technical sessions related to RSS/Atom and one for a Roller birds-of-a-feather (BOF) session.

Here's the link to submit proposals: http://www.cplan.com/sun/javaone07/cfp.

ROME 0.9 (beta) is available

2006-12-11T18:52:49+00:00

A new release of the RSS and Atom Utilities (ROME) project ROME 0.9 (beta) is now available on the project's Java.net website. This new release includes fixes to Atom relative URI resolution, easier parsing for RSS feeds that use <content:encoded>, better support for mapping of RSS to and from Atom and numerous small fixes. Here are some quick links to the relevant release docs and files.

Let's review a couple of the changes that might affect the ways you use ROME, starting with the RSS parsing changes.

Easier parsing of RSS feeds that use <content:encoded>

Now that ROME includes built-in support for <content:encoded>, most user will no longer need to download and configure the separate and optional Content Module plugin. That's important because many RSS feeds use <description> as item summary and <content:encoded> as item content. It's also important because Atom supports both entry <summary> and <content>. So, whether you are parsing RSS or Atom the code to get an RSS item or Atom entry summary and content is the same. To this this, we made a number of small changes:

Added a new com.sun.syndication.feed.rss.Content object to the RSS model
Added code to converters to convert RSS Content to SyndEntry Content and vice-versa
Added code to generate <content:encoded> if content is present

Here's a simple example that shows how to parse and print out parts of an RSS feed with entries that have both summary and content (using ROME's SyndFeed model):

    String urlstring = "http://scripting.wordpress.com/feed/";
    InputStream is = new URL(urlstring).openConnection().getInputStream();
    SyndFeedInput input = new SyndFeedInput();
    SyndFeed feed = (SyndFeed)input.build(
        new InputStreamReader(is, Charset.forName("UTF-8")));

    Iterator entries = feed.getEntries().iterator();
    while (entries.hasNext()) {
        SyndEntry entry = (SyndEntry)entries.next();
        System.out.println("-------------");
        System.out.println("Title: " + entry.getTitle());
        System.out.println("Published: " + entry.getPublishedDate());
        if (entry.getDescription() != null) {
            System.out.println("Description: "
               + entry.getDescription().getValue());
        }
       if (entry.getContents().size() > 0) {
           SyndContent content = (SyndContent)entry.getContents().get(0);
           System.out.println("Content type=" + content.getType());
           System.out.println("Content value=" + content.getValue());
       }
    }

That certainly makes things easier in most cases, but if you need full support for all of the features defined by the Content Module specification you can still download and use ROME's Content Module plugin -- it works exactly as it did before.

Better mapping for mapping Atom and RSS

As ROME users already know, ROME supports parsing feeds to three different object models the RSS model, the Atom model and "SyndFeed" -- an abstract model that you can use when you do not care about the source format of the feed. ROME includes converters to convert to and from the different models. We made two changes to improve the mapping done by the converters. First, as you saw above, we added support for Content to our RSS model. That allows us to map Atom summary/content nicely to and from RSS description/content, like so:

RSS <description> <--> SyndEntry.description <--> Atom <summary>
RSS <content:encoded> <--> SyndEntry.contents[0] <--> Atom <content>

Second, we added better support for Atom title and subtitles. In the old ROME, there was some potential for information loss because the SyndFeed model did not completely support the Atom model. Specifically, titles and subtitles did not support a type attribute. To fix that without breaking the API (i.e. ensuring that old ROME code still compiles and works), we added a number of new methods to make the type information available by returning title and description as SyndContent objects (which have type and value properties).

SyndFeed additions:

public SyndContent getTitleEx()
public void setTitleEx(SyndContent title)
public SyndContent getDescriptionEx()
public void setDescriptionEx(SyndContent desc)

SyndEntry

public SyndContent getTitleEx()
public void setTitleEx(SyndContent title)

We left the existing getters and setters in place, so your existing code will continue to compile and work properly. And that's it for the quick review...

What's next?

Download it, try it out, report bugs and give the ROME project some feedback and support. And what's next for ROME? I think ROME is just about ready for a 1.0 release, but there's one last feature we need to add: some form of feed security to strip potentially malicious JavaScript from feeds. Then onto ROME2.

For even more information on ROME...

There's a complete chapter on ROME in my recently published book RSS and Atom in Action.

Apache Abdera 0.2.0 (incubating) released

2006-12-07T11:45:49+00:00

Abdera is an open source Atom parser, generator, client and server tool-kit for Java. James Snell announced a new version of Apache Abdera (incubating) the other day and the feature list is impressive, especially for a "0.2.0" release. Here's an excerpt:

The goal of the Apache Abdera project is to build a functionally-complete, high-performance implementation of the IETF Atom Syndication Format (RFC 4287) and Atom Publishing Protocol (in-progress) specifications. [... incubator blah blah blah ...]
A reworked API that improves usability
Decoupled extensions from the underlying parser implementation
An Atom Publishing Protocol client implementation
Updated support for the current Atom Publishing Protocol draft specification
Added support for Internationalized Resource Identifiers (IRIs)
Improved Thread Safety
Fixed a number of Classloader issues that kept Abdera from working properly in application server environments.
Improved Javadocs
Added test cases and sample code
Added experimental Bidirectional Text support
Improved implementation of OpenSearch v1.0 and v1.1 extensions
Implementation of MediaRSS extensions
Implementation of Feed Paging and Archiving extensions
GoogleLogin Authentication Support

We might have to steal that IRI support for ROME. Actually, that's something that should be built right into the Java platform. Apparently IRI support was considered for Java SE 6 and something was implemented, but then rolled back.