Blogging Roller

This is the first in my series of Web Integration Patterns. Check out the intro at this URL http://rollerweblogger.org/roller/entry/web_integration_patterns

Synopsis

Use links as a way to integrate web sites & applications via navigation and relationships between resources.

I’ve been meaning to write on this topic for a some time and to explain how I’ve gone from being an advocate of RSS/Atom feeds, Atom Publishing Protocol and things bloggy to being a proponent of Linked Data (video), Semantic Web, RDF and other things that I previously considered to be nuisances. I've also got a new topic and blog series to announce, so here goes.

Mac-based weblog editor MarsEdit v3.0 has just been released by Red Sweater Software.

Because MarsEdit v3.0 supports the Atom Publishing Protocol (APP), it works with Apache Roller 5.0RC1's new and improved AtomPub support right out of the box. I'm writing this post in MarsEdit right now and publishing to Roller 5 via APP.

MarsEdit includes a rich text editor and supports image upload via drag-and-drop into the editor. You can edit a draft post locally or use Send to Blog to send the draft to the server for further editing before publish. You can specify multiple tags for your post and MarsEdit will send them to Roller as tags.

Generally, I prefer to edit my HTML by hand but for somebody who wants simplicity and a familiar Mac interface, MarsEdit v3.0 looks like a great choice. And the HTML it generates doesn't look too bad.

Here's a screenshot that shows what MarsEdit looks like when editing a post:

Here are the settings I'm using (URLs changed to protect the innocent).

MarsEdit is $39.95 and there's a 30-day free and fully-function trial version available too.

It's been a while since the BETA (over 6 months) but we now have a release candidate for Apache Roller 5.0 available for testing. This site is running Apache Roller 5.0 RC1 right now, as you can see in the itty bitty screenshot below:

Here's a What's New in Roller 5.0 page that summarizes what has changed since 4.0. One thing I forgot to mention on that page was that Roller now uses ROME Propono 1.0 for AtomPub and Roller 5.0's AtomPub support has been successfully tested with MarsEdit and Windows Live Writer.

(I just returned to work after vacation and a week of conferences in the SF bay area. Instead of posting my trip reports to the limited audience that reads my internal IBM blog, I'm going to post them here so that everybody can benefit from them.)

On Monday Nov. 2, I attended Social Web Camp at Sun's Santa Clara campus. There were about 40 people in attendance. The event was organized by Sun's Henry Story, an expert in semantic web technologies and inventor of the FOAF+SSL approach to implementing Social Networking features (relationship based privacy). Unfortunately, Henry was not able to attend the conference because he was detained by US immigration.

• Social Web Camp, Santa Clara, CA
  http://barcamp.org/SocialWebCamp-Santa-Clara
• Sun's Santa Clara Campus - AKA the Agnews Insane Asylum
  http://www.nps.gov/nr/travel/santaclara/agn.htm
• FOAF+SSL distributed/open social networking
  http://esw.w3.org/topic/foaf+ssl
• Henry detained for 6 days, case dismissed in 30 seconds
  http://twitter.com/bblfish/status/5509198135

During the camp, I lead a session on OpenSocial using my "What's up with OpenSocial" slides from BarCampRDU. Surprisingly, very few people were familiar with OpenSocial, so this was an introductory level discussion.

• What's up with OpenSocial preso from BarCampRDU
  http://docs.google.com/present/edit?id=0AWuMBlP1tnN6ZGcyY2ZuendfOThmcXMydjdmcQ&hl=en

I participated in a session on enterprise social networking and shared a little about we do with micro-blogging inside IBM, mentioning BlueTwit and the new features in Lotus Connections. A couple of folks from Boeing were present and described the home-grown social networking and micro-blogging system.

• BlueTwit mentioned in Business Week
  http://www.businessweek.com/magazine/content/08_22/b4086056643442.htm

John Panzer of Google pitched his new Salmon protocol, a distributed commenting system that allows comments made on items in downstream systems (e.g. aggregators, social networks, FriendFeed, etc.) to find their way back upstream to the source item. The protocol is based, in part, on AtomPub. Comments are signed and posted back upstream. Seems like this could be useful in both Lotus Connections river of news feature, Jazz-based products and Roller; so I'll going to track this one closely. It might be fun to try to implement Salmon for Roller.

• Salmon: comments and annotations to swim upstream, spawn more commentary
  http://www.salmon-protocol.org/

I missed a little of the conference because I had lunch with some of my former co-worker from Sun and I left a little early to return my vacation rental car and make my way to Oakland for ApacheCon US 2009. More about that later...

Yesterday I wrote about OAuth support in the upcoming Roller 5.0 release. Today I'm following up with a post about OAuth support in ROME Propono.

As you may remember, ROME Propono is a subproject of ROME, the Java-based RSS/Atom feed library. ROME Propono includes an AtomPub server library and an AtomPub client. I added OAuth support to the AtomPub client and in this post, I'll show how you can use it to post to the Roller 5.0-dev (i.e. the snapshot build that I made available yesterday).

ROME 1.0 and coming soon: ROME Propono 1.0

In case you haven't already heard, thanks to the recent hard work of Nick Lothian, ROME 1.0 is now available. You can find downloads at rome.dev.java.net and a list of changes in the Change Log there. To celebrate this momentous event, I'm planning on releasing ROME Propono 1.0 as well, and in preparation, I've made a release candidate available. The new Propono includes ROME 1.0 and support for OAuth. You can get it via the links below:

rome-propono-1.0RC1.tar.gz (2.0 mb)

rome-propono-1.0RC1.zip (3 mb)

Posting to Roller via AtomPub and OAuth

To use the Propono AtomPub client, you place the Propono jars in your Java VM classpath and then call the AtomClientFactory to get started, as described in the ROME Propono 1.0 Javadocs.

Below is a Groovy example that shows how to post a blog entry to Roller via AtomPub and OAuth. You can get the consumer key, secret and URLs you need to call your instance of Roller from the OAuth Credentials page in the Roller admin interface.

  import com.sun.syndication.propono.atom.client.*
  import com.sun.syndication.feed.atom.*

  def authStrategy = new OAuthStrategy(
    "roller",                               // username
    "55132608a2fb68816bcd3d1caeafc933",     // consumer key
    "bb420783-fdea-4270-ab83-36445c18c307", // consumer secret
    "HMAC-SHA1",                            // key type
    "http://blogs.example.com/roller-services/oauth/requestToken",
    "http://blogs.example.com/roller-services/oauth/authorize",
    "http://blogs.example.com/roller-services/oauth/accessToken")

  // get the AtomPub service
  def appService = AtomClientFactory.getAtomService(
    "http://blogs.example.com/roller-services/app", authStrategy)

  // find workspace of my blog
  def blog = appService.findWorkspace("Blogging Roller")

  // find collecton that will accept entries
  def entries = blog.findCollection(null, "application/atom+xml;type=entry")

  // create and post an entry
  def entry = entries.createEntry()
  entry.title = "TestPost"
  def content = new Content()
  content.setValue("This is a test post. w00t!")
  entry.setContent([content])
  entries.addEntry(entry)

If you have questions or feedback about ROME Propono 1.0 RC1, please post them to the ROME dev mail list and I'll do my best to respond there.

Over the past month or so I've been adding OAuth support to just about every open source project that I can commit to. I added OAuth support to Roller so that you can now use OAuth to protect Roller's AtomPub server and other things. I also added OAuth support to ROME Propono's AtomPub client so you can now use Propono to post to Roller (more about that later). Here's a quick overview of how OAuth in Roller works.

NOTE that this post applies to Roller 5.0, which has not yet been officially released.

Setting up OAuth for AtomPub in Roller

If you want to use OAuth with AtomPub on your Roller site, go to the Server Admin page and find the Web Services section, enable AtomPub and specify 'oauth' as the authentication mechanism, like so:

Getting your OAuth key, secret and URLs

Once you've done the setup, you'll find an OAuth Credentials link on the Roller Main Menu page, which will lead you a page like the one below showing your OAuth consumer key & secret and, if you are a site admin user, the site-wide key & secret. Currently, there's only one set of site-wide credentials; I plan to fix that.

Of course, those aren't my real keys. You'll want to keep your OAuth keys secret as they can enable anybody to access your Roller account via AtomPub.

Want to try it yourself?

I mentioned that Roller 5.0 has not yet been released and that's true. There's still a lot of work to be done on 5.0, but that doesn't mean you can't get your hands on the code and binaries now. To make it easy, I've made an unofficial snapshot version of Roller 5.0-dev available for testing purposes only. It's what I'm running on my site. You can get it here in two flavors:

apache-roller-5.0-dev-20090321-SNAPSHPOT.tar.gz (31 mb)

apache-roller-5.0-dev-20090321-SNAPSHPOT.zip (31 mb)

The instructions in the old Roller 4.0 installation guide should work fine, so follow them to install and configure the 5.0-dev SNAPSHOT. Please send questions and feedback to either the Roller dev mail list and I'll do my best to respond there.

You'll also need an OAuth capable AtomPub client. More on that topic tomorrow...

I'm going to be following up my OAuth everywhere! post, with several more OAuth related posts this week. So, just in case you are wondering "why is Dave going off on this cockamamie OAuth tangent?", I'll take some time now to explain a little about OAuth to help you understand.

OAuth is a emerging protocol that one web site can use to access your data on another website without asking you to reveal your username and password. For example, when the sinister BuddyNet9000(TM) Social Network site wants to access your GMail account so it can spam your "friends" on your behalf, you can use OAuth to give it access without telling it your username and password. Why risk your GMail security when all you want to do is spam some people? There are less snarky examples, but that one makes the point well, I think.

There's a good end-user oriented introduction on OAuth.net titled Beginner's Guide to OAuth: Protocol Workflow. OAuth is not that widely deployed yet, and is not perfect, but it is emerging and going the IETF standards route.

I'm interested in OAuth because it's part of the OpenSocial spec, used to authorize access to the OpenSocial REST API and to enable OpenSocial Gadgets to call out to OAuth protected resources. Also, because it's used to protect AtomPub-based services, including the Google Data APIs. I needed to learn about it for my Roller and SocialSite work and if you're going to be doing much OpenSocial work, you'll need to learn about it too.

Congratulations to the Apache Abdera team, who've just graduated to full Apache top level project status. The don't have the new site at abdera.apache.org up yet and they're still not quite at 1.0 yet, but this is a major milestone. They've got the best Atom format and protocol toolkit around, in my opinion.

via Garett and James.

A lot has changed since I did my first and even my second "how does Shindig/Java work" diagram. Believe it or not, there are now two separate web services protocols in OpenSocial and thus in Shindig. How did that happen you wonder, well read on...

Oh and by the way, I have not given up on that Delicious.com Blog Posting tool that I wrote about in August. I've got the feed fetching, entry composing parts done now and I'm currently working on the blog posting code. I hope to have something ready to blog about this week.

Here's a diagram I worked up over the weekend to explain Shindig REST API internals to my team mates. See the Project SocialSite blog for the full story.

• snellspace.com: Contact!
  "So Google announced it?s Contacts API? based on Atompub of course"
• netzooid: Writing Your First AtomPub Service with Abdera
  "This shows the much simplified server code base in action, so beware - this requires the latest trunk/ code. 0.4.0 should be out soon though"
• Astoria: AtomPub support in the ADO.NET
  "We have been looking for the last few months at adding first-class support for AtomPub to Project Astoria"
• Marc Hadley's Blog: Authentication in Jersey
  "Actions on some resources require authentication and I need access to the name of the authenticated user in the resource method"
• The Aquarium: Jersey 0.6 released
  "the Jersey client API, enhanced JSON suport, and better integration of Jersey with IoC containers"
• Carrot2 - Open Source Search Results Clustering Engine
  "Carrot2 is an Open Source Search Results Clustering Engine. It can automatically organize (cluster) search results into thematic categorie"
• MySQL Forums: Partitioning
  The official discussion forum for those interested in or experimenting with MySQL Partitioning technology
• Partha's Blog: MySQL Table Partitioning
  Nice series of posts on table partitioning in MySQL 5.1
• Simon Phipps: The Adoption-Led Market
  "emerging approach which is rapidly spreading for smaller software projects and in my view is the future of all software acquisition"
• Will Acquia ruin Drupal? | Open Source | ZDNet.com
  "Acquia, a commercial version with all the bells and whistles, including venture capital partners, who’ve put in $7 million."
• JPost.com | BlogCentral
  Jerusalem Post is blogging with Roller!
• triangletweetup wiki
  Triangle Tweetup Attendance & information

I missed this one in my social networking API link-fest yesterday: Google announced version 0.7 of the OpenSocial API, some of the data APIs are outlined in the spec and they're still using AtomPub protocol (just like GData).

I had heard there was some push-back against AtomPub, but I really don't know what is going on because there is no transparency at all in the specification development process. So, who knows, but I really don't think they have time to invent an all new protocol. In fact, they'd better wrap things up tout de suite because Google's planning to go live with OpenSocial on Orkut during the last week of February.

I've got to carve out some time ASAP to take a close look at this. The code is in Abdera SVN and there's <a href= "http://cwiki.apache.org/confluence/download/attachments/69763/abderaserverguide.pdf"> 20-minute implementation guide (PDF) too:

James Snell: Dan Diephouse and I have been spending the last week refactoring the Abdera server framework with the goal of making is less complicated, easier, and generally better.

There have been some interetsing Atom items recently on James Snell's blog. There was a link to a nice IBM DeveloperWorks article on using Abdera to write Atom feeds to JSON. (I was kinda hoping for an XSL/T, but not matter how you do it, it's a definitely feature that belongs in Abdera.) And there's a good quote from Adrian Sutton's blog post about the Atom features in IBM's Roller and Abdera based Lotus Connections product. And most recently, James linked to the new Google Feed Server project: an Abdera based AtomPub server from Google. All good stuff.

Gotta say, Abdera's looking better and better. It's got the features (IRI support, encrytion, pluggable auth, STAX parsing, etc.) and it's got Dan and appears to have some good momentum going. At this rate, I'm not sure how much more work I'll be putting into ROME Propono. If somebody were to, just for example, contribute an Abdera based AtomPub implementation to Roller, I might have to stop entirely.

• snellspace.com: Sync!
  "Within the course of implementing several Atompub servers, the issue of “feed synchronization” has come up repeatedly"
• Paul Fremantle's Blog: A new kind of (SOA) Registry
  "as we looked at the REST space, we kept noticing how close the [AtomPub] is to our needs"


• OpenSocial - Berlin Web 2.0 Expo
  Google API evangelist Pat Chanezon's OpenSocial presentation
• opensocket - Run OpenSocial Gadgets in Facebook
  An OpenSocial container written as a Facebook application.
• Ruby on Rails: OpenSocial container plugin 0.0.1
  "This is a very early version, but it is under very active development "


• IBM's Carol Jones on Web 2.0 Research and Collaborative Software
  Redmonk video discussion of Lotus Connections, Dogear, internal/private vs. external bookmarking
• Wayne Horkan's weblog eclectic: Weblog language translator
  With Roller specific functionality. Based on JavaScript and Google translation API
• ZDNet.com: Social nets and identity fragmentation
  FaberNovel Consulting has mapped out some trends in social networking and digital identity"
• Postgres at Hi5: June 2007 user group preso
  How unique features of PostgreSQL helped Hi5 scale (PDF presentation)
• Scaling Java and PostgreSQL with Hyperic
  Another perspective on PostgreSQL scalability at Hi5 (PDF presentation)
• Skrentablog: Scaling Facebook, Hi5 with memcached
  "discussion of high volume [sites] using memcached as a critical scaling tool"

Joe Cheng posted another entry in his series explaining the details of AtomPub support in Windows Live Writer (WLM), titled WLW+AtomPub, Part 2: Authentication.

Wondering what WLM looks like? Travelin' Librarian has a nice set of screen-shots of WLM on Flickr including shots of the installation process, HTML mode, preview mode and more. Looks pretty sweet.