Blogging Roller

WIP #6: Feed-based Integration

Dave Johnson — Mon, 8 Oct 2012 10:38:43 +0000

This is the sixth in my series of Web Integration Patterns. Check out the intro at this URL http://rollerweblogger.org/roller/entry/web_integration_patterns

Synopsys

This pattern is about integrating web sites and applications by using standard feed formats to convey timely information, updates, status messages, events and other things from one web application to another.

Motivations

Take advantage of the many tools and services that can produce, consume, aggregate and build work-flows around feeds as a way to integrate web applications.
Use a standards-based approach to ensure that that widest range of feed producers and consumers can be supported. Make it so everybody who needs can read your feeds.

Related Patterns

Search Integration
Web APIs

A feed is a list of entries, each with an timestamp, ID, title, content and metadata like categories and tags. Entries are arranged in reverse chronological order. The entries in a feed can represent just about anything from blog entries, Flickr photos, YouTube videos, source-code change sets or tasks in a change management system. A feed is an XML resource that is available at a URL. If you want updates, then you poll that URL, ideally using HTTP Conditional GET so that you only pull down the feed when it has been updated.

Generally speaking there are two standard feed formats in use on the web: RSS and Atom, both are based on XML. Both use different element names, for example: what Atom calls "entries" RSS calls "items." Because these standard formats are so widely supported, providing a feed is an effective way to share updates from your web site or application.

Elements of Atom (from my 2006 presentation on Atom)

Another flavor of feeds is ActivityStrea.ms, which is essentially a feed format with a schema for representing about 70 different types of activities. These activities can be social network activities like share or friend and they can also be business activities like assign, resolve or schedule. One advantage of using the ActivityStrea.ms standard is that it has both an Atom and a JSON mapping.

Producing Feeds

Feed-based Integration is listed as a basic pattern because it can be very easy to implement. The ability to produce feeds is built-in to many different types of web applications from blog and wikis to continuous integration servers. If you are writing your own web application, you can choose to use XML tools to produce your feeds, a templating engine or a dedicated feed toolkit like ROME. You’ll find plenty of XML tools and templating engines no matter what language you are using.

Which type of feed should you produce? That depends. Atom is the most complete specification and is a true IETF standard, so often it is the right choice. To make the right decision, you have to consider who is going to be consuming your feeds. If your consumers prefer RSS, them give them that. If your consumers prefer JSON over XML, then consider ActivityStrea.ms in JSON flavor.

Consuming Feeds

You can use a wide variety of tools to parse and process feed data. For example, there are many web sites and services that can digest feeds and trigger other events and processing. Services such as Yahoo Pipes and If This Then That can read feeds, process each item and perform other actions based on item values.

Processing a feed with Yahoo Pipes (from Marsh Gardiner's post)

You’ll also find a variety of JavaScript widgets that can parse and display feeds in a website. Most web portal and software with a web dashboard features will include a component that can display feeds. And, of course, you can use a reader that supports RSS/Atom feeds, such as Google Reader or Feedly or FlipBoard, to monitor and read feeds.

If you need to add RSS/Atom reading features to your own software, you can use standard XML parsing tools and, for most language, you’ll find that there are open source libraries specifically designed for parsing feeds.

For more information in producing and consuming feeds, RSS and Atom in Action. Also, you might find my presentation on Beyond Blogging: Atom Format and Protocol useful.

Beyond the basics

Feeds are a great way to do simple integrations, but there are limitations and there will be times you’ll need to go beyond the basics with RSS and Atom feeds. Here's an example. Normally, with feeds, clients have to repeatedly poll the feed URL for updates. This is annoying, and inefficient, even with HTTP Conditional GET. To address this problem, you can setup a PubSubHubub server that will subscribe to feeds and will then notify other subscribers instantly when updates are available, so that those other subscribers don’t have to poll.

Another problem is that, if you don’t poll often enough, you might miss some updates and they may “scroll” off the bottom of the feed before you see them. Feed providers can address this problem by supporting Feed Paging and Archiving, which allows clients to use next and previous links to “page” back to feed items that are no longer in the first page of the feed.

One more beyond-the-basics item to mention is the related pattern Web APIs, which we'll cover later. Web APIs are listed as a related pattern to Feed-based Integration. That's because feeds have been used as the basis for several "Web APIs" or protocols. These protocols specify how to use HTTP POST, GET, PUT and DELETE and create, retrieve, update and delete web resources that are represented as feed entries. Examples are the IETF's Atom Protocol, Microsoft's OData and Google's GData APIs.

That’s it for Feed-based Integration. In my next posts, we'll move in to the Advanced Patterns.

HTML5 Microdata and Schema.org

Dave Johnson — Thu, 9 Feb 2012 08:48:06 +0000

Worth a read and related to my Web Integration Patterns post on Embedded Properties in HTML:

HTML5 Microdata and Schema.org:HTML5 Microdata and Schema.org On June 2, 2011, Bing, Google, and Yahoo! announced the joint effort Schema.org. When the big search engines talk, Web site authors listen. This article is an introduction to Microdata and Schema.org. The first section describes what HTML5, Microdata and Schema.org are, and the problems they have been designed to solve. With this foundation in place section 2 provides a practical tutorial of how to use Microdata and Schema.org using a real life example from the cultural heritage sector. Along the way some tools for implementers will also be introduced. Issues with applying these technologies to cultural heritage materials will crop up along with opportunities to improve the situation.

WIP: Resource Preview

Dave Johnson — Fri, 3 Jun 2011 09:00:35 +0000

This is the fourth in my series of Web Integration Patterns. Check out the intro at this URL http://rollerweblogger.org/roller/entry/web_integration_patterns

Synopsis

Enhance links shown in HTML pages so that users can hover, mouse-over, or use some other gesture, to view a preview of the resource at the other end of the link.

Motivations

Make it convenient for a user to get information about a link but without having to navigate to the link and without having to leave the current web page in the browser.
Make applications appear to be part of one integrated whole by enabling them to delegate to each other's user interfaces for preview display.

Complements

Links
Delegated Resource Creation & Selection
Common Navigation

This pattern allows users to view a preview of links that are displayed in a web page. This preview might be a simple tool-tip with plain-text, a more rich JavaScript-driven display with hyper-text and graphics, or it might be a thumbnail view of the page at the other end of the link.

Resource Preview works well for web applications that use the Links pattern for integration and use links to establish relationships between resources. If a user is looking at a Bug Report, for example, they might see a list of links to associated resources like attachments to the bug, test cases impacted by the bug, etc. The user can hover over links and see a preview with key information for each link, without leaving the current web page that they are on. Below are three different approaches to applying the Resource Preview pattern in web applications.

Approach #1: Preview Site offers previews to other applications

With this approach, there is a special preview site that crawls a set of web sites and creates Resource Previews of the web pages of those sites. Then, web sites that wish to display previews on links use some special JavaScript magic to display those previews. An example of this preview site approach is Snap.com’s Snap-Shots service. Anybody with a web site can sign up for the service, get some special HTML code that can be added to their site to display a graphical and thumbnail-style preview whenever a user hovers over a link.

Here’s an example of a Snap.com preview on the company’s blog:

The advantages of the preview site approach are participating sites don’t have to do anything except for adding a small bit of JavaScript code to enable previews. The disadvantage is that the automatically generated previews often can’t offer more than a page title, thumbnail image and perhaps a short excerpt from the page being previewed. Also, it’s interesting to note that there was a backlash against Snap-Shots and some people thought they were distracting and gimmicky. Regardless of that criticism, this approach to Resource Previews is not very interesting from a Web Integration Patterns point of view because the focus is making one site more useful and interactive, and not integrating with others.

Approach #2: Application creates and displays previews as needed

With this approach, an application that needs to provide preview creates the previews that it needs and take case of displaying them. The best known examples of this approach are probably the preview features built into Microsoft’s Bing and Google’s search engine. This approach is also not very interesting as a WIP because its all about making one site more useful, not integrating with others.

Here’s an example of a Resource Preview in Microsoft Bing:

Approach #3: Applications offer previews of their own resources

This approach is to enable web applications provide previews of their own resources and make them available via a simple protocol. The best example of this approach is OSLC UI Preview, which is part of the OSLC specifications. The OSLC specs are designed to enable integration between ALM tools, but can be applied to integrating web applications of almost any stripe. Applications that implement OSLC UI Preview provide a UI Preview for each resource, one that provides a summary of the resource and links to large and small previews of the resource.

Here’s an example of an OSLC UI Preview from FusionForge, an open source ALM suite:

Here’s an example of a Resource Preview from IBM Rational Team Concert, which also implements the OSLC UI Preview spec:

There are several advantages to this approach. Preview-providing applications have complete control over the content and appearance of previews for their resources. Preview-consuming applications don’t have to create and format them, they just display what comes back from the provider. No special preview site is needed to enable previews. Another advantage to this approach to Resource Previews is the OSLC UI Preview specification, which defines an open and standard way to provide and consume previews.

Wrapping up

I've explained what I call the Resource Pattern and three different ways that I've seen it used. Resource Preview is a good way to add value to the Links pattern and to make links even more useful. The best way to implement this pattern is to ask all web applications that want to integrate to offer Resource Previews via a standard protocol such as OSLC UI Preview, and to display them whenever a user hovers over a link.

OSLC Core v2 specification now FINAL

Dave Johnson — Wed, 1 Jun 2011 14:38:13 +0000

I've been working on the OSLC Core specification for about 1.5 years now as workgroup lead, and OSLC fits squarely under the "open web technologies" and Web Integration Patterns topics of this blog, so I'm blogging this happy news.

Here's the announcement From the OSLC Core Workgroup mailing list:

From: Dave Johnson
To: oslc-core (a) open-services.net, community (a) open-services.net
Subject: OSLC Core v2 specification now FINAL

Today [1], I'm very happy to announce that the OSLC Core v2
specification is FINAL.

The OSLC Core v2 specification [2] defines a set of REST and Linked
Data-based patterns, resources and protocols for integration of application 
and product lifecycle resources (ALM and PLM). It's designed to be the
foundation for all other OSLC domain specifications and there are now
three final OSLC specifications that are based on the Core, those
being the OSLC Change Management (CM) [3], OSLC Quality Management
(QM) [4] and OSLC Requirements Management (RM) [5] specs.

I'd like to thank all of the members of the OSLC Core Workgroup and
community for their hard work, critical thinking and ability to work
together in such a productive and pleasant way. Also, special thanks
to those OSLC domain workgroups who rebased their work on the Core and
development teams that provided excellent feedback along the way.

Thanks,
- Dave

--
David M. Johnson
OSLC Core Workgroup Lead
IBM Rational Software

[1] Move to final was proposed last week, along with a small set of
changes which have since been applied to the specification. 
[2] OslcCoreSpecification
[3] CmSpecificationV2
[4] QmSpecificationV2
[5] RmSpecificationV2

I really do have another Web Integration Patterns post on the way shortly, so stay tuned.

WIP: Embedded Properties in HTML

Dave Johnson — Thu, 7 Apr 2011 21:57:52 +0000

This is the third in my series of Web Integration Patterns. Check out the intro at this URL http://rollerweblogger.org/roller/entry/web_integration_patterns

Synopsis

Enable easier integration and better search across integrated web applications and sites by using standard mechanisms (e.g. Microformats, RDFa) to embed property values in HTML pages.

Motivations

Enable integrated web applications to read, parse and make use of each others data
Enable better search across integrated web applications

Related Patterns

Links
Web APIs
Linked Data + REST

In this pattern, you define different types of resources or “objects," and the properties may be associated with each, or you adopt an existing vocabulary. You define a way to embed type information and properties values into HTML, by embedding name and value pairs in a way invisible to a user or by marking up strings of visible text as property values, or you adopt an existing technique.

For example, a cooking web site could share recipe data using this pattern by presenting each recipe with embedded property values to make it easy for other web applications, mash-ups and search engines to parse out the different parts of the recipe, title, ingredients list, list of steps, etc. This is a real example, by the way, Google indexes recipe data in HTML pages marked up in either Microformat-style or RDFa.

If you present each web page as a resource with a type, or multiple types, and embedded property values then you don’t need to define and provide a special “REST API” to allow other web applications to access your data.

This pattern can enable more powerful search capabilities across integrated web application because search engine’s can be aware of and provide special indexing for those different types of resources and properties.

There are at least three different standard ways to embed data in HTML: Microformats, HTML Microdata and RDFa, each of which I’ll explain below.

Microformats

According to Wikipedia “Microformats emerged as part of a grassroots movement to make recognizable data items (such as events, contact details or geographical locations) capable of automated processing by software, as well as directly readable by end-users.” A community grew around Microformats and this community has worked to define Microformats for a variety of use cases. The microformats.org web site lists nine microformats, including hCard for sharing contact information, hCalendar for sharing events and hReview for sharing movie and book reviews.

The basic idea of Microformats is to use existing HTML element attributes, specifically rel and class, to convey types of things and properties of those things. Here’s an example from Wikipedia: to represent information about a geographic location in HTML, the Geo Microformat uses the syntax below:


   The birds roosted at
     <span class="geo">
     <span class="latitude">52.48</span>,
     <span class="longitude">-1.89</span>
   </span>

The outer-most span’s class element indicates the type "geo" and the inner spans carry the "latitude" and "longitude" property values.

If you want to take advantage of Microformats in web application integration, then you pick one of the existing formats or you invent a new one, ideally by working with the Microformats community to do so.

Another approach for implementing Embedded Properties in HTML is HTML itself.

HTML(5) Microdata

HTML Microdata is a W3C working draft, a specification that defines ways to embed property values in HTML. As Mark Pilgrim explains in Dive Into HTML5, HTML Microdata “annotates the DOM with scoped name/value pairs from custom vocabularies.” This is very similar to what Microformats do, but instead of “overloading” the class attribute, HTML Microdata adds some new HTML attributes to enable embedded property values.

HTML Microdata enables you to mark-up an HTML element as an item that contains properties. You do this via the itemscope attribute, you indicate the item’s type via the itemtype attribute and each property is indicated by the itemprop attribute. Here’s an example, also from Pilgrim’s book:


   <TABLE itemscope itemtype="http://data-vocabulary.org/Person">
     <TR><TD>Name<TD>Mark Pilgrim
     <TR><TD>Link<TD>
       <span itemprop="url">
       <A href=# onclick=goExternalLink()>http://diveintomark.org/</A>
       </span>
   </TABLE>

The table element defines the scope of the item and the type of the item is a URL http://data-vocabulary.org/Person which indicates that the item represents a person. The enclosed span element carries the “url” property of the person.

It’s good to have a standard way to encode types, scope and property values in HTML, but you also need standard vocabularies of types and properties to make this approach successful. There are some HTML Microdata vocabularies defined at data-vocabulary.org and HTML Microdata can also be used with RDF, opening up a huge number of vocabularies.

RDFa

RDFa is a W3C recommendation, a specification that defines ways to embed RDF data in HTML and XHTML. Like HTML Microdata, RDFa adds new attributes to HTML to enable this. Let’s look at an example from the Wikipedia page on RDFa, which uses two of these attributes about and property:

 
   <div xmlns:dc="http://purl.org/dc/elements/1.1/"
     about="http://www.example.com/books/wikinomics">
     <span property="dc:title">Wikinomics</span>
     <span property="dc:creator">Don Tapscott</span>
     <span property="dc:date">2006-10-01</span>
   </div>

The above example embeds some property values about the book Wikinomics, which is identified by URL http://www.example.com/books/wikinomics. The example provides a title, creator and date values about that resource, and it doesn’t say what type of resource exists at the URL, but it could.

The advantage of the RDFa approach is that it ties in with the growing RDF / Linked Data movement, and the huge number of vocabularies from almost every field of endeavor. RDFa has some momentum and has been adopted by Facebook in its OpenGraph Protocol and Best Buy for exposing catalog data in standard ways. However, HTML Microdata can also be used with RDF, so RDFa is not the only way to go if you favor Linked Data.

Wrapping up

We've talked about three different ways to implement Embedded Properties in HTML, so which one do you choose? It's not to me whether one of these techniques will dominate. Google is hedging its bet, so to speak, by indexing all three types of data. If you don't already have a favorite, one way to choose is to look at what others in your community, company or "industry vertical" are doing. The network effect applies here, so if one technique is already favored amongst web application or sites in your space, then go with that.

Next up: Resource Preview

WIP: Common Navigation

Dave Johnson — Sun, 27 Mar 2011 10:08:44 +0000

This is the second in my series of Web Integration Patterns. Check out the intro at this URL http://rollerweblogger.org/roller/entry/web_integration_patterns

Synopsis

Make separate web sites and applications appear to be one by using common user interface elements for navigation.

Motivations

Provide easy navigation between integrated web sites & applications.
Make separate web sites appear to be one and parts of the same overall user interface.

Related patterns

Links

This pattern is an extension of the Links pattern. The idea is to use links combined with common user interface (UI) elements, as a way to provide navigation between integrated sites and to make the separate sites appear to be parts of a whole application. This pattern is usually implemented via a banner with links, a tabs or some other type of menu component.

To make this work, somebody has to create the common navigation, give it an attractive design, decide which web sites or applications are included in the navigation. Consequently, this pattern works well for a set of web sites owned by the same organization, or a suite of packaged web applications.

Pros and Cons

The advantages of Common Navigation are that it works, it does make separate web sites seem to be part of one integrated whole and it's relatively easy to implement. One disadvantage of this approach is that the Common Navigation elements can either conflict with, in a visual sense, or distract from the web sites themselves. But if you're selling a suite of web applications, you've got control of whole design and you can solve this problem.

Examples

Here are some examples of Common Navigation that I see almost everyday. The first is Google.

Common Navigation in Google web applications

Google Mail, Calendar and other Google apps all share a Common Navigation bar across the top of the page. Google decides which apps appear in the menu, and the tech blogs whine whenever an item is moved or removed from the line-up. This is a pretty shallow integration: just links to applications and no more.

Now, let's see a deeper example.

Common Navigation in IBM Lotus Connections

IBM Lotus Connections is a suite of social software applications including social networking, blogs, wiki, forums, file sharing and etc. As you can see below, Common Navigation is used to provide links, not just to applications, but into specific parts of applications. For example, the Latest Entries page in the Blogs app, and the Wikis that I own vs. Public Wikis.

Next, another suite example.

Common Navigation in IBM Rational Team Concert

IBM Rational's Jazz-based products use Common Navigation, but with a project-orientation, showing a user's project's within each application. You can see this below in Rational Team Concert. A development project spans different applications and may have requirements managed by one web application, defects tracked by another, test cases managed by a third and so on. Each user sees the right menu for their projects.

Next, an example that's not a suite of web applications.

Common Navigation in StumleUpon

Another example is StumbleUpon. When you use StumbleUpon, you see the banner at the top of every page. You press the Stumble! button and a you see a randomly selected web page from anywhere on the web, or from some specific category of web site, and you still see the StumbleUpon banner. The StumbleUpon banner makes the whole web seem like one big site, deigned for "stumbling" around and sharing the things you find with friends.

Wrapping up

I wasn't sure that Common Navigation really deserved its own pattern, as it's really just a simple and obvious application of the Links pattern, but I think this works. As always, feedback is welcome. What did I get wrong? What did I leave out?

Next up: Web Annotations.

WIP: Links

Dave Johnson — Mon, 21 Mar 2011 06:49:23 +0000

This is the first in my series of Web Integration Patterns. Check out the intro at this URL http://rollerweblogger.org/roller/entry/web_integration_patterns

Synopsis

Use links as a way to integrate web sites & applications via navigation and relationships between resources.

Motivations

Provide navigation between integrated web sites & applications
Create meaningful relationships between web resources

Related patterns

Common Navigation
Embedded Properties in HTML
Resource Preview
Delegated Selection & Creation
Linked Data & REST

The web is the web and hypertext is hyper because of links. Links allow for the most deep and also the most shallow forms of integration. Links enable navigation and with some discipline, links can also express relationships between resources. There are a variety of ways to express links; the standard and most popular way is to use HTML, but there are others and I’ll cover them below.

Links enable navigation

With links, you can provide navigation between web applications at the application, or the feature level. One example is providing a link from the blog application of a social software suite to the wiki, forums and other component applications. Another is providing a link from one application to one specific part or feature of another application. This type of linking is easy to implement but fairly shallow. It’s convenience and not much more.

Links can express relationships

What’s more useful and powerful, is using links to create meaningful relationships between the resources provided by web applications. For example, there's obvious value linking a defect in a bug tracking system to the source code changes that fixed the bug in a source code management system. Or, you might link a Requirement resource to the Work-Item resources that implement it. By the way, those example are drawn from the OSLC specs for ALM / PLM integration.

If you want to use links to establish relationships, then you do have to have some discipline about the URLs you offer and the URLs you link to. You have to treat links with the same rigor and respect you’d give primary and foreign keys in a relational database. For example, if you make a resource available in different formats, and each at a different URL, one that ends in “.html” and one with “.xml” and one with “.json” then you’ve got three URLs that refer same resource. In this case, it’s better to use Content Negotiation so you can offer one URL that others can use to link to the resource.

One way to get yourself some link discipline is to adopt Linked Data and the RDF data model, which ”extends the linking structure of the Web to use URIs to name the relationship between things as well as the two ends of the link.” See also the Web Annotations and Linked Data & REST pattern.

Links can be expressed in standard formats

The standard way to represent links on the web is through HTML and in HTML, links are expressed in a variety of ways, via anchor tags, via image tags, the link tag and others. There are other standard ways to express links, but none as popular as HTML.

HTML

In HTML, links can be expressed via the <a> element, for expressing a hyperlink in text, and the <link> element, for expressing a link to a resource associated with the page.

<a> and <link> links can specify not only the URL of the link but also the relationship type of the link, as well as content-type and language for the resource that lies at the other end of the link. For example, a link type of rel=”next” indicates that a link points to the next page in a series of pages, or multi-page article. A link type of rel=”stylesheet” indicates that the link is to style-sheet to be applied to the current page. There are a standard set of link relationship types in the HTML spec. Here's an example from this very blog page, the link to this blogs Atom feed:



<link rel="alternate" type="application/atom+xml" title="Recent Entries (Atom)"     
   href="http://rollerweblogger.org/roller/feed/entries/atom" />

Atom

The Atom Syndication Format defines a <link> element, modeled on the one in HTML. Atom adds a length attribute, the size in bytes of the resource at the other end of the link. Atom defines some new links types, alternate, related, self and enclosure. Atom also allows new links to be added and a Atom link-type registry is maintained at IANA.org. Here’s an example link from the Atom comments feed for this blog:



<link rel="alternate" type="text/html" href=
"http://rollerweblogger.org/roller/entry/apache_roller_5_0_rc3#comment-12941523" />

RDF

Another standard way to express links is RDF, using any of the RDF serializations: RDF/XML, Turtle, N3 and soon, there will be a W3C standard for RDF/JSON. Yes, there will be a standard way to express links in JSON and that's good news. For now, we're stuck with RDF/XML. In RDF/XML a link is expressed via the rdf:resource attribute, so for example a link from a Requirement to the user that created the requirement might look like this:



<dc:creator rdf:resource="https://localhost:9443/user/676" />

RDF (Wikipedia page) goes way beyond the couple dozen link attributes and relationship types provided by HTML and Atom. RDF provides ways to specify property values about any linked resource, and property values of each link. RDF also provides ways to define vocabularies of properties.

Wrapping up

It's time to wrap-up. For better or worse, I’ve written more than I expected to write and I’m nowhere near done with the topic of links. I hope I’ve covered the important parts. Comments are open so please correct me and make suggestions if you got some. This is the first pattern that I’ve written up, so I’m also interested in comments about the format, level of abstraction, value, etc. as well.

Next up: Common Navigation.

Web Integration Patterns

Dave Johnson — Tue, 15 Mar 2011 09:30:40 +0000

In my previous post, I promised to write a blog series on Web Integration Patterns. This post explains the concept and a bit about how I plan to write about it.

What I’m calling Web Integration Patterns are techniques for integrating software systems, web applications and web sites using the common technologies of the web. These patterns build on HTTP, HTML, JavaScript and sometimes JSON, RDF and XML to provide ways to integrate software systems and include both programmatic approaches and user interface integrations.

Not the same as Enterprise Integration Patterns

These patterns are different from Enterprise Integration Patterns (from the book of the same title). Those enterprise patterns make the most sense inside an organization where shared-databases and specifically messaging middle-wares are part of the common infrastructure. Web Integration Patterns assume only the web as common infrastructure, or at least the best of them do, and that means they can work equally as well in a controlled enterprise Intranet as they do on the big wild Internet. They can work well for a suite of web applications or a family of loosely related web sites.

Four categories of patterns covered

I plan to write about a dozen blog entries each covering one pattern, and I’ll do at least one per week. I’ll start with the basic patterns then introduce more advanced patterns. I’ve organized the patterns into four groupings, listed below in the order that I will cover them:

(I'll add links in below as I publish each pattern)

Basic Patterns: fundamental patterns which are relatively easy to implement, and, mostly, possible on a website made up of static HTML pages.
- #1 Links: Use links as a way to integrate web sites & applications via navigation and relationships between resources.
- #2 Common Navigation: Make separate web sites and applications appear to be one by using common user interface elements for navigation.
- #3 Embedded Properties in HTML: Enable easier integration and better search across integrated web applications and sites by using standard mechanisms (e.g. Microformats, RDFa) to embed property values in HTML pages.
- #4 Resource Preview: Enhance links shown in HTML pages so that users can hover, mouse-over, or use some other gesture, to view a preview of the resource at the other end of the link.
- #5 Widgets and Gadgets: Allow other web sites and applications to integrate your site into their web pages by providing an embeddable user interface, commonly known as a Gadget or Widget
- #6 Feed-based Integration integrating web sites and applications by using standard feed formats, e.g. RSS and Atom, to convey timely information, updates, status messages, events and other information
Advanced Patterns: more advanced patterns, which may require some server-side logic and infrastructure beyond a plain old web server.
Authentication Patterns: these are advanced patterns that concern authentication and authorization of users and web sites, including Single Sign On, OpenID and more.
Social Patterns: patterns from the world of social networking, including things like OpenSocial, Facebook apps and ActivityStreams.

Perhaps using the word “patterns” is a bit pretentious. I’m no gang of four or even one, and I don’t plan to give these patterns the comprehensive treatment examples that you’d find in a patterns book. I won’t be providing source code examples either. Still, I think Web Integration Patterns is the right name and format, so I’m sticking with it.

The first pattern that I’ll cover is Links. More Later...

More than bloggy

Dave Johnson — Mon, 7 Mar 2011 06:00:00 +0000

I’ve been meaning to write on this topic for a some time and to explain how I’ve gone from being an advocate of RSS/Atom feeds, Atom Publishing Protocol and things bloggy to being a proponent of Linked Data (video), Semantic Web, RDF and other things that I previously considered to be nuisances. I've also got a new topic and blog series to announce, so here goes.

Once upon a time, I actually said “the web is bloggy” in conference presentations and without shame. It sounds silly and, for a blog server developer self-serving, but it’s true and it’s still true to this day. So much of what we do on the web can be made more useful via feeds. That’s because so many things can be usefully represented as a well formed log entry, i.e. a URL, title, date, description and a chunk of content. No matter whether your web application is Flickr, Twitter or IBM/Rational Team Concert, you can enable all sorts of interesting integrations and mashups by providing feeds.

Atom Publishing Protocol (APP) applied the feed concept to web APIs, using a feed to represent a collection of entries and allowing API callers to use HTTP to create, retrieve, update and delete entries. Entries could be anything of course. In the case of Google’s APP-based GData Protocol, entries are Calendar Events, Picasa Photo uploads or YouTube videos. In the case of the IBM/Lotus Connections API, entries are Forum Posts, Social Network Profiles, File Uploads and more.

Feeds only get you so far

I’m not going to argue that RSS is dead (or Atom) because I don't believe that, but the feeds approach only gets you so far. With feeds, we agree on a small set of common properties, e.g. URL, title, date, description and content. Not all resources on the web have those properties and many resources have a lot more. So in some cases applying Atom means forcing things to fit into the Atom model and in some cases it means inventing new properties for Atom, something that is supported by Atom format.

APIs only get you so far

I’m not going to argue that APP is dead or a failure either, but APP is an API approach and, for web integration, APIs only get you so far. It’s definitely a good thing for developers when a web site or application provides an API, but if you have to integrate a set of N web sites and each has it’s own API and (even if they’re all based on APP) you’re stuck doing N x N point-to-point integrations. APIs are not necessarily the best way, and certainly not the only way, to enable integrations on the web.

Linked Data

Linked Data (W3C page) is a different approach. Instead of forcing everything to fit the blog model, Linked Data gives us a way to define common vocabularies of properties about resources on the web and, most importantly, the links between them. Linked Data enables integration by allowing sites to weave their resources into the interlinked web of data that is the web.

Next: Web Integration Patterns

These days, what's important to me at work is integration and in my mind, Feeds, APIs and Linked Data are different “patterns” for web-based integrations. They can be used in complementary ways. Just as we have defined and cataloged Enterprise Integration Patterns, I think it will be useful to do the same for Web Integration Patterns. I know I'll learn something in the process. In my next posts, I'll expand on this concept and after that, I'll start cataloging, in an informal and bloggy way, the patterns that I'm familiar with. More later...