Blogging Roller

A lot has changed since I did my first and even my second "how does Shindig/Java work" diagram. Believe it or not, there are now two separate web services protocols in OpenSocial and thus in Shindig. I'll explain that and then I'll share my updated diagram and synopsis of Shindig/Java internals.

NOTE: this post covers the social API implementation only and not the Gadget server portions of Shindig.

Here's what happened. There was a long discussion on the OpenSocial spec mailing list about the merits of REST vs. RPC for the OpenSocial web services APIs (e.g. here and here). I'm not going to characterize the two sides as the JavaScripty Gadgeteers and the XML-smokin' RESTafarians because that would be just plain silly. Long story short, neither side really "won" and so the latest rev 0.8.1 of OpenSocial includes two separate web services protocols:

• The OpenSocial REST API, which is based on AtomPub but with support for XML and JSON formats in addition to Atom format XML. There is no support for batching of operations. (There was a proposal for RESTful batching, but it did not gain consensus.)
• The OpenSocial JSON-RPC API where methods are called by HTTP POSTing JSON data and there is full support for batching of operations.

Seems a pity to have two separate protocols that serve essentially the same purpose, but thankfully there are a lot of similarities. For example, the JSON-RPC protocol methods are GET, POST, PUT and DELETE. And the JSON format used in the JSON-RPC protocol matches the one in the REST API. That makes things a little easier for implementers and if you use Shindig as the basis for your social network's OpenSocial implementation, it's even easier -- you implement one set of interfaces and you get support for both REST and JSON-RPC. Next, I'll explain how that works with my updated diagram:

And I'll explain by walking through the processing of one request; here's the walk-through (new stuff is highlighted in yellow:

1... A request enters Shindig via either the DataServiceServlet at /social/rest or the JsonRpcServlet at /social/rpc, which has a map of DataRequestHandlers (which it got from the HandlerProvider) keyed by the route that each handles. It's also got a pair of bean converters, which will come into play later when we need to convert outgoing POJOs to XML or JSON representation.

2... The Servlet creates a RequestItem, which parses the request for easy access later. The request item also has a pair of converters which can be used to convert incoming XML or JSON data into Java POJOs.

There are two different request items, one for REST RestfulRequestItem and one for RPC RpcRequestItem. The RESTful request item parses the incoming URL for parameters and treats data from a POST or PUT as typed data, to be automatically converted from JSON, XML or Atom into a Java POJO object from the SPI.

In the RPC case, everything is a POST and the request item grabs its parameters and the typed payload from the JSON object that was posted to the server.

3... The Servlet calls the appropriate handlers. If the request is a single request, then the Servlet looks up a DataRequestHandler based on the request's "route" (i.e. the first segment of URL's pathinfo). The Servlet calls the selected handler, hands it a request item. If the request is a batch request then the Servlet does the same as above, but in a loop collecting request items.

4... One of Shindig's three request handlers then handles the request and returns a ResponseItem or (plural) items containing the Plain Old Java Objects (POJOs) to be returned to the called. This is the point where incoming XML or JSON data is de-serialized to POJO form by a call to requestItem.getTypedParameter(). Shinding provides three DataRequestHandlers:

• PersonHandler: returns Person objects with detailed profile information
• ActivtyHandler: returns Activities of people or group, allows activity creation
• AppDataHandler: returns Gadget data, allows Gadets to store data

NOTE: If want to add your own REST/RPC APIs to extend Shindig, as we are doing in SocialSite, you can do it by hooking in your own handlers. Use Guice dependency injection to hook in your own custom HandlerProvider, one that returns your own custom handlers.

5... The Shindig handlers call the Shindig Service Provider (SPI), which is defined by a set of interfaces with methods that return ResponseItems. There is a set of model objects, aka POJOS, shown in green that represent the different data types that can be returned by the REST or RPC APIs. The service implementations return these POJO objects and expect the Shindig infrastructure to automatically map POJO to either JSON or XML as appropriate.

• PersonService: returns Person objects with detailed profile information
• ActivtyService: returns Activities of people or group, allows activity creation
• AppDataService: returns Gadget data, allows Gadgets to store data

NOTE: the SPI is where you hook in to Shindig if you want to expose your application's Social Graph data via the standard OpenSocial REST or RPC API. You implement these interfaces with code that calls your back-end to create, retrieve update and delete data. And you hook your implementation via Guice.

6... The handler returns a ResponseItem, which wraps one or a collection of POJOs. The Servlet may then write that response out or, in the batch case, continue to call handlers and collect more ResponseItems.

7...Servlet converts/serializes and returns ResponseItems. In the REST API case, once the handler, or handlers in the batch case, returns the response item(s), the DataServiceServlet calls the appropriate converter to serialize the item(s) to either JSON or XML format depending on what was specified in the 'format' request parameter.

And that's that...