Blogging Roller

We did the traditional tree routine last night. This involves lugging way too many boxes of ornaments, Christmas table-ware, various stuffed snowmen, and seasonal nick-nacks out of the attic; listening to the Peanuts Christmas CD; drinking spiked nog; watching the kids decorate the tree and goof around in front of the video camera. Here's the tree, for Russell:

<img src="http://www.rollerweblogger.org/resources/roller/tree-2003.png" alt="our Christmas tree" />

Juha Komulainen I find myself writing a Lisp-interpreter every couple of years.

Curses, foiled again! <a href= "http://www.manageability.org/blog/stuff/blogging-roller-technorati-top-100"> Carlos has uncovered my secret plan to embezzle huge amounts of bogus wuffie.

Matt's been busy. He just released Struts-Menu 2.0 and AppFuse 1.0, two truly useful projects for twisted Struts-monkeys like me. In case you haven't heard, Struts-Menu is a JSP tag library for creating XML-configurable web application menus with support for role-based permissions and for Velocity-based menu rendering. I'm going to be getting started with Struts-Menu myself this week, so this is perfect timing. AppFuse on the other hand, is a skeleton project to help new Struts-monkeys to get started with Struts, XDoclet, Hibernate, JUnit, Cactus, and a host of other open source Java development tools and libraries.

Andy wasn't a big fan of political correctness until he realized that it was yet another way to stir up controversy, draw attention to himself, and bash The Apache Software Foundation.

Roller has changed quite a bit since the Roller article was published on the O'Reilly site. Roller has moved from a two layer architecture with a Presentation Layer and a Business Layer to a three layer architecture with Presentation, Business, and Persistence Layers. This article covers the transition and will bring you up to date with the current Roller persistence architecture.

Then, as now, the Roller Business layer was a small collection of interfaces. The Roller interface was the entry point and provided access to the various Roller manager interfaces. Back then, the Business Layer interfaces were implemented in the org.roller.business package and used the Castor O/R mapping framework APIs directly. The diagram below illustrates this point. The org.roller.business package implements the Business Layer intefaces defined in org.roller.model and depends on org.exolab.castor.

<img src="http://www.rollerweblogger.org/resources/roller/roller-2layer.png" alt="Roller's original 2-layer architecture" />

Earlier this year, during Roller 0.9.7 development, we decided to create an implementation of the Roller Business Layer using the Hibernate O/R framework. We moved the Castor implementation into the org.roller.business.castor package and added the new Hibernate implementation in org.roller.business.hibernate. In the process we noticed a lot of code duplication and we tried to move common code back up into org.roller.business, but we had limited success. The diagram below illustrates this situation: separate Castor and Hibernate implementations of the Roller Business Layer with some shared code.

<img src="http://www.rollerweblogger.org/resources/roller/roller-2impls.png" alt="Two implementations of the Business Layer interfaces" />

Since Roller 0.9.7, the Roller Business Layer implementation has been evolving and moving towards that three layer architecture mentioned at the start of this article. We introduced a PersistenceStrategy interface and a persistence package in org.roller.persistence. Over time we have slowly moved almost all Castor and Hibernate specific code behind the PersistenceStrategy interface. I say almost because, there are a couple of cases where the Persistence Layer abstraction is not powerful enough to handle the queries needed by the Business Layer. Specifically, the RefererManager.getHits() and RefererManager.getDaysPopularWebsites() methods need to drop down to raw SQL because, apparently, neither Castor nor Hibernate can handle the queries we need to do.

The diagram below illustrates the current situation. The Business Layer calls upon the Persistence Layer for object storage, retrieval, and queries. The org.roller.business.castor and org.roller.business.hibernate packages are shown in grey because they still exist but are on the way out.

<img src="http://www.rollerweblogger.org/resources/roller/roller-3layer.png" alt="Two implementations of the Business Layer interfaces" />

Thus far, we've been looking at things at the package level. Let's zoom in on the org.roller.persistence package and take a close look at the key interfaces in that package: <a href= "http://www.rollerweblogger.org/javadoc/org/roller/persistence/PersistenceStrategy.html"> PersistenceStrategy, <a href= "http://www.rollerweblogger.org/javadoc/org/roller/persistence/QueryFactory.html"> QueryFactory, and <a href= "http://www.rollerweblogger.org/javadoc/org/roller/persistence/Query.html"> Query.

<img src="http://www.rollerweblogger.org/resources/roller/persistence-api.png" alt="Roller persistence and query API interfaces" />

PersistenceStrategy supports storage, retrieval, and removal of objects. To execute a query, you create a Query object using the QueryFactory, create a where clause by adding Conditions together, set a limit if desired, set an order-by if desired, and call the Query.execute() method. At that point, the Query implementation will generate an HQL query for Hibernate or an OQL query string for Castor and will call the appropriate method to execute the query. Below is some example code to illustrate typical usage of the Query API.

    public List getTodaysReferers(WebsiteData website) throws RollerException {
        if (website==null ) throw new RollerException("Website is null");
        QueryFactory factory = mStrategy.getQueryFactory();
        Query query = factory.createQuery(RefererData.class.getName());                
        Condition specifiedUser = factory.createCondition(
            "website", Query.EQ, website);            
        Condition hasDayHits = factory.createCondition(
            "dayHits", Query.GT, new Integer(0));            
        Condition userEnabled = factory.createCondition(
            "website.user.userEnabled", Query.EQ, new Boolean(true));            
        List conditions = new LinkedList();
        conditions.add(specifiedUser);
        conditions.add(hasDayHits);
        conditions.add(userEnabled);                
        query.setWhere(factory.createCondition(Query.AND, conditions));            
        query.setOrderBy(Query.DESC, "dayHits");       
        return query.execute();
    }

The PersistenceStrategy interface is a fairly simple abstraction, sort of a least common denominator approach, but it is expressive enough to handle all of the requirements of Roller. This approach allows the Roller Business Layer to be completely independent of the underlying persistence mechanism. This is really nothing new. PersistenceStrategy is essentially a generic Data Access Object (DAO). I use the word "generic" because most DAOs I've seen are specific to one type of object, a CustomerDAO or EmployeeDAO for example, but PersistenceStrategy handles any persistent object in the system. I'm interested to see how this approach holds up. Will we eventually find the simple API to be too limiting? Will it be possible to implement PersistentStrategy with other persistence technologies such as JDO?

This Thanksgiving I'm thankful for you, so keep on doing those things that you do. Those other guys? Well, they need to clean up their act, but I'm thankful for them too. It takes all kinds, you know? Happy Thanksgiving!

As part of Roller 0.9.9's new boomkark management features, I have also added the ability to export the bookmark folder hierarchy as OPML. I implemented this by changing Roller's RSSServlet into a FlavorServlet that serves "flavors" defined by Velocity templates found in the classpath at under /flavors and by adding a new flavor called opml.vm.

For example, the following URL will retrieve all of my folders in OPML format:

• /flavor/roller?flavor=opml

The following URL will retrieve only the contents of the my "/Subscriptions/Blogs - blogroll" folder:

• <a href= "http://rollerweblogger.org/flavor/roller?flavor=opml&path=/Subscriptions/Blogs%20-%20blogroll"> /flavor/roller?flavor=opml&path=/Subscriptions/Blogs%20-%20blogroll

And finally, the following URL will return my RSS2 feed:

• /flavor/roller?flavor=rss

Note that the FlavorServlet defaults to RSS and is mapped to /rss as well as /flavor so all existing Roller RSS URLs will continue to work.

• /rss/roller <-- still works

There is also a new macro #showOpmlLink that renders an OPML icon which links to the bookmark folder of your choice. For example, near the bottom of my Bookmarks page there are OPML links, one for my bookmarks and one for my subsriptions. These OPML links are displayed by the following template code:

<p>You can download my entire bookmark collection in OPML format 
using the links below:</p>
#showOpmlLink("/Bookmarks")Bookmarks (shown above)<br />
#showOpmlLink("/Subscriptions")RSS Subscriptions (not shown)

Over the past month or so, I've been re-working Roller's bookmark management features. As part of this work I added a new Query API so that we no longer have to hard-code Castor or Hibernate query language statements into the Roller Business Layer, I modified BookmarkManager to handle folder hierarchies of arbitrary depth, and I modified the bookmark POJOs to use Castor and Hibernate's lazy-loading facilities. I also rewrote the bookmark management JSP pages and used the JSTL <fmt:message> tag to externalize all strings on the page (for I18N).

Below is the new bookmark manager interface. It works fine now, but it still needs a little fit and finish.

The new bookmark management features will be included in Roller 0.9.9. If you want a preview, check out the new Bookmark Management docs I'm working on.

Now that I've got the hierarchy thing down, I'm turning my attention to the WeblogManager where I will add support for multiple categories per weblog entry and for hierarchical categories.

For RSS and OPML generation... OK, seems to be working now. Erik Thauvin donated his XML/HTML encoding methods to Roller. I just added them to Roller 0.9.9-dev which is running this site now I'll apply them to the Roller 0.9.8 branch for JRoller. Thanks Erik! Update: This is the class Erik donated: TextToHTML.java.

Kindergardener Linus Johnson shares his famous Thanksgiving Turkey recipe:

Get a turkey. Get it out of the wrapper. Then get some spicy sauce. Then you get some peas and put them around the turkey on a plate. Take the turkey and put it on a pan and put it in the oven (a big fat one) at 4000 degrees. Cook the turkey for 5-7 minutes then take it out of the oven. Put it back on the plate that has the peas on it. Eat.

Ovidiu Predescu has posted an interesting presentation on Cocoon's continuation-based control flow or "FlowScript". Coocon is a web application framework that is build around an XML pipelining technology. Since Java does not support continuations, FlowScript uses a version of the Rhino JavaScript engine that has been hacked to support continuations. Very interesting stuff.

I've always thought it would be cool to package Roller, Jetty, and Hsql-db together with a nice installer and make it really, really easy to download and try Roller. All you'd have to do to start Roller would be to run the installer and run a startup script. So, today I was very happy to see that somebody, by the name of Tarka, has documented the Roller installation process for Jetty. Thanks Tarka.

Jetty appears to be very flexible. The Jetty configuration files allow you to instantiate, configure, and call arbitrary Java objects. I wonder if there is a way to configure Jetty to start HSQL in-process, or if I need to write my own launcher to start/stop Jetty and HSQL? Later... hey, maybe that is what JettyPlus does...

This site was down for a couple of hours today. This is the first time that the site was down for a significant period and it was not my fault. The Kattare guys had to install a security patch on Apache HTTP and when they brought the system back up, everything looked OK. Tomcat was up. Apache HTTP was up. Unfortunately, Apache's mod_jk Tomcat connector was crashing and because it is not a monitored service, beepers were silent. After a quick call to Kattare (zero hold-time), service was restored. As I have said before: Kattare rocks!

The path to Roller 1.0 is coming into view.

Charles Miller: Despite what the name suggests, a stand-up meeting can be held sitting down.

Lenny and the Squigtones: Come on everybody, sit down and dance!

I love the idea of stand-up meetings, because I like short meetings. If your team's legs are aching because of stand-up meetings, then your team needs to work some more exercise time into the schedule.

According to the three amigos "architecture is the set of significant decisions about the organization of a software system." Here are some of the more significant decisions that have been made in the development of Roller. Some are questionable, some are bad and have been reversed, and some, as longtime followers of Roller internals will note, are new.

Presentation Layer

• Editor UI will be implemented as Struts actions and JSP pages.
  • JSP pages will use Struts tags, JSTL wherever possible, and custom tags as needed.
  • Header and footer includes will be used (rather than Struts Tiles).
  • JSP custom tags are to be preferered over JSP Scriplets.
• HTML Forms will be implemented with Struts form tags.
  • Struts form tags will be generated via XDoclet from business layer POJOs.
  • Each Struts form bean can copyTo a POJO or copyFrom a POJO
  • copyTo POJO methods will only copy primitives (String, Boolean, boolean, Integer, int, etc.).
• Weblog pages and RSS feeds will be implemented via Velocity templates.
  • Users will use Velocity templates to define pages of weblog.
  • Weblog page macros will be implemented as Veloci-Macros.
• Deployment descriptors
  • struts-config.xml will be generated by XDoclet.
  • web.xml and roller.tld will be generated by XDoclet.
• Interface with Business Layer
  • Presentation Layer will depend on Business Layer via interfaces only.
• Performance
  • Weblog pages and RSS feeds will be cached via Servlet Filter version of OSCache.
  • RSS feeds will obey HTTP IF-MODIFIED-SINCE header.
  • Main page will be cached via OSCache custom JSP taga

• Bad decision, reversed: Struts form beans will each wrap a POJO.
• Bad decision, not yet fully reversed: Editor UI will be implemented via large grained custom JSP tags.

Business Layer

• Business Layer will be defined by a set of interfaces.
• Business Layer objects will be implemented as hand-coded POJOs.
• Business Layer will be Presentation Layer and persistence engine independent
  • Business Layer will not depend on presentation APIs (eg. Struts, Servlet API).
  • Business Layer will use a Persistence Strategy interface
  • Persistence strategy will be implemented for both Hibernate and Castor.
  • Hibernate and Castor mapping files will be generated by XDoclet from POJOs.

• Bad decision, reversed: POJOs will be generated by EJBDoclet from abstract EJB bean classes.
• Bad decision, reversed: Each Business Layer method will open, commit or rollback, and close it's own persistence session.

Persistence Layer

• Open Session in View pattern will be used to ensure maximum of one persistence session per request.
  • At the start of each request, persistence session will be opened via Servlet Filter.
  • At the end of each request, persistence session will be rolled-back and closed via Servlet Filter.
  • Persistence Layer actions must exlicitly commit changes.
  • Persistence sessions will be managed via Thread Local Storage (similar to Thread Local Session).
• Load object by ID will return null if object not found.

via Christopher Kemper: part one of an excellent article by Ed Burnette about building applications on the Eclipse Rich Client Platform.

I think this is a very interesting topic and I've been exploring it further. Here is my current reading list.

Papers and talks on continuations and web programming:

• Modeling Web Interactions (Graunke et al., Apr 2003)
• Lisp in Web-based Applications (Graham, Apr 2001)
• Web Programming with Continuations (PDF) (Byrd, Nov 2002)

A couple of weblog posting on the topic:

• Ovidiu Predescu explains Cocoon's continuation control flow layer
• Blog discussion between <a href= "http://www.cincomsmalltalk.com/userblogs/avi/blogView?showComments=true&entry=3243274443"> Avi Bryant, Daniel von Fange and others on web continuations.

Java web application frameworks that support continuations:

• RIFE: a web application framework with continuation support
• Cocoon's Flowscript supports continuations by using a specially modified version of Rhino

Often in web applications, a user will be filling in an HTML form on one page and will realize the need to go to a second or even a third page to enter information needed by the first page.

For example, you are writing a weblog entry by typing text into an HTML form on the Edit Weblog Entry page. You type in about half of your entry and then you realize that you need to create a new category for your post. You click the Add Weblog Category button, arrive at the Add Weblog Category page, create your new Category, and click OK. At this point, you are returned to the Edit Weblog Entry page to find that your half-edited weblog entry is still there. You are ready to continue right where you left off.

How do you achieve this type of behavior in your web application? How do you allow users to continue working where they left off? Do you treat each case as a special case or do you have a general solution like a breadcrumb stack or a web continuation to this problem? It can get pretty messy. Do you cache half-completed forms in your HTTP session or in your database? Or, do you avoid the whole issue by carefully designing around it.