Blogging Roller

I've been making good progress on the Hibernate impementation of the Roller backend and I may even finish up this weekend. I said that I would blog about it, but I decided not to. It is just not that exciting. The Hibernate API is similar enough to the Castor-JDO API that I have been able to use the same basic patterns in my implementation. Once I finish, I'll do some benchmarking and see how Castor and Hibernate compare. I'm still shooting for Roller 0.9.7 in April, probably during my furlough week (April 14-18).

is available for <a href= "http://download2.eclipse.org/downloads/drops/R-2.1-200303272130/index.php">download.

Anthony reports that FreeRoller is getting a new home, thanks to the JavaLobby.

Anthony Eden: FreeRoller is going to have a new home. The JavaLobby has offered to host FreeRoller on one of their servers which is significantly more powerful than "el crappo", the dinky box which currently hosts FreeRoller [more...]

Brian Blakeley of labourunions.com is working to bring the community building power of weblogging to bear in the day-to-day activities of trade unions. Today, Brian launched cheblogs.com, running on Roller 0.9.7-dev fresh out of CVS.

Matthew Porter: Right now, I really want to blog about the way we manage Hibernate sessions in JLCP. My problem with the currently suggested Hibernate patterns was that they broke fine-grained DAOs, notabley in unit tests and when DAOs call other DAOs. I also did not like the non-OO nature of all the static methods. Until then, check out the HibernateSessionManager and HibernateSession in the code.

I really want you to blog about that too.

Chris Grindstaff will be giving an in-depth tutorial at the RTP Websphere Users Group tomorrow night, March 25, 2003. Details are on the RTP-WUG meeting page.

Sang Shin's slides (with speaker notes) from the TechEngage Dot-Net vs. J2EE shootout are online on the RTP-WUG download page.

I just uploaded Roller 0.9.6.5 to SourceForge. This is a minor "patch" release that includes a web.xml bug fix and some performance enhancements including: simplified main page, better last-update query, RSS caching, etc. This is not the long awaited <a href= "http://opensource.atlassian.com/projects/roller/IssueNavigator.jspa?reset=true&pid=10000&fixfor=10010">Roller 0.9.7 release with support for comments. If you want that you'll have to get it from CVS or wait until the third week of April.

In the evenings, I've been making progress on the Hibernate impementation of the Roller business tier (a.k.a backend) and I completed a Hibernate version of NewsfeedManager, but I keep getting bogged down in refactoring. I can't resist making some improvements to the Roller object model and when I make improvements in the objects, I have to make the corresponding changes in Castor. Then I get diverted into refeactoring the Castor implementation.

This is the drawback of having JUnit tests in place: it makes refactoring so easy that you do it just because you can. Sometimes, I'm not sure if I'm making useful changes or just macdinking. The only think that is holding me back is the lack of unit tests in the Roller presentation tier.

I need to put my nose to the grindstone and wrap this stuff up. I want to get back to the cool stuff, like fixing Roller's comments features and stealing <a href= "http://www.blojsom.com/blog/blojsom/?permalink=A480AB5244E2F1A1656518EA87A4801C.txt">Trackback and <a href= "http://www.blojsom.com/blog/blojsom/?permalink=blojsom-metaweblog.html">MetaWeblog API support from Blojsom.

Simon Brown: As authors, many of us are now in the situation where our work may never be published, or at least published in the form that we expected. Of course this isn't good, but don't forget about those staff at Wrox that have lost their jobs - they are in a much worse position than ourselves. I wish them all the luck for future.

alang: What a load of ill-conceived, patronising, self-serving and naive poppy-cock!

There have been <a href= "http://www.raibledesigns.com/page/rd/20030315#re_wrox_going_under">numerous weblog posts and news articles about the demise of WROX Press. This is sad news, but I'm not giving up on Professional JSP 2.0 yet. There is still some small chance that another publisher will pick up the book or that the authors will band together to publish the book. Hopefully, I'll know the fate of my chapters (Data Access with JDBC and Performance and Debugging) in the next couple of weeks.

No matter what and regardless of the whip count in the security council, blogger Stuart Halloway will be talking Java Classloaders tonight at the Triangle Java Users Group Meeting. Here is the scoop:

Java's class loader architecture provides a dynamic and extensible mechanism for building applications. You will learn how to use class loaders to deploy multiple versions of classes side-by-side in the same VM, and how to redeploy components withtout shutting down servers. You will also learn how to troubleshoot class loading problems such as inversion. You will learn to use the context class loader to correctly implement factory methods, and how to load non-code resources. [more...]

Here are my notes from the TechEngage J2EE vs. Dot-Net shootout. I tried to be objective while I took these notes Wednesday and while I typed them in today. I may have made some mistakes and I may have let some of my open-source/Java bias show through. You be the judge and leave a comment if you see something that does not look right.

Opening Statements

MS [Dot-Net]: Dot-Net is a vision of XML web services enabled by the Dot-Net Framework that happens on the Windows platform. The evolution of computing goes like this: Mainframe OLTP, Client-server OLTP, N-Tier TP monitors (MTS 1997, first OTM), web applications (J2EE, Cold Fusion, Windows DNA), and finally Web Services on the Dot-Net platform. When people learn about Dot-Net they wonder where is the app server? The app server is the Dot-Net Framework, Windows 2003 Server (with load balancing and clustering), plus developer tools (like Borland Sidewinder and Together Control Center). Windows is the app server.

Sang Shin, Sun [J2EE]: Let's talk about Microsoft's contribution to the evolution of computing. Microsoft's contribution is Fear, Uncertainty, and Doubt. F. U. D. FUD. Let's talk about some of the FUD that Microsoft is spreading. FUD #1 is that J2EE is expensive. Don't listen to them. J2EE is free. The analysts at Gartner say that the hidden cost of Dot-Net is 40%-60% (sorry, couldn't follow this). FUD #2 is that the Dot-Net server is a product. Dot-Net server is not a product yet. Once it is released, how many bugs will it have? How many security problems?

Microsoft says that portability is not important. This is very important. Microsoft wants to lock you in. Single vendor lock-in. Lock-in is not a horrible problem when you are talking about applications like office suites, but for infrastructure, single vendor lock-in is very dangerous and expensive. Microsoft touts interoperability, but only as a bait to draw you into single vendor lock-in. They also say that J2EE is no good for web services: wrong.

FUD #4 is that Dot-Net performs better. This is only true in Microsoft funded benchmarks that are tuned in favor of Microsoft. And look at this paper about the Dot-Net Petshop, it shows that the Petshop is pure spaghetti code. Look at this method call from the Dot-Net Petshop (everybody laughs), this method has 36 arguments. Spaghetti!

FUD #5 is that Microsoft believes in standards and interoperability, but Dot-Net is not a standard. Only about 5% of Dot-Net has been submitted as a standard. None of the important stuff you need to develop apps is standard: WinForms, ADO, etc. Even when Microsoft supports a standard, they always add that "Microsoft extra" that breaks compatility with other implementations of the standard. Here is a list of examples: Kerberos, etc. etc.

Greg Ackerman, IBM [J2EE]: I like Letterman and his top ten list so here is my top ten list for J2EE:
1. Openness, avoid single user lock-in
2. Web services, J2EE fully supports
3. Superior platform support, TCO, scalability, Linux
   J2EE websphere reference customers (eBay)
4. Products designed to fit your needs
5. J2EE connects to what you already have
6. World class leading development tools
7. Best dev support programs
8. Partner support
9. IBM does not compete with ISVs
10. J2EE is a complete platform

Mark Fleury, JBoss [J2EE]: We are free and we don't suck. JBoss supports all of the J2EE standards and helps Sun to write those standards. JBoss has a services oriented architecture, a microkernel, and a sophisticated net-boot capability. JBoss brings you unified classloading, no more ClassNotFoundExceptions. If JBoss can't find your class, then your class really can't be found.

I'm very impressed with Dot-Net. The method and class attributes support Aspect Oriented Programming. JBoss does some of the same things, adding capabilities to your classes by using attributes, dynamic proxies, and interceptors. All of this stuff can work outside of the JBoss platform too.

JBoss is the defacto standard. 150,000 downloads per month. App server market share is 48% JBoss, 28% BEA, and then the others. JBoss offers extreme stability. JBoss group is 30 people and growing fast. Some of our references customers: EA Games SIMs online is all JBoss, Playboy (I'm very proud of this), BASF, MITRE, McDonalds, etc. etc.

Richard Weeks, NetEdge [Dot-Net]: Customers use different programming languages and Dot-Net supports 40-50 different languages. Dot-Net is all about multi-language support. Those Java guys want you to rewrite all of your code, don't do it. Don't rewrite your code, get interoperability with web services. Dot-Net does not leave anything behind, you can still use your old code.

Lee Faus, Borland [Dot-Net]: Open source is good, but sometimes there is too much choice. Microsoft has learned from Java and open source and has taken the best aspects of them and has built them into Dot-Net. Borland knows how good Dot-Net is because Dot-Net made it possible for us to build products very quickly. Our Dot-Net products come out of our Rapid Application Development (RAD) group because they are so rapid. Dot-Net makes things so easy, Microsoft is not the dark side, they are our friends.

Q1: What sets your platform apart?

Sang Shin, Sun [J2EE]: Our vision has always been "the network is the computer". We have been active in open source software with Open Office, Netbeans, and Apache. J2ME is everywhere. Java is everywhere from cars, computers, and phones to rings, smart cards, etc. Java makes true end-to-end computing possible. Our innovation continues with the N1 project which promises complete virtualization of resources, ORION to solve maintenance and upgrade problems, and Madhatter to bring Linux and open source to the desktop.

Greg Ackerman, IBM [J2EE]: The things that set J2EE apart are openness, standards, and choice. The things that set Websphere apart are scalability/TCO, web services, comprehensiveness, dev support, business partner support, support for open source, and most importantly community.

Mark Fleury, JBoss [J2EE]: Everybody brags about open source. At least Microsoft is honest about it. Microsoft does not like open source and they say so. Sun is very hypocritical about open source. Both Sun and Microsoft have accused JBoss of taking away the license revenue money that drives R&D. Guilty as charged! The truth is this: the big guys can't compete at the container level and that is why you hear about portal this schmortal that. SOURCE CODE is what sets us apart. The advantages of J2EE are that it is mature, free, and ubiquitous. Dot-Net on the other hand is expensive and buggy. Remember, JBoss wants to commoditize the app server and Microsoft wants to commoditize the developer.

Richard Weeks, NetEdge [Dot-Net]: Dot-Net supports both managed and unmanaged code. Dot-Net is flexible. Sometimes web service implementations don't really follow the standard and Dot-Net helps you to get around this by allowing you to tweak how it's implemented. Plus, you can call COM objects. I know BEA has some Java2COM tool, but Dot-Net's COM support is better. The SDK and the class libraries are free and there are lots of free tools, SharpDevelop for example. You don't need a big expensive app server because Dot-Net gives you choices. You can use only the small parts that you need. Use your old code, don't rewrite it.

Microsoft [Dot-Net]: Windows is the app server and Windows is not expensive. Windows Advanced Server is only $5000. Microsoft and IBM are driving the web services standards, not Sun. Windows is a standards-based integration platform and platform integration gives Dot-Net much better performance than Java. The Dot-Net developer experience is fantastic. We've got multi-language support, the best web application development environment ever in ASP.NET, ADO.NET, and a strong versioning story.

REBUTTAL: Sang Shin, Sun [J2EE]: Sun was not involved in the Web services standards because Bill Gates was playing politics. Bill made sure that Sun did not get invited. Microsoft wants royalty based licensing included in standards. Microsoft wants to charge for every packet of information and monopolize the internet.

Q2: show and tell how you support web services?

Greg Ackerman, IBM [J2EE]: Websphere supports "on demand" computing which allows you to compose your applications as reusable services. Let's look at a demo of creating a web services using Websphere Studio (shows an AVI animation of Websphere Studio). Let's build a stock quote web service. You can build in Websphere Studio and then use a Dot-Net client to access the web service.

Mark Fleury, JBoss [J2EE]: This is when I go Phbbbbtbphphppph (makes a loud farting sound with his lips). Web services is totally vendor driven and all hype, but JBoss supports it fully. We integrate Axis. I don't have much else to say. Let me ask you some questions. How many people do Java and have a Java app in production? (20 or 30 people raise their hands) How many people do Dot-Net and have a Dot-Net app in production? (20 or 30 people raise hands)  How many people have a app that uses both Dot-Net and Java at the same time? (nobody raises their hands)  See? This interoperability stuff is just vendor noise. You need to avoid serialization and avoid RMI and remember SOAP is the biggest dog of them all. B2B will not work. I worked with SAP once and found that these guys can't even create a common object model across one company. B2B vendors are never going to be able to create object models that cross industries.

Lee Faus, Borland [Dot-Net]: Let's use Control Center to create a web services. You just set the project type to C# and you are off. You can add a class using UML notation and the code is generated automatically. If you change the code, the diagram changes. If you change the diagram, the code changes. You can use a wizard to make any object into a web service.

Kenny Jones, MS [Dot-Net]: Analysts say that Microsoft has the best support for Web services. We support web services through our entire product line from Office, MS SQL, and MSMQ, to BizTalk, Excel, etc. Look at how you do web services in Dot-Net Studio. You just add an attribute to your object and it becomes a web services. Let's look at how you use a web service in Dot-Net studio. You just add a "web reference" and then you have a proxy object that you can use to call a web service.

Sang Shin, Sun [J2EE]: Web services is like teenage sex: everybody talks about it but nobody is doing it. There are three phases of web services adoption: 1)  simple (now), 2) Enterprise Application Integration (beginning) and 3) Business web services (2004). #3 is the most important and Sun will support it through J2EE, UBL, ebXML, and the Liberty Alliance.  It is very easy to create web services by exposing EJBs, here is how you do it in Netbeans (shows a wizard).

REBUTTAL: Greg Ackerman, IBM [J2EE]: the analysts are mixed on who is the leader in web services. Some say Microsoft and some say IBM. IBM does web services for many more customers than Microsoft does.

REBUTTAL: Mark Fleury, JBoss [J2EE]: IBM is much better on standards than MS.

REBUTTAL: Microsoft [Dot-Net]: J2EE support for web services is irrelevant. App server vendors and open source software is pushing web services much harder than Sun.

Q3: how do you support building apps for hundreds of thousands of users?

Mark Fleury, JBoss [J2EE]: To support big applications you need grid computing, but grids are too expansive when you must pay for software licenses. You also need caching and JBoss has great support for caching. Don't use serialization. Use caching. Also, you need to integrate the stack within one virtual machine. Dow Jones uses JBoss to support 10,000 clients.

Lee Faus, Borland [Dot-Net]: How do you build N-Tier apps? You need to use modeling and code generation. You need to use Model Driven Architecture (MDA). Here is a demo of MDA in Together Control Center.

Kenny Jones, MS [Dot-Net]: We support this by the scalability of the Windows platform with load balancing, clustering, and caching built into ASP.Net that allows you to easily cache pages and portions of pages. Also with distributed session state and the ADO disconnected data set. You need technology, but you also need the knowledge and you can get the knowledge from the MSDN program and Microsoft Patterns and Practices. Look at all of these customers who use Microsoft to support giant customer bases Merril Lynch, London stock Exchange, etc.). To summarize, the platform is scalable and the knowledge is available.

Sang Shin, Sun [J2EE]: Java has the scalability. Sun has 64-bit support in the Java VM. When will Microsoft have that? One VM can scale up to 100 processors, X RAM, and X threads. Tremendous scalability of just one Java VM. J2EE is all about scalability and reliability. J2EE vendors compete on scalability and reliability to benefit you. Dot-Net is constrained by Windows, Dot-Net is not proven, Dot-Net is single-vendor lock-in. Why should you be the Dot-Net guinea pig?

Greg Ackerman, IBM [J2EE]: IBM has a great deal of experience in distributed computing (CORBA, Encina, etc.). J2EE is designed for this stuff.  Remember the case studies. Java and VMs are scalable.

REBUTTAL: Microsoft [Dot-Net]: The analysts say that portability across J2EE app servers is going to become more and more difficult. Java app servers do contain vendor lock-in features.  Java allows you to scale to bigger more expensive hardware. With Dot-Net you won't need to do that, you can stick with the hardware you already have.

REBUTTAL: Mark Fleury, JBoss [J2EE]: The EJB spec does not give you what you need to scale, you need the cache. The spec is fighting the implementation here. J2EE is not moving fast enough. We need Aspect Oriented Programming, from Xerox, where all good things emanate. Dot-Net does not have what it takes here either: ADO sucks and Dot-Net caching is not there.

Q4: explain your platform's security system?

Richard Weeks, NetEdge [Dot-Net]: Dot-Net provides code-access security. This allows you to say where what a piece of code is allowed to do. For example, if you have a consultant that you don't trust, you can lock his code down so that it does not threaten you. Encryption is built in. ASP.Net has forms based security. No more buffer overflows because of the Dot-Net runtime. Microsoft is putting a big emphasis on integrated securty.

Kenny Jones, MS [Dot-Net]: Windows security vulnerability is a myth. Security is an industry wide problem, not Microsoft problem. Windows has fewer CERT security advisories than Sun or Redhat. Microsoft has a serious "trustworthy computing" initiative going on. Microsoft won the Open Hack 4.0 contest. Let me show you how code that is downloaded from the internet is treated differently than code that lives on your hard-drive. See: this downloaded code is not allowed to run.

Sang Shin, Sun [J2EE]: Security IS a Microsoft problem. Security must be built-in from the beginning. You cannot just bolt it on as an afterthought. Look at the passport fiasco. Microsoft's Passport identity management system was centralized, single-point-of-failure, controlled by Microsoft, and single point-of-attack. No wonder everybody hated it and it failed. The Liberty Alliance on the other hand is a federated system, much better. There are 52,000 viruses for Windows and the analysts say it is time to switch away from Windows based web servers. Viruses are very expensive. The ILOVEU virus costs us $1 billion dollars. NIMBDA costs $2.6 billion. Recently, Microsoft's Craig Mundie said "we started thinking about this three years ago." Microsoft has been in business for 27 years. It took them 24 years to realized that security is important.

Greg Ackerman, IBM [J2EE]: Good for Microsoft! They finally realize that security is important, but security needs to be built-in from the start.  Look at Dot-Net security: sandboxing, code-access security, not exactly novel concepts. Dot-Net security is just a copy of Java security. IBM is cooperating with Microsoft on Web services security.

Mark Fleury, JBoss [J2EE]: UNIX has had better security than Windows for many years. Security cannot be bolted on. Java security is excellent. JBoss did JAAS security years ago, before all of the other app server vendors. JBoss can also use interceptors to add additional security.

REBUTTAL: Sang Shin, Sun [J2EE]: Microsoft FUD #9 is that Dot-Net is secure. Dot-Net depends on COM+ which is not managed code and is therefore unsecure. C# permits unsafe and unsecure code. Passport has already been hacked.

Closing Statements

MS [Dot-Net]: Let's take a look at what it takes to build a mobile web app, one with an adaptive UI that looks different depending on which device you use to access it. Let me cut-and-paste some code here and let's try to run it. Oh no, the Dot-Net server is not responding (his computer appears to lock-up). Analysts say that C# is going to be the number #1 language in two years.

Mark Fleury, JBoss [J2EE]: C# to be the #1 language in two years? You're freaking kidding me. Dot-Net has some good features, but multi-language support is just cute, no more. Without portability off of the Windows platform, Dot-Net will go nowhere. JBoss is a responsible, moral, and open player. Let JBoss be the standard, not Dot-Net or J2EE.

Sang Shin, Sun [J2EE]: Java is the most powerful development technology ever.  There are 3 million Java developers, 65 million Java enabled phones, 8 million lines of Java source contributed to open source, etc. etc. The development resources are all free: open source software is almost all Java, tutorials, knowledge base, community!

Greg Ackerman, IBM [J2EE]: C# is great, very Java like.  Make the rational choice (no pun intended). VB.Net has a big learning curve, even for veteran VB programmers. C# does too, so why don't you just go directly to Java. We will welcome you into the community. Come on out the the Websphere Users Group and the Java Users Group meetings in the Park.

Richard Weeks, NetEdge [Dot-Net]: We keep hearing about free this and free that. Open source is not free. Support costs money. Multi-language is not just a cute feature. Each language has it's own unique advantages and disadvantages. C# and Java are different. C# is better.

The shootout was a bit too long and a little too fluffy, but I enjoyed it. I took 20 pages of notes and I will probably write them up for you later but for right now, I'll just give you some quotes from the J2EE team:

Mark Fleury: The worst dog of them all is SOAP.

Sang Shin: Security is not an industry problem, it is a Microsoft problem.

Mark Fleury: C# to be the number one language in two years: give me a freaking break!

Mark Fleury: JBoss is a responsible, moral, and open player.

Sang Shin: Web services is like teenage sex, everybody is talking about it but nobody is doing it.

Sang Shin: My grandmother called me last night to tell me that she is doing web services.

Mark Fleury: ADO sucks, Dot-Net caching is not there.

Greg Ackerman: C# is great, very Java like.

eclipse.exe -vmargs -Xverify:none -XX:+UseParallelGC -XX:PermSize=20M  
 -XX:MaxNewSize=32M -XX:NewSize=32M -Xmx96m -Xms96m

Sosume applies some new GC configuration settings, written about in JavaWorld and recommended for Netbeans to improve Eclipse performance in his post: Improving Eclipse performance by using vm arguments. I'm trying these settings myself now, but with -Xmx256m.

I tried the kill -QUIT trick on FreeRoller this afternoon after it locked up. Looking at the stack traces for Tomcat's 75 threads, I found that most were either waiting for database connections or involved reading cache entries from the disk.

Recently, I had changed to OSCache settings so that Roller caches to disk rather than to memory. I did that because Roller was running out of memory. Caching to disk fixed the out-of-memory problem, but at the expense of heavy disk access. So, I changed the OSCache settings back to cache to both disk and to memory. OSCache seems to have some limitations in this area. Apparently, if you configure OSCache to cache in memory and to disk, you can limit the number of entries in the memory cache, but the disk cache is unlimited - which is not the best situation.

Also, I changed the DBCP connection pooling parameters to reduce the maxIdle and maxWait so that threads will timeout quickly rather than hanging around waiting for connections and thus forcing the creation of new threads.

Those changes kept FreeRoller up for a much longer period of time, but did not fix the problem.

Andy Oliver: I followed the instructions, step by step, substituting the postgresql script for the mysql and the driver name, etc. (Take for granted that I approximately know how to do these things since I do Java/database-related apps for a living) [...] Nope...It doesn't look like it worked...

Andy Oliver: competence is required to judge competency and that the incompetent often don't know that they are.

I'd like to point out that we have this thing, we call it a "<a href= "http://sourceforge.net/mail/?group_id=47722">mailing list," where we answer installation questions and otherwise help people who are trying to get started with Roller. Look into it and if you have any suggestions for making the Roller install process better, please share them with us.

FreeRoller keeps on running out of threads to process incoming requests. This could be a problem in Roller, but I suspect that we may have run up against this bug in the Tomcat 4.0.X HTTP connector:

BUG5735: HTTP connector running out of processors under heavy load

The bug is marked as fixed, but I think that means the fix is in the new Tomcat 4.1.X Coyote HTTP connections.

Next time this happens, I'll try Glen Nielson's advice (from the bug report):

I would recommend that you dump the stack for all running threads when
you experience this problem.  This can help identify what is causing the
problem. By reviewing the stack dump for each thread you can determine
whether the problem is due to Tomcat or your application.

On unix you do a kill -QUIT {tomcat java pid) to cause the thread stack's 
to be dumped to catalina .out.

A Processor for Tomcat runs your application code, delays in your code
can cause additional processor threads to be created to handle new requests.
Possible application or configuration problems which can delay requests:

   Connection delays due to networked services such as a db.
   Connection delays due to running out of pooled resources.
   Thread synchronization deadlocks.
   A cascading affect where many new processors get created due to
   excessively long JVM Garbage Collections.  start java with
   -verbose:gc to detect this.

Ted explains how to plug JRocket into an existing JDK 1.4.1 JRE:

Ted Neward: First, I created a subdirectory in my standard JDK 1.4.1 install structure, j2sdk1.4.1/jre/bin/jrockit. Into this directory I copied the jvm.dll and Xusage.txt files that came from JRockit itself. Then, after adding a line "-jrockit KNOWN" into my JDK 1.4.1's jvm.cfg file (so that the launcher would recognize "-jrockit" as a viable command-line option), I fired up the standard Sun JDK java launcher with the command line "java -jrockit Hello". Sure enough, it worked.