BlogQL: Storybook and Mocked Service Workers

One of the most interesting new technologies I learned about during the development of BlogQL is Storybook. Storybook allows you to develop and test React, Angular, and other web components in isolation. 

Once you create stories for your components, and thanks to Chromatic, you can setup GitHub Actions to check each incoming Pull Request for changes to your UI, a visual diff! You can require that a developer approve each change. I found that all pretty compelling and started to develop stories for all of my React components. 

The hardest part of this work was figuring out how to mock the GraphQL data needed for my components. It was difficult for me because I was still learning WebPack and polyfills in Node.js, but I got it working, in part thanks to his very useful blog post: Mock GraphQL and REST in Storybook and Jest with MSW.  

You can view the BlogQL Storybook online, thanks to Chromatic: https://63b0bf0f112b2d8b80a785f5-flfdjbdgsu.chromatic.com.

Blog QL Storybook

This is part of a series of short posts about BlogQL:  https://github.com/snoopdave/blogql.


BlogQL, Apollo Server and Google Login

The BlogQL server uses Apollo Server to provide a GraphQL API. Initially, I was using Apollo Server standalone, but when I stared integrating Google Login I realized I needed a couple of REST APIs so I added Express https://expressjs.com for that. On the frontend, I'm using React Oauth2 which makes adding Google Login support very easy to implement.  This all works fine, but if I were to start over I might consider using a server-side framework like Next.js https://nextjs.org instead of rolling my own Apollo Server and Express setup. 

This is part of a series of short posts about BlogQL:  https://github.com/snoopdave/blogql.


BlogQL and create-react-app

When I first started with BlogQL, I used create-react-app (CRA) to generate the code. It worked well and allowed me to focus on learning React, but eventually I found it too constraining. I wanted to learn the underlying technologies of WebPack and Babel and CRA tucks those away. It took some time, but was able to remove CRA and replace it with my own WebPack and Babel setup.

If I had to start again, I might consider using Vite, which is growing in popularity as an alternative to WebPack.

This is part of a series of short posts about BlogQL:  https://github.com/snoopdave/blogql.

BlogQL

Last summer, I started a side project to enhance my knowledge of Node, TypeScript, React, GitHub Actions and to dog-food Apollo GraphQL Studio's features that I was working on. In the past couple of months, I've continued to work on BlogQL and now that the UI is not a total embarrassment, I'm going to share a little about the project.

BlogQL is a blog app made up of a TypeScript/Node backend that provides a GraphQL API and a TypeScript/React-based frontend with a rich-text editor for editing blog entries. Below is a two-minute demo so you can see the features that BlogQL supports. You can find the complete source for BlogQL on GitHub https://github.com/snoopdave/blogql.

I'll be following this up with a series of short posts about the technologies I'm using in BlogQL. 


Thoughts on ActivityPub, Mastodon, etc.

(disclaimer: I'm new at this stuff and do not have full understanding of the AcitvityPub spec so please correct me if you think I got things wrong). Is the demise of Twitter going to spark a resurgence of the "open web" and is that a good thing?[Read More]

Upgraded to Roller 6.1.1

A note about this site: I just upgraded rollerweblogger.org to run the recently released Roller 6.1.1, and Java 17. There was one snag. This site uses the Roller-JSPWiki plugin and old Lucene dependency in that plugin prevented Tomcat from loading Roller. It took me a couple of hours to figure out how to upgrade the plugin to use the latest version of JSPWiki. That fixed it.


Building an Open Source J2EE Weblogger

I wrote this article for O'Reilly's OnJava.com over twenty years ago and it was published on April 17, 2002. Roller would not become Apache Roller until about five years later. Publishing this article changed my life and set my career on a new trajectory. I can't find it online anymore so to celebrate this anniversay, i'm going to publish it here on Roller. As a Java developer, you should be aware of the tremendous wealth of open source development software that is available for your use -- even if you have no desire to release any of your own software as open source. In this article, I will introduce you to some of the most useful open source Java development tools by showing you how I used these tools to develop a complete database-driven Web application called Roller.[Read More]

Blogging about Roller 6.1

I barely even update this blog, but I do update the software that powers it and I'm happy to still have the help of an awesome team of volunteers who pitch in when they want to and always when needed. Today we released Apache Roller 6.1.0, a release we had been talking about, but that was prompted by the Log4J vulnerability. See the project's blog for details.

I'm not the one making the most code changes in Roller now days, but I do help with releases. I've been spending my spare cycles hacking on BlogQL, a Node/TypeScript-based blog server with a GraphQL API and a React front-end. It's really more of an example app to help me understand those technologies, kind of like Roller was. Maybe I'll write about it someday. That's all for now.


Apache Roller 6.0.2 released!

The Roller PMC has approved the release of Apache Roller 6.0.2, a minor bug fix release.

You can download the release via Apache mirrors link here:

Roller 6.0.2 includes the fixes listed below:

Parse referrer URL instead of using string value directly

Improved: the UI for the blog entries page

Improved: tooltip content on the registration form

Updated: the main menu to have proper space when displaying permission message

Updated: the header to show the tagline only when it's available

UI fixes are thanks to new contributor Yash Maheshwari.

On behalf of the Roller PMC,
Enjoy and best regards!

Roller 6 released

This latest release of Roller includes a new UI that uses Twitter Bootstrap 3 and is based on work I started in 2015 and first committed on December 21, 2015 with this commit: 2da6c3c2e28419f68244e0c362c15be96013d5f9. You can find the details on on the Roller project blog. I got lot of help along the way with testing, fixes, dependency upgrades, Java 11 support and more, so thanks to all that helped make this happen.


Powered by Digital Ocean Kubernetes

Just a note to say that I've switched this site over to Digital Ocean Kubernetes service, which is in Limited Availability right now.

Digital Ocean's Kubernetes service is just as simple and well designed as the rest of Digital Ocean. I mentioned before that I rolled my own Kubernetes cluster via Ansible and Kubeadm. Now I can delete all those config files and that's a good thing. Plus, the price is right; I can get by with one $10/month node (1 CPU / 2 GB memory) and a $10/month load balancer.

To get this site up and running I had to deploy four things to my cluster. I installed the NGINX Ingress Controller, Cert-Manager for automatic creation of Let's Encrypt TLS certs, PostgreSQL and my custom build of Apache Roller. All of that went pretty smoothly and I didn't run into and problems that I could blame on Digital Ocean.


Roller 6.0.0-SNAPSHOT

Upgraded this site to Roller 6.0.0-SNAPSHOT today, which meant an hour of fiddling around with my private Docker registry, then giving up and using the one free private repository offered by DockerHub and then, another hour of futzing around trying to figure out my PostgreSQL JDBC driver doesn't work anymore (I inadvertently upgraded from JDK 1.7 to 1.8) and why I can't seem to upgrade it (Kubernetes caches Docker images unless you set imagePullPolicy to always). In the end, I got it working. This post is written in the yet to be officially release Apache Roller 6.0.0-SNAPSHOT version.

Side note: the new rich-text editor in Roller is now Summernote and it seems quite nice. I need to tweak it a bit because there is currently no way to set the font or add a link unless you switch to raw HTML mode.


Roller's new web UI

About three years ago I decided to modernize and improve the Apache Roller web UI by rewriting the JSP pages to use the Struts 2 Bootstrap tags, which use Twitter's Bootstrap v3 components and JavaScipt. I also wanted to replace all the HTML table-based formatting with div's and Bootstrap, do a bunch of other improvements and make Roller's web UI less clunky and annoying.

Converting Roller's eight-five JSP pages was a big task and I did not have much time for it. That's why it took three years. Ironically, the Roller modernization project leaves Roller three years out of date. Still, I think it is a huge improvement over the Roller v5 web UI and I want to get it released in Roller v6. Currently, this work is available as Pull Request #22 and you can find some screenshots there too. Here's one:

screenshot of Roller Edit Entry page

Try it with Docker-Compose

I also did some work to make it super-easy to try the Roller v6 snapshot pre-release for yourself, by using Docker Compose. You don't have to fiddle with Tomcat or PostgreSQL. You can find a simple Dockerfile for running Roller v2 snapshot and a docker-compose.yml file linked below. And you can find a Docker image in my DockerHub repo.

If you want to try Roller v6 snapshot, here's what you need to do:

1 - If you don't aleady have it, install Docker

2 - Create a directory on your computer where you want Roller to store it's data.

3 - Save this file docker-compose.yml to that new directory.

4 - Open a shell in that new directory and run:

docker-compose up

5 - Watch the PostgreSQL and Roller startup logs scroll by

6 - When the log scroll slows go to http://localhost:8080 to access Roller and go through the initial setup.

Alternatively, if you want to try Roller the hard way, you can get the regular-style v6 SNAPSHOT release files here roller/roller-6.0/v6.0.0.

Let us know how it goes

I hope you'll give Roller v6 snapshot a try and let the project know how it can be improved for your use. Send feedback to the Roller mailing lists or ttweet at us at @apache_roller.



Powered by Kubernetes

kubernetes logo Just a quick note to say that I ditched Docker Swarm and now this rarely updated blog is powered by Kubernetes. Total overkill, I know. Like Roller itself, I did it as a learning exercise. I hope to blog more about what I learned by doing this. For now, here's a quick summary of what I've done so far.

Created a cluster

I created a 2-node Kubernetes cluster on Digital Ocean using some hand-crafted Ansible scripts that call apt-get to install and kubeadm to start Kubernetes. I considered using Typhoon to create the cluster, but I really wanted to learn how to install Kubernetes "from scratch".

Ran two Ingress Controllers

To avoid using Digital Ocean's $20/month load balancer I'm running an Nginx Ingress controller on each node, and pinning containers to nodes using labels and nodeSelectors. I had to borrow Nginx Controller setup files from the Typhoon project because I'm still kind of bewildered by Ingresses.

Deployed my containers

Next, I wrote Kubernetes YAML files for deploying my containers: a private Docker Registry, PostgreSQL and my custom Roller image. Getting the private registry working properly was the biggest challenge. I need private because I don't want to make my custom Roller image public. Next, I'll install Jenkins next for CI/CD of my custom Roller build via the Jenkins Kubernetes plugin.

Let me know if there are any aspects of this that you'd like to see covered in a blog entry, or suggestions for running the cluster without two Ingress Controllers. I've already got a post cooking about installing a TLS secured Docker Registry on Kubernetes.


Powered by Postgresql and Docker Swarm

It was somewhat painful but due to some problems with MySQL and Docker, and some general uneasiness with MySQL, I switched this site from MySQL v5.7 to PostgreSQL v10. I also switched over to Docker Swarm. Here's the Docker-Compose file that I'm using now to run this site:
version: '3.2'

services:

   postgresql:
      image: "postgres:10.0"
      ports:
         - "5432:5432"
      deploy:
         resources:
           limits:
              memory: 50M
      volumes:
         - type: bind
           source: /var/lib/postgresql/data
           target: /var/lib/postgresql/data
      environment:
        - POSTGRES_USER=roller
        - POSTGRES_DB=rollerdb
        - POSTGRES_PASSWORD_FILE=/run/secrets/pg_passwd
      secrets:
        - source: db_passwd
          target: pg_passwd

   roller:
      image: "rwo:latest"
      ports:
        -  "80:8080"
      depends_on:
        - postgresql
      deploy:
         resources:
           limits:
              memory: 800M
      volumes:
        - type: bind
          source: /var/lib/roller
          target: /var/lib/roller
      environment:
        - DB_HOST=postgresql
        - STORAGE_ROOT=/var/lib/roller
        - JAVA_OPTS="-Xmx700m"

secrets:
  db_passwd:
    file: ./db_passwd.txt
It was a pain, but sometimes pain = gain and I learned a lot. I'm hoping the site will be a bit more stable now.

Modernizing the Roller UI

I don't blog very often but I still find time to work on my blog's software: Apache Roller.

Recently, I decided to focus on improving Roller's ancient Struts 2-based user interface (UI). I had considered adding a comprehensive API to Roller and building a new UI based on that API, but wow that is a huge amount of work. Instead, I decided to modernize the Roller UI by using Twitter's Bootstrap components and CSS styles.

So far, I've devoted a couple of weekends to this work and made some pretty good progress. I'm about half-way done. I'm using the Struts2-Bootstrap plugin, adding better client-side form validation with JavaScript and doing my best to improve the overall user experience. You can see an album of the pages I've done so far on Flickr:
Roller UI with Bootstrap
.

I would love any contributions, so if you are interested in helping out, please submit Pull Requests against the bootstrap-ui branch in the Apache Roller repo on GitHub.


Fractal fun with HTML5 and TypeScript

MbCanvas is a fun project that I did in 2015: a simple Mandelbrot Set viewer written in Typescript and using the HTML5 Canvas. I did the project to learn more about Typescript and the HTML5 Canvas and I must say, Typescript very nice -- so much easier to read and write than plain old JavaScript, at least for me.

Here's an example image from the viewer.

mandelbrot set image

The project is fairly easy to build if you've got Node and NPM installed, or you can play around with it here: mbcanvas - Mandelbrot viewer in TypeScript


Usergrid-Vagrant updated for Usergrid 2

GitHub logo

If you're interested in trying the not-yet-released Apache Usergrid 2 you might want to checkout my Usergrid-Vagrant project on GitHub. I just updated the project to support Usergrid 2, using the latest code from the Usergrid "release" brach. The big changes were switching to OpenJDK 8 and adding ElasticSearch. I also rewrote the scripts to use plain old Bash instead of Groovy.

https://github.com/snoopdave/usergrid-vagrant

If you want the old Usergrid 1 Vagrant-file then checkout the "1.x" branch.


Now hosted on DigitalOcean

After thirteen years of hosting this blog at Kattare.com, I've moved it over to DigitalOcean. Kattare was great, but nowadays I prefer managing my own server and DigitalOcean makes that very easy -- and costs less ($10/month vs. $26/month at Kattare).

The move was easy, or as easy as setting up OpenJDK 8, Tomcat 7 and MySQL 5.5 can be. I only hit one little snag. Once I added the Roller WAR to Tomcat, Tomcat would hang on startup. I used jstack to look at the Java VM threads and found some clues that led me to a post on ServerFault.com: Tomcat 7 hangs on deploying apps. As recommended in that post, I added -Djava.security.egd=file:/dev/./urandom to my CATALINA_OPTS and was back in action.


Apache Shiro for authentication in Roller

Shiro logo

This is the third of my 2014 side projects that I'm sharing and one that involves the Apache Roller blog server and the Apache Shiro security framework. You might find this interesting if you're considering using Shiro for authentication and authorization, or if your interested in how security works in Apache Roller.

Inspired by my work with Ember.js in Fall 2014, I started thinking about what it would take to build an Ember.js-based editor/admin interface for Apache Roller. To do that, I'd need to add a comprehensive REST API to Roller, and I'd need a way to implement secrity for the new API. I've enjoyed working with Apache Shiro, so I decided that a good first step would be to figure out how to use Apache Shiro in Roller for Roller's existing web interface.

Working over the winter break I was able to replace Roller's existing Spring security implementation with Shiro and remove all Spring dependencies from my Rollarcus fork of Roller. Below I'll describe what I had to do get Shiro working for Form-base Authentication in Roller.

Creating a Shiro Authorizing Realm

The first step in hooking Shiro into Roller is to implement a Shiro interface called ShiroAuthorizingRealm. This interface enables Shiro to do username and password checks for users when they attempt to login, and to get the user's roles.

Below is the first part of the class, which includes the doGetAuthenticationInfo() method, which returns the AuthenticationInfo for a user specified by an AuthenticationToken that includes the user's username. In other words, this method allows Shiro to look-up a user by providing a username and get back the user's (hashed) password, so that Shiro can validate a user's username and password.

ShiroAuthorizingRealm.java (link)
public class ShiroAuthorizingRealm extends AuthorizingRealm {

    public ShiroAuthorizingRealm(){
        setName("ShiroAuthorizingRealm");
        setCredentialsMatcher(
            new HashedCredentialsMatcher(Sha1Hash.ALGORITHM_NAME));
    }

    @Override
    public AuthenticationInfo doGetAuthenticationInfo(
        AuthenticationToken authToken) throws AuthenticationException {

        UsernamePasswordToken token = (UsernamePasswordToken) authToken;

        User user;
        try {
            user = loadUserByUsername( token.getUsername() );

        } catch (WebloggerException ex) {
            throw new AuthenticationException(
                "Error looking up user " + token.getUsername(), ex);
        }

        if (user != null) {
            return new SimpleAuthenticationInfo( 
                user.getUserName(), user.getPassword(), getName());

        } else {
            throw new AuthenticationException(
                "Username not found: " + token.getUsername());
        }
    }

In the code above you can see how we pull the username out of the authToken provided by Shiro and we call a method, loadUserByUserName(), which uses Roller's Java API to load a Roller user object specified by name.

The next method of interest is doGetAuthorizationInfo(), which allows Shiro to look-up a user's Role. This allows Shiro to detmerine if the user is a Roller admin user or a blog editor.

ShiroAuthorizingRealm.java (continued)

    public AuthorizationInfo doGetAuthorizationInfo(
        PrincipalCollection principals) {

        String userName = (String)
            (principals.fromRealm(getName()).iterator().next());

        User user;
        try {
            user = loadUserByUsername( userName );
        } catch (WebloggerException ex) {
            throw new RuntimeException("Error looking up user " + userName, ex);
        }

        Weblogger roller = WebloggerFactory.getWeblogger();
        UserManager umgr = roller.getUserManager();

        if (user != null) {
            List roles;
            try {
                roles = umgr.getRoles(user);
            } catch (WebloggerException ex) {
                throw new RuntimeException(
                    "Error looking up roles for user " + userName, ex);
            }
            SimpleAuthorizationInfo info = new SimpleAuthorizationInfo();
            for ( String role : roles ) {
                info.addRole( role );
            }
            log.debug("Returning " + roles.size() 
                + " roles for user " + userName + " roles= " + roles);
            return info;

        } else {
            throw new RuntimeException("Username not found: " + userName);
        }
    }

In the code above you can see that we use the loadUserByUsername() too look-up a user by username, then we use Roller's Java API to get the user's roles. We add those roles to an instance of the Shiro class SimpleAuthorizationInfo and return it to Shir.

Creating a Shiro Authorizing Filter

Now that we've implementated a realm, we've provided Shiro with everything needed to authenticate Roller users and get access to Roller user role information. Next, we need to configure Shiro to enforce roles for the URL apths found in Roller. Shiro includes a RolesAuthorizationFilter, which is close to what we need but not exactly right for Roller. I had to extend Shiro's roles filter so that we can allow a user who has any (not all) of the required roles for a resource.

RollerRolesAuthorizationFilter.java (link)
public class RollerRolesAuthorizationFilter 
    extends RolesAuthorizationFilter {

    @Override
    public boolean isAccessAllowed( 
        ServletRequest request, 
        ServletResponse response, 
        Object mappedValue) throws IOException {

        final Subject subject = getSubject(request, response);
        final String[] roles = (String[]) mappedValue;

        if (roles == null || roles.length == 0) {
            return true;
        }

        // user is authorized if they have ANY of the roles
        for (String role : roles) {
            if (subject.hasRole(role)) {
                return true;
            }
        }
        return false;
    }
}

Configuring Shiro for Roller

Now that we've seen the Java code needed to hook Shiro into Roller, lets look at how we configure Shiro to use that code. We do that using the Shiro configuration file: shiro.ini, as shown below.

shiro.ini (link)
[main]

defaultRealm = org.apache.roller.weblogger.auth.ShiroAuthorizingRealm
securityManager.realms = $defaultRealm

cacheManager = org.apache.shiro.cache.ehcache.EhCacheManager
securityManager.cacheManager = $cacheManager

authc = org.apache.shiro.web.filter.authc.FormAuthenticationFilter
authc.loginUrl = /roller-ui/login.rol
authc.successUrl = /roller-ui/menu.rol

rollerroles = org.apache.roller.weblogger.rest.auth.RollerRolesAuthorizationFilter

[urls]

/roller-ui/login.rol          = authc
/roller-ui/login-redirect.rol = authc, rollerroles[admin,editor]
/roller-ui/profile**          = authc, rollerroles[admin,editor]
/roller-ui/createWeblog**     = authc, rollerroles[admin,editor]
/roller-ui/menu**             = authc, rollerroles[admin,editor]
/roller-ui/authoring/**       = authc, rollerroles[admin,editor]
/roller-ui/admin/**           = authc, rollerroles[admin]
/rewrite-status/**            = authc, rollerroles[admin]
/roller-services/rest/**      = authcBasic, rollerroles[admin,editor]

In the configuration file above, you see how we hook in the new ShiroAuthorizingRealm on line 3. The next couple lines are boiler-plate code to hook in Shiro's caching mechanism and then, on line 9, we configure an authentication method called authc, which is configured to use Shiro's Form Authentication feature. And, on line 13, we hook in our new RollerRolesAuthorizationFilter.

Next, we tell Shiro that the login page for Roller is /roller-ui/login.rol and which page to direct a user to on a successful login, /roller-ui/menu.rol, if the user did not specify which page they wanted to access. And finally, on lines 17-25, you see the list of Roller URL patterns that need protection, which authentication method to use (authc or authcBasic) and the authorization filter and roles required for access to the URL pattern.

Wrapping up...

That's all there is to the story of Roller and Shiro so far. I was able to get Roller's form-based authentication working with Shiro, but I did not try to test with OpenID or LDAP, so I assume more work will be necessary to get them working. I did the work in my experimental Rollarcus fork of Roller. You can get the code from the shiro_not_spring branch. Pull requests are quite welcome as are suggestions for improvement. Please let me know if you see anything wrong in the above code.

This work may not find its way into Roller proper, but it plays a part in my the next side-project that I will share: A REST API for Roller with JAX-RS.

Main | Next page »