Showing posts with label ror. Show all posts
Showing posts with label ror. Show all posts

Monday, December 17, 2007

First Steps with Pylons

I have written previously about how impressed I was with Ruby on Rails (RoR) when I first came across it about a year or so ago. For me, it represented a shift in the way web applications were designed - rather than build an individual application from scratch, the framework generates a template which can be extended as needed for the given application. Obviously, convention over configuration is an important aspect of this approach, since the template code will expect to find files with the "right" name and in the "right" location.

However, I did not know the Ruby scripting language, and in spite of my best efforts, I haven't been able to muster up enough interest to take the time and learn it. For one, Maven 2.x offers a similar (although not as complete) functionality to build application archetypes, and since I mostly do Java web application development for a living, it seemed a better investment of my time to work with Maven2 and Java. Second, although RoR seems to have impressive mindshare among the web literati, my personal experience is that customers expect a mid to large scale apps to be in Java. They usually don't have an opinion about smaller scale (3-10 page) apps, but I find PHP far simpler, more natural, and more than up to the task for building one of these. So there is not enough incentive for me to learn a language in order to use a framework that I will hardly ever use.

However, PHP uses the "code-in-page" approach, which is not easy to maintain once the application grows beyond a certain size. What I needed was a MVC framework based on a scripting language I already knew and enjoyed working in. Happily, when looking recently, I found not one, but three popular open source MVC frameworks based on Python - Django, TurboGears and Pylons.

Of these, TurboGears and Pylons are more glue-like, assembling the framework from a variety of other open source sub-frameworks. It is also possible to switch out some of the sub-frameworks and replace it with another if its a better fit for the application or the programming environment.

I started playing with TurboGears, since that is the more mature (and more stable, I hoped) framework of the two. Unfortunately, I could not get tg-admin to build my application template, because there were some modules mismatches with Python 2.5. After spending an afternoon trying to debug the problem, I gave up and tried Pylons. Installation was a breeze, partly because I already had some of the components installed for TurboGears, but there were absolutely no surprises, and everything worked as expected. I was able to get through the Getting Started (a Hello World style web application) and the Quick Wiki Tutorials within the next couple of days.

As I mentioned above, Pylons is composed of a number of sub-frameworks, each of which are independent projects in their own right. The Paster component provides commands to build an application template, similar to Maven2's archetype:create command, and to deploy the application as a Python egg (similar to WAR files in Java web applications). It also provides a built in HTTP webserver to test the application, similar to Maven2's Jetty plugin.

For database access, Pylons uses the SQLAlchemy project, which provides a Hibernate like API to access database tables and rows as Python objects. These are called model objects in Pylons. Each table (or logical group of tables) is modelled as a Python class. The SQLAlchemy engine is injected into the environment using a Python call in environment.py. The engine exposes a Session object, similar to Hibernate, from which a Query object is extracted. Method calls on the Query object translate to SQL queries to the database. However, the caller only sees application objects. This approach works great when the application gets to decide the database schema, but would probably be slightly more difficult (although not impossible) when building an application against an existing database. In this respect, it has the same limitations as Hibernate. I still need to delve deeper into this though.

For the page rendering subsystem, Pylons uses the Mako templating framework. Mako templates have markup that seems to be a superset of JSTL-style tags and something like Velocity-style tags. The JSTL style tags work off predefined model objects which are injected by Pylons controllers. Personally, I think its a bit confusing to have two kinds of markup, but I guess it may have just evolved that way. Although I am still new to Pylons, I think this would be one of the candidate subsystems where I may be tempted to look for alternatives.

The controller subsystem is provided by Pylons itself, and it is all straight generated Python code. There is a provided BaseController object which all application controllers must extend. The structure of an application controller seems to be similar in style to a Spring MultiActionController, with each public method mapped to a URL pattern, and representing a user action.

To route user URLs to the appropriate controller method, Pylons uses the Routes project. The default routes are set up an in-memory Python dictionary data structure. Additional application specific mappings are added to this structure using additional map.connect() calls in routing.py. This approach appears to be more powerful than Spring XML configuration, in that it allows regular expressions as well. TurboGears uses @expose annotations within the controller methods to do this, which may or may not be better depending on your point of view.

Thus, for a Java programmer who has worked with Maven2, Spring and Hibernate, getting started with Pylons is likely to be quite easy, probably more so than RoR. Of course, it would help if you already knew some Python. I think the relative popularity of RoR among Java programmers is because a lot of them would list Perl as their favorite scripting language, and Perl to Ruby is not that much of a stretch. Perl used to my favorite scripting language at one point, but ever since I began to use Python, reading and writing Perl code is less fun than it used to be. Python is very Java like in a lot of respects, although it is much more concise.

I think that another reason for the relative obscurity of Python based MVC frameworks is the availability of too many choices. Given that Python's philosophy is that "there should be one — and preferably only one — obvious way to do it", I can see how multiple frameworks could be anathema to Python programmers. However, jokes apart, this does dilute the focus of any serious efforts at popularizing a framework. Unlike RoR, there is no one "right" way to do rapid MVC development in Python. However, this is changing, and the two top contenders (in my opinion) in this space are TurboGears and Pylons. The two are very similar, although each glues together different sets of sub-projects. As Ian Bicking states:

I'd have a hard time describing the technical differences (between TurboGears and Pylons) in a meaningful way to someone who didn't already know something about Python web development.
However, TurboGears 2.x will be based on Pylons, so soon Python web developers will have one uber MVC framework to work with, possibly composed of the best sub-projects from either framework. Of course, in keeping with the "glue" philosophy, I expect that the programmer would be able to switch a component out with another one in the same category if required.

Saturday, June 10, 2006

RoR style URLs with Spring MVC

One of the things that I find impressive about Ruby on Rails (RoR) is the simplicity of URLs used in RoR applications, and how they map back to the Controllers and View components. So in the RoR world, a URL of the form:

http://localhost:8080/app/entity/action/1234

means that the request would be forwarded by the web-application named "app" listening on port number 8080 on localhost to the Ruby EntityController class, which would then invoke the action() method on it with "1234" as an argument, then forward the request to the view component at app/entity/action.rhtml for presentation under the webserver's docroot.

This is, of course, both good and bad. It is good because it makes the application easier to understand and debug, both for the user and developer, and removes the need for some configuration, which can be a point of failure. It is bad for applications relying on security through obscurity, because malicious users can understand your application better too, so your application itself must be more security-concious for applications facing the outside world.

I have been thinking about how to do this using the Spring MVC framework, and it turns out to be quite simple. Here is how I did it. The flow in Spring is identical to the flow in RoR. However, I have changed the URL structure somewhat, to mimic what most Java developers and users are used to (including myself). So here is the new URL structure:

http://localhost:8080/app/entity/action.do?id=1234

would send the request to the DispatcherServlet configured in the "app" web application, which will forward it to the EntityController and call its action() method. The action() method would optionally consume the parameter id from the request. Lot of people associate the .do suffix with Struts, but I think its a nice convention to indicate that the URL points to some kind of Controller component, as opposed to static content indicated by .html, for example.

The DispatcherServlet is configured within the application's web.xml file. Currently, it is configured to respond to URLs ending with the .do suffix. The reference to the Spring Application Context, which contains references to beans that will be used by the DispatcherServlet, is set up by the ContextLoaderListener, as shown below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
<web-app>
  <listener>
    <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
  </listener>
                                                                                
  <servlet>
    <servlet-name>app</servlet-name>
    <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>app</servlet-name>
    <url-pattern>*.do</url-pattern>
  </servlet-mapping>
                                                                                
</web-app>

The DispatcherServlet looks for a file called ${servlet.name}-servlet.xml (in our case app-servlet.xml), which typically contains references for the beans that the DispatcherServlet needs. I like to keep it in applicationContext-*.properties in my classpath WEB-INF/classes to make the application unit test friendly and include it from app-servlet.xml. There are three beans that really need to be customized to support RoR style URLs - the HandlerMapping to route the incoming URL, the MethodNameResolver to get the method name to invoke on the Controller which was routed to, and the ViewResolver to do the actual presentation. We also create an abstract class ActiveController which has some convenience method and extends the Spring MultiActionController, but that is just so as to enforce that all Controllers in such applications should be MultiActionControllers.

The HandlerMapping: ActiveControllerUrlHandlerMapping

This is a drop in replacement for standard handler mappings such as SimpleUrlHandlerMapping. Unlike the SimpleUrlHandlerMapping, which reads its mapping configuration once at startup, the ActiveControllerUrlHandlerMapping computes the controller, method and view names each time it is passed a request. It does this by parsing the request URI and pulling out the entity name and adding a "Controller" suffix. It will look this bean up in the ApplicationContext and complain if it cannot find it, so it is important to remember to configure each Controller according to the pattern ${entityName}Controller. Since it is parsing the URL anyway at this stage, it also computes and validates the method name and the view names to use, and sticks them into request attributes.

The requirement to have the Controller bean reference named in a certain way is not there in a RoR app, since it is an interpreted language, so a Controller and a Controller method becomes visible as soon as you drop the new code in the docroot. We could probably set this up to auto-detect a Controller as soon as it becomes visible in a certain package, but the approach would not be totally platform agnostic until Java comes out with a Package.getClasses() method.

The configuration for the handlerMapping looks like this:

1
2
3
4
  <bean id="handlerMapping" class="cnwk.prozac.utils.controllers.ActiveControllerUrlHandlerMapping">
     <property name="defaultHandler" ref="defaultHandler" />
   </bean>
  <bean id="defaultHandler" class="cnwk.prozac.utils.controllers.ActiveControllerDefaultHandler" /> 

Notice that there is no explicit URL pattern to controller mappings here. The defaultHandler is not strictly necessary, but can help to return user friendly results if the ActiveControllerUrlHandlerMapping does not find a valid controller or method to go to. In such cases, rather than throw a 404, it executes the ${defaultHandler}.info() method.

Heres the code for the ActiveControllerUrlHandlerMapping

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
import java.lang.reflect.Method;
 
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
 
import org.apache.commons.beanutils.MethodUtils;
import org.apache.log4j.Logger;
import org.springframework.web.servlet.ModelAndView;
import org.springframework.web.servlet.handler.AbstractUrlHandlerMapping;
 
/**
 * Sets up mappings between the URL pattern and the corresponding Controller
 * beans. The convention for the URL is as follows:
 * <pre>
 * http://host:port/${webAppName}/${entityName}/${methodName}?(${arg}=${value})* * </pre>
 * If no controller bean is found in the application context, then the
 * lookupHandler method returns null.
 * The defaultHandler can be set as it is a property of AbstractHandlerMapping.
 */
public class ActiveControllerUrlHandlerMapping extends AbstractUrlHandlerMapping {
     
    private static final Logger log = Logger.getLogger(ActiveControllerUrlHandlerMapping.class);
     
    public ActiveControllerUrlHandlerMapping() {
        super();
    }
 
    /**
     * Returns a configured ActiveController bean that the URL resolves to.
     * If the URL is malformed, or the ActiveController for the specified URL
     * is not configured, or if the handling method is not available in the
     * resolved ActiveController instance, the ActiveControllerDefaultHandler
     * is returned, with the appropriate error message in the request attribute.     
     * @param request the HttpServletRequest object.
     * @return the ActiveController for this request.
     */
    @Override
    protected Object getHandlerInternal(HttpServletRequest request) {
        String urlPath = request.getRequestURI();
        String[] entityAndMethod = parseEntityAndMethodNamesFromUrl(urlPath);
        if (entityAndMethod[0] == null && entityAndMethod[1] == null) {
            request.setAttribute(ActiveController.ATTR_ERROR_MESSAGE,
                "Malformed URL:[" + urlPath + "], must be of the form " +
                "/${webapp}/${entity}/${method}?[${arg}=${value}&...]");
            return null;
        }
        String requestedController = entityAndMethod[0] + "Controller";
        Object handler = getApplicationContext().getBean(requestedController);
        if (handler == null) {
            request.setAttribute(ActiveController.ATTR_ERROR_MESSAGE,
                "The ActiveController instance " + requestedController + 
                " is not configured in the ApplicationContext");
            return null;
        } else if (!(handler instanceof ActiveController)) {
            request.setAttribute(ActiveController.ATTR_ERROR_MESSAGE,
                "The bean " + requestedController + " is not a ActiveController");
            return null;
        } else {
            Method requestedMethod = MethodUtils.getAccessibleMethod(
                handler.getClass(), entityAndMethod[1],
                new Class[] {HttpServletRequest.class, HttpServletResponse.class});
            if (requestedMethod == null) {
                request.setAttribute(ActiveController.ATTR_ERROR_MESSAGE,
                    "The method " + entityAndMethod[1] + 
                    "(HttpServletRequest, HttpServletResponse):ModelAndView is not defined in " + 
                    requestedController);
                return null;
            } else {
                String returnTypeClassName = requestedMethod.getReturnType().getName();
                if (!(ModelAndView.class.getName().equals(returnTypeClassName))) {
                    request.setAttribute(ActiveController.ATTR_ERROR_MESSAGE,
                        "The method " + entityAndMethod[1] + " has incorrect return type " + 
                        returnTypeClassName + 
                        ", should be org.springframework.web.servlet.ModelAndView, " +
                        "check your code");
                    return null;
                }
            }
        }
        request.setAttribute(ActiveController.ATTR_VIEW_NAME, entityAndMethod[0] + 
            "/" + entityAndMethod[1]);
        request.setAttribute(ActiveController.ATTR_METHOD_NAME, entityAndMethod[1]);
        return handler;
    }
     
    /**
     * Returns the entity and method names from the URL.
     * @param urlPath
     * @return
     */
    private String[] parseEntityAndMethodNamesFromUrl(String urlPath) {
        String[] parts = urlPath.split("[\\/|\\&]");
        if (parts.length < 4) {
            return new String[] {null, null};
        }
        String[] entityAndMethodNames = new String[2];
        entityAndMethodNames[0] = parts[2];
        if (parts[3].indexOf(".") > -1) { // remove any trailing suffix
            entityAndMethodNames[1] = parts[3].split("\\.")[0];
        } else {
            entityAndMethodNames[1] = parts[3];
        }
        return entityAndMethodNames;
    }
}

The MethodNameResolver: ActiveControllerMethodNameResolver

The ActiveControllerMethodNameResolver simply retrieves the value of the request attribute that was set by the HandlerMapping bean when it parsed and validated the incoming URL. If there was an error parsing the URL, the Spring framework will pass it off to the defaultHandler and the method attribute would be null in the incoming request. So all this bean does is to check if the method attribute is null, and if so, set it to the string "info".

Here is how it is configured:

1
  <bean id="methodNameResolver" class="cnwk.prozac.utils.controllers.ActiveControllerMethodNameResolver" />

and here is the code for the ActiveControllerMethodNameResolver

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
import javax.servlet.http.HttpServletRequest;
 
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.web.servlet.mvc.multiaction.AbstractUrlMethodNameResolver;
import org.springframework.web.servlet.mvc.multiaction.MethodNameResolver;
import org.springframework.web.servlet.mvc.multiaction.NoSuchRequestHandlingMethodException;
 
/**
 * Resolves the method name for the specified controller. If the method does
 * not exist, then it returns an error.
 */
public class ActiveControllerMethodNameResolver implements MethodNameResolver {
     
    private static Log log = LogFactory.getLog(ActiveControllerMethodNameResolver.class);
     
    public ActiveControllerMethodNameResolver() {
        super();
    }
 
    /**
     * Returns the method name that will be executed.
     * @see org.springframework.web.servlet.mvc.multiaction.MethodNameResolver#getHandlerMethodName(javax.servlet.http.HttpServletRequest)
     */
    public String getHandlerMethodName(HttpServletRequest request) 
            throws NoSuchRequestHandlingMethodException {
        // check to see if its already populated by ActiveControllerUrlHandlerMapping
        String methodName = (String) request.getAttribute(ActiveController.ATTR_METHOD_NAME);
        if (methodName == null) {
            return "info";
        }
    }
}

The ViewResolver: InternalResourceViewResolver

For the ViewResolver, we just use one of the standard view resolvers provided with Spring. The HandlerMapping already populates the view name as ${entityName}/${methodName}, and the ViewResolver is configured with a prefix and suffix as shown below:

1
2
3
4
5
  <bean id="viewResolver" class="org.springframework.web.servlet.view.InternalResourceViewResolver">
    <property name="prefix" value="/" />
    <property name="suffix" value=".jsp" />
    <property name="viewClass" value="org.springframework.web.servlet.view.JstlView" />
  </bean>

So in this case, a view name of the form "person/list" would be resolved by the JSP file person/list.jsp under the web application's docroot.

The Controller superclass: ActiveController

Finally, since all our Controllers need to be MultiActionControllers to avail of this RoR style URL mappings, we enforce this by requiring that all our Controller subclass the ActiveController class. If they do not, the HandlerMapping would refuse to resolve the URLs. The ActiveController does provide one useful method getDefaultViewName() which pulls out the view name attribute off the request. Here is the configuration.

1
2
3
  <bean id="activeController" class="cnwk.prozac.utils.controllers.ActiveController">
    <property name="methodNameResolver" ref="methodNameResolver" />
  </bean>

Notice that it needs a reference to the methodNameResolver, which is our ActiveControllerMethodName resolver. The code is here:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
 
import org.apache.log4j.Logger;
import org.springframework.context.ApplicationContextException;
import org.springframework.web.servlet.ModelAndView;
import org.springframework.web.servlet.mvc.multiaction.MultiActionController;
 
/**
 * Base class for our RoR URL handling controllers.
 */
public class ActiveController extends MultiActionController {
 
    private final static Logger log = Logger.getLogger(ActiveController.class);
     
    public static final String ATTR_REQUEST_OBJECT = "_request";
    public static final String ATTR_ERROR_MESSAGE = "_error";
    public static final String ATTR_METHOD_NAME = "_methodName";
    public static final String ATTR_VIEW_NAME = "_viewName";
    public static final String DEFAULT_METHOD_NAME = "info";
     
    /**
     * Default ctor.
     * @throws ApplicationContextException if one is thrown.
     */
    public ActiveController() throws ApplicationContextException {
        super();
    }
 
    /**
     * Alternate ctor.
     * @param delegate the Object to delegate to.
     * @throws ApplicationContextException if one is thrown.
     */
    public ActiveController(Object delegate) throws ApplicationContextException {
        super(delegate);
    }
 
    /**
     * The default info() method which is called whenever a method cannot be
     * resolved. The info() method contains information about the URL sent, any
     * error messages, and (optionally) other information useful for debugging.
     * @param request a HttpServletRequest object.
     * @param response a HttpServletResponse object.
     * @return a ModelAndView
     * @throws Exception if one is thrown during processing.
     */
    public ModelAndView info(HttpServletRequest request, HttpServletResponse response) 
            throws Exception {
        ModelAndView mav = new ModelAndView();
        mav.addObject(ActiveController.ATTR_REQUEST_OBJECT, request);
        mav.setViewName(getDefaultViewName(request));
        return mav;
    }
     
    /**
     * Returns the method name from the request. The method name is populated
     * by the ActiveControllerUrlHandlerMapping. The method name is the view
     * name in our "convention over configuration" world. The exact mechanics
     * of view resolution (eg whether it goes to a JSP or a Tile) is controlled
     * by the ViewResolver class injected into the DispatcherServlet.
     * @param request the HttpServletRequest object.
     * @return the view name to forward to.
     */
    protected String getDefaultViewName(HttpServletRequest request) {
        return (String) request.getAttribute(ActiveController.ATTR_VIEW_NAME);
    }
}

Usage

Once the HandlerMapping, MethodNameResolver and the ViewResolver are configured, and these classes added to the system classpath, Controller classes will need to extend the ActiveController, and provide methods with the following signature:

public ModelAndView ${methodName}(HttpServletRequest request, 
  HttpServletResponse response) throws Exception;

and once the Controller class itself is configured as per the convention ${entityName}Controller, these methods would be automatically accessible to the web application at the URL /${webapp.name}/${entityName}/${methodName}.do.

In the course of doing some googling, I also found that the author of the blog MemeStorm has written an article here - Convention over Configuration in Spring MVC which describes an approach very similar to mine. It also contains many more articles on Spring MVC so its a good place to look for more insights about Spring.

Saturday, March 18, 2006

Book Reviews with Ruby On Rails

Late last year, I came across this ONLamp article on Ruby on Rails. I had heard of the scripting language Ruby before, while I was looking at moving to Python from Perl as my scripting language of choice. To be honest, Ruby the language did not (and still does not) interest me very much. I find it lacking in many features that is taken for granted in Perl, and to a lesser extent, Python. I also find the syntax of Ruby a little "unnatural", since it overloads various syntax notations that mean different things in Perl, which I am already used to.

So, had it not been for Ruby on Rails, I probably wouldn't have looked at Ruby ever again. However, Ruby on Rails (RoR) provides functionality that is too good to pass up merely because I dont like the underlying language. To put it briefly, RoR is a web-development toolkit in a box. You provide the underlying database schema, and RoR provides scripts that generate a bunch of files that will provide basic CRUD (Create, Retrieve, Update, Delete) functionality via a web browser. All without writing a single line of Ruby code!

Of course, what RoR generates is almost never going to be good enough for your customers, unless your customers have exceptionally low expectations, or unless the customer is you and you are looking for an alternative to doing straight SQL calls on the database. This is because the database (and by extension, RoR) does not know how its various tables are linked together. To make the RoR application look remotely like something from the real world, you will have to learn about how RoR works, and you will have to learn Ruby. So the TANSTAAFL (There A'int No Such Thing As A Free Lunch) rule applies here as well.

Luckily, I fell into the latter category. I needed a tool to write and populate book reviews into a database. These book reviews would then be rendered into web pages for my Amazon.com affiliate website.

But first, a little history. Sometime in 1996, I signed up for a free web page from Geocities, a company which has since been acquired by Yahoo!. I knew some basic HTML, actually above-average for the time, since I had recently finished writing a CGI program to allow people to send me database change request forms over the corporate intranet. Two sites I found particularly helpful were Joe Burn's HTMLGoodies (for HTML coding) and Phil Greenspun's photo.net (for CGI scripting).

Some time later, Amazon.com started selling books, and they attempted to increase their marketing presence by offering the Amazon affiliate program, and I happily signed up for that too. I figured that I would be particularly well suited for this sort of thing, since I read (a lot, mostly technical books), both out of choice (because I enjoy reading) and necessity (because my career requires me to keep abreast of the very rapid changes in my chosen field). I also figured that if I did get some commissions on clickthrough traffic, maybe I could buy a couple of books a year with that money, not much I know, but every little bit helps. So I wrote up some book reviews of what I had read, and put up a page with the links to Amazon's store.

However, I was writing the pages manually with a text editor, so remembering to keep each review in the same format and keeping the links consistent was laborious and time consuming. I also never actually made any money off the affiliate storefront. I soon found other, more interesting things to do, and the site languished for a while. I then redesigned the entire site to be link-less (ie, no links between pages in my site) site, so it would cut down on the manual labor involved in keeping links up-to-date. But the site as a result of the redesign was (and still is) ugly and almost un-navigable.

I have been planning another site redesign, this time as static pages generated off content from a database. So RoR looked ideal for building a simple tool that would let me enter book reviews and categorize them into various broad categories. I could write a Python script to read the database and generate a bunch of HTML pages which I could upload to the geocities server. So in effect, I was getting my free lunch - a web based tool to enter my data, written in Ruby by RoR, and a Python script (which I would write myself, and which was "free" too, since I already knew Python) to generate the pages and upload to the server.

So last week I set up a simple RoR application with two tables in the database - books and categories. A category can have many books within it. The process to get the basic CRUD functionality up took less than an hour. Here is the database schema:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
create table books (
    id bigint not null auto_increment,
    name varchar(255) not null,
    author varchar(255) not null,
    amazon_asin varchar(32) not null,
    review text not null,
    category_id bigint not null,
    primary key(id)
) type=InnoDB;
                                                                                
create table categories (
    id bigint not null auto_increment,
    category_name varchar(128) not null,
    primary key(id)
) type=InnoDB;

RoR is optimized for MySQL. My database of choice for personal work is PostgreSQL, but I could not make RoR work with it. Although I did not try very hard, all I did was download postgres-pr (the Ruby binding for PostgreSQL) and update the config/database.yml file to use PostgreSQL. I may have to learn more about RoR to hook it up with PostgreSQL.

The only other changes I had to make to link categories and books together in the RoR object model was to add the belongs_to and has_many declarations to the model classes:

1
2
3
4
5
6
7
8
9
# app/models/book.rb
class Book < ActiveRecord::Base
    belongs_to :category
end

# app/models/category.rb
class Category < ActiveRecord::Base
    has_many :book
end

To make the selection list of categories to show up in my Book edit form, I added this single line to the edit method in app/controllers/books_controller.rb to generate the complete list of categories.

1
2
3
4
5
def edit
    @book = Book.find(params[:id])
    # added this line
    @categories = Category.find_all
end

And this snippet to the views/books/edit.rhtml file to display a select list of all categories with the currently selected category highlighted.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# views/books/edit.rhtml
...
<%= render :partial => 'form' %>
# added this snippet here                                                                                
&;lt;p><b>Category:</b><br>
<select name="book[category_id]">
 <% @categories.each do |category| %>
     <option value="<%= category.id %>"
       <%= ' selected' if category.id == @book.category_id %>>
       <%= category.category_name %>
     </option>
 <% end %>
</select></p>
...

All these changes did not require me to gain a deep understanding of the RoR framework, I really extrapolated from Curt Hibbs's ONLamp article referenced above.

However, I have been looking at the features of RoR in more depth since I first saw the ONLamp article. I have even purchased the book "Agile Web Development with Rails" by Dave Thomas and David Hansson, and have read through it to understand how RoR works under the covers. I find RoR to be a very well constructed toolkit. A lot of thought has been put into the correct way to do things, such as using HTTP POST requests to trigger operations that update the database. It also automatically generates unit test code skeletons as it generates the code for controllers, views and models, so there is built in support for a test driven development environment. RoR has fewer artefacts than comparitive J2EE frameworks, since RoR relies on convention rather than configuration, and for most of the artefacts that it does need, it generates at least a base template which the developer can customize. Because RoR generates the base application, its developers typically have more time to spend writing code thats important to the customer rather than trying to configure the framework to work with the application. As a result, a RoR developer is more likely to develop a well-designed application with fewer bugs than a Java/J2EE developer.

Saturday, February 11, 2006

WebApps Development the Rails way

When people think of Web application developers, presumably the first image they have is a graphic designer with some PHP or other scripting knowledge. While this was true 5-7 years ago, the web application development landscape has changed dramatically in recent years. I can only speak for Java based web development, but I am sure that there is a similar proliferation of choices in other languages too. Today's Java web developer needs to know object relational mapping (ORM) technologies such as EJB, Hibernate or JDO, web application frameworks such as Struts, Webwork or Spring, front end tools such as HTML, JSP, Velocity, Tapestry, JSTL, Java Server Faces or even Javascript with AJAX, with miscellaneous middleware components thrown in such as Lucene for search, JMS for asynchronous messaging, SOAP for remoting, and so on. It a wonder that a web developer doesn't get more respect.

However, I find that the biggest challenge in web development is not what technologies to use (although that choice does have impact on scaling, responsiveness and other usability concerns), but what the site's navigation should look like. Most sites attempt to layer logic over HTTP's stateless protocol to provide an illusion of continuity to the end-user. Figuring out the appropriate navigation strategy plays a big part in creating that illusion. Of course, it helps if the web developer can think like his end customer, such as a developer for an online games site who is himself a gamer. Some companies are not so lucky, such as a bank, whose developers are typically more financially savvy than the end users they serve.

It is easy to get bogged down by details of the navigation, and in the process, create web site flow logic that is difficult to maintain. Worse, the flow logic leaks into the application logic, making core application code that much more brittle.

I have been reading about Ruby on Rails in the Agile Web Development with Rails book. In short, what the Rails framework does is allow you to start with a database table and automatically generate your model, controller and view with rails scripts. This allows you to set up simple web based applications relatively painlessly. To make any significant customizations to the generated application, however, you would need to know Ruby, which I do not as yet. There are other efforts to replicate this functionality in the Java world, such as the Trails project and Appfuse (with AppGen). However, what excited me about Rails is not the code generation itself, but that it represents a simplified way to do web development.

Most web applications have as their basis one or more databases, with tables representing the business entities for the application. The Rails approach to web development allows you to build the controllers and views to store, retrieve or delete these entities first. Once this is done, you can then concentrate on building up the relationships between the entities by providing the appropriate navigation hooks.

For example, in a purchasing system, you would first build up the operations for the Order and Part entities, including the views for each. The Part entity would be a one-to-many relationship with Order, so in your database you would represent it with a foreign key in Part pointing back to Order.

1
2
3
4
5
6
create table Order (...);
create table Part (
   id int not null primary key,
   ...
   order_id int not null foreign key(Order)
);

In your model, you would represent this with a collection member variable, like so:

1
2
3
4
5
6
class Order {
   private Set<Part> parts;
   ...
   public Set<Part> getParts() { get parts; }
   public void setParts(Set<Part> parts) { this.parts = parts; }
}

So even before you attempted to build the view for the Purchase order which contains a set of Part line items for the Order entity, you would have the model, view and controller for the Part and Order entities ready.

Navigation would then be grafted on to the system at the entity level. For example, the Payment entity would be linked to the purchase order view by a navigation link such as "Pay for this order". At the database level it would be represented as a one-to-one mapping between Order and Payment.

1
2
3
4
5
6
7
8
create table Order (
   ...
   payment_id int not null foreign key (Part)
);
create table Payment (
   ...
   order_id int not null foreign key (Order)
);

The model for Payment and Order will look like this, with properties that point back to each other.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
class Payment {
   private Order order;
   ...
   public Order getOrder() { return order; }
   public void setOrder(Order order) { this.order = order; }
}
class Order {
   private Payment payment;
   ...
   public Payment getPayment() { return payment; }
   public void setPayment(Payment payment) { this.payment = payment; }
}

Effectively, you are letting the database model for the application drive the navigation, which is probably not a bad thing for a number of reasons.

First, by designing the database layer first, its very likely that the database model has a strong correlation with the business domain model. The domain model represents the aggregate of the way people have been doing this particular thing for a number of years, so inefficiencies have presumably been weeded out of the process. In some industries, domain models are standardized, so new developers joining your team are likely to be familiar with the domain model even before they sign up. Even if the domain model is not standard, it is easier to explain a domain model to customers and new hires than arcane data models. As for explaining navigation flows, I have found that it is most often never explained, much less justified with logic; new developers are left to navigate through the site themselves and find out on their own.

Second, the user is more likely to be familiar with the domain model for your business than navigation that you as a developer or designer can dream up, since you are more likely than not to be out of sync with your customer's needs, unless you are in the happy position of being able to think exactly like your customer. Thus the user is going to be more comfortable with a navigation strategy that is based on the domain model. Users who migrate to your application from other applications are also likely to have less of a re-learning curve, since the other application would have implemented at least part of its navigation to be domain-driven.

Third, and finally, it allows you to defer development of navigation details until you are done developing the core functionality for the business entities. From an agile viewpoint, this allows you to demonstrate working code to your client sooner than if you try to develop code with the navigation logic in place, and possibly mixed in with, the core logic for the entities. During the phase where you are grafting your navigation into your partially completed web application, you are really working with larger working components (the entities) and treating them like black boxes as you wire them together with your navigation strategy, so it also results in robust, easy to understand, and easy to maintain code.

I have recently been experimenting with this strategy on a open-source development project where the mandate was to develop a web application to front a model layer developed with Hibernate by members of the core team. I decided to use Tapestry to do the development (I did not know Tapestry and this was a chance to learn). The core entities are now ready and I have submitted it back to the core team for their feedback. The great thing about this strategy so far is that I did not have to have a complete understanding of the domain which the model is designed for. I can defer this work till the next stage, when I apply the navigation and refine the view layer based on the core team's feedback. The feedback itself will provide me with most of the understanding of the domain model that is needed.