Showing posts with label rome-custom-modules. Show all posts
Showing posts with label rome-custom-modules. Show all posts

Saturday, August 02, 2008

Parsing custom modules with ROME Fetcher

Last week, I described a fairly basic feed fetcher written using ROME's Fetcher library. My intent is provide clients of our RSS 2.0 based API a convenient way to access the XML as a Java object, negating the need for XML parsing on their end. Our API does return standard RSS 2.0, but we add extra information using a custom module at both the feed and the entry level, which the implementation described in the last post could not parse out as a module. Instead, it treated it as foreign markup, which a client could parse out using the following snippet.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
  SyndFeed feed = client.execute(...);
  List<SyndEntry> entries = feed.getEntries();
  for (SyndEntry entry : entries) {
    ...
    List<Element> foreignMarkups = (List<Element>) entry.getForeignMarkup();
    for (Element foreignMarkup : foreignMarkups) {
      if (foreignMarkup.getNamespaceURI().equals(MyModule.URI)) {
        // we got our custom module, now parse it
        if (foreignMarkup.getName().equals("score")) {
          // extract and populate the value of score
          float score = Float.valueOf(foreignMarkup.getValue());
        }
        ...
      }
    }
  }

This is a workable solution, but not ideal. What I would like is for clients to be able to get a reference to the custom module by URI, then use the getters and setters defined on the module to populate their objects. This post describes the changes I had to make to get this functionality to work.

ROME depends on a plug-in mechanism that is driven by the rome.properties file. ROME comes with one built in, but it can be overriden by placing one's custom rome.properties file at the root of the classpath. So here is my rome.properties file for reference.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
# rome.properties

rss_2.0my.feed.ModuleGenerator.classes=\
com.sun.syndication.io.impl.DCModuleGenerator \
com.sun.syndication.io.impl.SyModuleGenerator \
com.sun.syndication.feed.module.opensearch.impl.OpenSearchModuleGenerator \
com.mycompany.myapp.mymodule.MyModuleGenerator

rss_2.0my.feed.ModuleParser.classes=\
com.sun.syndication.io.impl.DCModuleParser \
com.sun.syndication.io.impl.SyModuleParser \
com.sun.syndication.feed.module.opensearch.impl.OpenSearchModuleParser \
com.mycompany.myapp.mymodule.MyModuleParser

rss_2.0my.item.ModuleParser.classes=\
com.mycompany.myapp.mymodule.MyModuleParser

rss_2.0my.item.ModuleGenerator.classes=\
com.mycompany.myapp.mymodule.MyModuleGenerator

WireFeedParser.classes=\
com.sun.syndication.io.impl.RSS090Parser \
com.sun.syndication.io.impl.RSS091NetscapeParser \
com.sun.syndication.io.impl.RSS091UserlandParser \
com.sun.syndication.io.impl.RSS092Parser \
com.sun.syndication.io.impl.RSS093Parser \
com.sun.syndication.io.impl.RSS094Parser \
com.sun.syndication.io.impl.RSS10Parser  \
com.sun.syndication.io.impl.RSS20wNSParser  \
com.sun.syndication.io.impl.RSS20Parser  \
com.sun.syndication.io.impl.Atom10Parser \
com.sun.syndication.io.impl.Atom03Parser \
com.mycompany.myapp.mymodule.MyRss20Parser

WireFeedGenerator.classes=\
com.sun.syndication.io.impl.RSS090Generator \
com.sun.syndication.io.impl.RSS091NetscapeGenerator \
com.sun.syndication.io.impl.RSS091UserlandGenerator \
com.sun.syndication.io.impl.RSS092Generator \
com.sun.syndication.io.impl.RSS093Generator \
com.sun.syndication.io.impl.RSS094Generator \
com.sun.syndication.io.impl.RSS10Generator  \
com.sun.syndication.io.impl.RSS20Generator  \
com.sun.syndication.io.impl.Atom10Generator \
com.sun.syndication.io.impl.Atom03Generator \
com.mycompany.myapp.mymodule.MyRss20Generator

Converter.classes=\
com.sun.syndication.feed.synd.impl.ConverterForAtom10 \
com.sun.syndication.feed.synd.impl.ConverterForAtom03 \
com.sun.syndication.feed.synd.impl.ConverterForRSS090 \
com.sun.syndication.feed.synd.impl.ConverterForRSS091Netscape \
com.sun.syndication.feed.synd.impl.ConverterForRSS091Userland \
com.sun.syndication.feed.synd.impl.ConverterForRSS092 \
com.sun.syndication.feed.synd.impl.ConverterForRSS093 \
com.sun.syndication.feed.synd.impl.ConverterForRSS094 \
com.sun.syndication.feed.synd.impl.ConverterForRSS10  \
com.sun.syndication.feed.synd.impl.ConverterForRSS20 \
com.mycompany.myapp.mymodule.ConverterForMyRss20

You will notice that I am using a custom version of RSS 2.0 called rss_2.0my. The reason for this is ROME SyndEntry does not have a way to set the rss/channel/item/source element, which I handled by extending ROME. Additionally, we use a custom module MyModule that carries information that is not possible to send using standard RSS. We also use the Amazon OpenSearch and the Content modules.

We have used ROME for a while to successfully generate the feeds, but this was the first time we were looking at eating our own dog food, as it were. I am not sure if our setup is that uncommon, but either there are gaps in the ROME parsing code or we are doing something wrong. If you have been through this and solved this more cleanly, your comments and suggestions would be appreciated.

Change to ROME: FeedParsers.java

Currently, the code for FeedParsers.getParserFor(Document) loops through the parsers defined for WireFeedParser.classes in rome.properties. Each parser's isMyType() method is invoked, and if satisfied, the first parser is selected.

The problem is that the RSS20Parser.isMyType(Document) is too loose, all it does is verify that the root element is "rss" and the value of rss.@version startswith "2.0". So what is selected is the RSS20Parser, which is not what I want, and besides all my modules are registered to the feed type rss-2.0my, which is what is supported by MyRss20Parser.

So I needed it to keep going until it found the last matched parser, so I made the method sticky. An alternative approach would be to traverse the list backwards, since the selection would then go from the most specific parser to the least specific. Here is my code for FeedParsers.getParserFor(Document).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
    public WireFeedParser getParserFor(Document document) {
        List parsers = getPlugins();
        WireFeedParser selectedParser = null;
        for (int i=0;i < parsers.size();i++) {
            WireFeedParser parser = (WireFeedParser) parsers.get(i);
            if (parser.isMyType(document)) {
                selectedParser = parser;
            }
        }
        return selectedParser;
    }

Change to MyRss20Parser.java

I added a isMyType(Document) method to my custom RSS20Parser so it does not call the superclass's isMyType(). It does piggyback on RSS20Parser.isMyType() to figure out if it RSS 2.0, and if so, whether it contains the namespace for MyModule.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
  public boolean isMyType(Document document) {
    boolean isValidRss20 = super.isMyType(document);
    boolean isValidMyRss20 = false;
    if (isValidRss20) {
      Element rssRoot = document.getRootElement();
      List<Namespace> namespaces = rssRoot.getAdditionalNamespaces();
      for (Namespace namespace : namespaces) {
        if (namespace.getURI().equals(MyModule.URI));
        isValidMyRss20 = true;
        break;
      }
    }
    return isValidRss20 && isValidMyRss20;
  }

The changes to ROME their CVS versions downloaded on 2008-07-23. I plan on submitting a patch for the changes, so hopefully it will be available in future releases of the software. Unless, of course, there is a workaround, which I would be happy to use.

Additionally, I am using the ROME Fetcher code from CVS. Unlike the 0.9 version available at the time of this writing, the CVS version has support for configurable connection and read timeouts, which I wanted to provide. But I already talked about that in the previous post.

With these changes, I am finally able to get results using the following code on my client application.

Posted by Sujit Pal at 12:21 PM 4 comments

Labels: , , ,

Share/Save: Digg Technorati Delicious StumbleUpon Reddit BlinkList Furl Mixx Facebook Google Bookmark Yahoo

Friday, October 19, 2007

Extending ROME to do RSS 2.0

In my previous post, I mentioned that I was trying to use ROME to convert an existing RSS 2.0 feed. ROME provides an abstraction of the various RSS and Atom flavors using its SyndFeed object. A Java programmer using the ROME library would use the SyndFeed object to build up a feed, then write it out into a WireFeed object of the appropriate type, using code like the one shown below:

1
2
3
4
5
  SyndFeed myfeed = new SyndFeedImpl();
  ... // populate the feed object
  WireFeedOutput outputter = new WireFeedOutput();
  WireFeed wirefeed = myfeed.createWireFeed("rss_2.0");
  System.out.println(outputter.output(wirefeed));

My problem was that I had a //rss/channel/item/source tag in the feed I was trying to convert. There did not seem to be any way to set anything in the SyndEntryImpl object (the ROME abstraction of the RSS item and the Atom entry elements) that would translate to a RSS item/source element. This seems strange, since the source element is valid RSS 2.0 according to the RSS 2.0 specifications. It may just be an oversight or a conscious decision by the ROME developers to exclude this element, but I needed it. Luckily, ROME was designed with extension in mind, and with the help of Dave Johnson's RAIA book, it was pretty simple to do. This post describes what I needed to do so that I could set a source attribute into the SyndEntryImpl object which would render and parse to and from a RSS 2.0 WireFeed object.

I chose the SyndEntry.setContributors() method which takes a List of SyndPerson objects as the method to set my source attribute from the SyndEntry object. The contributors object is used in Atom, so since I was not going to use Atom in this particular case, it was safe to use this method.

First I built a custom converter by extending the built in RSS 2.0 converter. The job of the converter is to translate between a SyndFeed and an RSS or Atom Feed object. However, since all I wanted to do was to be able to push a source object into my SyndEntry object and get it out when it was converted to an Item object, I extended protected methods of the superclass that does just that. Here is the code for my converter.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
// ConverterForMyRss20.java
package com.mycompany.myapp.mymodule;

import java.util.Arrays;
import java.util.List;

import com.sun.syndication.feed.rss.Item;
import com.sun.syndication.feed.rss.Source;
import com.sun.syndication.feed.synd.Converter;
import com.sun.syndication.feed.synd.SyndEntry;
import com.sun.syndication.feed.synd.SyndPerson;
import com.sun.syndication.feed.synd.SyndPersonImpl;
import com.sun.syndication.feed.synd.impl.ConverterForRSS20;

public class ConverterForMyRss20 extends ConverterForRSS20 {

  /**
   * Default Ctor. Must be implemented and delegates to type ctor.
   */
  public ConverterForMyRss20() {
    this("myrss_2.0");
  }

  /**
   * Type Ctor. Must be implemented and delegates to superclass type ctor.
   * @param feedType the feed type to convert.
   */
  public ConverterForMyRss20(String feedType) {
    super(feedType);
  }
  
  /**
   * Called from {@link Converter#createRealFeed(com.sun.syndication.feed.synd.SyndFeed)}
   * Delegates to the superclass to create a partially populated Item object, then
   * provides additional logic to pull extra elements from the SyndEntry object into
   * the Item object and returns it.
   * @param syndEntry the SyndEntry to populate from.
   * @return the Item object.
   */
  @Override
  @SuppressWarnings("unchecked")
  protected Item createRSSItem(SyndEntry syndEntry) {
    Item item = super.createRSSItem(syndEntry);
    List<SyndPerson> contributors = syndEntry.getContributors();
    if (contributors != null && contributors.size() > 0) {
      Source source = new Source();
      source.setValue(contributors.get(0).getName());
      item.setSource(source);
    }
    return item;
  }
  
  /**
   * Called from {@link Converter#copyInto(com.sun.syndication.feed.WireFeed, com.sun.syndication.feed.synd.SyndFeed)}
   * Delegates to the superclass to create a partially populated SyndEntry object, then
   * adds the extra elements from the Item object to the SyndEntry object.
   * @param item the Item object to convert to a SyndEntry object.
   */
  protected SyndEntry createSyndEntry(Item item) {
    SyndEntry syndEntry = super.createSyndEntry(item);
    Source source = item.getSource();
    if (source != null) {
      SyndPerson syndPerson = new SyndPersonImpl();
      syndPerson.setName(source.getValue());
      syndEntry.setContributors(Arrays.asList(new SyndPerson[] {syndPerson}));
    }
    return syndEntry;
  }
}

I also needed to write a custom WireFeedParser and WireFeedGenerator to parse RSS 2.0 feeds to and from ROME objects. I really only needed to build the generator, but I built the parser too, just in case I need to provide my clients with tools to parse the RSS feeds I generate. Here is the code for my custom WireFeed parser and generator. As before, I override the RSS 2.0 parser and generator, so I only need to override a single protected method in each superclass.

Here is the code for my custom WireFeed parser. As in the case of the custom converter, the default and type constructors are required, and class just overrides the parseItem() method in the built in WireFeed parser for RSS 2.0 to grab the source JDOM Element from the XML and add it to the RSS Item object.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
// MyRss20Parser.java
package com.mycompany.myapps.mymodule;

import org.jdom.Element;

import com.sun.syndication.feed.rss.Item;
import com.sun.syndication.feed.rss.Source;
import com.sun.syndication.io.impl.RSS20Parser;

public class MyRss20Parser extends RSS20Parser {
  
  /**
   * Default Ctor. Must be implemented and delegates to type ctor.
   */
  public MyRss20Parser() {
    this("myrss_2.0");
  }
  
  /**
   * Type Ctor. Must be implemented and delegates to superclass type ctor.
   * @param feedType the feed type for this parser.
   */
  public MyRss20Parser(String feedType) {
    super(feedType);
  }
  
  /**
   * Called from {@link com.sun.syndication.io.WireFeedParser#parse(org.jdom.Document, boolean)}
   * Delegates to the superclass to build an Item object, then adds the source to the
   * returned Item if it exists in the Element itemElement.
   * @param rssRoot the root Element of the RSS feed.
   * @param itemElement the Element representing the Item object.
   * @return the Item object with the source added if it exists.
   */
  @Override
  public Item parseItem(Element rssRoot, Element itemElement) {
    Item item = super.parseItem(rssRoot, itemElement);
    Element sourceElement = itemElement.getChild("source", getRSSNamespace());
    if (sourceElement != null) {
      Source source = new Source();
      source.setValue(sourceElement.getTextTrim());
      item.setSource(source);
    }
    return item;
  }
}

And here is the code for the custom WireFeedGenerator. As before, the default and the type constructors are required by the framework, and all this class does is to override the superclass's populateItem() method, to populate an Item JDOM Element object with the source Element if the Item.getSource() value is not null.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
// MyRss20Generator.java
package com.mycompany.myapp.mymodule;

import org.jdom.Element;

import com.sun.syndication.feed.rss.Item;
import com.sun.syndication.io.WireFeedGenerator;
import com.sun.syndication.io.impl.RSS20Generator;

public class MyRss20Generator extends RSS20Generator {

  /**
   * Default Ctor. Must be implemented and delegates to type ctor.
   */
  public MyRss20Generator() {
    this("myrss_2.0", "2.0");
  }
  
  /**
   * Type Ctor. Must be implemented and delegates to superclass type ctor.
   * @param feedType the feed type for this generator.
   * @param version the feed version for this generator.
   */
  public MyRss20Generator(String feedType, String version) {
    super(feedType, version);
  }
  
  /**
   * Called from {@link WireFeedGenerator#generate(com.sun.syndication.feed.WireFeed)}
   * Delegates to the superclass to partially populate an Item element from
   * an Item object. Adds on a source element to the item element if the Item
   * object has a non-null source.
   * @param item the Item object from which to populate.
   * @param itemElement the Element being populated from the Item.
   * @param index the index of the object, not used here.
   */
  @Override
  public void populateItem(Item item, Element itemElement, int index) {
    super.populateItem(item, itemElement, index);
    if (item.getSource() != null) {
      Element sourceElement = new Element("source", getFeedNamespace());
      sourceElement.setText(item.getSource().getValue());
      itemElement.addContent(sourceElement);
    }
  }
}

To let ROME know that I will be using my dialect of RSS 2.0 (myrss_2.0) which supports the source element, I need to update the rome.properties file with the mapping to the above classes. I also need to change the mapping for the Module parser and generator to use this dialect. My complete rome.properties looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
myrss_2.0.item.ModuleParser.classes=\
com.mycompany.myapp.mymodule.MyModuleParser

myrss_2.0.item.ModuleGenerator.classes=\
com.mycompany.myapp.mymodule.MyModuleGenerator

WireFeedParser.classes=\
com.mycompany.myapp.mymodule.MyRss20Parser

WireFeedGenerator.classes=\
com.mycompany.myapp.mymodule.MyRss20Generator

Converter.classes=\
com.mycompany.myapp.mymodule.ConverterForMyRss20

Once this is all done, the Java code to set a source element inside a SyndEntry object looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
  SyndEntry entry = new SyndEntryImpl();
  ...
  SyndPerson source = new SyndPersonImpl();
  source.setName(mydomainObject.getSourceName());
  entry.setContributors(Arrays.asList(new SyndPerson[] {source}));
  ...
  feed.getEntries().add(entry);

  WireFeedOutput outputter = new WireFeedOutput();
  WireFeed wirefeed = myfeed.createWireFeed("myrss_2.0");
  System.out.println(outputter.output(wirefeed));

And I am happy to say that the resulting item elements in the RSS 2.0 feed did contain the source as expected. However, it is very possible that I am doing something wrong and there is some settable field in SyndEntry that will allow the source to be populated without going through all this trouble. If there is, I would appreciate being corrected.

Update

I found that once I put in my custom WireFeed generator and parser, and the custom converter to convert between SyndEntry and Item objects, some of my old feeds began to fail. Specifically, it was not generating the OpenSearch element within the channel, even though they were being set in the code. Prior to the change, the OpenSearch elements were showing up fine. I also noticed that some namespace declarations were not showing up. The upshot was that I ended up changing my rome.properties file to explicitly declare all the properties for RSS 2.0 classes in addition to my custom myrss_2.0 dialect. Here is my complete rome.properties.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
# Copied from the base rome.properties for rss_2.0 since we are now using
# our own dialect of rss_2.0
myrss_2.0.feed.ModuleGenerator.classes=\
  com.sun.syndication.io.impl.DCModuleGenerator \
  com.sun.syndication.io.impl.SyModuleGenerator \
  com.sun.syndication.feed.module.opensearch.impl.OpenSearchModuleGenerator

myrss_2.0.feed.ModuleParser.classes=\
  com.sun.syndication.io.impl.DCModuleParser \
  com.sun.syndication.io.impl.SyModuleParser \
  com.sun.syndication.feed.module.opensearch.impl.OpenSearchModuleParser

myrss_2.0.item.ModuleParser.classes=\
  com.mycompany.myapp.mymodule.MyModuleParser

myrss_2.0.item.ModuleGenerator.classes=\
  com.mycompany.myapp.mymodule.MyModuleGenerator

WireFeedParser.classes=\
  com.sun.syndication.io.impl.RSS090Parser \
  com.sun.syndication.io.impl.RSS091NetscapeParser \
  com.sun.syndication.io.impl.RSS091UserlandParser \
  com.sun.syndication.io.impl.RSS092Parser \
  com.sun.syndication.io.impl.RSS093Parser \
  com.sun.syndication.io.impl.RSS094Parser \
  com.sun.syndication.io.impl.RSS10Parser  \
  com.sun.syndication.io.impl.RSS20wNSParser  \
  com.sun.syndication.io.impl.RSS20Parser  \
  com.sun.syndication.io.impl.Atom10Parser \
  com.sun.syndication.io.impl.Atom03Parser \
  com.mycompany.myapp.mymodule.MyRss20Parser

WireFeedGenerator.classes=\
  com.sun.syndication.io.impl.RSS090Generator \
  com.sun.syndication.io.impl.RSS091NetscapeGenerator \
  com.sun.syndication.io.impl.RSS091UserlandGenerator \
  com.sun.syndication.io.impl.RSS092Generator \
  com.sun.syndication.io.impl.RSS093Generator \
  com.sun.syndication.io.impl.RSS094Generator \
  com.sun.syndication.io.impl.RSS10Generator  \
  com.sun.syndication.io.impl.RSS20Generator  \
  com.sun.syndication.io.impl.Atom10Generator \
  com.sun.syndication.io.impl.Atom03Generator \
  com.mycompany.myapp.mymodule.MyRss20Generator

Converter.classes=\
  com.sun.syndication.feed.synd.impl.ConverterForAtom10 \
  com.sun.syndication.feed.synd.impl.ConverterForAtom03 \
  com.sun.syndication.feed.synd.impl.ConverterForRSS090 \
  com.sun.syndication.feed.synd.impl.ConverterForRSS091Netscape \
  com.sun.syndication.feed.synd.impl.ConverterForRSS091Userland \
  com.sun.syndication.feed.synd.impl.ConverterForRSS092 \
  com.sun.syndication.feed.synd.impl.ConverterForRSS093 \
  com.sun.syndication.feed.synd.impl.ConverterForRSS094 \
  com.sun.syndication.feed.synd.impl.ConverterForRSS10  \
  com.sun.syndication.feed.synd.impl.ConverterForRSS20 \
  com.mycompany.myapp.mymodule.ConverterForMyRss20

ROME uses a hierarchy of rome.properties files to configure itself, which works fine when we are adding modules as I did in my last post, but breaks down when we are overriding. Its like overriding a superclass method without calling super.method(), I guess. So we basically have to do the equivalent of the super method call by copying the classes for RSS 2.0 for the overriden properties.

Posted by Sujit Pal at 10:34 PM 0 comments

Labels: , , , ,

Share/Save: Digg Technorati Delicious StumbleUpon Reddit BlinkList Furl Mixx Facebook Google Bookmark Yahoo

Saturday, October 13, 2007

Custom Modules with ROME

I have been looking at ROME lately. ROME is a very popular open-source Java based RSS/Atom Feed parsing and generation library, originally developed by a group of Sun developers. I originally looked at ROME as a way to parse external feeds. Although the feeds were either in RSS or Atom, there are various versions of both RSS and Atom, which are quite different from each other in subtle ways. ROME abstracts away all the differences, allowing you to treat them as high level Java objects. ROME uses JDOM, my favorite XML parsing library, to do the parsing and building of XML behind the scenes.

My first application using ROME was to parse and aggregate a bunch of external feeds, and was ridiculously simple, about half a page of Java code. As Mark Woodman says in his article "Rome in a Day: Parse and Publish Feeds in Java" on XML.com, the sheer variety of RSS and Atom flavors are enough to make a grown programmer cry. However, the way in which ROME abstracts away all these variations and the simplicity of my resulting code almost made me cry... tears of joy.

However, given that most of the "smarts" of the application would be in the selection of the feeds themselves, and since that required domain expertise I did not have, I decided to put that project aside for a while and explore the other part of ROME, trying to use it to build a feed instead. In any case, in retrospect, I would probably be looking at using the rome-fetcher subproject instead to build the aggregator, since that already provides code to build "well-behaved" feed fetchers.

The feed I choose to rebuild was an existing RSS 2.0 feed. It was generated using JSP XML templates powered by Java services. As such, there were many custom extensions built in, which were not part of any of the standard modules that ROME uses. So I basically had to build a custom extension module for ROME to support the custom tags that this feed used. This post describes the (pretty simple) process.

I could not find any place where this process was described in sufficient newbie detail for me, so after quickly looking through the RSS/Atom/ROME books on Amazon, I settled on "RSS and Atom in Action" (RAIA) by Dave Johnson. One reason for my choice was that it was published by Manning and they provide downloadable PDF versions free with their printed book, so I got my book about 15 minutes after I ordered from my home computer. As it turned out, I was done reading the PDF version by the time the print edition arrived about 5 days later. The print version sits unopened on my desk for now, but I am sure I will need it some day.

But enough idle chatter. Lets get right down to building a custom module that supports three of my custom tags, called my:tag, my:tagDate and my:isTagged respectively. The first is a String, the second a Date and the third a Boolean.

Each module has four components that needs to be built and hooked up with ROME. An interface that describes the URI for the namespace in which the custom tags will live and the getters and setters for each custom tag supported, an implementation of that interface, a parser and a generator. They are shown below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
// MyModule.java
package com.mycompany.feeds.modules.mymodule;

import java.util.Date;

import com.sun.syndication.feed.module.Module;

public interface MyModule extends Module {
  
  public static final String URI = "http://www.my.com/spec";

  public String getTag();
  public void setTag(String tag);
  public Date getTagDate();
  public void setTagDate(Date tagDate);
  public Boolean getIsTagged();
  public void setIsTagged(Boolean isTagged);
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
// MyModuleImpl.java
package com.mycompany.feeds.modules.mymodule;

import java.util.Date;

import com.sun.syndication.feed.module.ModuleImpl;

public class MyModuleImpl extends ModuleImpl implements MyModule {

  private static final long serialVersionUID = -8275118704842545845L;

  private Boolean isTagged;
  private Date tagDate;
  private String tag;
  
  // boilerplate code. Eclipse will generate all but the constructor but
  // will keep reporting an error until you do it.
  public MyModuleImpl() {
    super(MyModule.class, MyModule.URI);
  }

  public void copyFrom(Object obj) {
    MyModule module = (MyModule) obj;
    setTag(module.getTag());
    setTagDate(module.getTagDate());
    setIsTagged(module.getIsTagged());
  }

  public Class getInterface() {
    return MyModule.class;
  }

  // getter and setter impls for MyModule interface
  public Boolean getIsTagged() {
    return isTagged;
  }

  public String getTag() {
    return tag;
  }

  public Date getTagDate() {
    return tagDate;
  }

  public void setIsTagged(Boolean isTagged) {
    this.isTagged = isTagged;
  }

  public void setTag(String tag) {
    this.tag = tag;
  }

  public void setTagDate(Date tagDate) {
    this.tagDate = tagDate;
  }
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
// MyModuleGenerator.java
package com.mycompany.feeds.modules.mymodule;

import java.text.SimpleDateFormat;
import java.util.Collections;
import java.util.HashSet;
import java.util.Set;

import org.jdom.Element;
import org.jdom.Namespace;

import com.sun.syndication.feed.module.Module;
import com.sun.syndication.io.ModuleGenerator;

public class MyModuleGenerator implements ModuleGenerator {

  // boilerplate code
  private static final Namespace NAMESPACE = Namespace.getNamespace("my", MyModule.URI);
  private static final Set NAMESPACES;
  static {
    Set<Namespace> namespaces = new HashSet<Namespace>();
    namespaces.add(NAMESPACE);
    NAMESPACES = Collections.unmodifiableSet(namespaces);
  }

  public String getNamespaceUri() {
    return MyModule.URI;
  }

  public Set getNamespaces() {
    return NAMESPACES;
  }

  // Implements the module generation logic
  private final SimpleDateFormat dateFormat = new SimpleDateFormat("MM/dd/yyyy");

  public void generate(Module module, Element element) {
    MyModule myModule = (MyModule) module;
    if (myModule.getTag() != null) {
      Element myElement = new Element("tag", NAMESPACE);
      myElement.setText(myModule.getTag());
      element.addContent(myElement);
    }
    if (myModule.getTagDate() != null) {
      Element myElement = new Element("tagDate", NAMESPACE);
      myElement.setText(dateFormat.format(myModule.getTagDate()));
      element.addContent(myElement);
    }
    if (myModule.getIsTagged() != null) {
      Element myElement = new Element("isTagged", NAMESPACE);
      myElement.setText(String.valueOf(myModule.getIsTagged()));
      element.addContent(myElement);
    }
  }
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
// MyModuleParser.java
package com.mycompany.feeds.modules.mymodule;

import java.text.ParseException;
import java.text.SimpleDateFormat;

import org.jdom.Element;
import org.jdom.Namespace;

import com.sun.syndication.feed.module.Module;
import com.sun.syndication.io.ModuleParser;

public class MyModuleParser implements ModuleParser {

  // boilerplate
  public String getNamespaceUri() {
    return MyModule.URI;
  }

  // implements the parsing for MyModule
  private final SimpleDateFormat dateFormat = new SimpleDateFormat("MM/dd/yyyy");

  public Module parse(Element element) {
    Namespace myNamespace = Namespace.getNamespace(MyModule.URI);
    MyModule myModule = new MyModuleImpl();
    if (element.getNamespace().equals(myNamespace)) {
      if (element.getName().equals("tag")) {
        myModule.setTag(element.getTextTrim());
      }
      if (element.getName().equals("tagDate")) {
        try {
          myModule.setTagDate(dateFormat.parse(element.getTextTrim()));
        } catch (ParseException e) {
          // don't set it if bad date format
        }
      }
      if (element.getName().equals("isTagged")) {
        myModule.setIsTagged(Boolean.valueOf(element.getTextTrim()));
      }
    }
    return myModule;
  }
}

To let ROME know that these modules should be used, we need to create a rome.properties file in our classpath. The ROME JAR file already has rome.properties files within it that controls its default configuration, and it will read our rome.properties in addition to its own configuration. Our three tags are all item level tags, so we will need to only map the parsers to the item level. The RAIA book shows you how to map feed level modules as well, but the process is quite similar. Here is my rome.properties file (it would be located in src/main/resources in a Maven2 project). I am building an RSS 2.0 feed, so I map it to that dialect here.

1
2
3
4
5
6
# rome.properties
rss_2.0.item.ModuleParser.classes=\
com.mycompany.feeds.modules.mymodule.MyModuleParser

rss_2.0.item.ModuleGenerator.classes=\
com.mycompany.feeds.modules.mymodule.MyModuleGenerator

Finally, here is how I would call it from within my code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
  @Test
  public void testFeedWithMyModule() throws Exception {
    // create the feed object
    SyndFeed feed = new SyndFeedImpl();
    feed.setFeedType("rss_2.0");
    feed.setTitle("My test feed");
    ...
    // add the MyModule namespace to the feed
    feed.getModules().add(new MyModuleImpl());

    // create the item
    SyndEntry entry = new SyndEntryImpl();
    ...
    // create the module, populate and add to the item
    MyModule myModule = new MyModuleImpl();
    myModule.setTag("tagValue");
    myModule.setTagDate(new Date());
    myModule.setIsTagged(true);
    entry.getModules().add(myModule);
    ...
    // add entry(s) to the feed
    feed.getEntries().add(entry);
    // print it out
    WireFeedOutput output = new WireFeedOutput();
    WireFeed wireFeed = feed.createWireFeed("rss_2.0");
    log.debug(output.outputString(wireFeed));
  }

As you can see, adding custom modules to ROME is a bit involved, but its really quite simple. Before I started on these two projects, I did not know much about all the various flavors of ROME and about these custom extensions. In fact, this is the first time I have used Namespaces in JDOM. However, Dave Johnson's book provides a lot of background information and a lot of nice examples in Java and C#. I would highly recommend it to anyone who needs to get up to speed quickly with ROME and RSS/Atom. Another very informative article is the article "ROME: Escape syndication hell, a Java developer's perspective" written by two of the original developers of ROME.

Posted by Sujit Pal at 9:40 AM 35 comments

Labels: , , , , , ,

Share/Save: Digg Technorati Delicious StumbleUpon Reddit BlinkList Furl Mixx Facebook Google Bookmark Yahoo

Subscribe to: Posts (Atom)