Once you realise the zope.testbrowser.browser.Browser.open accepts a urllib2.Request object as a parameter as well as a string then it all falls into place.  I could only get this working from a doc test:

     >>> from Products.Five.testbrowser import Browser
     >>> browser = Browser()
     >>> from urllib2 import Request
     >>> class HeadRequest(Request):
     ...     def get_method(self):
     ...         return "HEAD"
     >>> head_request = HeadRequest(url_of_object_to_test)
     >>> browser.open(head_request)
     >>> browser.headers['content-type']
     'image/jpeg'

Getting my image to support HEAD requests took a bit of work but that’s a different post…

Please hold as we route your chat to an Adobe Representative.

Welcome to Adobe.com! My name is Victor. May I assist you with your selection today?

Victor: Hi, How are you doing today?

Anthony: Great. Just trying to buy Adobe Fireworks

Anthony: I’m based in the United Kingdom

Victor: That’s good to know.

Anthony: There’s no option for that in the “country” drop down

Victor: I’ll help you with that.

Victor: What do you think the most convenient way of getting the software, downloading or getting CD shipped to you?

Anthony: downloading

Victor: Alright.

Anthony: although if it can be shipped quickly that would be OK

Victor: Let me provide you the direct link to place the order. Okay.

Anthony: ok

Victor: Click here to buy Adobe Fireworks CS5

Victor: Please let me know, if you get any error in the order process.

Anthony: That’s £100 more expensive!

Anthony: it’s $299 on the other page

Anthony: = £186

Victor: In Adobe UK store, Adobe Fireworks CS5 is priced at £285.60.

Anthony: how do you explain the price difference?

Anthony: even if you add VAT it’s still £50 more

Anthony: i’m downloading it

Anthony: from the same domain

Victor: We understand your concern. We want to make sure that we are giving our Customers the best products and at the right price.

Victor: Just so you know, there are many factors that decide pricing like ‘the cost of doing business in each country’ and the ‘difference in the perceived value of the product’.

Victor: However, we’re sure that the Adobe products add value by helping you get the desired output at ease, efficiency and speed.

Anthony: for me the right price is not £285

Victor: I understand price is your concern.

Anthony: $299 plus VAT sounds like the right price to me

Anthony: £224

Victor: just to make sure, do you own US credit card with the US billing address?

Anthony: no. i’m based in the UK

Anthony: i have a UK address and credit card

Victor: In that case you need to buy the software from UK store online.

Anthony: I thinking I don’t need it that much

Anthony: thanks for your help

Victor: You’re Welcome.

Anthony: I appreciate your time but I feel like I’m being ripped of

Anthony: off

Anthony: so won’t be purchasing the product

Victor: Do you’ve any further questions for me?

Anthony: no thank you victor

Victor: It was nice chatting with you today.  :)

Victor: Thank you for visiting Adobe.com. Have a great day!

Thank you for chatting with us today.

There are plans to add support from Plone 4.3 onwards but here’s an interim solution I came up with so that I could merge the CSS and JavaScript of my theme right now.  The approach I took is to register my theme’s CSS and JavaScript in the registry with conditional logic based on attributes of the request, use one of the existing custom views for an object to set that attribute on the request and then call templates within Products.ResourceRegistries.

Background to the site

The site I’m theming has two URLs: one for anonymous, public access; one for the editors.  The editors get the standard Plone sunburst theme with a preview pane whilst the public URL doesn’t include any CSS or JavaScript and little HTML from the default Plone UI.

The public view is built up of a number of custom views for an object in Plone all pulled together using rules.xml.

Create the custom view

Register a new view in configure.zcml

    <browser:page
        for="*"
        name="standard-page-elements"
        template="standard_page_elements.pt"
        permission="zope2.View"
        />

Create the custom view class

from Products.Five.browser.pagetemplatefile import ViewPageTemplateFile
from Products.ResourceRegistries import browser
from os.path import dirname
from sys import modules

class StandardPageElementsView(BaseThemeView):
    styles_viewlet = ViewPageTemplateFile('styles.pt',
                                  _prefix = dirname(modules['Products.ResourceRegistries.browser'].__file__))
    scripts_viewlet = ViewPageTemplateFile('scripts.pt',
                                  _prefix = dirname(modules['Products.ResourceRegistries.browser'].__file__))
    def mark_request(self):
        self.request.other['plone.app.theming.name'] = 'my.theme'

    def styles(self):
        self.mark_request()
        return self.styles_viewlet(self)

    def scripts(self):
        self.mark_request()
        return self.scripts_viewlet(self)

Next create the template for the view

<html xmlns="http://www.w3.org/1999/xhtml"
    xmlns:tal="http://xml.zope.org/namespaces/tal"
<head>
    <tal:styles tal:replace="structure view/styles" />
    <tal:styles tal:replace="structure view/scripts" />
</head>
<body></body>
</html>

Register the theme’s resources

In cssregistry.xml

<stylesheet title="" applyPrefix="True" authenticated="False"
    cacheable="True" compression="safe" conditionalcomment="" cookable="True"
    enabled="on"
    expression="python:portal.REQUEST.other.has_key('plone.app.theming.name')"
    id="++theme++my.them/css/main.css" media="screen" rel="stylesheet"
    rendering="link"/>
<stylesheet title="" applyPrefix="True" authenticated="False"
    cacheable="True" compression="safe" conditionalcomment="" cookable="True"
    enabled="on"
    expression="python:portal.REQUEST.other.has_key('plone.app.theming.name')"
    id="++theme++my.theme/css/other.css" media="screen" rel="stylesheet"
    rendering="link"
    insert-after="++theme++my.theme/css/main.css"/>

And do something similar for your scripts in jsregistry.xml

Switch off default Plone CSS and JavaScript

This is a complete pain (roll on Plone 4.3!). You need to disable each resource by hand. I wrote a test for this:

    def test_css_registry(self):
        css_registry = getToolByName(self.portal, 'portal_css')

        theme_css_ids = ('++theme++my.theme/css/main.css',
                         '++theme++my.theme/css/other.css')

        for css in (css for css in css_registry.getResourcesDict().values() if css.getId() not in theme_css_ids):
            self.assertTrue(css.getExpression().startswith(
                               "python:(not portal.REQUEST.other.has_key('plone.app.theming.name'))"),
                            '%s - %s' % (css.getId(), css.getExpression()))

And then in cssregistry.xml you need a line for each Plone css file like the following:

<stylesheet expression="python:(not portal.REQUEST.other.has_key('plone.app.theming.name'))" id="member.css"/>

And then do something similar for jsregistery.xml

Include via rules.xml

You need to block stylesheets (including ones in conditional comments) and scripts from the main view and then pull in ones from your custom view:

    <drop theme="/html/head/link[@rel = 'stylesheet']" />
    <drop theme="/html/head/comment()" />
    <drop theme="/html/head/script" />
    <append theme="/html/head" content="/html/head/*" href="@@standard-page-elements" />

Different resources for different pages

I have the requirement to serve different CSS and scripts for different pages.  I’ve extended the approach above to register different “standard-page-elements” view classes for different objects which set additional attributes on the request.

Roll on Plone 4.3

Work is in progress for Products.ResourceRegistries to look up resources based on an adapter and plone.app.theming to register adapters that provide such resources from link and script tags in the html head element of the theme.  If this can be achieved it would obviously be a big win although it sounds tricky for some aspects of existing resource registry functionality like conditional css for IE and specifying which scripts can be compressed safely.

Here’s the code I used.  A few things that I learnt along the way:

• contextual portlets are managed by Zope local utilities
• you have a IPortletManager local utility for each portlet area defined.  Plone comes with two of these ‘plone.leftcolumn’ and ‘plone.rightcolumn’ but you can add more
• An IPortletAssignmentMapping adaptor adapts your context and its portlet manager to provide storage for your portlet assignments for that context
• A ILocalPortletAssignmentManager adaptor adapts your context and its portlet manager to provide storage for your blocking settings for that context

from logging import getLogger
from plone.portlets.interfaces import ILocalPortletAssignable
from plone.portlets.interfaces import ILocalPortletAssignmentManager
from plone.portlets.interfaces import IPortletAssignmentMapping
from plone.portlets.interfaces import IPortletManager
from plone.portlets.constants import CONTEXT_CATEGORY, GROUP_CATEGORY, CONTENT_TYPE_CATEGORY

from zope.interface import alsoProvides
from zope.component import getMultiAdapter
from zope.component import getUtility
from zope.component import getUtilitiesFor
from zope.component import queryUtility
from zope.component import provideUtility

logger = getLogger('scarba05.utils')

def copy_portlet_assignments_and_settings(src, target):
    """Copy portlet assignments from src to target"""
    if not ILocalPortletAssignable.providedBy(src):
        alsoProvides(src, ILocalPortletAssignable)

    for manager_name, src_manager in getUtilitiesFor(IPortletManager, context=src):
        src_manager_assignments = getMultiAdapter((src, src_manager), IPortletAssignmentMapping)
        target_manager = queryUtility(IPortletManager, name=manager_name, context=target)
        if target_manager is None:
            logger.warning('New folder %s does not have portlet manager %s' %
                           (target.getId(), target_manager))
        else:
            target_manager_assignments = getMultiAdapter((target, target_manager),
                                                IPortletAssignmentMapping)
            for id, assignment in src_manager_assignments.items():
                target_manager_assignments[id] = assignment
            src_assignment_manager = getMultiAdapter((src, src_manager),
                                                 ILocalPortletAssignmentManager)
            target_assignment_manager = getMultiAdapter((target, target_manager),
                                                 ILocalPortletAssignmentManager)
            for category in (CONTEXT_CATEGORY, GROUP_CATEGORY, CONTENT_TYPE_CATEGORY):
                target_assignment_manager.setBlacklistStatus(category,
                                      src_assignment_manager.getBlacklistStatus(category))

I ended up tweaking this code a little for the actual migration step as in Lineage 0.1 Child Folders didn’t inherit their parents portlets, no matter what the parent portlet block settings were.

Following a suggestion on the Plone forum To get round this we’re going to try the following:

• Create a BrowserView with a custom traverser that renders the image
• Override the news item template to use the url expected by the new browser view

I’ve managed to get this working quite neatly:

from mimetypes import guess_type

from plone.app.layout.viewlets.common import BrowserView

from zope.publisher.interfaces import NotFound

class ImageView(BrowserView):
"""A view of a new item's image that supports file extensions"""

    def __getitem__(self, partid):
        (type, encoding) = guess_type(partid)
        image_type = self.context.getField('image').getContentType(self.context)

        if not type == image_type:
            raise NotFound(self.context, partid, self.request)

        return self.context.__bobo_traverse__(self.request, partid[:partid.find('.')])

This simple bit of code will work for all image scales and validates that the extension used matches the content type of the image.

Integration testing interceptors and @PathVariable

The most noticeable problem with my original approach is that it doesn’t test any interceptors that are configured and this is something you probably want to include in your integration tests.  One unexpected (for me at least) side effect of this is that methods that include @PathVariable annotations on their parameters don’t work either.  You get the following exception:

org.springframework.web.bind.annotation.support.HandlerMethodInvocationException: Failed to invoke handler method [public org.springframework.web.servlet.ModelAndView test.MyClass.myMethod(test.SomeType)]; nested exception is java.lang.IllegalStateException: Could not find @PathVariable [parameterName] in @RequestMapping

This is because an interceptor is used by Spring to extract path variables from the request, before it hits the controller and processes the corresponding annotated parameters.

Use common handle method in integration tests

Using the same example class before:

@Controller
@RequestMapping("/simple-form")
public class MyController {
    private final static String FORM_VIEW = null;

    @InitBinder
    public void initBinder(WebDataBinder binder) {
        binder.registerCustomEditor(String.class, new StringTrimmerEditor(false));
    }

    @RequestMapping(method = RequestMethod.GET)
    public MyForm newForm() {
        return new MyForm();
    }

    @RequestMapping(method = RequestMethod.POST)
    public String processFormSubmission(@Valid MyForm myForm,
            BindingResult result) {
        if (result.hasErrors()) {
            return FORM_VIEW;
        }
        // process the form
        return "success-view";
    }
}

Here’s my updated implementation of an integration test. In it I define a handle method that is called by each test after it has configured the request to mimic that sent by the browser. This handle method includes logic to execute each of the interceptors configured for that request first, before passing control to the controller. It also makes no assumption about what class the controller is.

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration({"file:web/WEB-INF/application-context.xml",
    "file:web/WEB-INF/dispatcher-servlet.xml"})
public class MyControllerIntegrationTest {

    @Inject
    private ApplicationContext applicationContext;

    private MockHttpServletRequest request;
    private MockHttpServletResponse response;
    private HandlerAdapter handlerAdapter;

    @Before
    public void setUp() throws Exception {
        this.request = new MockHttpServletRequest();
        this.response = new MockHttpServletResponse();

        this.handlerAdapter = applicationContext.getBean(HandlerAdapter.class);
    }

    ModelAndView handle(HttpServletRequest request, HttpServletResponse response)
            throws Exception {
        final HandlerMapping handlerMapping = applicationContext.getBean(HandlerMapping.class);
        final HandlerExecutionChain handler = handlerMapping.getHandler(request);
        assertNotNull("No handler found for request, check you request mapping", handler);

        final Object controller = handler.getHandler();
        // if you want to override any injected attributes do it here

        final HandlerInterceptor[] interceptors =
            handlerMapping.getHandler(request).getInterceptors();
        for (HandlerInterceptor interceptor : interceptors) {
            final boolean carryOn = interceptor.preHandle(request, response, controller);
            if (!carryOn) {
                return null;
            }
        }

        final ModelAndView mav = handlerAdapter.handle(request, response, controller);
        return mav;
    }

    @Test
    public void testNewForm() throws Exception {
        request.setMethod("GET");
        request.setRequestURI("/simple-form");

        final ModelAndView mav = handle(request, response);
        // make assertions on the ModelAndView here
    }

    @Test
    public void testProcessFormSubmission() throws Exception {
        request.setMethod("POST");
        request.setRequestURI("/simple-form");
        // set some request parameters for binding

        final ModelAndView mav = handle(request, response);
        // make assertions on the ModelAndView here plus any side effects
    }

Spring 3.0 includes some great features and one of my favourites is its integration with the JSR 303 bean validation API.  It uses the method-level validation proposal from the specification.  Note the word proposal: Method-level @Valid annotations are mentioned in an appendix rather than in the official part of the specification.  Basically you annotate the form bean parameter with @Valid and include a BindingResult paramater on your handler method and Spring validates the bean after binding, populating the BindingResult instance with any errors that occur.

    @RequestMapping(method = RequestMethod.POST)
    public String saveForm(@Valid MyForm myForm, BindingResult result) {
        if (result.hasErrors()) {
            // usually return the user back to the form
        }
        // process the form
    }

You obviously also need to annotate you form bean class with JSR 303 annotations.

What about multi page (wizard style) forms?

All this works great with one page forms but what happens when your form spans more than one page?  Validation and other Spring MVC 3.0 features work great for single page forms but there’s not much out of the box support wizard style forms yet.  These types of forms are very common for checkout style processes and forms where more complex interaction is required, for example, where a user is asked different sets of questions based on their earlier answers.

I’ve been working on an online donation, multi page, wizard and wanted to use the new Spring 3.0 MVC style rather than the old Spring 2.0 AbstractWizardFormController.  I’ll probably post more about my solution later but it involved using a HTTP request parameter to specify the page, a handler method per page in my controller and the params parameter of the @RequestMapping annotation to help Spring select the correct method.

    @RequestMapping(method = RequestMethod.POST, params = "page=1")
    public String handlePage1(MyForm myForm, BindingResult result) {
        // what about validation?
    }

Spring 3.0 integration with JSR 303 validation doesn’t work with multi page forms

My requirements (like most multi page forms) is to validate the data submitted by the user on the submission of the page where they entered it.  If I annotate my form bean method parameter with @Valid then it validates the whole bean and there’s currently no way round this.  This won’t work: mandatory fields on page 2 of the form won’t be filled in when the user submits page 1.

JSR 303 validation does have the concept of groups.  You can specify which groups each validation annotation applies to and map these groups to your various pages:

@GroupSequence({MyForm.class, MyForm.Page1.class, MyForm.Page2.class})
public class MyForm {
    public static interface Page1 {}
    public static interface Page2 {}

    @NotNull(groups = {Page1.class})
    private String page1MandatoryField;

    @NotNull(groups = {Page2.class})
    private String page2MandatoryField;

    ...
}

For real applications you’re better naming your groups based on what’s on the page, rather than its number.

JSR 303 then provides mechanisms for you to validate only a specify group or groups on a bean (great!) but there’s no way of integrating this with Spring MVC binding step (not great!).

Extend JSR 303 @Valid annotation to specify which group to validate

I can work round this pain by manually calling the group validation step in my handler methods.  This is exactly the kind of boiler plate repetitive code Spring is usually so good at doing for you. What I’d really like is to be able specify which groups I want Spring to validate using an annotation.  The most elegant solution I believe would be to extend the @Valid annotation to include a groups parameter:

    @RequestMapping(method = RequestMethod.POST, params = "page=1")
    public String handlePage1(@Valid(groups = {Page1.class}) MyForm myForm,
            BindingResult result) {
        if (result.hasErrors()) {
            return "page1";
        }
        return "page2";
}

The @Valid annotation primary purpose is for cascading validation down the object graph. I believe a groups parameter would also be useful for this purpose but don’t have a concrete example of this.

Obviously extending the specification and implementation the solution is out of Spring’s hands so in the interim maybe Spring could provide a @ValidationGroups parameter to do the job for you:

    @RequestMapping(method = RequestMethod.POST, params = "page=1")
    public String handlePage1(@Valid @ValidationGroups(Page1.class) MyForm myForm,
            BindingResult result) {
        if (result.hasErrors()) {
            return "page1";
        }
        return "page2";
}

There’s already a Spring feature request asking for something along these lines.   Please go and vote for it!

    @RequestMapping(method = POST, params = "page=1")
    public String handlePage1(@Valid MyForm form, BindingResult result) {
       if (result.hasErrors()) {
           return "page1";
       }
       return "page2";
    }

We’re using Tomcat 6 as our servlet container and have previously found that to get it to handle non-ASCII characters correctly you need to set the character encoding on the request.  In Spring 3.0 world we’ve been doing this in our @InitBinder annotated method.

    @InitBinder
    public void initBinder(HttpServletRequest request, ServletRequestDataBinder binder) {
        request.setCharacterEncoding("UTF-8");
        // initialize the binder
    }

For some reason this wasn’t working in my new form: special characters submitted in the form weren’t being encoded properly.  This had me baffled for a while until I realized what the culprit was: my use of the params parameter with the @RequestMapping annotation!  After debugging it became apparent that using it causes HTTP request parameters to be parsed before the initBinder method is called and any further calls to request.getParameter() ignore the character encoding specified (possibly because the result is cached by the Tomcat implementation?).  This means I needed to set the character encoding before the HandlerAdaptor starts working on the @RequestMapping annotated methods.

Solution: set the character encoding via an interceptor

I managed to resolve this by creating a Spring interceptor (equivalent of a JEE Filter) to set the character encoding .

package com.example;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.springframework.web.servlet.handler.HandlerInterceptorAdapter;

/**
 * Sets the character encoding for the request.  Defaults to UTF-8.
 */
public class SetCharacterEncodingInterceptor extends HandlerInterceptorAdapter {
    private String characterEncoding = "UTF-8";

    @Override
    public boolean preHandle(HttpServletRequest request, HttpServletResponse response,
            Object handler) throws Exception {
        request.setCharacterEncoding(this.characterEncoding);
        return true;
    }

    public void setCharacterEncoding(String characterEncoding) {
        this.characterEncoding = characterEncoding;
    }
}

and then configuring Spring to use this interceptor for all my controllers:

    <mvc:interceptors>
        <bean class="com.example.SetCharacterEncodingInterceptor"/>
    </mvc:interceptors>

This meant I no longer need to set the encoding in my initBinder methods: removing duplicate code as a bonus:

    @InitBinder
    public void initBinder(HttpServletRequest request, ServletRequestDataBinder binder) {
        request.setCharacterEncoding("UTF-8");
        // initialize the binder
    }

As a very simple example.  Say you want a controller that processes a simple form.

@Controller
@RequestMapping("/simple-form")
public class MyController {
    private final static String FORM_VIEW = null;

    @InitBinder
    public void initBinder(WebDataBinder binder) {
        binder.registerCustomEditor(String.class, new StringTrimmerEditor(false));
    }

    @RequestMapping(method = RequestMethod.GET)
    public MyForm newForm() {
        return new MyForm();
    }

    @RequestMapping(method = RequestMethod.POST)
    public String processFormSubmission(@Valid MyForm myForm,
            BindingResult result) {
        if (result.hasErrors()) {
            return FORM_VIEW;
        }
        // process the form
        return "success-view";
    }
}

This is great because:

• The controller doesn’t extend a Spring class
• You can call your methods that handle requests whatever you like (i.e. a name which best expresses their intention) with the @RequestMapping annotation reinforcing this view
• There is virtually no boiler plate code
• It’s integrated with the JSR 303 validation API
• You decide what parameters go in the controller methods and Spring calls the method passing in the appropriate object

Confession: I haven’t been writing tests for the @RequestMapping, @RequestParam, @PathVariable and other annotations

Throughout the last two months of using the new features I’ve have had a nagging feeling that I haven’t been writing enough JUnit tests.  I’m very keen on writing tests and we use test-driven-development most of the time.  Unit testing the logic inside the controller methods is no issue.  In fact, that’s another great thing about Spring 3 controllers: they’re really easy to unit test.  We also continue to build up our bank of end to end regression tests using Selenium (excellent tool!).

What is missing are tests that ensure I’ve used the correct annotation in the correct place.  In the Spring reference manual tests of this nature are called integration tests and it talks about how you might want to write tests for Hibernate entity annotations and similar.  Looking into this I haven’t been able to find a satisfactory approach for testing the MVC annotations:  I’ve read suggestions to test a method is annotated with a particular annotation or to write functional tests with something like HtmlUnit.  What I want is something similar to the recommendation in the Spring documentation for writing database integration tests.

I’ve also read suggestions that I don’t need to test the annotations because Spring will have tests that already do that.  To me that argument is specious:  I’m not testing that the Spring annotations do what they’re supposed to, I’m testing that I’ve used the annotations correctly.

How to write integration tests for Spring MVC annotations

Here’s my approach to writing integration tests for Spring MVC annotations.  Taking the example above my code that I want to test would be for a HTTP GET:

• results in a null view
• a new command object being placed in the model

and a HTTP POST:

• request parameters are bound to a command object
• the parameter values are trimmed
• the command object is validated and if not valid the errors are included in a binding result in the model and a null view returned

More generally I want to test that:

• my Spring configuration files correctly configure the appropriate HandlerAdaptor

import static org.springframework.test.web.ModelAndViewAssert.*;

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration({"file:web/WEB-INF/application-context.xml",
    "file:web/WEB-INF/dispatcher-servlet.xml"})
public class MyControllerIntegrationTest {

    @Inject
    private ApplicationContext applicationContext;

    private MockHttpServletRequest request;
    private MockHttpServletResponse response;
    private HandlerAdapter handlerAdapter;
    private MyController controller;

    @Before
    public void setUp() {
       request = new MockHttpServletRequest();
       response = new MockHttpServletResponse();
       handlerAdapter = applicationContext.getBean(HandlerAdapter.class);
       // I could get the controller from the context here
       controller = new MyController();
    }

    @Test
    public void testGet() throws Exception {
       request.setMethod("GET");
       final ModelAndView mav = handlerAdapter.handle(request, response, controller);
       assertViewName(mav, null);
       assertAndReturnModelAttributeOfType(mav, "myForm", MyForm.class);
    }

    @Test
    public void testPost() throws Exception {
       request.setMethod("POST");
       request.addParameter("firstName", "  Anthony  ");
       final ModelAndView mav = handlerAdapter.handle(request, response, controller);
       final MyForm myForm = assertAndReturnModelAttributeOfType(mav, "myForm", MyForm.class);
       assertEquals("Anthony", myForm.getFirstName());

       /* if myForm is not valid */
       assertViewName(mav, null);
       final BindingResult errors = assertAndReturnModelAttributeOfType(mav,
               "org.springframework.validation.BindingResult.myForm",
               BindingResult.class);
       assertTrue(errors.hasErrors());
    }
}

It should be fairly easy to extend this to more complex examples testing @RequestParam, @RequestMethod, @PathVariable, @RequestHeader,  @RequestBody.

What’s next

You might notice that I’m not testing the @RequestMapping("/simple-form") class level annotation in this test. I’ve not figured out what the best way to do this is just yet.

How do I know if Tomcat is using APR or not?

If Tomcat is not using APR then you’ll see the following log line (or something similar) when you start it.

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /home/tomcat6live/java.home/jre/lib/i386/server:/home/tomcat6live/java.home/jre/lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib

If it is using APR then you’ll see something like

INFO: Loaded APR based Apache Tomcat Native library 1.1.19.

If you don’t see either or these lines then either you’ve configured logging above INFO on org.apache.catalina.core.AprLifecycleListener or you haven’t included the APR listener in your server.xml (see below).

Prerequisites

You need the APR and openSSL libraries installed on your server. We’d already installed them on our server so I’m not going to blog about how to install them here.

Install the Tomcat APR native libraries

As root:

cd /usr/local/apache-tomcat-6.0.24/bin
tar xfz tomcat-native.tar.gz
cd tomcat-native-1.1.19-src/jni/native
./configure --with-apr=/usr/local/apr --with-ssl=/usr/local --prefix=/usr/local/apache-tomcat-6.0.24  --with-java-home=/usr/java/jdk1.6.0_17
make
make install

Where you should adjust the ./configure parameters so that they match you APR, SSL, Tomcat and Java installations on your server.

Configure Tomcat to use the APR native libraries

The LD_LIBRARY path variable needs to include the Tomcat lib directory.  I have a setenv file that is referenced by our init.d start-up script so I simply added the following lines to that.

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CATALINA_HOME/lib
export LD_LIBRARY_PATH

Your start-up script needs to do something similar.

Now all you need to do is ensure the APR listener is configured in server.xml.  We have the following (which you’ll need to adjust if you’re using SSL).

<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="off" />

All done!

Problems getting Tomcat to use the APR native libraries

Like I mentioned earlier I encounted a number of issues along the way.

Error initializing endpoint / Invalid Server SSL Protocol

Tomcat started but with lots of errors. This being the first:

SEVERE: Error initializing endpoint
java.lang.Exception: Invalid Server SSL Protocol
        at org.apache.tomcat.jni.SSLContext.make(Native Method)
        at org.apache.tomcat.util.net.AprEndpoint.init(AprEndpoint.java:716)
        at org.apache.coyote.http11.Http11AprProtocol.init(Http11AprProtocol.java:107)
        at org.apache.catalina.connector.Connector.initialize(Connector.java:1007)
        at org.apache.catalina.core.StandardService.initialize(StandardService.java:677)
        at org.apache.catalina.core.StandardServer.initialize(StandardServer.java:795)
        at org.apache.catalina.startup.Catalina.load(Catalina.java:540)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:261)
        at org.apache.catalina.startup.Bootstrap.init(Bootstrap.java:276)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:160)

This was because I had SSLEngine="off" in my APR listener configuration

<Listener className="org.apache.catalina.core.AprLifecycleListener"  SSLEngine="off" />

But had a http connector configured to use SSL

<Connector protocol="HTTP/1.1" ... SSLEnabled="true" ... />

Fix seemed simple – switch the ssl engine on!

<Listener className="org.apache.catalina.core.AprLifecycleListener"   SSLEngine="offon" />

symbol lookup error / undefined symbol: SSL_CTX_set_info_callback

Now Tomcat wouldn’t start at all! This was the error:

jsvc.exec: symbol lookup error: /usr/local/apache-tomcat-6.0.24/lib/libtcnative-1.so.0.1.19: undefined symbol: SSL_CTX_set_info_callback

This was due to the fact I hadn’t used the correct value for the --with-ssl option when I ran ./configure (see above).  I’d used /usr/share/ssl when I should have used /usr/local.  To fix I reran ./configure with the correct values.  You can check the output to make sure it finds a compatible version of ssl.  You then need to remake the libraries.

make clean
make
make install

Error initializing endpoint / No Certificate file specified or invalid file format

Now Tomcat starts but with a number of SEVERE errors.  In particular:

SEVERE: Error initializing endpoint
java.lang.Exception: No Certificate file specified or invalid file format
        at org.apache.tomcat.jni.SSLContext.setCertificate(Native Method)

This was caused by a misconfiguration of our http connector that was using SSL.  It turns out that: The APR connector uses different attributes for SSL keys and certificates. How annoying!  We only need SSL on our local development environments as Apache looks after the SSL handshake on our production servers so I simply switched off the SSL Engine on our APR listener and disabled SSL on the http connector in server.xml. If you need SSL with APR then you can find instructions in the Tomcat 6.0 documentation page on SSL.

<Listener className="org.apache.catalina.core.AprLifecycleListener"   SSLEngine="onoff" />
…
<Connector protocol="HTTP/1.1" ... SSLEnabled="truefalse" ... />

At last everything works!