Resources: The Code for Faceted Search Navigation

In my last post, I discussed the data-capture portion of our new Resources page on our corporate site. Now I’d like to get into the code for the page to show you how things are done.

The main template for the page is pretty much a standard STK template:

Template Definition

There are a couple of interesting things going on in the mainArea and extrasArea, though. Have a look:

Auto-Generated Paragraphs

You can see that, in both sections, we’re adding an automatically-generated paragraph. The main area adds the paragraph that’s responsible for rendering search results, and the extras area automatically includes the search navigation. Because we continue to rely on the normal structure of STK pages, we don’t have to do a lot more to build the page structure. The one remaining important piece we do need to put in place, however, is custom CSS and Javascript for the page template. (We’re using the excellent PrettyPhoto jQuery lightbox script and need to include its supporting files.) Here’s how we include the extra files in the page template:

Including CSS and Javascript files

As you can see, we can easily specify whether or not to use Far Future Caching (which improves caching performance significantly) and what media the CSS should apply to.

Now that our page template is built up, let’s take a look at the navigation paragraph. This paragraph’s job is to present a list of all of the different categories for which we have assets, and to allow site visitors to choose from that list. It’s dynamic, in that if a category is already selected, we only display the categories that have items with both that category and the selected category. Here’s the code from the Freemarker template:

    <ul>
        [#list model.categories as category]
            <li class="category">${category.displayName!category}</li>
            [#if category?children?size > 0]
                <ul>
                    [#list category?children?sort as subCategory]
                        [#if model.isCategoryActive( subCategory )]
                            <li class="selected">
                                ${subCategory} (${model.getCategoryCount( subCategory )})
                                <a href="${model.getClearCategoryLink( subCategory )}">${unselectCharacters}</a>
                            </li>
                        [#else]
                            [#if model.getCategoryCount( subCategory ) > 0]
                                <li>
                                    <a href="${model.getCategoryLink( subCategory )}">[#if subCategory.displayName?has_content]${subCategory.displayName}[#else]${subCategory}[/#if] (${model.getCategoryCount( subCategory )})</a>
                                </li>
                            [/#if]
                        [/#if]
                    [/#list]
                </ul>
            [/#if]
        [/#list]
    </ul>

You should be able to see that the Freemarker template gets a list of categories from the model object and iterates through them. It then iterates through the subcategories, displaying each as a link (if it’s not yet selected), or as plain text with a cancel option (if it’s already selected). But obviously, the model class is doing a fair bit of work under the covers to generate the data that Freemarker uses. Let’s take a look at some of the code there. The first thing that the template relies on the list of categories:

    public List<Content> getCategories() throws AccessDeniedException, PathNotFoundException, RepositoryException {
        if ( this.categories == null ) {
            Content getCategoriesFromNode = content;
            //The model is called form the searchResult paragraph, so the categories are on the auto generated navigation paragraph in extras.
            if(!content.hasNodeData("categories")) {
                //Getting the searchNavigation paragraph's node in extras
                getCategoriesFromNode = getRoot().getContent().getContent("extras").getContent("extras1").getContent("extrasItem0");
            }
            this.categories = CategoryUtil.getCategories( getCategoriesFromNode, "categories" );
        }
        return this.categories;
    }

We use lazy initialization for all of the model’s properties, which keeps us from having to compute things unnecessarily or too often. We then use the CategoryUtil class (which is part of the Categorization module) to get the list of Categories that have been specified in the search configuration dialog and return that list.

Next up, we want to determine if a particular subcategory is active — that is, whether the site visitor has already chosen to search on that category. Here’s the code for that:

public boolean isCategoryActive( Content category ) {
    String existingCategories = FCSSelectorUtil.getSelectorValue( CATEGORY_PARAMETER );
    if ( !StringUtils.isEmpty( existingCategories ) ) {
        List<String> categoriesList = Arrays.asList( StringUtils.split( existingCategories, CATEGORY_DELIMITER ) );
        return categoriesList.contains( getAbbreviatedPath( category ) );
    }
    return false;
}

The FCKSelectorUtil class is what we use to extract search parameters from the current URL string. It’s a bit outside the scope of this article, but we built it to allow us to use Selectors instead of query strings to determine what results to show. (Using Selectors allows us to cache the results, while using query strings turns off caching by default in Magnolia.)

Next up, we need to count the number of results available for a given category. This is pretty database-intensive, as we have to count the number of results for videos, slideshows, DMS documents, and webpages, and then add them all together. Note that we have here used Openmind’s very nice Criteria API to simplify the queries as much as possible:

    public int getCategoryCount( Content category ) {
        Criteria websiteCriteria = getWebsiteCriteria();
        websiteCriteria = addCurrentRestrictions( websiteCriteria );
        websiteCriteria.add( Restrictions.contains( "@categories", category.getUUID() ) );
        AdvancedResult websiteResult = websiteCriteria.execute();

        Criteria dmsCriteria = getDmsCriteria();
        dmsCriteria = addCurrentRestrictions( dmsCriteria );
        dmsCriteria.add( Restrictions.contains( "@categories", category.getUUID() ) );
        AdvancedResult dmsResult = dmsCriteria.execute();

        Criteria videoCriteria = getVideoCriteria();
        videoCriteria = addCurrentRestrictions( videoCriteria );
        videoCriteria.add( Restrictions.contains( "@categories", category.getUUID() ) );
        AdvancedResult videoResult = videoCriteria.execute();

        Criteria slideshowCriteria = getSlideshowCriteria();
        slideshowCriteria = addCurrentRestrictions( slideshowCriteria );
        slideshowCriteria.add( Restrictions.contains( "@categories", category.getUUID() ) );
        AdvancedResult slideshowResult = slideshowCriteria.execute();

        return websiteResult.getTotalSize() + dmsResult.getTotalSize() + videoResult.getTotalSize() + slideshowResult.getTotalSize();
    }

This method relies on a couple of supporting utility methods. One set is responsible for setting up the basics of a certain type of query. For example:

private Criteria getDmsCriteria() {
    Criteria criteria = JCRCriteriaFactory.createCriteria()
    .setWorkspace("dms")
    .setBasePath( getSearchRoot() )
    .add( Restrictions.eq( Criterion.JCR_PRIMARYTYPE, "mgnl:contentNode" ) );
    return criteria;
}

And the addCurrentRestrictions() method modifies the query to only return results from the currently-selected categories:

private Criteria addCurrentRestrictions(Criteria criteria) {
    String categoriesString = FCSSelectorUtil.getSelectorValue( CATEGORY_PARAMETER );
    if ( !StringUtils.isEmpty( categoriesString ) ) {
        HierarchyManager hm = MgnlContext.getHierarchyManager("data");
        for ( String category : StringUtils.split( categoriesString, CATEGORY_DELIMITER ) ) {
            Content categoryContent;
            try {
                categoryContent = hm.getContent( getFullPath( category ) );
                criteria.add( Restrictions.contains( "@categories", categoryContent.getUUID() ) );
            } catch (RepositoryException e) {
                log.warn( "Unable to locate category: " + category );
                criteria.add( Restrictions.contains( "@categories", "NO_MATCH" ) );
            }
        }
    }
    return criteria;
}

There are some more details, but that should give you a basic understanding of how the navigation paragraph works. In the next post, we’ll take a look at the search results paragraph and how it works.

Advertisement

Resources: Capturing Data for Faceted Search in Magnolia

As you Magnolia-watchers have doubtless noticed, there’s a new section on the Magnolia corporate website: Resources, a faceted search directory of media assets that are available through our site. The new page makes it easy to plow through a ton of available content to find what you want to see, and we’re pretty proud of it.

Since there’s some interesting stuff going on under the covers to allow us to aggregate slideshows, DMS documents, webpages, and videos into a common collection of content, I thought it might be worth sharing some of the details here.

We decided to use the Categorization Module to assign the media assets to particular facets. Because the module supports hierarchical categories, it makes it pretty straightforward to organize these how we like. In addition, we can often reuse these assignments just like we would normal categories — to generate category pages, RSS feeds, etc.

Once we’d set up our hierarchy of categories, we had to make it possible to capture all of the content we wanted to include. The Data Module allowed us to create data types for externally hosted videos (blip.tv or youtube) and slideshows (slideshare.net). This is pretty standard stuff, except for the fact that, in order to support multiple categories within the data workspace, we had to tweak the nodetype definitions. (This change is documented on the Categorization page.)

Next up, we want to be able to assign Categories to documents in the DMS. “But wait!” I hear you cry. “The DMS doesn’t support categorization!” With a little bit of Magnolia-Fu, however, it can. The Categorization module makes a tab control available for sharing, so all we have to do is add a reference to that control to our DMS dialog:

Finally, STK-based web pages already support Categorization, so we didn’t have to do anything there — they’re already good to go.

So with these few additions, we’re ready to start capturing data on media assets to show in our Resources section. Watch for my next blog post where I’ll get into the code behind the scenes.

Three Ring Circus: Making Content, Code, and Configuration Work Together

Since I started working at Magnolia, I’ve been asked one question over and over, more than any other:

“Can I please have some money?” This is because I’m a father.

The next most common question I’ve been asked, however, is this:

“I have content authors writing content on my production system. I have developers creating Java and Freemarker code on their own machines. I have system administrators setting up and tweaking configurations on our test server. How can I possibly get all the changes coming from all different directions where they need to be?”

This is a challenging problem, but — unlike convincing a teenager to wash the dishes without grumbling — not an insurmountable one. There are several approaches one can take; here’s one that we eventually settled on at Texas State University during my tenure there, and which worked quite well for us.

To start with, we need to decide what the canonical location for each of these sorts of information should be:

1. Content: Since the content that site editors enter is the most up-to-date at any time, the production Author or Public instances of Magnolia are the places we’ll look for this.
2. Code: Custom functionality in Magnolia is wrapped up into Modules, which are then compiled by Maven into JARs. Because these are built out of standard Java code, they’re usually managed with standard source control tools like Subversion, Git, or Mercurial, and the repository is the place to go to find the “real” version of the code.
3. Configuration: Each system running Magnolia, be it a production instance, a testing instance, or a development instance, will likely need some different configuration tweaks. (They might use different SMTP configurations, for example, or a different persistence manager.) In this case, where you store these data is less clear-cut, but since it’s not necessarily shared among instances, it’s reasonable to store it on each box in some place outside of the web app (so that it’s not erased when Magnolia is redeployed).

Now that we’ve identified the various bits that we need to pull together and where they live, let’s look at how we might assemble things on the various instances of Magnolia. Here’s a pretty typical deployment:

First off, let’s discuss the development instances. These are going to be the most changeable, as developers configure and reconfigure things to handle the need of the moment. (You know how developers are!) Code in this case comes from Subversion (or whatever SCM system your team uses), combined with whatever changes the developer is working on. Content can either be the sample content that’s distributed with Magnolia, or can be bootstrapped or imported from content exported from a production instance. Configuration may be left with default values, or can be overridden by the developer in a variety of ways.

Helpful hint: When developing a module, sometimes you may want to have the module’s configuration rebootstrapped without having to delete all of the content and start the instance from scratch. You can do this by deleting the module’s configuration node at Configuration -> modules -> [module name] and then bouncing Magnolia. The system will see the module’s JAR, but will think it has never been installed, and will dutifully go through the bootstrap process again.

Next, the test instance. Here we want to have the latest code the developers have committed combined with up-to-date content from content authors. To achieve this, our Continuous Integration system watches the source code repository, and whenever new versions of our module code are checked in, it rebuilds the WAR, deletes the content repositories on the test system, and deploys the new WAR there. (You do have a Continuous Integration system, don’t you? If not, set one up! It’s one of the best things you can do for your team’s development workflow. We like Hudson, but there are lots of great ones out there.)

But where does the content come from? We’ve set up an automatic process to fetch an XML backup of the content in our production instance each night and to store it in a known location. (This can be as simple as having cron call wget each night with the path to Magnolia’s export servlet.) By setting the magnolia.bootstrap.dir property in magnolia.properties to the directory where that content export is located, our instance will bootstrap this exported content along with all of its other initial setup tasks. We recommend a path outside of the web application directory in this case, so that any content you place here will survive redeployments.

Finally, let’s consider configuration. For things like SMTP that are set up in the JCR, it’s pretty simple to export those configurations to XML and put those in the bootstrap directory along with the content. Note, however, that this works best if you get down to the individual items that you want to have imported. For example, if you have a particular set of users you want bootstrapped, it’s best to export each of those users individually. And if you want particular settings to be used everywhere, export those settings — not the whole Config tree, since there are other settings in that tree that won’t be the same across all your instances.

But what about items — like which persistence manager to use — that have to be specified in magnolia.properties? For that, you can actually specify a different properties file for each of your environments, and have it applied automatically. (It’s like magic!)

The trick is to use a properties file that corresponds to one of these naming conventions:


WEB-INF/config/magnolia.properties
WEB-INF/config/default/magnolia.properties
WEB-INF/config/${webapp}/magnolia.properties
WEB-INF/config/${servername}/magnolia.properties
WEB-INF/config/${servername}/${webapp}/magnolia.properties


When it starts up, Magnolia marches down this list looking for properties files that match. ${servername} is the host name of the machine that Magnolia is running on, while ${webapp} reflects the application context in which it is deployed. Thus, if you’re running an instance called “magnoliaAuthor” on a box named “production”, Magnolia will check the following paths:


WEB-INF/config/magnolia.properties
WEB-INF/config/default/magnolia.properties
WEB-INF/config/magnoliaAuthor/magnolia.properties
WEB-INF/config/production/magnolia.properties
WEB-INF/config/production/magnoliaAuthor/magnolia.properties


Even better, properties defined in the more-specific paths will overwrite properties defined in the less-specific paths. Thus, you could define your persistence manager as Derby in WEB-INF/config/default/magnolia.properties and as Oracle in WEB-INF/config/production/magnolia.properties. The result would be that Derby would be used everywhere except in production, where Oracle would be used instead.

(For more information on this mechanism, see the article on Using a Single WAR File with Multiple Configurations on our Documentation site.)

Finally, let’s take a look at our production system. This is the canonical location of our content, so we’re always up to date here. And since this is a production system, and you’re an extremely clever person, you have your content in a RDBMS, which ensures that the content is safely retained when you deploy new versions of Magnolia or your module.

The Code for your module should also make its way to the production system through your continuous integration environment. (This ensures that the version you’re testing is the same version that gets deployed — always a good thing.) You can entirely automate the process if you’re a trusting sort, or simply have the CI environment dump your module’s JAR file somewhere for you to copy over yourself if you’re of a more skeptical nature.

“But I’ve added a new paragraph in my module! How does it get set up?” Well, I’m glad you asked, Hypothetical Reader! Back around Magnolia 3.5, Magnolia introduced the Version Handler, a mechanism for automating version upgrade tasks. This has been a tremendous boon for deployment, as you might imagine.

Before, we maintained a manual list of things that needed to be changed when a module was updated, and had to go in and perform those tasks manually, usually at three in the morning, under time pressure, and while possibly hallucinating due to lack of sleep. (We saw a porcupine in the parking garage after one particularly grueling deployment. Surprisingly, it turned out to be real.) With this reproducible mechanism, we can do dry runs of our upgrades first thing in the morning (around 11:00am — we’re programmers, after all) while we’re still fresh and have a high degree of confidence that we’ll have flushed out any problems before we try to upgrade our production system.

The best practice here is to tag your code in your SCM each time you deploy to production. Then, when you want to try out your new code, you can build the previous deployed version, bootstrap it, and then try running your upgrade against that instance on a machine where it won’t get you fired when things go horribly wrong.

Lastly, configuration which is stored in the JCR will persist across deployments, just as content does. Anything that needs to be set in magnolia.properties can be set up using the same per-host, per-webapp configuration mechanism that we discussed for the test environment.

And that’s it! A soup-to-nuts setup that lets everyone do their jobs, and gets your information where it needs to go. Of course, you may want to tweak this at times for various reasons — for example, if you’re developing a new feature and want to add content, you might make your test server the canonical location for content temporarily — but this should give you a good idea of where to start getting all of these pieces working together.

If you have any questions about this configuration, don’t hesitate to leave me a note here, email me at sean.mcmains@magnolia-cms.com, or to send a carrier pigeon. (Warn me first, though, so I can lock up the cats.)

Top 5 Reasons I’m Looking Forward to Magnolia 5

I’ve been working with Magnolia for a number of years now, and have seen it evolve from a great idea without much meat on its bones to a rich, flexible CMS that can be used for a huge variety of tasks. Even in its current mature state, however, there are areas that can still be even better. Here are some of the things that are planned for the next major revision which I’m most looking forward to:

1. Touchscreen support: the iPad and its imitators have ushered in a surprisingly viable alternative to notebook computing. In my experience, I can do about 70% of what I would do on a laptop with an iPad. Web sites and apps generally work remarkably well, thanks to the clever WebKit engineers and site designers. Unfortunately, though one can create great sites that work brilliantly on mobile devices, differences in how things like double-clicks are handled have kept editing in Magnolia from working 100% when using touch-based browsers. This will be remedied in Magnolia 5. Our UX engineer has been giving a lot of thought to the different ways one can use Magnolia, and is planning full support for Mouse, Keyboard, and Touch interaction models. (The focus here will also bring much improved accessibility for editors using screen readers or other assistive technologies.)
2. REST API: For all of its excellent functionality, Magnolia has been fairly walled in by the boundaries of Java. Introducing a fully-fledged REST API will allow other applications to benefit from Magnolia’s functionality without having to be written in the same language, making integrations across disparate systems, or even entirely new UIs, much more practical.
3. Templating Improvements: The templating system is being reworked to support designating “areas” within a page, to make designing templates even more straightforward, to give Freemarker templates more functionality, and to remain as backward-compatible as possible. In addition, the default rendering engine will allow content to be requested as XML or JSON, further easing reuse of Magnolia content in other contexts.
4. Undo: “Are you REALLY sure you want to do this?” confirmation dialogs are a pain. They get in your way when you really do want to do something, and you click blithely through them when you don’t. Magnolia 5 will do away with these confirmation dialogs, replacing them with an undo/redo queue. While this approach is still fairly uncommon for web applications (mostly for technical reasons), it provides a more fluid and safer feeling while using it, as there are fewer obstacles to experimentation, and it’s easier to recover if you don’t like the results.
5. Improved Configuration Experience: Much of Magnolia’s configuration has to date been done by changing values in a giant JCR tree. While flexible, this approach does leave folks who are new to it (and even those of us who aren’t) overwhelmed at times. Magnolia 5 will be adding task-specific configuration panes that will pull together the options related to a specific bit of functionality, making it much easier to configure the system for particular tasks.

This all sounds pretty good, right? Well, there’s more that I haven’t yet mentioned: use of Vaadin for the editing UI, better visual organization in editing screens, a clipboard for copying and pasting across pages and sites, clearer security, richer dialogs, a generic JCR UI, JCR 2.0 support, improved performance, and more. If you’re interested in additional details, you can visit the Magnolia 5.0 section of Magnolia’s wiki, or sign up for the Magnolia Conference in September, where the new software will be shown publicly for the first time.

As someone who works with Magnolia full-time these days, I’m really looking forward to it.

Pushing Magnolia: Creating a Command

When Magnolia CMS is set up normally, it generates content on a page-by-page basis as it’s requested. The HTML is never written out to the filesystem, but is generated on the fly by the app and then cached in memory. This “pull” approach has a number of advantages: content is only rendered when it’s actually needed, one can be sure that the content is up to date, and content activations are quick.

Recently, however, one of our new customers needed Magnolia’s web pages written out to disk so that their existing web infrastructure could do some unusual post-processing on the HTML. We had to figure out a way to “push” Magnolia’s content out to the filesystem.

To enable this functionality and make it available throughout Magnolia, I decided the best thing to do would be to wrap it up into a Command. The great thing about Commands is that they can be used in workflows, called by the user interface, triggered by the Observation modules, and generally reused in a ton of different contexts.

(Note: we don’t recommend this approach for serving content. This article is more useful as a way to see how commands are constructed and can interact with the rest of the system than as something one would generally want to emulate for a production system.)

To create a new command, the first thing one must do is to create a new Java class that extends one of the existing command classes. The class from which all commands are derived is MgnlCommand, though in this case BaseActivationCommand gives us some additional functionality that is useful, so we’ll use it instead:

public class PublishToFileSystemCommand extends BaseActivationCommand {
	@Override
	public boolean execute(Context ctx) throws Exception {
	}
}

Next up, we need to add some parameters to the command. This is done by adding properties to the object, along with standard getters and setters. Magnolia’s introspection is clever enough to pick these up and generate the appropriate hooks to allow them to be set in all of the contexts where a command can be used. (We’ll also set up a logger while we’re at it.)

	private static Logger log = LoggerFactory.getLogger(PublishToFileSystemCommand.class);

	private String fileSystemPath = "/tmp/";
	private String pathToWget = "/opt/local/bin/wget";
	private String baseUrl = "http://localhost:8080/magnoliaPublic";

	public void setFileSystemPath(String fileSystemPath) {
		this.fileSystemPath = fileSystemPath;
	}

	public String getFileSystemPath() {
		return fileSystemPath;
	}

	public void setPathToWget(String pathToWget) {
		this.pathToWget = pathToWget;
	}

	public String getPathToWget() {
		return pathToWget;
	}

	public void setBaseUrl(String baseUrl) {
		this.baseUrl = baseUrl;
	}

	public String getBaseUrl() {
		return baseUrl;
	}

As you can see, the various parameters we’re interested in letting the user set are:

• fileSystemList: Where in the file system to write the files that wget retrieves.
• pathToWget: Where to find the wget command-line utility.
• baseUrl: The URL of the Magnolia instance from which to download the rendered HTML pages. If you want a page without the editing widgets, you’ll want to use a public instance here.

Now, since there are already programs out there that do a great job at fetching web content and writing it to the filesystem, there’s no need for us to reinvent that wheel. Instead, we’ll just use wget, which is widely available across unix-y operating systems, and which can be called with tons of different command-line options to make it behave the way we want. Here we add a template for calling the wget command, and a function to fire up command-line commands from Java:

	private static String commandTemplate = "PATH_TO_WGET -q --auth-no-challenge --http-user=SUPERUSER_NAME --http-password=SUPERUSER_PASSWORD -P FILE_SYSTEM_PATH -k -r URL";

	private boolean runCommandLine(String command) {
		log.info( "About to execute: " + command );
		try {
			Runtime rt = Runtime.getRuntime();
			Process pr = rt.exec( command );

			BufferedReader input = new BufferedReader(new InputStreamReader( pr.getInputStream()) );

			String line = null;

			// since we're using quiet mode, we shouldn't be getting output,
			// but just in case...
			while ((line = input.readLine()) != null) {
				log.info(line);
			}

			int exitVal = pr.waitFor();
			log.info("Exited with code " + exitVal );
			return true;
		} catch (Exception e) {
			log.error(e.toString());
			e.printStackTrace();
			return false;
		}
	}

(Thanks to linglom.com for the sample code I drew upon for this.)

When Java calls other programs using the command-line, the read-write buffers can fill up very quickly. Thus, we use wget’s quiet mode to keep it from overwhelming Java with its voluminous output. Additionally, wget’s exit codes are often odd, at least with the version of it I was using, so we merely log the codes, but otherwise ignore them.

Because Magnolia supports content in multiple languages, we also need a way to get a list of additional languages that should be retrieved. This method iterates through the languages listed in the site definition. (Site definitions, of course, rely on Magnolia’s templating kits to work.) As the primary language is at the default URL, we don’t include that language in the list of additional ones we’ll want to retrieve:

	private List getAdditionalLocales( Context ctx ) throws RepositoryException {
		SiteManager siteManager = SiteManager.Factory.getInstance();
		Site site = siteManager.getAssignedSite( getNode( ctx ) );
		I18nContentSupport i18nSupport = site.getI18n();
		Collection locales = i18nSupport.getLocales();

		List returnLocales = new LinkedList();
		Iterator i = locales.iterator();
		while ( i.hasNext() ) {
			Locale locale = i.next();
			if ( !locale.toString().equals( i18nSupport.getFallbackLocale().toString() ) ) {
				returnLocales.add( locale.toString() );
			}
		}
		return returnLocales;
	}

Now the infrastructure is all set up. Let’s go back and fill in the execute() method, which is the code that actually gets called when a command is used:

	@Override
	public boolean execute(Context ctx) throws Exception {

		Content nodeToActivate = getNode( ctx );
		Content homePage = nodeToActivate.getAncestor( 1 );

		// These credentials are not needed when downloading from a public
		// instance, but it doesn't hurt to provide them, and we'll need them
		// if we're ever grabbing content from an Author instance.
		User superuser = Security.getSystemUser();
		String superuserName = superuser.getName();
		String superuserPassword = superuser.getPassword();

		log.info( "path to fetch: " + baseUrl + homePage.getHandle() );
		log.info( "fileSystemPath: " + this.getFileSystemPath() );
		log.info( "additionalLocales: " + this.getAdditionalLocales( ctx ) );

		String command = commandTemplate.replaceAll("FILE_SYSTEM_PATH", fileSystemPath).
			replaceAll("URL", baseUrl + homePage.getHandle() ).
			replaceAll("PATH_TO_WGET", pathToWget).
			replaceAll("SUPERUSER_NAME", superuserName ).
			replaceAll("SUPERUSER_PASSWORD", superuserPassword);

		if ( !runCommandLine(command) ) return false;
		Iterator i = this.getAdditionalLocales(ctx).iterator();
		while ( i.hasNext() ) {
			String localeName = i.next();
			command = commandTemplate.replaceAll("FILE_SYSTEM_PATH", fileSystemPath).
			replaceAll("URL", baseUrl + homePage.getHandle() + "/" + localeName + "/" ).
			replaceAll("PATH_TO_WGET", pathToWget).
			replaceAll("SUPERUSER_NAME", superuserName ).
			replaceAll("SUPERUSER_PASSWORD", superuserPassword);
			if ( !runCommandLine( command ) ) return false;
		}
		return true;
	}

This code could obviously be refactored a bit to make it cleaner and less repetitious, but it should give a good idea of what’s involved in actually making the command execute. The return value indicates whether the command was successful or not, and can affect the calling context in various ways.

With this final step, we have a fully functional, parameterized command! Here’s the finished, complete source:

package com.sample;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.util.Collection;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
import java.util.Locale;

import javax.jcr.RepositoryException;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import info.magnolia.cms.core.Content;
import info.magnolia.cms.core.HierarchyManager;
import info.magnolia.cms.core.NodeData;
import info.magnolia.cms.i18n.I18nContentSupport;
import info.magnolia.cms.security.Security;
import info.magnolia.cms.security.User;
import info.magnolia.context.Context;
import info.magnolia.context.MgnlContext;
import info.magnolia.module.admininterface.commands.BaseActivationCommand;
import info.magnolia.module.templatingkit.sites.Site;
import info.magnolia.module.templatingkit.sites.SiteManager;

public class PublishToFileSystemCommand extends BaseActivationCommand {

	private String fileSystemPath = "/tmp/";
	private String pathToWget = "/opt/local/bin/wget";
	private String baseUrl = "http://localhost:8080/magnoliaPublic";

	private static Logger log = LoggerFactory.getLogger(PublishToFileSystemCommand.class);
	private static String commandTemplate = "PATH_TO_WGET -q --auth-no-challenge --http-user=SUPERUSER_NAME --http-password=SUPERUSER_PASSWORD -P FILE_SYSTEM_PATH -k -r URL";

	@Override
	public boolean execute(Context ctx) throws Exception {

		Content nodeToActivate = getNode(ctx);
		Content homePage = nodeToActivate.getAncestor( 1 );

		// These credentials are not needed when downloading from a public
		// instance, but it doesn't hurt to provide them, and we'll need them
		// if we're ever grabbing content from an Author instance.
		User superuser = Security.getSystemUser();
		String superuserName = superuser.getName();
		String superuserPassword = superuser.getPassword();

		log.info( "path to fetch: " + baseUrl + homePage.getHandle() );
		log.info( "fileSystemPath: " + this.getFileSystemPath() );
		log.info( "additionalLocales: " + this.getAdditionalLocales( ctx ) );

		String command = commandTemplate.replaceAll("FILE_SYSTEM_PATH", fileSystemPath).
			replaceAll("URL", baseUrl + homePage.getHandle() ).
			replaceAll("PATH_TO_WGET", pathToWget).
			replaceAll("SUPERUSER_NAME", superuserName ).
			replaceAll("SUPERUSER_PASSWORD", superuserPassword);

		if ( !runCommand(command) ) return false;
		Iterator<String> i = this.getAdditionalLocales(ctx).iterator();
		while ( i.hasNext() ) {
			String localeName = i.next();
			command = commandTemplate.replaceAll("FILE_SYSTEM_PATH", fileSystemPath).
			replaceAll("URL", baseUrl + homePage.getHandle() + "/" + localeName + "/" ).
			replaceAll("PATH_TO_WGET", pathToWget).
			replaceAll("SUPERUSER_NAME", superuserName ).
			replaceAll("SUPERUSER_PASSWORD", superuserPassword);
			if ( !runCommand( command ) ) return false;
		}
		return true;
	}

	private boolean runCommand(String command) {
		log.info( "About to execute: " + command );
		try {
			Runtime rt = Runtime.getRuntime();
			Process pr = rt.exec( command );

			BufferedReader input = new BufferedReader(new InputStreamReader( pr.getInputStream()) );

			String line = null;

			// since we're using quiet mode, we shouldn't be getting output,
			// but just in case...
			while ((line = input.readLine()) != null) {
				log.info(line);
			}

			int exitVal = pr.waitFor();
			log.info("Exited with error code " + exitVal );
			return true;
		} catch (Exception e) {
			log.error(e.toString());
			e.printStackTrace();
			return false;
		}
	}

	private List<String> getAdditionalLocales( Context ctx ) throws RepositoryException {
		SiteManager siteManager = SiteManager.Factory.getInstance();
		Site site = siteManager.getAssignedSite( getNode( ctx ) );
		I18nContentSupport i18nSupport = site.getI18n();
		Collection<Locale> locales = i18nSupport.getLocales();

		List<String> returnLocales = new LinkedList<String>();
		Iterator<Locale> i = locales.iterator();
		while ( i.hasNext() ) {
			Locale locale = i.next();
			if ( !locale.toString().equals( i18nSupport.getFallbackLocale().toString() ) ) {
				returnLocales.add( locale.toString() );
			}
		}
		return returnLocales;
	}

	public void setFileSystemPath(String fileSystemPath) {
		this.fileSystemPath = fileSystemPath;
	}

	public String getFileSystemPath() {
		return fileSystemPath;
	}

	public void setPathToWget(String pathToWget) {
		this.pathToWget = pathToWget;
	}

	public String getPathToWget() {
		return pathToWget;
	}

	public void setBaseUrl(String baseUrl) {
		this.baseUrl = baseUrl;
	}

	public String getBaseUrl() {
		return baseUrl;
	}

}

Using Magnolia for Mobile Content

When we started work on version 2.0 of the Texas State Mobile app for the iPhone, the biggest new feature that the Marketing department wanted to see was a “What’s New” screen. Their idea was that they could use it to accomodate all of the folks who wanted a presence in the mobile app, but didn’t really have enough content to justify adding a permanent module to the app. Here’s what it looks like:

What you can’t see in the photo is that each of the three sections scrolls left to right when the user swipes on that area, bringing more buttons into view. Additionally, each button can link to a web page or to content within the app itself. Because of these specialized behaviors, we couldn’t implement this screen as a web page. But, of course, the Marketing department was used to being able to update content quickly and easily, and wasn’t interested in having to publish a new version of the mobile app each time the content on this screen needed updating. (Nor was my team interested in managing that many trips through Apple’s approval process!)

After giving it some thought, we hit upon the right solution: let the content be managed in our Magnolia CMS, and have the app ask Magnolia for the needed data when it starts up. We got the flexibility we were used to with our web content, but the additional functionality that the native Objective C code provided.

The way we did this was first to build a simple template in Magnolia for managing the content. For each button, we needed a standard-resolution image, a high-resolution image (for retina displays), and a URL. (The URL could either be “http:” or “txstatemobile:”, a scheme we invented for linking to content within the app itself.) We created a paragraph and dialog definition accordingly:

And a paragraph template so that Magnolia would know how to render one of these buttons when called upon to do so:

<cms:out nodeDataName="image" var="imageurl" />
<img src="${pageContext.request.contextPath}${imageurl}" />
<p><cms:out nodeDataName="link" /></p>
<br style="clear: both" />

(Note: all of the JSP code could be done more easily with Freemarker, but we weren’t up to speed on using it yet.)

In order to allow editors to manage the content, we needed to create collections to hold the content — one for each row on the app screen. Here’s a sample from our template’s JSP that includes the top and middle rows:

<div class="section">
	<h1>Large Buttons</h1>
	
	<cms:contentNodeIterator
		contentNodeCollectionName="largeButtons">
		<div>
			<cms:adminOnly>
				<cms:editBar />
			</cms:adminOnly>

			<cms:includeTemplate />
		</div>
	</cms:contentNodeIterator>
	
	<cms:adminOnly>
		<cms:newBar contentNodeCollectionName="largeButtons"
			paragraph="mobile-large-button" newLabel="Add New Large Button"/>
	</cms:adminOnly>
</div>

<div class="section">
	<h1>Small Buttons (Middle Row)</h1>
	
	<cms:contentNodeIterator
		contentNodeCollectionName="smallButtonsMiddle">
		<div>
			<cms:adminOnly>
				<cms:editBar />
			</cms:adminOnly>

			<cms:includeTemplate />
		</div>
	</cms:contentNodeIterator>
	
	<cms:adminOnly>
		<cms:newBar contentNodeCollectionName="smallButtonsMiddle"
			paragraph="mobile-small-button" newLabel="Add New Small Button"/>
	</cms:adminOnly>
</div>

This gives the editor a nice, friendly interface for managing the content:

Now, the tricky bit is that we don’t want to present this content as XHTML to the iPhone app. We generally use JSON as our data interchange format of choice, so we created another template file, this one with the iPhone in mind as the consumer. Here’s the bit in the JSON template that corresponds to the above section in the human-friendly template:

"largeButtons":[
<cms:contentNodeIterator
	contentNodeCollectionName="largeButtons"
	varStatus="status">
	<cms:out nodeDataName="image" var="imageurl" />
	<cms:out nodeDataName="image-hirez" var="imagehirezurl" />
	<cms:out nodeDataName="link" var="link" />
	{
		"image": "${ pageContext.request.scheme }://${serverUrl}${pageContext.request.contextPath}${imageurl}",
		"image-hirez": "${ pageContext.request.scheme }://${serverUrl}${pageContext.request.contextPath}${imagehirezurl}",
		"link": "${fn:replace( link, ' ', '%20' )}"
	}<c:if test="${not status.last}">,</c:if>
</cms:contentNodeIterator>
],

"smallButtonsMiddle":[
<cms:contentNodeIterator
	contentNodeCollectionName="smallButtonsMiddle"
	varStatus="status">
	<cms:out nodeDataName="image" var="imageurl" />
	<cms:out nodeDataName="image-hirez" var="imagehirezurl" />
	<cms:out nodeDataName="link" var="link" />
	{
		"image": "${ pageContext.request.scheme }://${serverUrl}${pageContext.request.contextPath}${imageurl}",
		"image-hirez": "${ pageContext.request.scheme }://${serverUrl}${pageContext.request.contextPath}${imagehirezurl}",
		"link": "${fn:replace( link, ' ', '%20' )}"
	}<c:if test="${not status.last}">,</c:if>
</cms:contentNodeIterator>
]

But how to get JSON when we want it and HTML when we want it? Simple! Sub-Templates. Here’s the template definition:

By defining the sub-template this way, we can ask for “whats-new.html” for the human-friendly view, and “whats-new.json” for the rendering the iPhone wants.

Now that we’ve got this data published, it’s a simple matter for the iPhone app to fetch the JSON, to download the images from the URLs Magnolia provides, and to apply them to the UIButtons we create in the iPhone app. Best of all, since we’re making the data available using internet standards like JSON and HTTP, we can take advantage of it elsewhere too. The University’s upcoming Android app uses exactly the same data for its What’s New screen as well, allowing the Marketing team to reach out to even more mobile users with no extra effort on their part.

Upgrades with Magnolia

During my stint at Texas State University, we adopted Magnolia CMS around 2007 after a disastrous two-year dalliance with another enterprise system. Since we wanted to be as responsive as possible to our users’ varied (and often changing) needs while still keeping our CMS reasonably stable, we decided to do once-a-month releases to fix bugs and incorporate new features. This gave us 3 weeks a month to do development, and a week to do testing and regression to make sure everything we were putting into the system worked as we hoped.

These monthly updates, which we had to do after-hours, were initially a big hassle: we made long checklists to remind ourselves what paragraph and dialog definitions would need updating, write scripts to update the existing data in the system’s database, and sometimes even had to export objects to XML, tweak them in an editor, and then import them back into the system to make the changes we wanted. Needless to say, this made release nights quite stressful, and we staggered out at 4:00am, seven hours after we had started, more than once. (Upon seeing a porcupine in the parking garage one night, we had to double-check with each other before we were convinced we weren’t hallucinating it.)

As the folks at Magnolia made the system more modular, they also introduced the Version Handler framework, which let us define all of these upgrade tasks programmatically. Instead of having to march down a checklist we meticulously prepared in advance, doing manual step after manual step, we could now let the system do all of that for us in a consistent, reproducible way.

But better than that, having the upgrade tasks defined in this way allowed us to do a “dry run” of our upgrades before actually doing them on a production system. I could run the upgrade process on my machine and find the issues during work hours when I was awake, alert, and had appropriate amounts of caffeinated beverages available. We were able to get our issues sorted out with much less stress, hair-pulling, and use of language my mother wouldn’t approve of than was the case when we were having to handle all of those problems on the fly, under time pressure, in the middle of the night.

Since we implemented this improvement, we had only one unexpected issue in a couple of years of monthly updates. (That one turned out to be our fault!) It was a huge change for the better, and is one of the things about Magnolia that we came to like best.