Crawljax

Crawljax can easily be extended through plugins. Any programmer with basic Java knowledge can write plugins using the documentation on this page.

Types of Plugins

The main interface is Plugin, which is extended by the different types of plugins. Each plugin type serves as an extension point that is called in a different phase of the Crawljax execution. To write a plugin you should implement one of the following plugin types.

Furthermore, if your plugin generates output in the form of files, it should implement the GenerateOutput inteface. This class gives you an easy way to get the absolute path the end-user would like to save any output files to.

Plugin invocation points

Crawljax plugins flow

In the picture above the different invocation points for the different type of plugins are visible, green boxes are the Plugins and the yellow boxes are the decisions / or operations of the core.

Example

Imagine we would like to save each dynamic DOM state as a static HTML page. We can easily add this functionality to Crawljax by creating a plugin that implements the OnNewStatePlugin interface. When a new state is visited the onNewState method is called with a session object as its parameter. This allows us to get the current DOM from the browser and save it to a file:

package com.crawljax.plugins;

import java.io.FileWriter;

import com.crawljax.core.CrawlSession;
import com.crawljax.core.plugin.OnNewStatePlugin;

public class MirrorGenerator implements OnNewStatePlugin {

	@Override
	public void onNewState(CrawlSession session) {
		try {
			String dom = session.getBrowser().getDom();
			String fileName = session.getCurrentState().getName();

			/* last parameter is false because we don't want to append if the file exists */
			FileWriter fw = new FileWriter(fileName, false);
			fw.write(dom);
			fw.close();
		} catch (Exception e) {
			e.printStackTrace();
		}
	}
}

Using the Plugin

You can use this plugin by adding it to your CrawljaxConfiguration instance in your “runner class”. This is done using the addPlugin method it provides. Note that in each execution point, the plugins are executed in the same order in which they are added using the addPlugin method.