1 Spring Batch Admin User Guide

Document Sample
1 Spring Batch Admin User Guide Powered By Docstoc
					1. Spring Batch Admin User Guide
Version : 1.0.0.M1
Authors : Dave Syer

Spring Batch Admin provides a web-based user interface that features an admin console for
Spring Batch applications and systems. It is an open-source project from SpringSource .

1. Main Use Cases

The easiest way to get a quick overview of Spring Batch Admin is to see some screenshots of the
main use cases.The user interface is a web application (built with Spring MVC).

2. Inspect Jobs

The user can inspect the jobs that are known to the system. Jobs are either launchable or non-
launchable (in the screenshot they are all launchable). The distinction is that a launchable job is
defined and configured in the application itself, whereas a non-launchable job is detected as state
left by the execution of a job in another process. (Spring Batch uses a relational database to track
the state of jobs and steps, so historic executions can be queried to show the non-launchable

3. Launch Job

Launchable jobs can be launched from the user interface with job parameters provided as name
value pairs, or by an incrementer configured into the application.

4. Inspect Executions

Once a job is executing, or has executed, this view can be used to see the most recent
executions, and a brief summary of their status (STARTED, COMPLETED, FAILED, etc.).

Each individual execution has a more detailed view (shown above), and from there the user can
click down to a view of each of the step executions in the job (only one in this case). A common
reason for wanting to do this is to see the cause of a failure.
The top of the step execution detail view shows the history of the execution of this step across all
job executions. This is useful for getting a statistical feel for performance characteristics. A
developer running a job in an integration test environment might use the statistics here to compare
different parameterisations of a job, to see what effect is of changing (for instance) the commit
interval in an item processing step.

The bottom of the step execution view has the detailed meta-data for the step (status, read count,
write count, commit count, etc.) as well as an extract of the stack trace from any exception that
caused a failure of the step (as in the example shown above).

5. Stop an Execution
A job that is executing can be stopped by the user (whether or not it is launchable). The stop
signal is sent via the database and once detected by Spring Batch in whatever process is running
the job, the job is stopped (status moves from STOPPING to STOPPED) and no further
processing takes place.
2. Installation

1. Installing Spring Batch Admin

There is a sample project in the distribution - it's a regular war file. Once it's deployed, browse with
a web browser (e.g. localhost:8080/spring-batch-admin-sample) and check out the features for
launching jobs and inspecting job executions.

1.1. Building your own applications:

A custom web application with the ability to inspect job execution data from external processes is
really easy to build:
- Create a war project with an index.jsp and a web.xml (from the sample or from the spring-
   batch-admin-resources.jar ).
- Include the spring-batch-admin-*.jar files in WEB-INF/lib , plus all their dependencies. In the
   sample this is done simply by making the WAR depend on those jar files in the Maven pom.
- Deploy the web app. It starts up with an in-memory database, so without any jobs defined in the
   application it isn't going to do much at first. See the section on environment settings for more


2. Running Spring Batch Admin

If you are deploying Spring Batch Admin as a WAR file (either the sample or one you rolled
yourself), just deploy it as normal to a servlet container like SpringSource tcServer .

If the WAR file is called spring-batch-admin-sample.war (it may have a version number
appended), then when you load the application in a browser at http://localhost:8080/spring-batch-
admin-sample) ) it should show a listing of the contents of the application:

This page is a list of URLs, or URL patterns, along with the HTTP method (GET, POST, etc.) that
you need to use to carry out an operation on that resource. If the URL is a pattern it has braces
({}) in it to represent the value of a parameter that needs to be supplied by the user. Users of
Spring REST support in MVC will recognise this convention, and also the idea that a GET
represents a request for information, and a POST is an "insert" operation. So, for example, to
request a list of all running Job executions you must GET the resource at /jobs/executions
(relative to the batch/ servlet in the application). And to launch a new job execution you POST to
/jobs/ jobName (with "{jobName}" replaced with the name of a launchable job).

Links on the home page that are not parameterised and require a GET method are clickable in the
browser and take you to that page in the application. The others are all navigable from there.

3. Menus

The basic menus for the application out of the box are
- Home - the URL list or site map listing the interface for the application
- Jobs - list and launch jobs. Only jobs which are either configured in the application or have
  been executed in other processes against the same database show up here.
- Executions - view and interact with running Job and Step executions.
- Status - a status page for the health of the application (may be removed before 1.0.0 is

You can learn how to extend the content of the application including the menus in a later section.
3. Customization

1. Environment Settings

The most likely thing you will want to customize is the location of the database. Spring Batch
Admin ships with an embedded HSQLDB database, which is initialized on start up.
- To change the database type add a file to the application classpath called batch-
  [type].properties , where [type]is the database type you want to use (e.g. mysql , oracle , db2 ).
  Copy the contents of the batch-hsql.properties from the Manager jar and change the values to
  suit your environment. Then launch the application with a system property -
- To stop the database from being wiped and re-created on start up just set
  batch.data.source.init=false (in the properties file or as a System property).

N.B. The use of the environment variable to switch on the database type is only a suggestion. You
can change that and other things by overriding and adding configuration fragments to the Spring
application context. See below for details.

2. Overriding Components from Spring Batch Admin

The system tries to provide some useful defaults for things like transaction manager, job
repository, job registry etc. Most of these live in the manager jar in a special place: META-
INF/bootstrap/*.xml . If you want to override them, just add your own versions of the same bean
definitions to a Spring XML config file in META-INF/spring (these are guaranteed to load after the
bootstrap files, so they can override default definitions). You could use this to override the data
source definition as an alternative to the environment settings described above.

2.1. Important Bootstrap Components

The important bootstrap components are listed below along with some common scenarios for
when you might want to override or replace them:

2.1.1. JDBC Data Source

- Bean ID : dataSource
- Default : DBCP BasicDataSource with placeholders for common properties.
- Override Scenarios : Change to another implememtation, or add additional placeholders.

2.1.2. Spring Transaction Manager

- Bean ID : transactionManager
- Default : DataSourceTransactionManager injected with the dataSource
- Override Scenarios : Change to another implementation. For example if you are using
  Hibernate in your jobs, you will need to provide a HibernateTransactionManager .

2.1.3. Job Launcher

- Bean ID : jobLauncher
- Default : SimpleJobLauncher injected with the jobRepository and transactionManager , and
  also a thread pool task executor with bean id poolTaskExector , so that launches happen in a
  background thread by default.
- Override Scenarios : Should be fine as it is for most use cases. To switch to synchronous
  launching change the task executor (see below).

2.1.4. Job Launcher Task Executor

- Bean ID : jobLauncherTaskExecutor
- Default : a task executor with pool-size=6 and rejection policy ABORT .
- Description : Injected into the jobLauncher to control the number of cocurrent jobs executing in-
  process. The effect of the defaults is that at most 6 jobs will run concurrently when launched
  locally from this application, and any submitted in excess of that will fail.
- Override Scenarios :
      - Change to a synchronous task executor for integration testing (see the sample unit tests
         for an example), e.g.

            <bean id="jobLauncherTaskExecutor"

      - Change the pool parameters. Note that these settings can be changed at runtime using
        JMX, if the task executor is exposed explictly.

2.1.5. Throttled Task Executor for Jobs and Steps

- Bean ID : throttledTaskExecutor
- Default : a task executor with pool-size=600 and rejection policy CALLER_RUNS and infinite
  throttle limit.
- Description : Not used anywhere in the manager, but provided as a convenience for
  applications that provide their own jobs and steps with concurrent behaviour.
- Override Scenarios : The sample contains one job with a concurrent step ("job2"), and it works
  by creating a local taskExecutor bean inheriting from the root definition, and adding a throttle

     <bean id="taskExecutor" parent="throttledTaskExecutor">
      <property name="throttleLimit" value="100"/>

  This task executor is then injected into a step in the sample which has the effect of limiting the
  concurrency of the step to 100, but within overall limits set globally by the bootstrap context,
  and without discarding or failing any tasks that are submitted. The throttle limit is set to 100 so
  that if this job is the only one running it will hardly be affected by the throttle, but if it is
  contending with other jobs, then they will all eventually run in the thread pool provided by the
  bootstrap (with 600 threads by default). The bootstrap pool can be modified by overriding its
  bean definition poolTaskExecutor , e.g.

     <task:executor id="poolTaskExecutor" pool-size="200" rejection-

  changes the pool size to 200, limiting all jobs that use it to at most 200 threads. Your mileage
  may vary with these thread pool settings depending on the workload that is executed. It is worth
  experimenting with the parameters to see if you can affect the overall throughput or execution
  times of your multi-threaded jobs.

2.1.6. Job Repository

- Bean ID : jobRepository
- Default : a JDBC version of SimpleJobRepository .
- Override Scenarios : Not common to override. But if you do override then probably the
  application can be used to launch but not inspect job executions, since it relies on JDBC to
  inspect the database for historic executions.

2.1.7. Job Loader

- Bean ID : jobLoader
- Default : a ClassPathXmlJobLoader with path locations set to classpath*:/META-
  INF/batch/*.xml .
- Override Scenarios :
     - Changing the path locations maybe (trivial but not that useful).
     - Change the inheritance strategy for post processors. By default the AOP configuration
        and the PropertyPLaceHolderConfigurer instances from the root context are copied into
        the child context created from the Job configuration files. You can change this behaviour
        just by providing your own versions in addition in the child context, or to make global
        changes you can override the jobLoader with one that copies different post processors
        down into the child, or one that does something completely different.
     - Override it to a dummy bean for the purposes of an integration test, so that it doesn't try
        to create multiple instances of the Job that you want to test. The sample application has
        an example. In JobExecutionTests-context.xml we want to load the Jobs explicitly into
        the test context, and prevent them from being loaded again by the job loader, so we do

            <!-- prevent loading of other jobs by overriding the loader in the main bootstrap
           context -->
            <bean id="jobLoader" class="java.lang.String"/>

3. Add your Own Job Executions

To add job executions from another process just execute a job (e.g. from command line) against
the same database that is used by Spring Batch Admin, and the UI picks up the meta data from
the usual Spring Batch tables.

3.1. Create Your Own Standalone Application

To create a standalone application and run jobs against the Spring Batch Admin database, the
easiest thing to do is include the spring-batch-admin-*.jar files in the classpath of your application
and use the same conventions to override and extend as you would for a web application.

If you do that then in particular for a standalone application that runs a single job in its own
process, you will want to override the jobLauncherTaskExecutor as descriobed above, and
possibly also the jobLoader in addition to providing data source properties. Here's an example
configuration file to drive a job:

  <beans ...>
 <import resource="classpath*:/META-INF/bootstrap/*.xml" />
 <import resource="classpath*:/META-INF/batch/*.xml" />

  <!-- ensure job runs in foreground thread -->
  <bean id="jobLauncherTaskExecutor"
  <!-- prevent loading of other jobs by overriding the loader in the main bootstrap context -->

With the job defined in /META-INF/batch/*.xml you can use the CommandLineJobRunner from
Spring Batch to load the file above, and launch the job by name, e.g.

  $ java org.springframework.batch.core.launch.support.CommandLineJobRunner \
   -next config/launch-context.xml jobName

4. Add your Own Jobs For Launching

Include Spring XML files in META-INF/batch . Each file should be self-contained (except for Batch
Because they are child contexts you don't have to worry about name clashes between different
XML files or different contributing JARs (except for the jobs which must have unique names, at
present, but perhaps not when we get to a release).

As a convenience, the child contexts inherit property placeholders and AOP configuration from the
parent (this is not the default behaviour for a child context in Spring). This means you can control
those things centrally if you need to. Of course, the child can always create its own placeholder
definition and AOP configuration, but these will not affect the parent or any of its siblings.

4.1. Create your own Web Application

Hints for custom applications:
- Look at the sample.
   batch-admin-resources.jar ).
- Include the spring-batch-admin-*.jar files in WEB-INF/lib . In the sample this is done simply by
- Optionally add your own jobs in the classpath under META-INF/batch/*.xml .
- Deploy the web app.

5. Design and Architecture

Spring Batch Admin is a layered, extensible application. Its main artifacts are 2 JAR files (libraries)
containing all the content and business logic for the web application. To deploy them you need a
deployment platform, like a WAR file in a servlet container. There is a sample application spring-
batch-admin-sample which show how this works in practice.
6. Application Context Structure

There is a root or parent application context, and a child context for the UI components. These are
both loaded in the standard Spring MVC way following directives in web.xml. There are also
possibly several separate child contexts (of the main root) containing job configurations.
- The root context is loaded from a file in the manager JAR, but all that does is import from well
  known locations on the classpath:

     <import resource="classpath*:/META-INF/spring/*.xml"/>

- The UI child context is loaded in the same way from

     <import resource="classpath*:/META-INF/servlet/*.xml"/>

  N.B. the child context by default only loads when you visit the first page in the application. You
  should be able to see which files it actually loads from the log statements that come out on the
  server console.
- The job contexts are loaded by a special component ( ClasspathXmlJobLoader from Spring
  Batch) in the parent context. It looks for individual files in the pattern classpath*/META-
  INF/batch/*.xml and loads each file individually as its own self-contained context, registering
  any instance of Job it finds in the JobRegistry at the top level.

The job contexts are loaded in this way so that you can provide your own jobs in multiple (possibly
many) JAR files included in the application, but you do not have to worry too much about duplicate
bean definitions because only the Job names have to be unique. A job context inherits the AOP
and PropertyPlaceholderConfigurer settings from the root context (not the standard behaviour of a
child context) as a convenience, but you can add your own settings to apply locally as well.

7. Extending the UI

There are two extension points for the UI: adding menus, and modifying the content of the existing

7.1. Adding a Menu

Create a component of type org.springframework.batch.admin.web.base.Menu in the UI child
context ( META-INF/servlet/*.xml ). It has a url property that should point to a controller mapping
(relative to the application root context). That's it really.

If you want the page you add to show the other menus and links, you will want to implement the
view for the URL that you add in a specific way. Namely you will use a Freemarker template to
render the body of the page, and insert it in a Spring MVC View definition in your servlet XML