Announcement Announcement Module
No announcement yet.
can it support a database with no transactions? Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • can it support a database with no transactions?

    Our core database we are looking to implement Spring Batch on a database with no transactions (Informix).

    Is it possible to customize the transaction handling in Spring Batch so we can avoid sending COMMITS/ROLLBACKs to the database and getting errors back - and do our own manual actions instead?

    Also - is nested transactions support required? Informix does not support these at all.

  • #2
    Technically, you can pass in any kind of PlatformTransactionManager to the StepExecutor you want. However, if you have no transactions then any errors whatsoever will require your entire batch job to shut down and could leave output in an inconsistent state. (which is the motivation behind transactions in the first place) I'm not familiar with Informix as a database, but I can't imagine why it wouldn't support transactions, or why you would want to run any type of application without transactions. The only use case I can think of is a pure file to file batch job. In this case, you could 'turn off' the repository, which would then not require any database. However, keep in mind that doing so would prevent you from seeing any status on the job, or restarting it.


    • #3
      don't get me started

      I can't imagine it without transactions either / but that's the legacy system we have. We are told that if we turned transactions on - with no code changes - then it would slow performance by 200-300%. All of the existing code is single threaded. One of the reasons we want to port to Java is to take advantage of a multi-threaded architecture. But we still need to integrate the existing platform as we move along.


      • #4
        Coincidentally Spring Batch does actually include a "dummy" transaction manager (ResourcelessTransactionManager) that could be used to make the database access non-transactional as a side effect. It was intended (and is used internally) for testing purposes.


        • #5
          Would you use ResourcelessTransactionManager if you are only doing a file-to-file step? Otherwise what transaction manager would you install in SimpleStepFactoryBean?


          • #6
            Even if doing File to File, I would still want a normal transaction manager and repository, because then you can see output in the status tables. However, if you don't care about the meta data, then you could just use the 'in-memory repository' that's covered in chapter 4:



            • #7
              So if you were doing a job with File input and File output, what PlatformTransactionManager subclass would you configure for the job? There doesn't seem to be a Spring implementation that's not coupled to a DB, JMS, or some other resource provider.

              Would you hook it up to the transaction manager for the batch database that stores the job executions etc?


              • #8
                Just as Dave suggests, the ResourcelessTransactionManager will fill this void. You can inject it and it doesn't do any transaction management. In terms of side effects, the really important thing to remember is that even if it doesn't break your file-to-file operations, updates to your Spring Batch metadata will not be protected, so you can actually have jobs break as a result.

                Spring's transaction support is pretty interesting - it handles a lot of concerns for you, including whether or not nesting is supported - you should read up more about it directly from the Spring documentation:


                • #9
                  My question was, when I have a Job that deals only with files, which impl should I pick? I have a txMgr for the batch meta data database, but it seems silly to hook that one up to the one that manages the file. Or is it?

                  Between Spring 2.5.2 and Batch, I have these impls:
                  CCI, DataSource, Hibernate, JDO, JMS, JPA, JTA, Resourceless, TopLink, WebSphere.

                  The only one that makes any possible sense is Resourceless. My question was just to confirm that that is the intent of the framework developers.


                  • #10
                    You should probably use the DataSourceTransactionManager if you are using the Jdbc*Daos from Spring Batch (i.e. with any "normal" database platform and with JobRepositoryFactiryBean, as in the samples). Just because your business data is file-based, doesn't mean that you shouldn't use a database for the batch meta data - generally the opposite is true. There might be some special cases where you can live without an RDBMS for the meta data (e.g. if you really don't care about restartability or reporting and tractability), but that won't often be the case.


                    • #11
                      I'm getting confused. It seems to me that there are more than one transaction managers involved.

                      First, there's the one for the batch database where the metadata (JobExecutions etc) are stored. I have a datasource, transaction manager, and the recommended AOP TXN configs (direct from the user manual), configured with

                      	    	pointcut="execution(* org.springframework.batch.core..*Repository+.*(..))"
                      	    	advice-ref="txAdvice" />
                      	        <tx:method name="create*" propagation="REQUIRES_NEW" isolation="SERIALIZABLE" />
                      	        <tx:method name="*" />
                      However, the transaction manager I'm asking about is the one configured in SimpleStepFactoryBean. When my Step is an ItemReader reading a file, should I use the same TransactionManager that points to the batch database?

                      	<bean id="myJob" class="org.springframework.batch.core.job.SimpleJob">
                      		<property name="steps">
                      				<bean id="myStep" class="org.springframework.batch.core.step.item.SimpleStepFactoryBean">
                      					<property name="transactionManager" ref="?????????????????????"/>
                      					<property name="itemReader" class="FlatFileItemReader"/>
                      					blah blah
                      I guess you are suggesting that I use my batch database transaction manager in place of the ????? above. That's fine, I can do that. However, it seems that if I used ResourcelessTransactionManager, it doesn't affect the transactions on the batch db; those are set by the AOP and TX config above.

                      Do I have that correct? I'm very sensitive about transaction configuration and want to make sure I get it right.

                      OTOH, if my step used a database item reader, then I'd configure a transaction manager for that particular database (or JTA) for the SimpleStepFactoryBean. Correct?

                      Sorry for the confusion.


                      • #12
                        That's pretty much what I was saying

                        There's no reason you can't use the same tx manager for both your metadata database and your business transactions. Hook the resourceless transaction manager into both and see if that solves your issue.


                        • #13
                          I think Doug was responding to the original post, not the last one, so let me clarify. The transaction manager injected into the step should always be the same as the one that is used to wrap the repository declaratively. Sorry for the confusion, but it is quite important for restartability that we can piggyback on the business transaction in the step - we only need to store position data for your open files, for instance, if the current chunk is successful.


                          • #14
                            I apologize for my confusing response - I most likely was responding to the wrong point in the thread - that's what I get for trolling the forums at that hour.

                            To be clear, you are 100% CORRECT that there can be up to two transaction managers involved - one for the AOP advice around the batch metadata repository, and one that is injected into the steps. However, these generally should be the same.

                            I'm confused as to whether or not you have tried using a transaction manager and encountered failures, or are just anticipating failure because Informix doesn't support transactions.

                            I believed (and I could be wrong) that the Spring Transaction module would properly handle this and not attempt to dispatch a transaction commit command if it was not supported by the underlying datasource.


                            • #15
                              Originally posted by Dave Syer View Post
                              Cool, that makes clear the file-to-file situation. I'll fix up my txn config. Thanks!

                              Now let me change the setup a bit; suppose the Step writes into a different database (call it "content"). Our batch jobs access a bunch of different databases. We have not installed the spring batch schema in all of them; instead, we have a central "batch" db to hold the repository.

                              Dave, in this case I would still inject the batch repository tx mgr into the step factory, and wrap my DAO to the content database that the step writes to with declarative txn for that db. Is that correct? Or should I use JTA if my batch repository schema is in a different db than the step writes the items to?