Getting "bitronix.tm.internal.BitronixXAException: resource already started on XID" error

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Getting "bitronix.tm.internal.BitronixXAException: resource already started on XID" error

ewestfal
Hello, we're having an issue that I'm hoping someone on this list may
know something about.

We have an application configured which uses two different databases.
It's using the transaction management support provided by the spring
framework configured for JTA.  Things seems to work well most of the
time, but when we run load tests against it we get errors similar to the
following occasionally:

* bitronix.tm.internal.BitronixXAException: resource already started on
XID a Bitronix XID [737072696E672D62746D0000013423394EA100000E41 :
737072696E672D62746D0000013423394F0100000E4F]
         at
bitronix.tm.resource.jdbc.lrc.LrcXAResource.start(LrcXAResource.java:135)
         at
bitronix.tm.internal.XAResourceHolderState.start(XAResourceHolderState.java:219)
         at
bitronix.tm.internal.XAResourceManager.enlist(XAResourceManager.java:111)
         at
bitronix.tm.BitronixTransaction.enlistResource(BitronixTransaction.java:93)
         at
bitronix.tm.resource.common.TransactionContextHelper.enlistInCurrentTransaction(TransactionContextHelper.java:70)
         at
bitronix.tm.resource.jdbc.JdbcConnectionHandle.enlistResource(JdbcConnectionHandle.java:84)
         at
bitronix.tm.resource.jdbc.JdbcConnectionHandle.prepareStatement(JdbcConnectionHandle.java:289)
         at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source)
     ...

Full stack trace here: http://www.pastie.org/2991516

Any ideas on why this might be happening?  It appears that somehow a
resource which is "started" is getting reused across multiple concurrent
transactions which seems like it shouldn't happen.

Here is our bitronix transaction manager configuration:

bitronix.tm.serverId=spring-btm
bitronix.tm.disableJmx=true
bitronix.tm.timer.defaultTransactionTimeout=3600
bitronix.tm.journal=null
bitronix.tm.timer.gracefulShutdownInterval=0
bitronix.tm.journal.disk.logPart1Filename=target/btm1.tlog
bitronix.tm.journal.disk.logPart2Filename=target/btm2.tlog
bitronix.tm.2pc.warnAboutZeroResourceTransactions=false
bitronix.tm.allowMultipleLrc=true
bitronix.tm.setIgnoreRecoveryFailures=true

Of particular note here, we are using the "LRC" setup with two
datasources which is why we have allowMultipleLrc=true

Relevant PoolingDataSource configuration from our spring configuration
is as follows:

<property name="useTmJoin" value="true" />
<property name="allowLocalTransactions" value="true" />
<property name="shareTransactionConnections" value="true" />
<property name="enableJdbc4ConnectionTest" value="true" />
<property name="automaticEnlistingEnabled" value="true"/>

Thanks in advance for any help!
Eric



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Getting "bitronix.tm.internal.BitronixXAException: resource already started on XID" error

Brett Wooldridge-2
Hi Eric,

Thanks for using Bitronix.  I'm sure Ludovic will be responding soon regarding this issue, but
if you're willing to test the BTM-2.2 development branch, I would be interested to see if this
issue has been fixed.

With BTM-2.2 we're moving from a Java 1.4 codebase to Java 5.  And in the process
converting the code to use type-safe collections and more importantly taking advantage of
the newer Java 5 concurrent collections and synchronization primitives.  While going
through this process, I found at least one theoretical race condition, though not one that
had yet been reported.

I've been running the BTM-2.2 codebase on our pre-production servers for the past
two months, and have run several tens of millions of transactions through without 
issue.  Admittedly, we do not have multiple datasources and are not using LRC, so 
your environment is definitely different.

So, if you want to try the snapshot and give me your feedback regarding this issue
or whether any other errors occur in your testing.  I've made a snapshot available here:


Regards,
Brett

On Sat, Dec 10, 2011 at 12:32 AM, Eric Westfall <[hidden email]> wrote:
Hello, we're having an issue that I'm hoping someone on this list may know something about.

We have an application configured which uses two different databases. It's using the transaction management support provided by the spring framework configured for JTA.  Things seems to work well most of the time, but when we run load tests against it we get errors similar to the following occasionally:

* bitronix.tm.internal.BitronixXAException: resource already started on XID a Bitronix XID [737072696E672D62746D0000013423394EA100000E41 : 737072696E672D62746D0000013423394F0100000E4F]
       at bitronix.tm.resource.jdbc.lrc.LrcXAResource.start(LrcXAResource.java:135)
       at bitronix.tm.internal.XAResourceHolderState.start(XAResourceHolderState.java:219)
       at bitronix.tm.internal.XAResourceManager.enlist(XAResourceManager.java:111)
       at bitronix.tm.BitronixTransaction.enlistResource(BitronixTransaction.java:93)
       at bitronix.tm.resource.common.TransactionContextHelper.enlistInCurrentTransaction(TransactionContextHelper.java:70)
       at bitronix.tm.resource.jdbc.JdbcConnectionHandle.enlistResource(JdbcConnectionHandle.java:84)
       at bitronix.tm.resource.jdbc.JdbcConnectionHandle.prepareStatement(JdbcConnectionHandle.java:289)
       at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source)
   ...

Full stack trace here: http://www.pastie.org/2991516

Any ideas on why this might be happening?  It appears that somehow a resource which is "started" is getting reused across multiple concurrent transactions which seems like it shouldn't happen.

Here is our bitronix transaction manager configuration:

bitronix.tm.serverId=spring-btm
bitronix.tm.disableJmx=true
bitronix.tm.timer.defaultTransactionTimeout=3600
bitronix.tm.journal=null
bitronix.tm.timer.gracefulShutdownInterval=0
bitronix.tm.journal.disk.logPart1Filename=target/btm1.tlog
bitronix.tm.journal.disk.logPart2Filename=target/btm2.tlog
bitronix.tm.2pc.warnAboutZeroResourceTransactions=false
bitronix.tm.allowMultipleLrc=true
bitronix.tm.setIgnoreRecoveryFailures=true

Of particular note here, we are using the "LRC" setup with two datasources which is why we have allowMultipleLrc=true

Relevant PoolingDataSource configuration from our spring configuration is as follows:

<property name="useTmJoin" value="true" />
<property name="allowLocalTransactions" value="true" />
<property name="shareTransactionConnections" value="true" />
<property name="enableJdbc4ConnectionTest" value="true" />
<property name="automaticEnlistingEnabled" value="true"/>

Thanks in advance for any help!
Eric



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

  http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|

Re: Getting "bitronix.tm.internal.BitronixXAException: resource already started on XID" error

Brett Wooldridge-2
Eric,

Another thing which will be useful whether you're running  btm 2.1.2 or the development branch
is to enable logging.  Bitronix has extensive logging, and usually if we can see the logs from
the time of the error, we can figure out what went wrong (whether it was a bitronix bug or a bug
in some other component in the stack).

Because of the volume of logging from Bitronix, I recommend logging to a different file than
your primary application log.  Also, because of the volume of logging -- particularly under a 
load test -- I recommend rolling the logs so you don't lose the logs at the crucial time of
failure.

Here is a sample recommended configuration for log4j:


log4j.appender.BTM=org.apache.log4j.RollingFileAppender
log4j.appender.BTM.File=btm.log
log4j.appender.BTM.MaxFileSize=10MB
log4j.appender.BTM.MaxBackupIndex=50
log4j.appender.BTM.layout=org.apache.log4j.PatternLayout
log4j.appender.BTM.layout.ConversionPattern=%d{HH:mm:ss,SSS} [%-25.25c{1}] [%-25t] %-5p - %m%n

This is going to create a 10MB log file, which rolls 50 times (btm.log.1, btm.log.2, etc.).  The [%-25t] part
is also crucial because it logs the thread name.

What you want to do is run your load test, and if you can, stop the test as soon after the error is
observed as possible.  In your application log, find the exception that occurred during the run, and note
the time (HH:mm:ss,SSS).  Now, start opening btm.log files (btm.log, btm.log.1, btm.log.2, etc) and
find the one that contains the same approximate timestamp as the application error (+/- a few milliseconds).

Once you find the corresponding btm log, zip that btm.log AND THE TWO NEXT OLDER LOGS together
with the application log file with the error.  So, if you the error timestamp corresponded to logs in the
btm.log.18 file, zip that file along with btm.log.19 and btm.log.20 files.

You can open a bug and attach the zipped log files and we'll take a look at them.

Regards,
Brett


On Sat, Dec 10, 2011 at 6:55 PM, Brett Wooldridge <[hidden email]> wrote:
Hi Eric,

Thanks for using Bitronix.  I'm sure Ludovic will be responding soon regarding this issue, but
if you're willing to test the BTM-2.2 development branch, I would be interested to see if this
issue has been fixed.

With BTM-2.2 we're moving from a Java 1.4 codebase to Java 5.  And in the process
converting the code to use type-safe collections and more importantly taking advantage of
the newer Java 5 concurrent collections and synchronization primitives.  While going
through this process, I found at least one theoretical race condition, though not one that
had yet been reported.

I've been running the BTM-2.2 codebase on our pre-production servers for the past
two months, and have run several tens of millions of transactions through without 
issue.  Admittedly, we do not have multiple datasources and are not using LRC, so 
your environment is definitely different.

So, if you want to try the snapshot and give me your feedback regarding this issue
or whether any other errors occur in your testing.  I've made a snapshot available here:


Regards,
Brett

On Sat, Dec 10, 2011 at 12:32 AM, Eric Westfall <[hidden email]> wrote:
Hello, we're having an issue that I'm hoping someone on this list may know something about.

We have an application configured which uses two different databases. It's using the transaction management support provided by the spring framework configured for JTA.  Things seems to work well most of the time, but when we run load tests against it we get errors similar to the following occasionally:

* bitronix.tm.internal.BitronixXAException: resource already started on XID a Bitronix XID [737072696E672D62746D0000013423394EA100000E41 : 737072696E672D62746D0000013423394F0100000E4F]
       at bitronix.tm.resource.jdbc.lrc.LrcXAResource.start(LrcXAResource.java:135)
       at bitronix.tm.internal.XAResourceHolderState.start(XAResourceHolderState.java:219)
       at bitronix.tm.internal.XAResourceManager.enlist(XAResourceManager.java:111)
       at bitronix.tm.BitronixTransaction.enlistResource(BitronixTransaction.java:93)
       at bitronix.tm.resource.common.TransactionContextHelper.enlistInCurrentTransaction(TransactionContextHelper.java:70)
       at bitronix.tm.resource.jdbc.JdbcConnectionHandle.enlistResource(JdbcConnectionHandle.java:84)
       at bitronix.tm.resource.jdbc.JdbcConnectionHandle.prepareStatement(JdbcConnectionHandle.java:289)
       at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source)
   ...

Full stack trace here: http://www.pastie.org/2991516

Any ideas on why this might be happening?  It appears that somehow a resource which is "started" is getting reused across multiple concurrent transactions which seems like it shouldn't happen.

Here is our bitronix transaction manager configuration:

bitronix.tm.serverId=spring-btm
bitronix.tm.disableJmx=true
bitronix.tm.timer.defaultTransactionTimeout=3600
bitronix.tm.journal=null
bitronix.tm.timer.gracefulShutdownInterval=0
bitronix.tm.journal.disk.logPart1Filename=target/btm1.tlog
bitronix.tm.journal.disk.logPart2Filename=target/btm2.tlog
bitronix.tm.2pc.warnAboutZeroResourceTransactions=false
bitronix.tm.allowMultipleLrc=true
bitronix.tm.setIgnoreRecoveryFailures=true

Of particular note here, we are using the "LRC" setup with two datasources which is why we have allowMultipleLrc=true

Relevant PoolingDataSource configuration from our spring configuration is as follows:

<property name="useTmJoin" value="true" />
<property name="allowLocalTransactions" value="true" />
<property name="shareTransactionConnections" value="true" />
<property name="enableJdbc4ConnectionTest" value="true" />
<property name="automaticEnlistingEnabled" value="true"/>

Thanks in advance for any help!
Eric



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

  http://xircles.codehaus.org/manage_email




Reply | Threaded
Open this post in threaded view
|

Re: Getting "bitronix.tm.internal.BitronixXAException: resource already started on XID" error

ewestfal
Hi Brett, thanks for following up.  I think after some investigation I'm
leaning toward this being an application issue and not something
inherently wrong in Bitronix.  I wrote a simple little test and verified
that this error happens whenever a javax.sql.Connection gets reused
across multiple concurrent transactions in different threads.  So I
think there is probably something going wrong "above" the tm/pooling layer.

The application is using Spring (version 2.0.8, so it's pretty old) as
well as an old ORM from apache called OJB.  I suspect right now that a
bug in one of these layers is causing the "cross-pollination" of
connections.

If I do think there might still be something going on in Bitronix here,
I will definitely be willing and able to try out that new version and
let you know if it fixes the problem.

Thanks!
Eric


Brett Wooldridge wrote:

> Eric,
>
> Another thing which will be useful whether you're running  btm 2.1.2 or
> the development branch
> is to enable logging.  Bitronix has extensive logging, and usually if we
> can see the logs from
> the time of the error, we can figure out what went wrong (whether it was
> a bitronix bug or a bug
> in some other component in the stack).
>
> Because of the volume of logging from Bitronix, I recommend logging to a
> different file than
> your primary application log.  Also, because of the volume of logging --
> particularly under a
> load test -- I recommend rolling the logs so you don't lose the logs at
> the crucial time of
> failure.
>
> Here is a sample recommended configuration for log4j:
>
> log4j.logger.bitronix.tm <http://log4j.logger.bitronix.tm>=debug, BTM
> log4j.additivity.bitronix.tm <http://log4j.additivity.bitronix.tm>=false
>
> log4j.appender.BTM=org.apache.log4j.RollingFileAppender
> log4j.appender.BTM.File=btm.log
> log4j.appender.BTM.MaxFileSize=10MB
> log4j.appender.BTM.MaxBackupIndex=50
> log4j.appender.BTM.layout=org.apache.log4j.PatternLayout
> log4j.appender.BTM.layout.ConversionPattern=%d{HH:mm:ss,SSS}
> [%-25.25c{1}] [%-25t] %-5p - %m%n
>
> This is going to create a 10MB log file, which rolls 50 times
> (btm.log.1, btm.log.2, etc.).  The [%-25t] part
> is also crucial because it logs the thread name.
>
> What you want to do is run your load test, and if you can, stop the test
> as soon after the error is
> observed as possible.  In your application log, find the exception that
> occurred during the run, and note
> the time (HH:mm:ss,SSS).  Now, start opening btm.log files (btm.log,
> btm.log.1, btm.log.2, etc) and
> find the one that contains the same approximate timestamp as the
> application error (+/- a few milliseconds).
>
> Once you find the corresponding btm log, zip that btm.log AND THE TWO
> NEXT OLDER LOGS together
> with the application log file with the error.  So, if you the error
> timestamp corresponded to logs in the
> btm.log.18 file, zip that file along with btm.log.19 and btm.log.20 files.
>
> You can open a bug and attach the zipped log files and we'll take a look
> at them.
>
> Regards,
> Brett
>
>
> On Sat, Dec 10, 2011 at 6:55 PM, Brett Wooldridge
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     Hi Eric,
>
>     Thanks for using Bitronix.  I'm sure Ludovic will be responding soon
>     regarding this issue, but
>     if you're willing to test the BTM-2.2 development branch, I would be
>     interested to see if this
>     issue has been fixed.
>
>     With BTM-2.2 we're moving from a Java 1.4 codebase to Java 5.  And
>     in the process
>     converting the code to use type-safe collections and more
>     importantly taking advantage of
>     the newer Java 5 concurrent collections and synchronization
>     primitives.  While going
>     through this process, I found at least one theoretical race
>     condition, though not one that
>     had yet been reported.
>
>     I've been running the BTM-2.2 codebase on our pre-production servers
>     for the past
>     two months, and have run several tens of millions of transactions
>     through without
>     issue.  Admittedly, we do not have multiple datasources and are not
>     using LRC, so
>     your environment is definitely different.
>
>     So, if you want to try the snapshot and give me your feedback
>     regarding this issue
>     or whether any other errors occur in your testing.  I've made a
>     snapshot available here:
>
>     http://www.dancernetworks.com/~brettw/btm-2.2.0-SNAPSHOT.jar
>
>     Regards,
>     Brett
>
>     On Sat, Dec 10, 2011 at 12:32 AM, Eric Westfall
>     <[hidden email] <mailto:[hidden email]>> wrote:
>
>         Hello, we're having an issue that I'm hoping someone on this
>         list may know something about.
>
>         We have an application configured which uses two different
>         databases. It's using the transaction management support
>         provided by the spring framework configured for JTA.  Things
>         seems to work well most of the time, but when we run load tests
>         against it we get errors similar to the following occasionally:
>
>         * bitronix.tm.internal.__BitronixXAException: resource already
>         started on XID a Bitronix XID
>         [__737072696E672D62746D0000013423__394EA100000E41 :
>         737072696E672D62746D0000013423__394F0100000E4F]
>                at
>         bitronix.tm.resource.jdbc.lrc.__LrcXAResource.start(__LrcXAResource.java:135)
>                at
>         bitronix.tm.internal.__XAResourceHolderState.start(__XAResourceHolderState.java:__219)
>                at
>         bitronix.tm.internal.__XAResourceManager.enlist(__XAResourceManager.java:111)
>                at bitronix.tm
>         <http://bitronix.tm>.__BitronixTransaction.__enlistResource(__BitronixTransaction.java:93)
>                at
>         bitronix.tm.resource.common.__TransactionContextHelper.__enlistInCurrentTransaction(__TransactionContextHelper.java:__70)
>                at
>         bitronix.tm.resource.jdbc.__JdbcConnectionHandle.__enlistResource(__JdbcConnectionHandle.java:84)
>                at
>         bitronix.tm.resource.jdbc.__JdbcConnectionHandle.__prepareStatement(__JdbcConnectionHandle.java:289)
>                at
>         sun.reflect.__GeneratedMethodAccessor50.__invoke(Unknown Source)
>            ...
>
>         Full stack trace here: http://www.pastie.org/2991516
>
>         Any ideas on why this might be happening?  It appears that
>         somehow a resource which is "started" is getting reused across
>         multiple concurrent transactions which seems like it shouldn't
>         happen.
>
>         Here is our bitronix transaction manager configuration:
>
>         bitronix.tm.serverId=spring-__btm
>         bitronix.tm.disableJmx=true
>         bitronix.tm.timer.__defaultTransactionTimeout=3600
>         bitronix.tm.journal=null
>         bitronix.tm.timer.__gracefulShutdownInterval=0
>         bitronix.tm.journal.disk.__logPart1Filename=target/btm1.__tlog
>         bitronix.tm.journal.disk.__logPart2Filename=target/btm2.__tlog
>         bitronix.tm.2pc.__warnAboutZeroResourceTransacti__ons=false
>         bitronix.tm.allowMultipleLrc=__true
>         bitronix.tm <http://bitronix.tm>.__setIgnoreRecoveryFailures=true
>
>         Of particular note here, we are using the "LRC" setup with two
>         datasources which is why we have allowMultipleLrc=true
>
>         Relevant PoolingDataSource configuration from our spring
>         configuration is as follows:
>
>         <property name="useTmJoin" value="true" />
>         <property name="allowLocalTransactions" value="true" />
>         <property name="__shareTransactionConnections" value="true" />
>         <property name="__enableJdbc4ConnectionTest" value="true" />
>         <property name="__automaticEnlistingEnabled" value="true"/>
>
>         Thanks in advance for any help!
>         Eric
>
>
>
>         ------------------------------__------------------------------__---------
>         To unsubscribe from this list, please visit:
>
>           http://xircles.codehaus.org/__manage_email
>         <http://xircles.codehaus.org/manage_email>
>
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Getting "bitronix.tm.internal.BitronixXAException: resource already started on XID" error

Brett Wooldridge-2
Hi Eric,

That's interesting that you say that.  I'm reluctant to point the finger at a third-party's code but to tell
truth, after your initial post, I started looking at:

org.apache.orb.broker.accesslayer.JdbcAccessImpl
org.apache.orb.broker.accesslayer.StatementManager
org.apache.orb.broker.accesslayer.ConnectionManagerImpl

The ConnectionManagerImpl immediately raised a flag in my mind, as it appeared to be either pooling
it's own connections or trying to maintain some kind of connection-to-local-tx mapping.  Something
like that inherently won't mix well with many JTA.

You may still able about to use Bitronix, but you might try turning off shareTransactionConnections
in Bitronix.  You might also investigate the "eager-release" configuration setting in the apache orb
configuration.  Search for "eager-release" here:


We'll be interested in what you find out.

Regards,
Brett


On Sun, Dec 11, 2011 at 3:59 PM, Eric Westfall <[hidden email]> wrote:
Hi Brett, thanks for following up.  I think after some investigation I'm leaning toward this being an application issue and not something inherently wrong in Bitronix.  I wrote a simple little test and verified that this error happens whenever a javax.sql.Connection gets reused across multiple concurrent transactions in different threads.  So I think there is probably something going wrong "above" the tm/pooling layer.

The application is using Spring (version 2.0.8, so it's pretty old) as well as an old ORM from apache called OJB.  I suspect right now that a bug in one of these layers is causing the "cross-pollination" of connections.

If I do think there might still be something going on in Bitronix here, I will definitely be willing and able to try out that new version and let you know if it fixes the problem.

Thanks!
Eric
Reply | Threaded
Open this post in threaded view
|

Re: Getting "bitronix.tm.internal.BitronixXAException: resource already started on XID" error

ewestfal
Thanks Brett, OJB does have it's own connection pooling support built
in, but we are not actually using that the way we have it configured.
One question about your development version of Bitronix (2.2).  You
don't happen to have that available as a snapshot in a maven repository
somewhere do you?  We use maven on the project so that tends to make it
easier for us if we can just change our dependency version ;)

Right now, we're checking to see if we are getting connections bound
across threads to the Spring Framework's
TransactionSynchronizationManager which is a problem we have had before
in the past (though for reasons we aren't entirely sure of).

Thanks,
Eric

Brett Wooldridge wrote:

> Hi Eric,
>
> That's interesting that you say that.  I'm reluctant to point the finger
> at a third-party's code but to tell
> truth, after your initial post, I started looking at:
>
> org.apache.orb.broker.accesslayer.JdbcAccessImpl
> org.apache.orb.broker.accesslayer.StatementManager
> org.apache.orb.broker.accesslayer.ConnectionManagerImpl
>
> The ConnectionManagerImpl immediately raised a flag in my mind, as it
> appeared to be either pooling
> it's own connections or trying to maintain some kind of
> connection-to-local-tx mapping.  Something
> like that inherently won't mix well with many JTA.
>
> You may still able about to use Bitronix, but you might try turning
> off shareTransactionConnections
> in Bitronix.  You might also investigate the "eager-release"
> configuration setting in the apache orb
> configuration.  Search for "eager-release" here:
>
> http://db.apache.org/ojb/docu/getting-started.html#Sample+project-N104AF 
> <http://db.apache.org/ojb/docu/getting-started.html#Sample+project-N104AF>
>
> We'll be interested in what you find out.
>
> Regards,
> Brett
>
>
> On Sun, Dec 11, 2011 at 3:59 PM, Eric Westfall <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi Brett, thanks for following up.  I think after some investigation
>     I'm leaning toward this being an application issue and not something
>     inherently wrong in Bitronix.  I wrote a simple little test and
>     verified that this error happens whenever a javax.sql.Connection
>     gets reused across multiple concurrent transactions in different
>     threads.  So I think there is probably something going wrong "above"
>     the tm/pooling layer.
>
>     The application is using Spring (version 2.0.8, so it's pretty old)
>     as well as an old ORM from apache called OJB.  I suspect right now
>     that a bug in one of these layers is causing the "cross-pollination"
>     of connections.
>
>     If I do think there might still be something going on in Bitronix
>     here, I will definitely be willing and able to try out that new
>     version and let you know if it fixes the problem.
>
>     Thanks!
>     Eric
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Getting "bitronix.tm.internal.BitronixXAException: resource already started on XID" error

Brett Wooldridge-2
Hi Eric,

The BTM-2.2 snapshots are not currently being published to the repository.  To be honest, I don't know how to make that happen.  Ludovic is the project owner/lead and would need to make the call on whether he wants to put BTM-2.2 out there.  Most of the changes in BTM-2.2 are cosmetic in nature (converting untyped collections to Java 5 typed collections), but there are a few fundamental changes around JDBC4 handling and synchronization (increased concurrency).  Recently Ludovic has had limited time to review incoming changes, and may not have enough of a comfort level to put BTM-2.2 out there.

Having put a lot of time into BTM-2.2, I am obviously biased toward wanting to get it out there so that it can be tested by adventurous developers.  As with any refactor that includes threading changes, the best way to get a comfort level is through having it run in various environments by users -- because obviously we can't desk-test such a large number of databases and development stacks.

As I said, I've been running BTM-2.2 in my company's own pre-production environment under load with high-thread counts and transaction volumes without issue.  We'll see what Ludovic things about putting BTM-2.2 into the snapshot maven repository.

Regards,
Brett

On Mon, Dec 12, 2011 at 6:13 AM, Eric Westfall <[hidden email]> wrote:
Thanks Brett, OJB does have it's own connection pooling support built in, but we are not actually using that the way we have it configured. One question about your development version of Bitronix (2.2).  You don't happen to have that available as a snapshot in a maven repository somewhere do you?  We use maven on the project so that tends to make it easier for us if we can just change our dependency version ;)

Right now, we're checking to see if we are getting connections bound across threads to the Spring Framework's TransactionSynchronizationManager which is a problem we have had before in the past (though for reasons we aren't entirely sure of).

Thanks,
Eric

Brett Wooldridge wrote:
Hi Eric,

That's interesting that you say that.  I'm reluctant to point the finger at a third-party's code but to tell
truth, after your initial post, I started looking at:

org.apache.orb.broker.accesslayer.JdbcAccessImpl
org.apache.orb.broker.accesslayer.StatementManager
org.apache.orb.broker.accesslayer.ConnectionManagerImpl

The ConnectionManagerImpl immediately raised a flag in my mind, as it appeared to be either pooling
it's own connections or trying to maintain some kind of connection-to-local-tx mapping.  Something
like that inherently won't mix well with many JTA.

You may still able about to use Bitronix, but you might try turning off shareTransactionConnections
in Bitronix.  You might also investigate the "eager-release" configuration setting in the apache orb
configuration.  Search for "eager-release" here:

http://db.apache.org/ojb/docu/getting-started.html#Sample+project-N104AF <http://db.apache.org/ojb/docu/getting-started.html#Sample+project-N104AF>


We'll be interested in what you find out.

Regards,
Brett


On Sun, Dec 11, 2011 at 3:59 PM, Eric Westfall <[hidden email] <mailto:[hidden email]>> wrote:

   Hi Brett, thanks for following up.  I think after some investigation
   I'm leaning toward this being an application issue and not something
   inherently wrong in Bitronix.  I wrote a simple little test and
   verified that this error happens whenever a javax.sql.Connection
   gets reused across multiple concurrent transactions in different
   threads.  So I think there is probably something going wrong "above"
   the tm/pooling layer.

   The application is using Spring (version 2.0.8, so it's pretty old)
   as well as an old ORM from apache called OJB.  I suspect right now
   that a bug in one of these layers is causing the "cross-pollination"
   of connections.

   If I do think there might still be something going on in Bitronix
   here, I will definitely be willing and able to try out that new
   version and let you know if it fixes the problem.

   Thanks!
   Eric


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

  http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|

Re: Getting "bitronix.tm.internal.BitronixXAException: resource already started on XID" error

Ludovic Orban-2
Brett,

It definitely is a good idea to upload the BTM-2.2 artifacts into the codehaus snapshot repository.

You should have enough privileges to perform the upload, just edit (or create) $HOME/.m2/settings.xml and add a server element in it with your codehaus credentials:

<settings xmlns="http://maven.apache.org/POM/4.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">

  <servers>
    <server>
      <id>codehaus-nexus-snapshots</id>
      <username>your-xircles-userid</username>
      <password>your-xircles-password</password>
    </server>
  </servers>

</settings>

After that, you should be able to invoke 'mvn deploy' which will upload the artifacts at the right place.

--
Ludovic


2011/12/12 Brett Wooldridge <[hidden email]>
Hi Eric,

The BTM-2.2 snapshots are not currently being published to the repository.  To be honest, I don't know how to make that happen.  Ludovic is the project owner/lead and would need to make the call on whether he wants to put BTM-2.2 out there.  Most of the changes in BTM-2.2 are cosmetic in nature (converting untyped collections to Java 5 typed collections), but there are a few fundamental changes around JDBC4 handling and synchronization (increased concurrency).  Recently Ludovic has had limited time to review incoming changes, and may not have enough of a comfort level to put BTM-2.2 out there.

Having put a lot of time into BTM-2.2, I am obviously biased toward wanting to get it out there so that it can be tested by adventurous developers.  As with any refactor that includes threading changes, the best way to get a comfort level is through having it run in various environments by users -- because obviously we can't desk-test such a large number of databases and development stacks.

As I said, I've been running BTM-2.2 in my company's own pre-production environment under load with high-thread counts and transaction volumes without issue.  We'll see what Ludovic things about putting BTM-2.2 into the snapshot maven repository.

Regards,
Brett

On Mon, Dec 12, 2011 at 6:13 AM, Eric Westfall <[hidden email]> wrote:
Thanks Brett, OJB does have it's own connection pooling support built in, but we are not actually using that the way we have it configured. One question about your development version of Bitronix (2.2).  You don't happen to have that available as a snapshot in a maven repository somewhere do you?  We use maven on the project so that tends to make it easier for us if we can just change our dependency version ;)

Right now, we're checking to see if we are getting connections bound across threads to the Spring Framework's TransactionSynchronizationManager which is a problem we have had before in the past (though for reasons we aren't entirely sure of).

Thanks,
Eric

Brett Wooldridge wrote:
Hi Eric,

That's interesting that you say that.  I'm reluctant to point the finger at a third-party's code but to tell
truth, after your initial post, I started looking at:

org.apache.orb.broker.accesslayer.JdbcAccessImpl
org.apache.orb.broker.accesslayer.StatementManager
org.apache.orb.broker.accesslayer.ConnectionManagerImpl

The ConnectionManagerImpl immediately raised a flag in my mind, as it appeared to be either pooling
it's own connections or trying to maintain some kind of connection-to-local-tx mapping.  Something
like that inherently won't mix well with many JTA.

You may still able about to use Bitronix, but you might try turning off shareTransactionConnections
in Bitronix.  You might also investigate the "eager-release" configuration setting in the apache orb
configuration.  Search for "eager-release" here:

http://db.apache.org/ojb/docu/getting-started.html#Sample+project-N104AF <http://db.apache.org/ojb/docu/getting-started.html#Sample+project-N104AF>


We'll be interested in what you find out.

Regards,
Brett


On Sun, Dec 11, 2011 at 3:59 PM, Eric Westfall <[hidden email] <mailto:[hidden email]>> wrote:

   Hi Brett, thanks for following up.  I think after some investigation
   I'm leaning toward this being an application issue and not something
   inherently wrong in Bitronix.  I wrote a simple little test and
   verified that this error happens whenever a javax.sql.Connection
   gets reused across multiple concurrent transactions in different
   threads.  So I think there is probably something going wrong "above"
   the tm/pooling layer.

   The application is using Spring (version 2.0.8, so it's pretty old)
   as well as an old ORM from apache called OJB.  I suspect right now
   that a bug in one of these layers is causing the "cross-pollination"
   of connections.

   If I do think there might still be something going on in Bitronix
   here, I will definitely be willing and able to try out that new
   version and let you know if it fixes the problem.

   Thanks!
   Eric


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

  http://xircles.codehaus.org/manage_email




Reply | Threaded
Open this post in threaded view
|

Re: Getting "bitronix.tm.internal.BitronixXAException: resource already started on XID" error

ewestfal
Thanks for being willing to throw those snapshots up into the maven
repository.  Though we ended up working up another way to get the code
in.  Unfortunately, it didn't resolve the enlistment error we are seeing
so we are continuing to look for causes of the problem.

Thanks,
Eric

Ludovic Orban wrote:

> Brett,
>
> It definitely is a good idea to upload the BTM-2.2 artifacts into the
> codehaus snapshot repository.
>
> You should have enough privileges to perform the upload, just edit (or
> create) $HOME/.m2/settings.xml and add a server element in it with your
> codehaus credentials:
>
> <settings xmlns="http://maven.apache.org/POM/4.0.0"
>           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>           xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
> http://maven.apache.org/xsd/settings-1.0.0.xsd">
>
>   <servers>
>     <server>
>       <id>codehaus-nexus-snapshots</id>
>       <username>your-xircles-userid</username>
>       <password>your-xircles-password</password>
>     </server>
>   </servers>
>
> </settings>
>
> After that, you should be able to invoke 'mvn deploy' which will upload
> the artifacts at the right place.
>
> --
> Ludovic
>
>
> 2011/12/12 Brett Wooldridge <[hidden email]
> <mailto:[hidden email]>>
>
>     Hi Eric,
>
>     The BTM-2.2 snapshots are not currently being published to the
>     repository.  To be honest, I don't know how to make that happen.
>      Ludovic is the project owner/lead and would need to make the call
>     on whether he wants to put BTM-2.2 out there.  Most of the changes
>     in BTM-2.2 are cosmetic in nature (converting untyped collections to
>     Java 5 typed collections), but there are a few fundamental changes
>     around JDBC4 handling and synchronization (increased concurrency).
>      Recently Ludovic has had limited time to review incoming changes,
>     and may not have enough of a comfort level to put BTM-2.2 out there.
>
>     Having put a lot of time into BTM-2.2, I am obviously biased toward
>     wanting to get it out there so that it can be tested by adventurous
>     developers.  As with any refactor that includes threading changes,
>     the best way to get a comfort level is through having it run in
>     various environments by users -- because obviously we can't
>     desk-test such a large number of databases and development stacks.
>
>     As I said, I've been running BTM-2.2 in my company's own
>     pre-production environment under load with high-thread counts and
>     transaction volumes without issue.  We'll see what Ludovic things
>     about putting BTM-2.2 into the snapshot maven repository.
>
>     Regards,
>     Brett
>
>     On Mon, Dec 12, 2011 at 6:13 AM, Eric Westfall <[hidden email]
>     <mailto:[hidden email]>> wrote:
>
>         Thanks Brett, OJB does have it's own connection pooling support
>         built in, but we are not actually using that the way we have it
>         configured. One question about your development version of
>         Bitronix (2.2).  You don't happen to have that available as a
>         snapshot in a maven repository somewhere do you?  We use maven
>         on the project so that tends to make it easier for us if we can
>         just change our dependency version ;)
>
>         Right now, we're checking to see if we are getting connections
>         bound across threads to the Spring Framework's
>         TransactionSynchronizationMana__ger which is a problem we have
>         had before in the past (though for reasons we aren't entirely
>         sure of).
>
>         Thanks,
>         Eric
>
>         Brett Wooldridge wrote:
>
>             Hi Eric,
>
>             That's interesting that you say that.  I'm reluctant to
>             point the finger at a third-party's code but to tell
>             truth, after your initial post, I started looking at:
>
>             org.apache.orb.broker.__accesslayer.JdbcAccessImpl
>             org.apache.orb.broker.__accesslayer.StatementManager
>             org.apache.orb.broker.__accesslayer.__ConnectionManagerImpl
>
>             The ConnectionManagerImpl immediately raised a flag in my
>             mind, as it appeared to be either pooling
>             it's own connections or trying to maintain some kind of
>             connection-to-local-tx mapping.  Something
>             like that inherently won't mix well with many JTA.
>
>             You may still able about to use Bitronix, but you might try
>             turning off shareTransactionConnections
>             in Bitronix.  You might also investigate the "eager-release"
>             configuration setting in the apache orb
>             configuration.  Search for "eager-release" here:
>
>             http://db.apache.org/ojb/docu/__getting-started.html#Sample+__project-N104AF
>             <http://db.apache.org/ojb/docu/getting-started.html#Sample+project-N104AF>
>             <http://db.apache.org/ojb/__docu/getting-started.html#__Sample+project-N104AF
>             <http://db.apache.org/ojb/docu/getting-started.html#Sample+project-N104AF>>
>
>
>             We'll be interested in what you find out.
>
>             Regards,
>             Brett
>
>
>             On Sun, Dec 11, 2011 at 3:59 PM, Eric Westfall
>             <[hidden email] <mailto:[hidden email]>
>             <mailto:[hidden email] <mailto:[hidden email]>>>
>             wrote:
>
>                Hi Brett, thanks for following up.  I think after some
>             investigation
>                I'm leaning toward this being an application issue and
>             not something
>                inherently wrong in Bitronix.  I wrote a simple little
>             test and
>                verified that this error happens whenever a
>             javax.sql.Connection
>                gets reused across multiple concurrent transactions in
>             different
>                threads.  So I think there is probably something going
>             wrong "above"
>                the tm/pooling layer.
>
>                The application is using Spring (version 2.0.8, so it's
>             pretty old)
>                as well as an old ORM from apache called OJB.  I suspect
>             right now
>                that a bug in one of these layers is causing the
>             "cross-pollination"
>                of connections.
>
>                If I do think there might still be something going on in
>             Bitronix
>                here, I will definitely be willing and able to try out
>             that new
>                version and let you know if it fixes the problem.
>
>                Thanks!
>                Eric
>
>
>         ------------------------------__------------------------------__---------
>         To unsubscribe from this list, please visit:
>
>           http://xircles.codehaus.org/__manage_email
>         <http://xircles.codehaus.org/manage_email>
>
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Getting "bitronix.tm.internal.BitronixXAException: resource already started on XID" error

ewestfal
FYI, my colleague from Cornell University who has been testing this
locally created the following jira including some of the logging that
you requested earlier:

http://jira.codehaus.org/browse/BTM-116

Thanks in advance if you have any insight here!

Eric

Eric Westfall wrote:

> Thanks for being willing to throw those snapshots up into the maven
> repository.  Though we ended up working up another way to get the code
> in.  Unfortunately, it didn't resolve the enlistment error we are seeing
> so we are continuing to look for causes of the problem.
>
> Thanks,
> Eric
>
> Ludovic Orban wrote:
>> Brett,
>>
>> It definitely is a good idea to upload the BTM-2.2 artifacts into the
>> codehaus snapshot repository.
>>
>> You should have enough privileges to perform the upload, just edit (or
>> create) $HOME/.m2/settings.xml and add a server element in it with
>> your codehaus credentials:
>>
>> <settings xmlns="http://maven.apache.org/POM/4.0.0"
>>           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>           xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
>> http://maven.apache.org/xsd/settings-1.0.0.xsd">
>>
>>   <servers>
>>     <server>
>>       <id>codehaus-nexus-snapshots</id>
>>       <username>your-xircles-userid</username>
>>       <password>your-xircles-password</password>
>>     </server>
>>   </servers>
>>
>> </settings>
>>
>> After that, you should be able to invoke 'mvn deploy' which will
>> upload the artifacts at the right place.
>>
>> --
>> Ludovic
>>
>>
>> 2011/12/12 Brett Wooldridge <[hidden email]
>> <mailto:[hidden email]>>
>>
>>     Hi Eric,
>>
>>     The BTM-2.2 snapshots are not currently being published to the
>>     repository.  To be honest, I don't know how to make that happen.
>>      Ludovic is the project owner/lead and would need to make the call
>>     on whether he wants to put BTM-2.2 out there.  Most of the changes
>>     in BTM-2.2 are cosmetic in nature (converting untyped collections to
>>     Java 5 typed collections), but there are a few fundamental changes
>>     around JDBC4 handling and synchronization (increased concurrency).
>>      Recently Ludovic has had limited time to review incoming changes,
>>     and may not have enough of a comfort level to put BTM-2.2 out there.
>>
>>     Having put a lot of time into BTM-2.2, I am obviously biased toward
>>     wanting to get it out there so that it can be tested by adventurous
>>     developers.  As with any refactor that includes threading changes,
>>     the best way to get a comfort level is through having it run in
>>     various environments by users -- because obviously we can't
>>     desk-test such a large number of databases and development stacks.
>>
>>     As I said, I've been running BTM-2.2 in my company's own
>>     pre-production environment under load with high-thread counts and
>>     transaction volumes without issue.  We'll see what Ludovic things
>>     about putting BTM-2.2 into the snapshot maven repository.
>>
>>     Regards,
>>     Brett
>>
>>     On Mon, Dec 12, 2011 at 6:13 AM, Eric Westfall <[hidden email]
>>     <mailto:[hidden email]>> wrote:
>>
>>         Thanks Brett, OJB does have it's own connection pooling support
>>         built in, but we are not actually using that the way we have it
>>         configured. One question about your development version of
>>         Bitronix (2.2).  You don't happen to have that available as a
>>         snapshot in a maven repository somewhere do you?  We use maven
>>         on the project so that tends to make it easier for us if we can
>>         just change our dependency version ;)
>>
>>         Right now, we're checking to see if we are getting connections
>>         bound across threads to the Spring Framework's
>>         TransactionSynchronizationMana__ger which is a problem we have
>>         had before in the past (though for reasons we aren't entirely
>>         sure of).
>>
>>         Thanks,
>>         Eric
>>
>>         Brett Wooldridge wrote:
>>
>>             Hi Eric,
>>
>>             That's interesting that you say that.  I'm reluctant to
>>             point the finger at a third-party's code but to tell
>>             truth, after your initial post, I started looking at:
>>
>>             org.apache.orb.broker.__accesslayer.JdbcAccessImpl
>>             org.apache.orb.broker.__accesslayer.StatementManager
>>             org.apache.orb.broker.__accesslayer.__ConnectionManagerImpl
>>
>>             The ConnectionManagerImpl immediately raised a flag in my
>>             mind, as it appeared to be either pooling
>>             it's own connections or trying to maintain some kind of
>>             connection-to-local-tx mapping.  Something
>>             like that inherently won't mix well with many JTA.
>>
>>             You may still able about to use Bitronix, but you might try
>>             turning off shareTransactionConnections
>>             in Bitronix.  You might also investigate the "eager-release"
>>             configuration setting in the apache orb
>>             configuration.  Search for "eager-release" here:
>>
>>            
>> http://db.apache.org/ojb/docu/__getting-started.html#Sample+__project-N104AF 
>>
>>            
>> <http://db.apache.org/ojb/docu/getting-started.html#Sample+project-N104AF>
>>
>>            
>> <http://db.apache.org/ojb/__docu/getting-started.html#__Sample+project-N104AF 
>>
>>            
>> <http://db.apache.org/ojb/docu/getting-started.html#Sample+project-N104AF>>
>>
>>
>>
>>             We'll be interested in what you find out.
>>
>>             Regards,
>>             Brett
>>
>>
>>             On Sun, Dec 11, 2011 at 3:59 PM, Eric Westfall
>>             <[hidden email] <mailto:[hidden email]>
>>             <mailto:[hidden email] <mailto:[hidden email]>>>
>>             wrote:
>>
>>                Hi Brett, thanks for following up.  I think after some
>>             investigation
>>                I'm leaning toward this being an application issue and
>>             not something
>>                inherently wrong in Bitronix.  I wrote a simple little
>>             test and
>>                verified that this error happens whenever a
>>             javax.sql.Connection
>>                gets reused across multiple concurrent transactions in
>>             different
>>                threads.  So I think there is probably something going
>>             wrong "above"
>>                the tm/pooling layer.
>>
>>                The application is using Spring (version 2.0.8, so it's
>>             pretty old)
>>                as well as an old ORM from apache called OJB.  I suspect
>>             right now
>>                that a bug in one of these layers is causing the
>>             "cross-pollination"
>>                of connections.
>>
>>                If I do think there might still be something going on in
>>             Bitronix
>>                here, I will definitely be willing and able to try out
>>             that new
>>                version and let you know if it fixes the problem.
>>
>>                Thanks!
>>                Eric
>>
>>
>>        
>> ------------------------------__------------------------------__---------
>>         To unsubscribe from this list, please visit:
>>
>>           http://xircles.codehaus.org/__manage_email
>>         <http://xircles.codehaus.org/manage_email>
>>
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email