BTM Spring ActiveMQ number of consumers keeps increasing

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

BTM Spring ActiveMQ number of consumers keeps increasing

James Black
Hello,

we are currently testing BTM and our having some issues.

Basically the number of consumers on the ActiveMQ queue keeps going up and up each second.

Our concurrency is set to 5-10 for our DefaultMessageListener.

What we are using:

Spring 3.0.5
Hibernate 3.6.2
ActiveMQ 5.5.0
BTM 2.1.0

I have configured everything as per the BTM docs.

<bean id="txManager"
          class="org.springframework.transaction.jta.JtaTransactionManager"> 
        <property name="transactionManager" ref="btmTransactionManager" />
        <property name="userTransaction" ref="btmTransactionManager" />
    </bean>

    <bean id="btmConfig" factory-method="getConfiguration" class="bitronix.tm.TransactionManagerServices">
         <property name="serverId" value="spring-api-btm" />
    </bean>

    <bean id="btmTransactionManager" factory-method="getTransactionManager"
          class="bitronix.tm.TransactionManagerServices" depends-on="btmConfig" destroy-method="shutdown" />

    <bean id="entityManagerFactory"
          depends-on="btmConfig"
          class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
        <property name="persistenceUnitName" value="platformEntityManager" />
        <property name="dataSource" ref="dataSource" />
    </bean>

I left out the data source config but it is the correct PoolingDataSource, now the JMS stuff.

<bean id="btmConnectionFactory"
          class="bitronix.tm.resource.jms.PoolingConnectionFactory"
          init-method="init" destroy-method="close">
         
        <property name="className" value="org.apache.activemq.ActiveMQXAConnectionFactory" />
        <property name="uniqueName" value="XAJMS" />
        <property name="minPoolSize" value="40" />
        <property name="maxPoolSize" value="40" />
        <property name="driverProperties">
            <props>
                <prop key="brokerURL">tcp://localhost:61616</prop>
            </props>
        </property>
    </bean>

<bean id="MessageListenerAdapter1"  -->
           class="org.springframework.jms.listener.adapter.MessageListenerAdapter">
        <property name="delegate" ref="targetEntity" /> 
         <property name="defaultListenerMethod" value="doStuff" /> 
         <property name="messageConverter" ref="MyMessageConverter" /> 
     </bean> 

    <bean id="MessageListenerContainer1"
          class="org.springframework.jms.listener.DefaultMessageListenerContainer"
          depends-on="txManager">
        <property name="connectionFactory" ref="btmConnectionFactory"/>
        <property name="destination" ref="jmsQueue1"/>
        <property name="messageListener" ref="MessageListenerAdapter1" />
        <property name="concurrency" value="5-10" />
        <property name="transactionManager" ref="txManager" />
        <property name="sessionTransacted" value="true" /> 
        <property name="clientId" value="LIST1" />
    </bean>

Any help much appreciated, if you fix it I will send you beer tokens through Paypal :>

James
Reply | Threaded
Open this post in threaded view
|

Re: BTM Spring ActiveMQ number of consumers keeps increasing

snicoll
Hi,

I've seen quite a few quirks with ActiveMQ, especially when running with XA. I actually discourage you from trying to integrate ActiveMQ XA support with BTM. For instance, see [1]. 

Can you try with LRC? The related issue in BTM has a configuration snippet [2]

Is there any reason why you want 5 message listeners at all time? If you don't need to ramp up very fast, you can use 1-10 which will only use one listener when your queue is empty. The drawback of course is that it might take some time before you get up to 10 threads if a lot of messages are being sent in a short period of time.

If ActiveMQ is not a strong choice for you, I also encourage you to try HornetQ [3]. It has been working quite well for us

HTH,
S.

[3] http://www.jboss.org/hornetq/

On Fri, Jun 3, 2011 at 1:31 PM, James Black <[hidden email]> wrote:

Hello,

we are currently testing BTM and our having some issues.

Basically the number of consumers on the ActiveMQ queue keeps going up and
up each second.

Our concurrency is set to 5-10 for our DefaultMessageListener.

What we are using:

Spring 3.0.5
Hibernate 3.6.2
ActiveMQ 5.5.0
BTM 2.1.0

I have configured everything as per the BTM docs.

<bean id="txManager"
         class="org.springframework.transaction.jta.JtaTransactionManager">
       <property name="transactionManager" ref="btmTransactionManager" />
       <property name="userTransaction" ref="btmTransactionManager" />
   </bean>

   <bean id="btmConfig" factory-method="getConfiguration"
class="bitronix.tm.TransactionManagerServices">
        <property name="serverId" value="spring-api-btm" />
   </bean>

   <bean id="btmTransactionManager" factory-method="getTransactionManager"
         class="bitronix.tm.TransactionManagerServices"
depends-on="btmConfig" destroy-method="shutdown" />

   <bean id="entityManagerFactory"
         depends-on="btmConfig"

class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
       <property name="persistenceUnitName" value="platformEntityManager"
/>
       <property name="dataSource" ref="dataSource" />
   </bean>

I left out the data source config but it is the correct PoolingDataSource,
now the JMS stuff.

<bean id="btmConnectionFactory"
         class="bitronix.tm.resource.jms.PoolingConnectionFactory"
         init-method="init" destroy-method="close">

       <property name="className"
value="org.apache.activemq.ActiveMQXAConnectionFactory" />
       <property name="uniqueName" value="XAJMS" />
       <property name="minPoolSize" value="40" />
       <property name="maxPoolSize" value="40" />
       <property name="driverProperties">
           <props>
               <prop key="brokerURL">tcp://localhost:61616</prop>
           </props>
       </property>
   </bean>

<bean id="MessageListenerAdapter1"  -->

class="org.springframework.jms.listener.adapter.MessageListenerAdapter">
       <property name="delegate" ref="targetEntity" />
        <property name="defaultListenerMethod" value="doStuff" />
        <property name="messageConverter" ref="MyMessageConverter" />
    </bean>

   <bean id="MessageListenerContainer1"

class="org.springframework.jms.listener.DefaultMessageListenerContainer"
         depends-on="txManager">
       <property name="connectionFactory" ref="btmConnectionFactory"/>
       <property name="destination" ref="jmsQueue1"/>
       <property name="messageListener" ref="MessageListenerAdapter1" />
       <property name="concurrency" value="5-10" />
       <property name="transactionManager" ref="txManager" />
       <property name="sessionTransacted" value="true" />
       <property name="clientId" value="LIST1" />
   </bean>

Any help much appreciated, if you fix it I will send you beer tokens through
Paypal :>

James
--
View this message in context: http://old.nabble.com/BTM-Spring-ActiveMQ-number-of-consumers-keeps-increasing-tp31764702p31764702.html
Sent from the Bitronix Transaction Manager mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|

Re: BTM Spring ActiveMQ number of consumers keeps increasing

Dennis Brakhane-2
On Sat, Jun 4, 2011 at 9:55 AM, Stephane Nicoll
<[hidden email]> wrote:
> Hi,
> I've seen quite a few quirks with ActiveMQ, especially when running with XA.
> I actually discourage you from trying to integrate ActiveMQ XA support with
> BTM. For instance, see [1].

I'd also like to generally caution you about using ActiveMQ, I know of a project
where it was used and they had serious problems with it. Since I wasn't directly
involved, I cannot say what they were exactly, but they seemed to be
pretty severe.
YMMV, but it seems they were not the only one having problems:

http://goodstuff.im/activemq-not-ready-for-prime-time

I'm not saying that you cannot use AMQ, but if you are building are
high-availability
heavy-traffic application, you should be aware that you will run into
problems. You should
at least invest some time evaluating alternatives.

Just my 2 cents

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: BTM Spring ActiveMQ number of consumers keeps increasing

MarkFletcher
In reply to this post by James Black
Hello James,
 Sorry to restart an old thread. We're running into the same issues in a similar environment. The number of consumers goes up every second and the messages are getting stuck in the queue without being passed onto the Spring Message Handlers. Were you able to figure out what was the problem? Your response will be much appreciated.

Thanks.

James Black wrote
Hello,

we are currently testing BTM and our having some issues.

Basically the number of consumers on the ActiveMQ queue keeps going up and up each second.

Our concurrency is set to 5-10 for our DefaultMessageListener.

What we are using:

Spring 3.0.5
Hibernate 3.6.2
ActiveMQ 5.5.0
BTM 2.1.0

I have configured everything as per the BTM docs.

<bean id="txManager"
          class="org.springframework.transaction.jta.JtaTransactionManager"> 
        <property name="transactionManager" ref="btmTransactionManager" />
        <property name="userTransaction" ref="btmTransactionManager" />
    </bean>

    <bean id="btmConfig" factory-method="getConfiguration" class="bitronix.tm.TransactionManagerServices">
         <property name="serverId" value="spring-api-btm" />
    </bean>

    <bean id="btmTransactionManager" factory-method="getTransactionManager"
          class="bitronix.tm.TransactionManagerServices" depends-on="btmConfig" destroy-method="shutdown" />

    <bean id="entityManagerFactory"
          depends-on="btmConfig"
          class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
        <property name="persistenceUnitName" value="platformEntityManager" />
        <property name="dataSource" ref="dataSource" />
    </bean>

I left out the data source config but it is the correct PoolingDataSource, now the JMS stuff.

<bean id="btmConnectionFactory"
          class="bitronix.tm.resource.jms.PoolingConnectionFactory"
          init-method="init" destroy-method="close">
         
        <property name="className" value="org.apache.activemq.ActiveMQXAConnectionFactory" />
        <property name="uniqueName" value="XAJMS" />
        <property name="minPoolSize" value="40" />
        <property name="maxPoolSize" value="40" />
        <property name="driverProperties">
            <props>
                <prop key="brokerURL">tcp://localhost:61616</prop>
            </props>
        </property>
    </bean>

<bean id="MessageListenerAdapter1"  -->
           class="org.springframework.jms.listener.adapter.MessageListenerAdapter">
        <property name="delegate" ref="targetEntity" /> 
         <property name="defaultListenerMethod" value="doStuff" /> 
         <property name="messageConverter" ref="MyMessageConverter" /> 
     </bean> 

    <bean id="MessageListenerContainer1"
          class="org.springframework.jms.listener.DefaultMessageListenerContainer"
          depends-on="txManager">
        <property name="connectionFactory" ref="btmConnectionFactory"/>
        <property name="destination" ref="jmsQueue1"/>
        <property name="messageListener" ref="MessageListenerAdapter1" />
        <property name="concurrency" value="5-10" />
        <property name="transactionManager" ref="txManager" />
        <property name="sessionTransacted" value="true" /> 
        <property name="clientId" value="LIST1" />
    </bean>

Any help much appreciated, if you fix it I will send you beer tokens through Paypal :>

James
Reply | Threaded
Open this post in threaded view
|

Re: BTM Spring ActiveMQ number of consumers keeps increasing

James Black
Hi Mark,

unfortunately we did not solve our problem with BTM and AMQ.  We were testing BTM because of issues with Atomikos and AMQ.  In the end we decided to go with no XA transactions and deal with any extra work required in our own code.

To be honest I think that this has worked out better for us.

Sorry I cannot be of help :<

James

MarkFletcher wrote
Hello James,
 Sorry to restart an old thread. We're running into the same issues in a similar environment. The number of consumers goes up every second and the messages are getting stuck in the queue without being passed onto the Spring Message Handlers. Were you able to figure out what was the problem? Your response will be much appreciated.

Thanks.

James Black wrote
Hello,

we are currently testing BTM and our having some issues.

Basically the number of consumers on the ActiveMQ queue keeps going up and up each second.

Our concurrency is set to 5-10 for our DefaultMessageListener.

What we are using:

Spring 3.0.5
Hibernate 3.6.2
ActiveMQ 5.5.0
BTM 2.1.0

I have configured everything as per the BTM docs.

<bean id="txManager"
          class="org.springframework.transaction.jta.JtaTransactionManager"> 
        <property name="transactionManager" ref="btmTransactionManager" />
        <property name="userTransaction" ref="btmTransactionManager" />
    </bean>

    <bean id="btmConfig" factory-method="getConfiguration" class="bitronix.tm.TransactionManagerServices">
         <property name="serverId" value="spring-api-btm" />
    </bean>

    <bean id="btmTransactionManager" factory-method="getTransactionManager"
          class="bitronix.tm.TransactionManagerServices" depends-on="btmConfig" destroy-method="shutdown" />

    <bean id="entityManagerFactory"
          depends-on="btmConfig"
          class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
        <property name="persistenceUnitName" value="platformEntityManager" />
        <property name="dataSource" ref="dataSource" />
    </bean>

I left out the data source config but it is the correct PoolingDataSource, now the JMS stuff.

<bean id="btmConnectionFactory"
          class="bitronix.tm.resource.jms.PoolingConnectionFactory"
          init-method="init" destroy-method="close">
         
        <property name="className" value="org.apache.activemq.ActiveMQXAConnectionFactory" />
        <property name="uniqueName" value="XAJMS" />
        <property name="minPoolSize" value="40" />
        <property name="maxPoolSize" value="40" />
        <property name="driverProperties">
            <props>
                <prop key="brokerURL">tcp://localhost:61616</prop>
            </props>
        </property>
    </bean>

<bean id="MessageListenerAdapter1"  -->
           class="org.springframework.jms.listener.adapter.MessageListenerAdapter">
        <property name="delegate" ref="targetEntity" /> 
         <property name="defaultListenerMethod" value="doStuff" /> 
         <property name="messageConverter" ref="MyMessageConverter" /> 
     </bean> 

    <bean id="MessageListenerContainer1"
          class="org.springframework.jms.listener.DefaultMessageListenerContainer"
          depends-on="txManager">
        <property name="connectionFactory" ref="btmConnectionFactory"/>
        <property name="destination" ref="jmsQueue1"/>
        <property name="messageListener" ref="MessageListenerAdapter1" />
        <property name="concurrency" value="5-10" />
        <property name="transactionManager" ref="txManager" />
        <property name="sessionTransacted" value="true" /> 
        <property name="clientId" value="LIST1" />
    </bean>

Any help much appreciated, if you fix it I will send you beer tokens through Paypal :>

James
Reply | Threaded
Open this post in threaded view
|

Re: BTM Spring ActiveMQ number of consumers keeps increasing

MarkFletcher
Thank you, James. I appreciate your response. We had some problems with Atomikos too with the database that we were using. :-)

James Black wrote
Hi Mark,

unfortunately we did not solve our problem with BTM and AMQ.  We were testing BTM because of issues with Atomikos and AMQ.  In the end we decided to go with no XA transactions and deal with any extra work required in our own code.

To be honest I think that this has worked out better for us.

Sorry I cannot be of help :<

James

MarkFletcher wrote
Hello James,
 Sorry to restart an old thread. We're running into the same issues in a similar environment. The number of consumers goes up every second and the messages are getting stuck in the queue without being passed onto the Spring Message Handlers. Were you able to figure out what was the problem? Your response will be much appreciated.

Thanks.

James Black wrote
Hello,

we are currently testing BTM and our having some issues.

Basically the number of consumers on the ActiveMQ queue keeps going up and up each second.

Our concurrency is set to 5-10 for our DefaultMessageListener.

What we are using:

Spring 3.0.5
Hibernate 3.6.2
ActiveMQ 5.5.0
BTM 2.1.0

I have configured everything as per the BTM docs.

<bean id="txManager"
          class="org.springframework.transaction.jta.JtaTransactionManager"> 
        <property name="transactionManager" ref="btmTransactionManager" />
        <property name="userTransaction" ref="btmTransactionManager" />
    </bean>

    <bean id="btmConfig" factory-method="getConfiguration" class="bitronix.tm.TransactionManagerServices">
         <property name="serverId" value="spring-api-btm" />
    </bean>

    <bean id="btmTransactionManager" factory-method="getTransactionManager"
          class="bitronix.tm.TransactionManagerServices" depends-on="btmConfig" destroy-method="shutdown" />

    <bean id="entityManagerFactory"
          depends-on="btmConfig"
          class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
        <property name="persistenceUnitName" value="platformEntityManager" />
        <property name="dataSource" ref="dataSource" />
    </bean>

I left out the data source config but it is the correct PoolingDataSource, now the JMS stuff.

<bean id="btmConnectionFactory"
          class="bitronix.tm.resource.jms.PoolingConnectionFactory"
          init-method="init" destroy-method="close">
         
        <property name="className" value="org.apache.activemq.ActiveMQXAConnectionFactory" />
        <property name="uniqueName" value="XAJMS" />
        <property name="minPoolSize" value="40" />
        <property name="maxPoolSize" value="40" />
        <property name="driverProperties">
            <props>
                <prop key="brokerURL">tcp://localhost:61616</prop>
            </props>
        </property>
    </bean>

<bean id="MessageListenerAdapter1"  -->
           class="org.springframework.jms.listener.adapter.MessageListenerAdapter">
        <property name="delegate" ref="targetEntity" /> 
         <property name="defaultListenerMethod" value="doStuff" /> 
         <property name="messageConverter" ref="MyMessageConverter" /> 
     </bean> 

    <bean id="MessageListenerContainer1"
          class="org.springframework.jms.listener.DefaultMessageListenerContainer"
          depends-on="txManager">
        <property name="connectionFactory" ref="btmConnectionFactory"/>
        <property name="destination" ref="jmsQueue1"/>
        <property name="messageListener" ref="MessageListenerAdapter1" />
        <property name="concurrency" value="5-10" />
        <property name="transactionManager" ref="txManager" />
        <property name="sessionTransacted" value="true" /> 
        <property name="clientId" value="LIST1" />
    </bean>

Any help much appreciated, if you fix it I will send you beer tokens through Paypal :>

James
Reply | Threaded
Open this post in threaded view
|

Re: BTM Spring ActiveMQ number of consumers keeps increasing

Ludovic Orban-2
As said earlier, ActiveMQ's XA support is very poor and should be avoided if you can.

LRC might be the easiest thing to do, otherwise your options are:

 - tune the Spring DefaultMessageListenerContainer (it caching can confuse ActiveMQ a lot) and maybe disable BTM PoolingDataSource's cacheProducersConsumers property
 - use a different JMS server
 - avoid XA.

2011/10/5 MarkFletcher <[hidden email]>

Thank you, James. I appreciate your response. We had some problems with
Atomikos too with the database that we were using. :-)