MQ Listener [ADA.600.1007] – Data disappearing from Queue

Few days back we faced a very strange issue. Whenever source application was putting data to MQ, data was getting consumed but nothing was coming into webMethods server. There used to be no data left in MQ however we were not able to see anything for listener document and no instance for interface was getting triggered. We were able to see errors [ADA.600.1007] for the listener in server error logs. Below is the error message which we were seeing in logs.

MQ ERROR –

Mq_Adapter.listeners:receive  java.lang.NullPointerException at com.wm.adapter.wmmqadapter.connection.wmMQMessage.getJMSHeader(wmMQMessage.java:1634)
Stack trace data … 0f969420-a307-11e0-b119-e96436d8b712  NULL  0f969420-a307-11e0-b119-e96436d8b712
Mq_Adapter.listeners:receive  [ADA.600.1007] 1007
null
Stack trace data … 0f969420-a307-11e0-b119-e96436d8b712  NULL  0f969420-a307-11e0-b119-e96436d8b712
2011-06-30 07:08:48 EDT    java.lang.NullPointerException at com.wm.adapter.wmmqadapter.connection.wmMQMessage.getJMSHeader(wmMQMessage.java:1634)
Stack trace data … 0f969420-a307-11e0-b119-e96436d8b712  0f969420-a307-11e0-b119-e96436d8b712  538baa10-a309-11e0-826c-fdb750e73640
2011-06-30 07:08:47 EDT    [ADA.600.1007] 1007
null
Stack trace data … 0f969420-a307-11e0-b119-e96436d8b712  0f969420-a307-11e0-b119-e96436d8b712  538baa10-a309-11e0-826c-fdb750e73640
Mq_Adapter.listeners:receive  java.lang.NullPointerException at com.wm.adapter.wmmqadapter.connection.wmMQMessage.getJMSHeader(wmMQMessage.java:1634)

For troubleshooting purpose, we created a dummy put service in webMethods which was pushing the same external message to same MQ to which external application was trying to put. To our surprise when message was put through dummy service it was consumed well and everything was getting processed in webMethods absolutely fine.

Also that messages from external application was getting processed fine in DEV environment. Problem was environment specific. That ruled out any issue in our code as it was working fine in DEV.

We started looking at the configuration and difference between DEV and UAT environments. At last we were able to figure out the problem. The issue was that MQ adapter fix was missing in our UAT environment. We recently updated to wM8 from 712 and Fix Patch MQS_6-5_Fix8 was not applied to the new UAT servers. Becuase of this fix data from external application was not getting through. though it was not happening with all the interface.

webMethods do behave very weird some time.

Next time if you also facing the same kind of issue, please check if you have got all latest fixes from SAG.

Please feel free to share your thoughts and experiences. If you have faced similar issues, please do share your experience in the comments section.