Oracle Service Bus: transport thread blocking evidences

The common case scenario for a service exposed via Oracle Service Bus is to receive a message, validate/enrich/transform it, call a service provider and then do some additional work with the response from the provider before returning it to the caller. The initial steps are grouped inside a request pipeline; the message then goes through a route to the provider, and the final processing is done by a response pipeline before sending it back to the caller:

proxy.struct.png
Usual structure of an OSB service

The threading model used by Oracle Service Bus states that the request pipeline must execute on a thread (the Inbound Request Message Thread) and the response pipeline must use another thread to process the response (properly named the Outbound Response Message Thread). The route is done by a transport provider, which can or cannot use the request thread to wait for the provider’s response.

When the transport holds the request thread until it gets a response, it is called a blocking transport. Conversely, if the transport puts the call and releases the thread, it’s called a non-blocking transport. Some of the transports are listed below, with info about their support to non-blocking calls:

non-blocking.list.png

The table above is available from Oracle Service Bus 10g’s documentation, but the newest one, 12c, doesn’t have it (link here). This may have to do with the introduction of JCA Transports, but we can prove our point anyway. So, as per documentation HTTP transport is non-blocking, meaning it doesn’t keep a thread blocked while it waits for a response from the service provider. In order to check if this is (still) true I tried a couple scenarios, described below.

  • 1. Testing HTTP transport with a Route action

This is the most common use for a proxy, so I tried it first. I created a syncronous BPEL with just a wait time inside it:

non-blocking.bpel.png
Delayed response flow

Then, I mapped the endpoint with a Business Service inside OSB and created a Proxy Service routing to the Business. Everything in place, I started a few instances and went to SOA’s Enterprise Manager. Sure enough, the instances were running (and the completed one shows the time it took to finish, 30 seconds):

Delayed.Instances

Then, I checked the executing threads from OSB’s WebLogic Administration Console and found no activity (no threads running), as expected. I also put a few dozen instances to spin using SOAP UI’s and the behaviour didn’t change – a lot of instances running at BPEL, no threads waiting at OSB.

Conclusion: it still is a non-blocking transport.

  • 2. Testing HTTP transport with a Service Callout action

Even though the HTTP transport is non-blocking, if you use it inside a Service Callout action the behaviour is somewhat different, as stated by the documentation (same links I mentioned before):

Service Callout: The pipeline processor blocks the thread until the response arrives asynchronously. The blocked thread then resumes execution of the pipeline. The purpose is to bind variables that can later be used in pipeline actions to perform business logic. Therefore, these actions must block so that the business logic can be performed before the response comes back..

To test this, I removed the Route from the Proxy Service and added a Service Callout to the Request pipeline:

non-blocking.callout.png
Service Callout

Then I started a few instances and watched the execution threads. There they were. A thread dump from one of them shows that it is locked waiting for a response:

"[ACTIVE] ExecuteThread: '22' for queue: 'weblogic.kernel.Default (self-tuning)'" Id=110 WAITING on java.lang.Object@3db8e907
	at java.lang.Object.wait(Native Method)
	-  waiting on java.lang.Object@3db8e907
	at java.lang.Object.wait(Object.java:502)
	at com.bea.wli.sb.pipeline.SynchronousCallback.waitForResponse(SynchronousCallback.java:254)
	at com.bea.wli.sb.pipeline.PipelineContextImpl.dispatchSync(PipelineContextImpl.java:457)
	at stages.transform.runtime.WsCalloutRuntimeStep$WsCalloutDispatcher.dispatch(WsCalloutRuntimeStep.java:1396)
	at stages.transform.runtime.WsCalloutRuntimeStep.processMessage(WsCalloutRuntimeStep.java:245)
	at com.bea.wli.sb.stages.StageMetadataImpl$WrapperRuntimeStep.processMessage(StageMetadataImpl.java:384)
	at com.bea.wli.sb.pipeline.components.PipelineStage.processMessage(PipelineStage.java:89)
	at com.bea.wli.sb.pipeline.PipelineContextImpl.execute(PipelineContextImpl.java:902)
	at com.bea.wli.sb.pipeline.components.Pipeline.processMessage(Pipeline.java:153)
	at com.bea.wli.sb.pipeline.PipelineContextImpl.execute(PipelineContextImpl.java:902)
	at com.bea.wli.sb.pipeline.components.PipelineNode.doRequest(PipelineNode.java:64)
	at com.bea.wli.sb.pipeline.components.Node.processMessage(Node.java:82)
	at com.bea.wli.sb.pipeline.PipelineContextImpl.execute(PipelineContextImpl.java:902)
	at com.bea.wli.sb.pipeline.components.Router.processMessage(Router.java:205)
	at com.bea.wli.sb.pipeline.MessageProcessor.processRequest(MessageProcessor.java:87)
	at com.bea.wli.sb.pipeline.dispatcher.PipelineDispatcher$1.run(PipelineDispatcher.java:139)
	at com.bea.wli.sb.pipeline.dispatcher.PipelineDispatcher$1.run(PipelineDispatcher.java:136)
	at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:368)
	at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:163)
	at com.bea.wli.sb.util.security.SecurityUtils.executeAs(SecurityUtils.java:102)
	at com.bea.wli.sb.security.WLSSecurityContextService.executeAs(WLSSecurityContextService.java:55)
	at com.bea.wli.sb.pipeline.dispatcher.PipelineDispatcher.dispatch(PipelineDispatcher.java:134)
	at com.bea.wli.sb.service.handlerchain.handlers.ServiceInvoker.dispatch(ServiceInvoker.java:156)
	at com.bea.wli.sb.service.handlerchain.handlers.AbstractHandler.dispatch(AbstractHandler.java:132)
	at com.bea.wli.sb.service.handlerchain.handlers.UpdateInboundOperationalStatistics.dispatch(UpdateInboundOperationalStatistics.java:75)
	at com.bea.wli.sb.service.handlerchain.handlers.AbstractHandler.dispatch(AbstractHandler.java:132)
	at com.bea.wli.sb.service.handlerchain.handlers.ComputeOperation.dispatch(ComputeOperation.java:110)
	at com.bea.wli.sb.service.handlerchain.handlers.AbstractHandler.dispatch(AbstractHandler.java:132)
	at com.bea.wli.sb.service.handlerchain.handlers.InboundMessageContentHandler.dispatch(InboundMessageContentHandler.java:170)
...

I copied just the start of the trace as it has relevant information for our scenario – last line shows the Inbound Message Handler receiving the message and passing it down to Pipeline Dispatcher / Stage / Node Processor, eventually reaching the Callout Dispatcher that makes the request and then waits for a response as expected.

Conclusion: HTTP Transport is still non-blocking but when invoked by a Service Callout action it will block a thread.

 

So, if you don’t want to get into a thread starvation scenario because your pipelines have a lot of Service Callouts happening in parallel, don’t use it unless it’s absolutely necessary.

If there’s a lot of those actions inside a pipeline then you may have a design problem, not a threading one – I saw a few developers doing this instead of using the more logical Route construct and when challenged about their “architectural decision” the answer was “well, it works this way, so it should be fine.”. A really strong argument indeed.

Cheers!

Advertisements

One thought on “Oracle Service Bus: transport thread blocking evidences

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s