Continue of Bridges Construction Site (1/3): Asynchronous – synchronous bridge.
———————————————-
Hello again, colleagues!
Let’s continue to review solutions for the problem of integration in heterogeneous interfaces landscape using bridges.
Uff.. most complex part of this post is finished! 🙂
——————-
See also:
Bridges Construction Site (1/3): Asynchronous – synchronous bridge.
Bridges Construction Site (3/3): SAP PI bridges – “exotic” and recommendations.
——————-
- 3. Sync-Async Bridge and its modules – RequestOnewayBean, WaitResponseBean and NotifyResponseBean.
- 3.1 Synchronous-asynchronous bridge with modules in the Sender Communication Channel.
- 3.2 Example of synchronous – asynchronous bridge with modules in Sender Communication Channel.
- 3.3 Synchronous – asynchronous bridge with modules in Receiver Communication Channel.
- 3.4 Example : Synchronous – asynchronous bridge with modules in Receiver Communication Channel.
3. Sync-Async Bridge and its modules – RequestOnewayBean, WaitResponseBean and NotifyResponseBean.
Let’s think about a new problem: we are running the process in the one system and want to send some information to the external system. It’s simple. But we also need immediate confirmation of of successful reception (or an error message).
Our external system works in asynchronous manner only, ie it takes the input by one service (it can be a file, asynchronous SOAP, SQL, HTTP, etc.), and send an acknowledgment by another service after while.
Let me remind you the scheme of such interfaces:
Figure 1 : Synchronous – Asynchronous Bridge
Sync-async bridge using following modules:
• RequestOnewayBean is responsible for converting synchronous message to asynchronous.
• WaitResponseBean – leaves a synchronious communication channel open and waits for the response. When receiving asynchronous response – converts it to a synchronous message and sends it back to the original system. If the answer does not come within the specified time – sends an error message to the source system.
• NotifyResponseBean is used instead of the standard adapter module in the asynchronous response sender channel to transfer the response message directly to WaitResponseBean ( bypassing the processing in the messaging subsystem of PI).
RequestOnewayBean and WaitResponseBean can be used in a sender communication channel or in a receiver communication channel.
RequestOnewayBean must be inserted before the standard adapter module (usually CallSAPAdapter); WaitResponseBean – after.
For using modules in the Communication Channel you should go to the tab «Module» and enter the following values:
«Module Key» can be anything – so long as it is unique and the keys would be difficult to confuse – it will help us later to determine the parameters of modules.
Further settings depends on the placement of modules: in a sender communication channel or in a receiver communication channel.
But we have one more problem connected to synchronous-asynchronous bridge.
Imagine that several messages are going through the bridge simultaneously. So, at some point few queries are waiting for responses. Some response is coming from the target system – but how we know which of pending channels must be used to send a specific answer?
Figure 2 : the task of determining the recipient of asynchronous response
Here comes new entity – a correlation. Correlation is an identifier , the “key” , which is held on the correspondence between the response and awaiting queue.
The term ” correlation ” can also be found in the Sync-Async Bridge in ccBPM – there may be a correlation using not only one but a set of identifiers .
So the task is to “remember” some id during the sending of asynchronous message and check this id during the receiving an answer:
Figure 3: Determination of the recipient using the asynchronous response correlation
The correlationID parameter (defined in the PI message header) is used to correlate a response from asynchronous system. The value in correlationID must be equal to the message GUID of the synchronous request message . This parameter belongs to a group of Adapter-Specific Message Attributes and can be changed using the standard module – DynamicConfigurationBean.
Another example in SAP standard documentation («Configuring the Async / Sync Bridge Using the JMS Adapter») correlate messages using standard settings of communication channels JMS (“Set PI Conversation ID” parameter).
In other kinds of adapters similar standard parameter is missing and you have to make some tweaks – in particular , to save Message GUID and fill correlationID parameter using PI or with the help of the target system. In the example below (3.2) , you can find one of the ways to work with correlations in file adapter.
3.1 Synchronous-asynchronous bridge with modules in the Sender Communication Channel.
If we use Sender Communication Channel for bridge’s modules, the scheme of message processing would be next:
Figure 4: The logic of the synchronous-asynchronous bridge with modules in the Sender CC.
- Receiving a synchronous message from external system via communication channel, sending it to the module RequestOnewayBean. The module switch the message type from synchronous to asynchronous by changing the message header.
- Depending on the parameter passThrough, module pass the message down the chain of modules or directly sending it to the message processing subsystem of PI (skipping step 3).
- Message processing by the standard adapter module, transfering it to PI message processing subsystem.
- Sending a request to asynchronous communication channel.
- Asynchronous call to the external system
- The response from external system is coming to async sender communication channel
- Return the response to the NotifyResponseBean, which directly transmits a response to WaitResponseBean, bypassing the messaging subsystem PI.
- WaitResponseBean changes message type from asynchronous to synchronous and returns it to the source system.
Module parameters for RequestOnewayBean when placing it in the Sender Communication Channel:
Module parameters for WaitResponseBean when placing it in the Sender Communication Channel:
NotifyResponseBean module in this configuration must be located in the recieving communication channel for a response from asynchronous system.
The module must be used instead of the standard adapter module.
NotifyResponseBean have following module parameters:
Once we set up all the necessary modules , we need to configure the request and response correlation.
To do so, we need keep the message GUID somehow and pass it to the target system. The system, in turn, should return a response with a the GUID in some part of message. When the answer is coming, we need to fill out the parameter in the message header responsible for the correlation – correlationId (Dynamic Configuration header section) with stored GUID value.
Solution can be different for each of the adapter types. I’ll show you one of possible file adapter solution in the example below.
After setting up the correlation the only thing to do is to create routing rules linking source synchronous interface to the target asynchronous interface.
For the routing of response it is sufficient to create a Sender Agreement only – everything else will be complete by the NotifyResponseBean module.
3.2 Example of synchronous – asynchronous bridge with modules in Sender Communication Channel.
Let’s take a look to the specific example.
Business case: we have an external client who gives us an information about new books available in the store using a synchronous web-service .
PI should receive this information and send it to our database. This DB is pretty old, so the Interface of DB is based on the file exchange. The status of operation is also returned as a file, so PI should get it and return it to the client.
Figure 5: Example diagram.
To implement this interface, we need to create the following development objects:
Figure 6: Development objects in the Integration Repository
Please note the operations interfaces mapping – it synchronous, but “one-way” – it works in one direction only. To create it we need to use a “trick” with a temporary changing of receiver interface type, like it was done in the mapping from the previous article about the asynchronous-synchronous bridges.
After that, create all necessary routing objects in the Integration Builder:
Figure 7: Configuration objects
The question may arise – how the routing of response can be done by Sender Agreement only?
You can configure full routing rule for the answer or make response routing via Integrated Configuration. But in this particular case it does not make sense – the answer still will be transferred to the module NotifyResponseBean and directly to waiting module WaitResponseBean after that, bypassing the messages processing subsystem of PI.
Now let’s put the modules into place.
Synchronous SOAP communication channel:
Figure 8: Synchronous SOAP Sender communication channel
Please don’t forget to set Quality Of Service = Best Effort, ie identify the channel as synchronous.
Then add modules RequestOnewayBean and WaitResponseBean:
Fig. 9: Modules in SOAP Sender synchronous communication channel
Then set up a file communication channel to submit a request to the database:
Fig.: 10: Asynchronous File communication channel for DB request.
Please note the configuration of file name:
Filename is formed by a variable that have Message ID value. Filename = message GUID without any extension.
Why do so? Thus, in our example, we store the GUID of request message to use it in correlation later. In reality, you can do it same way. or you can pass GUID in the message body. The main thing – target system must get it, save it and return it in an accessible manner to PI along with the response.
Next step: set up a file communication channel for the response:
Fig. 11: File communication channel to get a response from the target system.
We assume that the system produces a response in another local directory on the PI-server as an XML-file.
The name of the file must be the same GUID as request.
Modules configuration:
Fig.: 12: Modules in asynchronous file sender comm. channel
Standard module DynamicConfigurationBean writes the file name to the correlationId parameter in the header of PI message – ie fills correlation for further processing of the response by the bridge.
We are using NotifyResponseBean module instead of standard adapter module CallSAPAdapter. NotifyResponseBean forwards the response directly to the waiting module WaitResponseBean. If nobody expects the answer with such correlation value – an error will be written to the log.
Well, that’s all settings now, time to check our bridge.
Form a WSDL for web-service (use context menu on Integrated Configuration -> Display WSDL). Then, use WSDL to form and send SOAP request using any suitable SOAP-tool (I used a freeware tool SOAPUI).
Fig.: 13: Sending a synchronous request from SOAPUI
After sending the file appears in the directory with the name of the message GUID and request data inside.
Fig.: 14: File request received by the target system
Now let’s simulate the recipient DB system.
Edit file – put the response data in it.
Fig.: 15: File with response
Move it into the directory books/status. If we were quick enough, the file will be processed by the PI and the response in SOAPUI will look like this:
Fig.: 16: Response successfully received by SOAP-client
Synchronous-asynchronous bridge successfully built.
All fine, but ..
Just imagine: our customer came and asked us to add one more field to the answer – timestamp.
We don’t have response mapping, the target system also “can’t” provide us response in new format.
What should we do?
Fig.17: Additional development objects in Integration Repository
Also define the mapping, which will convert the response from the system BS_ExtDB in new response format for BS_WebClient.
Figure 18: Add timestamp to the response
Now configure the routing in Integration Directory:
• create a new “pseudo” File communication channel CC_Dummy_file_receiver and move bridge’s modules DynamicConfigurationBean and NotifyResponseBean there, with all configuration parameters;
• restore the module CallSAPAdapter in communication channel SS_ExtDB_FileSender;
• remove Sender Agreement for SS_ExtDB_FileSender;
• create and configure the Integrated Configuration for response routing from BS_ExtDB to BS_WebClient.
Fig.: 19: Additional configuration objects
Check our bridge as before and we must get the right answer:
Fig.: 20: Request and response with timestamp.
Ok, the customer is satisfied now, our synchronous-asynchronous bridge works perfect.
Congratulations! 🙂
3.3 Synchronous – asynchronous bridge with modules in Receiver Communication Channel.
There are nothing specific in this version of bridge comparing to the previous. There are the same three modules: two of them – in asynchronous Receiver Communication Channel, transmitting a request to target system; one more – in asynchronous Sender Communication Channel for proceessing the response.
Figure 21 : The logic of the synchronous-asynchronous bridge with modules in the receiver communication channel.
- Receiving the synchronous request from external system.
- Transfer the message to PI messaging system.
- Message processing in PI, transfer it to a synchronous communication channel – to RequestOnewayBean module. The module switch the message processing type from synchronous to asynchronous by changing the message header.
- Message goes to the standard adapter module.
- Asynchronous call to external system.
- Asynchronous response from external system. NotifyResponseBean processing the module instead of standard adapter module.
- Transfer request to WaitResponseBean module, correlation check.
- The module switch the message processing type from asynchronous to synchronous, transfer the message to messaging sub-system of PI.
- Response message is going to synchronous communication channel.
- Response to external system.
Modules options are independent from the placement .
Please note: there is another information in SAP help, there are more options for modules when placing in a Receiver Channel. I have not found any meaningful application of these parameters.
RequestOnewayBean parameters:
WaitResponseBean parameters:
NotifyResponseBean parameters:
You also should take care about correlation.
3.4 Example : Synchronous – asynchronous bridge with modules in Receiver Communication Channel.
Let’s rebuild our example from 3.2.
Figure 22: Development objects in Integration Repository
At this time we need two synchronous interfaces – SI_WebClient_SaveBook_sync and SI_ExtDB_SaveBook_sync, as well as mapping between them-OM_SaveBook_WebClient_to_ExtDB. We also need asynchronous interface SI_ExtDB_Status (should be based on the same message type as the response in SI_ExtDB_SaveBook_sync) – we will use it for response from an asynchronous system.
Configure routing in Integration Directory as follows:
Fig.: 23: Configuration settings, Integration Directory
Remove all modules from SOAP sender communication channel and move them to file receiver communication channel:
Fig.: 24: asynchronous file receiver communication channel for request.
Fig.: 25: asynchronous file receiver communication channel for response.
Reconfigure Integrated Configuration – we have new communication channels for sender and receiver, new interface and mapping; all we have from the previous example – target system only.
To get the file with response, we need only the Sender Agreement connected to the channel CC_ExtDB_File_Sender_WithModule and interface SI_ExtDB_Status.
Start SOAP test tool (SOAPUI in my case) and send test request. You should have a file with request in the file system.
Replace its content with the answer, move it to the folder with responses, wait for PI processing – and you should get following picture:
Fig.: 26: bridge at work – request and response in SOAPUI.
If your picture looks the same – congratulations! You have just built live sync-async bridge! 🙂
—————————————————————–
See also:
Bridges Construction Site (1/3): Asynchronous – synchronous bridge.
Bridges Construction Site (3/3): SAP PI bridges – “exotic” and recommendations.
—————————————————————–
With best regards,
Alexey Petrov
Thanks Alexey petrov . well explained . Thanks for sharing …!!!
Hi Alexey,
Nice Blog!! I have a query, appreciate if you could help.
Is there anyway we could set correlation in response async message if Receiver is ABAP Proxy in SAP ECC. I know SAP ECC is synchronous system, but for PPO we went with approach of having ECC as asynchronous system. Our scenario is SOAP sender (sync) – ABAP Proxy(SOAP) as Receiver.
Thanks,
Miten
Hi Miten,
try to go this way:
1) Get message ID in inbound proxy
or–xi-message-header
2) Save it somethere.
3) Pass it back via client proxy – use additional field in the message structure cause it’s not possible to change message header in client proxy.
4) Use mapping or Dynamic Configuration Bean to put saved ID into correlationId.
With best regards,
Alex
Hi Alexey,
We mapped Message id to Inbound Proxy. In Response mapping we assigned Message id to Adapter attribute and then used Dynamic Configuration Bean in Receiver Agreement of Response.
It works well. Thank you.
Regards,
Miten
Hi Miten,
you are welcome!
Thanks for sharing the result. 🙂
With best regards,
Alex
Hi Alexey,
excellent series of articles, thank you.
Please add the link to the third blog in this one:
Bridges Construction Site (1/3): Asynchronous – synchronous bridge.
Best regards,
Andy.
Hi Andy,
thanks for the advice!
I’ve added cross-links to all 3 posts.
With best regards,
Alex
Alexey petrov
Hello, very interesting your article but I have a question another way to get the correlation ID for the sender channel.
Thanks
Hello Erick,
the main task is to save message GUID and fill CorrelationID parameter in the message header. This can be done in the sender channel too.
Most adapters (except JMS) have no standard tools for it.
So this would be custom solution – with standard module DynamicConfigurationBean, own module, message mapping or something else.
In this particular case with file adapter – if I need to use another way – I would think about saving the GUID into the message body.
With best regards,
Alex
Hello Alexey Pitroff.
Please could you tell me how you would use the message ID in the message body, this point is not clear to me. Please explain to me your example.
Thanks.
Hello Alexey,
I also use Sync Async Bridge and did the trick for the OM. Sync SOAP Sender, Async IDoc Receiver (and some hidden stuffs to do the correlation), but I’m having problems with SAP PI’s processing, cause it forces to do a Response Mapping. Other than that, the Sync SI, when transported, does not overwrite the Async settings because of conflicts (SAP Note: 1580750). Did you manage to solve that issue?
This is already resolved.
Great information Alexey Pitroff.
I wonder if you anyone can shed some light on how to make this work with
SOAP ABAP Proxy (Sync) – > PI -> SOAP Webservice (Async)
I’m using the Get Weather operation at
This all works great synchronously.
I’ve followed the steps above but most blogs all use the FILE adapter as it’s easier to demonstrate but with a SOAP receiver the message get sent but because I’ve used the sync/async pattern no response comes back and it times out.
I suppose the issue is SOAP async means there’s no response unless there is some sort of callback mechanism that could point it back to a SOAP Sender….but that’s me thinking out loud.
Can you or anyone give some guidance on how to realise this scenario…get the response form the SOAP service that works synchronously to work asynchronously as part of the sync/async pattern?
Thanks.
Hi Peter,
Good day! Is there another SOAP Web Service that responds with your ‘Request Message’?
Request Message = ABAP Proxy to SOAP
Response Message = SOAP to ABAP Proxy
If yes then you should set an Async – Sync Bridge with your request flow’s receiver channel, set the Sender Interface and Namespace of the message expected to be received there. After that, create an ICo or Classical Configuration using those sender interface and namespace with Virtual Receiver – your virtual receiver system should be the sender system of request flow.
If not, I don’t think you need to use S/A bridge since its designed to have the response flow. Check with your ABAP team if they need a response and that’s when you’ll need Sync / Async Brdige, if not, change the Service Interface to Async and let your ABAP team regenerate the proxy on their side.
Best Regards,
Nica
Thanks Genica Bocalan for the prompt response!
” Is there another SOAP Web Service that responds with your ‘Request Message’?” – No there isn’t sadly. I do like your design though. If we did this though you’d never get the response back to the ABAP system would you as it would be an async proxy?
“Check with your ABAP team if they need a response and that’s when you’ll need Sync / Async Bridge” – Yes the response from the service is required as it needs to be acted upon.
So I’m still stuck with how to get the response back from the service and I’m wondering if the service itself doesn’t fit the scenario. Would a true async web service have some call back mechanism where it would know to direct a response to a PI Sender channel??
Thanks,
Peter
Hello Peter,
There will be no response if the receiver was an Async SOAP interface (it has HTTP 200 but only a blank soap envelope) but you can do something like this link below.
The only problem if you’re going to align this reference with your design is, whenever PI splits the message into more than 1 message (which will be like Sender Proxy to Receiver SOAP and Receiver File setup), the message ID changes which will also be the problem of closing the S/A Bridge – because you need to send back the ‘correlation ID’ for PI to identify that response message was the pair of a request message. So better create a file somewhere in PI’s file directory in the mapping part of your build, don’t add another target message in the ICo.
This is case to case, you can use this for ‘acknowledgement’ only, but if you need the receiver’s response, then they should design another interface for that, or better yet, send the response back to the original SOAP interface.
This is not the best practice and this poses performance issues, so if you still can change the design, propose Async Proxies. Async Outbound Request and Async Inbound Response. In PI’s request design, Async Outbound Proxy to Async SOAP and Async Inbound Proxy.
SYNC ASYN Bridge but the receiver is IDOC