Mulesoft Parallel processing-2 (Splitter,Collection Splitter, Chunk Splitter)

I described in my previous blog about parallel processing, in this blog also I am continuing my previous blog but adding new flavor of parallel processing in Mulesoft.
Splitter flow control splits a message into separate fragments and then sends these fragments parallel and concurrent to the next message processor in the flow. Segments are identified based on an expression parameter, usually written in Mule Expression Language (MEL), but other formats can be employed also. There are three ways we can spilt Mulesoft message.

Splitter — Splitter can split all types of data like Object, XML, JSON, and Payload based on MEL (Mule expression language) and processes each split into individual thread. .

Collection Splitter – If input type is collection, then collection splitter split data based on collection and process each element in individual thread.

Chunk Splitter – Chunk of splitter spilts message into chunk of bytes based on user input and processes each chunk of bytes in individual thread.

Since Collection splitter is one of the most usable splitter in Mulesoft. So I am showing example about Collection splitter.
In this example HasMap is coming as payload. Collection Splitter splits HasMap into different threads (limiting max 50 threads) and processes these threads in parallel.

 Flow diagrame to implement parallel processing through Mulesoft Collection Splitter
splitter

Mulesoft code for parallel processing (Splitter)

<flow name="Vanrish-processFlow" processingStrategy="allow50Threads">

    <logger message="*** Starting Vanrish-processFlow ***" category="edi-vanrish-process" level="INFO" doc:name="Flow Start Log"/>

    <set-payload value="#[map-payload:processing]" doc:name="Payload Processing"/>
    <set-variable variableName="numberOfMessages" value="#[payload.entrySet().size()]" doc:name="Variable"/>
    <logger message="Processing #[flowVars['numberOfMessages']] entities" level="INFO" doc:name="logger-status go to database"/>
    <splitter doc:name="Collection Splitter" expression="#[payload.entrySet()]"/>
    <vm:outbound-endpoint exchange-pattern="one-way" path="VanrishVM" doc:name="VM"/>

    <logger message="*** Ending Vanrish-processFlow ***" category="edi-Vanrish-process" level="INFO" doc:name="Flow End Log"/>
</flow>

<flow name="Vanrish_Splitter_Demo" processingStrategy="allow50Threads">
    <vm:inbound-endpoint exchange-pattern="one-way" path="VanrishVM" doc:name="VM"/>
    <logger message="Company Canonical Start Time -&gt; #[server.dateTime]" level="INFO" doc:name="Company Logger"/>
    <flow-ref name="vanrishMsgPrivateFlow" doc:name="companyMsgPrivateFlow"/>
</flow>

Java Profiler is to show how Mulesoft Splitter splits collection message into multiple threads and process data
splitter-profiler

This entry was posted in Java, mulesoft, SOA by Rajnish Kumar. Bookmark the permalink.
Rajnish Kumar
Follow me

Rajnish Kumar

Entrepreneur, Associate MuleSoft Developer certified and Java Programmer Sun certified with over 19 years in Project Management, Architecture of Enterprise application and application development . Complete Software Life Cycle Experience for Multi-Tier Systems, involving Scope, Specifications, Architecture, Design, Estimation, Parallel Iterative Development, Integration, Testing, Release, Revision, and Maintenance. Extensively worked on IOT, Microservices, APIs,SOA application, Cloud, Amazon AWS, Big Data, Analytics, Artificial intelligence and Security(Application Security, Web Security, Data Security)

2 thoughts on “Mulesoft Parallel processing-2 (Splitter,Collection Splitter, Chunk Splitter)

  1. Hello Rajnish
    You are doing good work …. I am also part of similar work of SOA but at quite a less experience ….. I would really like to work with you and understand the things better ….
    Regards,
    Varun S

  2. Dear Rajnish,
    This article about Splitter was very good , you have written step by step and it was very easy to follow and implement.
    I would really like to meet you and work with you

    Thanks
    Raj

Leave a Reply

Your email address will not be published. Required fields are marked *