Question: I have a stream that watch output of multi file in a directory, process data and put it to HDFS. Here is my stream creat command: 

```
stream create --name fileHdfs --definition "file --dir=/var/log/supervisor/ --pattern=tracker.out-*.log --outputType=text/plain | logHdfsTransformer | hdfs --fsUri=hdfs://192.168.1.115:8020 --directory=/data/log/appsync --fileName=log --partitionPath=path(dateFormat('yyyy/MM/dd'))" --deploy

```

Problem is source:file module send all data read from file to log processing module instead of one line each turn, becase of that, payload string have millions of char, i can't process it. Ex:

```
--- PAYLOAD LENGTH---- 9511284

```

Please tell me how to read line by line when use source:file module, thanks !!!

Answer: 
I know that this may be late, but for any googlers out there looking for the solution:

Even though there is no module or option that does it automatically, it's as simple as adding a splitter that separates the incoming message into multiple outgoing messages.

Please note that you have to decide between using \n and \r\n. Check your files to see what they're using.

Example:

```
stream create --name filetest --definition "file --outputType=text/plain --dir=/tmp/examplefiles/| splitter --expression=payload.split('\\n') | log" --deploy

```

Cheers!
Spring Integration has a FileSplitter that splits a text file into lines. You can use this to create a custom processor module, let's call it file-split:

1. Create file-split.xml file with the following content:

```
<?xml version="1.0" encoding="UTF-8"?>
beans xmlns="http://www.springframework.org/schema/beans"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns:int="http://www.springframework.org/schema/integration"
   xsi:schemaLocation="http://www.springframework.org/schema/beans        
   http://www.springframework.org/schema/beans/spring-beans.xsd
   http://www.springframework.org/schema/integration 
   http://www.springframework.org/schema/integration/spring-integration.xsd">

<int:splitter input-channel="input" output-channel="output">
        <bean class="org.springframework.integration.file.splitter.FileSplitter">
                <constructor-arg value="false"/>
        </bean>
</int:splitter>

<int:channel id="output"/> 
</beans>

```
2. Copy the file to ${XD\_HOME}/xd/modules/processor/file-split/config/ (create the path if needed)
3. Sample usage: 

```
stream create --name splitFile --definition "file --dir=/data --ref=true | file-split | log" --deploy

```

You can further customize the module to take more options if needed.