title: How can I improve the efficiency of my simple file-splitting program
body: <p>I have a simple program that reads a .txt file, and then splits it up into many files of "pMaxRows" number of rows. These .txt files are huge - some are nearly 25Gb. Right now it is not running fast enough for my liking, I feel that there should be a way to improve the efficiency by maybe reading/writing multiple lines at once, but I am not very experienced with vb.net streamreader/streamwriter. </p>
  
  <p>Code is below:</p>
  
  <pre><code>Public Sub Execute(ByVal pFileLocation As String, _
                     ByVal pMaxRows As Int32)
  
      Dim sr As IO.StreamReader
      Dim Row As String
      Dim SourceRowCount As Int64
      Dim TargetRowCount As int64
      Dim TargetFileNumber As Int32
      ''Does the file exist in that location?
      If IO.File.Exists(pFileLocation) = False Then
          Throw New Exception("File does not exist at " &amp; pFileLocation)
      End If
  
      ''Split FileLocation into FileName and Folder Location
      Dim arrFileLoc() As String = pFileLocation.Split("\")
      Dim i As Integer = arrFileLoc.Length - 1
      Dim FileName As String = arrFileLoc(i)
      Dim FileLocationLength As Integer = pFileLocation.Length
      Dim FileNameLength As Integer = FileName.Length
      Dim Folder As String = pFileLocation.Remove(FileLocationLength - FileNameLength, FileNameLength)
  
  
  
      ''Read the file
      sr = New IO.StreamReader(pFileLocation)
      SourceRowCount = 0
      TargetRowCount = 0
      TargetFileNumber = 1
  
  
      ''Create First Target File Name
      Dim TargetFileName As String
      TargetFileName = TargetFileNumber &amp; "_" &amp; FileName
  
      ''Open streamreader and start reading lines
      Do While Not sr.EndOfStream
  
  
          ''if it hits the target number of rows: 
          If (TargetRowCount = pMaxRows) Then
  
              ''Advance target file number
              TargetFileNumber += 1
              ''Create New file with target file number
              TargetFileName = TargetFileNumber &amp; "_" &amp; FileName
  
              ''Set target row count back to 0
              TargetRowCount = 0
  
          End If
          ''Read line
          Row = sr.ReadLine()
  
          ''Write line
          Using sw As New StreamWriter(Folder &amp; TargetFileName, True)
              sw.WriteLine(Row)
          End Using
  
          SourceRowCount += 1
          TargetRowCount += 1
  
      Loop
  End Sub
  </code></pre>
  
  <p>Anyone have any suggestions? Even directing me to the right place if this has been answered before would be much appreciated </p>