title: Segfault resulting from strdup and strtok
body: <p>I've been assigned a homework from my college professor and I seem to have found some strange behavior of <code>strtok</code></p>
  
  <p>Basically, we have to parse a CSV file for my class, where the number of tokens in the CSV is known and the last element may have extra <code>","</code> characters.</p>
  
  <p>An example of a line:</p>
  
  <pre><code>Hello,World,This,Is,A lot, of Text
  </code></pre>
  
  <p>Where the tokens should be output as </p>
  
  <pre><code>1. Hello
  2. World
  3. This
  4. Is
  5. A lot, of Text
  </code></pre>
  
  <p>For this assignment we <em>MUST</em> use <code>strtok</code>. Because of this I found on some other SOF post that using <code>strtok</code> with an empty string (or passing <code>"\n"</code> as the second argument) results in reading until the end of the line.  This is perfect for my application since the extra commas always appear in the last element. </p>
  
  <p>I've created this code which works:</p>
  
  <pre><code>#include &lt;string.h&gt;
  #include &lt;stdlib.h&gt;
  #include &lt;stdio.h&gt;
  
  #define NUM_TOKENS 5
  
  const char *line = "Hello,World,This,Is,Text";
  
  char **split_line(const char *line, int num_tokens)
  {
      char *copy = strdup(line);
  
      // Make an array the correct size to hold num_tokens
      char **tokens = (char**) malloc(sizeof(char*) * num_tokens);
  
      int i = 0;
      for (char *token = strtok(copy, ",\n"); i &lt; NUM_TOKENS; token = strtok(NULL, i &lt; NUM_TOKENS - 1 ? ",\n" : "\n"))
      {
          tokens[i++] = strdup(token);
      }
  
      free(copy);
  
      return tokens;
  }
  
  int main()
  {
      char **tokens = split_line(line, NUM_TOKENS);
  
      for (int i = 0; i &lt; NUM_TOKENS; i++)
      {
          printf("%s\n", tokens[i]);
          free(tokens[i]);
      }
  }
  </code></pre>
  
  <p>Now this works and should get me full credit but I hate this ternary that shouldn't be needed: </p>
  
  <pre><code>token = strtok(NULL, i &lt; NUM_TOKENS - 1 ? ",\n" : "\n");
  </code></pre>
  
  <p>I'd like to replace the method with this version: </p>
  
  <pre><code>char **split_line(const char *line, int num_tokens)
  {
      char *copy = strdup(line);
  
      // Make an array the correct size to hold num_tokens
      char **tokens = (char**) malloc(sizeof(char*) * num_tokens);
  
      int i = 0;
      for (char *token = strtok(copy, ",\n"); i &lt; NUM_TOKENS - 1; token = strtok(NULL, ",\n"))
      {
          tokens[i++] = strdup(token);
      }
  
      tokens[i] = strdup(strtok(NULL, "\n"));
  
      free(copy);
  
      return tokens;
  }
  </code></pre>
  
  <p>This tickles my fancy much nicer since it is much easier to see that there is a final case. You also get rid of the strange ternary operator. </p>
  
  <p>Sadly though, this segfaults! I can't for the life of me figure out why.</p>
  
  <p>Edit: Add some output examples:</p>
  
  <pre><code>[11:56:06] gravypod:test git:(master*) $ ./test_no_fault 
  Hello
  World
  This
  Is
  Text
  [11:56:10] gravypod:test git:(master*) $ ./test_seg_fault 
  [1]    3718 segmentation fault (core dumped)  ./test_seg_fault
  [11:56:14] gravypod:test git:(master*) $ 
  </code></pre>