Question: I am trying to parse a large text file and split it up into single words using strtok. The delimiters remove all special characters, whitespace, and new lines. For some reason when I printf() it, it only prints the first word and a bunch of (null) for the rest.

```cpp
    ifstream textstream(textFile);
    string textLine;
    while (getline(textstream, textLine))
    {
        struct_ptr->numOfCharsProcessedFromFile[TESTFILEINDEX] += textLine.length() + 1;
        char *line_c = new char[textLine.length() + 1]; // creates a character array the length of the line
        strcpy(line_c, textLine.c_str());               // copies the line string into the character array
        char *word = strtok(line_c, delimiters);        // removes all unwanted characters
        while (word != nullptr && wordCount(struct_ptr->dictRootNode, word) > struct_ptr->minNumOfWordsWithAPrefixForPrinting)
        {
            MyFile << word << ' ' << wordCount(struct_ptr->dictRootNode, word) << '\n'; // writes each word and number of times it appears as a prefix in the tree
            word = strtok(NULL, delimiters);                                            // move to next word
            printf("%s", word);
        }
    }

```

Answer: 
Rather than jumping through the hoops necessary to use `strtok`, I'd write a little replacement that works directly with strings, without modifying its input, something on this general order:

```cpp
std::vector<std::string> tokenize(std::string const &input, std::string const &delims = " ") {
    std::vector<std::string> ret;
    int start = 0;

    while ((start = input.find_first_not_of(delims, start)) != std::string::npos) {
        auto stop = input.find_first_of(delims, start+1);
        ret.push_back(input.substr(start, stop-start));
        start = stop;
    }
    return ret;
}

```

At least to me, this seems to simplify the rest of the code quite a bit:

```cpp
std::string textLine;
while (std::getline(textStream, textLine)) {
    struct_ptr->numOfCharsProcessedFromFile[TESTFILEINDEX] += textLine.length() + 1;
    auto words = tokenize(textLine, delims);
    for (auto const &word : words) {
        MyFile << word << ' ' << wordCount(struct_ptr->dictRootNode, word) << '\n';
        std::cout << word << '\n';
    }
}

```

This also avoids (among other things) the massive memory leak you had, allocating memory every iteration of your loop, but never freeing any of it.
As @j23 pointed out, your `printf` is in the wrong location.

As @Jerry-Coffin points out, there are more c++-ish and modern ways to accomplish, what you try to do. Next to avoiding mutation, you can also avoid copying the words out of the text string. (In my code below, we read line by line, but if you know your whole text fits into memory, you could as well read the whole content into a `std::string`.)

So, using `std::string_view` avoids to perform extra copies, it being just something like a pointer into your string and a length.

Here, how it looks like, for a use case, where you need not store the words in another data structure - some kind of one-pass processing of words:

```
#include <iostream>
#include <fstream>
#include <string>
#include <string_view>
#include <cctype>

template <class F>
void with_lines(std::istream& stream, F body) {
  for (std::string line; std::getline(stream,line);) {
    body(line);
  }
}

template <class F>
void with_words(std::istream& stream, F body) {
  with_lines(stream,[&body](std::string& line) {
    std::string_view line_view{line.cbegin(),line.cend()};
    while (!line_view.empty()) {
      // skip whitespaces
      for (; !line_view.empty() && isspace(line_view[0]);
       line_view.remove_prefix(1));
      size_t position = 0;
      for (; position < line_view.size() &&
         !isspace(line_view[position]);
       position++);
      if (position > 0) {
        body(line_view.substr(0,position));
        line_view.remove_prefix(position);
      }
    }
  });
}

int main (int argc, const char* argv[]) {
  size_t word_count = 0;
  std::ifstream stream{"input.txt"};
  if(!stream) {
    std::cerr
      << "could not open file input.txt" << std::endl;
    return -1;
  }
  with_words(stream, [&word_count] (std::string_view word) {
    std::cout << word_count << " " << word << std::endl;
    word_count++;
  });
  std::cout
    << "input.txt contains "
    << word_count << " words."
    << std::endl;
  return 0;
}

```