Search this site


Metadata

Articles

Projects

Presentations

Chasing down a bug while using strtok.

While coding tonight, I spent *at least* 20 minutes trying to figure out why my program was crashing in strtok_r. Turns out I forgot that strtok (and strtok_r) modify the string in-place (with null bytes), and I was passing in a string literal.

In C, "string literals" are stored in a special read-only bit of the program, so if you try to modify them your program crashes. I was calling a function several levels up with a string literal trying to test functionality and it kept crashing with sigbus. Wee. Lesson relearned ;)

Eliminiating special cases in strtok loops

strtok has a "first case" and "other case" usage. The first time you call strtok, you pass it the string. Future calls must pass NULL for this same session. This leads to this kind of code:
void foo(char *mystr) {
  char *tok;

  tok = strtok(mystr, " ");
  while (tok != NULL) {
    // Do something with tok

    tok = strtok(NULL, " ");
  }
}
Notice the above duplicate code. You can use pointers properly and achieve this same result with only one line using strtok:
void foo(char *mystr) {
  char *tok;
  char *strptr = mystr;

  while ( (tok = strtok(strptr, " ")) != NULL ) {
    if (strptr != NULL)
      strptr = NULL;
    // Do something with tok
  }
}
This method lets you still invoke both first and nonfirst cases of strtok but you only have one line of code using strtok, making your code more maintainable and readable. This way has the great benefit of being able to use 'continue' inside your loop and you still move on to the next token.