Description
A common operation for input strings is to tokenize, or separate, strings with a delimiter of the
user’s choice. You are to write a string tokenizer function for the STL string class. The
prototype is
void tokenizer(const string input, const string delimitier, list &tokenList);
The tokens extracted from the input based on the delimiter will be stored in the list tokenList.
This function should be the contents of the file named Tokenizer.cpp. You should also create a
file named Tokenizer.h containing the prototype for the tokenizer function. You are also to
write a program to test your tokenzier function. This program should be in a separate file
called TokenizerTester.cpp You will submit all three files. The grading will consist of using both
your test program and one of my own.
Your test program should prompt the user for an input string, then prompt the user for a
delimiter string; after that, the tokens should be printed, one per line.
Example execution (user input shown underlined!):
Enter an input string: It–is–going–to–be–a–long–season
Enter the delimiter string: —
Tokens:
It
is
going
to
be
a
long
season
Due date: September 13.
Suggestion: investigate the STL string and list classes for useful functions (Google anyone?)
