This is a project I have been working on and off of for about a year now. It's basically a tokenizer on steroids.

Instead of supplying a list of single character delimiters, you can actually specify entire strings as token separators. Along with this, you can attach additional logic to the token, such as
-- Should the token be returned? (Think space, new line, ect... removal)
-- Change Token Sets (Allows you to switch what tokens are used at runtime. This is very useful for things like comment removal in code)
-- Callbacks (When a token is found, a function is called. It's nicer then a huge switch statement)

Other features include
-- Built in File I/O, with a specified buffer size (how many bytes of the file to keep in memory at runtime)
-- Various Binary Functions
-- Multiple Token Sets
-- Modifiable at runtime
-- Portable (If ANSI C is supported, it should work)
-- Documentation!
-- -- Lots of code comments
-- -- 70 KB HTML document (NOT from Doxygen or similar)
-- -- 4 Sample drivers

My main development version of the code is in C; however, there is a C++ class version as well.

No executables are included; however, it is very simple to compile. Just pick a driver, and compile it with the Parser (one .h and .c file).

Any comments, suggestions, or reports on warnings/errors will appreciated.

Updated the Parser. There were several bug fixes, and you can now pass a parameter to the callback.

Performance Delta:
GNU: 6% Faster then 7.1
MS: No difference

Memory Delta:
Global Data: Increased by sizeof(int) bytes
Token: Increased by sizeof(void *) bytes

Edit: Cleaned up a few compiler warnings. Cosmetic only.
Edit: Ha! Compiles under DeSmet C again. Also, I did find a slight bug. In any case, this should be the last update for this version.

This post has been edited by Cerolobo: 24 Jun, 2008 - 09:22 PM

I'm going to move this to the Discuss Projects Forum so it will get the attention it deserves

EDIT: Whoops Didn't realize it already was, sorry

Well, I finally broke down and picked up a SourceForge account. Further versions of the library can be picked up here

http://sourceforge.net/projects/ceroparser/

I will continue to post updates here. Regardless, I tend to only get a desire to work on it once every few months, so don't expect any updates any time soon.

About time.