Offset Separatorclass offset_separator The offset_separator class is an implementation of the TokenizerFunction concept that can be used with the tokenizer class to break text up into tokens. The offset_separator breaks a sequence of Char's into strings based on a sequence of offsets. For example, if you had the string "12252001" and offsets (2,2,4) it would break the string into 12 25 2001. Here is an example. Example// simple_example_3.cpp #include<iostream> #include<boost/tokenizer.hpp> #include<string> int main(){ using namespace std; using namespace boost; string s = "12252001"; int offsets[] = {2,2,4}; offset_separator f(offsets, offsets+3); tokenizer<offset_separator> tok(s,f); for(tokenizer<offset_separator>::iterator beg=tok.begin(); beg!=tok.end();++beg){ cout << *beg << "\n"; } }
Construction and UsageThe offset_separator has 1 constructor of interest. (The default constructor is just there to make some compilers happy). The declaration is below template<typename Iter> offset_separator(Iter begin,Iter end,bool bwrapoffsets = true, bool breturnpartiallast = true)
To use this class, pass an object of it anywhere a TokenizerFunction is required. If you default constructruct the object, it will just return every character in the parsed sequence as a token. (ie it defaults to an offset of 1, and bwrapoffsets is true).
Model ofRevised 25 December, 2006 Copyright © 2001 John R. Bandela Distributed under the Boost Software License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) |