Definitions
from Wiktionary, Creative Commons Attribution/Share-Alike License.
- verb Third-person singular simple present indicative form of
tokenize .
Etymologies
Sorry, no etymologies found.
Support
Help support Wordnik (and make this page ad-free) by adopting the word tokenizes.
Examples
-
The call "FIFFConsole grid1. html grid2. html" will return many differences because it tokenizes it by words and not lines.
The Code Project Latest Articles Ilka Guigova 2010
-
Is he making fun of the way the Special Olympics tokenizes and infantalizes those with disabilities?
VivirLatino 2009
-
Since that is inefficient, it now parses and tokenizes each expression at load time.
AutoHotkey Community 2009
Comments
Log in or sign up to get involved in the conversation. It's quick and easy.