Definitions
from Wiktionary, Creative Commons Attribution/Share-Alike License.
- noun Plural form of
tokenizer .
Etymologies
Sorry, no etymologies found.
Support
Help support Wordnik (and make this page ad-free) by adopting the word tokenizers.
Examples
-
As a program manager working with computational linguists, I find it fascinating because it epitomizes the numerous difficult decisions one has to make when building natural language processing systems like word-breakers, tokenizers, spell-checkers, etc.
MSDN Blogs 2009
Comments
Log in or sign up to get involved in the conversation. It's quick and easy.