Definitions
from Wiktionary, Creative Commons Attribution/Share-Alike License.
- verb Alternative spelling of
tokenize .
Etymologies
Sorry, no etymologies found.
Support
Help support Wordnik (and make this page ad-free) by adopting the word tokenise.
Examples
-
Until cis people accept us for the human beings we are, without feeling the need to tokenise, exoticise and objectify us, then such policies will continue to be needed.
-
I knew they didn't understand the code, but I hadn't realised how many of them don't parse it or even tokenise it in their minds.
Making Light: The lily knows not why it blossoms in the spring 2010
-
The way we do this is really pretty simple: tokenise the text, then keep a count for each pair of tokens.
Planet PHP 2009
-
AWID: How can women's movements build spaces that significantly incorporate and support rather than tokenise young women's contributions to gender equality and women's rights struggles?
AWID RSS Feed 2008
-
If you present to me as, let’s say, a woman, I will respect that – why, then, do so many people believe they know what’s beneath my knickers and think they have the right to make such judgemental and ill-informed comments; to objectify and tokenise me?
Comments
Log in or sign up to get involved in the conversation. It's quick and easy.