Definitions

from Wiktionary, Creative Commons Attribution/Share-Alike License

  • v. Third-person singular simple present indicative form of tokenise.

Etymologies

Sorry, no etymologies found.

Examples

  • Last January, I spoke by telephone with Nigerian-American author Nnedi Okorafor about Afrofuturism, a branch of science fiction that features, rather than excludes or tokenises, African characters.

    Archive 2009-11-01

  • PHP_CodeSniffer tokenises PHP, JavaScript and CSS files and detects violations of a defined set of coding standards. phpmd scans PHP source code and looks for potential problems such as possible bugs, suboptimal code or overcomplicated expressions.

    Planet PHP

  • When given a file to generate text based from, as an argument to the learn function, it first tokenises the file then stores pairs in an array of [first word] [second word].

    Planet PHP

Comments

Log in or sign up to get involved in the conversation. It's quick and easy.