n-grams

You can get approx. one trillion word-tokens (n-grams, from unigrams to 5-grams) of text from publicly available web-pages in 24GB gzip'ed text files over the Linguistic Data Consortium.

Singularity Summit 2007: When will the machines exceed human intelligence? 2007

You can get approx. one trillion word-tokens (n-grams, from unigrams to 5-grams) of text from publicly available web-pages in 24GB gzip'ed text files over the Linguistic Data Consortium.

Archive 2007-09-01 2007

Anagrams: "n-grams it was the best of times it was the worst of times"

Steve Rosenbaum: Is Google Afraid of the Big, Bad, Wolf? 2009

So, to make a start we decided to write a short Perl script to extract word level n-grams from ...

Perl Collocates 2008

So, to make a start we decided to write a short Perl script to extract word level n-grams from some text so we could start looking for interesting collocates.

Perl Collocates — バカな火星人 2008

- Call the NGramGenerator UDF to compose the n-grams of the query.

Site Home Avkash Chauhan - MSFT 2012

- Use the DISTINCT command to get the unique n-grams for all records.

Site Home Avkash Chauhan - MSFT 2012

With this data in hand, they performed a similar process for what they call "n-grams," or short phrases of up to five words.

Ars Technica John Timmer 2010

One of the many brilliant things that Google indexing has created is something known as the, as I understand it, n-grams have to do with the frequency with which one unit in a language is followed by another unit - e.g. how many times in a given body of text is the word "love" followed by the word "fifteen," and what, then, is the predictability of this 2-gram occuring when "love" occurs.

Slaw 2009

The categorisation of the documents is based on n-grams, which are used to determine significant features of the documents.

Search Engine Watch Blog 2009