A few language trigram utilities.

• What is this?
• When should I use this?
• Install
• Use
• API
  • clean(value)
  • trigrams(value)
  • asDictionary(value)
  • asTuples(value)
  • tuplesAsDictionary(tuples)
• Types
• Compatibility
• Security
• Related
• Contribute
• License

This package contains a few utilities that can help when working with trigram (an n-gram where each slice is 3 characters) based natural language detection.

Probably not often, except when you want to create something like franc, but build it in something other than UDHR.

This package is ESM only. In Node.js (version 14.14+, 16.0+), install with npm:

npm install trigram-utils

In Deno with esm.sh:

import * as trigramUtils from 'https://esm.sh/trigram-utils@2'

In browsers with esm.sh:

<script type="module">
  import * as trigramUtils from 'https://esm.sh/trigram-utils@2?bundle'
</script>

import {clean, trigrams, asDictionary, asTuples, tuplesAsDictionary} from 'trigram-utils'

clean(' t@rololol ') // => 't rololol'

trigrams(' t@rololol ')
// => [' t ', 't r', ' ro', 'rol', 'olo', 'lol', 'olo', 'lol', 'ol ']

asDictionary(' t@rololol ')
// => {'ol ': 1, lol: 2, olo: 2, rol: 1, ' ro': 1, 't r': 1, ' t ': 1}

const tuples = asTuples(' t@rololol ')
// => [
//   ['ol ', 1],
//   ['rol', 1],
//   [' ro', 1],
//   ['t r', 1],
//   [' t ', 1],
//   ['lol', 2],
//   ['olo', 2]
// ]

tuplesAsDictionary(tuples)
// => {olo: 2, lol: 2, ' t ': 1, 't r': 1, ' ro': 1, rol: 1, 'ol ': 1}

This package exports the identifiers clean, trigrams, asDictionary, asTuples, and tuplesAsDictionary. There is no default export.

Clean a value (string). Strips some (for language detection) useless punctuation, symbols, and numbers. Collapses white space, trims, and lowercases.

From a value (string), make clean, padded trigrams (see n-gram) (Array<string>).

From a value (string), get clean trigrams as a dictionary (Record<string, number>): keys are trigrams, values are occurrence counts.

From a value (string), get clean trigrams with occurrence counts as a tuple (Array<[string, number]>): first index (0) the trigram, second (1) the occurrence count.

Turn trigram tuples (Array<[string, number]>) into a dictionary (Record<string, number>).

This package is fully typed with TypeScript. It exports the additional types TrigramTuple, TrigramTuples, and TrigramDictionary.

This package is at least compatible with all maintained versions of Node.js. As of now, that is Node.js 14.14+ and 16.0+. It also works in Deno and modern browsers.

This package is safe.

• words/trigrams — trigrams for 400+ languages based on UDHR
• words/n-gram — get n-grams from text
• wooorm/franc — natural language detection

Yes please! See How to Contribute to Open Source.

MIT © Titus Wormer