Skip to main content

tokenize

canonic::index

Function tokenize 

Source 
pub fn tokenize(text: &str) -> Vec<String>
Expand description

Tokenize for pure lexical helpers (tests / jaccard).