C# (CSharp) Lucene.Net.Analysis.Core LowerCaseTokenizer - 7 ejemplos encontrados. Estos son los ejemplos en C# (CSharp) del mundo real mejor valorados de Lucene.Net.Analysis.Core.LowerCaseTokenizer extraídos de proyectos de código abierto. Puedes valorar ejemplos para ayudarnos a mejorar la calidad de los ejemplos.
Relacionados
Related in langs
LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together. It divides text at non-letters and converts them to lower case. While it is functionally equivalent to the combination of LetterTokenizer and LowerCaseFilter, there is a performance advantage to doing the two tasks at once, hence this (redundant) implementation.
Note: this does a decent job for most European languages, but does a terrible job for some Asian languages, where words are not separated by spaces.