Esempi in C# (CSharp) per NumberAndSurrogatePairTokenizer

Esempio n. 1

File: TestCharTokenizers.cs Progetto: zalintyre/lucenenet

            protected internal override TokenStreamComponents CreateComponents(string fieldName, TextReader reader)
            {
                Tokenizer tokenizer = new NumberAndSurrogatePairTokenizer(TEST_VERSION_CURRENT, reader);

                return(new TokenStreamComponents(tokenizer, tokenizer));
            }

Esempio n. 2

File: TestCharTokenizers.cs Progetto: ChristopherHaws/lucenenet

 public override TokenStreamComponents CreateComponents(string fieldName, TextReader reader)
 {
     Tokenizer tokenizer = new NumberAndSurrogatePairTokenizer(TEST_VERSION_CURRENT, reader);
     return new TokenStreamComponents(tokenizer, tokenizer);
 }