CaptureGroup uses Java regexes to emit multiple tokens - one for each capture group in one or more patterns.

For example, a pattern like:

"(https?://([a-zA-Z\-_0-9.]+))"

when matched against the string "http://www.foo.com/index" would return the tokens "https://www.foo.com" and "www.foo.com".

If none of the patterns match, or if preserveOriginal is true, the original token will be preserved.

Each pattern is matched as often as it can be, so the pattern "(...)", when matched against "abcdefghi" would produce ["abc","def","ghi"]

A camelCaseFilter could be written as:

"([A-Z]{2,})",
"(?<![A-Z])([A-Z][a-z]+)",
"(?:^|\\b|(?<=[0-9_])|(?<=[A-Z]{2}))([a-z]+)",
"([0-9]+)"

plus if #preserveOriginal is true, it would also return "camelCaseFilter

Inheritance: TokenFilter
PatternCaptureGroupTokenFilter Class Documentation
Beispiel #1
0
        private void TestPatterns(string input, string[] regexes, string[] tokens, int[] startOffsets, int[] endOffsets, int[] positions, bool preserveOriginal)
        {
            Regex[] patterns = new Regex[regexes.Length];
            for (int i = 0; i < regexes.Length; i++)
            {
                patterns[i] = new Regex(regexes[i], RegexOptions.Compiled);
            }
            TokenStream ts = new PatternCaptureGroupTokenFilter(new MockTokenizer(new StringReader(input), MockTokenizer.WHITESPACE, false), preserveOriginal, patterns);

            AssertTokenStreamContents(ts, tokens, startOffsets, endOffsets, positions);
        }
Beispiel #2
0
 private void TestPatterns(string input, string[] regexes, string[] tokens, int[] startOffsets, int[] endOffsets, int[] positions, bool preserveOriginal)
 {
     Regex[] patterns = new Regex[regexes.Length];
     for (int i = 0; i < regexes.Length; i++)
     {
         patterns[i] = new Regex(regexes[i], RegexOptions.Compiled);
     }
     TokenStream ts = new PatternCaptureGroupTokenFilter(new MockTokenizer(new StringReader(input), MockTokenizer.WHITESPACE, false), preserveOriginal, patterns);
     AssertTokenStreamContents(ts, tokens, startOffsets, endOffsets, positions);
 }