private PredictionTree GetFollowDPathSet(FiniteAutomataMultiTargetTransitionTable <GrammarVocabulary, Tuple <int, PredictionTreeBranch> > transitionTableResult, GrammarVocabulary transition, Dictionary <PredictionTreeBranch, int[]> epDepthLookup)
{
//var minMinDepth = (from deviationPath in transitionTableResult[transition]
// select deviationPath.Item2.MinDepth).Min();
//minDepth = minMinDepth;
//epDepthLookup = epDepthLookup.ToDictionary(dps => new PredictionTreeBranch(dps.Key.Skip(minMinDepth).ToList(), dps.Key.Depth - minMinDepth, minDepth: dps.Key.MinDepth - minMinDepth), dps => dps.Value);
var result = PredictionTree.GetPathSet(transition, epDepthLookup.Keys.ToList(), this, ProductionRuleProjectionType.FollowAmbiguity, PredictionDerivedFrom.LookAhead_FollowPrediction);
result.SetFollowEpsilonData(provider => epDepthLookup[provider]);
return(result);
}
private static IEnumerable <PredictionTreeBranch> PushIntersection(
PredictionTreeBranch sourceMasterPath,
FiniteAutomataMultiTargetTransitionTable <GrammarVocabulary, Tuple <int, PredictionTreeBranch> > transitionTableResult,
Stack <Tuple <int, PredictionTreeBranch> > toProcess,
HashSet <PredictionTreeBranch> seen,
Tuple <int, PredictionTreeBranch> currentPathTuple,
PredictionTreeBranch currentPath,
PredictionTreeBranch path,
GrammarVocabulary intersection,
GrammarVocabulary transitionKey = null)
{
/* * * * * * * * * * * * * * * * * * * * * * * * * *\
* Construct the transition table with the matching *
* paths. *
* * * * * * * * * * * * * * * * * * * * * * * * * */
var intersectingCurrent = from key in currentPath.CurrentNode.Veins.Keys
let intersect = key.Intersect(intersection)
where !intersect.IsEmpty
let subPathSet = currentPath.CurrentNode.Veins[key]
from subPath in subPathSet.UnalteredOriginals
let rPath = new PredictionTreeBranch(sourceMasterPath.Take(currentPath.Depth).Concat(subPath).ToList(), currentPath.Depth + subPath.Depth, true, false, currentPath.GetDeviationsUpTo(currentPath.Depth), minDepth: currentPath.Depth)
where rPath.Valid
select rPath;
var set = new List <Tuple <int, PredictionTreeBranch> >(from ip in intersectingCurrent
select Tuple.Create(currentPathTuple.Item1, ip))
{
Tuple.Create(currentPathTuple.Item1 + 1, new PredictionTreeBranch(path.baseList, path.Depth, false, false, minDepth: path.Depth))
};
var distinctNodeArray = (from element in set
select element.Item2).Distinct().ToArray();
foreach (var element in distinctNodeArray)
{
yield return(element);
}
transitionTableResult.Add(transitionKey ?? intersection, set);
/* * * * * * * * * * * * * * * * * * * * * * * * * * * *\
* If the current point is an edge, walk further *
* up the parse tree to see what else lies in store. *
* * * * * * * * * * * * * * * * * * * * * * * * * * * **
* Further ambiguities might result in additional look- *
* ahead being necessary. *
* * * * * * * * * * * * * * * * * * * * * * * * * * * */
if (seen.Add(path) && path.CurrentNode.Veins.DFAOriginState.IsEdge)
{
toProcess.Push(Tuple.Create(currentPathTuple.Item1 + 1, path));
}
}
private void ObtainTerminalAmbiguitiesOnPath(
PredictionTreeBranch sourceMasterPath,
Dictionary <SyntacticalDFAState, PredictionTreeLeaf> fullSeries,
ControlledDictionary <IOilexerGrammarProductionRuleEntry, SyntacticalDFARootState> ruleDFAs,
Dictionary <IOilexerGrammarProductionRuleEntry, GrammarVocabulary> ruleLookup,
FiniteAutomataMultiTargetTransitionTable <GrammarVocabulary, Tuple <int, PredictionTreeBranch> > transitionTableResult,
GrammarSymbolSet grammarSymbols)
{
var ambiguousSymbols = grammarSymbols.AmbiguousSymbols.ToList();
Stack <Tuple <int, PredictionTreeBranch> > toProcess = new Stack <Tuple <int, PredictionTreeBranch> >();
var fullCheck = this.Veins.Keys.Aggregate(GrammarVocabulary.UnionAggregateDelegate);
/* * * * * * * * * * * * * * * * * * * * * * * * * * * *\
* Kick things off with the master path, this is derived *
* from the incoming states from the parent rule. *
* * * * * * * * * * * * * * * * * * * * * * * * * * * */
toProcess.Push(Tuple.Create(0, sourceMasterPath));
//foreach (var transition in this.Value.Keys)
// foreach (var target in this.Value[transition])
// toProcess.Push(new PredictionTreeBranch(sourceMasterPath.Take(sourceMasterPath.Depth).Concat(target).ToList(), sourceMasterPath.Depth, true, false, sourceMasterPath.GetDeviationsUpTo(sourceMasterPath.Depth)));
HashSet <PredictionTreeBranch> seen = new HashSet <PredictionTreeBranch>();
while (toProcess.Count > 0)
{
var currentPathTuple = toProcess.Pop();
var currentPath = currentPathTuple.Item2;
if (currentPath.Depth == 0)
{
continue;
}
if (currentPath.CurrentNode.Veins.Keys.Aggregate(GrammarVocabulary.UnionAggregateDelegate).Intersect(fullCheck).IsEmpty)
{
continue;
}
/* *
* Walk up the tree one level into the state that
* follows the current rule in whatever the current
* calling rule might be.
* */
var incomingTransitions = currentPath.FollowTransition(fullSeries, ruleLookup[currentPath.CurrentNode.Veins.Rule], ruleLookup, true, false, true, true);
/* *
* Aggregate the transitions from our previously
* calculated LL(1) look-ahead information.
* */
/* *
* On the state following this rule, gather up the
* targets which do overlap one or more of our
* transitions.
* */
var fullAmbiguityContext = incomingTransitions.FullCheck | fullCheck;
var intersectionPoints = (from t in incomingTransitions.Keys
let intersection = t.Intersect(fullCheck)
where !intersection.IsEmpty
from path in incomingTransitions[t]
select new { Intersection = intersection, Path = path }).ToArray();
var overlap = GrammarVocabulary.NullInst;
if ((fullAmbiguityContext != null && !fullAmbiguityContext.IsEmpty) && grammarSymbols.IsGrammarPotentiallyAmbiguous(fullAmbiguityContext))
{
foreach (var ambiguity in ambiguousSymbols)
{
var localIntersect = fullCheck & ambiguity.AmbiguityKey;
var incomingIntersect = incomingTransitions.FullCheck & ambiguity.AmbiguityKey;
if (localIntersect.IsEmpty || incomingIntersect.IsEmpty)
{
/* If the intersection isn't present on one or the other sides; then, for the purposes of
* this follow check, the union would not be relevant, it would likely be identified within the
* prediction upon the state itself. */
continue;
}
var aggregateIntersect = fullAmbiguityContext & ambiguity.AmbiguityKey;
var overlapIntersect = overlap & ambiguity.AmbiguityKey;
if (aggregateIntersect.Equals(ambiguity.AmbiguityKey) && !overlapIntersect.Equals(ambiguity.AmbiguityKey))
{
/* */
var ambiguityIntersection =
(from t in incomingTransitions.Keys
let intersection = t.Intersect(incomingIntersect)
where !intersection.IsEmpty
from path in incomingTransitions[t]
select new { Intersection = intersection, Path = path }).ToArray();
var ambiguityVocabulary = new GrammarVocabulary(grammarSymbols, ambiguity);
foreach (var intersectingPoint in ambiguityIntersection)
{
var path = intersectingPoint.Path;
var intersection = localIntersect;
if (path.Equals(currentPath))
{
continue;
}
/* Notify the nodes of the ambiguities so their state-machines can be patched up.
* This is done explicitly to avoid the ambiguity cropping up and allowing it everywhere,
* which is incorrect. If it shows up where it's not supposed to, we have a logic error
* somwehere. */
foreach (var node in (from ambigPath in PushIntersection(sourceMasterPath, transitionTableResult, toProcess, seen, currentPathTuple, currentPath, path, intersection, ambiguityVocabulary)
select ambigPath.CurrentNode).Distinct().ToArray())
{
node.DenoteLexicalAmbiguity(ambiguity);
}
}
ambiguity.Occurrences++;
overlap |= aggregateIntersect;
}
}
}
foreach (var intersectingPoint in intersectionPoints)
{
var path = intersectingPoint.Path;
var intersection = intersectingPoint.Intersection;
if (path.Equals(currentPath))
{
continue;
}
PushIntersection(sourceMasterPath, transitionTableResult, toProcess, seen, currentPathTuple, currentPath, path, intersection).ToArray();
}
}
}
internal MultikeyedDictionary <GrammarVocabulary, int, List <Tuple <Tuple <int[], PredictionTree>[], int[]> > > ObtainTerminalAmbiguities(
Dictionary <SyntacticalDFAState, PredictionTreeLeaf> fullSeries,
ControlledDictionary <IOilexerGrammarProductionRuleEntry, SyntacticalDFARootState> ruleDFAs,
Dictionary <IOilexerGrammarProductionRuleEntry, GrammarVocabulary> ruleLookup,
GrammarSymbolSet grammarSymbols)
{
if (!this.Veins.DFAOriginState.IsEdge)
{
return(new MultikeyedDictionary <GrammarVocabulary, int, List <Tuple <Tuple <int[], PredictionTree>[], int[]> > >());
}
var tempDictionary = new MultikeyedDictionary <PredictionTreeBranch, GrammarVocabulary, Tuple <int[], PredictionTree> >();
var rootNode = fullSeries[ruleDFAs[this.Rule]];
var currentIncoming = new List <PredictionTreeBranch>(rootNode.incoming);
var reductions = rootNode.PointsOfReduction.ToArray();
if (reductions.Length > 0)
{
}
var totalIncoming = new HashSet <PredictionTreeBranch>();
Dictionary <PredictionTree, PredictionTree> uniqueSet = new Dictionary <PredictionTree, PredictionTree>();
foreach (var path in currentIncoming)
{
var transitionTableResult = new FiniteAutomataMultiTargetTransitionTable <GrammarVocabulary, Tuple <int, PredictionTreeBranch> >();
var masterPathChop = new PredictionTreeBranch(path.Take(path.Depth).Concat(new[] { this }).ToArray(), path.Depth, false, false);
masterPathChop.SetDeviations(path.GetDeviationsUpTo(path.Depth));
ObtainTerminalAmbiguitiesOnPath(masterPathChop, fullSeries, ruleDFAs, ruleLookup, transitionTableResult, grammarSymbols);
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * \
* Construct the sequence of Path->Grammar->PathSets based *
* off of the table provided by the ObtainTerminalAmbiguitiesOnPath. *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * **/
foreach (var transition in transitionTableResult.Keys)
{
var epDepthLookup =
(from entry in transitionTableResult[transition]
group entry.Item1 by entry.Item2).ToDictionary(k => k.Key, v => v.ToArray());
//int minDepth;
//var uniqueCurrent = GetUniqueDPathSet(uniqueSet, transitionTableResult, transition, epDepthLookup);
//var subPath = new PredictionTreeBranch(path.Skip(minDepth).ToList(), path.Depth - minDepth, minDepth: path.MinDepth);
tempDictionary.TryAdd(path, transition, Tuple.Create((from epDepth in transitionTableResult[transition]
select epDepth.Item1).ToArray(), GetFollowDPathSet(transitionTableResult, transition, epDepthLookup)));
}
}
var regrouping = (from ksvp in tempDictionary
group new { Path = ksvp.Keys.Key1, Set = ksvp.Value } by ksvp.Keys.Key2).ToDictionary(k => k.Key, v => v.ToArray());
/*
* foreach (var pathSet in from key in regrouping.Keys
* let value = regrouping[key]
* from r in value
* select r.Set)
* pathSet.Item2.FixAllPaths();//*/
var comparisons = (from key in regrouping.Keys
let value = regrouping[key]
let kRewrite = from r in value
select r.Set
let commonalities = PredictionTree.GetCompoundRightSideSimilarities(kRewrite)
select new { Commonalities = commonalities, Transition = key }).ToDictionary(k => k.Transition, v => v.Commonalities);
var resultDictionary = new MultikeyedDictionary <GrammarVocabulary, int, List <Tuple <Tuple <int[], PredictionTree>[], int[]> > >();
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * \
* Once we're done, we need to rebuild the paths with the MinDepth set to *
* their current Depth. This is to ensure that it the PathSets don't try *
* to reduce more than necessary. *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Then we take that set and perform a right-hand-side comparison between *
* the elements. *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* This will be used later to reduce the walking necessary to isolate the *
* proper machine to use to disambiguate the look-ahead. *
\ * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * **/
foreach (var comparison in comparisons.Keys)
{
var comparisonElements = comparisons[comparison];
foreach (var memberCount in comparisonElements.Keys)
{
var comparisonElement = comparisonElements[memberCount];
resultDictionary.Add(comparison, memberCount, comparisonElement);
}
}
return(resultDictionary);
}
public bool ConstructEpsilonLookAheadProjection(Dictionary <SyntacticalDFAState, PredictionTreeLeaf> fullSeries, Dictionary <IOilexerGrammarProductionRuleEntry, GrammarVocabulary> ruleVocabulary, int cycleDepth)
{
if (cycleDepth > 1)
{
this.cyclePaths[cycleDepth - 2].Clear();
}
var currentCycle = this.cyclePaths[cycleDepth - 1];
bool result = false;
FiniteAutomataMultiTargetTransitionTable <GrammarVocabulary, PredictionTreeBranch> resultTable = new FiniteAutomataMultiTargetTransitionTable <GrammarVocabulary, PredictionTreeBranch>();
var previousLeftRecursive = leftRecursionType;
if (leftRecursionType == ProductionRuleLeftRecursionType.None)
{
var temp =
(from p in currentCycle
where p.Depth == 0
select p.GetRecursionType()).Aggregate(PredictionTreeBranch.LeftRecursionAggregateDelegate);
if (leftRecursionType != (leftRecursionType | temp))
{
leftRecursionType = temp;
}
}
if (previousLeftRecursive != leftRecursionType)
{
result = true;
}
var transitionalPaths = (from p in currentCycle
from ePath in p.ObtainEpsilonTransitions(fullSeries)
select ePath).ToArray();
foreach (var transitionalPath in transitionalPaths)
{
if (!currentCycle.Contains(transitionalPath))
{
currentCycle.Add(transitionalPath);
var leftRecursionChange =
(from path in currentCycle
select path.GetRecursionType()).Aggregate(PredictionTreeBranch.LeftRecursionAggregateDelegate);
}
}
var leftRecursivePaths =
(from p in currentCycle
where p.Depth == 0
let recurType = p.GetRecursionType()
where recurType != ProductionRuleLeftRecursionType.None
select p.GetFirstRecursion()).ToArray();
foreach (var path in currentCycle)
{
/* *
* Use a transition table to sort out ambiguities.
* */
foreach (var transition in path.CurrentNode.Veins.DFAOriginState.OutTransitions.Keys)
{
if (leftRecursionType == ProductionRuleLeftRecursionType.None)
{
//var tokVariant = transition.GetTokenVariant();
var tokenVar = transition.GetTokenVariant();
var reducedRules = transition.GetRuleVariant().GetSymbols().Select(k => (IGrammarRuleSymbol)k).Select(k => new { Node = fullSeries.GetRuleNodeFromFullSeries(k.Source), Symbol = k }).Where(k => k.Node.HasBeenReduced).Select(k => k.Symbol);
var ruleVar = new GrammarVocabulary(transition.symbols, reducedRules.ToArray());
if (!tokenVar.IsEmpty)
{
resultTable.Add(tokenVar, new List <PredictionTreeBranch>()
{
path
});
}
if (!ruleVar.IsEmpty)
{
resultTable.Add(ruleVar, new List <PredictionTreeBranch>()
{
path
});
}
}
else
{
var currentSet = new List <PredictionTreeBranch>()
{
path
};
//if (this.Rule.Name == "RelationalExp")
//{
// if (transition.ToString() == "Identifier")
// {
foreach (var lrPath in leftRecursivePaths)
{
currentSet.Add(lrPath.GetTransitionPath(lrPath, lrPath.Take(lrPath.Depth).ToArray(), path, ruleVocabulary, false, false, false));
}
// }
//}
resultTable.Add(transition, currentSet);
}
}
}
int currentCount = this.Count;
if (currentCount != resultTable.Count)
{
result = true;
if (this.Count > 0)
{
this._Clear();
}
foreach (var transition in resultTable.Keys)
{
var pathSet = PredictionTree.GetPathSet(transition, resultTable[transition], this.Leaf, ProductionRuleProjectionType.LookAhead, PredictionDerivedFrom.Lookahead_Epsilon);
this._Add(transition, pathSet);
pathSet.CheckReductionState(fullSeries, ruleVocabulary);
//pathSet.ProcessCommonSymbol(fullSeries, ruleVocabulary);
}
}
return(result);
}
FollowTransition(
Dictionary <SyntacticalDFAState, PredictionTreeLeaf> fullSeries,
GrammarVocabulary transitionKey,
Dictionary <IOilexerGrammarProductionRuleEntry, GrammarVocabulary> ruleLookup,
bool usePrevious,
bool injectIncomingPaths,
bool includeRootEdges = false,
bool isEpsilonTransition = false,
bool silentlyFail = false,
bool isReduction = false)
{
var resultData = from u in FollowTransitionInternal(fullSeries, transitionKey, ruleLookup, usePrevious, isEpsilonTransition, silentlyFail, isReduction)
group u.Item2 by u.Item1;
FiniteAutomataMultiTargetTransitionTable <GrammarVocabulary, PredictionTreeBranch> result = new FiniteAutomataMultiTargetTransitionTable <GrammarVocabulary, PredictionTreeBranch>();
foreach (var element in resultData)
{
if (element.Key.IsEmpty)
{
result.SetAutoSegment(false);
}
result.Add(element.Key, element.ToList());
if (element.Key.IsEmpty)
{
result.SetAutoSegment(true);
}
}
return(result);
}