private void WatersImsMseChromatogramTest(DriftFilterType mode,
IonMobilityWindowWidthCalculator.IonMobilityPeakWidthType driftPeakWidthCalcType,
RefinementSettings.ConvertToSmallMoleculesMode asSmallMolecules = RefinementSettings.ConvertToSmallMoleculesMode.none)
{
if (asSmallMolecules != RefinementSettings.ConvertToSmallMoleculesMode.none && !RunSmallMoleculeTestVersions)
{
Console.Write(MSG_SKIPPING_SMALLMOLECULE_TEST_VERSION);
return;
}
string subdir = (asSmallMolecules == RefinementSettings.ConvertToSmallMoleculesMode.none) ? null : asSmallMolecules.ToString();
var testFilesDir = new TestFilesDir(TestContext, ZIP_FILE, subdir);
TestSmallMolecules = false; // Don't need that extra magic node
bool withDriftTimePredictor = (mode == DriftFilterType.predictor); // Load the doc that has a drift time predictor?
bool withDriftTimeFilter = (mode != DriftFilterType.none); // Perform drift time filtering? (either with predictor, or with bare times in blib file)
string docPath;
SrmDocument document = InitWatersImsMseDocument(testFilesDir, driftPeakWidthCalcType, withDriftTimeFilter, withDriftTimePredictor, out docPath);
AssertEx.IsDocumentState(document, null, 1, 1, 1, 8); // Drift time lib load bumps the doc version, so does small mol conversion
var listChromatograms = new List <ChromatogramSet>();
// A small subset of the QC_HDMSE_02_UCA168_3495_082213 data set (RT 21.5-22.5) from Will Thompson
const string mz5Path = @"waters-mobility.mz5";
string testModeStr = withDriftTimePredictor ? "with drift time predictor" : "without drift time info";
if (withDriftTimeFilter && !withDriftTimePredictor)
{
testModeStr = "with drift times from spectral library";
}
listChromatograms.Add(AssertResult.FindChromatogramSet(document, new MsDataFilePath(mz5Path)) ??
new ChromatogramSet(Path.GetFileName(mz5Path).Replace('.', '_'), new[] { mz5Path }));
using (var docContainer = new ResultsTestDocumentContainer(document, docPath))
{
var doc = docContainer.Document;
var docResults = doc.ChangeMeasuredResults(new MeasuredResults(listChromatograms));
Assume.IsTrue(docContainer.SetDocument(docResults, doc, true));
docContainer.AssertComplete();
document = docContainer.Document;
}
document = ConvertToSmallMolecules(document, ref docPath, new[] { mz5Path }, asSmallMolecules);
using (var docContainer = new ResultsTestDocumentContainer(document, docPath))
{
float tolerance = (float)document.Settings.TransitionSettings.Instrument.MzMatchTolerance;
double maxHeight = 0;
var results = document.Settings.MeasuredResults;
Assume.AreEqual(1, document.MoleculePrecursorPairs.Count());
foreach (var pair in document.MoleculePrecursorPairs)
{
ChromatogramGroupInfo[] chromGroupInfo;
Assume.IsTrue(results.TryLoadChromatogram(0, pair.NodePep, pair.NodeGroup,
tolerance, true, out chromGroupInfo));
Assume.AreEqual(1, chromGroupInfo.Length, testModeStr);
var chromGroup = chromGroupInfo[0];
int expectedPeaks;
if (withDriftTimeFilter)
{
expectedPeaks = 3;
}
else if (asSmallMolecules != RefinementSettings.ConvertToSmallMoleculesMode.masses_only)
{
expectedPeaks = 5;
}
else
{
expectedPeaks = 6; // No libraries
}
Assume.AreEqual(expectedPeaks, chromGroup.NumPeaks, testModeStr); // This will be higher if we don't filter on DT
foreach (var tranInfo in chromGroup.TransitionPointSets)
{
maxHeight = Math.Max(maxHeight, tranInfo.MaxIntensity);
}
}
Assume.AreEqual(withDriftTimeFilter ? 5226 : 20075, maxHeight, 1, testModeStr); // Without DT filtering, this will be much greater
// now drill down for specific values
int nPeptides = 0;
foreach (var nodePep in document.Molecules.Where(nodePep => !nodePep.Results[0].IsEmpty))
{
// expecting just one peptide result in this small data set
if (nodePep.Results[0].Sum(chromInfo => chromInfo.PeakCountRatio > 0 ? 1 : 0) > 0)
{
Assume.AreEqual(21.94865, (double)nodePep.GetMeasuredRetentionTime(0), .0001, testModeStr);
Assume.AreEqual(1.0, (double)nodePep.GetPeakCountRatio(0), 0.0001, testModeStr);
nPeptides++;
}
}
Assume.AreEqual(1, nPeptides);
if (withDriftTimePredictor || withDriftTimeFilter)
{
// Verify that the .imdb pr .blib file goes out in the share zipfile
for (int complete = 0; complete <= 1; complete++)
{
var sharePath =
testFilesDir.GetTestPath(complete == 1 ? "share_complete.zip" : "share_minimized.zip");
var share = new SrmDocumentSharing(document, docPath, sharePath,
new ShareType(complete == 1, SkylineVersion.CURRENT)); // Explicitly declaring version number forces a save before zip
share.Share(new SilentProgressMonitor());
var files = share.ListEntries().ToArray();
var imdbFile = withDriftTimePredictor ? "scaled.imdb" : "waters-mobility.filtered-scaled.blib";
if (asSmallMolecules != RefinementSettings.ConvertToSmallMoleculesMode.none)
{
var ext = "." + imdbFile.Split('.').Last();
imdbFile = imdbFile.Replace(ext, BiblioSpecLiteSpec.DotConvertedToSmallMolecules + ext);
}
Assume.IsTrue(files.Contains(imdbFile));
// And round trip it to make sure we haven't left out any new features in minimized imdb or blib files
share.Extract(new SilentProgressMonitor());
using (var cmdline = new CommandLine())
{
Assume.IsTrue(cmdline.OpenSkyFile(share.DocumentPath)); // Handles any path shifts in database files, like our .imdb file
var document2 = cmdline.Document;
Assume.IsNotNull(document2);
Assume.IsTrue(docContainer.SetDocument(document2, docContainer.Document, true));
docContainer.AssertComplete();
document2 = docContainer.Document;
var im = document2.Settings.GetIonMobilities(new MsDataFilePath(mz5Path));
var pep = document2.Molecules.First();
foreach (TransitionGroupDocNode nodeGroup in pep.Children)
{
double windowDT;
var centerDriftTime = document2.Settings.PeptideSettings.Prediction.GetIonMobility(
pep, nodeGroup, im, null, driftTimeMax, out windowDT);
Assume.AreEqual(3.86124, centerDriftTime.IonMobility.Mobility.Value, .0001, testModeStr);
Assume.AreEqual(0.077224865797235934, windowDT, .0001, testModeStr);
}
}
}
}
}
}
public void DoThermoRatioTest(RefinementSettings.ConvertToSmallMoleculesMode smallMoleculesTestMode)
{
if (smallMoleculesTestMode != RefinementSettings.ConvertToSmallMoleculesMode.none && !RunSmallMoleculeTestVersions)
{
System.Console.Write(MSG_SKIPPING_SMALLMOLECULE_TEST_VERSION);
return;
}
TestSmallMolecules = false; // We do this explicitly
var testFilesDir = new TestFilesDir(TestContext, ZIP_FILE);
string docPath;
SrmDocument doc = InitThermoDocument(testFilesDir, out docPath);
SrmSettings settings = doc.Settings.ChangePeptideModifications(mods =>
mods.ChangeInternalStandardTypes(new[] { IsotopeLabelType.light }));
doc = doc.ChangeSettings(settings);
if (smallMoleculesTestMode != RefinementSettings.ConvertToSmallMoleculesMode.none)
{
var docOrig = doc;
var refine = new RefinementSettings();
doc = refine.ConvertToSmallMolecules(doc, smallMoleculesTestMode);
// This is our first example of a converted label doc - check roundtripping
AssertEx.ConvertedSmallMoleculeDocumentIsSimilar(docOrig, doc);
AssertEx.Serializable(doc);
}
using (var docContainer = new ResultsTestDocumentContainer(doc, docPath))
{
string extRaw = ExtensionTestContext.ExtThermoRaw;
var listChromatograms = new List <ChromatogramSet>
{
new ChromatogramSet("rep03", new[]
{
MsDataFileUri.Parse(testFilesDir.GetTestPath(
"Site20_STUDY9P_PHASEII_QC_03" + extRaw))
}),
new ChromatogramSet("rep05", new[]
{
MsDataFileUri.Parse(testFilesDir.GetTestPath(
"Site20_STUDY9P_PHASEII_QC_05" + extRaw))
})
};
var docResults = doc.ChangeMeasuredResults(new MeasuredResults(listChromatograms));
Assert.IsTrue(docContainer.SetDocument(docResults, doc, true));
docContainer.AssertComplete();
docResults = docContainer.Document;
// Make sure all groups have at least 5 transitions (of 6) with ratios
int ratioGroupMissingCount = 0;
foreach (var nodeGroup in docResults.MoleculeTransitionGroups)
{
if (nodeGroup.TransitionGroup.LabelType.IsLight)
{
foreach (var result in nodeGroup.Results)
{
Assert.IsFalse(result[0].Ratio.HasValue, "Light group found with a ratio");
}
foreach (TransitionDocNode nodeTran in nodeGroup.Children)
{
foreach (var resultTran in nodeTran.Results)
{
Assert.IsFalse(resultTran[0].Ratio.HasValue, "Light transition found with a ratio");
}
}
}
else
{
bool missingRatio = false;
foreach (ChromInfoList <TransitionGroupChromInfo> chromInfoList in nodeGroup.Results)
{
var ratioHeavy = chromInfoList[0].Ratio;
if (!ratioHeavy.HasValue)
{
missingRatio = true;
}
}
int ratioCount1 = 0;
int ratioCount2 = 0;
foreach (TransitionDocNode nodeTranHeavy in nodeGroup.Children)
{
float?ratioHeavy = nodeTranHeavy.Results[0][0].Ratio;
if (ratioHeavy.HasValue)
{
Assert.IsFalse(float.IsNaN(ratioHeavy.Value) || float.IsInfinity(ratioHeavy.Value));
ratioCount1++;
}
ratioHeavy = nodeTranHeavy.Results[1][0].Ratio;
if (ratioHeavy.HasValue)
{
Assert.IsFalse(float.IsNaN(ratioHeavy.Value) || float.IsInfinity(ratioHeavy.Value));
ratioCount2++;
}
}
Assert.AreEqual(3, ratioCount1);
if (ratioCount2 < 2)
{
ratioGroupMissingCount++;
}
else
{
Assert.IsFalse(missingRatio, "Precursor missing ratio when transitions have ratios");
}
}
}
// 3 groups with less than 2 transition ratios
Assert.AreEqual(3, ratioGroupMissingCount);
// Remove the first light transition, checking that this removes the ratio
// from the corresponding heavy transition, but not the entire group, until
// after all light transitions have been removed.
IdentityPath pathFirstPep = docResults.GetPathTo((int)SrmDocument.Level.Molecules, 0);
var nodePep = (PeptideDocNode)docResults.FindNode(pathFirstPep);
Assert.AreEqual(2, nodePep.Children.Count);
var nodeGroupLight = (TransitionGroupDocNode)nodePep.Children[0];
IdentityPath pathGroupLight = new IdentityPath(pathFirstPep, nodeGroupLight.TransitionGroup);
Assert.IsNull(nodeGroupLight.Results[0][0].Ratio, "Light group has ratio");
var nodeGroupHeavy = (TransitionGroupDocNode)nodePep.Children[1];
IdentityPath pathGroupHeavy = new IdentityPath(pathFirstPep, nodeGroupHeavy.TransitionGroup);
float? ratioStart = nodeGroupHeavy.Results[0][0].Ratio;
Assert.IsTrue(ratioStart.HasValue, "No starting heavy group ratio");
var expectedValues = new[] { 1.403414, 1.38697791, 1.34598482 };
for (int i = 0; i < 3; i++)
{
var pathLight = docResults.GetPathTo((int)SrmDocument.Level.Transitions, 0);
var pathHeavy = docResults.GetPathTo((int)SrmDocument.Level.Transitions, 3);
TransitionDocNode nodeTran = (TransitionDocNode)docResults.FindNode(pathHeavy);
float? ratioTran = nodeTran.Results[0][0].Ratio;
Assert.IsTrue(ratioTran.HasValue, "Expected transition ratio not found");
Assert.AreEqual(ratioTran.Value, expectedValues[i], 1.0e-5);
docResults = (SrmDocument)docResults.RemoveChild(pathLight.Parent, docResults.FindNode(pathLight));
nodeTran = (TransitionDocNode)docResults.FindNode(pathHeavy);
Assert.IsFalse(nodeTran.Results[0][0].Ratio.HasValue, "Unexpected transiton ratio found");
Assert.AreEqual(pathGroupHeavy, pathHeavy.Parent, "Transition found outside expected group");
// nodePep = (PeptideDocNode) docResults.FindNode(pathFirstPep);
nodeGroupHeavy = (TransitionGroupDocNode)docResults.FindNode(pathGroupHeavy);
// Assert.AreEqual(nodePep.Results[0][0].RatioToStandard, nodeGroupHeavy.Results[0][0].Ratio,
// "Peptide and group ratios not equal");
if (i < 2)
{
float?ratioGroup = nodeGroupHeavy.Results[0][0].Ratio;
Assert.IsTrue(ratioGroup.HasValue, "Group ratio removed with transition ratios");
Assert.AreEqual(ratioStart.Value, ratioGroup.Value, 0.1,
"Unexpected group ratio change by more than 0.1");
}
else
{
Assert.IsFalse(nodeGroupHeavy.Results[0][0].Ratio.HasValue,
"Group ratio still present with no transition ratios");
}
}
bool asSmallMolecules = (smallMoleculesTestMode != RefinementSettings.ConvertToSmallMoleculesMode.none);
if (!asSmallMolecules) // GetTransitions() doesn't work the same way for small molecules - it only lists existing ones
{
bool firstAdd = true;
var nodeGroupLightOrig = (TransitionGroupDocNode)doc.FindNode(pathGroupLight);
DocNode[] lightChildrenOrig = nodeGroupLightOrig.Children.ToArray();
foreach (var nodeTran in nodeGroupLightOrig.GetTransitions(docResults.Settings,
null, nodeGroupLightOrig.PrecursorMz, null, null, null, false))
{
var transition = nodeTran.Transition;
if (!firstAdd && lightChildrenOrig.IndexOf(node => Equals(node.Id, transition)) == -1)
{
continue;
}
// Add the first transition, and then the original transitions
docResults = (SrmDocument)docResults.Add(pathGroupLight, nodeTran);
nodeGroupHeavy = (TransitionGroupDocNode)docResults.FindNode(pathGroupHeavy);
if (firstAdd)
{
Assert.IsNull(nodeGroupHeavy.Results[0][0].Ratio, "Unexpected heavy ratio found");
}
else
{
Assert.IsNotNull(nodeGroupHeavy.Results[0][0].Ratio,
"Heavy ratio null after adding light children");
}
firstAdd = false;
}
Assert.AreEqual(ratioStart, nodeGroupHeavy.Results[0][0].Ratio);
}
}
testFilesDir.Dispose();
}
public void ThermoCancelImportTest()
{
var testFilesDir = new TestFilesDir(TestContext, ZIP_FILE);
string resultsPath = testFilesDir.GetTestPath("Site20_STUDY9P_PHASEII_QC_03" +
ExtensionTestContext.ExtThermoRaw);
string dirPath = Path.GetDirectoryName(resultsPath) ?? "";
string docPath;
SrmDocument doc = InitThermoDocument(testFilesDir, out docPath);
// Give this two chances to succeed. It can succeed tens of thousands of times
// in a row, but it still occasionally fails on nightly tests. Hopefully two
// tries will make this extremely unlikely.
for (int tries = 0; tries < 2; tries++)
{
using (var docContainer = new ResultsTestDocumentContainer(doc, docPath))
{
// Remove any existing temp and cache files
foreach (var path in Directory.GetFiles(dirPath))
{
if (IsCacheOrTempFile(path))
{
FileEx.SafeDelete(path);
}
}
string name = Path.GetFileNameWithoutExtension(resultsPath);
var listChromatograms = new List <ChromatogramSet> {
new ChromatogramSet(name, new[] { MsDataFileUri.Parse(resultsPath) })
};
var docResults = doc.ChangeMeasuredResults(new MeasuredResults(listChromatograms));
// Start cache load, but don't wait for completion
Assert.IsTrue(docContainer.SetDocument(docResults, doc));
// Wait up to 10 second for the cache to start being written
bool cacheFound = false;
for (int i = 0; i < 1000; i++)
{
if (Directory.GetFiles(dirPath).IndexOf(IsCacheOrTempFile) != -1)
{
cacheFound = true;
break;
}
Thread.Sleep(10);
}
if (!cacheFound)
{
Assert.Fail(TextUtil.LineSeparate("Failed to create cache file. Found files:", TextUtil.LineSeparate(Directory.GetFiles(dirPath))));
}
// Cancel by reverting to the original document
Assert.IsTrue(docContainer.SetDocument(doc, docResults));
// Wait up to 10 seconds for cancel to occur
bool cancelOccurred = false;
for (int i = 0; i < 1000; i++)
{
if (docContainer.LastProgress != null && docContainer.LastProgress.IsCanceled)
{
cancelOccurred = true;
break;
}
Thread.Sleep(10);
}
// Wait up to 20 seconds for the cache to be removed
bool cacheRemoved = false;
for (int i = 0; i < 200; i++)
{
if (Directory.GetFiles(dirPath).IndexOf(IsCacheOrTempFile) == -1)
{
cacheRemoved = true;
break;
}
Thread.Sleep(100);
}
if (!cacheRemoved)
{
if (tries == 0 && File.Exists(Path.ChangeExtension(docPath, ChromatogramCache.EXT)))
{
// Allow a single failure where we end up with the final cache instead of a cancelation
FileEx.SafeDelete(docPath);
continue; // Try again
}
if (!cancelOccurred)
{
Assert.Fail("Attempt to cancel results load failed on try {0}. {1}", tries + 1,
docContainer.LastProgress != null && docContainer.LastProgress.ErrorException != null
? docContainer.LastProgress.ErrorException.Message : string.Empty);
}
Assert.Fail(TextUtil.LineSeparate("Failed to remove cache file. Found files:", TextUtil.LineSeparate(Directory.GetFiles(dirPath))));
}
break; // If we make it here then, successful
}
}
// Cache file has been removed
testFilesDir.Dispose();
}
public void TestMProphetResultsHandler()
{
var testFilesDir = new TestFilesDir(TestContext, ZIP_FILE);
var documentFile = testFilesDir.GetTestPath("MProphetGold-trained.sky");
SrmDocument doc = ResultsUtil.DeserializeDocument(documentFile);
// Load libraries
doc = doc.ChangeSettings(doc.Settings.ChangePeptideLibraries(libraries =>
{
var lib = libraries.Libraries[0];
return(libraries.ChangeLibrarySpecs(new LibrarySpec[]
{
new BiblioSpecLiteSpec(lib.Name, testFilesDir.GetTestPath(lib.FileNameHint))
}));
}));
// Load an empty doc, so that we can make a change and
// cause the .skyd to be loaded
var docContainer = new ResultsTestDocumentContainer(null, documentFile);
docContainer.SetDocument(doc, null, true);
docContainer.AssertComplete();
SrmDocument docOriginal = docContainer.Document;
var peakScoringModel = docOriginal.Settings.PeptideSettings.Integration.PeakScoringModel;
var resultsHandler = new MProphetResultsHandler(docOriginal, peakScoringModel)
{
QValueCutoff = Q_CUTOFF
};
// 1. Reintegrate and export report produces expected file
resultsHandler.ScoreFeatures();
var docNew = resultsHandler.ChangePeaks();
var reportSpec = MakeReportSpec();
if (IsSaveAll)
{
// For regenerating expected files if things change
ReportToCsv(reportSpec, docNew, testFilesDir.GetTestPath(REPORT_EXPECTED), CultureInfo.GetCultureInfo("en-US"));
ReportToCsv(reportSpec, docNew, testFilesDir.GetTestPathIntl(REPORT_EXPECTED), CultureInfo.GetCultureInfo("fr-FR"));
}
string docNewActual = testFilesDir.GetTestPath(REPORT_ACTUAL);
string docNewExpected = testFilesDir.GetTestPathLocale(REPORT_EXPECTED);
ReportToCsv(reportSpec, docNew, docNewActual, CultureInfo.CurrentCulture);
AssertEx.FileEquals(docNewExpected, docNewActual);
// 2. Reintegrating again gives no change in document
var resultsHandlerRepeat = new MProphetResultsHandler(docNew, peakScoringModel)
{
QValueCutoff = Q_CUTOFF
};
resultsHandlerRepeat.ScoreFeatures();
var docRepeat = resultsHandlerRepeat.ChangePeaks();
Assert.AreSame(docRepeat, docNew);
Assert.AreNotSame(docOriginal, docNew);
// 3. Export mProphet results gives expected file
var calcs = peakScoringModel.PeakFeatureCalculators;
var mProphetActual = testFilesDir.GetTestPath(MPROPHET_ACTUAL);
var mProphetExpected = testFilesDir.GetTestPathLocale(MPROPHET_EXPECTED);
if (IsSaveAll)
{
// For regenerating files
SaveMProphetFeatures(resultsHandler, testFilesDir.GetTestPath(MPROPHET_EXPECTED), CultureInfo.GetCultureInfo("en-US"), calcs);
SaveMProphetFeatures(resultsHandler, testFilesDir.GetTestPathIntl(MPROPHET_EXPECTED), CultureInfo.GetCultureInfo("fr-FR"), calcs);
}
SaveMProphetFeatures(resultsHandler, mProphetActual, CultureInfo.CurrentCulture, calcs);
AssertEx.FileEquals(mProphetExpected, mProphetActual);
// 4. Export mProphet -> Import Peak Boundaries leads to same result as reintegrate
var resultsHandlerQAll = new MProphetResultsHandler(docOriginal, peakScoringModel)
{
QValueCutoff = 1.0
};
resultsHandlerQAll.ScoreFeatures();
var docNewQAll = resultsHandlerQAll.ChangePeaks();
var peakBoundaryImporter = new PeakBoundaryImporter(docNewQAll);
long lineCount = Helpers.CountLinesInFile(mProphetActual);
peakBoundaryImporter.Import(mProphetActual, null, lineCount);
var docImport = peakBoundaryImporter.Document;
// Serialized documents are easier to debug when something is different
string strDocNew = SerializeDoc(docNewQAll);
string strDocImport = SerializeDoc(docImport);
AssertEx.NoDiff(strDocNew, strDocImport);
Assert.AreSame(docNewQAll, docImport);
// 5. Reintegration with q value cutoff of <0 causes all peaks set to null
var handlerAllNull = new MProphetResultsHandler(docOriginal, peakScoringModel)
{
QValueCutoff = -0.001
};
handlerAllNull.ScoreFeatures();
var docNull = handlerAllNull.ChangePeaks();
foreach (var transitionNode in docNull.PeptideTransitions)
{
foreach (var chromInfo in transitionNode.ChromInfos)
{
Assert.IsTrue(chromInfo.IsEmpty || transitionNode.IsDecoy);
}
}
// 6. Reintegration adjusts example peak to null at q=0.005 cutoff, but adjusts it to a non-null peak at q=0.20
const int groupNum = 11;
var midQNode = resultsHandler.Document.PeptideTransitionGroups.ToList()[groupNum];
foreach (var chromInfo in midQNode.Transitions.SelectMany(transition => transition.ChromInfos))
{
Assert.IsTrue(chromInfo.IsEmpty);
}
resultsHandler.QValueCutoff = Q_CUTOFF_HIGH;
resultsHandler.ChangePeaks();
var midQNodeNew = resultsHandler.Document.PeptideTransitionGroups.ToList()[groupNum];
foreach (var chromInfo in midQNodeNew.Transitions.SelectMany(transition => transition.ChromInfos))
{
Assert.IsFalse(chromInfo.IsEmpty);
}
// 7. Labeled peptide pairs still have matching peaks
foreach (var peptideNode in resultsHandler.Document.Peptides)
{
Assert.AreEqual(2, peptideNode.TransitionGroupCount);
var groupList = peptideNode.TransitionGroups.ToList();
var lightGroup = groupList[0];
var heavyGroup = groupList[0];
var lightChromInfo = lightGroup.ChromInfos.ToList()[0];
var heavyChromInfo = heavyGroup.ChromInfos.ToList()[0];
Assert.AreEqual(lightChromInfo.StartRetentionTime, heavyChromInfo.StartRetentionTime);
Assert.AreEqual(lightChromInfo.EndRetentionTime, heavyChromInfo.EndRetentionTime);
Assert.AreEqual(lightChromInfo.RetentionTime, heavyChromInfo.RetentionTime);
}
// 8. Verify that chosen peaks and q values are the same as those in mProphet paper:
// http://www.nature.com/nmeth/journal/v8/n5/full/nmeth.1584.html#/supplementary-information
// TODO: Grab this data from the mProphet paper
// Release open streams
docContainer.Release();
}
private void PerformTestMeasuredDriftValues(bool asSmallMolecules)
{
if (asSmallMolecules)
{
if (!RunSmallMoleculeTestVersions)
{
Console.Write(MSG_SKIPPING_SMALLMOLECULE_TEST_VERSION);
return;
}
}
var testFilesDir = new TestFilesDir(TestContext, @"TestData\Results\BlibDriftTimeTest.zip"); // Re-used from BlibDriftTimeTest
// Open document with some peptides but no results
var docPath = testFilesDir.GetTestPath("BlibDriftTimeTest.sky");
// This was a malformed document, which caused problems after a fix to not recalculate
// document library settings on open. To avoid rewriting this test for the document
// which now contains 2 precursors, the first precursor is removed immediately.
SrmDocument docOriginal = ResultsUtil.DeserializeDocument(docPath);
var pathFirstPeptide = docOriginal.GetPathTo((int)SrmDocument.Level.Molecules, 0);
var nodeFirstPeptide = (DocNodeParent)docOriginal.FindNode(pathFirstPeptide);
docOriginal = (SrmDocument)docOriginal.RemoveChild(pathFirstPeptide, nodeFirstPeptide.Children[0]);
if (asSmallMolecules)
{
var refine = new RefinementSettings();
docOriginal = refine.ConvertToSmallMolecules(docOriginal, testFilesDir.FullPath);
}
using (var docContainer = new ResultsTestDocumentContainer(docOriginal, docPath))
{
var doc = docContainer.Document;
// Import an mz5 file that contains drift info
const string replicateName = "ID12692_01_UCA168_3727_040714";
var chromSets = new[]
{
new ChromatogramSet(replicateName, new[]
{ new MsDataFilePath(testFilesDir.GetTestPath("ID12692_01_UCA168_3727_040714" + ExtensionTestContext.ExtMz5)), }),
};
var docResults = doc.ChangeMeasuredResults(new MeasuredResults(chromSets));
Assert.IsTrue(docContainer.SetDocument(docResults, docOriginal, true));
docContainer.AssertComplete();
var document = docContainer.Document;
document = document.ChangeSettings(document.Settings.ChangePeptidePrediction(prediction => new PeptidePrediction(null, IonMobilityPredictor.EMPTY)));
// Verify ability to extract predictions from raw data
var newPred = document.Settings.PeptideSettings.Prediction.IonMobilityPredictor.ChangeMeasuredIonMobilityValuesFromResults(
document, docContainer.DocumentFilePath, true);
var result = newPred.MeasuredMobilityIons;
Assert.AreEqual(1, result.Count);
const double expectedDT = 4.0019;
var expectedOffset = .4829;
Assert.AreEqual(expectedDT, result.Values.First().IonMobility.Mobility.Value, .001);
Assert.AreEqual(expectedOffset, result.Values.First().HighEnergyIonMobilityValueOffset, .001);
// Check ability to update, and to preserve unchanged
var revised = new Dictionary <LibKey, IonMobilityAndCCS>();
var libKey = result.Keys.First();
revised.Add(libKey, IonMobilityAndCCS.GetIonMobilityAndCCS(IonMobilityValue.GetIonMobilityValue(4, eIonMobilityUnits.drift_time_msec), null, 0.234)); // N.B. CCS handling would require actual raw data in this test, it's covered in a perf test
var libKey2 = new LibKey("DEADEELS", asSmallMolecules ? Adduct.NonProteomicProtonatedFromCharge(2) : Adduct.DOUBLY_PROTONATED);
revised.Add(libKey2, IonMobilityAndCCS.GetIonMobilityAndCCS(IonMobilityValue.GetIonMobilityValue(5, eIonMobilityUnits.drift_time_msec), null, 0.123));
document =
document.ChangeSettings(
document.Settings.ChangePeptidePrediction(prediction => new PeptidePrediction(null, new IonMobilityPredictor("test", revised, null, null, IonMobilityWindowWidthCalculator.IonMobilityPeakWidthType.resolving_power, 40, 0, 0))));
newPred = document.Settings.PeptideSettings.Prediction.ChangeDriftTimePredictor(
document.Settings.PeptideSettings.Prediction.IonMobilityPredictor.ChangeMeasuredIonMobilityValuesFromResults(
document, docContainer.DocumentFilePath, true)).IonMobilityPredictor;
result = newPred.MeasuredMobilityIons;
Assert.AreEqual(2, result.Count);
Assert.AreEqual(expectedDT, result[libKey].IonMobility.Mobility.Value, .001);
Assert.AreEqual(expectedOffset, result[libKey].HighEnergyIonMobilityValueOffset, .001);
Assert.AreEqual(5, result[libKey2].IonMobility.Mobility.Value, .001);
Assert.AreEqual(0.123, result[libKey2].HighEnergyIonMobilityValueOffset, .001);
}
}
public void DoAsymmetricIsolationTest(RefinementSettings.ConvertToSmallMoleculesMode asSmallMolecules)
{
if (asSmallMolecules != RefinementSettings.ConvertToSmallMoleculesMode.none && !RunSmallMoleculeTestVersions)
{
Console.Write(MSG_SKIPPING_SMALLMOLECULE_TEST_VERSION);
return;
}
LocalizationHelper.InitThread(); // TODO: All unit tests should be correctly initialized
var testFilesDir = new TestFilesDir(TestContext, ZIP_FILE);
string docPath = testFilesDir.GetTestPath("Asym_DIA.sky");
string cachePath = ChromatogramCache.FinalPathForName(docPath, null);
FileEx.SafeDelete(cachePath);
SrmDocument doc = ResultsUtil.DeserializeDocument(docPath);
var refine = new RefinementSettings();
doc = refine.ConvertToSmallMolecules(doc, testFilesDir.FullPath, asSmallMolecules);
const int expectedMoleculeCount = 1; // At first small molecules did not support multiple charge states, and this was 2 for that test mode
AssertEx.IsDocumentState(doc, null, 1, expectedMoleculeCount, 2, 4);
var fullScanInitial = doc.Settings.TransitionSettings.FullScan;
Assert.IsTrue(fullScanInitial.IsEnabledMsMs);
Assert.AreEqual(FullScanAcquisitionMethod.DIA, fullScanInitial.AcquisitionMethod);
Assert.AreEqual(25, fullScanInitial.PrecursorFilter);
AssertEx.Serializable(doc);
using (var docContainer = new ResultsTestDocumentContainer(doc, docPath))
{
// Import the first RAW file (or mzML for international)
string rawPath = testFilesDir.GetTestPath("Asym_DIA_data.mzML");
var measuredResults = new MeasuredResults(new[] { new ChromatogramSet("Single", new[] { rawPath }) });
TransitionGroupDocNode nodeGroup;
double ratio;
const double poorRatio = 0.25;
const double fixedRatio = 1.05;
{
// Import with symmetric isolation window
SrmDocument docResults = docContainer.ChangeMeasuredResults(measuredResults, expectedMoleculeCount, 1, 1, 2, 2);
nodeGroup = docResults.MoleculeTransitionGroups.First();
ratio = nodeGroup.Results[0][0].Ratio ?? 0;
// The expected ratio is 1.0, but the symmetric isolation window should produce poor results
if (asSmallMolecules != RefinementSettings.ConvertToSmallMoleculesMode.masses_only) // Can't use labels without a formula
{
Assert.AreEqual(poorRatio, ratio, 0.05);
}
// Revert to original document, and get rid of results cache
Assert.IsTrue(docContainer.SetDocument(doc, docResults, false));
FileEx.SafeDelete(testFilesDir.GetTestPath("Asym_DIA.skyd"));
}
{
// Import with asymmetric isolation window
SrmDocument docAsym = doc.ChangeSettings(doc.Settings.ChangeTransitionFullScan(fullScan =>
fullScan.ChangeAcquisitionMethod(fullScan.AcquisitionMethod, new IsolationScheme("Test asym", 5, 20))));
AssertEx.Serializable(docAsym);
Assert.IsTrue(docContainer.SetDocument(docAsym, doc, false));
SrmDocument docResults = docContainer.ChangeMeasuredResults(measuredResults, expectedMoleculeCount, 1, 1, 2, 2);
nodeGroup = docResults.MoleculeTransitionGroups.First();
ratio = nodeGroup.Results[0][0].Ratio ?? 0;
// Asymmetric should be a lot closer to 1.0
if (asSmallMolecules != RefinementSettings.ConvertToSmallMoleculesMode.masses_only) // Can't use labels without a formula
{
Assert.AreEqual(fixedRatio, ratio, 0.05);
}
// Revert to original document, and get rid of results cache
Assert.IsTrue(docContainer.SetDocument(doc, docResults, false));
FileEx.SafeDelete(testFilesDir.GetTestPath("Asym_DIA.skyd"));
}
{
// Import with prespecified isolation windows
var windowList = new List <IsolationWindow>
{
new IsolationWindow(999.2702214, 1024.270221),
new IsolationWindow(1024.27267, 1049.27267)
};
SrmDocument docPrespecified = doc.ChangeSettings(doc.Settings.ChangeTransitionFullScan(fullScan =>
fullScan.ChangeAcquisitionMethod(fullScan.AcquisitionMethod, new IsolationScheme("Test prespecified", windowList))));
AssertEx.Serializable(docPrespecified);
Assert.IsTrue(docContainer.SetDocument(docPrespecified, doc, false));
SrmDocument docResults = docContainer.ChangeMeasuredResults(measuredResults, expectedMoleculeCount, 1, 1, 2, 2);
nodeGroup = docResults.MoleculeTransitionGroups.First();
ratio = nodeGroup.Results[0][0].Ratio ?? 0;
// Asymmetric should be a lot closer to 1.0
if (asSmallMolecules != RefinementSettings.ConvertToSmallMoleculesMode.masses_only) // Can't use labels without a formula
{
Assert.AreEqual(fixedRatio, ratio, 0.05);
}
// Revert to original document, and get rid of results cache
Assert.IsTrue(docContainer.SetDocument(doc, docResults, false));
FileEx.SafeDelete(testFilesDir.GetTestPath("Asym_DIA.skyd"));
}
{
// Import with prespecified targets
var windowList = new List <IsolationWindow>
{
new IsolationWindow(999.2702214, 1024.270221, 1004.27),
new IsolationWindow(1024.27267, 1049.27267, 1029.27)
};
SrmDocument docPrespecified = doc.ChangeSettings(doc.Settings.ChangeTransitionFullScan(fullScan =>
fullScan.ChangeAcquisitionMethod(fullScan.AcquisitionMethod, new IsolationScheme("Test target", windowList))));
AssertEx.Serializable(docPrespecified);
Assert.IsTrue(docContainer.SetDocument(docPrespecified, doc, false));
SrmDocument docResults = docContainer.ChangeMeasuredResults(measuredResults, expectedMoleculeCount, 1, 1, 2, 2);
nodeGroup = docResults.MoleculeTransitionGroups.First();
ratio = nodeGroup.Results[0][0].Ratio ?? 0;
// Asymmetric should be a lot closer to 1.0
if (asSmallMolecules != RefinementSettings.ConvertToSmallMoleculesMode.masses_only) // Can't use labels without a formula
{
Assert.AreEqual(fixedRatio, ratio, 0.05);
}
// Revert to original document, and get rid of results cache
Assert.IsTrue(docContainer.SetDocument(doc, docResults, false));
FileEx.SafeDelete(testFilesDir.GetTestPath("Asym_DIA.skyd"));
}
{
// Import with ambiguous prespecified targets
var windowList = new List <IsolationWindow>
{
new IsolationWindow(999.2702214, 1024.270221, 1004.27),
new IsolationWindow(1000.0, 1049.27267, 1004.28)
};
SrmDocument docAmbiguous = doc.ChangeSettings(doc.Settings.ChangeTransitionFullScan(fullScan =>
fullScan.ChangeAcquisitionMethod(fullScan.AcquisitionMethod, new IsolationScheme("Test ambiguous", windowList))));
AssertEx.Serializable(docAmbiguous);
Assert.IsTrue(docContainer.SetDocument(docAmbiguous, doc, false));
try
{
docContainer.ChangeMeasuredResults(measuredResults, expectedMoleculeCount, 1, 1, 2, 2);
Assert.Fail("Expected ambiguous isolation targets.");
}
catch (Exception x)
{
AssertEx.AreComparableStrings(Resources.SpectrumFilter_FindFilterPairs_Two_isolation_windows_contain_targets_which_match_the_isolation_target__0__, x.Message, 1);
}
// Revert to original document, and get rid of results cache
Assert.IsTrue(docContainer.SetDocument(doc, docContainer.Document, false));
FileEx.SafeDelete(testFilesDir.GetTestPath("Asym_DIA.skyd"));
}
{
// Import with one isolation window, so one result is discarded.
var windowList = new List <IsolationWindow>
{
new IsolationWindow(999.2702214, 1024.270221),
};
SrmDocument docOneWindow = doc.ChangeSettings(doc.Settings.ChangeTransitionFullScan(fullScan =>
fullScan.ChangeAcquisitionMethod(fullScan.AcquisitionMethod, new IsolationScheme("Test one window", windowList))));
AssertEx.Serializable(docOneWindow);
Assert.IsTrue(docContainer.SetDocument(docOneWindow, doc, false));
SrmDocument docResults = docContainer.ChangeMeasuredResults(measuredResults, 1, 1, 0, 2, 0);
nodeGroup = docResults.MoleculeTransitionGroups.First();
Assert.IsNull(nodeGroup.Results[0][0].Ratio);
// Revert to original document, and get rid of results cache
Assert.IsTrue(docContainer.SetDocument(doc, docResults, false));
FileEx.SafeDelete(testFilesDir.GetTestPath("Asym_DIA.skyd"));
}
}
testFilesDir.Dispose();
}
private void DoFullScanFilterTest(RefinementSettings.ConvertToSmallMoleculesMode asSmallMolecules,
out List <SrmDocument> docCheckpoints, bool centroided = false)
{
docCheckpoints = new List <SrmDocument>();
TestSmallMolecules = false; // We test small molecules explicitly
var testFilesDir = new TestFilesDir(TestContext, ZIP_FILE);
string docPath = testFilesDir.GetTestPath("BSA_Protea_label_free_20100323_meth3_multi.sky");
var expectedPepCount = 7;
var expectedTransGroupCount = 7;
var expectedTransCount = 49;
var doc = InitFullScanDocument(docPath, 2, ref expectedPepCount, ref expectedTransGroupCount, ref expectedTransCount, asSmallMolecules);
if (centroided && ExtensionTestContext.CanImportThermoRaw)
{
const double ppm20 = 20.0;
doc = doc.ChangeSettings(doc.Settings.ChangeTransitionFullScan(fs =>
fs.ChangePrecursorResolution(FullScanMassAnalyzerType.centroided, ppm20, 0)));
}
using (var docContainer = new ResultsTestDocumentContainer(doc, docPath))
{
// Import the first RAW file (or mzML for international)
string rawPath = testFilesDir.GetTestPath("ah_20101011y_BSA_MS-MS_only_5-2" +
ExtensionTestContext.ExtThermoRaw);
var measuredResults = new MeasuredResults(new[] { new ChromatogramSet("Single", new[] { new MsDataFilePath(rawPath) }) });
SrmDocument docResults = docContainer.ChangeMeasuredResults(measuredResults, 3, 3, 21);
docCheckpoints.Add(docResults);
// Refilter allowing multiple precursors per spectrum
SrmDocument docMulti = doc.ChangeSettings(doc.Settings.ChangeTransitionFullScan(
fs => fs.ChangeAcquisitionMethod(FullScanAcquisitionMethod.DIA, new IsolationScheme("Test", 2))));
AssertEx.Serializable(docMulti, AssertEx.DocumentCloned);
// Release data cache file
Assert.IsTrue(docContainer.SetDocument(docMulti, docResults));
// And remove it
FileEx.SafeDelete(Path.ChangeExtension(docPath, ChromatogramCache.EXT));
docCheckpoints.Add(docContainer.ChangeMeasuredResults(measuredResults, 6, 6, 38));
// Import full scan Orbi-Velos data
docPath = testFilesDir.GetTestPath("BSA_Protea_label_free_20100323_meth3_long_acc_template.sky");
expectedPepCount = 3;
expectedTransGroupCount = 3;
expectedTransCount = 21;
doc = InitFullScanDocument(docPath, 1, ref expectedPepCount, ref expectedTransGroupCount, ref expectedTransCount, asSmallMolecules);
docCheckpoints.Add(doc);
Assert.AreEqual(FullScanMassAnalyzerType.orbitrap, doc.Settings.TransitionSettings.FullScan.ProductMassAnalyzer);
// Make sure saving this type of document works
AssertEx.Serializable(doc, AssertEx.DocumentCloned);
Assert.IsTrue(docContainer.SetDocument(doc, docContainer.Document));
rawPath = testFilesDir.GetTestPath("ah_20101029r_BSA_CID_FT_centroid_3uscan_3" +
ExtensionTestContext.ExtThermoRaw);
measuredResults = new MeasuredResults(new[] { new ChromatogramSet("Accurate", new[] { rawPath }) });
docCheckpoints.Add(docContainer.ChangeMeasuredResults(measuredResults, 3, 3, 21));
// Import LTQ data with MS1 and MS/MS
docPath = testFilesDir.GetTestPath("BSA_Protea_label_free_20100323_meth3_test4.sky");
expectedPepCount = 3;
expectedTransGroupCount = 4;
expectedTransCount = 32;
doc = InitFullScanDocument(docPath, 3, ref expectedPepCount, ref expectedTransGroupCount, ref expectedTransCount, asSmallMolecules);
Assert.AreEqual(FullScanMassAnalyzerType.none, doc.Settings.TransitionSettings.FullScan.ProductMassAnalyzer);
Assert.AreEqual(FullScanMassAnalyzerType.none, doc.Settings.TransitionSettings.FullScan.PrecursorMassAnalyzer);
docCheckpoints.Add(doc);
var docBoth = doc.ChangeSettings(doc.Settings.ChangeTransitionFullScan(fs =>
fs.ChangeAcquisitionMethod(FullScanAcquisitionMethod.Targeted, null)
.ChangePrecursorResolution(FullScanMassAnalyzerType.qit, TransitionFullScan.DEFAULT_RES_QIT, null)));
docCheckpoints.Add(docBoth);
AssertEx.Serializable(docBoth, AssertEx.DocumentCloned);
Assert.IsTrue(docContainer.SetDocument(docBoth, docContainer.Document));
string dataPath = testFilesDir.GetTestPath("klc_20100329v_Protea_Peptide_Curve_200fmol_uL_tech1.mzML");
var listResults = new List <ChromatogramSet>
{
new ChromatogramSet("MS1 and MS/MS", new[] { dataPath }),
};
measuredResults = new MeasuredResults(listResults.ToArray());
docCheckpoints.Add(docContainer.ChangeMeasuredResults(measuredResults, expectedPepCount, expectedTransGroupCount, expectedTransCount - 6));
// The mzML was filtered for the m/z range 410 to 910.
foreach (var nodeTran in docContainer.Document.PeptideTransitions)
{
Assert.IsTrue(nodeTran.HasResults);
Assert.IsNotNull(nodeTran.Results[0]);
if (410 > nodeTran.Mz || nodeTran.Mz > 910)
{
Assert.IsTrue(nodeTran.Results[0][0].IsEmpty);
}
else
{
Assert.IsFalse(nodeTran.Results[0][0].IsEmpty);
}
}
// Import LTQ data with MS1 and MS/MS using multiple files for a single replicate
listResults.Add(new ChromatogramSet("Multi-file", new[]
{
testFilesDir.GetTestPath("both_DRV.mzML"),
testFilesDir.GetTestPath("both_KVP.mzML"),
}));
measuredResults = new MeasuredResults(listResults.ToArray());
docCheckpoints.Add(docContainer.ChangeMeasuredResults(measuredResults, expectedPepCount - 1, expectedTransGroupCount - 1, expectedTransCount - 6));
int indexResults = listResults.Count - 1;
foreach (var nodeTran in docContainer.Document.PeptideTransitions)
{
Assert.IsTrue(nodeTran.HasResults);
Assert.AreEqual(listResults.Count, nodeTran.Results.Count);
var peptide = nodeTran.Transition.Group.Peptide;
// DRV without FASTA sequence should not have data for non-precursor transitions
if (!peptide.Sequence.StartsWith("DRV") || !peptide.Begin.HasValue)
{
Assert.IsNotNull(nodeTran.Results[indexResults]);
Assert.IsFalse(nodeTran.Results[indexResults][0].IsEmpty);
}
else if (nodeTran.Transition.IonType != IonType.precursor)
{
Assert.IsNull(nodeTran.Results[indexResults]);
}
else
{
// Random, bogus peaks chosen in both files
Assert.IsNotNull(nodeTran.Results[indexResults]);
Assert.AreEqual(2, nodeTran.Results[indexResults].Count);
Assert.IsFalse(nodeTran.Results[indexResults][0].IsEmpty);
Assert.IsFalse(nodeTran.Results[indexResults][1].IsEmpty);
}
}
if (asSmallMolecules == RefinementSettings.ConvertToSmallMoleculesMode.masses_only)
{
return; // Can't work with isotope distributions when we don't have ion formulas
}
// Verify handling of bad request for vendor centroided data - out-of-range PPM
docPath = testFilesDir.GetTestPath("Yeast_HI3 Peptides_test.sky");
expectedPepCount = 2;
expectedTransGroupCount = 2;
expectedTransCount = 2;
doc = InitFullScanDocument(docPath, 2, ref expectedPepCount, ref expectedTransGroupCount, ref expectedTransCount, asSmallMolecules);
Assert.AreEqual(FullScanMassAnalyzerType.none, doc.Settings.TransitionSettings.FullScan.ProductMassAnalyzer);
Assert.AreEqual(FullScanMassAnalyzerType.none, doc.Settings.TransitionSettings.FullScan.PrecursorMassAnalyzer);
var docBad = doc;
AssertEx.ThrowsException <InvalidDataException>(() =>
docBad.ChangeSettings(docBad.Settings.ChangeTransitionFullScan(fs =>
fs.ChangePrecursorIsotopes(FullScanPrecursorIsotopes.Count, 1, IsotopeEnrichments.DEFAULT)
.ChangePrecursorResolution(FullScanMassAnalyzerType.centroided, 50 * 1000, 400))),
string.Format(Resources.TransitionFullScan_ValidateRes_Mass_accuracy_must_be_between__0__and__1__for_centroided_data_,
TransitionFullScan.MIN_CENTROID_PPM, TransitionFullScan.MAX_CENTROID_PPM));
// Verify relationship between PPM and resolving power
const double ppm = 20.0; // Should yield same filter width as resolving power 50,000 in TOF
var docNoCentroid = doc.ChangeSettings(doc.Settings.ChangeTransitionFullScan(fs =>
fs.ChangePrecursorIsotopes(FullScanPrecursorIsotopes.Count, 1, IsotopeEnrichments.DEFAULT)
.ChangePrecursorResolution(FullScanMassAnalyzerType.centroided, ppm, 0)));
AssertEx.Serializable(docNoCentroid, AssertEx.DocumentCloned);
Assert.IsTrue(docContainer.SetDocument(docNoCentroid, docContainer.Document));
const double mzTest = 400.0;
var filterWidth = docNoCentroid.Settings.TransitionSettings.FullScan.GetPrecursorFilterWindow(mzTest);
Assert.AreEqual(mzTest * 2.0 * ppm * 1E-6, filterWidth);
// Verify handling of bad request for vendor centroided data - ask for centroiding in mzML
const string fileName = "S_2_LVN.mzML";
var filePath = testFilesDir.GetTestPath(fileName);
AssertEx.ThrowsException <AssertFailedException>(() =>
{
listResults = new List <ChromatogramSet> {
new ChromatogramSet("rep1", new[] { new MsDataFilePath(filePath, null, true) }),
};
docContainer.ChangeMeasuredResults(new MeasuredResults(listResults.ToArray()), 1, 1, 1);
},
string.Format(Resources.NoCentroidedDataException_NoCentroidedDataException_No_centroided_data_available_for_file___0_____Adjust_your_Full_Scan_settings_, filePath));
// Import FT data with only MS1
docPath = testFilesDir.GetTestPath("Yeast_HI3 Peptides_test.sky");
expectedPepCount = 2;
expectedTransGroupCount = 2;
expectedTransCount = 2;
doc = InitFullScanDocument(docPath, 2, ref expectedPepCount, ref expectedTransGroupCount, ref expectedTransCount, asSmallMolecules);
Assert.AreEqual(FullScanMassAnalyzerType.none, doc.Settings.TransitionSettings.FullScan.ProductMassAnalyzer);
Assert.AreEqual(FullScanMassAnalyzerType.none, doc.Settings.TransitionSettings.FullScan.PrecursorMassAnalyzer);
var docMs1 = doc.ChangeSettings(doc.Settings.ChangeTransitionFullScan(fs =>
fs.ChangePrecursorIsotopes(FullScanPrecursorIsotopes.Count, 1, IsotopeEnrichments.DEFAULT)
.ChangePrecursorResolution(FullScanMassAnalyzerType.tof, 50 * 1000, null)));
Assert.AreEqual(filterWidth, docMs1.Settings.TransitionSettings.FullScan.GetPrecursorFilterWindow(mzTest));
docMs1 = doc.ChangeSettings(doc.Settings.ChangeTransitionFullScan(fs =>
fs.ChangePrecursorIsotopes(FullScanPrecursorIsotopes.Count, 1, IsotopeEnrichments.DEFAULT)
.ChangePrecursorResolution(FullScanMassAnalyzerType.ft_icr, 50 * 1000, mzTest)));
AssertEx.Serializable(docMs1, AssertEx.DocumentCloned);
Assert.IsTrue(docContainer.SetDocument(docMs1, docContainer.Document));
const string rep1 = "rep1";
listResults = new List <ChromatogramSet>
{
new ChromatogramSet(rep1, new[] { filePath }),
};
measuredResults = new MeasuredResults(listResults.ToArray());
docCheckpoints.Add(docContainer.ChangeMeasuredResults(measuredResults, 1, 1, 1));
// Because of the way the mzML files were filtered, all of the LVN peaks should be present
// in the first replicate, and all of the NVN peaks should be present in the other.
foreach (var nodeTranGroup in docContainer.Document.MoleculeTransitionGroups)
{
foreach (var docNode in nodeTranGroup.Children)
{
var nodeTran = (TransitionDocNode)docNode;
Assert.IsTrue(nodeTran.HasResults);
Assert.AreEqual(1, nodeTran.Results.Count);
if ((nodeTran.Transition.Group.Peptide.IsCustomIon
? nodeTranGroup.CustomIon.Name
: nodeTran.Transition.Group.Peptide.Sequence).StartsWith("LVN"))
{
Assert.IsFalse(nodeTran.Results[0][0].IsEmpty);
}
else
{
Assert.IsTrue(nodeTran.Results[0][0].IsEmpty);
}
}
}
const string rep2 = "rep2";
listResults.Add(new ChromatogramSet(rep2, new[] { testFilesDir.GetTestPath("S_2_NVN.mzML") }));
measuredResults = new MeasuredResults(listResults.ToArray());
docCheckpoints.Add(docContainer.ChangeMeasuredResults(measuredResults, 1, 1, 1));
// Because of the way the mzML files were filtered, all of the LVN peaks should be present
// in the first replicate, and all of the NVN peaks should be present in the other.
foreach (var nodeTranGroup in docContainer.Document.MoleculeTransitionGroups)
{
foreach (var docNode in nodeTranGroup.Children)
{
var nodeTran = (TransitionDocNode)docNode;
Assert.IsTrue(nodeTran.HasResults);
Assert.AreEqual(2, nodeTran.Results.Count);
if ((nodeTran.Transition.Group.Peptide.IsCustomIon
? nodeTranGroup.CustomIon.Name
: nodeTran.Transition.Group.Peptide.Sequence).StartsWith("LVN"))
{
Assert.IsTrue(nodeTran.Results[1][0].IsEmpty);
}
else
{
Assert.IsFalse(nodeTran.Results[1][0].IsEmpty);
}
}
}
// Chromatograms should be present in the cache for a number of isotopes.
var docMs1Isotopes = docContainer.Document.ChangeSettings(doc.Settings
.ChangeTransitionFullScan(fs => fs.ChangePrecursorIsotopes(FullScanPrecursorIsotopes.Count,
3, IsotopeEnrichments.DEFAULT))
.ChangeTransitionFilter(filter => filter.ChangeIonTypes(new[] { IonType.precursor })));
docCheckpoints.Add(docMs1Isotopes);
AssertEx.IsDocumentState(docMs1Isotopes, null, 2, 2, 2); // Need to reset auto-manage for transitions
var refineAutoSelect = new RefinementSettings {
AutoPickChildrenAll = PickLevel.transitions
};
docMs1Isotopes = refineAutoSelect.Refine(docMs1Isotopes);
AssertEx.IsDocumentState(docMs1Isotopes, null, 2, 2, 6);
AssertResult.IsDocumentResultsState(docMs1Isotopes, rep1, 1, 1, 0, 3, 0);
AssertResult.IsDocumentResultsState(docMs1Isotopes, rep2, 1, 1, 0, 3, 0);
docCheckpoints.Add(docMs1Isotopes);
// Add M-1 transitions, and verify that they have chromatogram data also, but
// empty peaks in all cases
var docMs1All = docMs1Isotopes.ChangeSettings(docMs1Isotopes.Settings
.ChangeTransitionFullScan(fs => fs.ChangePrecursorIsotopes(FullScanPrecursorIsotopes.Percent,
0, IsotopeEnrichments.DEFAULT))
.ChangeTransitionIntegration(i => i.ChangeIntegrateAll(false))); // For compatibility with v2.5 and earlier
docCheckpoints.Add(docMs1All);
AssertEx.IsDocumentState(docMs1All, null, 2, 2, 10);
AssertResult.IsDocumentResultsState(docMs1All, rep1, 1, 1, 0, 4, 0);
AssertResult.IsDocumentResultsState(docMs1All, rep2, 1, 1, 0, 4, 0);
var ms1AllTranstions = docMs1All.MoleculeTransitions.ToArray();
var tranM1 = ms1AllTranstions[0];
Assert.AreEqual(-1, tranM1.Transition.MassIndex);
Assert.IsTrue(tranM1.Results[0] != null && tranM1.Results[1] != null);
Assert.IsTrue(tranM1.Results[0][0].IsEmpty && tranM1.Results[1][0].IsEmpty);
tranM1 = ms1AllTranstions[5];
Assert.AreEqual(-1, tranM1.Transition.MassIndex);
Assert.IsTrue(tranM1.Results[0] != null && tranM1.Results[1] != null);
Assert.IsTrue(tranM1.Results[0][0].IsEmpty && tranM1.Results[1][0].IsEmpty);
}
}
/* TODO bspratt drift time libs for small molecules
*
* [TestMethod]
* public void WatersImsMsePredictedDriftTimesChromatogramTestAsSmallMolecules()
* {
* WatersImsMseChromatogramTest(DriftFilterType.predictor, true);
* }
*
* [TestMethod]
* public void WatersImsMseLibraryDriftTimesChromatogramTestAsSmallMolecules()
* {
* WatersImsMseChromatogramTest(DriftFilterType.library, true);
* }
*
*/
private void WatersImsMseChromatogramTest(DriftFilterType mode,
RefinementSettings.ConvertToSmallMoleculesMode asSmallMolecules = RefinementSettings.ConvertToSmallMoleculesMode.none)
{
string subdir = (asSmallMolecules == RefinementSettings.ConvertToSmallMoleculesMode.none) ? null : asSmallMolecules.ToString();
var testFilesDir = new TestFilesDir(TestContext, ZIP_FILE, subdir);
TestSmallMolecules = false; // Don't need that extra magic node
bool withDriftTimePredictor = (mode == DriftFilterType.predictor); // Load the doc that has a drift time predictor?
bool withDriftTimeFilter = (mode != DriftFilterType.none); // Perform drift time filtering? (either with predictor, or with bare times in blib file)
string docPath;
SrmDocument document = InitWatersImsMseDocument(testFilesDir, withDriftTimePredictor ? "single_with_driftinfo.sky" : "single_no_driftinfo.sky", asSmallMolecules, out docPath);
AssertEx.IsDocumentState(document, (withDriftTimePredictor || (asSmallMolecules != RefinementSettings.ConvertToSmallMoleculesMode.none)) ? 1 : 0, 1, 1, 1, 8); // Drift time lib load bumps the doc version
var docContainer = new ResultsTestDocumentContainer(document, docPath);
var doc = docContainer.Document;
var docOriginal = doc;
string testModeStr = withDriftTimePredictor ? "with drift time predictor" : "without drift time info";
if (withDriftTimeFilter && !withDriftTimePredictor)
{
// Use the bare drift times in the spectral library
var librarySpec = new BiblioSpecLiteSpec("drift test",
testFilesDir.GetTestPath("mse-mobility.filtered-scaled.blib"));
doc = doc.ChangeSettings(
doc.Settings.ChangePeptideLibraries(lib => lib.ChangeLibrarySpecs(new[] { librarySpec })).
ChangePeptidePrediction(p => p.ChangeLibraryDriftTimesResolvingPower(100)).
ChangePeptidePrediction(p => p.ChangeUseLibraryDriftTimes(true))
);
testModeStr = "with drift times from spectral library";
}
var listChromatograms = new List <ChromatogramSet>();
// A small subset of the QC_HDMSE_02_UCA168_3495_082213 data set (RT 21.5-22.5) from Will Thompson
const string path = @"waters-mobility.mz5";
listChromatograms.Add(AssertResult.FindChromatogramSet(doc, new MsDataFilePath(path)) ??
new ChromatogramSet(Path.GetFileName(path).Replace('.', '_'), new[] { path }));
var docResults = doc.ChangeMeasuredResults(new MeasuredResults(listChromatograms));
Assert.IsTrue(docContainer.SetDocument(docResults, docOriginal, true));
docContainer.AssertComplete();
document = docContainer.Document;
float tolerance = (float)document.Settings.TransitionSettings.Instrument.MzMatchTolerance;
double maxHeight = 0;
var results = document.Settings.MeasuredResults;
Assert.AreEqual(1, document.MoleculePrecursorPairs.Count());
foreach (var pair in document.MoleculePrecursorPairs)
{
ChromatogramGroupInfo[] chromGroupInfo;
Assert.IsTrue(results.TryLoadChromatogram(0, pair.NodePep, pair.NodeGroup,
tolerance, true, out chromGroupInfo));
Assert.AreEqual(1, chromGroupInfo.Length, testModeStr);
var chromGroup = chromGroupInfo[0];
var expectedPeaks = ((asSmallMolecules == RefinementSettings.ConvertToSmallMoleculesMode.masses_only) ? 6 : 5);
Assert.AreEqual(withDriftTimeFilter ? 3 : expectedPeaks, chromGroup.NumPeaks, testModeStr); // This will be higher if we don't filter on DT
foreach (var tranInfo in chromGroup.TransitionPointSets)
{
maxHeight = Math.Max(maxHeight, tranInfo.MaxIntensity);
}
}
Assert.AreEqual(withDriftTimeFilter? 5226 : 20075, maxHeight, 1, testModeStr); // Without DT filtering, this will be much greater
// now drill down for specific values
int nPeptides = 0;
foreach (var nodePep in document.Molecules.Where(nodePep => nodePep.Results[0] != null))
{
// expecting just one peptide result in this small data set
if (nodePep.Results[0].Sum(chromInfo => chromInfo.PeakCountRatio > 0 ? 1 : 0) > 0)
{
Assert.AreEqual(21.94865, (double)nodePep.GetMeasuredRetentionTime(0), .0001, testModeStr);
Assert.AreEqual(1.0, (double)nodePep.GetPeakCountRatio(0), 0.0001, testModeStr);
nPeptides++;
}
}
Assert.AreEqual(1, nPeptides);
if (withDriftTimePredictor || withDriftTimeFilter)
{
// Verify that the .imdb pr .blib file goes out in the share zipfile
for (int complete = 0; complete <= 1; complete++)
{
var sharePath = testFilesDir.GetTestPath(complete == 1?"share_complete.zip":"share_minimized.zip");
var share = new SrmDocumentSharing(document, docPath, sharePath, complete == 1);
using (var longWaitDlg = new LongWaitDlg
{
// ReSharper disable once LocalizableElement
Text = "unit test WatersImsTest -- sharing document",
})
{
longWaitDlg.PerformWork(null, 1000, share.Share);
Assert.IsFalse(longWaitDlg.IsCanceled);
}
var files = share.ListEntries().ToArray();
Assert.IsTrue(files.Contains(withDriftTimePredictor ? "scaled.imdb" : "mse-mobility.filtered-scaled.blib"));
// And round trip it to make sure we haven't left out any new features in minimized imdb or blib files
using (var longWaitDlg = new LongWaitDlg
{
// ReSharper disable once LocalizableElement
Text = "unit test WatersImsTest",
})
{
longWaitDlg.PerformWork(null, 1000, share.Extract);
Assert.IsFalse(longWaitDlg.IsCanceled);
}
using (TextReader reader = new StreamReader(share.DocumentPath))
{
XmlSerializer documentSerializer = new XmlSerializer(typeof(SrmDocument));
var document2 = (SrmDocument)documentSerializer.Deserialize(reader);
Assert.IsNotNull(document2);
var im = document.Settings.GetIonMobilities(new MsDataFilePath(path));
var pep = document2.Molecules.First();
foreach (TransitionGroupDocNode nodeGroup in pep.Children)
{
double windowDT;
var centerDriftTime = document.Settings.PeptideSettings.Prediction.GetDriftTime(
pep, nodeGroup, im, out windowDT);
Assert.AreEqual(3.86124, centerDriftTime.DriftTimeMsec(false) ?? 0, .0001, testModeStr);
Assert.AreEqual(0.077224865797235934, windowDT, .0001, testModeStr);
}
}
}
}
// Release file handles
docContainer.Release();
testFilesDir.Dispose();
string cachePath = ChromatogramCache.FinalPathForName(docPath, null);
FileEx.SafeDelete(cachePath);
}
public void DocLoadLibrary()
{
// Load the document
var testFilesDir = new TestFilesDir(TestContext, TEST_ZIP_PATH);
string loadPath = testFilesDir.GetTestPath("DocWithLibrary.sky");
string libraryPath = testFilesDir.GetTestPath("Yeast_MRMer_min.blib");
var doc = ResultsUtil.DeserializeDocument(loadPath);
doc = doc.ChangeSettings(doc.Settings.ChangePeptideLibraries(
lib => lib.ChangeLibrarySpecs(new[] { new BiblioSpecLiteSpec(lib.Libraries[0].Name, libraryPath), })));
// Cause library load and subsequent document update
var docContainer = new ResultsTestDocumentContainer(null, loadPath);
docContainer.SetDocument(doc, null, true);
docContainer.AssertComplete();
// Check that library info on peptides and transitions were not recalculated
// during document load
var docLoaded = docContainer.Document;
Assert.AreEqual(6, docLoaded.PeptideCount);
Assert.AreEqual(36, docLoaded.PeptideTransitionCount);
var transitions = docLoaded.PeptideTransitions.ToArray();
Assert.AreEqual("y12", transitions[0].FragmentIonName);
Assert.AreEqual(1, transitions[0].LibInfo.Rank);
Assert.AreEqual("y12", transitions[3].FragmentIonName);
Assert.AreEqual(1, transitions[3].LibInfo.Rank);
Assert.AreEqual("b3", transitions[14].FragmentIonName);
Assert.AreEqual(2, transitions[14].LibInfo.Rank);
Assert.AreEqual("b3", transitions[17].FragmentIonName);
Assert.AreEqual(2, transitions[17].LibInfo.Rank);
var docLibraryChanged = docLoaded.ChangeSettings(docLoaded.Settings.ChangePeptideLibraries(
lib => lib.ChangeLibraries(new LibrarySpec[0], new Library[0])
.ChangeLibrarySpecs(new[] { new BiblioSpecLiteSpec("Test reload", libraryPath), })));
docContainer.SetDocument(docLibraryChanged, docLoaded, true);
var docChangedLoaded = docContainer.Document;
// Check that document changed to be in synch with the library
Assert.AreEqual(3, docChangedLoaded.PeptideCount);
Assert.AreEqual(18, docChangedLoaded.PeptideTransitionCount);
var transitionsNew = docChangedLoaded.PeptideTransitions.ToArray();
Assert.AreEqual("y7", transitionsNew[0].FragmentIonName);
Assert.AreEqual(1, transitionsNew[0].LibInfo.Rank);
Assert.AreEqual("y7", transitionsNew[3].FragmentIonName);
Assert.AreEqual(1, transitionsNew[3].LibInfo.Rank);
Assert.AreEqual("y6", transitionsNew[8].FragmentIonName);
Assert.AreEqual(2, transitionsNew[8].LibInfo.Rank);
Assert.AreEqual("y6", transitionsNew[11].FragmentIonName);
Assert.AreEqual(2, transitionsNew[11].LibInfo.Rank);
for (int i = 1; i < 3; i++)
{
Assert.AreSame(transitions[i], transitionsNew[i]);
Assert.AreSame(transitions[i + 3], transitionsNew[i + 3]);
}
for (int i = 12; i < 14; i++)
{
Assert.AreSame(transitions[i], transitionsNew[i - 6]);
Assert.AreSame(transitions[i + 3], transitionsNew[i - 3]);
}
for (int i = 24; i < 27; i++)
{
Assert.AreSame(transitions[i], transitionsNew[i - 12]);
Assert.AreSame(transitions[i + 3], transitionsNew[i - 9]);
}
// Release open streams
docContainer.Release();
}
public void DoAgilentMseChromatogramTest(RefinementSettings.ConvertToSmallMoleculesMode asSmallMolecules, small_mol_mode smallMolMode = small_mol_mode.simple, string expectedError = null)
{
if (asSmallMolecules != RefinementSettings.ConvertToSmallMoleculesMode.none && !RunSmallMoleculeTestVersions && smallMolMode == small_mol_mode.simple)
{
System.Console.Write(MSG_SKIPPING_SMALLMOLECULE_TEST_VERSION);
return;
}
var testFilesDir = new TestFilesDir(TestContext, ZIP_FILE);
string docPath;
SrmDocument document = InitAgilentMseDocument(testFilesDir, out docPath);
if (asSmallMolecules != RefinementSettings.ConvertToSmallMoleculesMode.none)
{
var refine = new RefinementSettings();
document = refine.ConvertToSmallMolecules(document, testFilesDir.FullPath, asSmallMolecules, smallMolMode == small_mol_mode.simple ? RefinementSettings.ConvertToSmallMoleculesChargesMode.none : RefinementSettings.ConvertToSmallMoleculesChargesMode.invert);
}
using (var docContainer = new ResultsTestDocumentContainer(document, docPath))
{
var doc = docContainer.Document;
var listChromatograms = new List <ChromatogramSet>();
var path = MsDataFileUri.Parse(smallMolMode == small_mol_mode.invert_charges_and_data ? @"AgilentMse\BSA-AI-0-10-25-41_first_100_scans_neg.mzML" : @"AgilentMse\BSA-AI-0-10-25-41_first_100_scans.mzML");
listChromatograms.Add(AssertResult.FindChromatogramSet(doc, path) ??
new ChromatogramSet(path.GetFileName().Replace('.', '_'), new[] { path }));
var docResults = doc.ChangeMeasuredResults(new MeasuredResults(listChromatograms));
Assert.IsTrue(docContainer.SetDocument(docResults, doc, true));
if (expectedError != null)
{
docContainer.AssertError(expectedError);
}
else
{
docContainer.AssertComplete();
document = docContainer.Document;
float tolerance = (float)document.Settings.TransitionSettings.Instrument.MzMatchTolerance;
var results = document.Settings.MeasuredResults;
foreach (var pair in document.MoleculePrecursorPairs)
{
ChromatogramGroupInfo[] chromGroupInfo;
Assert.IsTrue(results.TryLoadChromatogram(0, pair.NodePep, pair.NodeGroup,
tolerance, true, out chromGroupInfo));
Assert.AreEqual(1, chromGroupInfo.Length);
VerifyMs1Truncated(chromGroupInfo.First());
}
// now drill down for specific values
int nPeptides = 0;
foreach (var nodePep in document.Molecules.Where(nodePep => !nodePep.Results[0].IsEmpty))
{
// expecting just one peptide result in this small data set
if (nodePep.Results[0].Any(chromInfo => chromInfo.PeakCountRatio > 0))
{
Assert.AreEqual(0.25205,
(double)nodePep.GetMeasuredRetentionTime(0), .0001, "averaged retention time differs in node " + nodePep.ModifiedTarget);
Assert.AreEqual(0.3333, (double)nodePep.GetPeakCountRatio(0), 0.0001);
nPeptides++;
}
}
Assert.AreEqual(smallMolMode == small_mol_mode.invert_charges ? 0 : 1, nPeptides); // If we switched document polarity, we'd expect no chromatograms extracted
}
}
testFilesDir.Dispose();
}
public void WiffResultsTest()
{
TestFilesDir testFilesDir = new TestFilesDir(TestContext, ZIP_FILE);
SrmDocument doc = InitWiffDocument(testFilesDir);
using (var docContainer = new ResultsTestDocumentContainer(doc,
testFilesDir.GetTestPath("SimpleWiffTest.sky")))
{
FileEx.SafeDelete(ChromatogramCache.FinalPathForName(docContainer.DocumentFilePath, null));
var listChromatograms = new List <ChromatogramSet>();
if (ExtensionTestContext.CanImportAbWiff)
{
string pathWiff = testFilesDir.GetTestPath("051309_digestion.wiff");
string[] dataIds = MsDataFileImpl.ReadIds(pathWiff);
for (int i = 0; i < dataIds.Length; i++)
{
string nameSample = dataIds[i];
if (!Equals(nameSample, "test") && listChromatograms.Count == 0)
{
continue;
}
string pathSample = SampleHelp.EncodePath(pathWiff, nameSample, i, LockMassParameters.EMPTY, false, false);
listChromatograms.Add(new ChromatogramSet(nameSample, new[] { MsDataFileUri.Parse(pathSample) }));
}
}
else
{
listChromatograms.Add(new ChromatogramSet("test",
new[] { MsDataFileUri.Parse(testFilesDir.GetTestPath("051309_digestion-test.mzML")) }));
listChromatograms.Add(new ChromatogramSet("rfp9,before,h,1",
new[] { MsDataFileUri.Parse(testFilesDir.GetTestPath("051309_digestion-rfp9,before,h,1.mzML")) }));
}
// Should have added test and one after
Assert.AreEqual(2, listChromatograms.Count);
var docResults = doc.ChangeMeasuredResults(new MeasuredResults(listChromatograms));
Assert.IsTrue(docContainer.SetDocument(docResults, doc, true));
docContainer.AssertComplete();
docResults = docContainer.Document;
AssertEx.IsDocumentState(docResults, 6, 9, 9, 18, 54);
Assert.IsTrue(docResults.Settings.MeasuredResults.IsLoaded);
foreach (var nodeTran in docResults.PeptideTransitions)
{
Assert.IsTrue(nodeTran.HasResults);
Assert.AreEqual(2, nodeTran.Results.Count);
}
// Remove the last chromatogram
listChromatograms.RemoveAt(1);
var docResultsSingle = docResults.ChangeMeasuredResults(new MeasuredResults(listChromatograms));
AssertResult.IsDocumentResultsState(docResultsSingle, "test", 9, 2, 9, 10, 27);
// Add mzXML version of test sample
listChromatograms.Add(new ChromatogramSet("test-mzXML", new[] { MsDataFileUri.Parse(testFilesDir.GetTestPath("051309_digestion-s3.mzXML")) }));
var docMzxml = docResults.ChangeMeasuredResults(new MeasuredResults(listChromatograms));
Assert.IsTrue(docContainer.SetDocument(docMzxml, docResults, true));
docContainer.AssertComplete();
docMzxml = docContainer.Document;
// Verify mzXML and native contained same results
// Unfortunately mzWiff produces chromatograms with now zeros, which
// need to be interpolated into place. This means a .wiff file and
// its mzWiff mzXML file will never be the same.
AssertResult.MatchChromatograms(docMzxml, 0, 1, -1, 0);
}
// TODO: Switch to a using clause when PWiz is fixed, and this assertion fails
// AssertEx.ThrowsException<IOException>(() => testFilesDir.Dispose());
}
public void WatersMultiReplicateTest()
{
var testFilesDir = new TestFilesDir(TestContext, ZIP_FILE);
string docPath;
SrmDocument docOriginal = InitWatersDocument(testFilesDir, out docPath);
SrmDocument doc = docOriginal;
var docContainer = new ResultsTestDocumentContainer(doc, docPath);
string extRaw = ExtensionTestContext.ExtWatersRaw;
string[] replicatePaths =
{
testFilesDir.GetTestPath("160109_Mix1_calcurve_070.mzML"),
testFilesDir.GetTestPath("160109_Mix1_calcurve_073.mzML"),
testFilesDir.GetTestPath("160109_Mix1_calcurve_075" + extRaw),
testFilesDir.GetTestPath("160109_Mix1_calcurve_078.mzML")
};
// Count peaks where higher concentration replicates show less area
int outOfOrder = 0;
foreach (string path in replicatePaths)
{
var listChromatograms = new List <ChromatogramSet>();
if (doc.Settings.HasResults)
{
listChromatograms.AddRange(doc.Settings.MeasuredResults.Chromatograms);
}
string name = Path.GetFileNameWithoutExtension(path);
if (name != null)
{
name = name.Substring(name.Length - 12);
}
listChromatograms.Add(new ChromatogramSet(name, new[] { MsDataFileUri.Parse(path) }));
int len = listChromatograms.Count;
var docResults = doc.ChangeMeasuredResults(new MeasuredResults(listChromatograms));
// Adding unloaded results should add a new null result.
foreach (var nodeTran in docResults.PeptideTransitions)
{
Assert.IsTrue(nodeTran.HasResults);
Assert.AreEqual(listChromatograms.Count, nodeTran.Results.Count);
Assert.IsNull(nodeTran.Results[len - 1]);
}
Assert.IsTrue(docContainer.SetDocument(docResults, doc, true),
string.Format("Failed adding results for {0}.", path));
docContainer.AssertComplete();
docResults = docContainer.Document;
Assert.IsTrue(docResults.Settings.MeasuredResults.IsLoaded);
var transOld = doc.PeptideTransitions.ToArray();
var transNew = docResults.PeptideTransitions.ToArray();
Assert.AreEqual(transOld.Length, transNew.Length);
int countPeaks = 0;
for (int i = 0; i < transNew.Length; i++)
{
// Make sure new peak was added to each transition
var nodeTranNew = transNew[i];
Assert.IsTrue(nodeTranNew.HasResults);
Assert.AreEqual(len, nodeTranNew.Results.Count);
var chromInfo = nodeTranNew.Results[len - 1][0];
Assert.IsNotNull(chromInfo);
if (!chromInfo.IsEmpty)
{
countPeaks++;
}
// Make sure previously loaded peaks did not change
for (int j = 0; j < len - 1; j++)
{
var chromInfoPrevious = transOld[i].Results[j][0];
Assert.AreSame(chromInfoPrevious, nodeTranNew.Results[j][0]);
if ((chromInfo.IsEmpty && !chromInfoPrevious.IsEmpty) ||
(!chromInfo.IsEmpty && chromInfoPrevious.Area >= chromInfo.Area))
{
outOfOrder++;
}
}
}
// Allow 2 missed peaks
Assert.IsTrue(countPeaks >= transNew.Length - (TestSmallMolecules ? 1 : 0) - 2);
// Check results calculations for peptides and groups
foreach (var nodePep in docResults.Peptides)
{
Assert.AreEqual(len, nodePep.Results.Count);
Assert.IsTrue(nodePep.HasResults);
var chromInfo = nodePep.Results[len - 1][0];
Assert.AreEqual(1, nodePep.Children.Count);
var nodeGroup = (TransitionGroupDocNode)nodePep.Children[0];
Assert.IsTrue(nodeGroup.HasResults);
Assert.AreEqual(len, nodeGroup.Results.Count);
var chromInfoGroup = nodeGroup.Results[len - 1][0];
Assert.IsTrue(chromInfoGroup.PeakCountRatio >= 0.5);
Assert.IsTrue(chromInfoGroup.RetentionTime.HasValue);
Assert.IsTrue(chromInfoGroup.Area.HasValue && chromInfoGroup.Area > 290);
Assert.AreEqual(chromInfo.RetentionTime, chromInfoGroup.RetentionTime);
Assert.AreEqual(chromInfo.PeakCountRatio, chromInfoGroup.PeakCountRatio);
}
doc = docResults;
}
Assert.AreEqual(13, outOfOrder, 1);
// Remove the original data
foreach (string path in replicatePaths)
{
if (File.Exists(path))
{
FileEx.SafeDelete(path);
}
else
{
DirectoryEx.SafeDelete(path);
}
}
FileEx.SafeDelete(docPath);
// Save the document
string xmlSaved = null;
var docPersisted = AssertEx.RoundTrip(doc, ref xmlSaved);
Assert.IsTrue(!docPersisted.Settings.MeasuredResults.IsLoaded);
// Make sure the persisted document round-trips.
// The orginal doesn't because of changing precision in the results info.
AssertEx.Serializable(docPersisted, AssertEx.DocumentCloned);
// Make sure the loaded document has reasonable results info
// before the cache files are loaded
for (int i = 0; i < doc.Children.Count; i++)
{
PeptideGroupDocNode nodePepGroup1 = (PeptideGroupDocNode)doc.Children[i];
if (TestSmallMolecules && nodePepGroup1.Name.Equals(SrmDocument.TestingNonProteomicMoleculeGroupName))
{
continue;
}
PeptideGroupDocNode nodePepGroup2 = (PeptideGroupDocNode)docPersisted.Children[i];
Assert.AreNotSame(nodePepGroup1, nodePepGroup2);
for (int j = 0; j < nodePepGroup1.Children.Count; j++)
{
PeptideDocNode nodePep1 = (PeptideDocNode)nodePepGroup1.Children[j];
PeptideDocNode nodePep2 = (PeptideDocNode)nodePepGroup2.Children[j];
Assert.AreNotSame(nodePep1, nodePep2);
Assert.AreEqual(nodePep1.Results.Count, nodePep2.Results.Count);
for (int k = 0; k < nodePep1.Results.Count; k++)
{
Assert.AreEqual(nodePep1.Results[k][0].PeakCountRatio, nodePep2.Results[k][0].PeakCountRatio);
}
for (int k = 0; k < nodePep1.Children.Count; k++)
{
TransitionGroupDocNode nodeGroup1 = (TransitionGroupDocNode)nodePep1.Children[k];
TransitionGroupDocNode nodeGroup2 = (TransitionGroupDocNode)nodePep2.Children[k];
Assert.AreNotSame(nodeGroup1, nodeGroup2);
Assert.AreEqual(nodeGroup1.Results.Count, nodeGroup2.Results.Count);
for (int l = 0; l < nodeGroup1.Results.Count; l++)
{
Assert.AreEqual(nodeGroup1.Results[l][0].PeakCountRatio,
nodeGroup2.Results[l][0].PeakCountRatio);
}
for (int l = 0; l < nodeGroup1.Children.Count; l++)
{
TransitionDocNode nodeTran1 = (TransitionDocNode)nodeGroup1.Children[l];
TransitionDocNode nodeTran2 = (TransitionDocNode)nodeGroup2.Children[l];
Assert.AreNotSame(nodeTran1, nodeTran2);
Assert.AreEqual(nodeTran1.Results.Count, nodeTran2.Results.Count);
for (int m = 0; m < nodeTran1.Results.Count; m++)
{
if (nodeTran1.Results[m] != null && nodeTran2.Results[m] != null)
{
Assert.AreEqual(nodeTran1.Results[m][0].IsEmpty, nodeTran2.Results[m][0].IsEmpty);
}
else
{
Assert.AreEqual(nodeTran1.Results[m], nodeTran2.Results[m]); // both null
}
}
}
}
}
}
// Reload data from .skyd files
Assert.IsTrue(docContainer.SetDocument(docPersisted, doc, true));
docContainer.AssertComplete();
doc = docContainer.Document;
var results = doc.Settings.MeasuredResults;
const float tolerance = (float)TransitionInstrument.DEFAULT_MZ_MATCH_TOLERANCE;
foreach (var pair in doc.PeptidePrecursorPairs)
{
foreach (var chromSet in results.Chromatograms)
{
ChromatogramGroupInfo[] chromGroupInfo;
Assert.IsTrue(results.TryLoadChromatogram(chromSet, pair.NodePep, pair.NodeGroup,
tolerance, true, out chromGroupInfo));
}
}
// The single final cache path should be open now
var listCachePaths = new List <string>(doc.Settings.MeasuredResults.CachePaths);
// Should only have one cache file at this point
Assert.AreEqual(1, listCachePaths.Count);
foreach (var cachePath in listCachePaths)
{
// Attempting to delete should throw
string path = cachePath;
AssertEx.ThrowsException <IOException>(() => FileEx.SafeDelete(path));
}
// Release the .skyd file
docContainer.Release();
foreach (var cachePath in listCachePaths)
{
// Cache files should be closed now, and delete successfully.
FileEx.SafeDelete(cachePath);
}
testFilesDir.Dispose();
}
public void WatersCacheTest()
{
// First test transition from per-replicate caching strategy to
// single cache per document strategy.
var testFilesDir = new TestFilesDir(TestContext, ZIP_FILE);
// Open the replicate document, and let it reload the data from mzML
// showing the document can find data files by name in its own directory,
// since the document paths will not match those on disk.
string docPath;
var doc = InitWatersDocument(testFilesDir, out docPath);
var docReload = InitWatersDocument(testFilesDir, "160109_Mix1_calcurve_rep.sky", out docPath);
var docContainer = new ResultsTestDocumentContainer(doc, docPath);
var streamManager = docContainer.ChromatogramManager.StreamManager;
Assert.IsTrue(docContainer.SetDocument(docReload, doc, true));
docContainer.AssertComplete();
docReload = docContainer.Document;
// Release file handles to cache files created during load
Assert.IsTrue(docContainer.SetDocument(doc, docReload));
// Delete the cache
string cachePath = Path.ChangeExtension(docPath, ".skyd");
FileEx.SafeDelete(cachePath);
// Then try using cached replicate files
// Move per-replicate cache files into place
var replicateCacheNames = new[]
{
"160109_Mix1_calcurve_rep_calcurve_070.skyd",
"160109_Mix1_calcurve_rep_calcurve_073.skyd"
};
GetCacheFiles(testFilesDir, replicateCacheNames);
// Delete the files these cache
DeleteFiles(testFilesDir,
new[]
{
"160109_Mix1_calcurve_070.mzML",
"160109_Mix1_calcurve_073.mzML",
});
var docCached = InitWatersDocument(testFilesDir, "160109_Mix1_calcurve_rep.sky", out docPath);
Assert.IsTrue(docContainer.SetDocument(docCached, doc, true));
docContainer.AssertComplete();
docCached = docContainer.Document;
// The document with data from the .mzML files should be the same as
// the one loaded from the .skyd files.
// Unfortunately, this is to hard to maintain when cache changes are made.
// AssertEx.Cloned(docCached, docReload);
// The one cache should be present
Assert.IsTrue(File.Exists(cachePath));
// And the replicate cache files should have been removed
foreach (var cacheName in replicateCacheNames)
{
Assert.IsFalse(File.Exists(testFilesDir.GetTestPath(cacheName)));
}
// Save the cache file time stamp
var cacheInfo = new FileInfo(cachePath);
long cacheSize = cacheInfo.Length;
// Adding files already in the document should have no impact on the cache.
string extRaw = ExtensionTestContext.ExtWatersRaw;
var listChromatograms = new List <ChromatogramSet>(docCached.Settings.MeasuredResults.Chromatograms)
{
new ChromatogramSet("extra1",
new[] { MsDataFileUri.Parse(testFilesDir.GetTestPath("160109_Mix1_calcurve_075" + extRaw)) }),
new ChromatogramSet("extra2",
new[] { MsDataFileUri.Parse(testFilesDir.GetTestPath("160109_Mix1_calcurve_078.mzML")) })
};
// Adding a new file should cause the cache to grow.
var settings = docCached.Settings.MeasuredResults.ChangeChromatograms(listChromatograms);
var docGrow = docCached.ChangeMeasuredResults(settings);
Assert.IsTrue(docContainer.SetDocument(docGrow, docCached, true));
docContainer.AssertComplete();
docGrow = docContainer.Document;
cacheInfo = new FileInfo(cachePath);
Assert.IsTrue(cacheSize < cacheInfo.Length);
cacheSize = cacheInfo.Length;
var writeTime = cacheInfo.LastWriteTime;
listChromatograms.Add(
new ChromatogramSet("double",
new[]
{
testFilesDir.GetTestPath("160109_Mix1_calcurve_075" + extRaw),
testFilesDir.GetTestPath("160109_Mix1_calcurve_078.mzML")
}));
settings = docGrow.Settings.MeasuredResults.ChangeChromatograms(listChromatograms);
var docNoCacheChange1 = docGrow.ChangeMeasuredResults(settings);
Assert.IsTrue(docContainer.SetDocument(docNoCacheChange1, docGrow, true));
docContainer.AssertComplete();
docNoCacheChange1 = docContainer.Document;
Assert.AreEqual(writeTime, File.GetLastWriteTime(cachePath));
// Removing files should have no impact, until optimized
listChromatograms.RemoveRange(listChromatograms.Count - 2, 2);
listChromatograms.RemoveAt(1);
settings = docNoCacheChange1.Settings.MeasuredResults.ChangeChromatograms(listChromatograms);
var docNoCacheChange2 = docNoCacheChange1.ChangeMeasuredResults(settings);
Assert.IsTrue(docContainer.SetDocument(docNoCacheChange2, docNoCacheChange1, true));
docContainer.AssertComplete();
docNoCacheChange2 = docContainer.Document;
Assert.AreEqual(writeTime, File.GetLastWriteTime(cachePath));
// Optimizing should shrink the cache
var results = docNoCacheChange2.Settings.MeasuredResults.OptimizeCache(docPath, streamManager);
var docOptimized = new SrmDocument(docNoCacheChange2,
docNoCacheChange2.Settings.ChangeMeasuredResults(results),
docNoCacheChange2.Children);
// This should not cause a reload
Assert.IsTrue(docContainer.SetDocument(docOptimized, docNoCacheChange2, false));
cacheInfo = new FileInfo(cachePath);
Assert.IsTrue(cacheSize > cacheInfo.Length);
// Test file caches
// First reload the files from .mzML
docReload = InitWatersDocument(testFilesDir, "160109_Mix1_calcurve_file.sky", out docPath);
// Change the path to use the right .skyd file
docContainer.DocumentFilePath = docPath;
Assert.IsTrue(docContainer.SetDocument(docReload, docOptimized, true));
docContainer.AssertComplete();
docReload = docContainer.Document;
// Release file handles to cache files created during load
Assert.IsTrue(docContainer.SetDocument(doc, docReload));
// Delete the cache
cachePath = Path.ChangeExtension(docPath, ".skyd");
FileEx.SafeDelete(cachePath);
// Then try using cached files
// Move per-file cache files into place
var fileCacheNames = new[]
{
"160109_Mix1_calcurve_075.mzML.skyd",
"160109_Mix1_calcurve_078.mzML.skyd"
};
GetCacheFiles(testFilesDir, fileCacheNames);
// Swap the mzML files, so the test will fail, if not reading from the cache
// CONSIDER: Should this really work, since they have different time stamps?
string file075 = testFilesDir.GetTestPath("160109_Mix1_calcurve_075.mzML");
string file078 = testFilesDir.GetTestPath("160109_Mix1_calcurve_078.mzML");
string fileTemp = file075 + ".tmp";
File.Move(file075, fileTemp);
File.Move(file078, file075);
File.Move(fileTemp, file078);
docCached = InitWatersDocument(testFilesDir, "160109_Mix1_calcurve_file.sky", out docPath);
// Make sure cache files exactly match the names the loader will look for
var listResultsFiles = new List <MsDataFileUri>();
foreach (var chromatogram in docCached.Settings.MeasuredResults.Chromatograms)
{
listResultsFiles.AddRange(chromatogram.MSDataFilePaths);
}
for (int i = 0; i < fileCacheNames.Length; i++)
{
string partPath = ChromatogramCache.PartPathForName(docPath, listResultsFiles[i]);
File.Move(testFilesDir.GetTestPath(fileCacheNames[i]), partPath);
}
Assert.IsTrue(docContainer.SetDocument(docCached, doc, true));
docContainer.AssertComplete();
// docCached = docContainer.Document;
// The document with data from the .mzML files should be the same as
// the one loaded from the .skyd files.
// Unfortunately, this is to hard to maintain when cache changes are made.
// AssertEx.Cloned(docCached, docReload);
// The one cache should be present
Assert.IsTrue(File.Exists(Path.ChangeExtension(docPath, ".skyd")));
// And the replicate cache files should have been removed
foreach (var cacheName in fileCacheNames)
{
Assert.IsFalse(File.Exists(testFilesDir.GetTestPath(cacheName)));
}
// Release file handles
docContainer.Release();
testFilesDir.Dispose();
}
public void WatersMultiFileTest()
{
var testFilesDir = new TestFilesDir(TestContext, ZIP_FILE);
string docPath;
SrmDocument docOriginal = InitWatersDocument(testFilesDir, out docPath);
SrmDocument doc = docOriginal;
var docContainer = new ResultsTestDocumentContainer(doc, docPath);
string extRaw = ExtensionTestContext.ExtWatersRaw;
var listChromatograms = new List <ChromatogramSet>
{
new ChromatogramSet("double", new[]
{
MsDataFileUri.Parse(testFilesDir.GetTestPath("160109_Mix1_calcurve_070.mzML")),
MsDataFileUri.Parse(testFilesDir.GetTestPath("160109_Mix1_calcurve_073.mzML"))
}),
new ChromatogramSet("trouble", new[]
{
MsDataFileUri.Parse(testFilesDir.GetTestPath("160109_Mix1_calcurve_075" + extRaw)),
MsDataFileUri.Parse(testFilesDir.GetTestPath("160109_Mix1_calcurve_078.mzML"))
})
};
var docResults = doc.ChangeMeasuredResults(new MeasuredResults(listChromatograms));
Assert.IsTrue(docContainer.SetDocument(docResults, doc, true));
docContainer.AssertComplete();
docResults = docContainer.Document;
Assert.IsTrue(docResults.Settings.HasResults);
var measuredResults = docResults.Settings.MeasuredResults;
var chromatograms = measuredResults.Chromatograms;
Assert.AreEqual(2, chromatograms.Count);
const float tolerance = (float)TransitionInstrument.DEFAULT_MZ_MATCH_TOLERANCE;
foreach (var pair in docResults.PeptidePrecursorPairs)
{
var nodePep = pair.NodePep;
var nodeGroup = pair.NodeGroup;
Assert.IsTrue(nodeGroup.HasResults);
Assert.AreEqual(2, nodeGroup.Results.Count);
foreach (var result in nodeGroup.Results)
{
Assert.AreEqual(2, result.Count);
}
for (int i = 0; i < 2; i++)
{
ChromatogramGroupInfo[] chromInfos;
Assert.IsTrue(measuredResults.TryLoadChromatogram(i, nodePep, nodeGroup, tolerance, true, out chromInfos));
Assert.AreEqual(2, chromInfos.Length);
double[] peakAreas = new double[2];
for (int j = 0; j < 2; j++)
{
var chromInfo = chromInfos[j];
Assert.IsTrue(chromInfo.BestPeakIndex != -1);
foreach (var tranInfo in chromInfo.TransitionPointSets)
{
var peakInfo = tranInfo.GetPeak(chromInfo.BestPeakIndex);
if (peakInfo.IsEmpty || peakInfo.IsForcedIntegration)
{
continue;
}
// Check times
var times = tranInfo.Times;
int iStart = Array.BinarySearch(times, peakInfo.StartTime);
Assert.IsTrue(iStart >= 0);
int iEnd = Array.BinarySearch(times, peakInfo.EndTime);
Assert.IsTrue(iEnd >= 0);
int iPeak = Array.BinarySearch(times, iStart, iEnd - iStart, peakInfo.RetentionTime);
// Check intensities at times
var intensities = tranInfo.Intensities;
Assert.IsTrue(intensities[iStart] < intensities[iPeak]);
Assert.IsTrue(intensities[iEnd] < intensities[iPeak]);
// Sum peak area
peakAreas[j] += peakInfo.Area;
}
}
Assert.IsTrue(peakAreas[0] < peakAreas[1]);
}
}
// Release file handles
docContainer.Release();
testFilesDir.Dispose();
}
public void WatersMultiFileTest()
{
var testFilesDir = new TestFilesDir(TestContext, ZIP_FILE);
string docPath;
SrmDocument docOriginal = InitWatersDocument(testFilesDir, out docPath);
SrmDocument doc = docOriginal;
using (var docContainer = new ResultsTestDocumentContainer(doc, docPath))
{
string extRaw = ExtensionTestContext.ExtWatersRaw;
var listChromatograms = new List <ChromatogramSet>
{
new ChromatogramSet("double", new[]
{
MsDataFileUri.Parse(testFilesDir.GetTestPath("160109_Mix1_calcurve_070.mzML")),
MsDataFileUri.Parse(testFilesDir.GetTestPath("160109_Mix1_calcurve_073.mzML"))
}),
new ChromatogramSet("trouble", new[]
{
MsDataFileUri.Parse(testFilesDir.GetTestPath("160109_Mix1_calcurve_075" + extRaw)),
MsDataFileUri.Parse(testFilesDir.GetTestPath("160109_Mix1_calcurve_078.mzML"))
})
};
var docResults = doc.ChangeMeasuredResults(new MeasuredResults(listChromatograms));
Assert.IsTrue(docContainer.SetDocument(docResults, doc, true));
docContainer.AssertComplete();
docResults = docContainer.Document;
Assert.IsTrue(docResults.Settings.HasResults);
var measuredResults = docResults.Settings.MeasuredResults;
var chromatograms = measuredResults.Chromatograms;
Assert.AreEqual(2, chromatograms.Count);
var dictPathToIndex = new Dictionary <MsDataFileUri, int>();
for (int i = 0; i < 2; i++)
{
Assert.AreEqual(listChromatograms[i].BatchName, chromatograms[i].BatchName);
for (int j = 0; j < 2; j++)
{
Assert.AreEqual(listChromatograms[i].MSDataFileInfos[j].FilePath, chromatograms[i].MSDataFileInfos[j].FilePath);
dictPathToIndex.Add(listChromatograms[i].MSDataFileInfos[j].FilePath, j);
}
}
const float tolerance = (float)TransitionInstrument.DEFAULT_MZ_MATCH_TOLERANCE;
foreach (var pair in docResults.PeptidePrecursorPairs)
{
var nodePep = pair.NodePep;
var nodeGroup = pair.NodeGroup;
Assert.IsTrue(nodeGroup.HasResults, string.Format("Missing results on {0}", nodeGroup));
Assert.AreEqual(2, nodeGroup.Results.Count);
foreach (var result in nodeGroup.Results)
{
Assert.AreEqual(2, result.Count);
}
for (int i = 0; i < 2; i++)
{
ChromatogramGroupInfo[] chromInfos;
Assert.IsTrue(measuredResults.TryLoadChromatogram(i, nodePep, nodeGroup, tolerance, true, out chromInfos),
string.Format("Missing chromatogram {0} - {1}", nodeGroup, i));
Assert.AreEqual(2, chromInfos.Length);
double[] peakAreas = new double[2];
for (int j = 0; j < 2; j++)
{
var chromInfo = chromInfos[j];
// No guarantee that chromInfos will be in the same order as the ChromatogramSet.MsDataFileInfos
int peakAreaIndex = dictPathToIndex[chromInfo.FilePath];
Assert.IsTrue(chromInfo.BestPeakIndex != -1, string.Format("Missing peak {0} - {1}:{2}", nodeGroup, i, j));
foreach (var tranInfo in chromInfo.TransitionPointSets)
{
var peakInfo = tranInfo.GetPeak(chromInfo.BestPeakIndex);
if (peakInfo.IsEmpty || peakInfo.IsForcedIntegration)
{
continue;
}
// Check times
var times = tranInfo.Times;
int iStart = CollectionUtil.BinarySearch(times, peakInfo.StartTime);
Assert.IsTrue(iStart >= 0, string.Format("Start time not {0}", iStart));
int iEnd = CollectionUtil.BinarySearch(times, peakInfo.EndTime);
Assert.IsTrue(iEnd >= 0, string.Format("End time not found {0}", iEnd));
int iPeak = CollectionUtil.BinarySearch(times, peakInfo.RetentionTime);
// Check intensities at times
var intensities = tranInfo.Intensities;
Assert.IsTrue(intensities[iStart] < intensities[iPeak],
string.Format("Start intensity {0} >= peak intensity {1}", intensities[iStart], intensities[iPeak]));
Assert.IsTrue(intensities[iEnd] < intensities[iPeak],
string.Format("End intensity {0} >= peak intensity {1}", intensities[iEnd], intensities[iPeak]));
// Sum peak area
peakAreas[peakAreaIndex] += peakInfo.Area;
}
}
Assert.IsTrue(peakAreas[0] < peakAreas[1],
string.Format("{0} analyte area >= {1} standard area", peakAreas[0], peakAreas[1]));
}
}
}
testFilesDir.Dispose();
}
public void TestImportPeakBoundary()
{
// Load the SRM document and relevant files
var testFilesDir = new TestFilesDir(TestContext, TEST_ZIP_PATH);
bool isIntl = (TextUtil.CsvSeparator != TextUtil.SEPARATOR_CSV);
var precursorMzs = isIntl ? _precursorMzsIntl : _precursorMzsUs;
var peakBoundaryFileTsv = testFilesDir.GetTestPath(isIntl
? "PeakBoundaryTsvIntl.tsv"
: "PeakBoundaryTsv.tsv");
var peakBoundaryFileCsv = testFilesDir.GetTestPath(isIntl
? "PeakBoundaryIntl.csv"
: "PeakBoundaryUS.csv");
var peakBoundaryDoc = testFilesDir.GetTestPath("Chrom05.sky");
SrmDocument doc = ResultsUtil.DeserializeDocument(peakBoundaryDoc);
var cult = LocalizationHelper.CurrentCulture;
var cultI = CultureInfo.InvariantCulture;
string csvSep = TextUtil.CsvSeparator.ToString(cultI);
string spaceSep = TextUtil.SEPARATOR_SPACE.ToString(cultI);
// Load an empty doc, so that we can make a change and
// cause the .skyd to be loaded
using (var docContainer = new ResultsTestDocumentContainer(null, peakBoundaryDoc))
{
docContainer.SetDocument(doc, null, true);
docContainer.AssertComplete();
SrmDocument docResults = docContainer.Document;
// Test Tsv import, looking at first .raw file
DoFileImportTests(docResults, peakBoundaryFileTsv, _precursorCharge,
_tsvMinTime1, _tsvMaxTime1, _tsvIdentified1, _tsvAreas1, _peptides, 0, precursorMzs, annote);
// Test Tsv import, looking at second .raw file
DoFileImportTests(docResults, peakBoundaryFileTsv, _precursorCharge,
_tsvMinTime2, _tsvMaxTime2, _tsvIdentified2, _tsvAreas2, _peptides, 1, precursorMzs, annote);
// Test Csv import for local format
DoFileImportTests(docResults, peakBoundaryFileCsv, _precursorCharge,
_csvMinTime1, _csvMaxTime1, _csvIdentified1, _csvAreas1, _peptides, 0, precursorMzs, annote);
DoFileImportTests(docResults, peakBoundaryFileCsv, _precursorCharge,
_csvMinTime2, _csvMaxTime2, _csvIdentified2, _csvAreas2, _peptides, 1, precursorMzs, annote);
// Test that importing same file twice leads to no change to document the second time
var docNew = ImportFileToDoc(docResults, peakBoundaryFileTsv);
var docNewSame = ImportFileToDoc(docNew, peakBoundaryFileTsv);
Assert.AreSame(docNew, docNewSame);
Assert.AreNotSame(docNew, docResults);
// Test that exporting peak boundaries and then importing them leads to no change
string peakBoundaryExport = testFilesDir.GetTestPath("TestRoundTrip.csv");
ReportSpec reportSpec = MakeReportSpec();
ReportToCsv(reportSpec, docNew, peakBoundaryExport);
var docRoundTrip = ImportFileToDoc(docNew, peakBoundaryExport);
Assert.AreSame(docNew, docRoundTrip);
// 1. Empty file -
ImportThrowsException(docResults, string.Empty,
Resources.PeakBoundaryImporter_Import_Failed_to_read_the_first_line_of_the_file);
// 2. No separator in first line
ImportThrowsException(docResults, "No-valid-separators",
TextUtil.CsvSeparator == TextUtil.SEPARATOR_CSV
? Resources.PeakBoundaryImporter_DetermineCorrectSeparator_The_first_line_does_not_contain_any_of_the_possible_separators_comma__tab_or_space_
: Resources.PeakBoundaryImporter_DetermineCorrectSeparator_The_first_line_does_not_contain_any_of_the_possible_separators_semicolon__tab_or_space_);
// 3. Missing field names
ImportThrowsException(docResults, string.Join(csvSep, PeakBoundaryImporter.STANDARD_FIELD_NAMES.Take(3).ToArray()),
Resources.PeakBoundaryImporter_Import_Failed_to_find_the_necessary_headers__0__in_the_first_line);
string[] values =
{
"TPEVDDEALEK", "Q_2012_0918_RJ_13.raw", (4.0).ToString(cult), (3.5).ToString(cult), (4.5).ToString(cult), 2.ToString(cult), 0.ToString(cult)
};
string headerRow = string.Join(csvSep, PeakBoundaryImporter.STANDARD_FIELD_NAMES.Take(values.Length));
string headerRowSpaced = string.Join(spaceSep, PeakBoundaryImporter.STANDARD_FIELD_NAMES.Take(values.Length));
// 4. Mismatched field count
ImportThrowsException(docResults, TextUtil.LineSeparate(headerRow, string.Join(spaceSep, values)),
Resources.PeakBoundaryImporter_Import_Line__0__field_count__1__differs_from_the_first_line__which_has__2_);
// 5. Invalid charge state
string[] valuesBadCharge = new List <string>(values).ToArray();
valuesBadCharge[(int)PeakBoundaryImporter.Field.charge] = (3.5).ToString(cult);
ImportThrowsException(docResults, TextUtil.LineSeparate(headerRow, string.Join(csvSep, valuesBadCharge)),
Resources.PeakBoundaryImporter_Import_The_value___0___on_line__1__is_not_a_valid_charge_state_);
valuesBadCharge[(int)PeakBoundaryImporter.Field.charge] = TextUtil.EXCEL_NA;
ImportThrowsException(docResults, TextUtil.LineSeparate(headerRow, string.Join(csvSep, valuesBadCharge)),
Resources.PeakBoundaryImporter_Import_The_value___0___on_line__1__is_not_a_valid_charge_state_);
// 6. Invalid start time
string[] valuesBadTime = new List <string>(values).ToArray();
valuesBadTime[(int)PeakBoundaryImporter.Field.start_time] = "bad";
ImportThrowsException(docResults, TextUtil.LineSeparate(headerRowSpaced, string.Join(spaceSep, valuesBadTime)),
Resources.PeakBoundaryImporter_Import_The_value___0___on_line__1__is_not_a_valid_start_time_);
valuesBadTime[(int)PeakBoundaryImporter.Field.end_time] = "bad";
ImportThrowsException(docResults, TextUtil.LineSeparate(headerRowSpaced, string.Join(spaceSep, valuesBadTime)),
Resources.PeakBoundaryImporter_Import_The_value___0___on_line__1__is_not_a_valid_start_time_);
// Still not okay when not changing peaks, because we now store start and end times as annotations
ImportThrowsException(docResults, TextUtil.LineSeparate(headerRowSpaced, string.Join(spaceSep, valuesBadTime)),
Resources.PeakBoundaryImporter_Import_The_value___0___on_line__1__is_not_a_valid_start_time_, true, false, false);
// 7. Invalid end time
valuesBadTime[(int)PeakBoundaryImporter.Field.start_time] =
values[(int)PeakBoundaryImporter.Field.start_time];
ImportThrowsException(docResults, TextUtil.LineSeparate(headerRowSpaced, string.Join(spaceSep, valuesBadTime)),
Resources.PeakBoundaryImporter_Import_The_value___0___on_line__1__is_not_a_valid_end_time_);
// Still not okay when not changing peaks, because we now store start and end times as annotations
ImportThrowsException(docResults, TextUtil.LineSeparate(headerRowSpaced, string.Join(spaceSep, valuesBadTime)),
Resources.PeakBoundaryImporter_Import_The_value___0___on_line__1__is_not_a_valid_end_time_, true, false, false);
// #N/A in times ok
valuesBadTime[(int)PeakBoundaryImporter.Field.start_time] =
valuesBadTime[(int)PeakBoundaryImporter.Field.end_time] = TextUtil.EXCEL_NA;
ImportNoException(docResults, TextUtil.LineSeparate(headerRowSpaced, string.Join(spaceSep, valuesBadTime)));
// If only start time #N/A throws exception
valuesBadTime[(int)PeakBoundaryImporter.Field.start_time] = (3.5).ToString(cult);
ImportThrowsException(docResults, TextUtil.LineSeparate(headerRowSpaced, string.Join(spaceSep, valuesBadTime)),
Resources.PeakBoundaryImporter_Import_Missing_end_time_on_line__0_);
// If only end time #N/A throws exception
valuesBadTime[(int)PeakBoundaryImporter.Field.start_time] = TextUtil.EXCEL_NA;
valuesBadTime[(int)PeakBoundaryImporter.Field.end_time] = (3.5).ToString(cult);
ImportThrowsException(docResults, TextUtil.LineSeparate(headerRowSpaced, string.Join(spaceSep, valuesBadTime)),
Resources.PeakBoundaryImporter_Import_Missing_start_time_on_line__0_);
// Empty times throws exception
valuesBadTime[(int)PeakBoundaryImporter.Field.start_time] =
valuesBadTime[(int)PeakBoundaryImporter.Field.end_time] = string.Empty;
ImportThrowsException(docResults, TextUtil.LineSeparate(headerRowSpaced, string.Join(spaceSep, valuesBadTime)),
Resources.PeakBoundaryImporter_Import_The_value___0___on_line__1__is_not_a_valid_start_time_);
// 8. Not imported file gets skipped
string[] valuesBadFile = new List <string>(values).ToArray();
valuesBadFile[(int)PeakBoundaryImporter.Field.filename] = "Q_2012_0918_RJ_15.raw";
ImportNoException(docResults, TextUtil.LineSeparate(headerRowSpaced, string.Join(spaceSep, valuesBadFile)));
// 9. Unknown modification state gets skipped
string[] valuesBadSequence = new List <string>(values).ToArray();
valuesBadSequence[(int)PeakBoundaryImporter.Field.modified_peptide] = "T[+80]PEVDDEALEK";
ImportNoException(docResults, TextUtil.LineSeparate(headerRow, string.Join(csvSep, valuesBadSequence)));
// 10. Unknown peptide sequence gets skipped
valuesBadSequence[(int)PeakBoundaryImporter.Field.modified_peptide] = "PEPTIDER";
ImportNoException(docResults, TextUtil.LineSeparate(headerRow, string.Join(csvSep, valuesBadSequence)));
// 11. Bad value in decoy field
string[] valuesBadDecoys = new List <string>(values).ToArray();
valuesBadDecoys[(int)PeakBoundaryImporter.Field.is_decoy] = 3.ToString(cult);
ImportThrowsException(docResults, TextUtil.LineSeparate(headerRow, string.Join(csvSep, valuesBadDecoys)),
Resources.PeakBoundaryImporter_Import_The_decoy_value__0__on_line__1__is_invalid__must_be_0_or_1_);
// 12. Import with bad sample throws exception
string[] valuesSample =
{
"TPEVDDEALEK", "Q_2012_0918_RJ_13.raw", (4.0).ToString(cult), (3.5).ToString(cult), (4.5).ToString(cult), 2.ToString(cult), 0.ToString(cult), "badSample"
};
string headerRowSample = string.Join(csvSep, PeakBoundaryImporter.STANDARD_FIELD_NAMES.Take(valuesSample.Length));
ImportThrowsException(docResults, TextUtil.LineSeparate(headerRowSample, string.Join(csvSep, valuesSample)),
Resources.PeakBoundaryImporter_Import_Sample__0__on_line__1__does_not_match_the_file__2__);
// 13. Decoys, charge state, and sample missing ok
var valuesFourFields = valuesSample.Take(5).ToArray();
string headerFourFields = string.Join(csvSep, PeakBoundaryImporter.STANDARD_FIELD_NAMES.Take(valuesFourFields.Length));
ImportNoException(docResults, TextUtil.LineSeparate(headerFourFields, string.Join(csvSep, valuesFourFields)));
// 14. Valid (charge state, fileName, peptide) combo that is not in document gets skipped
string[] valuesBadCombo = new List <string>(values).ToArray();
valuesBadCombo[(int)PeakBoundaryImporter.Field.charge] = (5).ToString(cult);
ImportNoException(docResults, TextUtil.LineSeparate(headerRow, string.Join(csvSep, valuesBadCombo)));
// Note: Importing with all 7 columns is tested as part of MProphetResultsHandlerTest
}
// Now check a file that has peptide ID's, and see that they're properly ported
var peptideIdPath = testFilesDir.GetTestPath("Template_MS1Filtering_1118_2011_3-2min.sky");
SrmDocument docId = ResultsUtil.DeserializeDocument(peptideIdPath);
docId = docId.ChangeSettings(docId.Settings.ChangePeptideLibraries(libraries =>
{
var lib = libraries.Libraries[0];
return(libraries.ChangeLibrarySpecs(new LibrarySpec[]
{
new BiblioSpecLiteSpec(lib.Name, testFilesDir.GetTestPath(lib.FileNameHint))
}));
}));
using (var docContainerId = new ResultsTestDocumentContainer(null, peptideIdPath))
{
docContainerId.SetDocument(docId, null, true);
docContainerId.AssertComplete();
SrmDocument docResultsId = docContainerId.Document;
var peakBoundaryFileId = testFilesDir.GetTestPath(isIntl
? "Template_MS1Filtering_1118_2011_3-2min_new_intl.tsv"
: "Template_MS1Filtering_1118_2011_3-2min_new.tsv");
DoFileImportTests(docResultsId, peakBoundaryFileId, _precursorChargeId,
_idMinTime1, _idMaxTime1, _idIdentified1, _idAreas1, _peptidesId, 0);
// 15. Decminal import format ok
var valuesUnimod = new[]
{
"LGGLRPES[+" + string.Format("{0:F01}", 80.0) + "]PESLTSVSR", "100803_0005b_MCF7_TiTip3.wiff", (80.5).ToString(cult), (82.0).ToString(cult)
};
var headerUnimod = string.Join(csvSep, PeakBoundaryImporter.STANDARD_FIELD_NAMES.Where((s, i) =>
i != (int)PeakBoundaryImporter.Field.apex_time).Take(valuesUnimod.Length));
ImportNoException(docResultsId, TextUtil.LineSeparate(headerUnimod, string.Join(csvSep, valuesUnimod)));
// 16. Integer import format ok
valuesUnimod[0] = "LGGLRPES[+80]PESLTSVSR";
ImportNoException(docResultsId, TextUtil.LineSeparate(headerUnimod, string.Join(csvSep, valuesUnimod)));
// 17. Unimod import format ok
valuesUnimod[0] = "LGGLRPES(UniMod:21)PESLTSVSR";
ImportNoException(docResultsId, TextUtil.LineSeparate(headerUnimod, string.Join(csvSep, valuesUnimod)));
// 18. Strange capitalizations OK
valuesUnimod[0] = "LGGLRPES(uniMoD:21)PESLTSVSR";
ImportNoException(docResultsId, TextUtil.LineSeparate(headerUnimod, string.Join(csvSep, valuesUnimod)));
// 18. Unimod with brackets OK
valuesUnimod[0] = "LGGLRPES[uniMoD:21]PESLTSVSR";
ImportNoException(docResultsId, TextUtil.LineSeparate(headerUnimod, string.Join(csvSep, valuesUnimod)));
}
}