internal static void Predict <TData, TListener>(string args, string inputFile, string referenceFile = null) where TData : BaseData where TListener : VowpalWabbitListenerToEvents <TData>, new() { float[] references = null; var index = 0; if (referenceFile != null) { references = File.ReadAllLines(referenceFile) .Select(l => float.Parse(l.Split(' ')[0], CultureInfo.InvariantCulture)) .ToArray(); } using (var vwRef = new VowpalWabbit(args)) using (var vwModel = new VowpalWabbitModel(args)) using (var vwInMemoryShared2 = new VowpalWabbit <TData>(new VowpalWabbitSettings(model: vwModel))) { var listener = new TListener(); listener.Created = (x, label) => { var expected = vwRef.Predict(x.Line, VowpalWabbitPredictionType.Scalar); var actual = vwInMemoryShared2.Predict(x, VowpalWabbitPredictionType.Scalar, label); Assert.AreEqual(expected, actual, 1e-5); if (references != null) { Assert.AreEqual(references[index++], actual, 1e-5); } }; } }
public VowpalWabbit(VowpalWabbitModel model, VowpalWabbitSerializerSettings settings = null) : base(model) { var visitor = new VowpalWabbitInterfaceVisitor(this); this.serializer = VowpalWabbitSerializerFactory.CreateSerializer <TExample>(visitor, settings); }
/// <summary> /// Update VW model from stream. /// </summary> /// <param name="modelStream">The model stream to load from.</param> public void Update(Stream modelStream) { if (modelStream == null) { return; } var model = new VowpalWabbitModel( new VowpalWabbitSettings { ModelStream = modelStream, MaxExampleCacheSize = 1024, TypeInspector = this.typeInspector, EnableStringExampleGeneration = this.developmentMode, EnableStringFloatCompact = this.developmentMode }); if (this.vwPool == null) { this.vwPool = new TPool(); this.vwPool.UpdateModel(model); } else { this.vwPool.UpdateModel(model); } }
private void InternalTestModelRandomCorrupt(string modelFile) { const int numBytesToCorrupt = 10; var rand = new Random(0); byte[] modelBytes = File.ReadAllBytes(modelFile); for (int i = 0; i < 100; i++) { var corruptBytes = new byte[modelBytes.Length]; Array.Copy(modelBytes, corruptBytes, corruptBytes.Length); for (int j = 0; j < numBytesToCorrupt; j++) { corruptBytes[rand.Next(corruptBytes.Length)] = (byte)rand.Next(byte.MaxValue); } try { using (var modelStream = new MemoryStream(corruptBytes)) using (var vw = new VowpalWabbitModel(new VowpalWabbitSettings("--quiet -t", modelStream))) { // chances of reaching this point after reading a corrupt model are low Assert.IsTrue(false); } } catch (Exception) { Assert.IsTrue(true); } } }
public static void TestMemoryLeak() { string outModelFile = "cb_adf_mem_leak.model"; using (var vw = new VowpalWabbit <DataString, DataStringADF>("--cb_adf --rank_all")) { DataString[] sampleData = CreateStringCbAdfData(1000); foreach (DataString example in sampleData) { vw.Learn(example, example.ActionDependentFeatures, example.SelectedActionIndex, example.Label); } vw.Native.SaveModel(outModelFile); } var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings(string.Format("--quiet -t -i {0}", outModelFile)) { MaxExampleCacheSize = 1024 }); var pool = new VowpalWabbitThreadedPrediction <DataString, DataStringADF>(vwModel); while (true) { vwModel = new VowpalWabbitModel(new VowpalWabbitSettings(string.Format("--quiet -t -i {0}", outModelFile)) { MaxExampleCacheSize = 1024 }); pool.UpdateModel(vwModel); } }
public void TestID() { using (var vw = new VowpalWabbit("--id abc")) { Assert.AreEqual("abc", vw.ID); vw.SaveModel("model"); vw.ID = "def"; vw.SaveModel("model.1"); } using (var vw = new VowpalWabbit("-i model")) { Assert.AreEqual("abc", vw.ID); } using (var vw = new VowpalWabbit("-i model.1")) { Assert.AreEqual("def", vw.ID); } using (var vwm = new VowpalWabbitModel("-i model.1")) { Assert.AreEqual("def", vwm.ID); using (var vw = new VowpalWabbit(new VowpalWabbitSettings(model: vwm))) { Assert.AreEqual("def", vw.ID); Assert.AreEqual(vwm.ID, vw.ID); } } }
/// <summary> /// Initializes a new instance of the <see cref="VowpalWabbitThreadedPredictionBase{TVowpalWabbit}"/> class. /// </summary> /// <param name="model">The initial model to use.</param> protected VowpalWabbitThreadedPredictionBase(VowpalWabbitModel model) { this.vwPool = new ObjectPool <VowpalWabbitModel, TVowpalWabbit>( ObjectFactory.Create( model, m => this.InternalCreate(new VowpalWabbit(m.Settings.ShallowCopy(model: m))))); }
private TVowpalWabbit CreateVowpalWabbitChild(VowpalWabbitModel model) { var newSettings = (VowpalWabbitSettings)this.settings.Clone(); newSettings.Model = model; var vw = new VowpalWabbit(newSettings); return(this.InternalCreate(vw)); }
public void TestExampleCache() { var random = new Random(123); var examples = new List <CachedData>(); for (int i = 0; i < 1000; i++) { examples.Add(new CachedData { Label = new SimpleLabel { Label = 1 }, Feature = random.NextDouble() }); } for (int i = 0; i < 1000; i++) { var cachedData = new CachedData { Label = new SimpleLabel { Label = 2 }, Feature = 10 + random.NextDouble() }; examples.Add(cachedData); examples.Add(cachedData); } using (var vw = new VowpalWabbit <CachedData>(new VowpalWabbitSettings("-k -c --passes 10", enableExampleCaching: false))) { foreach (var example in examples) { vw.Learn(example, example.Label); } vw.Native.RunMultiPass(); vw.Native.SaveModel("models/model1"); } using (var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings("-t", modelStream: File.OpenRead("models/model1")))) using (var vwCached = new VowpalWabbit <CachedData>(new VowpalWabbitSettings(model: vwModel, enableExampleCaching: true, maxExampleCacheSize: 5))) using (var vw = new VowpalWabbit <CachedData>(new VowpalWabbitSettings(model: vwModel, enableExampleCaching: false))) { foreach (var example in examples) { var cachedPrediction = vwCached.Predict(example, VowpalWabbitPredictionType.Scalar); var prediction = vw.Predict(example, VowpalWabbitPredictionType.Scalar); Assert.AreEqual(prediction, cachedPrediction); Assert.AreEqual(example.Label.Label, Math.Round(prediction)); } } }
/// <summary> /// Updates the model used for prediction in a thread-safe manner. /// </summary> /// <param name="model">The new model to be used.</param> public void UpdateModel(VowpalWabbitModel model) { this.vwPool.UpdateFactory(ObjectFactory.Create( model, m => { var settings = (VowpalWabbitSettings)m.Settings.Clone(); settings.Model = m; return(this.InternalCreate(new VowpalWabbit(settings))); })); }
public void VwModelRefCounting() { var model = new VowpalWabbitModel(""); //var i1 = new VowpalWabbit(model); //var i2 = new VowpalWabbit(model); //i1.Dispose(); model.Dispose(); //i1.Dispose(); }
private void Dispose(bool disposing) { if (disposing) { if (model != null) { this.model.Dispose(); this.model = null; } } }
/// <summary> /// Updates the model used for prediction in a thread-safe manner. /// </summary> /// <param name="model">The new model to be used.</param> public void UpdateModel(VowpalWabbitModel model) { this.vwPool.UpdateFactory(ObjectFactory.Create( model, m => { var settings = (VowpalWabbitSettings)m.Settings.Clone(); settings.Model = m; // avoid duplicate arguments (e.g. -i) and force testing mode settings.Arguments = m.Arguments.CommandLine.Contains("-t") ? string.Empty : "-t"; return(this.InternalCreate(new VowpalWabbit(settings))); })); }
private void InternalTestModel(string modelFile, bool shouldPass) { try { using (var vw = new VowpalWabbitModel(string.Format("--quiet -t -i {0}", modelFile))) { // should only reach this point if model is valid Assert.IsTrue(shouldPass); } } catch (VowpalWabbitException ex) { Assert.IsTrue(ex.Message.Contains("corrupted")); } }
public void TestQuietAndTestArguments() { using (var vw = new VowpalWabbit("--quiet -t")) { vw.SaveModel("args.model"); } using (var vw = new VowpalWabbitModel(new VowpalWabbitSettings { ModelStream = File.Open("args.model", FileMode.Open) })) { Assert.IsFalse(vw.Arguments.CommandLine.Contains("--quiet")); Assert.IsFalse(vw.Arguments.CommandLine.Contains("-t")); using (var vwSub = new VowpalWabbit(new VowpalWabbitSettings("-t") { Model = vw })) { Assert.IsTrue(vwSub.Arguments.CommandLine.Contains("--quiet")); Assert.IsTrue(vwSub.Arguments.CommandLine.Contains("-t")); } } using (var vw = new VowpalWabbit("")) { vw.SaveModel("args.model"); } using (var vw = new VowpalWabbitModel(new VowpalWabbitSettings { ModelStream = File.Open("args.model", FileMode.Open) })) { Assert.IsFalse(vw.Arguments.CommandLine.Contains("--quiet")); Assert.IsFalse(vw.Arguments.CommandLine.Contains("-t")); using (var vwSub = new VowpalWabbit(new VowpalWabbitSettings("-t") { Model = vw })) { Assert.IsTrue(vwSub.Arguments.CommandLine.Contains("--quiet")); Assert.IsTrue(vwSub.Arguments.CommandLine.Contains("-t")); } } }
public static void MultiThreadedPrediction() { var example = new MyExample { Income = 40, Age = 25 }; var vwModel = new VowpalWabbitModel("-t -i m1.model"); using (var pool = new VowpalWabbitThreadedPrediction<MyExample>(vwModel)) { // thread-safe using (var vw = pool.GetOrCreate()) { // vw.Value is not thread-safe vw.Value.Predict(example); } // thread-safe pool.UpdateModel(new VowpalWabbitModel("-t -i m2.model")); } }
/// <summary> /// Initializes a new instance of the <see cref="VowpalWabbitThreadedPredictionBase{TVowpalWabbit}"/> class. /// </summary> /// <param name="model">The initial model to use.</param> protected VowpalWabbitThreadedPredictionBase(VowpalWabbitModel model = null) { this.vwPool = new ObjectPool <VowpalWabbitModel, TVowpalWabbit>( ObjectFactory.Create( model, m => { if (m == null) { return(default(TVowpalWabbit)); } var settings = (VowpalWabbitSettings)m.Settings.Clone(); settings.Model = m; // avoid duplicate arguments settings.Arguments = null; return(this.InternalCreate(new VowpalWabbit(settings))); })); }
/// <summary> /// Initializes a new instance of the <see cref="VowpalWabbitThreadedPredictionBase{TVowpalWabbit}"/> class. /// </summary> /// <param name="model">The initial model to use.</param> protected VowpalWabbitThreadedPredictionBase(VowpalWabbitModel model = null) { this.vwPool = new ObjectPool <VowpalWabbitModel, TVowpalWabbit>( ObjectFactory.Create( model, m => { if (m == null) { return(default(TVowpalWabbit)); } var settings = (VowpalWabbitSettings)m.Settings.Clone(); settings.Model = m; // avoid duplicate arguments (e.g. -i) and force testing mode settings.Arguments = m.Arguments.CommandLine.Contains("-t") ? string.Empty : "-t"; return(this.InternalCreate(new VowpalWabbit(settings))); })); }
public void TestSaveLoadSkip() { using (var vw = new VowpalWabbit <SimpleData>("--binary -f saveload.model")) { for (int i = 0; i < 100; i++) { vw.Learn(new SimpleData { Value = 1 }, new SimpleLabel { Label = 1 }); vw.Learn(new SimpleData { Value = -1 }, new SimpleLabel { Label = -1 }); } Assert.AreEqual(1, vw.Predict(new SimpleData { Value = 1 }, VowpalWabbitPredictionType.Scalar)); Assert.AreEqual(-1, vw.Predict(new SimpleData { Value = -1 }, VowpalWabbitPredictionType.Scalar)); } using (var model = new VowpalWabbitModel(new VowpalWabbitSettings { Arguments = "--binary", ModelStream = File.Open("saveload.model", FileMode.Open) })) using (var pool = new VowpalWabbitThreadedPrediction <SimpleData>(new VowpalWabbitSettings { Model = model })) { using (var vw = pool.GetOrCreate()) { Assert.AreEqual(-1, vw.Value.Predict(new SimpleData { Value = -1 }, VowpalWabbitPredictionType.Scalar)); } } }
public static void MultiThreadedPrediction() { var example = new MyExample { Income = 40, Age = 25 }; var vwModel = new VowpalWabbitModel("-t -i m1.model"); using (var pool = new VowpalWabbitThreadedPrediction <MyExample>(vwModel)) { // thread-safe using (var vw = pool.GetOrCreate()) { // vw.Value is not thread-safe vw.Value.Predict(example); } // thread-safe pool.UpdateModel(new VowpalWabbitModel("-t -i m2.model")); } }
public static void TestMemoryLeak() { string outModelFile = "cb_adf_mem_leak.model"; using (var vw = new VowpalWabbit<DataString, DataStringADF>("--cb_adf --rank_all")) { DataString[] sampleData = CreateStringCbAdfData(1000); foreach (DataString example in sampleData) { vw.Learn(example, example.ActionDependentFeatures, example.SelectedActionIndex, example.Label); } vw.Native.SaveModel(outModelFile); } var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings(string.Format("--quiet -t -i {0}", outModelFile), maxExampleCacheSize: 1024)); var pool = new VowpalWabbitThreadedPrediction<DataString, DataStringADF>(vwModel); while (true) { vwModel = new VowpalWabbitModel(new VowpalWabbitSettings(string.Format("--quiet -t -i {0}", outModelFile), maxExampleCacheSize: 1024)); pool.UpdateModel(vwModel); } }
private void InternalTestModel(string modelFile, bool shouldPass) { bool passed = false; try { using (var vw = new VowpalWabbitModel(string.Format("--quiet -t -i {0}", modelFile))) { // should only reach this point if model is valid passed = true; } } catch (VowpalWabbitException ex) { Assert.IsTrue(ex.Message.Contains("corrupted")); } if (shouldPass) { // TODO: test currently fails, should update model version to 8.0.2? Assert.IsTrue(passed); } }
/// <summary> /// Initializes a new <see cref="VowpalWabbitFactory"/> instance. /// </summary> /// <param name="model">The shared model.</param> public VowpalWabbitFactory(VowpalWabbitModel model) : base(model) { }
public void TestSharedModel() { string cbadfModelFile = "models/cb_adf.model"; var sampleData = CreateSampleCbAdfData(); using (var vw = new VowpalWabbit <DataString, DataStringADF>("--cb_adf --rank_all")) { foreach (DataString example in sampleData) { vw.Learn(example); } vw.SaveModel(cbadfModelFile); } // Get ground truth predictions var expectedPredictions = new List <DataStringADF[]>(); using (var vw = new VowpalWabbit <DataString, DataStringADF>(string.Format("-t -i {0}", cbadfModelFile))) { foreach (DataString example in sampleData) { expectedPredictions.Add(vw.Predict(example)); } } // Test synchronous VW instances using shared model using (var vwModel = new VowpalWabbitModel("-t", File.OpenRead(cbadfModelFile))) using (var vwShared1 = new VowpalWabbit <DataString, DataStringADF>(vwModel)) using (var vwShared2 = new VowpalWabbit <DataString, DataStringADF>(vwModel)) { for (int i = 0; i < sampleData.Length; i++) { DataStringADF[] actualPrediction = vwShared1.Predict(sampleData[i]); ReferenceEquals(expectedPredictions[i], actualPrediction); } } // Test concurrent VW instances using shared model and model pool using (var vwModel = new VowpalWabbitModel("-t", File.OpenRead(cbadfModelFile))) using (var vwPool = new ObjectPool <VowpalWabbit <DataString, DataStringADF> >(new VowpalWabbitFactory <DataString, DataStringADF>(vwModel))) { Parallel.For ( fromInclusive: 0, toExclusive: 20, parallelOptions: new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount * 2 }, body: i => { using (PooledObject <VowpalWabbit <DataString, DataStringADF> > vwObject = vwPool.Get()) { var actualPredictions = new List <DataStringADF[]>(); foreach (DataString example in sampleData) { actualPredictions.Add(vwObject.Value.Predict(example)); } Assert.AreEqual(expectedPredictions.Count, actualPredictions.Count); for (int j = 0; j < expectedPredictions.Count; j++) { ReferenceEquals(expectedPredictions[j], actualPredictions[j]); } } } ); } }
public void TestSharedModel() { string cbadfModelFile = "models/cb_adf.model"; var sampleData = CreateSampleCbAdfData(); using (var vw = new VowpalWabbit <DataString, DataStringADF>("--cb_adf --rank_all")) using (var vwSharedValidation = new VowpalWabbitExampleValidator <DataString>("--cb_adf --rank_all")) using (var vwADFValidation = new VowpalWabbitExampleValidator <DataStringADF>("--cb_adf --rank_all")) { foreach (DataString example in sampleData) { Validate(vwSharedValidation, vwADFValidation, example); vw.Learn(example, example.ActionDependentFeatures, example.SelectedActionIndex, example.Label); } vw.Native.SaveModel(cbadfModelFile); } // Get ground truth predictions var expectedPredictions = new List <DataStringADF[]>(); using (var vw = new VowpalWabbit <DataString, DataStringADF>(string.Format("-t -i {0}", cbadfModelFile))) { foreach (DataString example in sampleData) { var pred = vw.Predict(example, example.ActionDependentFeatures); if (pred == null) { expectedPredictions.Add(null); } else { expectedPredictions.Add(pred.Select(p => p.Feature).ToArray()); } } } // Test synchronous VW instances using shared model using (var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings("-t") { ModelStream = File.OpenRead(cbadfModelFile) })) using (var vwShared1 = new VowpalWabbit <DataString, DataStringADF>(new VowpalWabbitSettings { Model = vwModel })) using (var vwShared2 = new VowpalWabbit <DataString, DataStringADF>(new VowpalWabbitSettings { Model = vwModel })) { for (int i = 0; i < sampleData.Length; i++) { var actualPrediction = vwShared1.Predict(sampleData[i], sampleData[i].ActionDependentFeatures); if (actualPrediction == null) { ReferenceEquals(expectedPredictions[i], actualPrediction); } else { ReferenceEquals(expectedPredictions[i], actualPrediction.Select(p => p.Feature).ToArray()); } } } // Test concurrent VW instances using shared model and model pool using (var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings("-t") { ModelStream = File.OpenRead(cbadfModelFile) })) using (var vwPool = new VowpalWabbitThreadedPrediction <DataString, DataStringADF>(vwModel)) { Parallel.For ( fromInclusive: 0, toExclusive: 20, parallelOptions: new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount * 2 }, body: i => { using (var vwObject = vwPool.GetOrCreate()) { var actualPredictions = new List <DataStringADF[]>(); foreach (DataString example in sampleData) { actualPredictions.Add(vwObject.Value.Predict(example, example.ActionDependentFeatures).Select(p => p.Feature).ToArray()); } Assert.AreEqual(expectedPredictions.Count, actualPredictions.Count); for (int j = 0; j < expectedPredictions.Count; j++) { ReferenceEquals(expectedPredictions[j], actualPredictions[j]); } } } ); } }
/// <summary> /// Initializes a new <see cref="VowpalWabbitFactoryBase{TVowpalWabbit}"/> instance. /// </summary> /// <param name="model">The shared model.</param> public VowpalWabbitFactoryBase(VowpalWabbitModel model) { this.model = model; }
public void TestCbAdfExplore() { var json = JsonConvert.SerializeObject(new { U = new { age = "18" }, _multi = new[] { new { G = new { _text = "this rocks" }, K = new { constant = 1, doc = "1" } }, new { G = new { _text = "something NYC" }, K = new { constant = 1, doc = "2" } }, }, _label_Action = 2, _label_Probability = 0.1, _label_Cost = -1, _labelIndex = 1 }); using (var vw = new VowpalWabbitJson("--cb_explore_adf --bag 4 --epsilon 0.0001 --cb_type mtr --marginal K -q UG -b 26 --power_t 0 --l1 1e-9 -l 4e-3")) { for (int i = 0; i < 50; i++) { var pred = vw.Learn(json, VowpalWabbitPredictionType.ActionProbabilities); Assert.AreEqual(2, pred.Length); if (i > 40) { Assert.AreEqual(1, (int)pred[0].Action); Assert.IsTrue(pred[0].Score > .9); Assert.AreEqual(0, (int)pred[1].Action); Assert.IsTrue(pred[1].Score < .1); } } vw.Native.SaveModel("cbadfexplore.model"); } using (var vw = new VowpalWabbitJson(new VowpalWabbitSettings { Arguments = "-t", ModelStream = File.Open("cbadfexplore.model", FileMode.Open) })) { var predObj = vw.Predict(json, VowpalWabbitPredictionType.Dynamic); Assert.IsInstanceOfType(predObj, typeof(ActionScore[])); var pred = (ActionScore[])predObj; Assert.AreEqual(1, (int)pred[0].Action); Assert.IsTrue(pred[0].Score > .9); Assert.AreEqual(0, (int)pred[1].Action); Assert.IsTrue(pred[1].Score < .1); } using (var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings { ModelStream = File.Open("cbadfexplore.model", FileMode.Open) })) using (var vwSeeded = new VowpalWabbitJson(new VowpalWabbitSettings { Model = vwModel })) { var pred = vwSeeded.Predict(json, VowpalWabbitPredictionType.ActionProbabilities); Assert.AreEqual(1, (int)pred[0].Action); Assert.IsTrue(pred[0].Score > .9); Assert.AreEqual(0, (int)pred[1].Action); Assert.IsTrue(pred[1].Score < .1); } using (var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings { ModelStream = File.Open("cbadfexplore.model", FileMode.Open) })) { using (var vwPool = new VowpalWabbitJsonThreadedPrediction(vwModel)) using (var vw = vwPool.GetOrCreate()) { var predObj = vw.Value.Predict(json, VowpalWabbitPredictionType.Dynamic); Assert.IsInstanceOfType(predObj, typeof(ActionScore[])); var pred = (ActionScore[])predObj; Assert.AreEqual(1, (int)pred[0].Action); Assert.IsTrue(pred[0].Score > .9); Assert.AreEqual(0, (int)pred[1].Action); Assert.IsTrue(pred[1].Score < .1); } } }
public void TestExampleCache() { var random = new Random(123); var examples = new List <CachedData>(); for (int i = 0; i < 1000; i++) { examples.Add(new CachedData { Label = new SimpleLabel { Label = 1 }, Feature = random.NextDouble() }); } for (int i = 0; i < 1000; i++) { var cachedData = new CachedData { Label = new SimpleLabel { Label = 2 }, Feature = 10 + random.NextDouble() }; examples.Add(cachedData); examples.Add(cachedData); } using (var vw = new VowpalWabbit <CachedData>("-k -c --passes 10", new VowpalWabbitSerializerSettings { EnableExampleCaching = false })) { foreach (var example in examples) { using (var vwExample = vw.ReadExample(example)) { vwExample.Learn(); } } vw.RunMultiPass(); vw.SaveModel("model1"); } using (var vwModel = new VowpalWabbitModel("-t", File.OpenRead("model1"))) using (var vwCached = new VowpalWabbit <CachedData>(vwModel, new VowpalWabbitSerializerSettings { EnableExampleCaching = true, MaxExampleCacheSize = 5 })) using (var vw = new VowpalWabbit <CachedData>(vwModel, new VowpalWabbitSerializerSettings { EnableExampleCaching = false })) { foreach (var example in examples) { using (var vwCachedExample = vwCached.ReadExample(example)) using (var vwExample = vw.ReadExample(example)) { var cachedPrediction = vwCachedExample.Predict <VowpalWabbitScalarPrediction>(); var prediction = vwExample.Predict <VowpalWabbitScalarPrediction>(); Assert.AreEqual(prediction.Value, cachedPrediction.Value); Assert.AreEqual(example.Label.Label, Math.Round(prediction.Value)); } } } }
public void Test1and2() { var references = File.ReadAllLines(@"pred-sets\ref\0001.predict").Select(l => float.Parse(l, CultureInfo.InvariantCulture)).ToArray(); var input = new List <Test1>(); using (var vwStr = new VowpalWabbit(" -k -c test1and2.str --passes 8 -l 20 --power_t 1 --initial_t 128000 --ngram 3 --skips 1 --invariant --holdout_off")) using (var vw = new VowpalWabbit <Test1>(" -k -c test1and2 --passes 8 -l 20 --power_t 1 --initial_t 128000 --ngram 3 --skips 1 --invariant --holdout_off")) { var lineNr = 0; VWTestHelper.ParseInput( File.OpenRead(@"train-sets\0001.dat"), new MyListener(data => { input.Add(data); var expected = vwStr.Learn <VowpalWabbitScalarPrediction>(data.Line); using (var example = vw.ReadExample(data)) { var actual = example.LearnAndPredict <VowpalWabbitScalarPrediction>(); Assert.AreEqual(expected.Value, actual.Value, 1e-6, "Learn output differs on line: " + lineNr); } lineNr++; })); vwStr.RunMultiPass(); vw.RunMultiPass(); vwStr.SaveModel("models/str0001.model"); vw.SaveModel("models/0001.model"); VWTestHelper.AssertEqual(@"train-sets\ref\0001.stderr", vwStr.PerformanceStatistics); VWTestHelper.AssertEqual(@"train-sets\ref\0001.stderr", vw.PerformanceStatistics); } Assert.AreEqual(input.Count, references.Length); using (var vwModel = new VowpalWabbitModel("-k -t --invariant", File.OpenRead("models/0001.model"))) using (var vwInMemoryShared1 = new VowpalWabbit(vwModel)) using (var vwInMemoryShared2 = new VowpalWabbit <Test1>(vwModel)) using (var vwInMemory = new VowpalWabbit("-k -t --invariant", File.OpenRead("models/0001.model"))) using (var vwStr = new VowpalWabbit("-k -t -i models/str0001.model --invariant")) using (var vw = new VowpalWabbit <Test1>("-k -t -i models/0001.model --invariant")) { for (var i = 0; i < input.Count; i++) { var actualStr = vwStr.Predict <VowpalWabbitScalarPrediction>(input[i].Line); var actualShared1 = vwInMemoryShared1.Predict <VowpalWabbitScalarPrediction>(input[i].Line); var actualInMemory = vwInMemory.Predict <VowpalWabbitScalarPrediction>(input[i].Line); using (var example = vw.ReadExample(input[i])) using (var exampleInMemory2 = vwInMemoryShared2.ReadExample(input[i])) { var actual = example.Predict <VowpalWabbitScalarPrediction>(); var actualShared2 = exampleInMemory2.Predict <VowpalWabbitScalarPrediction>(); Assert.AreEqual(references[i], actualStr.Value, 1e-5); Assert.AreEqual(references[i], actualShared1.Value, 1e-5); Assert.AreEqual(references[i], actualInMemory.Value, 1e-5); Assert.AreEqual(references[i], actual.Value, 1e-5); Assert.AreEqual(references[i], actualShared2.Value, 1e-5); } } // VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vwInMemoryShared2.PerformanceStatistics); //VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vwInMemoryShared1.PerformanceStatistics); VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vwInMemory.PerformanceStatistics); VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vwStr.PerformanceStatistics); VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vw.PerformanceStatistics); } }
public void TestSharedModel() { string cbadfModelFile = "models/cb_adf.model"; var sampleData = CreateSampleCbAdfData(); using (var vw = new VowpalWabbit<DataString, DataStringADF>("--cb_adf --rank_all")) { foreach (DataString example in sampleData) { vw.Learn(example, example.ActionDependentFeatures, example.SelectedActionIndex, example.Label); } vw.Native.SaveModel(cbadfModelFile); } // Get ground truth predictions var expectedPredictions = new List<DataStringADF[]>(); using (var vw = new VowpalWabbit<DataString, DataStringADF>(string.Format("-t -i {0}", cbadfModelFile))) { foreach (DataString example in sampleData) { expectedPredictions.Add(vw.Predict(example, example.ActionDependentFeatures)); } } // Test synchronous VW instances using shared model using (var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings("-t", modelStream: File.OpenRead(cbadfModelFile)))) using (var vwShared1 = new VowpalWabbit<DataString, DataStringADF>(new VowpalWabbitSettings(model: vwModel))) using (var vwShared2 = new VowpalWabbit<DataString, DataStringADF>(new VowpalWabbitSettings(model: vwModel))) { for (int i = 0; i < sampleData.Length; i++) { DataStringADF[] actualPrediction = vwShared1.Predict(sampleData[i], sampleData[i].ActionDependentFeatures); ReferenceEquals(expectedPredictions[i], actualPrediction); } } // Test concurrent VW instances using shared model and model pool using (var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings("-t", modelStream: File.OpenRead(cbadfModelFile)))) using (var vwPool = new VowpalWabbitThreadedPrediction<DataString, DataStringADF>(vwModel)) { Parallel.For ( fromInclusive: 0, toExclusive: 20, parallelOptions: new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount * 2 }, body: i => { using (var vwObject = vwPool.GetOrCreate()) { var actualPredictions = new List<DataStringADF[]>(); foreach (DataString example in sampleData) { actualPredictions.Add(vwObject.Value.Predict(example, example.ActionDependentFeatures)); } Assert.AreEqual(expectedPredictions.Count, actualPredictions.Count); for (int j = 0; j < expectedPredictions.Count; j++) { ReferenceEquals(expectedPredictions[j], actualPredictions[j]); } } } ); } }
/// <summary> /// Initializes a new <see cref="VowpalWabbitFactory{TExample, TActionDependentFeature}"/> instance. /// </summary> /// <param name="model">The shared model.</param> /// <param name="settings">The serializer settings.</param> public VowpalWabbitPredictorFactory(VowpalWabbitModel model, VowpalWabbitSerializerSettings settings = null) : base(model) { this.settings = settings; }
/// <summary> /// Updates the model used for prediction in a thread-safe manner. /// </summary> /// <param name="model">The new model to be used.</param> public void UpdateModel(VowpalWabbitModel model) { this.vwPool.UpdateFactory(ObjectFactory.Create( model, m => this.InternalCreate(new VowpalWabbit(m.Settings.ShallowCopy(model: m))))); }
public void TestExampleCache() { var random = new Random(123); var examples = new List<CachedData>(); for (int i = 0; i < 1000; i++) { examples.Add(new CachedData { Label = new SimpleLabel { Label = 1 }, Feature = random.NextDouble() }); } for (int i = 0; i < 1000; i++) { var cachedData = new CachedData { Label = new SimpleLabel { Label = 2 }, Feature = 10 + random.NextDouble() }; examples.Add(cachedData); examples.Add(cachedData); } using (var vw = new VowpalWabbit<CachedData>(new VowpalWabbitSettings("-k -c --passes 10", enableExampleCaching: false))) { foreach (var example in examples) { vw.Learn(example, example.Label); } vw.Native.RunMultiPass(); vw.Native.SaveModel("model1"); } using (var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings("-t", modelStream: File.OpenRead("model1")))) using (var vwCached = new VowpalWabbit<CachedData>(new VowpalWabbitSettings(model: vwModel, enableExampleCaching: true, maxExampleCacheSize: 5 ))) using (var vw = new VowpalWabbit<CachedData>(new VowpalWabbitSettings(model: vwModel, enableExampleCaching: false ))) { foreach (var example in examples) { var cachedPrediction = vwCached.Predict(example, VowpalWabbitPredictionType.Scalar); var prediction = vw.Predict(example, VowpalWabbitPredictionType.Scalar); Assert.AreEqual(prediction, cachedPrediction); Assert.AreEqual(example.Label.Label, Math.Round(prediction)); } } }
public void Test1and2() { var references = File.ReadAllLines(@"pred-sets\ref\0001.predict").Select(l => float.Parse(l, CultureInfo.InvariantCulture)).ToArray(); var input = new List<Test1>(); using (var vwStr = new VowpalWabbit(" -k -c test1and2.str --passes 8 -l 20 --power_t 1 --initial_t 128000 --ngram 3 --skips 1 --invariant --holdout_off")) using (var vw = new VowpalWabbit<Test1>(" -k -c test1and2 --passes 8 -l 20 --power_t 1 --initial_t 128000 --ngram 3 --skips 1 --invariant --holdout_off")) { var lineNr = 0; VWTestHelper.ParseInput( File.OpenRead(@"train-sets\0001.dat"), new MyListener(data => { input.Add(data); var expected = vwStr.Learn<VowpalWabbitScalarPrediction>(data.Line); using (var example = vw.ReadExample(data)) { var actual = example.LearnAndPredict<VowpalWabbitScalarPrediction>(); Assert.AreEqual(expected.Value, actual.Value, 1e-6, "Learn output differs on line: " + lineNr); } lineNr++; })); vwStr.RunMultiPass(); vw.RunMultiPass(); vwStr.SaveModel("models/str0001.model"); vw.SaveModel("models/0001.model"); VWTestHelper.AssertEqual(@"train-sets\ref\0001.stderr", vwStr.PerformanceStatistics); VWTestHelper.AssertEqual(@"train-sets\ref\0001.stderr", vw.PerformanceStatistics); } Assert.AreEqual(input.Count, references.Length); using (var vwModel = new VowpalWabbitModel("-k -t --invariant", File.OpenRead("models/0001.model"))) using (var vwInMemoryShared1 = new VowpalWabbit(vwModel)) using (var vwInMemoryShared2 = new VowpalWabbit<Test1>(vwModel)) using (var vwInMemory = new VowpalWabbit("-k -t --invariant", File.OpenRead("models/0001.model"))) using (var vwStr = new VowpalWabbit("-k -t -i models/str0001.model --invariant")) using (var vw = new VowpalWabbit<Test1>("-k -t -i models/0001.model --invariant")) { for (var i = 0; i < input.Count; i++) { var actualStr = vwStr.Predict<VowpalWabbitScalarPrediction>(input[i].Line); var actualShared1 = vwInMemoryShared1.Predict<VowpalWabbitScalarPrediction>(input[i].Line); var actualInMemory = vwInMemory.Predict<VowpalWabbitScalarPrediction>(input[i].Line); using (var example = vw.ReadExample(input[i])) using (var exampleInMemory2 = vwInMemoryShared2.ReadExample(input[i])) { var actual = example.Predict<VowpalWabbitScalarPrediction>(); var actualShared2 = exampleInMemory2.Predict<VowpalWabbitScalarPrediction>(); Assert.AreEqual(references[i], actualStr.Value, 1e-5); Assert.AreEqual(references[i], actualShared1.Value, 1e-5); Assert.AreEqual(references[i], actualInMemory.Value, 1e-5); Assert.AreEqual(references[i], actual.Value, 1e-5); Assert.AreEqual(references[i], actualShared2.Value, 1e-5); } } // VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vwInMemoryShared2.PerformanceStatistics); //VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vwInMemoryShared1.PerformanceStatistics); VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vwInMemory.PerformanceStatistics); VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vwStr.PerformanceStatistics); VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vw.PerformanceStatistics); } }
public void VwModelRefCountingTest() { var model = new VowpalWabbitModel(""); model.Dispose(); }
public void Test1and2() { var references = File.ReadAllLines(@"pred-sets\ref\0001.predict").Select(l => float.Parse(l, CultureInfo.InvariantCulture)).ToArray(); var input = new List<Test1>(); using (var vwStr = new VowpalWabbit(" -k -c test1and2.str --passes 8 -l 20 --power_t 1 --initial_t 128000 --ngram 3 --skips 1 --invariant --holdout_off")) using (var vw = new VowpalWabbit<Test1>(new VowpalWabbitSettings(" -k -c test1and2 --passes 8 -l 20 --power_t 1 --initial_t 128000 --ngram 3 --skips 1 --invariant --holdout_off") { EnableExampleCaching = false })) using (var vwValidate = new VowpalWabbitExampleValidator<Test1>("-l 20 --power_t 1 --initial_t 128000 --ngram 3 --skips 1 --invariant --holdout_off")) { var lineNr = 0; VWTestHelper.ParseInput( File.OpenRead(@"train-sets\0001.dat"), new MyListener(data => { input.Add(data); vwValidate.Validate(data.Line, data, data.Label); var expected = vwStr.Learn(data.Line, VowpalWabbitPredictionType.Dynamic); Assert.IsInstanceOfType(expected, typeof(float)); var actual = vw.Learn(data, data.Label, VowpalWabbitPredictionType.Scalar); Assert.AreEqual((float)expected, actual, 1e-6, "Learn output differs on line: " + lineNr); lineNr++; })); vwStr.RunMultiPass(); vw.Native.RunMultiPass(); vwStr.SaveModel("models/str0001.model"); vw.Native.SaveModel("models/0001.model"); VWTestHelper.AssertEqual(@"train-sets\ref\0001.stderr", vwStr.PerformanceStatistics); VWTestHelper.AssertEqual(@"train-sets\ref\0001.stderr", vw.Native.PerformanceStatistics); } Assert.AreEqual(input.Count, references.Length); using (var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings("-k -t --invariant") { ModelStream = File.OpenRead("models/0001.model") })) using (var vwInMemoryShared1 = new VowpalWabbit(new VowpalWabbitSettings { Model = vwModel })) using (var vwInMemoryShared2 = new VowpalWabbit<Test1>(new VowpalWabbitSettings { Model = vwModel })) using (var vwInMemory = new VowpalWabbit(new VowpalWabbitSettings("-k -t --invariant") { ModelStream = File.OpenRead("models/0001.model") })) using (var vwStr = new VowpalWabbit("-k -t -i models/str0001.model --invariant")) using (var vwNative = new VowpalWabbit("-k -t -i models/0001.model --invariant")) using (var vw = new VowpalWabbit<Test1>("-k -t -i models/0001.model --invariant")) using (var vwModel2 = new VowpalWabbitModel("-k -t --invariant -i models/0001.model")) using (var vwInMemoryShared3 = new VowpalWabbit<Test1>(new VowpalWabbitSettings { Model = vwModel2 })) { for (var i = 0; i < input.Count; i++) { var actualStr = vwStr.Predict(input[i].Line, VowpalWabbitPredictionType.Scalar); var actualNative = vwNative.Predict(input[i].Line, VowpalWabbitPredictionType.Scalar); var actualInMemory = vwInMemory.Predict(input[i].Line, VowpalWabbitPredictionType.Scalar); var actual = vw.Predict(input[i], VowpalWabbitPredictionType.Scalar, input[i].Label); var actualShared1 = vwInMemoryShared1.Predict(input[i].Line, VowpalWabbitPredictionType.Scalar); var actualShared2 = vwInMemoryShared2.Predict(input[i], VowpalWabbitPredictionType.Scalar, input[i].Label); var actualShared3 = vwInMemoryShared3.Predict(input[i], VowpalWabbitPredictionType.Scalar, input[i].Label); Assert.AreEqual(references[i], actualStr, 1e-5); Assert.AreEqual(references[i], actualNative, 1e-5); Assert.AreEqual(references[i], actualInMemory, 1e-5); Assert.AreEqual(references[i], actual, 1e-5); Assert.AreEqual(references[i], actualShared1, 1e-5); Assert.AreEqual(references[i], actualShared2, 1e-5); Assert.AreEqual(references[i], actualShared3, 1e-5); } // due to shared usage the counters don't match up //VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vwInMemoryShared2.Native.PerformanceStatistics); //VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vwInMemoryShared1.PerformanceStatistics); VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vwInMemory.PerformanceStatistics); VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vwStr.PerformanceStatistics); VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vw.Native.PerformanceStatistics); } }
public void TestExampleCache() { var random = new Random(123); var examples = new List<CachedData>(); for (int i = 0; i < 1000; i++) { examples.Add(new CachedData { Label = new SimpleLabel { Label = 1 }, Feature = random.NextDouble() }); } for (int i = 0; i < 1000; i++) { var cachedData = new CachedData { Label = new SimpleLabel { Label = 2 }, Feature = 10 + random.NextDouble() }; examples.Add(cachedData); examples.Add(cachedData); } using (var vw = new VowpalWabbit<CachedData>("-k -c --passes 10", new VowpalWabbitSerializerSettings { EnableExampleCaching = false })) { foreach (var example in examples) { using (var vwExample = vw.ReadExample(example)) { vwExample.Learn(); } } vw.RunMultiPass(); vw.SaveModel("model1"); } using (var vwModel = new VowpalWabbitModel("-t", File.OpenRead("model1"))) using (var vwCached = new VowpalWabbit<CachedData>(vwModel, new VowpalWabbitSerializerSettings { EnableExampleCaching = true, MaxExampleCacheSize = 5 })) using (var vw = new VowpalWabbit<CachedData>(vwModel, new VowpalWabbitSerializerSettings { EnableExampleCaching = false })) { foreach (var example in examples) { using (var vwCachedExample = vwCached.ReadExample(example)) using (var vwExample = vw.ReadExample(example)) { var cachedPrediction = vwCachedExample.Predict<VowpalWabbitScalarPrediction>(); var prediction = vwExample.Predict<VowpalWabbitScalarPrediction>(); Assert.AreEqual(prediction.Value, cachedPrediction.Value); Assert.AreEqual(example.Label.Label, Math.Round(prediction.Value)); } } } }
/// <summary> /// Initializes a new instance of <see cref="VowpalWabbitThreadedPrediction"/>. /// </summary> /// <param name="model">The model used by each pool instance.</param> public VowpalWabbitThreadedPrediction(VowpalWabbitModel model) : base(model) { }
public void TestSaveLoadSkip() { using (var vw = new VowpalWabbit<SimpleData>("--binary -f saveload.model")) { for (int i = 0; i < 100; i++) { vw.Learn(new SimpleData { Value = 1 }, new SimpleLabel { Label = 1 }); vw.Learn(new SimpleData { Value = -1 }, new SimpleLabel { Label = -1 }); } Assert.AreEqual(1, vw.Predict(new SimpleData { Value = 1 }, VowpalWabbitPredictionType.Scalar)); Assert.AreEqual(-1, vw.Predict(new SimpleData { Value = -1 }, VowpalWabbitPredictionType.Scalar)); } using (var model = new VowpalWabbitModel(new VowpalWabbitSettings { Arguments = "--binary", ModelStream = File.Open("saveload.model", FileMode.Open) })) using (var pool = new VowpalWabbitThreadedPrediction<SimpleData>(new VowpalWabbitSettings { Model = model })) { using (var vw = pool.GetOrCreate()) { Assert.AreEqual(-1, vw.Value.Predict(new SimpleData { Value = -1 }, VowpalWabbitPredictionType.Scalar)); } } }