public void RunTest() { double[][] input = { new double[] { 55, 0 }, // 0 - no cancer new double[] { 28, 0 }, // 0 new double[] { 65, 1 }, // 0 new double[] { 46, 0 }, // 1 - have cancer new double[] { 86, 1 }, // 1 new double[] { 56, 1 }, // 1 new double[] { 85, 0 }, // 0 new double[] { 33, 0 }, // 0 new double[] { 21, 1 }, // 0 new double[] { 42, 1 }, // 1 }; double[] output = { 0, 0, 0, 1, 1, 1, 0, 0, 0, 1 }; int[] labels = output.Apply(x => x > 0 ? +1 : -1); var svm = new SupportVectorMachine(inputs: 2); var teacher = new ProbabilisticNewtonMethod(svm, input, labels); teacher.Tolerance = 1e-10; teacher.Complexity = 1e+10; double error = teacher.Run(); var regression = LogisticRegression.FromWeights(svm.ToWeights()); double[] actual = new double[output.Length]; for (int i = 0; i < actual.Length; i++) { actual[i] = regression.Compute(input[i]); } double ageOdds = regression.GetOddsRatio(1); // 1.0208597028836701 double smokeOdds = regression.GetOddsRatio(2); // 5.8584748789881331 Assert.AreEqual(0.3, error); Assert.AreEqual(1.0208597028836701, ageOdds, 1e-4); Assert.AreEqual(5.8584748789881331, smokeOdds, 1e-4); Assert.IsFalse(Double.IsNaN(ageOdds)); Assert.IsFalse(Double.IsNaN(smokeOdds)); Assert.AreEqual(-2.4577464307294092, regression.Intercept, 1e-8); Assert.AreEqual(-2.4577464307294092, regression.Coefficients[0], 1e-8); Assert.AreEqual(0.020645118265359252, regression.Coefficients[1], 1e-8); Assert.AreEqual(1.7678893101571855, regression.Coefficients[2], 1e-8); }
public static void train_one(Problem prob, Parameters param, out double[] w, double Cp, double Cn) { double[][] inputs = prob.Inputs; int[] labels = prob.Outputs.Apply(x => x >= 0 ? 1 : -1); double eps = param.Tolerance; int pos = 0; for (int i = 0; i < labels.Length; i++) { if (labels[i] >= 0) { pos++; } } int neg = prob.Outputs.Length - pos; double primal_solver_tol = eps * Math.Max(Math.Min(pos, neg), 1.0) / prob.Inputs.Length; SupportVectorMachine svm = new SupportVectorMachine(prob.Dimensions); ISupportVectorMachineLearning teacher = null; switch (param.Solver) { case LibSvmSolverType.L2RegularizedLogisticRegression: // l2r_lr_fun teacher = new ProbabilisticNewtonMethod(svm, inputs, labels) { PositiveWeight = Cp, NegativeWeight = Cn, Tolerance = primal_solver_tol }; break; case LibSvmSolverType.L2RegularizedL2LossSvc: // fun_obj=new l2r_l2_svc_fun(prob, C); teacher = new LinearNewtonMethod(svm, inputs, labels) { PositiveWeight = Cp, NegativeWeight = Cn, Tolerance = primal_solver_tol }; break; case LibSvmSolverType.L2RegularizedL2LossSvcDual: // solve_l2r_l1l2_svc(prob, w, eps, Cp, Cn, L2R_L2LOSS_SVC_DUAL); teacher = new LinearDualCoordinateDescent(svm, inputs, labels) { Loss = Loss.L2, PositiveWeight = Cp, NegativeWeight = Cn, }; break; case LibSvmSolverType.L2RegularizedL1LossSvcDual: // solve_l2r_l1l2_svc(prob, w, eps, Cp, Cn, L2R_L1LOSS_SVC_DUAL); teacher = new LinearDualCoordinateDescent(svm, inputs, labels) { Loss = Loss.L1, PositiveWeight = Cp, NegativeWeight = Cn, }; break; case LibSvmSolverType.L1RegularizedLogisticRegression: // solve_l1r_lr(&prob_col, w, primal_solver_tol, Cp, Cn); teacher = new ProbabilisticCoordinateDescent(svm, inputs, labels) { PositiveWeight = Cp, NegativeWeight = Cn, Tolerance = primal_solver_tol }; break; case LibSvmSolverType.L2RegularizedLogisticRegressionDual: // solve_l2r_lr_dual(prob, w, eps, Cp, Cn); teacher = new ProbabilisticDualCoordinateDescent(svm, inputs, labels) { PositiveWeight = Cp, NegativeWeight = Cn, Tolerance = primal_solver_tol, }; break; } Trace.WriteLine("Training " + param.Solver); // run the learning algorithm var sw = Stopwatch.StartNew(); double error = teacher.Run(); sw.Stop(); // save the solution w = svm.ToWeights(); Trace.WriteLine(String.Format("Finished {0}: {1} in {2}", param.Solver, error, sw.Elapsed)); }
public void logistic_regression_sparse_test() { #region doc_logreg_sparse // Declare some training data. This is exactly the same // data used in the LogisticRegression documentation page // Suppose we have the following data about some patients. // The first variable is continuous and represent patient // age. The second variable is dichotomic and give whether // they smoke or not (This is completely fictional data). // We also know if they have had lung cancer or not, and // we would like to know whether smoking has any connection // with lung cancer (This is completely fictional data). Sparse <double>[] input = { // age, smokes?, had cancer? Sparse.FromDense(new double[] { 55, 0 }), // false - no cancer Sparse.FromDense(new double[] { 28, 0 }), // false Sparse.FromDense(new double[] { 65, 1 }), // false Sparse.FromDense(new double[] { 46, 0 }), // true - had cancer Sparse.FromDense(new double[] { 86, 1 }), // true Sparse.FromDense(new double[] { 56, 1 }), // true Sparse.FromDense(new double[] { 85, 0 }), // false Sparse.FromDense(new double[] { 33, 0 }), // false Sparse.FromDense(new double[] { 21, 1 }), // false Sparse.FromDense(new double[] { 42, 1 }), // true }; double[] output = // Whether each patient had lung cancer or not { 0, 0, 0, 1, 1, 1, 0, 0, 0, 1 }; // Create the probabilistic-SVM learning algorithm var teacher = new ProbabilisticNewtonMethod <Linear, Sparse <double> >() { Tolerance = 1e-10, Complexity = 1e+10, // learn a hard-margin model }; // Learn the support vector machine var svm = teacher.Learn(input, output); // Convert the svm to logistic regression var regression = (LogisticRegression)svm; // Compute the predicted outcome for inputs bool[] predicted = regression.Decide(input.ToDense(regression.NumberOfInputs)); // Compute probability scores for the outputs double[] scores = regression.Score(input.ToDense(regression.NumberOfInputs)); // Compute odds-ratio as in the LogisticRegression example double ageOdds = regression.GetOddsRatio(1); // 1.0208597028836701 double smokeOdds = regression.GetOddsRatio(2); // 5.8584748789881331 // Compute the classification error as in SVM example double error = new ZeroOneLoss(output).Loss(predicted); #endregion Assert.AreEqual(0.2, error); Assert.AreEqual(1.0208597028836701, ageOdds, 1e-4); Assert.AreEqual(5.8584748789881331, smokeOdds, 1e-4); Assert.AreEqual(-2.4577464307294092, regression.Intercept, 1e-8); Assert.AreEqual(-2.4577464307294092, regression.Coefficients[0], 1e-8); Assert.AreEqual(0.020645118265359252, regression.Coefficients[1], 1e-8); Assert.AreEqual(1.7678893101571855, regression.Coefficients[2], 1e-8); }