private int RecvCompleteCommandFromSlave() { int source; int command = Mpi.RecvInt(Mpi.AnySource, (int)Tag.Command, Mpi.CommWorld, out source); if (command == (int)Command.TaskIsComplete) { return(source); } throw new InvalidOperationException(); }
public static int CalcLocalNxStart(this Mpi mpi, LateralDimensions lateral) { var fullNx = lateral.Nx * 2; if (fullNx % mpi.Size != 0) { throw new ArgumentException($"fullNx % mpi.Size != 0, mpi.Size = [{mpi.Size}], fullNx = [{fullNx}]"); } var localNxSize = fullNx / mpi.Size; return(localNxSize * mpi.Rank); }
private static void RunFullVersion(ForwardProject project) { using (var mpi_world = Mpi.Init()) { var mpi = mpi_world.Dup(); using (var memoryProvider = new FftWMemoryProvider()) { using (var runner = new ExtremeMtSolverRunner(project, memoryProvider, mpi)) { var model = ModelGenUtils.LoadCartesianModel(project.ModelFile, mpi); runner.Run(model); } } mpi.Dispose(); } }
override public void initialize() { itmax = Instance.itmax; problem_class = Instance.CLASS; isiz01 = Instance.isiz01; isiz02 = Instance.isiz02; isiz03 = Instance.isiz03; nx0 = Blocks.nx0; ny0 = Blocks.ny0; nz0 = Blocks.nz0; worldcomm = Mpi.worldComm(); }
public static CartesianModel DistributedLoadModel(Mpi mpi, string path) { if (mpi == null || !mpi.IsParallel) { return(LoadModel(path)); } if (mpi.IsMaster) { return(DistributedLoadModelMaster(mpi, path)); } else { return(DistributedLoadModelSlave(mpi, path)); } }
public void SerializeClientSecondMessage() { BigInteger key = BigInteger.Parse(clientPublicKey, NumberStyles.HexNumber); Mpi _publicKey = new Mpi(key); ClientMessage2 m = new ClientMessage2(_publicKey, clientProof); m.InitialVector = clientInitialVector; foreach (KeyValuePair <string, string> p in clientOptions) { m.Options.Add(p.Key, p.Value); } byte[] serialized = m.Serialize(); Assert.IsTrue(serialized.SequenceEqual(expectedClientMessage2)); }
public AtoAGreenTensorCalculatorComponent(ForwardSolver solver) : base(solver) { if (Solver.Engine == ForwardSolverEngine.Giem2g) { if (!Solver.IsParallel) { throw new NotImplementedException("GIEM2G works only in parallel mode!"); } _plan = null; _scalarCalc = null; _tensorCalc = null; _mirroringX = false; _realCalcNxStart = -1; _totalNxLength = -1; _calcNxLength = -1; return; } _plan = new ScalarPlansCreater(Model.LateralDimensions, HankelCoefficients.LoadN40(), Solver.Settings.NumberOfHankels) .CreateForAnomalyToAnomaly(); _scalarCalc = GreenScalarCalculator.NewAtoACalculator(Logger, Model); _tensorCalc = new AtoAGreenTensorCalculator(Logger, Model, MemoryProvider); if (Solver.IsParallel) { var localNxStart = Mpi.CalcLocalNxStart(Model.LateralDimensions); _totalNxLength = Mpi.CalcLocalNxLength(Model.LateralDimensions); _mirroringX = localNxStart >= Model.Nx; _realCalcNxStart = _mirroringX ? 2 * Model.Nx - localNxStart - _totalNxLength + 1 : localNxStart; _calcNxLength = _totalNxLength; if (_realCalcNxStart + _totalNxLength == Model.Nx + 1) { _calcNxLength--; } } else { _mirroringX = false; _realCalcNxStart = 0; _totalNxLength = 2 * Model.Nx; _calcNxLength = Model.Nx; } }
public CustomFftPlan CreatePlan(Complex *input, Complex *output, Mpi mpi, int fullNx, int fullNy, int nc, int dimension) { var localNx = mpi.CalcLocalNxLength(fullNx); var numberOfLocalFftsAlongSecondDimension = GetNumberOfLocalFftsAlongSecondDimension(mpi, fullNy, nc); var plan = new CustomFftPlan(input, output, dimension) { Ny = fullNy, Nc = nc, NumberOfMpiProcesses = mpi.Size, LocalNx = localNx, NumberOfLocalFftsAlongSecondDimension = numberOfLocalFftsAlongSecondDimension }; CreateInternalPlans(plan); return(plan); }
public void AllocateBuffersAndCreatePlansParallel(int nx, int ny, int nz, Mpi mpi) { CustomFft = new CustomDistributedFft(mpi, _profiler); IsParallel = true; var localSize3Nz = CustomDistributedFft.GetLocalSize(mpi, nx * 2, ny * 2, 3 * nz); var ranks = Enumerable.Range(0, mpi.Size / 2).ToArray(); RealModelPart = mpi.Merge(ranks); _inputBuffer = _memoryProvider.AllocateComplex(localSize3Nz); _outputBuffer = _memoryProvider.AllocateComplex(localSize3Nz); Plan3Nz = CreatePlan2D(CustomFft, mpi, _inputBuffer, _outputBuffer, nx * 2, ny * 2, 3 * nz); Plan1Nz = CreatePlan2D(CustomFft, mpi, _inputBuffer, _outputBuffer, nx * 2, ny * 2, 1 * nz); Plan3 = CreatePlan2D(CustomFft, mpi, _inputBuffer, _outputBuffer, nx * 2, ny * 2, 3); Plan1 = CreatePlan2D(CustomFft, mpi, _inputBuffer, _outputBuffer, nx * 2, ny * 2, 1); }
private static Tuple <int, int> CalcHalfLocal(this Mpi mpi, int rank, int nx) { if (mpi.Size == 1) { return(new Tuple <int, int>(0, nx)); } var halfNx = nx; var halfSize = (mpi.Size / 2); if (halfNx % halfSize != 0) { throw new ArgumentException($"halfNx % halfSize != 0, halfSize = [{halfSize}], halfNx = [{halfNx}]"); } var halfNxLength = halfNx / halfSize; var halfNxStart = halfNxLength * rank; return(new Tuple <int, int>(halfNxStart, halfNxLength)); }
public ParallelConsoleLogger(Mpi mpi, bool permit = false) { _mpi = mpi; if (mpi == null) { throw new ArgumentNullException(nameof(mpi)); } _name = mpi.GetProcessorName(); _rank = mpi.Rank; if (_mpi.IsMaster) { Console.WriteLine($"Parallel logger started on [master {_name}] at {CreationTime}"); _permitToWrite = true; } else { _permitToWrite = permit; } }
private static CartesianModel GenerateModel(Mpi mpi, ModelSettings settings, Func <NonMeshedModel> genNonMeshed) { var nonMeshed = genNonMeshed(); var converter = new NonMeshedToCartesianModelConverter(nonMeshed); var mesh = settings.Mesh; converter.SetManualBoundaries(settings.ManualBoundaries); var nxStart = 0; var nxLength = mesh.Nx; if (mpi != null && mpi.IsParallel) { nxStart = mpi.CalcLocalHalfNxStart(mesh.Nx); nxLength = mpi.CalcLocalHalfNxLength(mesh.Nx); } var cartesian = converter.Convert(mesh, nxStart, nxLength); return(cartesian); }
public ExtremeMtSolverRunner(ForwardProject project, INativeMemoryProvider memoryProvider, Mpi mpi = null) { if (project == null) { throw new ArgumentNullException(nameof(project)); } if (memoryProvider == null) { throw new ArgumentNullException(nameof(memoryProvider)); } _project = project; _memoryProvider = memoryProvider; _mpi = mpi; _logger = (_mpi == null || !_mpi.IsParallel) ? (ILogger) new ConsoleLogger() : new ParallelConsoleLogger(_mpi); _solver = new Mt3DForwardSolver(_logger, memoryProvider, _project.ForwardSettings); _solver.SetProfiler(_profiler); }
override public void initialize() { itmax = Instance.itmax; inorm = Instance.inorm; dt = Instance.dt; nz = Blocks.nz; ist = Blocks.ist; jst = Blocks.jst; iend = Blocks.iend; jend = Blocks.jend; isiz1 = Problem.isiz1; isiz2 = Problem.isiz2; u = Problem.Field_u; a = Problem.Field_a; b = Problem.Field_b; c = Problem.Field_c; d = Problem.Field_d; rsd = Problem.Field_rsd; worldcomm = Mpi.worldComm(); }
private static CartesianModel DistributedLoadModelMaster(Mpi mpi, string path) { var model = LoadModelWithoutExternFiles(path); UpdateAnomalyLocalSize(mpi, model.Anomaly); for (int k = 0; k < model.Anomaly.Layers.Count; k++) { var layer = model.Anomaly.Layers[k]; var xFromFile = layer.UnderlyingXml.Element(ModelSerializer.AnomalyFromFile); var fileType = xFromFile?.Attribute(ModelSerializer.AnomalyFileType).Value; var fileName = xFromFile?.Attribute(ModelSerializer.AnomalyFileName); if (fileType == "plain-text" && fileName != null) { var fullPath = Path.Combine(Path.GetDirectoryName(path), fileName.Value); DistributedLoadModelMaster(mpi, fullPath, model, k); } } return(model); }
public static CartesianModel LoadCartesianModel(string modelFile, Mpi mpi = null) { try { if (ModelSettingsSerializer.IsModelCommemi(modelFile)) { var settings = ModelSettingsSerializer.LoadCommemiFromXml(modelFile); return(CreateCommemiModel(settings, mpi)); } if (ModelSettingsSerializer.IsModelCommemi3D3(modelFile)) { var settings = ModelSettingsSerializer.LoadCommemi3D3FromXml(modelFile); return(CreateCommemi3D3Model(settings, mpi)); } if (ModelSettingsSerializer.IsModelOneBlock(modelFile)) { var settings = ModelSettingsSerializer.LoadOneBlockFromXml(modelFile); return(CreateOneBlockModel(settings, mpi)); } if (ModelSettingsSerializer.IsModelNaser(modelFile)) { var settings = ModelSettingsSerializer.LoadNaserFromXml(modelFile); return(CreateNaserModel(settings, mpi)); } return(SerializationManager.DistributedLoadModel(mpi, modelFile)); } catch (Exception e) { throw new InvalidOperationException($"Can't load model {modelFile}", e); } }
static void Main(string[] args) { Console.WriteLine("Starting up an MPI program!"); var dataFilePath = args[0]; // Start up MPI // We can wrap the MPI helper library in a using statement so that we can instantiate the MPI connection // and have MPI_Finalize automatically called when we are done. using (var mpi = new Mpi(args)) { // How big is the cluster, and which node are we? int worldSize = mpi.GetWorldSize(); int worldRank = mpi.GetWorldRank(); Console.WriteLine($"Rank: {worldRank}, Size: {worldSize}"); // Do a quick allreduce to show that everything works int sum = 0; mpi.AllReduce(worldRank, ref sum); Console.WriteLine($"AllReduce Rank-Sum: {sum}"); // Now let's give a little demo showing how we can cooperatively build a model. Console.WriteLine("Training a linear model...."); TrainLinearModel(dataFilePath, (uint)worldRank, out float[] biasAndWeights); var averageBiasAndWeights = new float[biasAndWeights.Length]; mpi.AllReduce(biasAndWeights, averageBiasAndWeights); for (int i = 0; i < averageBiasAndWeights.Length; i++) { averageBiasAndWeights[i] /= (float)worldSize; } Console.WriteLine($"Rank-{worldRank}: bias={biasAndWeights[0]} weight[0]={biasAndWeights[1]} | bias={averageBiasAndWeights[0]} weight[0]={averageBiasAndWeights[1]}"); } }
public CustomFftPlan CreatePlan3D(Complex *input, Complex *output, Mpi mpi, int fullNx, int fullNy, int nc) => CreatePlan(input, output, mpi, fullNx, fullNy, nc, 3);
public RealEx3DAuthRequest(string secret, string merchantId, string account, string orderId, Amount amount, Card card, TssInfo tssInfo, ThreeDSecure threeDSecure, bool autoSettle, string custNum, string prodId, string varRef, Comments comments) : base(secret, merchantId, account, orderId, amount, card, tssInfo, autoSettle, custNum, prodId, varRef, comments) { Mpi = new Mpi(threeDSecure.Cavv, threeDSecure.Xid, threeDSecure.Eci); }
public static int CalcLocalHalfNxStart(this Mpi mpi, int nx) => CalcLocalHalfNxStart(mpi, mpi.Rank, nx);
public static CartesianModel CreateCommemi3D3Model(Commemi3D3ModelSettings settings, Mpi mpi = null) { return(GenerateModel(mpi, settings, Commemi3D3ModelCreater.CreateNonMeshedModel)); }
public static CartesianModel CreateCommemiModel(CommemiModelSettings settings, Mpi mpi = null) { var creater = new SimpleCommemi3DModelCreater(settings.AnomalySizeInMeters); return(GenerateModel(mpi, settings, () => creater.CreateNonMeshedModel(settings.LeftConductivity, settings.RightConductivity))); }
private static void PrepareBuffersForModel(ModelSize ms, INativeMemoryProvider memoryProvider, Mpi mpi, IProfiler profiler) { if (Buffers.ContainsKey(ms)) { throw new InvalidOperationException("Buffer for such model size is already created"); } using (profiler?.StartAuto(ProfilerEvent.FftwPlanCalculation)) { var buffer = new FftBuffer(memoryProvider, profiler); if (IsParallel(mpi)) { buffer.AllocateBuffersAndCreatePlansParallel(ms.Nx, ms.Ny, ms.Nz, mpi); } else { buffer.AllocateBuffersAndCreatePlansLocal(ms.Nx, ms.Ny, ms.Nz); } Buffers.Add(ms, buffer); } }
private static bool IsParallel(Mpi mpi) => mpi != null && mpi.Size > 1;
public void PaymentRequestXmlSettersTest() { Card card = new Card(); card.ExpiryDate = sx.CARD_EXPIRY_DATE; card.Number = sx.CARD_NUMBER; card.Type = CardType.VISA; card.CardHolderName = sx.CARD_HOLDER_NAME; card.IssueNumber = sx.CARD_ISSUE_NUMBER; Cvn cvn = new Cvn(); cvn.Number = sx.CARD_CVN_NUMBER; cvn.PresenceIndicator = sx.CARD_CVN_PRESENCE; card.Cvn = cvn; PaymentRequest request = new PaymentRequest(); request.Account = sx.ACCOUNT; request.MerchantId = sx.MERCHANT_ID; request.Type = PaymentType.AUTH; RxpAmount amount = new RxpAmount(); amount.Amount = sx.AMOUNT; amount.Currency = sx.CURRENCY; request.Amount = amount; AutoSettle autoSettle = new AutoSettle(); autoSettle.Flag = sx.AUTO_SETTLE_FLAG; request.AutoSettle = autoSettle; request.Card = card; request.Timestamp = sx.TIMESTAMP; request.Channel = sx.CHANNEL; request.OrderId = sx.ORDER_ID; request.Hash = sx.REQUEST_HASH; List <RxpComment> comments = new List <RxpComment>(); RxpComment comment = new RxpComment(); comment.Id = 1; comment.Comment = sx.COMMENT1; comments.Add(comment); comment = new RxpComment(); comment.Id = 2; comment.Comment = sx.COMMENT2; comments.Add(comment); request.Comments = comments; request.PaymentsReference = sx.PASREF; request.AuthCode = sx.AUTH_CODE; request.RefundHash = sx.REFUND_HASH; request.FraudFilter = sx.FRAUD_FILTER; Recurring recurring = new Recurring(); recurring.Flag = sx.RECURRING_FLAG; recurring.Sequence = sx.RECURRING_SEQUENCE; recurring.Type = sx.RECURRING_TYPE; request.Recurring = recurring; TssInfo tssInfo = new TssInfo(); tssInfo.CustomerNumber = sx.CUSTOMER_NUMBER; tssInfo.ProductId = sx.PRODUCT_ID; tssInfo.VariableReference = sx.VARIABLE_REFERENCE; tssInfo.CustomerIpAddress = sx.CUSTOMER_IP; List <Address> addresses = new List <Address>(); Address address = new Address(); address.Type = sx.ADDRESS_TYPE_BUSINESS; address.Code = sx.ADDRESS_CODE_BUSINESS; address.Country = sx.ADDRESS_COUNTRY_BUSINESS; addresses.Add(address); address = new Address(); address.Type = sx.ADDRESS_TYPE_SHIPPING; address.Code = sx.ADDRESS_CODE_SHIPPING; address.Country = sx.ADDRESS_COUNTRY_SHIPPING; addresses.Add(address); tssInfo.Addresses = addresses; request.TssInfo = tssInfo; Mpi mpi = new Mpi(); mpi.Cavv = sx.THREE_D_SECURE_CAVV; mpi.Xid = sx.THREE_D_SECURE_XID; mpi.Eci = sx.THREE_D_SECURE_ECI; request.Mpi = mpi; //convert to XML string xml = request.ToXml(); //Convert from XML back to PaymentRequest PaymentRequest fromXmlRequest = new PaymentRequest().FromXml(xml); sx.checkUnmarshalledPaymentRequest(fromXmlRequest); }
public static int CalcLocalNxLength(this Mpi mpi, LateralDimensions lateral) => CalcLocalNxLength(mpi, lateral.Nx * 2);
public static int CalcLocalHalfNxLength(this Mpi mpi, int nx) => CalcLocalHalfNxLength(mpi, mpi.Rank, nx);
public static void PrepareBuffersForModel(CartesianModel model, INativeMemoryProvider memoryProvider, Mpi mpi = null, IProfiler profiler = null) { var modelSize = new ModelSize(model.Nx, model.Ny, model.Nz); PrepareBuffersForModel(modelSize, memoryProvider, mpi, profiler); }
public double readTimerGlobal(int n) { Intracommunicator comm_setup = Mpi.localComm(this); double tmax = comm_setup.Allreduce<double>(this.readTimer(n), Operation<double>.Max); return tmax; }
public static CartesianModel CreateNaserModel(NaserModelSettings settings, Mpi mpi = null) { var creater = new NaserModelCreater(settings); return(GenerateModel(mpi, settings, creater.CreateNonMeshedModel)); }