private static void TestSet( Action <string> log, int numberOfThreads, RingMasterBackendCore backendCore, ClientSession clientSession) { var sw = new Stopwatch(); var data = new byte[PayloadLength]; var countPerThread = TotalNodeCount / numberOfThreads; var threads = Enumerable.Range(0, numberOfThreads) .Select(n => new Thread(() => { for (int i = 0; i < countPerThread; i++) { var setReq = new RequestSetData($"/test/{n}/{i}", null, data, -1, null); backendCore.ProcessMessage(setReq, clientSession, null); } })) .ToArray(); int gen0 = GC.CollectionCount(0), gen1 = GC.CollectionCount(1), gen2 = GC.CollectionCount(2); sw.Start(); Parallel.ForEach(threads, t => t.Start()); Parallel.ForEach(threads, t => t.Join()); sw.Stop(); var rate = countPerThread * numberOfThreads / sw.Elapsed.TotalSeconds; log($"Update nodes: {sw.Elapsed} QPS={rate:G3}"); log($" Gen0={GC.CollectionCount(0) - gen0} Gen1={GC.CollectionCount(1) - gen1} Gen2={GC.CollectionCount(2) - gen2}\n"); }
/// <summary> /// Creates a transaction request which contains only one book keeping request. /// </summary> /// <param name="transactionId">Id of the transaction</param> /// <returns>A <see cref="RequestMulti"/> that contains just one book keeping request</returns> private static RequestMulti CreateTransaction(ulong transactionId) { var recordTransactionIdRequest = new RequestSetData(TestExecutionQueue.LastAppliedTransactionIdPath, data: BitConverter.GetBytes(transactionId), version: -1); var multiRequest = new RequestMulti(new IRingMasterRequest[] { recordTransactionIdRequest }, completeSynchronously: true); ulong transactionTime = (ulong)DateTime.UtcNow.ToFileTimeUtc(); multiRequest.Overrides = new OperationOverrides() { TxTime = transactionTime, TxId = transactionId }; return(multiRequest); }
/// <summary> /// Create a batch request which contains the given requests and book keeping requests. /// </summary> /// <param name="batchId">Id of the batch</param> /// <param name="requests">Requests to include in the batch</param> /// <returns>A <see cref="RequestBatch"/> that contains the given requests and book keeping requests</returns> private static RequestBatch CreateBatch(ulong batchId, params IRingMasterRequest[] requests) { var requestArray = new IRingMasterRequest[2 + requests.Length]; requestArray[0] = new RequestSetData(TestExecutionQueue.LastAppliedBatchIdPath, data: BitConverter.GetBytes(batchId), version: -1); requests.CopyTo(requestArray, 1); requestArray[requestArray.Length - 1] = new RequestGetData(TestExecutionQueue.TransactionCrcPath, options: RequestGetData.GetDataOptions.None, watcher: null); var batchRequest = new RequestBatch(requestArray, completeSynchronously: true); batchRequest.Uid = batchId; batchRequest.ExecutionQueueId = TransactionManagerExecutionQueueId; batchRequest.ExecutionQueueTimeoutMillis = 10000; return(batchRequest); }
private void IssueRequest(IRingMasterRequestHandler ringMaster, string nodePath, byte[] nodeData, CountdownEvent operationsCompletedEvent) { var setDataRequest = new RequestSetData(nodePath, nodeData, -1); this.semaphore.Wait(); operationsCompletedEvent.AddCount(); var timer = Stopwatch.StartNew(); ringMaster.Request(setDataRequest).ContinueWith(responseTask => { try { this.semaphore.Release(); timer.Stop(); RequestResponse response = responseTask.Result; if (response.ResultCode == (int)RingMasterException.Code.Ok) { this.instrumentation?.SetDataMultiSucceeded(1, timer.Elapsed); } else { this.instrumentation?.SetDataMultiFailed(1); } } catch (Exception) { this.instrumentation?.SetDataMultiFailed(1); } finally { operationsCompletedEvent.Signal(); } }); }
/// <summary> /// Deserialize <see cref="IZooKeeperRequest"/> /// </summary> /// <param name="xid">the callid</param> /// <param name="type">type of the call</param> /// <param name="sessionState">The PerSession State</param> /// <param name="ringMasterRequest">The ring master request.</param> /// <exception cref="System.ArgumentException">unknown type + type</exception> /// <returns>The Zookeeper Request</returns> private IZooKeeperRequest DeserializeZooKeeperRequest(int xid, ZooKeeperRequestType type, ZkprPerSessionState sessionState, out IRingMasterRequest ringMasterRequest) { ringMasterRequest = null; switch (type) { case ZooKeeperRequestType.Notification: // "0" for Createing a session ringMasterRequest = null; return(this.DeserializeNotification(xid)); case ZooKeeperRequestType.CreateSession: ZkprProtocolMessages.CreateSession cs = this.DeserializeCreateSession(); ringMasterRequest = new RequestInit((ulong)cs.SessionId, cs.IsNullPassword ? string.Empty : cs.Password); sessionState.ConnectRecieved = true; return(cs); case ZooKeeperRequestType.Exists: ZkprProtocolMessages.Exists ex = this.DeserializeExists(xid); ringMasterRequest = new RequestExists(ex.Path, ex.Watch == false ? null : new Watcher((ulong)xid, WatcherKind.OneUse)); return(ex); case ZooKeeperRequestType.GetChildren: ZkprProtocolMessages.GetChildren gc = this.DeserializeGetChildren(xid); ringMasterRequest = new RequestGetChildren(gc.Path, gc.Watch == false ? null : new Watcher((ulong)xid, WatcherKind.OneUse), null); return(gc); case ZooKeeperRequestType.GetChildren2: ZkprProtocolMessages.GetChildren2 gc2 = this.DeserializeGetChildren2(xid); ringMasterRequest = new RequestGetChildren(gc2.Path, gc2.Watch == false ? null : new Watcher((ulong)xid, WatcherKind.OneUse), null); return(gc2); case ZooKeeperRequestType.GetData: ZkprProtocolMessages.GetData gd = this.DeserializeGetData(xid); ringMasterRequest = new RequestGetData(gd.Path, RequestGetData.GetDataOptions.None, gd.Watch == false ? null : new Watcher((ulong)xid, WatcherKind.OneUse)); return(gd); case ZooKeeperRequestType.Create: ZkprProtocolMessages.Create cr = this.DeserializeCreate(xid); IReadOnlyList <Acl> acls = this.TranslateZkprAclListToRMAclList(cr.Acls); CreateMode cm = this.TranslateZkprCreatFlagsToRmCreateMode(cr.Flags); ringMasterRequest = new RequestCreate(cr.Path, cr.Data, acls, cm); return(cr); case ZooKeeperRequestType.Create2: ZkprProtocolMessages.Create2 cr2 = this.DeserializeCreate2(xid); IReadOnlyList <Acl> acls2 = this.TranslateZkprAclListToRMAclList(cr2.Acls); CreateMode cm2 = this.TranslateZkprCreatFlagsToRmCreateMode(cr2.Flags); ringMasterRequest = new RequestCreate(cr2.Path, cr2.Data, acls2, cm2); return(cr2); case ZooKeeperRequestType.SetData: ZkprProtocolMessages.SetData sd = this.DeserializeSetData(xid); ringMasterRequest = new RequestSetData(sd.Path, sd.Data, sd.Version); return(sd); case ZooKeeperRequestType.Delete: ZkprProtocolMessages.Delete dl = this.DeserializeDelete(xid); ringMasterRequest = new RequestDelete(dl.Path, dl.Version, false); return(dl); case ZooKeeperRequestType.Ping: ringMasterRequest = null; return(this.DeserializePing(xid)); case ZooKeeperRequestType.CloseSession: ringMasterRequest = null; sessionState.ConnectRecieved = false; // Renegotiate the CreateSession return(this.DeserializeCloseSession(xid)); case ZooKeeperRequestType.GetACL: ZkprProtocolMessages.GetACL ga = this.DeserializeGetACL(xid); ringMasterRequest = new RequestGetAcl(ga.Path, null); return(ga); case ZooKeeperRequestType.SetACL: ZkprProtocolMessages.SetACL sa = this.DeserializeSetACL(xid); IReadOnlyList <Acl> sa_acls = this.TranslateZkprAclListToRMAclList(sa.Acls); ringMasterRequest = new RequestSetAcl(sa.Path, sa_acls, sa.Version); return(sa); case ZooKeeperRequestType.Multi: ZkprProtocolMessages.Multi mu = this.DeserializeMulti(xid); IReadOnlyList <Op> rmOps = this.TranslateZkprOpsListToRmOpsList(mu.Ops); ringMasterRequest = new RequestMulti(rmOps, false); return(mu); case ZooKeeperRequestType.Auth: ZkprProtocolMessages.Auth au = this.DeserializeAuth(xid); ringMasterRequest = new RequestSetAuth(au.RmAuthId); return(au); case ZooKeeperRequestType.Check: case ZooKeeperRequestType.Sync: case ZooKeeperRequestType.Reconfig: case ZooKeeperRequestType.SetWatches: case ZooKeeperRequestType.RemoveWatches: case ZooKeeperRequestType.CreateContainer: case ZooKeeperRequestType.DeleteContainer: case ZooKeeperRequestType.Sasl: case ZooKeeperRequestType.Error: default: break; } return(null); }
/// <summary> /// Serialize <see cref="RequestSetData"/>. /// </summary> /// <param name="request">Request to serialize</param> private void SerializeRequestSetData(RequestSetData request) { this.binaryWriter.Write((int)request.Version); this.SerializeData((byte[])request.Data); this.binaryWriter.Write((bool)request.IsDataCommand); }