public static ReplicatedTableQuorumWriteResult TryWriteBlobQuorum <T>(List <CloudBlockBlob> blobs, T configuration, Func <string, T> ParseBlobFunc, Func <T, T, bool> ConfigIdComparer, Func <T, T> GenerateConfigId) where T : ReplicatedTableConfigurationBase, new() { // Fetch all blobs ... List <T> valuesArray; List <string> eTagsArray; List <ReplicatedTableReadBlobResult> resultArray = TryReadAllBlobs(blobs, out valuesArray, out eTagsArray, ParseBlobFunc); // Find majority ... int quorumIndex; ReplicatedTableQuorumReadResult majority = FindMajority(resultArray, valuesArray, out quorumIndex); string readDetails = majority.ToString(); switch (majority.Code) { case ReplicatedTableQuorumReadCode.NotFound: // Create blobs ... break; case ReplicatedTableQuorumReadCode.UpdateInProgress: return(new ReplicatedTableQuorumWriteResult(ReplicatedTableQuorumWriteCode.ConflictDueToUpdateInProgress, readDetails)); case ReplicatedTableQuorumReadCode.Exception: return(new ReplicatedTableQuorumWriteResult(ReplicatedTableQuorumWriteCode.ReadExceptions, readDetails)); case ReplicatedTableQuorumReadCode.NullOrLowSuccessRate: return(new ReplicatedTableQuorumWriteResult(ReplicatedTableQuorumWriteCode.ReadNoQuorum, readDetails)); case ReplicatedTableQuorumReadCode.Success: // Blob has changed since ... if (ConfigIdComparer(valuesArray[quorumIndex], configuration) == false) { return(new ReplicatedTableQuorumWriteResult(ReplicatedTableQuorumWriteCode.ConflictDueToBlobChange, readDetails)); } // Update blobs ... break; case ReplicatedTableQuorumReadCode.BlobsNotInSyncOrTransitioning: return(new ReplicatedTableQuorumWriteResult(ReplicatedTableQuorumWriteCode.BlobsNotInSyncOrTransitioning, readDetails)); default: { var msg = string.Format("Unexpected value majority=\'{0}\' ", majority.Code); throw new Exception(msg); } } // Generate a new Id for the copy of the input configuration T newConfiguration = GenerateConfigId(configuration); string content = newConfiguration.ToString(); // Update blobs ... int numberOfBlobs = blobs.Count; ReplicatedTableWriteBlobResult[] writeResultArray = new ReplicatedTableWriteBlobResult[numberOfBlobs]; Parallel.For(0, numberOfBlobs, index => { T currentValue = valuesArray[index]; string currentETag = eTagsArray[index]; if (currentValue == null) { currentETag = null; } else if (!ConfigIdComparer(currentValue, configuration)) { currentETag = "*"; } writeResultArray[index] = TryWriteBlob(blobs[index], content, currentETag); }); int successRate = writeResultArray.Count(e => e.Success == true); int quorum = (numberOfBlobs / 2) + 1; if (successRate >= quorum) { // Return new config Id to the caller for record string newConfId = newConfiguration.GetConfigId().ToString(); return(new ReplicatedTableQuorumWriteResult(ReplicatedTableQuorumWriteCode.Success, newConfId, writeResultArray.ToList())); } return(new ReplicatedTableQuorumWriteResult(ReplicatedTableQuorumWriteCode.QuorumWriteFailure, writeResultArray.ToList())); }
public void TurnReplicaOn(string storageAccountName, List <string> tablesToRepair, out List <ReplicatedTableRepairResult> failures) { if (string.IsNullOrEmpty(storageAccountName)) { throw new ArgumentNullException("storageAccountName"); } if (tablesToRepair == null) { throw new ArgumentNullException("tablesToRepair"); } ReplicatedTableConfiguration configuration = null; failures = new List <ReplicatedTableRepairResult>(); // - Retrieve configuration ... ReplicatedTableQuorumReadResult readResult = RetrieveConfiguration(out configuration); if (readResult.Code != ReplicatedTableQuorumReadCode.Success) { var msg = string.Format("TurnReplicaOn={0}: failed to read configuration, \n{1}", storageAccountName, readResult.ToString()); ReplicatedTableLogger.LogError(msg); throw new Exception(msg); } /* - Phase 1: * Move the *replica* to the front and set it to None. * Make the view ReadOnly. **/ #region Phase 1 configuration.MoveReplicaToHeadAndSetViewToReadOnly(storageAccountName); // - Write back configuration, refresh its Id with the new one, // but don't validate it is loaded bcz if all views of the config are empty, the config won't be refreshed by RefreshReadAndWriteViewsFromBlobs() thread! SaveConfigAndRefreshItsIdAndValidateIsLoaded(configuration, "Phase 1", false); #endregion /** * Chain is such: [None] -> [RO] -> ... -> [RO] * or: [None] -> [None] -> ... -> [None] **/ // - Wait for L + CF to make sure no pending transaction working on old views Thread.Sleep(TimeSpan.FromSeconds(Constants.LeaseDurationInSec + Constants.ClockFactorInSec)); /* - Phase 2: * Set the *replica* (head) to WriteOnly and other active replicas to ReadWrite. * Or, in case of one replic chain * Set only the *replica* (head) to ReadWrite. **/ #region Phase 2 configuration.EnableWriteOnReplicas(storageAccountName); // - Write back configuration, refresh its Id with the new one, // and then validate it is loaded now (it has to be working since next Phase is "Repair") SaveConfigAndRefreshItsIdAndValidateIsLoaded(configuration, "Phase 2"); #endregion /** * Chain is such: [W] -> [RW] -> ... -> [RW] * or: [RW] -> [None] -> ... -> [None] **/ // To be safe: // - Wait for L + CF to make sure no pending transaction working on old views Thread.Sleep(TimeSpan.FromSeconds(Constants.LeaseDurationInSec + Constants.ClockFactorInSec)); /* - Phase 3: * Repair all tables **/ #region Phase 3 foreach (var tableName in tablesToRepair) { ReplicatedTableRepairResult result = RepairTable(tableName, storageAccountName, configuration); if (result.Code != ReplicatedTableRepairCode.Error) { ReplicatedTableLogger.LogInformational(result.ToString()); continue; } ReplicatedTableLogger.LogError(result.ToString()); // List of tables (and corresponding views) failed to repair! failures.Add(result); } #endregion /* - Phase 4: * Set the *replica* (head) to ReadWrite. **/ #region Phase 4 // TODO: re-evaluate if we will support this API for partitioned tables ? configuration.EnableReadWriteOnReplicas(storageAccountName, failures.Select(r => r.ViewName).ToList()); // - Write back configuration, refresh its Id with the new one, // and then validate it is loaded now (i.e. it is a working config) SaveConfigAndRefreshItsIdAndValidateIsLoaded(configuration, "Phase 4"); #endregion /** * Chain is such: [RW] -> [RW] -> ... -> [RW] if all configured table repaired * or: [W] -> [RW] -> ... -> [RW] if at least one configured table failed repair ! **/ }