public void Should_RemoveNodeMetricsAndDisposeMetricsContext_When_HostIsRemoved() { _metricsRoot = new MetricsBuilder().Build(); var cluster = GetNewTemporaryCluster(b => b.WithMetrics(_metricsRoot.CreateDriverMetricsProvider())); var session = cluster.Connect(); var metrics = session.GetMetrics(); Assert.AreEqual(3, cluster.Metadata.Hosts.Count); Assert.AreEqual(3, metrics.NodeMetrics.Count); // get address for host that will be removed from the cluster in this test var address = TestCluster.ClusterIpPrefix + "2"; var hostToBeRemoved = cluster.Metadata.Hosts.First(h => h.Address.Address.Equals(IPAddress.Parse(address))); // check node metrics are valid var gauge = metrics.GetNodeGauge(hostToBeRemoved, NodeMetric.Gauges.OpenConnections); var appMetricsGaugeValue = _metricsRoot.Snapshot.GetGaugeValue(gauge.Context, gauge.Name); Assert.Greater(gauge.GetValue().Value, 0); Assert.AreEqual(gauge.GetValue().Value, appMetricsGaugeValue); // check node metrics context in app metrics is valid var context = _metricsRoot.Snapshot.GetForContext(gauge.Context); Assert.True(context.IsNotEmpty()); Assert.AreEqual(2, context.Gauges.Count()); // remove host from cluster if (!TestClusterManager.SupportsDecommissionForcefully()) { TestCluster.DecommissionNode(2); } else { TestCluster.DecommissionNodeForcefully(2); } TestCluster.Stop(2); try { TestHelper.RetryAssert(() => { Assert.AreEqual(2, cluster.Metadata.Hosts.Count, "metadata hosts count failed"); }, 200, 50); TestHelper.RetryAssert(() => { Assert.AreEqual(2, metrics.NodeMetrics.Count, "Node metrics count failed"); }, 10, 500); } catch { TestCluster.Start(2, "--jvm_arg=\"-Dcassandra.override_decommission=true\""); throw; } // Check node's metrics were removed from app metrics registry context = _metricsRoot.Snapshot.GetForContext(gauge.Context); Assert.False(context.IsNotEmpty()); Assert.AreEqual(0, context.Gauges.Count()); TestCluster.Start(2, "--jvm_arg=\"-Dcassandra.override_decommission=true\""); TestHelper.RetryAssert(() => { Assert.AreEqual(3, cluster.Metadata.Hosts.Count, "metadata hosts count after bootstrap failed"); }, 200, 50); // when new host is chosen by LBP, connection pool is created foreach (var _ in Enumerable.Range(0, 5)) { session.Execute("SELECT * FROM system.local"); } TestHelper.RetryAssert(() => { Assert.AreEqual(3, metrics.NodeMetrics.Count, "Node metrics count after bootstrap failed"); }, 10, 500); // Check node's metrics were added again context = _metricsRoot.Snapshot.GetForContext(gauge.Context); Assert.True(context.IsNotEmpty()); Assert.AreEqual(2, context.Gauges.Count()); }