default LBP: warn on preferred DC misconfiguration

DefaultPolicy now issues respective warnings:
1. if the preferred datacenter is not present in the cluster;
2. if all nodes in the preferred datacenter are disabled by
   the HostFilter.

This helps avoid confusion when the user expects requests to be sent
to a specific datacenter, but it is not available.

As warnings are going to be emitted on every request (because
the configuration is most likely session-wide), the messages will
quickly flood the logs and the user should notice.
Alternatively, we could implement some frequency limiting of those
warnings, but we rather prefer to:
- keep it simple for now,
- let the user notice the misconfiguration as quickly as possible.

Author: wprzytula

scylla/src/policies/load_balancing/default.rs

@@ -589,8 +589,9 @@ impl DefaultPolicy {
     }
 
     /// Checks for misconfiguration and warns if any is discovered.
-    fn pick_sanity_checks(&self, routing_info: &ProcessedRoutingInfo, _cluster: &ClusterState) {
-        if let Some(_preferred_dc) = self.preferences.datacenter() {
+    fn pick_sanity_checks(&self, routing_info: &ProcessedRoutingInfo, cluster: &ClusterState) {
+        if let Some(preferred_dc) = self.preferences.datacenter() {
+            // Preferred DC + no datacenter failover + SimpleStrategy is an anti-pattern.
             if let Some(ref token_with_strategy) = routing_info.token_with_strategy {
                 if !self.permit_dc_failover
                     && matches!(
@@ -606,6 +607,55 @@ or refrain from preferring datacenters (which may ban all other datacenters, if
                     );
                 }
             }
+
+            // Verify that the preferred datacenter is actually present in the cluster.
+            // If not, shout a warning.
+            if !cluster.all_nodes.iter().any(|node| {
+                node.datacenter
+                    .as_deref()
+                    .is_some_and(|dc| dc == preferred_dc)
+            }) {
+                if self.permit_dc_failover {
+                    warn!(
+                        "\
+The preferred datacenter (\"{preferred_dc}\") is not present in the cluster! \
+Datacenter failover is enabled, so the request will be always sent to remote DCs. \
+This is most likely not what you want!"
+                    );
+                } else {
+                    warn!(
+                        "\
+The preferred datacenter (\"{preferred_dc}\") is not present in the cluster! \
+Datacenter failover is disabled, so the query plans will be empty! \
+You won't be able to execute any requests!"
+                    );
+                }
+            }
+            // Verify that there exist any enabled nodes in the preferred datacenter.
+            // If not, shout a warning.
+            else if !cluster.all_nodes.iter().any(|node| {
+                node.is_enabled()
+                    && node
+                        .datacenter
+                        .as_deref()
+                        .is_some_and(|dc| dc == preferred_dc)
+            }) {
+                if self.permit_dc_failover {
+                    warn!(
+                        "\
+All nodes in the preferred datacenter (\"{preferred_dc}\") are disabled by the HostFilter! \
+Datacenter failover is enabled, so the request will be always sent to remote DCs. \
+This is most likely a misconfiguration!"
+                    );
+                } else {
+                    warn!(
+                        "\
+All nodes in the preferred datacenter (\"{preferred_dc}\") are disabled by the HostFilter! \
+Datacenter failover is disabled, so the query plans will be empty! \
+You won't be able to execute any requests! This is most likely a misconfiguration!"
+                    );
+                }
+            }
         }
     }