8.6 Support HOTKEYS

docs/HotKeys.md

@@ -0,0 +1,71 @@
+Hot Keys
+===
+
+The `HOTKEYS` command allows for server-side profiling of CPU and network usage by key. It is available in Redis 8.6 and later.
+
+This command is available via the `IServer.HotKeys*` methods:
+
+``` c#
+// Get the server instance.
+IConnectionMultiplexer muxer = ... // connect to Redis 8.6 or later
+var server = muxer.GetServer(endpoint); // or muxer.GetServer(key)
+
+// Start the capture; you can specify a duration, or manually use the HotKeysStop[Async] method; specifying
+// a duration is recommended, so that the profiler will not be left running in the case of failure.
+// Optional parameters allow you to specify the metrics to capture, the sample ratio, and the key slots to include;
+// by default, all metrics are captured, every command is sampled, and all key slots are included.
+await server.HotKeysStartAsync(duration: TimeSpan.FromSeconds(30));
+
+// Now either do some work ourselves, or await for some other activity to happen:
+await Task.Delay(TimeSpan.FromSeconds(35)); // whatever happens: happens 
+
+// Fetch the results; note that this does not stop the capture, and you can fetch the results multiple times
+// either while it is running, or after it has completed - but only a single capture can be active at a time.
+var result = await server.HotKeysGetAsync();
+
+// ...investigate the results... 
+
+// Optional: discard the active capture data at the server, if any.
+await server.HotKeysResetAsync();
+```
+
+The `HotKeysResult` class (our `result` value above) contains the following properties:
+
+- `Metrics`: The metrics captured during this profiling session.
+- `TrackingActive`: Indicates whether the capture currently active.
+- `SampleRatio`: Profiling frequency; effectively: measure every Nth command. (also: `IsSampled`)
+- `SelectedSlots`: The key slots active for this profiling session.
+- `CollectionStartTime`: The start time of the capture.
+- `CollectionDuration`: The duration of the capture.
+- `AllCommandsAllSlotsTime`: The total CPU time measured for all commands in all slots, without any sampling or filtering applied.
+- `AllCommandsAllSlotsNetworkBytes`: The total network usage measured for all commands in all slots, without any sampling or filtering applied.
+
+When slot filtering is used, the following properties are also available:
+
+- `AllCommandsSelectedSlotsTime`: The total CPU time measured for all commands in the selected slots.
+- `AllCommandsSelectedSlotsNetworkBytes`: The total network usage measured for all commands in the selected slots.
+
+When slot filtering *and* sampling is used, the following properties are also available:
+
+- `SampledCommandsSelectedSlotsTime`: The total CPU time measured for the sampled commands in the selected slots.
+- `SampledCommandsSelectedSlotsNetworkBytes`: The total network usage measured for the sampled commands in the selected slots.
+
+If CPU metrics were captured, the following properties are also available:
+
+- `TotalCpuTimeUser`: The total user CPU time measured in the profiling session.
+- `TotalCpuTimeSystem`: The total system CPU time measured in the profiling session.
+- `TotalCpuTime`: The total CPU time measured in the profiling session.
+- `CpuByKey`: Hot keys, as measured by CPU activity; for each:
+  - `Key`: The key observed.
+  - `Duration`: The time taken. 
+
+If network metrics were captured, the following properties are also available:
+
+- `TotalNetworkBytes`: The total network data measured in the profiling session.
+- `NetworkBytesByKey`: Hot keys, as measured by network activity; for each:
+  - `Key`: The key observed.
+  - `Bytes`: The network activity, in bytes.
+
+Note: to use slot-based filtering, you must be connected to a Redis Cluster instance. The
+`IConnectionMultiplexer.HashSlot(RedisKey)` method can be used to determine the slot for a given key. The key
+can also be used in place of an endpoint when using `GetServer(...)` to get the `IServer` instance for a given key.

docs/ReleaseNotes.md

@@ -6,12 +6,12 @@ Current package versions:
 | ------------ | ----------------- | ----- |
 | [![StackExchange.Redis](https://img.shields.io/nuget/v/StackExchange.Redis.svg)](https://www.nuget.org/packages/StackExchange.Redis/) | [![StackExchange.Redis](https://img.shields.io/nuget/vpre/StackExchange.Redis.svg)](https://www.nuget.org/packages/StackExchange.Redis/) | [![StackExchange.Redis MyGet](https://img.shields.io/myget/stackoverflow/vpre/StackExchange.Redis.svg)](https://www.myget.org/feed/stackoverflow/package/nuget/StackExchange.Redis) |
 
-## unreleased
-
+## 2.11.unreleased
 
+- Add support for `HOTKEYS` ([#3008 by mgravell](https://github.com/StackExchange/StackExchange.Redis/pull/3008))
 - Add support for keyspace notifications ([#2995 by mgravell](https://github.com/StackExchange/StackExchange.Redis/pull/2995))
+- Add support for idempotent stream entry (`XADD IDMP[AUTO]`) support  ([#3006 by mgravell](https://github.com/StackExchange/StackExchange.Redis/pull/3006))
 - (internals) split AMR out to a separate options provider ([#2986 by NickCraver and philon-msft](https://github.com/StackExchange/StackExchange.Redis/pull/2986))
-- Implement idempotent stream entry (IDMP) support  ([#3006 by mgravell](https://github.com/StackExchange/StackExchange.Redis/pull/3006))
 
 ## 2.10.14
 

docs/index.md

@@ -40,6 +40,7 @@ Documentation
 - [Events](Events) - the events available for logging / information purposes
 - [Pub/Sub Message Order](PubSubOrder) - advice on sequential and concurrent processing
 - [Pub/Sub Key Notifications](KeyspaceNotifications) - how to use keyspace and keyevent notifications
+- [Hot Keys](HotKeys) - how to use `HOTKEYS` profiling
 - [Using RESP3](Resp3) - information on using RESP3
 - [ServerMaintenanceEvent](ServerMaintenanceEvent) - how to listen and prepare for hosted server maintenance (e.g. Azure Cache for Redis)
 - [Streams](Streams) - how to use the Stream data type

src/StackExchange.Redis/ClusterConfiguration.cs

@@ -45,6 +45,11 @@ private SlotRange(short from, short to)
         /// </summary>
         public int To => to;
 
+        internal const int MinSlot = 0, MaxSlot = 16383;
+
+        private static SlotRange[]? s_SharedAllSlots;
+        internal static SlotRange[] SharedAllSlots => s_SharedAllSlots ??= [new(MinSlot, MaxSlot)];
+
         /// <summary>
         /// Indicates whether two ranges are not equal.
         /// </summary>

src/StackExchange.Redis/CommandMap.cs

@@ -41,7 +41,7 @@ public sealed class CommandMap
 
             RedisCommand.BGREWRITEAOF, RedisCommand.BGSAVE, RedisCommand.CLIENT, RedisCommand.CLUSTER, RedisCommand.CONFIG, RedisCommand.DBSIZE,
             RedisCommand.DEBUG, RedisCommand.FLUSHALL, RedisCommand.FLUSHDB, RedisCommand.INFO, RedisCommand.LASTSAVE, RedisCommand.MONITOR, RedisCommand.REPLICAOF,
-            RedisCommand.SAVE, RedisCommand.SHUTDOWN, RedisCommand.SLAVEOF, RedisCommand.SLOWLOG, RedisCommand.SYNC, RedisCommand.TIME,
+            RedisCommand.SAVE, RedisCommand.SHUTDOWN, RedisCommand.SLAVEOF, RedisCommand.SLOWLOG, RedisCommand.SYNC, RedisCommand.TIME, RedisCommand.HOTKEYS,
         });
 
         /// <summary>
@@ -65,7 +65,7 @@ public sealed class CommandMap
 
             RedisCommand.BGREWRITEAOF, RedisCommand.BGSAVE, RedisCommand.CLIENT, RedisCommand.CLUSTER, RedisCommand.CONFIG, RedisCommand.DBSIZE,
             RedisCommand.DEBUG, RedisCommand.FLUSHALL, RedisCommand.FLUSHDB, RedisCommand.INFO, RedisCommand.LASTSAVE, RedisCommand.MONITOR, RedisCommand.REPLICAOF,
-            RedisCommand.SAVE, RedisCommand.SHUTDOWN, RedisCommand.SLAVEOF, RedisCommand.SLOWLOG, RedisCommand.SYNC, RedisCommand.TIME,
+            RedisCommand.SAVE, RedisCommand.SHUTDOWN, RedisCommand.SLAVEOF, RedisCommand.SLOWLOG, RedisCommand.SYNC, RedisCommand.TIME, RedisCommand.HOTKEYS,
 
             // supported by envoy but not enabled by stack exchange
             // RedisCommand.BITFIELD,

src/StackExchange.Redis/Enums/RedisCommand.cs

@@ -81,6 +81,7 @@ internal enum RedisCommand
     HLEN,
     HMGET,
     HMSET,
+    HOTKEYS,
     HPERSIST,
     HPEXPIRE,
     HPEXPIREAT,
@@ -432,6 +433,7 @@ internal static bool IsPrimaryOnly(this RedisCommand command)
             case RedisCommand.HKEYS:
             case RedisCommand.HLEN:
             case RedisCommand.HMGET:
+            case RedisCommand.HOTKEYS:
             case RedisCommand.HPEXPIRETIME:
             case RedisCommand.HPTTL:
             case RedisCommand.HRANDFIELD:

src/StackExchange.Redis/HotKeys.ResultProcessor.cs

@@ -0,0 +1,217 @@
+namespace StackExchange.Redis;
+
+public sealed partial class HotKeysResult
+{
+    internal static readonly ResultProcessor<HotKeysResult?> Processor = new HotKeysResultProcessor();
+
+    private sealed class HotKeysResultProcessor : ResultProcessor<HotKeysResult?>
+    {
+        protected override bool SetResultCore(PhysicalConnection connection, Message message, in RawResult result)
+        {
+            if (result.IsNull)
+            {
+                SetResult(message, null);
+                return true;
+            }
+
+            // an array with a single element that *is* an array/map that is the results
+            if (result is { Resp2TypeArray: ResultType.Array, ItemsCount: 1 })
+            {
+                ref readonly RawResult inner = ref result[0];
+                if (inner is { Resp2TypeArray: ResultType.Array, IsNull: false })
+                {
+                    var hotKeys = new HotKeysResult(in inner);
+                    SetResult(message, hotKeys);
+                    return true;
+                }
+            }
+
+            return false;
+        }
+    }
+
+    private HotKeysResult(in RawResult result)
+    {
+        var metrics = HotKeysMetrics.None; // we infer this from the keys present
+        var iter = result.GetItems().GetEnumerator();
+        while (iter.MoveNext())
+        {
+            ref readonly RawResult key = ref iter.Current;
+            if (!iter.MoveNext()) break; // lies about the length!
+            ref readonly RawResult value = ref iter.Current;
+            var hash = key.Payload.Hash64();
+            long i64;
+            switch (hash)
+            {
+                case tracking_active.Hash when tracking_active.Is(hash, key):
+                    TrackingActive = value.GetBoolean();
+                    break;
+                case sample_ratio.Hash when sample_ratio.Is(hash, key) && value.TryGetInt64(out i64):
+                    SampleRatio = i64;
+                    break;
+                case selected_slots.Hash when selected_slots.Is(hash, key) & value.Resp2TypeArray is ResultType.Array:
+                    var len = value.ItemsCount;
+                    if (len == 0)
+                    {
+                        _selectedSlots = [];
+                        continue;
+                    }
+
+                    var items = value.GetItems().GetEnumerator();
+                    var slots = len == 1 ? null : new SlotRange[len];
+                    for (int i = 0; i < len && items.MoveNext(); i++)
+                    {
+                        ref readonly RawResult pair = ref items.Current;
+                        if (pair.Resp2TypeArray is ResultType.Array)
+                        {
+                            long from = -1, to = -1;
+                            switch (pair.ItemsCount)
+                            {
+                                case 1 when pair[0].TryGetInt64(out from):
+                                    to = from; // single slot
+                                    break;
+                                case 2 when pair[0].TryGetInt64(out from) && pair[1].TryGetInt64(out to):
+                                    break;
+                            }
+
+                            if (from < SlotRange.MinSlot)
+                            {
+                                // skip invalid ranges
+                            }
+                            else if (len == 1 & from == SlotRange.MinSlot & to == SlotRange.MaxSlot)
+                            {
+                                // this is the "normal" case when no slot filter was applied
+                                slots = SlotRange.SharedAllSlots; // avoid the alloc
+                            }
+                            else
+                            {
+                                slots ??= new SlotRange[len];
+                                slots[i] = new((int)from, (int)to);
+                            }
+                        }
+                    }
+                    _selectedSlots = slots;
+                    break;
+                case all_commands_all_slots_us.Hash when all_commands_all_slots_us.Is(hash, key) && value.TryGetInt64(out i64):
+                    AllCommandsAllSlotsMicroseconds = i64;
+                    break;
+                case all_commands_selected_slots_us.Hash when all_commands_selected_slots_us.Is(hash, key) && value.TryGetInt64(out i64):
+                    AllCommandSelectedSlotsMicroseconds = i64;
+                    break;
+                case sampled_command_selected_slots_us.Hash when sampled_command_selected_slots_us.Is(hash, key) && value.TryGetInt64(out i64):
+                case sampled_commands_selected_slots_us.Hash when sampled_commands_selected_slots_us.Is(hash, key) && value.TryGetInt64(out i64):
+                    SampledCommandsSelectedSlotsMicroseconds = i64;
+                    break;
+                case net_bytes_all_commands_all_slots.Hash when net_bytes_all_commands_all_slots.Is(hash, key) && value.TryGetInt64(out i64):
+                    AllCommandsAllSlotsNetworkBytes = i64;
+                    break;
+                case net_bytes_all_commands_selected_slots.Hash when net_bytes_all_commands_selected_slots.Is(hash, key) && value.TryGetInt64(out i64):
+                    NetworkBytesAllCommandsSelectedSlotsRaw = i64;
+                    break;
+                case net_bytes_sampled_commands_selected_slots.Hash when net_bytes_sampled_commands_selected_slots.Is(hash, key) && value.TryGetInt64(out i64):
+                    NetworkBytesSampledCommandsSelectedSlotsRaw = i64;
+                    break;
+                case collection_start_time_unix_ms.Hash when collection_start_time_unix_ms.Is(hash, key) && value.TryGetInt64(out i64):
+                    CollectionStartTimeUnixMilliseconds = i64;
+                    break;
+                case collection_duration_ms.Hash when collection_duration_ms.Is(hash, key) && value.TryGetInt64(out i64):
+                    CollectionDurationMicroseconds = i64 * 1000; // ms vs us is in question: support both, and abstract it from the caller
+                    break;
+                case collection_duration_us.Hash when collection_duration_us.Is(hash, key) && value.TryGetInt64(out i64):
+                    CollectionDurationMicroseconds = i64;
+                    break;
+                case total_cpu_time_sys_ms.Hash when total_cpu_time_sys_ms.Is(hash, key) && value.TryGetInt64(out i64):
+                    metrics |= HotKeysMetrics.Cpu;
+                    TotalCpuTimeSystemMicroseconds = i64 * 1000; // ms vs us is in question: support both, and abstract it from the caller
+                    break;
+                case total_cpu_time_sys_us.Hash when total_cpu_time_sys_us.Is(hash, key) && value.TryGetInt64(out i64):
+                    metrics |= HotKeysMetrics.Cpu;
+                    TotalCpuTimeSystemMicroseconds = i64;
+                    break;
+                case total_cpu_time_user_ms.Hash when total_cpu_time_user_ms.Is(hash, key) && value.TryGetInt64(out i64):
+                    metrics |= HotKeysMetrics.Cpu;
+                    TotalCpuTimeUserMicroseconds = i64 * 1000; // ms vs us is in question: support both, and abstract it from the caller
+                    break;
+                case total_cpu_time_user_us.Hash when total_cpu_time_user_us.Is(hash, key) && value.TryGetInt64(out i64):
+                    metrics |= HotKeysMetrics.Cpu;
+                    TotalCpuTimeUserMicroseconds = i64;
+                    break;
+                case total_net_bytes.Hash when total_net_bytes.Is(hash, key) && value.TryGetInt64(out i64):
+                    metrics |= HotKeysMetrics.Network;
+                    TotalNetworkBytesRaw = i64;
+                    break;
+                case by_cpu_time_us.Hash when by_cpu_time_us.Is(hash, key) & value.Resp2TypeArray is ResultType.Array:
+                    metrics |= HotKeysMetrics.Cpu;
+                    len = value.ItemsCount / 2;
+                    if (len == 0)
+                    {
+                        _cpuByKey = [];
+                        continue;
+                    }
+
+                    var cpuTime = new MetricKeyCpu[len];
+                    items = value.GetItems().GetEnumerator();
+                    for (int i = 0; i < len && items.MoveNext(); i++)
+                    {
+                        var metricKey = items.Current.AsRedisKey();
+                        if (items.MoveNext() && items.Current.TryGetInt64(out var metricValue))
+                        {
+                            cpuTime[i] = new(metricKey, metricValue);
+                        }
+                    }
+
+                    _cpuByKey = cpuTime;
+                    break;
+                case by_net_bytes.Hash when by_net_bytes.Is(hash, key) & value.Resp2TypeArray is ResultType.Array:
+                    metrics |= HotKeysMetrics.Network;
+                    len = value.ItemsCount / 2;
+                    if (len == 0)
+                    {
+                        _networkBytesByKey = [];
+                        continue;
+                    }
+
+                    var netBytes = new MetricKeyBytes[len];
+                    items = value.GetItems().GetEnumerator();
+                    for (int i = 0; i < len && items.MoveNext(); i++)
+                    {
+                        var metricKey = items.Current.AsRedisKey();
+                        if (items.MoveNext() && items.Current.TryGetInt64(out var metricValue))
+                        {
+                            netBytes[i] = new(metricKey, metricValue);
+                        }
+                    }
+
+                    _networkBytesByKey = netBytes;
+                    break;
+            } // switch
+        } // while
+        Metrics = metrics;
+    }
+
+#pragma warning disable SA1134, SA1300
+    // ReSharper disable InconsistentNaming
+    [FastHash] internal static partial class tracking_active { }
+    [FastHash] internal static partial class sample_ratio { }
+    [FastHash] internal static partial class selected_slots { }
+    [FastHash] internal static partial class all_commands_all_slots_us { }
+    [FastHash] internal static partial class all_commands_selected_slots_us { }
+    [FastHash] internal static partial class sampled_command_selected_slots_us { }
+    [FastHash] internal static partial class sampled_commands_selected_slots_us { }
+    [FastHash] internal static partial class net_bytes_all_commands_all_slots { }
+    [FastHash] internal static partial class net_bytes_all_commands_selected_slots { }
+    [FastHash] internal static partial class net_bytes_sampled_commands_selected_slots { }
+    [FastHash] internal static partial class collection_start_time_unix_ms { }
+    [FastHash] internal static partial class collection_duration_ms { }
+    [FastHash] internal static partial class collection_duration_us { }
+    [FastHash] internal static partial class total_cpu_time_user_ms { }
+    [FastHash] internal static partial class total_cpu_time_user_us { }
+    [FastHash] internal static partial class total_cpu_time_sys_ms { }
+    [FastHash] internal static partial class total_cpu_time_sys_us { }
+    [FastHash] internal static partial class total_net_bytes { }
+    [FastHash] internal static partial class by_cpu_time_us { }
+    [FastHash] internal static partial class by_net_bytes { }
+
+    // ReSharper restore InconsistentNaming
+#pragma warning restore SA1134, SA1300
+}