Blog Post: Hash field expiration in Valkey #354

ranshid · 2025-09-08T13:08:26Z

Description

Blog Post: Introducing Hash Field expiration support

Issues Resolved

closes: #326

Pending tasks

Add section for future work
Add section with example usage

By submitting this pull request, I confirm that my contribution is made under the terms of the BSD-3-Clause License.

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

Signed-off-by: Kyle J. Davis <kyledvs@amazon.com>

stockholmux

This is a very good blog post.
I left a PR on your fork with several, mostly formatting, and nits.

There area a few more expansive ones here.

content/blog/2025-09-15-hash-fields-expiration/hfe-coarse-buckets.png

content/blog/2025-09-15-hash-fields-expiration/index.md

stockholmux · 2025-09-08T20:39:10Z

content/blog/2025-09-15-hash-fields-expiration/index.md

+
+The benchmarks demonstrate that field-level expirations can be added to Valkey without compromising performance or stability.
+The memory overhead remains modest and predictable, command throughput is unaffected, and the shared active expiration job efficiently reclaims memory even under heavy ingestion workloads.
+Together, these results validate that the coarse-bucket design with adaptive encoding delivers the right balance of efficiency, scalability, and correctness, while preserving Valkey’s reputation for high performance and low latency.


Needs a call to action. What should the reader do next if it was interesting - read the code? look at the tests? just try it out?

Do you think adding a section of how to use it with examples of command usage is the right solution?

I would suggest next steps of trying out the Valkey 9 rc to try it out. Maybe like to documentation for how to use it.

Also include next steps and ask for people to help!

I agree with @madolson.

Added 2 sections describing how to examples and future plans. can you PTAL?

fixes some formatting issues

madolson

I also like the blog. I left some comments about some stuff that was confusing while reading it.

content/blog/2025-09-15-hash-fields-expiration/hfe-benchmark-hash-commands.png

content/blog/2025-09-15-hash-fields-expiration/index.md

Co-authored-by: Madelyn Olson <madelyneolson@gmail.com> Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

content/blog/2025-09-15-hash-fields-expiration/index.md

2. fix section headers Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

stockholmux

Alt text looks fine.

Nit: add a bit more to your bio and I think I'm pretty good.

content/authors/ranshid.md

1. Using Hash Field Expiration in Valkey 2. Future plans Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

stockholmux

Very small changes required.

content/blog/2025-09-15-hash-fields-expiration/index.md

content/authors/ranshid.md

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

cherukum-Amazon

Overall good looking blog post.

cherukum-Amazon · 2025-09-17T05:45:51Z

content/blog/2025-09-15-hash-fields-expiration/index.md

+![illustration](hfe-coarse-buckets.png)
+
+This solution introduces a semi-sorted data structure which we named 'vset' (stands for "volatile set").
+The vset manages buckets in different time window resolutions and alternating encodings. 


The current diagram is helpful, but it may unintentionally suggest that the TTL fields inside the hash map(in the image - content/blog/2025-09-15-hash-fields-expiration/hfe-alternative-skiplist.png) are time-ordered (since the green boxes are laid out in sequence). In reality, the hash table is unordered — TTL fields can be scattered arbitrarily. The only place where time ordering exists is in the TTL index (skiplist or coarse buckets).

To avoid confusion, I’d suggest:

Making the hash map visualization unordered (random scatter of TTL vs non-TTL fields).

Emphasizing that ordering happens exclusively in the skiplist/bucket layer.

Optionally, add a side-by-side view showing Skiplist vs Coarse Buckets, so readers understand both designs clearly.

cherukum-Amazon · 2025-09-17T05:50:09Z

content/blog/2025-09-15-hash-fields-expiration/index.md

+  This is cheap, but untouched keys can linger indefinitely and waste memory.
+
+* Active expiration – A background cron job runs 10 times per second, sampling a small set of keys with expiration. 
+  Expired keys are deleted until a time budget is reached, ensuring memory is reclaimed proactively without introducing latency spikes.


Are expirations approximate (bucketed) or exact? What’s the worst-case delay in expiring a field after its set expiration time.

I cannot provide an answer since this can be user driven. for example, running a 15 minutes LUA script will delay the expiration by at least 15 minutes...

content/blog/2025-09-15-hash-fields-expiration/index.md

cherukum-Amazon · 2025-09-17T05:52:35Z

content/blog/2025-09-15-hash-fields-expiration/index.md

+To deliver hash field expirations in a way that is simple, performant, and memory-efficient, we made some deliberate trade-offs in the initial implementation.
+As a result, there are a few limitations today that we plan to revisit in future releases:
+
+1. Expired fields are treated as if they don’t exist when accessed through commands like [HGET](/commands/hget/) or [HGETALL](/commands/hgetall/).


During the execution of MULTI/EXEC, Lua, and functions—are expirations observable mid-transaction?

Yes. transactions/scripts are no different than regular keys.

cherukum-Amazon · 2025-09-17T05:54:08Z

content/blog/2025-09-15-hash-fields-expiration/index.md

+  If you try to read or write an expired key, Valkey notices it has passed its TTL and deletes it immediately. 
+  This is cheap, but untouched keys can linger indefinitely and waste memory.
+
+* Active expiration – A background cron job runs 10 times per second, sampling a small set of keys with expiration. 


What happens if sweeps fall behind the clock(e.g., high CPU)? Is there a catch-up strategy or rate limit applied?

There are some ways the job is accelerated when it lags behind. again this is not new to the HFE (as it shares the old mechanism) and I am not sure going into this level of details will do good. for example the job has 2 run modes:
Fast and Slow. the fast mode is running as part of the beforeSleep step which happens every time the server complete writing response to clients. The fast path is something that is managed in terms of CPU cycles.

cherukum-Amazon · 2025-09-17T05:55:40Z

content/blog/2025-09-15-hash-fields-expiration/index.md

+   In practice, this can result in [HRANDFIELD](/commands/hrandfield/) returning an empty reply more often than expected, and it reduces the fairness of field sampling.
+3. Valkey normally uses a highly compact representation for small hashes, which saves significant memory.
+   To simplify the expiration design in Valkey 9.0, we chose to disable this compact encoding when a hash contains volatile fields (i.e., fields with TTLs).
+   The impact is limited: only small hashes with field expirations are affected, but in those cases memory usage will be slightly higher than the fully compacted representation.


Do we provide metrics for observability - active buckets, members per encoding, sweeps/sec, fields expired/sec, avg/percentile sweep latency, skipped work due to budget.

In 9.0 the statistics added are minimal. we wanted to gather some requirements around observability, so we basically added what Redis did (which is not much). I can mention this in the blog, but it will make it look like not much (which is true). I prefer focusing on the algorithm, rather than on observability. I also want to make room for the followup blog when we will improve the active expiration using (I hope) Hierarchal timer wheel.

content/blog/2025-09-15-hash-fields-expiration/index.md

stockholmux

Two small changes requested.

content/authors/ranshid.md

content/blog/2025-09-15-hash-fields-expiration/index.md

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

stockholmux

LGTM

stockholmux · 2025-09-22T13:13:46Z

@ranshid I think we're pretty good on this post. According the tracker we're supposed to cluster db post up before this one, but I'd like to love this to publish stage tomorrow (Sept 23). Can you change the publish date in index.md frontmatter and the directory to 2025-09-23?

madolson · 2025-09-24T20:19:13Z

I have no objections, but didn't look closely enough to press ship it. @stockholmux Ran is out on holidays for a bit, you might just update and push it.

ranshid · 2025-09-25T06:45:43Z

@ranshid I think we're pretty good on this post. According the tracker we're supposed to cluster db post up before this one, but I'd like to love this to publish stage tomorrow (Sept 23). Can you change the publish date in index.md frontmatter and the directory to 2025-09-23?

Sorry. was out on holidays... Changing to 2025-09-25?

ranshid · 2025-09-25T06:46:14Z

content/blog/2025-09-15-hash-fields-expiration/index.md

@@ -0,0 +1,333 @@
+++
+title= "Introducing Hash Field Expirations"
+date= 2025-09-15 00:00:00


Suggested change

date= 2025-09-15 00:00:00

date= 2025-09-25 00:00:00

I went ahead with the Numbered DB post - let's schedule yours for Tuesday the 30th. Please change both the publish date here and the directory.

Hash field expiration in Valkey

8926905

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

ranshid requested review from madolson and stockholmux as code owners September 8, 2025 13:08

ranshid removed the request for review from stockholmux September 8, 2025 13:08

ranshid changed the title ~~Hash field expiration in Valkey~~ Blog Post: Hash field expiration in Valkey Sep 8, 2025

ranshid requested a review from stockholmux September 8, 2025 13:09

fix some headers sizing

204cb6b

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

ranshid force-pushed the hfe-blog-post branch from 5994598 to 204cb6b Compare September 8, 2025 13:16

ranshid and others added 3 commits September 8, 2025 16:34

fix broken hashtable link

c8408f7

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

fix some wrong comment

e58e6b4

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

fixes some formatting issues

7a23055

Signed-off-by: Kyle J. Davis <kyledvs@amazon.com>

stockholmux requested changes Sep 8, 2025

View reviewed changes

Merge pull request #1 from stockholmux/hfe-blog-update

b2cde34

fixes some formatting issues

madolson reviewed Sep 9, 2025

View reviewed changes

ranshid and others added 6 commits September 9, 2025 08:52

Update content/blog/2025-09-15-hash-fields-expiration/index.md

c3cdbb0

Co-authored-by: Madelyn Olson <madelyneolson@gmail.com> Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

change most vset to volatile set

8882a0e

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

fix image for coarse buckets

b39609e

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

remove the known limitations section

3403195

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

add CPU to the reclaim time chart

c1b4131

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

add data about the CPU time in the expiration time chart

964cd3b

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

stockholmux requested changes Sep 9, 2025

View reviewed changes

ranshid added 3 commits September 9, 2025 20:49

1. reduce dimension of charts

ee0b609

2. fix section headers Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

modify some of the messages.

15d9444

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

add images alt-text

6278fa6

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

stockholmux requested changes Sep 9, 2025

View reviewed changes

content/authors/ranshid.md Outdated Show resolved Hide resolved

add sections for:

0ebc5f9

1. Using Hash Field Expiration in Valkey 2. Future plans Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

ranshid self-assigned this Sep 10, 2025

stockholmux requested changes Sep 12, 2025

View reviewed changes

ranshid added 2 commits September 14, 2025 10:06

add info to bio

1c6a744

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

address PR comments

5d92351

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

cherukum-Amazon reviewed Sep 17, 2025

View reviewed changes

stockholmux requested changes Sep 18, 2025

View reviewed changes

content/authors/ranshid.md Outdated Show resolved Hide resolved

content/blog/2025-09-15-hash-fields-expiration/index.md Outdated Show resolved Hide resolved

address PR comments

32a1330

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>

stockholmux approved these changes Sep 22, 2025

View reviewed changes

ranshid commented Sep 25, 2025

View reviewed changes

Blog Post: Hash field expiration in Valkey #354

Are you sure you want to change the base?

Blog Post: Hash field expiration in Valkey #354

Uh oh!

Conversation

ranshid commented Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issues Resolved

Pending tasks

Uh oh!

stockholmux left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

madolson left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stockholmux left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

stockholmux left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cherukum-Amazon left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ranshid commented Sep 8, 2025 •

edited

Loading

madolson left a comment •

edited

Loading