Beyond Speed Limits: Exploring the Performance Power of Valkey

Most performance work on a read-heavy application eventually hits the same wall: the source-of-truth database answers in milliseconds, and no amount of vertical or horizontal scaling makes a disk-backed relational engine answer in microseconds. Viktor Vedmich's central claim is that an in-memory data store is the standard answer to that wall, and that Valkey is worth learning not because caching is novel but because its data structures turn several distinct problems — sessions, leaderboards, feature serving, rate limiting, cardinality counting — into small amounts of code against one engine.

Vedmich is a Senior Solutions Architect at AWS. He gave this 49-minute, 39-second talk at InfoQ Dev Summit Munich; InfoQ published the recording and transcript on June 8, 2026. He speaks explicitly as an AWS employee, and many slides show Amazon ElastiCache rather than Valkey directly, because ElastiCache is the managed service in which he runs Valkey as the engine. Where he says "we recommend," he means AWS guidance rather than a vendor-neutral consensus, and these notes keep that attribution.

What You Will Learn

• What Valkey is, where it came from, and what its Redis compatibility guarantee means in practice.
• How lazy loading, write-through, and hybrid invalidation differ, and when each is appropriate.
• How to detect a soon-to-expire hot key and how to solve the thundering-herd problem with a distributed lock.
• How client-side caching works, including the two invalidation-tracking designs and why connection pooling matters.
• Which Valkey data structure fits sessions, online feature serving, leaderboards, unique-visitor counting, and rate limiting, and the complexity of each.
• How to run server-side Lua with SCRIPT LOAD and EVALSHA, and why rate limiting belongs there.
• The operational rules Vedmich recommends for TTL, jitter, eviction policy, and cache sizing.

Why an In-Memory Layer at All

The talk opens with the one number that motivates everything else: reading from memory is roughly 20 times faster than reading from disk, and Vedmich is explicit that this holds even against NVMe rather than only against spinning disks. That ratio is the entire economic argument for a cache tier, and it also sets up the corresponding cost warning he returns to at the end — memory is 20 times faster but also more expensive per byte than disk.

He frames the problem with a running example used through the whole talk: you have joined a promising startup, the "Night Sky Marketplace," which sells human-made (explicitly not GenAI-generated) astrophotography images. The initial architecture is EC2 instances talking to Amazon RDS, chosen because a relational database is the easy way to start. Growth then exposes the limits:

• Vertical scaling (more CPU and memory on the RDS instance) works, but has a ceiling.
• Read replicas scale reads horizontally, but you will still reach a performance limit.
• Regardless of instance size, a disk-backed relational database answers in milliseconds. Complex queries that join several tables can take considerably longer.

His conclusion is that once a query is expensive and its result is reused, the better move is to store the result of execution in Valkey rather than to keep buying more database capacity. Two benefits follow. The obvious one is sub-millisecond read latency for the application. The less obvious one is cost: because reads move off the origin, you may be able to remove read replicas entirely, so the cache tier partially pays for itself.

What Valkey Is

Valkey is an open-source in-memory data store, created as a fork of Redis in 2024 after Redis moved its licensing away from open source. Vedmich characterises the situation as political and does not dwell on it, but two engineering consequences matter:

• API compatibility. Because Valkey is based on the Redis codebase, he states it is 100% compatible at the API level, so migrating an existing Redis application is intended to require no application-code changes. Practically, this means existing Redis client libraries, command vocabulary, and data structures carry over.
• Governance. Valkey sits under the Linux Foundation and CNCF umbrella. His argument for developers is that this is a structural guarantee that the project stays open source long-term, rather than a promise from a single vendor.

On contribution, he reports roughly 40 organisations and 150 contributors, with AWS as a major but not sole contributor. The contributions he highlights are performance and memory: AWS-driven work reached 1 million requests per second using threading, which he describes as more than a 2x performance improvement, alongside reduced memory consumption. Treat these as the speaker's reported project benchmarks rather than as guarantees for your workload; he does not state the instance type, payload size, or benchmark methodology behind the figure.

He also introduces Amazon ElastiCache early, because his slides show it throughout. ElastiCache is AWS's managed in-memory caching service; Valkey is one of the engines it can run. The managed service supplies availability and easy scaling; the data structures and API discussed in the talk are Valkey's, and apply equally to a self-hosted deployment.

A framing point worth carrying through the rest of the notes: Vedmich observes that the use cases for an in-memory store look unrelated on the surface — simple caching, leaderboards, machine learning feature serving — but the thing they share is the underlying data structures and API. Learning the data structures is what makes all of them cheap to build.

Caching Strategies

Lazy Loading

The simplest strategy, also known as cache-aside. The application asks Valkey for an item first. On a hit, the response takes on the order of hundreds of microseconds. On a miss, the application falls back to the source of truth — RDS or whatever the origin is — which costs roughly 10 milliseconds, depending on database configuration and provisioned resources. The application then writes the fetched object into the cache so that subsequent reads are fast.

The strength of lazy loading is that only data that is actually requested ever occupies memory. Its weakness is that the first request after a miss pays the full origin cost, and there is no mechanism to keep the cache correct when the origin changes.

Write-Through

The inverse: when a new item is created, the application writes it to RDS and simultaneously writes it to the cache. Subsequent reads are served from the cache without ever paying a miss. The strength is freshness and no cold-read penalty; the weakness is that you cache data nobody may read, and every write path must remember to do both writes.

Combining Both, With Asynchronous Invalidation

Vedmich's point is that neither strategy is universally better, so his practical design combines them. Reads use lazy loading. Writes use write-through semantics to drive invalidation, which is the problem lazy loading alone cannot solve.

His concrete example uses DynamoDB rather than RDS as the origin, and he flags this substitution explicitly: he needs a specific database capability, namely change streams. Suppose the price of a popular image changes. The application writes the new price to DynamoDB. DynamoDB's internal change mechanism triggers a Lambda function, which asynchronously invalidates the corresponding key in Valkey. The next reader either receives a freshly written correct value or misses and re-reads DynamoDB.

The engineering property that makes this attractive is that the write path does not block on cache invalidation, and that invalidation is driven by the database's own record of what changed rather than by every service that might write. The cost is that invalidation is eventually consistent: there is a window between the origin write and the cache delete during which readers see a stale value. That window is acceptable for a product price and would not be acceptable for, say, an authorization decision.

He also notes that Valkey is a binary engine: anything you can serialise, you can store. His example is caching S3 objects in Valkey instead of repeatedly fetching large objects from S3, which saves both latency and S3 request cost. The implicit constraint is memory — large blobs consume the most expensive storage tier you own — so this is a technique for hot objects, not for a general S3 mirror.

Refreshing Hot Keys Before They Expire

Before addressing the full thundering-herd problem, Vedmich shows a lighter preventive technique for known hot keys. Clients issue a MULTI transaction that performs both a GET on the hot key and a TTL on the same key, then executes and reads back the remaining time to live. MULTI matters here because it batches the two commands into one atomic round trip, so the value and its remaining lifetime are consistent with each other.

When the remaining TTL crosses a threshold — his example is five seconds — a client proactively repopulates the key from the origin. The intent is that the key never actually expires under load, so later clients never see a miss at all. This is a useful pattern precisely because it is cheap: no locking, no coordination, just a client noticing that a value it already needed is about to go stale.

The Thundering Herd Problem

The harder case is a key so popular that millions or billions of requests target it, served by many clients in a distributed system, and it expires. Every client misses simultaneously, and every client independently redirects its read to the origin database. The consequences Vedmich lists are cumulative: heavy pressure on RDS, a latency spike visible throughout the application, and possibly downtime, because the relational database was never provisioned for that many simultaneous connections.

The fix is a distributed lock in Valkey. One client wins the right to repopulate the key. Every other client observes that the key is locked, refrains from going to the origin, and waits, polling periodically to see whether the lock has been released. When the winner has written the fresh value, it releases the lock and the waiters read from the cache.

Vedmich walks the two-client version step by step, and the ordering detail is the instructive part. Client 1 does a GET and receives nil. A few milliseconds later Client 2 also does a GET and also receives nil — the miss is not a single event, it is a window. Client 1 has by then set the lock and begun its roughly 10-millisecond database query. Client 2 attempts to set the same lock, because which client wins is essentially arbitrary, discovers the lock already exists, and enters its wait-and-recheck loop. When Client 1 finishes writing the value it unlocks, and Client 2's next check succeeds, so Client 2 reads the value from Valkey without ever touching the database.

sequenceDiagram
    participant C1 as Client 1
    participant C2 as Client 2
    participant V as Valkey
    participant DB as Origin database
    C1->>V: GET hot_key
    V-->>C1: nil (miss)
    C2->>V: GET hot_key
    V-->>C2: nil (miss)
    C1->>V: SET lock:hot_key (acquired)
    C2->>V: SET lock:hot_key (already held)
    V-->>C2: lock exists, wait
    C1->>DB: query origin (~10 ms)
    DB-->>C1: fresh value
    C1->>V: SET hot_key, release lock
    C2->>V: GET hot_key
    V-->>C2: fresh value

Two caveats the talk does not spell out but that matter in production. First, the lock must carry its own expiry, otherwise a client that crashes mid-repopulation leaves every other client waiting indefinitely. Second, the waiting clients must bound their wait, because a lock held longer than expected should degrade into a slow origin read rather than a hung request.

Client-Side Caching

If Valkey's sub-millisecond latency is still not fast enough, the next step is to cache values inside the client process itself, so a repeat read costs nothing but a local map lookup and never crosses the network.

The problem this creates is invalidation across many client processes. Vedmich offers a simple baseline — put a TTL on the local entry — and a more precise option: subscribe to notifications so the client learns that a specific item is no longer valid. He then contrasts the two server-side designs for driving those notifications:

He is direct that per-client tracking "costs a lot of memory from server-side" and is not ideal when memory is constrained. The trade-off is therefore precision against server memory, and the right choice depends on how many distinct keys each client holds and how tolerant you are of unnecessary local evictions.

Connection Pools

Embedded in the client-side caching discussion is a general performance point. Every new TCP connection to Valkey requires a handshake, which costs both wall time and CPU. Vedmich's strong recommendation is a long-lived connection pool so connections are reused rather than re-established per operation.

His illustrative layout: a client opens roughly 10 connections, and dedicates connection 0 exclusively to invalidation messages while the other nine carry data traffic. When an invalidation arrives on connection 0, the client resets the affected entry in its local cache; subsequent reads then flow back out over the data connections and refetch from Valkey. Separating the control channel from the data channel means invalidation notices are not queued behind large data responses.

Session Store

Session storage is one of the most common uses for an in-memory store, especially in e-commerce, banking, and gaming. In the marketplace example, remembering who the user is allows each visitor to see a personalised set of images, cookies, and shopping-cart contents.

The architectural payoff is statelessness. Requests arrive at a load balancer and are dispatched to executors — EC2 instances in his diagram — that hold no session state at all, because all of it lives in Valkey. This is what makes dynamic scaling safe: you can scale out, scale in, or let Kubernetes reschedule pods without stranding a user's session on a terminated instance.

The recommended data structure is the hash, because a session is naturally a set of named fields under one session identifier. Commands use the H prefix — HSET to write fields, HGETALL to read them all back. He notes that Valkey's data structures are systematically prefixed by type, which is why the sorted-set commands later use Z. Hash field access is O(1), so read and write cost does not degrade as the number of users grows.

His Python sketch uses GLIDE, a Valkey client library, and follows a short sequence: open the connection to Valkey, build a dictionary of session attributes, generate a unique session identifier with uuid, store the dictionary under that identifier with HSET, and later use HGETALL to retrieve it and rebuild the equivalent Python dictionary. The reconstruction step is worth noting, because Valkey returns field values as strings and the application is responsible for restoring the original types.

Feature Store

Vedmich then moves from "who is this user" to "what should we predict for this user." A feature store is a component of machine learning infrastructure that holds the input signals — features — used to make predictions. In the marketplace example, a user based in Germany who consistently views German photographers' work should be shown more of that work.

He draws a boundary that is easy to get wrong and states it plainly: Valkey is not part of your training infrastructure. It is not involved in model training at all. Feature stores split into two halves:

• Offline feature store. Where training, batch scoring, and heavy computation happen. Latency does not matter; throughput and history do. His example uses Amazon Redshift.
• Online feature store. Queried at request time, when a user logs in and a decision must be made immediately. Latency is the entire requirement. This is where Valkey fits.

The component tying them together in his architecture is FEAST, an open-source feature store framework that acts as the registry for both halves. Configuration is a YAML file that declares the online store as Valkey with its connection string, and the offline store as Redshift with its authentication and configuration. The application code then asks FEAST for features and does not need to know which backend served them.

As with sessions, the recommended structure is the hash, because the access pattern is random reads of named features at constant time, which is precisely what an online feature store demands.

Real-Time Analytics With Sorted Sets

The next requirement is understanding what is happening now — specifically, which photo is currently the most viewed. Vedmich frames this as a leaderboard problem, with the twist that the ranked entity is an image and the score is a view count rather than a player and a game score.

Valkey's sorted set is implemented as a combination of two structures: a hash that maps each member to its score, and a skip list that maintains members in score order. A skip list is a probabilistic layered linked list that supports ordered insertion and range queries in logarithmic time without the rebalancing machinery of a tree. The property Vedmich emphasises is that it is bidirectional: you can traverse from lowest to highest or highest to lowest equally cheaply, so "top 10" and "bottom 10" are both trivial. Operations are O(log N).

Commands use the Z prefix:

He is careful to say that the realistic production operation is not ZADD with an absolute value but ZINCRBY: an image sitting at 31 views has its count incremented on each view, and the structure automatically reorders it — in his example, moving image 1 up to second place. For reads, his ZRANGE example uses unbounded start and stop values to return the entire leaderboard, with each member returned alongside its score, and he stresses that the sort direction is a parameter rather than a property of the data.

(One transcript note: the audio renders ZINCRBY as "Z in groupby." The surrounding description — increasing the number of views for one image — makes the intended command unambiguous.)

Counting Unique Viewers With HyperLogLog

Total views are easy. Unique views per photo are not. The obvious implementation is a set per image containing every user identifier that viewed it, but set memory grows linearly with the number of distinct users. At millions of users across millions of images, this is prohibitive.

Vedmich's answer, and the data structure he calls the most interesting in the talk, is HyperLogLog: a probabilistic cardinality estimator. It does not store the members at all; it stores a compact sketch derived from hashing them, from which the approximate count of distinct items can be reconstructed. The trade-off is explicit — you give up exactness in return for near-constant memory.

The properties he cites:

• Memory never exceeds 12 kilobytes, regardless of whether you inserted ten thousand users or ten million.
• The error rate is approximately 1%.
• Both read and write are O(1).

Commands use the PF prefix (PFADD to add, PFCOUNT to read). His walkthrough demonstrates the semantics: adding three distinct users returns 1, meaning the sketch changed; counting returns 3. Adding one of the same users again returns 0, because the estimated cardinality did not change, and counting still returns 3. That return value is a useful signal — it is an approximate "was this new?" answer, not a guaranteed one.

The scale comparison is the memorable part. With 10,000 unique users, a set consumes roughly 400-something kilobytes. HyperLogLog with the same input does not return exactly 10,000 — Vedmich reports being off by a little over ten — and uses 12 kilobytes. He is careful to qualify the acceptability: this is "ok-ish" for views, likes, and similar engagement metrics, and depends on your implementation and requirements. The implicit rule is that HyperLogLog belongs wherever the count informs a ranking or a dashboard, and never where the count feeds billing, quota enforcement, or compliance reporting.

Rate Limiting

Rate limiting in a distributed system is hard for a specific reason: the counter must be shared. If each microservice instance keeps its own count, the effective limit is the configured limit multiplied by the instance count, and the limit changes every time you scale. Vedmich's framing is that the counter should therefore live somewhere separate from the services enforcing it — which is exactly what a shared in-memory store provides. He presents two designs.

Fixed Window With a Counter

The simple approach stores the count as a string. Because Valkey is a binary engine, a string can hold numeric data and still support numeric operations such as increment, at O(1).

The algorithm is a fixed window: start from zero, increment on each request, and attach a TTL to the counter key. When the TTL expires, the key disappears, the bucket is empty, and the user has a fresh allowance. His visualisation walks through three permitted requests followed by a rejection, after which the user waits for expiry and is allowed again.

The critical design decision is that this logic runs server-side as a Lua script, not in the application. The reason is atomicity: read-modify-write across a network from many concurrent callers cannot be made correct with separate round trips, and pushing it into the engine collapses it into one atomic operation. It also removes network round trips from the hot path.

The mechanics involve two commands. First you upload the script — passed as a string — to Valkey, which returns a unique SHA identifier. Thereafter you invoke the script by that SHA using EVALSHA, optionally passing arguments, so the script body is not resent on every call. In his example the script returns 1, meaning the request is authorised.

Vedmich openly admits the first script is simplified: the limit and the expiration are hardcoded, at 4 requests per 10 seconds for every user. The logic is: read the current value if it exists; if it does not, create it and start its TTL; then decide whether the request is allowed and whether to increment.

Token Bucket

The more sophisticated design is a token bucket: a bucket with fixed capacity and a defined refill rate — his example is one token per second — from which each request consumes tokens. Vedmich notes that this model is especially familiar in generative AI contexts, where rate limits are themselves expressed in tokens.

Token bucket needs more state than a single integer, so it goes back to the hash. Per user, the hash holds the bucket capacity, the refill rate, the current token consumption (7 in his example), and a timestamp recording when the interaction began, so elapsed time can be converted into refilled tokens.

The Lua script is correspondingly longer, spanning several slides, and its logic runs as follows. Read the configuration values from the script arguments rather than hardcoding them, as in the simple version. Obtain the current time using Valkey's internal time operator — an important detail, because using the client's clock would make the limiter incorrect whenever client clocks drift. Read the existing hash. Compute the available tokens from the previous count plus the refill earned over the elapsed interval; if no hash exists, initialise the user to the full bucket, for example 10 tokens. Check whether at least one token is available and therefore whether the request is allowed. Write the updated state back to the hash.

The final line of the script is the one Vedmich singles out, and it is not about correctness at all: set a TTL on the bucket hash. If a user stops making requests and sleeps for eight hours, there is no reason to keep their bucket resident in expensive memory. Expiry is safe because an absent bucket is reinitialised to full capacity, which is the same state a long-idle user would have reached anyway.

He closes the section by noting these are not all the possible data structures and use cases, only a representative set.

Architecture And Data Flow

The following diagram assembles the pieces from the talk into the single flow they describe: an application tier that is stateless because Valkey holds sessions, serves reads via lazy loading with a lock guarding the origin, and receives asynchronous invalidations driven by change events at the source of truth.

flowchart TD
    U[Users] --> LB[Load balancer]
    LB --> APP[Stateless app instances
local cache + connection pool]
    APP -->|1. GET| V[(Valkey / ElastiCache)]
    V -->|hit: hundreds of microseconds| APP
    APP -->|2. miss: acquire lock| V
    APP -->|3. read origin ~10 ms| DB[(RDS / DynamoDB)]
    APP -->|4. SET value, release lock| V
    DB -->|change stream| L[Lambda invalidator]
    L -->|DEL key| V
    V --- S[Sessions: hash]
    V --- F[Online features: hash]
    V --- R[Leaderboard: sorted set]
    V --- H[Unique views: HyperLogLog]
    V --- RL[Rate limits: string / hash + Lua]
    OFF[(Redshift offline store)] -.->|batch materialisation| F

Operational Best Practices

The final section is Vedmich's AWS-recommended operational guidance.

Treat the Cache as Losable

His first and most emphatic rule is that an in-memory cache is not persistent storage, and he stresses that this is a general recommendation for any in-memory cache rather than a criticism of Valkey. You should store only ephemeral data — data whose loss you can absorb — and you must have a plan for what happens when it disappears. He offers three levels of response:

• Tolerate the loss. Accept the misses and let lazy loading rebuild the cache.
• Detect and refill automatically. Notice the loss and repopulate programmatically, for example through a Lambda function or application logic.
• Rebuild or repair part of the cache deliberately, specifically to avoid a mass-miss situation — which is the thundering-herd scenario again, now triggered by cache loss rather than by expiry.

If none of these are acceptable and you still need in-memory speed, his suggested alternative within AWS is Amazon MemoryDB, which provides data consistency by replicating a transaction log across availability zones. The trade-off he implies but does not quantify is that durability costs write latency; the notes should not be read as saying MemoryDB is as fast as a pure cache for writes.

TTL Discipline

Vedmich states that AWS does not recommend setting a TTL at the global server level, and that a single default TTL value for everything is a poor approach. Different data has different volatility, and a global default guarantees that some of it is either stale or evicted too early. Instead, set a per-item TTL, using whichever expression fits:

• Relative expiry — expire in 30 seconds from now.
• Absolute expiry — expire at a specific time. His example: when his talk ends, expire every cached item relating to his talk.

Add Random Jitter

The single most transferable operational tip in the talk. If your application loads many items from the origin at roughly the same moment and writes them all to Valkey with an identical TTL, they will all expire at the same moment, and you have reproduced the thundering herd at scale — every client stampeding back to RDS or DynamoDB simultaneously. Adding a small random offset to each TTL spreads expiry across a window, so origin load is smeared out rather than spiked. Note that this is a distinct failure mode from the single hot key discussed earlier: the lock protects one key, jitter protects against correlated expiry across many keys.

Eviction Policy

Independently of TTL, you must decide what happens when memory fills. The choices he lists:

• allkeys — evict from the entire keyspace.
• volatile — evict only from keys that carry a TTL, leaving keys without one untouched.

Within either scope, the selection algorithm can be least recently used (LRU), least frequently used (LFU), or simply the shortest remaining TTL. His repeated qualifier is "it depends" — specifically, on whether your application can survive the eviction of an arbitrary key.

Size the Cache Deliberately

The closing point returns to economics. Cache size is a direct cost, because memory is the expensive tier: 20 times faster than disk, but also more expensive than disk. The question to ask is whether you actually consume everything you are storing. His AWS recommendation is to apply autoscaling to find the balance empirically rather than provisioning for the worst case, and he ends with the blunt version: you probably do not need half a terabyte of memory.

Trade-offs And Limitations

• Approximation versus memory. HyperLogLog's ~1% error is the price of constant 12 KB memory. It is appropriate for views and likes; it is not appropriate anywhere the count must be exact or auditable.
• Cache invalidation is eventually consistent. The stream-and-Lambda design leaves a real window in which readers see stale data. Any workload that cannot tolerate that window must read the origin.
• The lock pattern trades latency for origin protection. Waiting clients are slower than clients that stampede, but the database survives. Vedmich does not discuss lock expiry or waiter timeouts, and both are needed in production to avoid a crashed lock holder blocking everyone.
• Client-side cache invalidation forces a memory-versus-precision choice. Per-client tracking is precise but expensive in server memory; prefix broadcast is cheap but invalidates entries that did not change.
• No durability. Valkey is explicitly not persistent storage; you must design for data loss, or move to something like MemoryDB and accept its costs.
• Not a training system. Valkey serves online features. It has no role in model training or batch scoring; that stays in the offline store.
• Memory is expensive. Every caching decision — object size, TTL length, eviction scope, S3 blob caching — is ultimately a spend decision.
• Change-stream invalidation is not portable. This is the substance of the first audience question. Vedmich's DynamoDB example relies on a database-native trigger mechanism. When you use a third-party, external, or self-managed database, his advice is to first check whether the engine itself offers something equivalent — he acknowledges Postgres has triggering capabilities — and otherwise to put the logic in the application: on update, enqueue an invalidation task on a queue, and have a consumer remove the item from the cache. He prefers a mechanism inside the database layer where one exists, particularly when combining lazy loading with write-through.
• You cannot invalidate without compute in the AWS path. A second audience question asked whether a DynamoDB stream could reach Valkey through API destinations or an EventBridge pipe and delete the key directly, avoiding Lambda entirely. Vedmich initially said he would need to check the documentation, and after the questioner asserted it is not possible, he confirmed that today you still need a Lambda function. Plan for that component and its cost, and re-verify against current AWS documentation, since this is exactly the kind of integration that changes.
• Vendor framing. The talk is delivered by an AWS architect, and its recommendations, managed-service references, and benchmark figures come from that perspective. The Valkey data structures and patterns are portable to any deployment; the ElastiCache, DynamoDB Streams, Lambda, Redshift, and MemoryDB specifics are not.

Practical Takeaways

• Reach for a cache when a query result is both expensive and reused. Remember that removing read replicas can offset part of the cache's cost.
• Default to lazy loading for reads, and drive invalidation from the origin's change stream rather than asking every writer to remember the cache.
• Add jitter to every TTL you set in bulk. This is a one-line change that prevents a correlated-expiry stampede.
• Protect known hot keys with a repopulation lock, and give the lock its own expiry plus a bounded wait on the client side.
• Watch remaining TTL on hot keys with a MULTI of GET and TTL, and refresh proactively at a threshold such as five seconds so the key never actually expires.
• Use a long-lived connection pool, and dedicate one connection to invalidation messages so control traffic is not queued behind data.
• Pick the structure by access pattern: hash for sessions and online features (O(1)), sorted set for rankings (O(log N)), string plus INCR for fixed-window counters, HyperLogLog for approximate distinct counts.
• Put multi-step rate-limiting logic in a Lua script loaded once via SCRIPT LOAD and invoked by EVALSHA, so the read-modify-write is atomic and the script body is not resent per call.
• In time-dependent scripts, read the clock from the server rather than from the client.
• Set TTLs per item, never globally, and put a TTL even on state like token buckets that has no natural expiry, purely to reclaim memory from idle users.
• Choose the eviction policy consciously: volatile scope protects keys you never gave a TTL, and LRU, LFU, and TTL-based selection encode different assumptions about reuse.
• Write down what happens if the entire cache is lost, and make sure the answer is something better than "the origin database falls over."

Key Terms

• Valkey — Open-source in-memory data store, forked from Redis in 2024, hosted under the Linux Foundation and CNCF, API-compatible with Redis.
• ElastiCache — AWS managed in-memory caching service that can run Valkey as its engine.
• MemoryDB — AWS in-memory database that adds durability by replicating a transaction log across availability zones.
• Lazy loading (cache-aside) — Read the cache first; on a miss, read the origin and populate the cache.
• Write-through — Write to the origin and the cache in the same operation so the cache is never cold for that item.
• Thundering herd — A simultaneous cache miss across many clients that redirects a flood of concurrent traffic to the origin database.
• Distributed lock — A key in the shared store that grants exactly one client the right to repopulate a value while others wait.
• Jitter — A small random offset added to TTLs so that keys written together do not expire together.
• Hash — Valkey structure storing named fields under one key, with O(1) access; commands are H-prefixed (HSET, HGETALL).
• Sorted set — Valkey structure combining a hash with a skip list to keep members ordered by score at O(log N); commands are Z-prefixed (ZADD, ZINCRBY, ZRANGE).
• Skip list — Layered linked list supporting ordered insertion and bidirectional range traversal in logarithmic time.
• HyperLogLog — Probabilistic cardinality estimator using at most 12 KB with roughly 1% error; commands are PF-prefixed.
• Token bucket — Rate-limiting model with a fixed capacity refilled at a defined rate, from which each request consumes tokens.
• EVALSHA — Command that executes a previously uploaded Lua script by its SHA identifier, giving atomic server-side logic without resending the script.
• Feature store — ML infrastructure holding model input signals, split into an offline half for training and batch work and an online half for low-latency serving.
• FEAST — Open-source feature store framework that registers and abstracts the offline and online stores.
• GLIDE — Valkey client library, used in the talk's Python examples.
• Eviction policy — Rule determining which keys are removed when memory is full, by scope (allkeys or volatile) and algorithm (LRU, LFU, or TTL).

Reference: Beyond Speed Limits: Exploring the Performance Power of Valkey