What’s New in Redis 4.0?

As one of the most popular NoSQL databases, Redis earned its name by offering outstanding performance and extensive data structure support. Besides, it never stops evolving and, ever since its birth, it has been continuously launching new features.

Redis 4.0, the latest GA version in the community, has been released for almost a year, and ApsaraDB for Redis 4.0 has been released for half a year. There are a series of articles that analyze Redis 4.0’s new features from the perspective of source code implementation. This article will discuss how to quickly understand and use benefits of Redis 4.0 from the perspective of users.

Lazyfree Feature

Many of you must have encountered the problem of deleting large keys. In addition to strings, Redis also supports list, set, hash, sorted set, and other complex data structures. These data structures have enriched Redis usage. However, improper usage may cause problems of over-sized single keys.

To better understand the problem, let’s look at a simple example. Let’s assume that there is an influential social networking account, “Mr. V”, who has millions of subscribers. We can use the set type data structure to store Mr. V’s fans’ IDs. The key that stores the fans set is called funs; let’s take a look at the number of the subscribers:

Mr. V has over 6 million fans! Unfortunately, Mr. V canceled his account one day and we need to delete his information. Let’s use the DEL command to delete this key.

Redis 4.0 expands slowlog’s return results and displays the IP:Port of clients that generated slow logs for convenient tracing.

We can see that the deletion operation took 3 seconds. This means that Redis was unable to execute other commands within these 3 seconds. This may cause adverse effect for online businesses. How can we avoid the blockage caused by deleting large keys? Redis 4.0 launched the Lazyfree function. Lazyfree uses the UNLINK command to delete large keys. The main thread is only responsible for removing the key from the database and the actual release operation is done by BIO backend threads. Let’s take a look at the result:

We can see that the UNLINK execution was very fast and no slowlog was generated.

Lazyfree involves three commands:

  1. UNLINK: asynchronously deletes the key.
  2. FLUSHDB ASYNC: asynchronously clears the current DB.
  3. FLUSHALL ASYNC: asynchronously clears all DBs.

And 4 configuration items:

  1. lazyfree-lazy-expire: asynchronously deletes expired keys.
  2. lazyfree-lazy-eviction: asynchronously evicts keys.
  3. lazyfree-lazy-server-del: uses the asynchronous deletion function in the case of implicit deletion. For example, rename b, if b exists, delete b.
  4. slave-lazy-flush: in the case of full data synchronization, the slave asynchronously clears all DBs.

Lua Script Supports Random Operations

Redis is built with a Lua environment to allow users to expand features. However, for the sake of consistency, the scripts must be pure functions. In other words, when a Lua script is given the same parameters, the same results will be returned for repeated executions.

Why? Because Redis not only is a standalone memory database, Redis also supports Master/slave replication and persistence. An executed Lua script will be replicated to the slave and be persisted to disk. If the results for repeated executions are not the same, then data in memory, disk, and slave will be inconsistent. This will cause data disorder after failover and restart and cause adverse effect to the business.

Here is an actual example. Assume that we have a Lua script to simply record the current time:

It uses Redis’ TIME command to obtain the timestamp, and store the timestamp to a key named now. However, an exception was thrown when executing this script:

The error message is simple: Write commands not allowed after non deterministic commands. Because TIME is random, Redis does not allow executing the write command, in order to ensure data consistency. How can we achieve the random write operation? The answer is provided in the above error message: call redis.replicate_commands(). After executing redis.replicate_commands(), Redis does not synchronize the complete Lua script to Slave or persist it to disk. Instead, it directly replicates the write command to call Redis from the script and then the same results will be returned at the slave and after persistence.

The script is modified as follows:

Execute it again to achieve random write:

LFU-based Hotspot Key Discovery Mechanism

LFU is a memory eviction policy newly introduced in Redis 4.0. LFU provides a more precise memory eviction algorithm. Its core function is to record the key access frequency within a period of time. It brings an additional benefit — hotspot key discovery.

Simply put, LFU uses 0–255 to indicate the access frequency of a key. A larger value indicates a higher frequency. The frequency increase is recorded in a logarithm-based manner. When LFU is 255, it indicates the key has been accessed by 1 million times.

We can use the OBJECT FREQ command to obtain a specified key’s access frequency. But before that, we must set the memory eviction policy to allkeys-lfu or volatile-lfu:

Use the SCAN command to traverse all keys, use OBJECT FREQ to retrieve the access frequency, and then sort the results to obtain the hotspot key. To ease user usage, Redis’ built-in client redis-cli also provides the hotspot discovery function. You can simply append --hotkeys to the end of redis-cli and then execute it:

MEMORY Analysis Command

Memory analysis helps optimize Redis usage. The completely new MEMORY command can help users perform this operation. The MEMORY command has 5 sub-commands. These 5 sub-commands can be viewed from MEMORY HELP.

You can learn more about theses sub-commands on Redis’s official documentation.

Summary

We looked at some of the key features of Redis 4.0, focusing on features like Lazytree, Lua, and Memory. To learn more about Redis on Alibaba Cloud, visit the ApsaraDB for Redis product page.

Reference:

https://www.alibabacloud.com/blog/what%27s-new-in-redis-4-0%3F_593819?spm=a2c41.11779313.0.0

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store