A Small Victory — Acceleration of BITFIELD Commands for ApsaraDB for Redis

1) Problems

1.1) Principles of Read/Write Splitting

  • The keys are written to the main node and then synchronized to the standby nodes.
  • User requests are judged by the proxy.
  • Write requests are forwarded to the main node while read requests are distributed to the standby nodes.
Figure 1: Example of forwarding read and write commands for ApsaraDB for Redis read/write splitting

1.2) BITFIELD Commands

  • Use bitmaps to record a user’s daily application logon. If the $ID user logs on, “SETBIT logins:20200404 $ID 1” is recorded, indicating that the $ID user logged on on April 4, 2020. Run “BITCOUNT logins:20200404” to obtain the number of users that logged on on that day and run “BITOP AND logins:20200404–05 logins:20200404 logins:20200404” to calculate the number of users that logged on for two consecutive days.
  • Determine whether users have read the same articles or watched the same videos.
  • Use bitmaps to easily determine whether a user correctly answered all the questions in a live Q&A activity.
Figure 2: A live Q&A system designed by using Redis bitmaps
[GET type offset] // Obtain the value of the specified bit
[SET type offset value] // Set the value for the specified bit
[INCRBY type offset increment] // Increase the value of the specified bit
[OVERFLOW WRAP|SAT|FAIL] // Control the INCR threshold

1.3) BITFIELD Problems in Read/Write Splitting Instances

2) Ideas and Problem Resolution

2.1) Solutions

  • Solution 1: Mark the BITFIELD command as a read attribute in the ApsaraDB for Redis kernel. When it contains sub-commands of the write attribute, such as SET and INCRBY, the BITFIELD command is synchronized to the standby nodes. When you use this solution, you do not need to modify the external components (proxy and client) but do need to specially process the BITFIELD command, which destroys the consistency of the unified processing of engine commands.
  • Solution 2: Add the BITFIELD_RO command, which is similar to the GEORADIUS_RO command, to only support GET commands. Because all these commands are read operations, this ensures that the standby nodes can process BITFIELD commands. This solution is clear and reliable but requires the adaptation of the proxy and client.

2.2) Adding BITFIELD_RO

"read-only fast @bitmap",
tair-redis > SLAVEOF 6379
tair-redis > set k v
(error) READONLY You can't write against a read only replica.
tair-redis > BITFIELD mykey GET u4 0
(error) READONLY You can't write against a read only replica.
tair-redis > BITFIELD_RO mykey GET u4 0
1) (integer) 0

2.3) Forwarding the Proxy

Figure 4: BITFIELD processing logic after the BITFIELD_RO command is added

2.4) Contribution to the Community

Figure 5: Contribution Rankings for Redis 6.0 RC

3) Extension and Discussion

3.1) Summary

3.2) Discussion Questions

  • Big keys cause data skew, which hinders the linear expansion of Redis capacity and service capability.
  • A big key is very likely a hot spot.
  • If you accidentally perform range operations on a big key, slow queries may occur and the bandwidth is likely to burst.

Original Source:



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alibaba Cloud

Alibaba Cloud

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com