Introducing the Redis-full-check Tool

By Zhu Zhao

Redis-full-check is a tool from the Alibaba Cloud Redis & MongoDB team that checks data consistency between two Redis databases and is usually used to check the correctness after Redis data migration (redis-shake).

Basic Principle

redis-full-check performs data verification by conducting a full comparison of the data between the source side and the target side in Redis. This comparison is performed by using the multi-round comparison method: The data from the source side and the target side is fetched for comparing the data differences and inconsistent data is recorded (in sqlite3 db) for the next-round comparison. After multiple rounds of comparison, data is continuously converged to reduce data inconsistency between the source database and the target database due to incremental data synchronization. The final data in sqlite is the final data differences.

The comparison conducted by redis-full-check is unidirectional: redis-full-check fetches data from source database A and checks if the data in A is also present in database B. It will not conduct reverse detection. That is, it checks whether the target database is a subset of the source database. If you want a bidirectional comparison, you need to compare data twice. The first comparison uses A as the source database and B as the target database. The second comparison uses B as the source database and A as the target database.

The following is the basic data flow diagram. redis-full-check uses the multi-round comparison, as shown in the yellow box. For each comparison, keys are fetched. In the first-round comparison, keys are fetched from the source database and the subsequent rounds of comparison fetch keys from sqlite3 db. After keys are fetched, the corresponding field and value of a key are fetched for comparison. Inconsistent data is stored in sqlite3 db for the next round of comparison.

Inconsistency Types

Redis-full-check divides data inconsistency into two types: key inconsistency and value inconsistency.

Key Inconsistency

Key inconsistency falls into the following subtypes:

  • lack_target: A key exists in the source database but does not exist in the target database.
  • type: A key exists both in the source database and the target database, but the type is inconsistent.
  • value: A key exists in both the source database and the target database and is of the same type, but the value is inconsistent.

Value Inconsistency

Different data types have different comparison criteria:

  • string: The value is different.
  • hash: A field exists and meets one of the following conditions:
  • A field exists on the source side but not on the target side.
  • A field exists on the target side but not on the source side.
  • A field exists both on the source side and the target side, but the value is different.
  • set/zset: similar to hash.
  • list: similar to hash.

The field conflict type falls into the following cases (only applicable to keys of types hash, set, zset, and list ):

  • lack_source: A field exists in a source-side key but not in a target-side key.
  • lack_target: A field does not exist in a source-side key, but the field exists in a target-side key.
  • value: A field exists both in a source-side key and a target-side key, but the values of the two fields are different.

Comparison Principle

Three compare modes (comparemode) are available:

  • KeyOutline: only compares if key values are equal.
  • ValueOutline: only compares if values have the equal length.
  • FullValue: compares if key values, value length, and values are equal.

The number of comparison rounds is determined by comparetimes (comparetimes is set to 3 by default):

  • In the first-round comparison, all keys in the source database are found. Then keys are fetched from the source database and the target database respectively.
  • The second round starts the iterative comparison and only compares inconsistent keys and fields found from the last round of comparison.
  • For key inconsistency (including lack_source , lack_target , and type), re-fetch keys and values from the source and the target databases for comparison.
  • For keys of string that have inconsistent values, compare these keys again: Fetch keys and values from the source and target databases.
  • For keys of hash, set, and zset that have inconsistent values, only re-compare inconsistent fields. Fields that have been compared and are found to be consistent do not need to be compared again. This prevents big keys from always failing the verification if updates are frequently performed.
  • For keys of list that have inconsistent values, re-compare keys: Fetch keys and values from the source and target values.
  • There is a specific interval between two rounds of comparison.

For big keys of hash, set, zset, and list, follow these rules:

  • If len is smaller or equal to 5192, use the following commands and fetch all fields and values for comparison: hgetall, smembers, zrange 0 -1 withscores, and lrange 0 -1.
  • If len is greater than 5192, use hscan, sscan, zscan, and lrange to batch-fetch fields and values.

Parameter Description

The following are the main parameters in redis-full-check:

For example, the source Redis database is and the target database is

The metric information uses the following format:

Sqlite 3 DB File

Results will be saved in the sqlite3 db file. If no file is specified, the result.db file under the current directory is used. If a third comparison round exists, the three following files are present: result.db. 1, result.db. 2, and result.db. 3.

  • Table key: saves inconsistent keys
  • Table field: saves inconsistent fields of hash, set, zset, and list. The list saves subscript values.
  • The key_id field in the table field is associated with the id field in the table key.
  • Table key_<N> and field_<N>: save the results after the N comparison round (that is, the intermediate results).


Reference Materials for the open-source project

Here are some reference Materials for the open-source project:


Data migration tool redis-shake

Feel free to post your problems or suggestions in Issues on GitHub. You are welcome to join our open-source project development.


Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store