Redis

Background Information

Redis is an open source, in-memory data structure store used as a database, cache, message broker, and streaming engine. Flink supports it as an output of streaming data. The information supported by Redis Connector is as follows:

Prerequisite

• A Redis instance has been created.

Usage Restrictions

• Only the Flink computing engine VERA 1.0.3 and above supports the Redis connector.
• Only supported as result table and dimension table, not as source tables.
• Currently, configuration of multiple hosts is not supported.

Grammatical Structures

Source table:

1    CREATE TABLE redis_table (
2    a STRING,
3    b STRING,
4    PRIMARY KEY (a) NOT ENFORCED -- required.
5    ) WITH (
6    'connector' = 'redis',
7    'host' = '<yourHost>'
8    );

Description:

• The DDL of Redis dimension table must meet the following requirements:
• Redis dimension table must declare and only one primary key can be declared.
• Redis dimension table only supports the declaration of two fields, and the field type must be STRING.
• Redis dimension table only supports reading data of type STRING and HASHMAP in Redis.
• The ON condition of Redis dimension table must include the equivalent conditions of all primary keys.
• Redis dimension table only supports two cache strategies: None and LRU.

WITH Parameter

Universal for Sink and Dimension

Sink Exclusive

Exclusive to Dimension

Type Mapping

Description:

1. Redis connector result table supports five Redis data structures, and its DDL must be defined in the following format and the primary key must be defined:

• STRING type DDL has two columns: the first column is key (STRING type), and the second column is value (STRING type). The command to insert data in Redis is set key value.
• LIST type DDL has two columns: the first column is key (STRING type), and the second column is value (STRING type). The command to insert data in Redis is lpush key value.
• SET type DDL has two columns: the first column is key (STRING type), and the second column is value (STRING type). The command to insert data in Redis is sadd key value.
• HASHMAP type The DDL has three columns: the first column is key (STRING type), the second column is hash_key (STRING type), and the third column is hash_value (STRING type) corresponding to hash_key. The command to insert data in Redis is hmset key hash_key hash_value.
• SORTEDSET type DDL has three columns: the first column is key (STRING type), the second column is score (DOUBLE), and the third column is value (STRING type). The command to insert data in Redis is zadd key score value.

1. Because the SCORE type of Redis is applied to SORTEDSET (ordered set), it is necessary to manually set a DOUBLE type SCORE for each Value, so that the Value can be sorted according to the SCORE from small to large.

Example

Sink example:

1    CREATE TEMPORARY TABLE datagen_source (
2      v STRING,
3      p STRING
4    ) with (
5      'connector' = 'datagen'
6    );

Dimension example:

1    CREATE TEMPORARY TABLE datagen_source (
2      id STRING,
3      data STRING,
4      proctime as PROCTIME()
5    ) WITH (
6      'connector' = 'datagen'
7    );