Secret Values

2 min read

Ververica Unified Streaming Data Platform allows you to configure variables and keys to prevent security risks that are caused by the use of information such as plaintext AccessKey pairs and passwords. You can reuse variables and keys to avoid repeatedly writing the same code or values and simplify configuration management. You can reference a variable or a key in various scenarios, such as the development of SQL, JAR, or Python drafts, log output configuration, and UI-based parameter configuration. This topic describes how to create and reference a variable or a key.

Background Information

Variables and secret value keys are both composed of a name and a corresponding value. The name should reflect the meaning of value and is completely customizable. The value is the actual data referenced by a deployment.

There are some important distinctions between a variable and a secret value key:

Variable: A variable can be referenced in only one deployment. You can reference variables in the DDL statements of SQL drafts and the input parameters of a Main function of JAR and Python drafts. Variables must be in the ${Variable name} format.
Key: A key can be referenced in only one namespace. can reference keys when you write DDL statements of SQL drafts and create catalogs on the UI. Keys must be in the ${secret_values.Key_name} format.

Variables

Before getting started with Variables, note that within a single draft, each variable name must be unique.

Reference a Variable in an SQL Draft

You can reference a variable when you develop an SQL draft and change the variable value when you start the deployment.

In the development console, click the SQL Editor in the left-side navigation pane.
In the Drafts tab, select your desired draft and input variables in the ${Variable name} format in the SQL Editor. The following code provides an example using variables in several places within a SQL draft.

Info

If you reference a variable in the WITH clause, you must enclose the variable with a pair of single quotation marks (').
If you reference a variable as a table name or field name in an SQL statement, you must enclose the variable with a pair of grave accents (`).

``sql showLineNumbers create temporary table ${source_name}`( id varchar, name varchar ) with ( 'connector' = 'datagen' );

create temporary table blackhole( id varchar, ${test_name} varchar ) with ( 'connector' = '${blackhole}' );

insert into blackhole select * from ${source_name};

TEXT

13. Ververica Unified Streaming Data Platform automatically identifies variables within the draft that are in the required `${Variable name}` format.
2You can assign the variables' default values in one of two methods:
3- Method 1: Enter the variable value in the **Variables** panel of the SQL Editor.
4
5![Method 1](media/variables-sql-draft-method1.png)
6
7- Method 2: Click the *Configurations* tab on the right side of the SQL Editor.
8Enter the default variable values in the *Variables* section.
9
10![Method 2](media/variables-sql-draft-method2.png)
11
124. After deploying the draft, view the deployment variables in the *Configuration* tab of the *Deployments* page.
13
14![Deployment Configuration](media/variables-deployment-start.png)
15
165. On the *Deployments* page, find the desired deployment and click *Start* in the *Actions* column to change variable values.
17
18![Start Job](media/variables-start-job.png)
19
20### Reference a Variable in a JAR or Python Draft
21
22In addition to SQL drafts, you can also reference a variable in the parameters of a Main function of a JAR or Python draft.
23
241. In the Ververica Unified Streaming Data Platform console, navigate to *Deployments* on the left side pane, followed then by clicking *Create Deployment*.
252. Enter variable names in the *Entry Point Main Arguments* field based on the format, `-- input ${test1}${test2}${test3}`, with `test1`, `test2`, and `test3` each being respective variable names.
26Aftewards, enter the variable values in the *Variables* section below.
27
28![Variables as Entry Point Arguments](media/variables-python-deployment.png)
29
303. In the Create Deployment dialog box, click *Deploy*.
314. On the *Deployments* page, find the desired deployment and click *Start* in the *Actions* column to change variable values.
32
33## Add a Secret Value
34
35You can only add and delete keys. You cannot change or view the values of keys.
36
37To configure a secret value key:
38
391. Go to **Security** → **Secret Values**.
402. Click **Add Secret Value**.
413. Fill in **Secret Name** and **Secret Value**. Note that they both are case-sensitive. Then click **OK**.
42
43![Limitations](media/secret-values-11-refresh.png)
44
45Within our namespace, we are now able to reference a key named `secret_key1`.
46After you configure a Secret Value, you can reference the key in SQL scripts or when creating a new catalog for a job in the console of fully managed Flink.
47
48## Reference a Secret Value Key
49
50After you configure a key, you can reference the key in SQL scripts and in catalog creation.
51
52Wherever you use the key, use the ``${secret_values.Key_name}`` format.
53
54### Reference the Key in the DDL Statements in SQL Scripts
55
561. Go to **SQL Editor**.
572. Reference the key in the SQL scripts. Note that `accessKey` has a value assigned from our key created previously, `secret_key1`.
58
59Sample code:
60
61    CREATE TEMPORARY table datahub_source(
62      name VARCHAR
63    ) WITH (
64      'connector' = 'datagen'
65    );
66
67    CREATE TEMPORARY table datahub_sink(
68      name  VARCHAR 
69    ) WITH (
70      'connector'='datahub',
71      'endpoint'='<yourEndpoint>',
72      'project'='<yourProject>',
73      'topic'='<yourTopic>',
74      'accessId'='<yourAccessId>',
75      'accessKey'='${secret_values.secret_key1}'
76    );
77
78    INSERT INTO datahub_sink
79    SELECT
80      LOWER(name)
81    from datahub_source;

Reference a Secret Value Key in the Deployment Logging Dection

Navigate to the Logging (/byoc/user-guides/admin-operator-guide/manage-deployments/modify-deployment#logging-section) section of your deployment.
Reference a key in the Logging section. The following sample code provides an example on how to reference a key named accessKeyId and a key named accessKeySecret when you configure parameters to export the logs of a deployment to Simple Log Service.

TEXT

1<Appender name="SLS" type="SLS">
2  <Layout pattern="%d{yyyy-MM-dd HH:mm:ss,SSS}{GMT+8} %-5p %-60c %x - %m%n" type="PatternLayout" charset="UTF-8"/>  
3
4  <!-- The final effective log path is: ${baseUri}/logs/${namespace}/${deploymentId}/{jobId}/ -->
5  <Property name="namespace">{{ namespace }}</Property> <!-- Do not modify this line -->
6  <Property name="project">YOUR-SLS-PROJECT</Property>  
7  <Property name="logStore">YOUR-SLS-LOGSTORE</Property> 
8  <Property name="endpoint">YOUR-SLS-ENDPOINT</Property> 
9  <Property name="accessKeyId">${secret_values.accessKeyId}</Property> 
10  <Property name="accessKeySecret">${secret_values.accessKeySecret}</Property> 
11  <Property name="topic">{{ namespace }}:{{ deploymentId }}:{{ jobId }}</Property>
12  <Property name="deploymentName">{{ deploymentName }}</Property>
13  <Property name="flushIntervalSeconds">10</Property>
14  <Property name="flushIntervalEventCount">100</Property>
15</Appender>

Reference a Secret Value Key in Catalog UI

When you create a catalog in Ververica Unified Streaming Data Platform's console, you can reference a namespace secret value key.

In the left-side pane, click Catalogs.
After selecting Create Catalog, fill in the required details and then reference a secret value key as the password. The following example shows us creating a new MySQL catalog with the key named secret_key1 from earlier.

Was this helpful?

Yes No