Edit

Set Variable Activity in Azure Data Factory and Azure Synapse Analytics

  • Article

APPLIES TO: Azure Data Factory Azure Synapse Analytics

Tip

Try out Data Factory in Microsoft Fabric, an all-in-one analytics solution for enterprises. Microsoft Fabric covers everything from data movement to data science, real-time analytics, business intelligence, and reporting. Learn how to start a new trial for free!

Use the Set Variable activity to set the value of an existing variable of type String, Bool, or Array defined in a Data Factory or Synapse pipeline or use the Set Variable activity to set a pipeline return value (preview).

Create a Set Variable activity with UI

To use a Set Variable activity in a pipeline, complete the following steps:

  1. Select the background of the pipeline canvas and use the Variables tab to add a variable:

Screenshot of an empty pipeline canvas with the Variables tab selected with an array type variable named TestVariable.

  1. Search for Set Variable in the pipeline Activities pane, and drag a Set Variable activity to the pipeline canvas.

  2. Select the Set Variable activity on the canvas if it isn't already selected, and then select the Settings tab to edit its details.

  3. Select Pipeline variable for your Variable type.

  4. Select the variable for the Name property.

  5. Enter an expression to set the value for the variables. This expression can be a literal string expression, or any combination of dynamic expressions, functions, system variables, or outputs from other activities.

Screenshot of the UI for a Set variable activity.

Setting a pipeline return value with UI

We have expanded Set Variable activity to include a special system variable, named Pipeline Return Value, allowing communication from the child pipeline to the calling pipeline, in the following scenario.

You don't need to define the variable, before using it. For more information, see Pipeline Return Value

Screenshot with ExecutePipeline Activity.

Type properties

Property Description Required
name Name of the activity in pipeline yes
description Text describing what the activity does no
type Must be set to SetVariable yes
variableName Name of the variable that is set by this activity yes
value String literal or expression object value that the variable is assigned to yes

Incrementing a variable

A common scenario involving variable is to use a variable as an iterator within an Until or ForEach activity. In a Set variable activity, you can't reference the variable being set in the value field, that is, no self-referencing. To work around this limitation, set a temporary variable and then create a second Set variable activity. The second Set variable activity sets the value of the iterator to the temporary variable. Here's an example of this pattern:

  • First you define two variables: one for the iterator, and one for temporary storage.

Screenshot shows defining variables.

  • Then you use two activities to increment values

Screenshot shows increment variable. 

{
    "name": "pipeline1",
    "properties": {
        "activities": [
            {
                "name": "Increment J",
                "type": "SetVariable",
                "dependsOn": [],
                "policy": {
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "variableName": "temp_j",
                    "value": {
                        "value": "@add(variables('counter_i'),1)",
                        "type": "Expression"
                    }
                }
            },
            {
                "name": "Set I",
                "type": "SetVariable",
                "dependsOn": [
                    {
                        "activity": "Increment J",
                        "dependencyConditions": [
                            "Succeeded"
                        ]
                    }
                ],
                "policy": {
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "variableName": "counter_i",
                    "value": {
                        "value": "@variables('temp_j')",
                        "type": "Expression"
                    }
                }
            }
        ],
        "variables": {
            "counter_i": {
                "type": "Integer",
                "defaultValue": 0
            },
            "temp_j": {
                "type": "Integer",
                "defaultValue": 0
            }
        },
        "annotations": []
    }
}

Variables are scoped at the pipeline level. This means that they're not thread safe and can cause unexpected and undesired behavior if they're accessed from within a parallel iteration activity such as a ForEach loop, especially when the value is also being modified within that foreach activity.

Related content

Learn about another related control flow activity: