variable assignment in jinja2

TTL255 - Przemek Rogala's blog

Computer networks, python and automation.

• Jinja2 Tutorial - Part 2 - Loops and conditionals

Welcome to part 2 of my Jinja2 Tutorial. In part 1 we learned what Jinja2 is, what are its uses, and we started looking at templating basics. Coming up next are loops and conditionals, sprinkled with tests and a healthy dose of examples!

Jinja2 Tutorial series

• Jinja2 Tutorial - Part 1 - Introduction and variable substitution
• Jinja2 Tutorial - Part 3 - Whitespace control
• Jinja2 Tutorial - Part 4 - Template filters
• Jinja2 Tutorial - Part 5 - Macros
• Jinja2 Tutorial - Part 6 - Include and Import
• J2Live - Online Jinja2 Parser

Looping over dictionaries

Comparisons, logical operators, loop filtering.

• in Operator
• GitHub repository with resources for this post

Control structures

In Jinja2 loops and conditionals come under name of control structures, since they affect flow of a program. Control structures use blocks enclosed by {% and %} characters.

First of the structures we'll look at is loops.

Jinja2 being a templating language has no need for wide choice of loop types so we only get for loop.

For loops start with {% for my_item in my_collection %} and end with {% endfor %} . This is very similar to how you'd loop over an iterable in Python.

Here my_item is a loop variable that will be taking values as we go over the elements. And my_collection is the name of the variable holding reference to the iterated collection.

Inside of the body of the loop we can use variable my_item in other control structures, like if conditional, or simply display it using {{ my_item }} statement.

Ok, but where would you use loops you ask? Using individual variables in your templates works fine for the most part but you might find that introducing hierarchy, and loops, will help with abstracting your data model.

For instance, prefix lists or ACLs are composed of a number of lines. It wouldn't make sense to have these lines represented as individual variables.

Initially you could model a specific prefix list using one variable per line, like so:

Which could be used in the following template:

Rendering results:

This approach, while it works, has a few problems.

If we wanted to have more lines in our prefix list we'd have to create another variable, and then another one, and so on. We not only have to add these new items to our data structure, templates would also have to have all of these new variables included individually. This is not maintainable, consumes a lot of time and is very error prone.

There is a better way, consider the below data structure:

And the template rendering prefix list configuration:

After rendering:

If you look closely you'll notice this is essentially modeling the same thing, a prefix list with a number of entries. But by using list we clearly state our intent. Even visually you can tell straight away that all of the indented lines belong to the PL_AS_65003_IN.

Adding to the prefix list here is simple, we just need to append a new line to the block. Also, our templates don't have to change at all. If we used loop to iterate, like we did here, over this list then the new lines will be picked up if we re-run the rendering. Small change but makes things a lot easier.

You might have noticed that there's still room for improvement here. Name of the prefix list is hardcoded in the prefix list definition and in our for loop. Fear not, that's something we'll be improving upon shortly.

Let's now see how we can loop over dictionaries. We will again use for loop construct, remember, that's all we've got!

We can use the same syntax we used for iterating over elements of the list but here we'll iterate over dictionary keys. To retrieve value assigned to the key we need to use subscript, i.e. [] , notation.

One advantage of using dictionaries over lists is that we can use names of elements as a reference, this makes retrieving objects and their values much easier.

Say we used list to represent our collection of interfaces:

There is no easy way of retrieving just Ethernet2 entry. We would either have to iterate over all elements and do key name comparison or we'd have to resort to advanced filters.

One thing to note, and this is hopefully becoming apparent, is that we need to spend some time modeling our data so that it's easy to work with. This is something you will rarely get right on your first attempt so don't be afraid to experiment and iterate.

Following with our example, we can keep data on individual interfaces assigned to keys in interfaces dictionary, instead of having them in a list:

Now we can access this data in our template like so:

Giving us end result:

Here intf refers to Ethernet1 and Ethernet2 keys. To access attributes of each interface we need to use interfaces[intf] notation.

There is another way of iterating over dictionary, which I personally prefer. We can retrieve key and its value at the same time by using items() method.

The end result is the same but by using items() method we simplify access to the attributes. This becomes especially important if you want to recursively iterate over deeply nested dictionaries.

I also promised to show how prefix list example can be improved upon, and that's where items() comes in.

We make small modification to our data structure by making each prefix list name a key int the dictionary prefix_lists

We now add outer loop iterating over key, value pairs in dictionary:

Rendering gives us the same result:

And here you go, no more hardcoded references to the prefix list names! If you need another prefix list you just need to add it to the prefix_lists dictionary and it will be picked up automatically by our for loop.

Note: If you're using version of Python < 3.6 then dictionaries are not ordered. That means order in which you recorded your data might differ from the order in which items will be processed inside of a template.

If you rely on the order in which they've been recorded you should either use collections.OrderedDict if using Jinja2 in Python script, or you can apply dictsort filter in your template to order your dictionary by key or value.

To sort by key:

To sort by value:

This concludes basics of looping in Jinja2 templates. The above use cases should cover 95% of your needs.

If you're looking for discussion of some advanced features connected to looping, rest assured I will be doing write up on those as well. I decided to leave more in depth Jinja2 topics for the final chapters of this tutorial and focus on the core stuff that lets you become productive quicker.

Conditionals and tests

Now that we're done with loops it's time to move on to conditionals.

Jinja2 implements one type of conditional statement, the if statement. For branching out we can use elif and else .

Conditionals in Jinja2 can be used in a few different ways. We'll now have a look at some use cases and how they combine with other language features.

First thing we look at is comparing values with conditionals, these make use of ==, !=, >, >=, <, <= operators. These are pretty standard but I will show some examples nonetheless.

One common scenario where comparison is used is varying command syntax based on the version, or vendor, of the installed OS. For instance some time ago Arista had to change a number of commands due to the lawsuit and we could use a simple if statement to make sure our templates work with all of the EOS versions:

Template, vars, and rendered template for host using EOS 4.19:

And same for device running EOS 4.22:

Quite simple really yet very useful. All we did is check if recorded EOS version is less than, or greater/equal than 4.22, and this is enough to make sure correct syntax makes it to the configs.

To show more complex branching with comparisons I've got here na example of template supporting multiple routing protocols where only relevant config is generated for each device.

First we define some data for hosts.

Device running BGP:

Device running OSPF:

Device with default route only:

Then we create a template using conditionals with branching. Additional protocols choices can be easily added as needed.

Rendering results for all devices:

So there you have it, one template supporting 3 different configuration options, pretty cool.

No implementation of conditionals would be complete without logical operators. Jinja2 provides these in the form of and , or and not .

There is not an awful lot to talk about here so here's just a short example showing all of these in action:

This is is a good place to look at different variable types and their truthiness. As is the case in Python, strings, lists, dictionaries, etc., variables evaluate to True if they're not empty. For empty values evaluation results in False.

I created an example illustrating thruthiness of, non-empty and empty, string, list and dictionary:

Personally I would advise against testing non-boolean types for truthiness. There aren't that many cases where this could be useful and it might make your intent non-obvious. If you simply want to check if the variable exists then is defined test, which we'll look at shortly, is usually a better choice.

Tests in Jinja2 are used with variables and return True or False, depending on whether the value passes the test or not. To use this feature add is and test name after the variable.

The most useful test is defined which I already mentioned. This test simply checks if given variable is defined, that is if rendering engine can find it in the data it received.

Checking if variable is defined is something I use in most of my templates. Remember that by default undefined variables will simply evaluate to an empty string. By checking if variable is defined before its intended use you make sure that your template fails during rendering. Without this test you could end with incomplete document and no indication that something is amiss.

Another family of tests that I find handy are used for checking type of the variable. Certain operations require both operands to be of the same type, if they're not Jinja2 will throw an error. This applies to things like comparing numbers or iterating over lists and dictionaries.

boolean - check is variable is a boolean integer - check if variable is an integer float - check if variable is a float number - check if variable is number, will return True for both integer and float string - check if variable is a string mapping - check if variable is a mapping, i.e. dictionary iterable - check if variable can be iterated over, will match string, list, dict, etc. sequence - check if variable is a sequence

Below is an example of some variables having these tests applied:

You might've noticed that some of these tests might seem a bit ambiguous. For instance to test if variable is a list it is not enough to check if it's a sequence or an iterable. Strings also are both sequences and iterables. So are the dictionaries, even though vanilla Python classes them as Iterable and Mapping but not Sequence:

So what all of this means? Well, I suggest the following tests for each type of variable:

Number, Float, Integer - these work just as expected, so choose whatever fits your use case.

Strings - it's enough to use string test:

{{ my_string is string }}

• Dictionary - using mapping test is sufficient:

{{ my_dict is mapping }}

• Lists - this is a tough one, full check should tests if variable is a sequence but at the same time it cannot be a mapping or a string:

{{ my_list is sequence and my list is not mapping and my list is not string }}

In some cases we know dictionary, or a string, is unlikely to appear so we can shorten the check by getting rid of mapping or string test:

{{ my_list is sequence and my list is not string }} {{ my_list is sequence and my list is not mapping }}

For the full list of available tests follow the link in References .

Last thing I wanted to touch on briefly are loop filtering and in operator.

Loop filtering does exactly what its name implies. It allows you to use if statement with for loop to skip elements that you're not interested in.

We could for instance loop over dictionary containing interfaces and process only the ones that have IP addresses:

As you can see we have 6 interfaces in total but only 4 of them have IP addresses assigned. With is defined test added to the loop we filter out interfaces with no IP addresses.

Loop filtering can be especially powerful when iterating over large payload returned from the device. In some cases you can ignore most of the elements and focus on things that are of interest.

in operator

in operator which is placed between two values can be used to check if value on the left is contained in the value on the right one. You can use it to test if an element appears in the list or if a key exists in a dictionary.

The obvious use cases for in operator is to check if something we're interested in just exists in a collection, we don't necessarily need to retrieve the item.

Looking at the previous example, we could check if Loopback0 is in the list interfaces, and if it does, we will use it to source Management Plane packets, if not we'll use Management1 interface.

Notice that even though interfaces is a dictionary containing a lot of data we didn't iterate over it or retrieve any of the keys. All we wanted to know was the presence of Loopback0 key.

To be completely honest, the above template could use some tweaking, we essentially duplicated 3 lines of config and hardcoded interface names. That's not a very good practice, and I'll show you in the next post how we can make improvements here.

And with that we've come to the end of part 2 of the Jinja2 tutorial. Next I'll cover whitespaces, so you can make your documents look just right, and we'll continue looking at the language features. I hope you learned something useful here and do come back for more!

• Official documentation for the latest version of Jinja2 (2.11.x). Available at: https://jinja.palletsprojects.com/en/2.11.x/
• Jinja2 built-in tests. Available at: https://jinja.palletsprojects.com/en/2.11.x/templates/#list-of-builtin-tests
• Jinja2 Python library at PyPi. Available at: https://pypi.org/project/Jinja2/
• GitHub repo with source code for Jinja. Available at: https://github.com/pallets/jinja/
• GitHub repo with resources for this post. Available at: https://github.com/progala/ttl255.com/tree/master/jinja2/jinja-tutorial-p2-loops-conditionals

SaltStack: Setting a jinja2 variable from an inner block scope

As an example of a solution that will not work , let’s say you have a global flag ‘foundUser’ set to False, then want to iterate through a group of users, and if a condition is met inside the loop, then ‘foundUser’ would be set to True.

This will render to:

So, while you can see that inside the loop, the value of ‘foundUser’ has indeed been modified, once you get outside the loop again, the value of ‘foundUser’ is still False.

This scope behavior seems wrong to those coming from Java, Python, and most other modern programming languages.  But it is unfortunately the way jinja2 works.

The workaround is that instead of setting a simple variable directly, you can instead use a dictionary.  And although the dictionary object pointer itself cannot change, the key/value pair entries can be modified.

Which renders as:

http://stackoverflow.com/questions/9486393/jinja2-change-the-value-of-a-variable-inside-a-loop

http://stackoverflow.com/questions/4870346/can-a-jinja-variables-scope-extend-beyond-in-an-inner-block

https://github.com/pallets/jinja/issues/164

View Cart Checkout

• No products in the cart.

Subtotal: 0,00  €

• SERVICE PROVIDER mpls, segment routing,mpls services like mpls vpn, vpls and mpls te
• DEVOPS_AUTOMATION devops and automation courses like ansible, cisco devnet and bash scripting
• Cisco Security
• Data Center
• SDN-based NETWORK sd-access, sd-wan and sdn based data center
• Juniper Security
• Free Courses
• Video Courses
• Service Provider
• DevOps / Automation
• SDN-based Networks
• About Teachers
• Privacy Policy

25. Python Jinja2 Template with Loops and Conditonals

• 25. Python Jinja2 Template with…

Table of Contents

Python Jinja2 template also has loops, conditionals, and some other features in addition to the variable substitution discussed in the previous section .

In this section, we demonstrate a jinja2 configuration template with all these features.

Python Jinja2 Template Fundamental

Usually, when we send the configuration to multiple devices, the configuration of the devices are not exactly the same and some configuration data may be different for network devices.

For example, the network number in the routing protocol, the access rules in the access list, and the AS number are some of the configuration data that are usually different among network devices.

With jinja2 Template we can generate a single configuration template in which variables are used instead of configuration data.

When the configuration template is applied to the network devices, the variables are replaced with configuration data related to each device which is retrieved from the automation data structure.

But unlike the example demonstrated in the previous section, variable substitution alone cannot always produce an efficient configuration template.

In this simple example of OSPF configuration in device1 and device2, I used a “for loop” in addition to “variable substitution” to generate an efficient configuration template that covers the OSPF configuration of both devices.

This figure shows the main features of the jinja2 template.

In variable substitution , we use variables in the place of configuration data that differ between network devices.

With condition we can apply a configuration based on a condition, for example, version number of the operating system, the routing protocol used in network device, AS number of the device, the device vendor and many other conditions.

With loop , as we saw in the example, we can iterate through something like access rules in the access list, or network numbers is a routing protocol.

You will find the most advanced features of the jinja2 template on this website , which is highly recommended. Here are many practical examples of the jinja2 template’s capabilities.

Review Inventory Files

Before we check the automation script and the jinja2 configuration template example, let’s review the inventory files and the Nornir data structure.

They’re just like the previous section and don’t need any further explanation, but let’s just go through them quickly once again.

There is only one device, “R1”, configured within “hosts.yaml” that is part of the “cisco” group.

The files “groups.yaml” and “defaults.yaml” are as usual and there is nothing special to talk about.

Review Configuration Data

The Nornir data structure is exactly what we talked about in the previous section.

We have three files, “R1.yaml” in the “host_vars” directory, which contain configuration data for the R1 router. “cisco.yaml” and “all.yaml” within “group_vars” directory keep the configuration data shared between devices within the cisco group and between all devices.

The SNMP community and NTP server variables were used in the jinja2 template used in the previous section. In this section, we use a list of NTP servers and authentication key information in the jinja2 configuration template.

NTP servers list are defined in all three files, “R1.yaml”, “cisco.yaml” and “all.yaml”.  But authentication key information including authentication protocol and authentication keys are only defied in “R1.yaml” file.

Python Nornir Automation Script using Ninja2 Template

This is the automation script that uses the jinja2 template configuration template to resolve and apply the configuration of each network device.

This is very similar to what we have demonstrated in the previous section. Here we use the template file, “ios-ntp-.j2”, to be resolved and applied to network devices. Shortly we will check inside the template file.

This script uses “load_yaml” to load data stored in “R1.yaml” into host variable with “hdata” key, “cisco.yaml” data into host variable with “ciscodata” key, and finally “all .yaml” into the host variable with “alldata” as the key.

Then the template file “ios-ntp-.j2” is converted to the final configuration list based on the loaded data and sent to the network devices using the nornir scrapli plugin.

Now let’s check inside the jinja2 configuration template used in automation script.

Jinja2 Template with Loop and Condition

This is the Jinja2 template we’ll be using in this section, and it includes all three of Jinja2’s main features: variable substitution, conditional, and looping.

We iterate over all the NTP server lists loaded in the “hdata”, “ciscodata”, and “alldata” keys and apply them to the network devices via the “ntp server” command.

Then with “if condition”, we check if NTP authentication information is defined in the loaded data.

When NTP authentication is defined, we enable NTP authentication with the command “ntp authenticate” and iterate over authentication keys with “for loop” and enable the keys.

This example is just a simple jinja2 template with variable substitution, conditionals and loops. But the jinja2 template has many other features that you can find with examples on the “ https://ttl255.com ” website.

Now let’s run the script to see how the list of NTP servers and optionally NTP authentication configured in “R1.yaml”, “cisco.yaml”, and “all.yaml” run on network devices.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• AnsibleFest
• Webinars & Training

• Docs »
• User Guide »
• Working With Playbooks »
• Using Variables
• Edit on GitHub

You are reading an unmaintained version of the Ansible documentation. Unmaintained Ansible versions can contain unfixed security vulnerabilities (CVE). Please upgrade to a maintained version. See the latest Ansible documentation .

Using Variables ¶

Creating valid variable names

Defining variables in inventory

Defining variables in a playbook

Defining variables in included files and roles

Using variables with Jinja2

Transforming variables with Jinja2 filters

Hey wait, a YAML gotcha

Variables discovered from systems: Facts

Disabling facts

Local facts (facts.d)

Ansible version

Caching Facts

Registering variables

Accessing complex variable data

Accessing information about other hosts with magic variables

Defining variables in files

Passing variables on the command line

Variable precedence: Where should I put a variable?

Scoping variables

Examples of where to set a variable

Using advanced variable syntax

While automation exists to make it easier to make things repeatable, all systems are not exactly alike; some may require configuration that is slightly different from others. In some instances, the observed behavior or state of one system might influence how you configure other systems. For example, you might need to find out the IP address of a system and use it as a configuration value on another system.

Ansible uses variables to help deal with differences between systems.

To understand variables you’ll also want to read Conditionals and Loops . Useful things like the group_by module and the when conditional can also be used with variables, and to help manage differences between systems.

The ansible-examples github repository contains many examples of how variables are used in Ansible.

Creating valid variable names ¶

Before you start using variables, it’s important to know what are valid variable names.

Variable names should be letters, numbers, and underscores. Variables should always start with a letter.

foo_port is a great variable. foo5 is fine too.

foo-port , foo port , foo.port and 12 are not valid variable names.

YAML also supports dictionaries which map keys to values. For instance:

You can then reference a specific field in the dictionary using either bracket notation or dot notation:

These will both reference the same value (“one”). However, if you choose to use dot notation be aware that some keys can cause problems because they collide with attributes and methods of python dictionaries. You should use bracket notation instead of dot notation if you use keys which start and end with two underscores (Those are reserved for special meanings in python) or are any of the known public attributes:

add , append , as_integer_ratio , bit_length , capitalize , center , clear , conjugate , copy , count , decode , denominator , difference , difference_update , discard , encode , endswith , expandtabs , extend , find , format , fromhex , fromkeys , get , has_key , hex , imag , index , insert , intersection , intersection_update , isalnum , isalpha , isdecimal , isdigit , isdisjoint , is_integer , islower , isnumeric , isspace , issubset , issuperset , istitle , isupper , items , iteritems , iterkeys , itervalues , join , keys , ljust , lower , lstrip , numerator , partition , pop , popitem , real , remove , replace , reverse , rfind , rindex , rjust , rpartition , rsplit , rstrip , setdefault , sort , split , splitlines , startswith , strip , swapcase , symmetric_difference , symmetric_difference_update , title , translate , union , update , upper , values , viewitems , viewkeys , viewvalues , zfill .

Defining variables in inventory ¶

Often you’ll want to set variables for an individual host, or for a group of hosts in your inventory. For instance, machines in Boston may all use ‘boston.ntp.example.com’ as an NTP server. The Working with Inventory page has details on setting Assigning a variable to one machine: host variables and Assigning a variable to many machines: group variables in inventory.

Defining variables in a playbook ¶

You can define variables directly in a playbook:

This can be nice as it’s right there when you are reading the playbook.

Defining variables in included files and roles ¶

As described in Roles , variables can also be included in the playbook via include files, which may or may not be part of an Ansible Role. Usage of roles is preferred as it provides a nice organizational system.

Using variables with Jinja2 ¶

Once you’ve defined variables, you can use them in your playbooks using the Jinja2 templating system. Here’s a simple Jinja2 template:

This expression provides the most basic form of variable substitution.

You can use the same syntax in playbooks. For example:

Here the variable defines the location of a file, which can vary from one system to another.

Inside a template you automatically have access to all variables that are in scope for a host. Actually it’s more than that – you can also read variables about other hosts. We’ll show how to do that in a bit.

ansible allows Jinja2 loops and conditionals in templates, but in playbooks, we do not use them. Ansible playbooks are pure machine-parseable YAML. This is a rather important feature as it means it is possible to code-generate pieces of files, or to have other ecosystem tools read Ansible files. Not everyone will need this but it can unlock possibilities.

More information about Jinja2 templating

Transforming variables with Jinja2 filters ¶

Jinja2 filters let you transform the value of a variable within a template expression. For example, the capitalize filter capitalizes any value passed to it; the to_yaml and to_json filters change the format of your variable values. Jinja2 includes many built-in filters and Ansible supplies many more filters .

Hey wait, a YAML gotcha ¶

YAML syntax requires that if you start a value with {{ foo }} you quote the whole line, since it wants to be sure you aren’t trying to start a YAML dictionary. This is covered on the YAML Syntax documentation.

This won’t work:

Do it like this and you’ll be fine:

Variables discovered from systems: Facts ¶

There are other places where variables can come from, but these are a type of variable that are discovered, not set by the user.

Facts are information derived from speaking with your remote systems. You can find a complete set under the ansible_facts variable, most facts are also ‘injected’ as top level variables preserving the ansible_ prefix, but some are dropped due to conflicts. This can be disabled via the INJECT_FACTS_AS_VARS setting.

An example of this might be the IP address of the remote host, or what the operating system is.

To see what information is available, try the following in a play:

To see the ‘raw’ information as gathered:

This will return a large amount of variable data, which may look like this on Ansible 2.7:

In the above the model of the first disk may be referenced in a template or playbook as:

Similarly, the hostname as the system reports it is:

Facts are frequently used in conditionals (see Conditionals ) and also in templates.

Facts can be also used to create dynamic groups of hosts that match particular criteria, see the Importing Modules documentation on group_by for details, as well as in generalized conditional statements as discussed in the Conditionals chapter.

Disabling facts ¶

If you know you don’t need any fact data about your hosts, and know everything about your systems centrally, you can turn off fact gathering. This has advantages in scaling Ansible in push mode with very large numbers of systems, mainly, or if you are using Ansible on experimental platforms. In any play, just do this:

Local facts (facts.d) ¶

New in version 1.3.

As discussed in the playbooks chapter, Ansible facts are a way of getting data about remote systems for use in playbook variables.

Usually these are discovered automatically by the setup module in Ansible. Users can also write custom facts modules, as described in the API guide. However, what if you want to have a simple way to provide system or user provided data for use in Ansible variables, without writing a fact module?

“Facts.d” is one mechanism for users to control some aspect of how their systems are managed.

Perhaps “local facts” is a bit of a misnomer, it means “locally supplied user values” as opposed to “centrally supplied user values”, or what facts are – “locally dynamically determined values”.

If a remotely managed system has an /etc/ansible/facts.d directory, any files in this directory ending in .fact , can be JSON, INI, or executable files returning JSON, and these can supply local facts in Ansible. An alternate directory can be specified using the fact_path play keyword.

For example, assume /etc/ansible/facts.d/preferences.fact contains:

This will produce a hash variable fact named general with asdf and bar as members. To validate this, run the following:

And you will see the following fact added:

And this data can be accessed in a template/playbook as:

The local namespace prevents any user supplied fact from overriding system facts or variables defined elsewhere in the playbook.

The key part in the key=value pairs will be converted into lowercase inside the ansible_local variable. Using the example above, if the ini file contained XYZ=3 in the [general] section, then you should expect to access it as: {{ ansible_local['preferences']['general']['xyz'] }} and not {{ ansible_local['preferences']['general']['XYZ'] }} . This is because Ansible uses Python’s ConfigParser which passes all option names through the optionxform method and this method’s default implementation converts option names to lower case.

If you have a playbook that is copying over a custom fact and then running it, making an explicit call to re-run the setup module can allow that fact to be used during that particular play. Otherwise, it will be available in the next play that gathers fact information. Here is an example of what that might look like:

In this pattern however, you could also write a fact module as well, and may wish to consider this as an option.

Ansible version ¶

New in version 1.8.

To adapt playbook behavior to specific version of ansible, a variable ansible_version is available, with the following structure:

Caching Facts ¶

As shown elsewhere in the docs, it is possible for one server to reference variables about another, like so:

With “Fact Caching” disabled, in order to do this, Ansible must have already talked to ‘asdf.example.com’ in the current play, or another play up higher in the playbook. This is the default configuration of ansible.

To avoid this, Ansible 1.8 allows the ability to save facts between playbook runs, but this feature must be manually enabled. Why might this be useful?

With a very large infrastructure with thousands of hosts, fact caching could be configured to run nightly. Configuration of a small set of servers could run ad-hoc or periodically throughout the day. With fact caching enabled, it would not be necessary to “hit” all servers to reference variables and information about them.

With fact caching enabled, it is possible for machine in one group to reference variables about machines in the other group, despite the fact that they have not been communicated with in the current execution of /usr/bin/ansible-playbook.

To benefit from cached facts, you will want to change the gathering setting to smart or explicit or set gather_facts to False in most plays.

Currently, Ansible ships with two persistent cache plugins: redis and jsonfile.

To configure fact caching using redis, enable it in ansible.cfg as follows:

To get redis up and running, perform the equivalent OS commands:

Note that the Python redis library should be installed from pip, the version packaged in EPEL is too old for use by Ansible.

In current embodiments, this feature is in beta-level state and the Redis plugin does not support port or password configuration, this is expected to change in the near future.

To configure fact caching using jsonfile, enable it in ansible.cfg as follows:

fact_caching_connection is a local filesystem path to a writeable directory (ansible will attempt to create the directory if one does not exist).

fact_caching_timeout is the number of seconds to cache the recorded facts.

Registering variables ¶

Another major use of variables is running a command and registering the result of that command as a variable. When you execute a task and save the return value in a variable for use in later tasks, you create a registered variable. There are more examples of this in the Conditionals chapter.

For example:

Results will vary from module to module. Each module’s documentation includes a RETURN section describing that module’s return values. To see the values for a particular task, run your playbook with -v .

Registered variables are similar to facts, with a few key differences. Like facts, registered variables are host-level variables. However, registered variables are only stored in memory. (Ansible facts are backed by whatever cache plugin you have configured.) Registered variables are only valid on the host for the rest of the current playbook run. Finally, registered variables and facts have different precedence levels .

When you register a variable in a task with a loop, the registered variable contains a value for each item in the loop. The data structure placed in the variable during the loop will contain a results attribute, that is a list of all responses from the module. For a more in-depth example of how this works, see the Loops section on using register with a loop.

If a task fails or is skipped, the variable still is registered with a failure or skipped status, the only way to avoid registering a variable is using tags.

Accessing complex variable data ¶

We already described facts a little higher up in the documentation.

Some provided facts, like networking information, are made available as nested data structures. To access them a simple {{ foo }} is not sufficient, but it is still easy to do. Here’s how we get an IP address:

OR alternatively:

Similarly, this is how we access the first element of an array:

Accessing information about other hosts with magic variables ¶

Whether or not you define any variables, you can access information about your hosts with the Special Variables Ansible provides, including “magic” variables, facts, and connection variables. Magic variable names are reserved - do not set variables with these names. The variable environment is also reserved.

The most commonly used magic variables are hostvars , groups , group_names , and inventory_hostname .

hostvars lets you access variables for another host, including facts that have been gathered about that host. You can access host variables at any point in a playbook. Even if you haven’t connected to that host yet in any play in the playbook or set of playbooks, you can still get the variables, but you will not be able to see the facts.

If your database server wants to use the value of a ‘fact’ from another node, or an inventory variable assigned to another node, it’s easy to do so within a template or even an action line:

groups is a list of all the groups (and hosts) in the inventory. This can be used to enumerate all hosts within a group. For example:

A frequently used idiom is walking a group to find all IP addresses in that group.

You can use this idiom to point a frontend proxy server to all of the app servers, to set up the correct firewall rules between servers, etc. You need to make sure that the facts of those hosts have been populated before though, for example by running a play against them if the facts have not been cached recently (fact caching was added in Ansible 1.8).

group_names is a list (array) of all the groups the current host is in. This can be used in templates using Jinja2 syntax to make template source files that vary based on the group membership (or role) of the host:

inventory_hostname is the name of the hostname as configured in Ansible’s inventory host file. This can be useful when you’ve disabled fact-gathering, or you don’t want to rely on the discovered hostname ansible_hostname . If you have a long FQDN, you can use inventory_hostname_short , which contains the part up to the first period, without the rest of the domain.

Other useful magic variables refer to the current play or playbook, including:

New in version 2.2.

ansible_play_hosts is the full list of all hosts still active in the current play.

ansible_play_batch is available as a list of hostnames that are in scope for the current ‘batch’ of the play. The batch size is defined by serial , when not set it is equivalent to the whole play (making it the same as ansible_play_hosts ).

New in version 2.3.

ansible_playbook_python is the path to the python executable used to invoke the Ansible command line tool.

These vars may be useful for filling out templates with multiple hostnames or for injecting the list into the rules for a load balancer.

Also available, inventory_dir is the pathname of the directory holding Ansible’s inventory host file, inventory_file is the pathname and the filename pointing to the Ansible’s inventory host file.

playbook_dir contains the playbook base directory.

We then have role_path which will return the current role’s pathname (since 1.8). This will only work inside a role.

And finally, ansible_check_mode (added in version 2.1), a boolean magic variable which will be set to True if you run Ansible with --check .

Defining variables in files ¶

It’s a great idea to keep your playbooks under source control, but you may wish to make the playbook source public while keeping certain important variables private. Similarly, sometimes you may just want to keep certain information in different files, away from the main playbook.

You can do this by using an external variables file, or files, just like this:

This removes the risk of sharing sensitive data with others when sharing your playbook source with them.

The contents of each variables file is a simple YAML dictionary, like this:

It’s also possible to keep per-host and per-group variables in very similar files, this is covered in Organizing host and group variables .

Passing variables on the command line ¶

In addition to vars_prompt and vars_files , it is possible to set variables at the command line using the --extra-vars (or -e ) argument. Variables can be defined using a single quoted string (containing one or more variables) using one of the formats below

key=value format:

Values passed in using the key=value syntax are interpreted as strings. Use the JSON format if you need to pass in anything that shouldn’t be a string (Booleans, integers, floats, lists etc).

JSON string format:

vars from a JSON or YAML file:

This is useful for, among other things, setting the hosts group or the user for the playbook.

Escaping quotes and other special characters:

Ensure you’re escaping quotes appropriately for both your markup (e.g. JSON), and for the shell you’re operating in.:

In these cases, it’s probably best to use a JSON or YAML file containing the variable definitions.

Variable precedence: Where should I put a variable? ¶

A lot of folks may ask about how variables override another. Ultimately it’s Ansible’s philosophy that it’s better you know where to put a variable, and then you have to think about it a lot less.

Avoid defining the variable “x” in 47 places and then ask the question “which x gets used”. Why? Because that’s not Ansible’s Zen philosophy of doing things.

There is only one Empire State Building. One Mona Lisa, etc. Figure out where to define a variable, and don’t make it complicated.

However, let’s go ahead and get precedence out of the way! It exists. It’s a real thing, and you might have a use for it.

If multiple variables of the same name are defined in different places, they get overwritten in a certain order.

Here is the order of precedence from least to greatest (the last listed variables winning prioritization):

command line values (eg “-u user”) role defaults 1 inventory file or script group vars 2 inventory group_vars/all 3 playbook group_vars/all 3 inventory group_vars/* 3 playbook group_vars/* 3 inventory file or script host vars 2 inventory host_vars/* 3 playbook host_vars/* 3 host facts / cached set_facts 4 play vars play vars_prompt play vars_files role vars (defined in role/vars/main.yml) block vars (only for tasks in block) task vars (only for the task) include_vars set_facts / registered vars role (and include_role) params include params extra vars (always win precedence)

Basically, anything that goes into “role defaults” (the defaults folder inside the role) is the most malleable and easily overridden. Anything in the vars directory of the role overrides previous versions of that variable in namespace. The idea here to follow is that the more explicit you get in scope, the more precedence it takes with command line -e extra vars always winning. Host and/or inventory variables can win over role defaults, but not explicit includes like the vars directory or an include_vars task.

Tasks in each role will see their own role’s defaults. Tasks defined outside of a role will see the last role’s defaults.

Variables defined in inventory file or provided by dynamic inventory.

Includes vars added by ‘vars plugins’ as well as host_vars and group_vars which are added by the default vars plugin shipped with Ansible.

When created with set_facts’s cacheable option, variables will have the high precedence in the play, but will be the same as a host facts precedence when they come from the cache.

Within any section, redefining a var will overwrite the previous instance. If multiple groups have the same variable, the last one loaded wins. If you define a variable twice in a play’s vars: section, the second one wins.

The previous describes the default config hash_behaviour=replace , switch to merge to only partially overwrite.

Group loading follows parent/child relationships. Groups of the same ‘parent/child’ level are then merged following alphabetical order. This last one can be superceeded by the user via ansible_group_priority , which defaults to 1 for all groups. This variable, ansible_group_priority , can only be set in the inventory source and not in group_vars/ as the variable is used in the loading of group_vars/.

Another important thing to consider (for all versions) is that connection variables override config, command line and play/role/task specific options and keywords. See Controlling how Ansible behaves: precedence rules for more details. For example, if your inventory specifies ansible_user: ramon and you run:

This will still connect as ramon because the value from the variable takes priority (in this case, the variable came from the inventory, but the same would be true no matter where the variable was defined).

For plays/tasks this is also true for remote_user . Assuming the same inventory config, the following play:

will have the value of remote_user overwritten by ansible_user in the inventory.

This is done so host-specific settings can override the general settings. These variables are normally defined per host or group in inventory, but they behave like other variables.

If you want to override the remote user globally (even over inventory) you can use extra vars. For instance, if you run:

the lola value is still ignored, but ansible_user=maria takes precedence over all other places where ansible_user (or remote_user ) might be set.

A connection-specific version of a variable takes precedence over more generic versions. For example, ansible_ssh_user specified as a group_var would have a higher precedence than ansible_user specified as a host_var.

You can also override as a normal variable in a play:

Scoping variables ¶

You can decide where to set a variable based on the scope you want that value to have. Ansible has three main scopes:

Global: this is set by config, environment variables and the command line Play: each play and contained structures, vars entries (vars; vars_files; vars_prompt), role defaults and vars. Host: variables directly associated to a host, like inventory, include_vars, facts or registered task outputs

Examples of where to set a variable ¶

Let’s show some examples and where you would choose to put what based on the kind of control you might want over values.

First off, group variables are powerful.

Site-wide defaults should be defined as a group_vars/all setting. Group variables are generally placed alongside your inventory file. They can also be returned by a dynamic inventory script (see Working With Dynamic Inventory ) or defined in things like Red Hat Ansible Tower from the UI or API:

Regional information might be defined in a group_vars/region variable. If this group is a child of the all group (which it is, because all groups are), it will override the group that is higher up and more general:

If for some crazy reason we wanted to tell just a specific host to use a specific NTP server, it would then override the group variable!:

So that covers inventory and what you would normally set there. It’s a great place for things that deal with geography or behavior. Since groups are frequently the entity that maps roles onto hosts, it is sometimes a shortcut to set variables on the group instead of defining them on a role. You could go either way.

Remember: Child groups override parent groups, and hosts always override their groups.

Next up: learning about role variable precedence.

We’ll pretty much assume you are using roles at this point. You should be using roles for sure. Roles are great. You are using roles aren’t you? Hint hint.

If you are writing a redistributable role with reasonable defaults, put those in the roles/x/defaults/main.yml file. This means the role will bring along a default value but ANYTHING in Ansible will override it. See Roles for more info about this:

If you are writing a role and want to ensure the value in the role is absolutely used in that role, and is not going to be overridden by inventory, you should put it in roles/x/vars/main.yml like so, and inventory values cannot override it. -e however, still will:

This is one way to plug in constants about the role that are always true. If you are not sharing your role with others, app specific behaviors like ports is fine to put in here. But if you are sharing roles with others, putting variables in here might be bad. Nobody will be able to override them with inventory, but they still can by passing a parameter to the role.

Parameterized roles are useful.

If you are using a role and want to override a default, pass it as a parameter to the role like so:

This makes it clear to the playbook reader that you’ve made a conscious choice to override some default in the role, or pass in some configuration that the role can’t assume by itself. It also allows you to pass something site-specific that isn’t really part of the role you are sharing with others.

This can often be used for things that might apply to some hosts multiple times. For example:

In this example, the same role was invoked multiple times. It’s quite likely there was no default for name supplied at all. Ansible can warn you when variables aren’t defined – it’s the default behavior in fact.

There are a few other things that go on with roles.

Generally speaking, variables set in one role are available to others. This means if you have a roles/common/vars/main.yml you can set variables in there and make use of them in other roles and elsewhere in your playbook:

There are some protections in place to avoid the need to namespace variables. In the above, variables defined in common_settings are most definitely available to ‘something’ and ‘something_else’ tasks, but if “something’s” guaranteed to have foo set at 12, even if somewhere deep in common settings it set foo to 20.

So, that’s precedence, explained in a more direct way. Don’t worry about precedence, just think about if your role is defining a variable that is a default, or a “live” variable you definitely want to use. Inventory lies in precedence right in the middle, and if you want to forcibly override something, use -e .

If you found that a little hard to understand, take a look at the ansible-examples repo on GitHub for a bit more about how all of these things can work together.

Using advanced variable syntax ¶

For information about advanced YAML syntax used to declare variables and have more control over the data placed in YAML files used by Ansible, see Advanced Syntax .

An introduction to playbooks

Conditional statements in playbooks

Jinja2 filters and their uses

Looping in playbooks

Playbook organization by roles

Best practices in playbooks

List of special variables

Have a question? Stop by the google group!

#ansible IRC chat channel

About variable overriding behavior of Jinja2

While working on a site made with flask microframework, one of the things that kept me puzzled for hours was Jinja2’s variable overriding behavior ( variable scoping, as the pedantic will call it ). To better demonstrate my case, I am going to make two jinja templates. The first one is the base or parent template, base.html -

The child template, page.html inherits from the base.html and tries to override the slogan variable -

The output of base.html is as expected -

And the output of page.html is -

Surprise! The slogan did not get replaced in the child template! I wont dare to comment about the intuition behind this behavior, since front-end development is not my speciality. Anyway, proper clarifications came from an old documentation of jinja2 . Here’s a relevent excerpt from it:

What is happenning here is that, the slogan is first set by page.html , then again set by base.html , notice the ordering! The way to achieve the intended behavior is to change the slogan assignment in the base.html , so that it can capture the assignment done by the page.html . Here’s a version of base.html that works -

# Jinja2: Conditional Statements

Conditional statements [1] perform different computations or actions depending on whether a programmer-specified boolean condition evaluates to true or false . Simply speaking, the if-else construct in your code is generally referred to as conditional statements.

Instead of dealing with if...else conditionals in the Flask code, you can directly embed them into the Jinja2 templates. With the default syntax, control structures appear inside {% ... %} blocks.

# Basic Comparisons

Define a template named conditionals_basics.html in your Flask project's /templates folder. In this example, we'll render different lists of products depending on the name of different companies provided by a user-defined variable called company . Here, we'll directly embed the conditional statements into the template.

Notice how we added conditional blocks to the template via {% %} blocks in line 6, 14 and 22. The {% endif %} block in line 25 marks the end of the conditional block. Now let's create an endpoint named /conditionals-basics/ in your Flask project's app.py file to render the template:

(opens new window) on your browser. You should be able to see the following list:

Now change the variable company in the app.py from Apple to Microsoft and see what happens. Your app.py should look like this:

(opens new window) on your brwoser should show a different list of products.

If you assign a different name other than Apple and Microsoft to the variable company , you'll see a "No product available" page.

# Checking Truthy / Falsy Values

You can also use conditional statements to test if a variable is truthy [2] (or falsy ) and take action based on that. For a variable to be truthy , it has to be defined, not empty and not false. Make a new template named conditionals_truthy.html and add the following contents to it:

The above template first checks whether the variable user is truthy and then it also checks whether attribute user.username is also a truthy value. If both of the conditions are true, the template returns the name defined in the user class.

Let's define an endpoint named /conditionals-truthy in the app.py file:

In line 8, we've defined a class called User . The class takes a single argument username . An instance of the class has been created in line 18. Here, the value of the argument username = "Adam" is truthy .

(opens new window) in your browser. You should see the template gets rendered like this:

Now if you instantiate the class User with a falsy value - for example, an empty string "" - the greeting statement of the template won't be rendered. Here, your app.py will look like this:

(opens new window) , you'll notice that the greeting statement hasn't been rendered.

# Conclusion

In this post, you've learned how to perform basic comparisons in Jinja2 templates using conditional statements. You've also seen how you can leverage truthy and falsy values of variables to control your logic directly from the Jinja2 template.

Enjoyed this article? You'll love the complete video course!

• Complete video lessons for each topic
• Get the most out of your learning
• Build a portfolio of projects
• Get one-on-one help from a real person

Get the video course

(opens new window) ↩︎

← Data Structures in Jinja2 Jinja2: Loops →