HOW TO WORK WITH ANSIBLE TEST

Reading Time: 4 minutes

The primary purpose of these tests is to enforce Ansible coding standards and requirements. ansible-test includes a variety of sanity tests to perform the code analysis, which can be found in the documentation.

Ansible Testing

As automation becomes crucial for more and more business cases, there is an increased need to test the automation code itself. This is where ansible-test comes in, Both ansible-core and ansible-base come packaged with a CLI tool called ansible-test, which can be collection developers to test their Collection and its content.

The ansible-test knows how to perform a wide variety of testing-related tasks, from linting module documentation and code to running unit and integration tests.

Types of tests

Ansible-test provides ways to run different types of tests on your Collections, broadly these tests are of types listed below:

  • Sanity Tests
  • Unit Tests
  • Integration Tests
  • Compile Tests

SANITY TESTS

Sanity tests consist of scripts and tools used to perform static code analysis. The main purpose of these tests is to apply Ansible coding standards and requirements.

Tests are run with ansible-test sanity. All available tests are run unless the --test option is used.

# Run all sanity tests
ansible-test sanity

# Run all sanity tests including disabled ones
ansible-test sanity --allow-disabled

# Run all sanity tests against against certain files
ansible-test sanity lib/ansible/modules/files/template.py

# Run all tests inside docker (good if you don't have dependencies installed)
ansible-test sanity --docker default

# Run validate-modules against a specific file
ansible-test sanity --test validate-modules lib/ansible/modules/files/template.py

UNIT TESTS

Unit tests are small isolation tests that target a particular library or module. Collection developers/administrators need to make sure that their code is unit tests. Ansible test provides a way to run and report unit tests for collections.

The Ansible unit tests can be use across the whole codebase by doing:

cd /path/to/ansible/source
source hacking/env-setup
ansible-test units --docker -v

Against a single file by doing:

ansible-test units --docker -v apt

Or against a specific Python version by doing:

ansible-test units --docker -v --python 2.7 apt

If you are running unit tests against things other than modules, such as module utilities, specify the whole file path:

ansible-test units --docker -v test/units/module_utils/basic/test_imports.py

INTEGRATION TESTS

These are end-to-end tests to check the functionality of the code path as expected and detect basic product changes to automate. In the context of the Ansible test, it basically looks like this:

  • Testing / Integration is where everything related to integration testing resides.
  • The test / integration / target directory contains all the test cases. Each test case is the role of Bare bone Ansible.

Non-destructive Tests

These tests will modify files in subdirectories, but will not do things that install or remove packages or things outside of those test subdirectories. They will also not reconfigure or bounce system services.

Run as follows for all POSIX platform tests executed by our CI system in a Fedora 34 container:

ansible-test integration shippable/ --docker fedora34

You can target a specific test as well, such as for individual modules:

ansible-test integration ping

You can use the -v option to make the output more verbose:

ansible-test integration lineinfile -vvv

Use the following command to list all the available targets:

ansible-test integration --list-targets

Destructive Tests

These tests are for the installation and removal of some trivial packages. You will likely want to devote these to a virtual environment, such as Docker. They won’t reformat your filesystem:

ansible-test integration destructive/ --docker fedora34

COMPILE TESTS

Compile tests check source files for valid syntax on all supported python versions

However, these tests can be run across the whole codebase by doing:

cd /path/to/ansible/source
source hacking/env-setup
ansible-test sanity --test compile

Against a single file by doing:

ansible-test sanity --test compile lineinfile

Or against a specific Python version by doing:

ansible-test sanity --test compile --python 2.7 lineinfile

Testing within GitHub & Azure Pipelines

When Pull Requests (PRs) are created they are tested using Azure Pipelines, a Continuous Integration (CI) tool. Results are at the end of every PR.

When Azure Pipelines detects an error and it can be linked back to a file that has been modified in the PR then the relevant lines will be added as a GitHub comment. For example:

The test `ansible-test sanity --test pep8` failed with the following errors:

lib/ansible/modules/network/foo/bar.py:509:17: E265 block comment should start with '# '

The test `ansible-test sanity --test validate-modules` failed with the following error:
lib/ansible/modules/network/foo/bar.py:0:0: E307 version_added should be 2.4. Currently 2.3

From the above example, we can see that --test pep8 and --test validate-modules have identified an issue. The commands given allow you to run the same tests locally to ensure you’ve fixed all issues without having to push your changes to GitHub and wait for Azure Pipelines, for example:

If you haven’t already got Ansible available, use the local checkout by running:

source hacking/env-setup

Then run the tests detailed in the GitHub comment:

ansible-test sanity --test pep8
ansible-test sanity --test validate-modules

Testing the Pull Request

At this point, you should be ready to begin testing!

Some ideas of what to test are:

  • Create a test Playbook with the examples in and check if they function correctly
  • Test to see if any Python backtraces returned (that’s a bug)
  • Test on different operating systems, or against different library versions

Run sanity tests

ansible-test sanity

Run unit tests

ansible-test units

To Run integration tests

ansible-test integration -v ping

Conclusion

As shown above, ansible-test can provide a lot of value testing Ansible Content Collections thoroughly.

If you’re a developer, one of the most valuable things you can do is look at the GitHub issues list and help fix bugs. We almost always prioritize bug fixing over feature development.

Even for non-developers, helping to test pull requests for bug fixes and features is still immensely valuable. Ansible users who understand writing playbooks and roles should be able to add integration tests and so GitHub pull requests with integration tests that show bugs in action will also be a great way to help.

Written by 

Deeksha Tripathi is a Software Intern at Knoldus Inc Software. She has a keen interest toward learning new technologies. Her practice area is Devops. When not working, she will be busy in listening music , and spending time with her family .

Leave a Reply