salt.modules.saltcheck

A module for testing the logic of states and highstates

codeauthor:William Cannon <william.cannon@gmail.com>
maturity:new

Saltcheck provides unittest like functionality requiring only the knowledge of salt module execution and yaml. Saltcheck uses salt modules to return data, then runs an assertion against that return. This allows for testing with all the features included in salt modules.

In order to run state and highstate saltcheck tests, a sub-folder in the state directory must be created and named saltcheck-tests. Tests for a state should be created in files ending in *.tst and placed in the saltcheck-tests folder. tst files are run through the salt rendering system, enabling tests to be written in yaml (or renderer of choice), and include jinja, as well as the usual grain and pillar information. Like states, multiple tests can be specified in a tst file. Multiple tst files can be created in the saltcheck-tests folder, and should be named the same as the associated state. The id of a test works in the same manner as in salt state files and should be unique and descriptive.

Usage

Example file system layout:

/srv/salt/apache/
    init.sls
    config.sls
    saltcheck-tests/
        init.tst
        config.tst
        deployment_validation.tst

Tests can be run for each state by name, for all apache/saltcheck/*.tst files, or for all states assigned to the minion in top.sls. Tests may also be created with no associated state. These tests will be run through the use of saltcheck.run_state_tests, but will not be automatically run by saltcheck.run_highstate_tests.

salt '*' saltcheck.run_state_tests apache,apache.config
salt '*' saltcheck.run_state_tests apache check_all=True
salt '*' saltcheck.run_highstate_tests
salt '*' saltcheck.run_state_tests apache.deployment_validation

Saltcheck Keywords

module_and_function:
(str) This is the salt module which will be run locally, the same as salt-call --local <module>. The saltcheck.state_apply module name is special as it bypasses the local option in order to resolve state names when run in a master/minion environment.
args:
(list) Optional arguments passed to the salt module
kwargs:
(dict) Optional keyword arguments to be passed to the salt module
assertion:
(str) One of the supported assertions and required except for saltcheck.state_apply
expected-return:
(str) Required except by assertEmpty, assertNotEmpty, assertTrue, assertFalse. The return of module_and_function is compared to this value in the assertion.
assertion_section:
(str) Optional keyword used to parse the module_and_function return. If a salt module returns a dictionary as a result, the assertion_section value is used to lookup a specific value in that return for the assertion comparison.
print_result:
(bool) Optional keyword to show results in the assertEqual, assertNotEqual, assertIn, and assertNotIn output. Defaults to True.
pillar-data:
(dict) Optional keyword for passing in pillar data. Intended for use in potential test setup or teardown with the saltcheck.state_apply function.
skip:
(bool) Optional keyword to skip running the individual test

Sample Cases/Examples

Basic Example

echo_test_hello:
  module_and_function: test.echo
  args:
    - "hello"
  kwargs:
  assertion: assertEqual
  expected-return:  'hello'

Example with jinja

{% for package in ["apache2", "openssh"] %}
{# or another example #}
{# for package in salt['pillar.get']("packages") #}
test_{{ package }}_latest:
  module_and_function: pkg.upgrade_available
  args:
    - {{ package }}
  assertion: assertFalse
{% endfor %}

Example with setup state including pillar

setup_test_environment:
  module_and_function: saltcheck.state_apply
  args:
    - common
  pillar-data:
    data: value

verify_vim:
  module_and_function: pkg.version
  args:
    - vim
  assertion: assertNotEmpty

Example with skip

package_latest:
  module_and_function: pkg.upgrade_available
  args:
    - apache2
  assertion: assertFalse
  skip: True

Example with assertion_section

validate_shell:
  module_and_function: user.info
  args:
    - root
  assertion: assertEqual
  expected-return: /bin/bash
  assertion_section: shell

Example suppressing print results

validate_env_nameNode:
  module_and_function: hadoop.dfs
  args:
    - text
    - /oozie/common/env.properties
  expected-return: nameNode = hdfs://nameservice2
  assertion: assertNotIn
  print_result: False

Supported assertions

  • assertEqual
  • assertNotEqual
  • assertTrue
  • assertFalse
  • assertIn
  • assertNotIn
  • assertGreater
  • assertGreaterEqual
  • assertLess
  • assertLessEqual
  • assertEmptyassertNotEmpty

Warning

The saltcheck.state_apply function is an alias for state.apply. If using the ACL system saltcheck.* might provide more capability than intended if only saltcheck.run_state_tests and saltcheck.run_highstate_tests are needed.

salt.modules.saltcheck.run_highstate_tests(saltenv=None)

Execute all tests for states assigned to the minion through highstate and return results

CLI Example:

salt '*' saltcheck.run_highstate_tests
salt.modules.saltcheck.run_state_tests(state, saltenv=None, check_all=False)

Execute all tests for a salt state and return results Nested states will also be tested

Parameters:
  • state (str) -- the name of a user defined state
  • check_all (bool) -- boolean to run all tests in state/saltcheck-tests directory

CLI Example:

salt '*' saltcheck.run_state_tests postfix,common
salt.modules.saltcheck.run_test(**kwargs)

Execute one saltcheck test and return result

Parameters:arg test (keyword) --

CLI Example:

salt '*' saltcheck.run_test
    test='{"module_and_function": "test.echo",
           "assertion": "assertEqual",
           "expected-return": "This works!",
           "args":["This works!"] }'
salt.modules.saltcheck.state_apply(state_name, **kwargs)

Runs state.apply with given options to set up test data. Intended to be used for optional test setup or teardown

Reference the state.apply module documentation for arguments and usage options

CLI Example:

salt '*' saltcheck.state_apply postfix
salt.modules.saltcheck.update_master_cache(saltenv=u'base')

Updates the master cache onto the minion - transfers all salt-check-tests Should be done one time before running tests, and if tests are updated Can be automated by setting "auto_update_master_cache: True" in minion config

CLI Example:

salt '*' saltcheck.update_master_cache