SMACC Automated Run-time Testing

Testing Levels inside SMACC

Testing inside SMACC consists of 3 levels…

  • Core: Threading tests, signal handler/signal detector race conditions, SM startup, base-client class**..
  • Client: Mainly via xterm, cause main terminal is used, built around actions with a server, cycling through components/planners.
  • State Machine: Time, Destination (with counter) and Events.

Testing at the State Machine Level

Three basic test types at SM level.

First launch the machine…

  • Time, did the statemachine run for at least x minutes. (sm_atomic)
  • Destination, did the state actually reach this state. (sm_threesome or sm_calendar)
  • Events, which are tied to ROS logging.. (if FATAL or ERROR msgs, then…)

…Then shut down.

Centralized Test Node

We’ve built a centralized test node to improve continuous integration with a specified testing interface for the library, which should dramatically increase the code reuse problem that commonly becomes a problem…

This package is named smacc_runtime_test and contains a special ROS node called “smacc_runtime_test_node” that implements some “gtest/rostest” code.

https://github.com/reelrbtx/SMACC/tree/master/smacc_diagnostics/smacc_runtime_test

https://github.com/reelrbtx/SMACC/blob/master/smacc_diagnostics/smacc_runtime_test/src/smacc_runtime_test_node.cpp

YAML Test Configuration

We developed a “yaml description” where you set the conditions you want to test your state machine with…

Here is an example file:

state_machine_rosparam_ws: /SmAtomic
success_switch:
   - type: state_reached
     state_name: "sm_atomic::State2"
failure_switch:
   - type: timeout
     duration: 25.0 # sec


The smacc_runtime_test_node subscribes to “standard” smacc topics and reads from the “yaml test specification” to validate that all the succcess conditions are satisfied. The c++ code of this node is developed to easily add new success or failure policies.

How to

The process of adding smacc_runtime_tests to any new state machine sample is the following:

  • Copy the test folder that contains the .test and .yaml file to the new package from some existing package
  • There are two files:
    • the <sm_name>.yaml file: with the description of the test
    • the <sm_name>.test file: that references/include the main launch file of the current example
  • Rename the files to adapt to your current state machine names.
  • Make sure that your main launch file of your example (referenced from the test file) doES not use any graphical application  (such as gazebo or xterm). (Xterm typical usage can be disabled overloading some launch file arguments)
  • Replace the contents of these files to adapt to the names of your specific state machine (name of states, name of  the state machine, etc)
  •  Add a rostest dependency in the package.xml file <test_depend>rostest</test_depend>
  • Add 4 lines into the CMakeLists.txt to include the test into the building process. These are the lines…
if (CATKIN_ENABLE_TESTING)
  find_package(rostest REQUIRED)
  add_rostest(test/sm_atomic.test)
endif()

Testing at the Core Level

Testing at this level uses gtest

https://github.com/reelrbtx/SMACC/tree/master/test

https://github.com/reelrbtx/SMACC/blob/master/test/sm_coretest_z_bob_1/ssm/test/ssm_test.cpp

Testing at this level most-closely resembles testing at Boost Statechart…

https://github.com/boostorg/statechart/tree/develop/test

SMACC CI

We use Travis CI, via a docker shell and ros_industrial_ci to… – Pablo Add Here.