It is also possible to perform a headless running of Validator tests using a pre-configured simulation target and the names of tests. After creating the tests in Designer and creating a simulation launch, local or remote, a user can run a test or a group of tests from the command line or scripting environment.
All of the Storyboard executables will be located in:
PATH_TO_INSTALL/Storyboard_Designer/storyboard
directories, though on each desktop platform they are named slightly differently
The tests argument is optional. If it is ommitted, all Validator tests for this project will be run.
Windows:
..../Storyboard.exe -application sb.validator model=<PathToGDEFile> tests="test_name[:test_name_2:test_name_3:...]" config=<ConfigurationName>
Mac:
..../Storyboard.app/Contents/MacOS/Storyboard -application sb.validator model=<PathToGDEFile> tests="test_name[:test_name_2:test_name_3:...]" config=<ConfigurationName>
Linux:
..../Storyboard -application sb.validator model=<PathToGDEFile> tests="test_name[:test_name_2:test_name_3:...]" config=<ConfigurationName>
Note: Storyboard requires a display, so to run in a true headless environment a Virtual frame buffer needs to be setup.
Xvfb :1 -ac -screen 0 1024x768x8 export DISPLAY=:1 ..../storyboard -application sb.validator model=<PathToGDEFile> tests="test_name[:test_name_2:test_name_3:...]" config=<ConfigurationName>
Where the model is the full path to the Storyboard Designer model file. This model file will be used to search for tests to run. The tests_to_run option specifies the tests that need to be run. The config_name parameter specifies the name of the launch configuration to run. This needs to be provided so that Validator knows whether or not to run the tests locally or remotely.
Additional Options
The -data option Is used to specify the workspace to be used when performing the export. This workspace should not be opened by other instances of Storyboard. This is helpful for Storyboard to find your launch configurations in your project.
It is specified as follows:
..../Storyboard.exe -application sb.validator -data <PathToWorkspace> model=<PathToGDEFile> tests="test_name[:test_name_2:test_name_3:...]" config=<ConfigurationName>
The -consoleLog option is used to enable additional logging to the console. The -noExit option is used to prevent the application from closing automatically after performing the export. This is sometimes useful to keep the log console open post-mortem.
Introduction This section will provide a brief introduction to Jenkins and demonstrate how Storyboard Validator can be triggered as part of your continuous integration workflow.
Follow https://www.jenkins.io/doc/pipeline/tour/getting-started/ for instructions on how to download, install and launch Jenkins, if you already have a Jenkins environment setup, you can skip this step.
Integration of Validator into Jenkins requires two steps. One windows batch command (or shell command if running Jenkins in a Unix environment), and one post-build action to publish JUnit test result report. It is assumed that you already have a project created. In this example, we started with a Freestyle project
Prerequisites The JUnit plugin for Jenkins https://plugins.jenkins.io/junit
Adding Validator Launch
-
Add a build step to execute Windows Batch Command
Adjust the following command to suit your needs, change the Storyboard installation, GDE model file path, launch configuration name and Storyboard workspace as required:
"C:\Program Files\Crank_Software\Storyboard_Designer\Crank Storyboard.exe" -application sb.validator model="%USERPROFILE%\storyboard_workspace\Thermostat\Thermostat.gde" config=Thermostat.gdeSimulator xml="%WORKSPACE%\test_report.xml" -data "%USERPROFILE%\storyboard_workspace"
-
Add a post-build action for Publish JUnit test result report.
Set the test filter to test_report.xml
-
From the dashboard you can trigger a new build for verification:
The Project will maintain a history of the build and test results:
The test results can be examined:
The presentation of data from within Storyboard Validator is more detailed and easier to work with than the experience in the Jenkins user interface, but it is useful to be able to dig into some results without having to open Storyboard.
When one or more tests fail the build is marked as unstable:
For more information on working with Jenkins, visit https://www.jenkins.io/doc/