Building great HMI's on the i.MX RT-1060
This guide is a modification of the "How To Create Graphics with i.MX RT" course taught at the 2019 NXP technology days. Our goal is to help people get Storyboard applications up and running on the i.MX RT 1060 with FreeRTOS and the MCUXpresso tool.
To complete this guide you will require the following:
- Storyboard 6.0, download the evaluation from www.cranksoftware.com
- Storyboard i.MX RT1060 Real-time executive installer, contact firstname.lastname@example.org
- This course content archive
- NXP i.MX RT 1060 EVK and Display kit.
The integration guide found in the installer will take you through the steps required to download the BSP and MCUXpresso environment. You will then configure a FreeRTOS project to run Storyboard on the i.MX RT 1060 evk board and display. When you can successfully flash and run a FreeRTOS project on your hardware you are ready to begin building your application in Storyboard.
You should already be familiar with the MCUXpresso environment after following the integration guide found in the i.MX RT 1060 installer. For the purposes of the next few steps, this guide assumes that you are using the hello_storyboard.h file included in the course content archive. This file is a sample Storyboard application which blinks the text "hello storyboard" there is a dependency on the animation plugin so please ensure it is enabled in your sbengine_plugins.h file. Start by extracting the course content somewhere on your computer, this course will assume you extract the content to your desktop.
Testing Our Setup
Step 1 - Check hardware dip switches
Make sure the hardware is configured to boot from flash. Check the dip switches on the underside of the board and ensure that are set to “Off, Off, On, Off”.
Plug the i.MX RT-1060 into your laptop.
Step 2 - Launch MCUXpresso
Step 3 - Build and flash
Select the blue debug icon in the toolbar.
A dialog will appear asking you to select a debug probe. Leave the default selected one and press OK.
You will see the application build and then the flash progress in the bottom left corner of the IDE status bar.
When flashing is complete the program will hit a breakpoint on the first line of main. Press the debugger resume button in the toolbar or F8 to continue.
You should now see “Hello Storyboard!” blinking on your RT-1060 display.
When you are done close the debug session with the stop button.
Hint: The debugger uses sockets so if it doesn’t close down properly you might need to use buttons labelled 2 and 3 to clean up all debug sessions and reboot the debug probe.
Creating a Storyboard Application from a PSD File
Importing a Photoshop file and adding screen transitions.
Step 1 - Open Designer
Launch Storyboard Designer. If this is your first time opening Storyboard you will be greeted with the welcome screen.
Step 2 - Import a Photoshop file
Once you close the welcome panel you should see the empty editor. Import a PSD file by selecting the PS icon in the toolbar or press the “New Project from Photoshop Content” button in the Application Model Panel.
Step 3 - Import PSD Options
A dialog titled “Storyboard PSD Import” should open. Press the browse button and select the following PSD.
Leave everything else as is and press Finish.
Step 4 - Explore your new project
After the import process has completed you should a workspace like this. I will refer to the numbered sections as follows.
- Main editor
- Application Model View
- Actions/Animation/Images Panel
- Properties View
Step 5 - Adding a Screen Transitions
Let's add a screen transition, select the Start button. Right click > Add Action.
Step 6 - Configure your transition
Nothing in Storyboard happens without an event to trigger an action. So we need to pick an event to trigger the screen transition. Select the “Touch” event in the event list. Then select
Select the “Screen Transition: Path” action in the action list. Note: Your MCUXpresso project will need to be configured to use this transition type, do so in sbengine_plugins.h. Animated effects require extra memory, on systems without extra memory available these should be avoided.
Fill out the transition parameters. Here we can choose where we want to transition to and how. Select “cycle” as the screen and direction as “Right”.
Step 7 - Simulate and test
Once you have completed the transitions let's test it out. Press the simulate button in the toolbar.
If you are done early hook up transitions to move backward through your application. Use the “BACK” buttons and in the transition parameters use the direction “left”.
Creating an Animation
Step 1 - Create an animation
Storyboard allows you to bootstrap an animation with the handy “Animation Record” feature. Press the green camera icon situated at the right end of the toolbar.
The icon will turn red and the main editor window will have a red highlight.
Step 2 - Create an animation step
Storyboard will now record all the changes you make and convert them to an animation. Select the start button on the menu screen.
Set the alpha property to 0
Press the Animation snapshot icon to turn this change into an animation “step”. You will notice a label in the bottom right corner of the main editor update to say ”Recording.. [Frame 2]”
Now that the Start button is transparent we want to make sure it’ marked as hidden so it doesn’t receive any touch input. In the Application Model View toggle the “visible” property (eye icon). Then take another animation snapshot. You will now be on “Frame 3”.
Now that we’ve hidden the start button we can do something interesting with the buttons underneath. Slide them out horizontally. Since this is the last step we don’t need to take another snapshot. Press the animation record button and it will finalize your animation.
You will be presented with a dialog. Give your animation a meaningful name and select the “Use Absolute Start Values” option. Then press OK.
Your animation will automatically open in the timeline (Action/Animation Panel area). You can preview it by pressing the animation preview button. Each step can be selected, and edited in the properties panel if you want to make changes.
Step 3 - Trigger the animation
When you are happy with your animation leave the timeline by pressing the filmstrip icon in the timeline panel. Then the Actions tab.
Select the Start button, right click and add an action.
Select a “Touch” event and an “Animation” as the action. Using the drop down for the name property of the animation select your newly created animation (StartMenuOpen). Click finish.
In the actions panel you will notice that you have two touch events bound to the start button. Right click on the Screen transition event and select delete. We will move this action to wash button that will be visible at the end of the animation.
Select the wash button in the Application Model view. Right click and add the screen path transition action. Refer to the notes in the “adding a screen transition”.
Simulate and test. You should now have an animation and screen transitions!
Preparing Your application for export
Step 1 - Create a resource export configuration.
Now that we have an application with screen transitions and animations we should test it out on the RT 1060. To do this we need to create an export configuration. Press the Export Configurations icon in the toolbar.
Step 2 - Create a new resource export configuration.
The resource export configuration panel will open in the Main editor section. Create a new configuration for the RT 1060 by pressing the new configuration button. Give it a meaningful name (eg: 1060) and press OK.
Step 3 - Resource Export Options.
Now that we have a new configuration we can setup how the application and its assets will be exported for the 1060. Select the following options, notice how the metrics update based on what you change.
- Storage Type:Virtual Filesystem
- Font Export Format: TTF
- Image ExportFormat: Direct RGB8888
- Image Start Alignment: 4
- Script Export Format: Raw
Press CTRL + S to save the configuration and close the panel.
Step 4 - Configure Application export.
With this new configuration we can now export our application into a C header file for inclusion in our FreeRTOS project. Press the “Storyboard Application Export configurations” button in the toolbar. Or Run > Storyboard Application Export configurations.
Step 5 - Export the application.
The Storyboard Application Export Configurations dialog will have opened up. Select the following options. In the “Packager” field select “Storyboard Embedded Resource Header”. In the “Resource Export Configurations” field select your new created configuration. (eg. 1060). Press Apply and then Run. the dialog will close and you should see a success dialog in a few seconds.
Step 6 - Find the exported header file.
The export will have created a header file called “sbengine_model.h”. This can be found in your project directory. Select the Navigator tab in the Application Model view then expand your project. Right-click on “sbengine_model.h” and copy it.
Deploying your application to hardware
Step 1 - Add the model to the FreeRTOS Project.
Switch back to MCUXpresso and expand the project directory. Paste sbengine_model.h into the source directory.
Step 2 - Include your new header
Open sbengine_task.c and include your new sbengine_model.h.
Step 3 - Clean and flash to hardware
Clean your project and launch with debugging.
The flashing process will take longer since we are transferring the application and all assets this time.
Reskinning your UI
Step 1 - Re-import a PSD
Storyboard allows you to quickly and easily reskin/rebrand your GUI via the PSD Re-import function. Select the PSD reimport button to begin.
Step 2 - Pick your PSD
Select the following PSD in the "RT1060-KnowledgeGuide" folder on the Desktop.
Leave the other settings default and select Next.
Step 3 - Manage incoming content
Storyboard gives you the ability to manage how the new content is coming into your application. This is useful if the new PSD has different naming then the original. Storyboard will do its best to match content based on names however sometimes you will need to manually pair elements. In this case our PSD files follow the same conventions so you can press “Select All” then “Finish
Step 4 - Simulate
You now have a new look and feel for your application and it was imported in only a few clicks. You’ve also retained all the work and behaviour you implemented. Simulate your project and test it out.
Step 5 - Optimize your assets
Now that you’ve reskinned your UI we should take a moment to optimize our assets so we take up less space on the RT-1060. This will also help speed up deployment. Open the metrics tab in the Properties Panel to get a view of our current resource consumption. Select the 1060 configuration.
Step 6 - Condense project images
Now open your images tab in the Action Panel area. Notice that there are excess images from the original design. In the metrics view, you can see how much space they take up in flash memory.
Select the resource clean up icon in the panel toolbar. This will open up a wizard that will scan your project for unreferenced images. Note: This tool will not catch images referenced in scripts. So you may need to remove some images that are suggested.
This application doesn’t have any images referenced via script so we can press Next. The tool will also scan for unused fonts but won’t find any in this project. Press finish and opt to delete the images. Notice all the space you’ve saved.
Step 7 - Export the application
Re-export your application for deployment to hardware. You can do this with a single click by pressing the Storyboard Application Export button in the toolbar. This will run the last export configuration.
You should see a message saying sbengine_model.h was successfully exported.
Copy the header file from the Project directory.
Step 8 - Update the MCUXpresso project
Switch back to MCUXpresso and paste “sbengine_model.h” into the source directory. Overwrite the previous file.
Clean and build then flash the project to the i.MX RT-1060.