Course: Graphics on the i.MX RT1060

360034653011-mceclip0.png

Building great HMI's on the i.MX RT-1060

This mini-course is a modification of the "How To Create Graphics with i.MX RT" course taught at the 2019 NXP technology days and can be used to help you get Storyboard applications up and running on the i.MX RT1060 with FreeRTOS and the MCUXpresso tool.

Prerequisites

The integration guide found in the installer will take you through the steps required to download the BSP and MCUXpresso environment. You will then configure a FreeRTOS project to run Storyboard on the i.MX RT1060 evk board and display. When you can successfully flash and run a FreeRTOS project on your hardware you are ready to begin building your application in Storyboard.

MCUXpresso Environment

You should already be familiar with the MCUXpresso environment after following the integration guide found in the i.MX RT1060 installer. For the purposes of the next steps, use the hello_storyboard.h file included in the course content archive. This file is a sample Storyboard application which blinks the text "hello storyboard" and has a dependency on the animation plugin so ensure it is enabled in your sbengine_plugins.h file. Start by extracting the course content onto your computer. For the purposes of this course, we will assume you extract the content to your desktop.

Testing the Setup

  1. Check hardware dip switches.

    Ensure the hardware is configured to boot from flash. Check the dip switches on the underside of the board and ensure that are set to “Off, Off, On, Off”. Plug the i.MX RT-1060 into your laptop.

    360034635672-mceclip1.png
  2. Launch MCUXpresso.

  3. Build and flash.

    Select the blue debug icon in the toolbar

    360034635792-mceclip3.png

    A dialog will appear asking you to select a debug probe. Leave the default and press OK.

    360034653311-mceclip4.png

    You will see the application build and then the flash progress in the bottom left corner of the IDE status bar.

    360034653351-mceclip5.png

    When flashing is complete the program will hit a breakpoint on the first line of main. Press the debugger resume button in the toolbar or F8 to continue.

    360034653411-mceclip6.png

    You should now see “Hello Storyboard!” blinking on your RT-1060 display.

    When you are done close the debug session with the stop button.

    360034635912-mceclip7.png

    Note

    The debugger uses sockets so if it doesn’t close down properly you might need to use buttons labelled 2 and 3 to clean up all debug sessions and reboot the debug probe.

Creating a Storyboard Application from a PSD File

Importing a Photoshop file and adding screen transitions.

  1. Launch Storyboard Designer. If this is your first time opening Storyboard you will be greeted with the welcome screen.

    360034654191-mceclip8.png
  2. Import a Photoshop file.

    Once you close the welcome panel you should see the empty editor. Import a PSD file by selecting the PS icon in the toolbar or press the “New Project from Photoshop Content” button in the Application Model Panel.

    360034636912-mceclip9.png
  3. Import PSD Options.

    A dialog titled “Storyboard PSD Import” should open. Press the browse button and select the following PSD.

    /Desktop/RT1060-KnowledgeGuide/Assets/washing_machine.psd

    Leave everything else as is and press Finish.

    360034657831-mceclip10.png
  4. Explore your new project.

    After the import process is complete you should see a workspace like this. Refer to the numbered sections as follows:

    • 1 - Main editor

    • 2 - Application Model View

    • 3 - Actions/Animation/Images Panel

    • 4 - Properties View

    360034640272-mceclip11.png

    Main editor

  5. To add a screen transition, right-click the Start button and select Add Action.

    360034640852-mceclip12.png
  6. Next, let's pick an event to trigger the screen transition. Select the Touch event in the event list.

    360034641192-mceclip13.png

    Select the Screen Transition: Path action.

    360034659691-mceclip14.png

    Note

    Your MCUXpresso project will need to be configured in sbengine_plugins.h to use this transition type. Animated effects require extra memory, so systems without available memory should be avoided.

    Fill out the transition parameters. Here we can choose where we want to transition to and how. Select “cycle” as the screen and direction as “Right”.

    360034660071-mceclip15.png
  7. Once you've completed the transitions, let's test it out. Press the simulate button on the toolbar.

    You can also hook up transitions to move backward through your application. Use the “BACK” buttons and in the transition parameters use the direction “left”.

    360034660451-mceclip16.png

Creating an Animation

  1. Create an animation.

    Storyboard allows you to bootstrap an animation with the handy “Animation Record” feature. Press the green camera icon situated at the right end of the toolbar.

    360034644212-mceclip17.png

    The icon will turn red and the main editor window will have a red highlight.

    360034644592-mceclip18.png
  2. Create an animation step.

    Storyboard will now record all the changes you make and convert them to an animation. Select the start button on the menu screen.

    360034663351-mceclip19.png

    Set the alpha property to 0

    360034645492-mceclip20.png

    Press the Animation snapshot icon to turn this change into an animation “step”. You will notice a label in the bottom right corner of the main editor update to say ”Recording.. [Frame 2]”

    360034645792-mceclip21.png

    Now that the Start button is transparent we want to make sure it’ marked as hidden so it doesn’t receive any touch input. In the Application Model View toggle the “visible” property (eye icon). Then take another animation snapshot. You will now be on “Frame 3”.

    360034646032-mceclip22.png

    Now that we’ve hidden the start button we can do something interesting with the buttons underneath. Slide them out horizontally. Since this is the last step we don’t need to take another snapshot. Press the animation record button and it will finalize your animation.

    360034664111-mceclip23.png

    You will be presented with a dialog. Give your animation a meaningful name and select the “Use Absolute Start Values” option. Then press OK.

    360034664391-mceclip24.png

    Your animation will automatically open in the timeline (Action/Animation Panel area). You can preview it by pressing the animation preview button. Each step can be selected, and edited in the properties panel if you want to make changes.

    360034646852-mceclip25.png
  3. Trigger the animation.

    When you are happy with your animation leave the timeline by pressing the filmstrip icon in the timeline panel. Then the Actions tab.

    360034649972-mceclip28.png

    Select the Start button, right click and add an action.

    360034650012-mceclip29.png

    Select a “Touch” event and an “Animation” as the action. Using the drop down for the name property of the animation select your newly created animation (StartMenuOpen). Click finish.

    360034668011-mceclip30.png

    In the actions panel you will notice that you have two touch events bound to the start button. Right click on the Screen transition event and select delete. We will move this action to wash button that will be visible at the end of the animation.

    360034668031-mceclip31.png

    Select the wash button in the Application Model view. Right click and add the screen path transition action. Refer to the notes in the “adding a screen transition”.

    360034650232-mceclip32.png

    Simulate and test. You should now have an animation and screen transitions!

Preparing your application for export

  1. Now that we have an application with screen transitions and animations we can test it out on the RT1060. To do this we need to create an export configuration. Press the Export Configurations icon in the toolbar.

    360034747871-mceclip0.png
  2. The resource export configuration panel will open in the Main editor section. Create a new configuration for the RT1060 by pressing the new configuration button. Give it a meaningful name (eg: 1060) and press OK.

    360034668151-mceclip34.png
  3. Now that we have a new configuration we can set up how the application and its assets will be exported for the RT1060. Select the following options and notice how the metrics update based on what you change:

    • Storage Type: Virtual Filesystem

    • Font Export Format: TTF

    • Image Expor Format: Direct RGB8888

    • Image Start Alignment: 4

    • Script Export Format: Raw

    Press CTRL + S to save the configuration and close the panel.

    360034668191-mceclip35.png
  4. With this new configuration we can now export our application into a C header file for inclusion in our FreeRTOS project. Press the Storyboard Application Export configuration button in the toolbar. Or Run > Storyboard Application Export Configurations.

    360034650412-mceclip36.png
  5. Select the following options in the Storyboard Application Export Configurations dialog. In the Packager field select Storyboard Embedded Resource Header. In the Resource Export Configurations field select your new configuration. (eg. 1060). Press Apply and then Run. the dialog will close and you should see a success dialog in a few seconds.

    360034650492-mceclip37.png
  6. The export will have created a header file called “sbengine_model.h”. This can be found in your project directory. Select the Navigator tab in the Application Model view then expand your project. Right-click on “sbengine_model.h” and copy it.

    360034668451-mceclip38.png

Deploying your application to hardware

Step 1 - Add the model to the FreeRTOS Project.

Switch back to MCUXpresso and expand the project directory. Paste sbengine_model.h into the source directory.

360034696731-mceclip39.png

Step 2 - Include your new header

Open sbengine_task.c and include your new sbengine_model.h.

360034697011-mceclip40.png

Step 3 - Clean and flash to hardware

Clean your project and launch with debugging.

The flashing process will take longer since we are transferring the application and all assets this time.

360034697071-mceclip41.png
360034697111-mceclip42.png

Reskinning your UI

  1. Storyboard allows you to quickly and easily reskin/rebrand your GUI via the PSD Re-import function. Select the PSD reimport button to begin.

    360034697211-mceclip43.png
  2. Select the following PSD in the "RT1060-KnowledgeGuide" folder on the Desktop.

    /Desktop/RT1060-KnowledgeGuide/Assets\washing_machine_reimport.psd

    Leave the other settings default and select Next.

    360034677112-mceclip44.png
  3. Storyboard gives you the ability to manage how the new content is coming into your application. This is useful if the new PSD has different naming then the original. Storyboard will do its best to match content based on names however sometimes you will need to manually pair elements. In this case our PSD files follow the same conventions so you can press “Select All” then “Finish

    360034697611-mceclip45.png
  4. You now have a new look and feel for your application and it was imported in only a few clicks. You’ve also retained all the work and behavior you implemented. Simulate your project and test it out.

    360034697871-mceclip46.png
  5. Now that you’ve reskinned your UI we should take a moment to optimize our assets so we take up less space on the RT-1060. This will also help speed up deployment. Open the metrics tab in the Properties Panel to get a view of our current resource consumption. Select the 1060 configuration.

    360034697931-mceclip47.png
  6. Now open your images tab in the Action Panel area. Notice that there are excess images from the original design. In the metrics view, you can see how much space they take up in flash memory.360034678132-mceclip48.png

    Select the resource clean up icon in the panel toolbar. This will open up a wizard that will scan your project for unreferenced images.

    Note

    This tool will not catch images referenced in scripts. So you may need to remove some images that are suggested.

    360034678212-mceclip49.png
  7. Re-export your application for deployment to hardware. You can do this with a single click by pressing the Storyboard Application Export button in the toolbar. This will run the last export configuration.360034682712-mceclip1.png

    You should see a message saying sbengine_model.h was successfully exported.360034702791-mceclip2.png

    Copy the header file from the Project directory.

    360034702931-mceclip3.png
  8. Switch back to MCUXpresso and paste sbengine_model.h into the source directory. Overwrite the previous file.

    360034683032-mceclip4.png

    Clean and build then flash the project to the i.MX RT1060.

    360034683152-mceclip5.png
Was this article helpful?
0 out of 0 found this helpful