NXP RT10XX EVK Storyboard Integration with MCUXpresso

This tutorial outlines how to integrate the Storyboard Engine into an existing project and run that project using the contents of this installer package with the following configuration:

  • FreeRTOS

  • An NXP RT10xx EVK board, which is one of the RT1050 EVK B, RT1060 EVK, or RT1064 EVK boards

  • 480x272 display resolution

The tutorial describes the process of configuring a project to support the Storyboard Engine and how to integrate the Storyboard Engine into the project.  Finally, it provides instructions on how to deploy the project to the hardware so that you can see the Storyboard Engine running on the target.

For knowledge base entries or updated instructions related to integrating Storyboard on the hardware referenced in this tutorial, visit our MCU documentation page.

Prerequisites

To follow this tutorial, you need the following:

  • A RT10xx EVK board with a 480x272 display

  • MCUXpresso IDE v11.3.0 [Build 5222] [2021-01-11] or higher

  • The  SDK_2.9.1_EVK-MIMXRT10xx SDK or higher from the MCUXpresso SDK Builder site, where xx represents 50, 60 or 64

  • The 7.0 version of the freertos-mcuxpresso-cortexm7-swrender-DFP-obj Crank Storyboard runtime

  • The contents of the installation package

The folder where the package that contains this tutorial is referred to as the <PACKAGE_DIR>. The folder for the support files is based on the type of package that is installed.

<PACKAGE_DIR>/integration-support-files/<BOARD_TYPE>

Where <BOARD_TYPE> is one of RT1050, RT1060 or RT1064.

The support files folder is referred to as <SUPPORT_FILES_DIR> from this point forward.

Also, before integrating the engine into a project, reading the Storyboard documentation is advised.

Setting up MCUXpresso

The MCUXpresso IDE can be downloaded from this website. To download the product, a user account must be created on the NXP website. Once the product has been downloaded, install it on the system.

Note

When copying paths that include quotes (“”), those quotes may be pasted in the MCUXpresso tool incorrectly. After pasting paths that include quotes, ensure that the resulting quotes are correct.

Installing a BSP/SDK for RT10xx EVK

The NXP SDKs are created through the Dashboard tool and are customizable for a particular IDE toolchain and feature set. The following steps are needed to create the correct setup:

  • Configure and download the latest SDK (currently 2.9.1 for the chosen RT10xx EVK) from the Dashboard tool. Select the RT10xx EVK board from the list of boards and then build the SDK for that board.

  • Request the MCUXpresso IDE flavor of the SDK.

  • Be sure to include Amazon FreeRTOS (select the Amazon-FreeRTOS Kernel component by clicking the "Add software component" button under "Select Optional Middleware”)

Open MCUXpresso and import the NXP SDK for the RT10xx EVK by pointing at the downloaded NXP SDK zip file. This can be done by clicking on the 'Installed NXP SDK's' tab in the MCUXpresso IDE and dragging and dropping the NXP SDK ZIP file to the view.

The SDK can also be extracted to a convenient folder to add additional files from the SDK to our project later on. This folder is referred to as <RT10xx_NXPSDK_ROOT> later in this tutorial.

Getting a BSP/SDK project ready for Storyboard integration

Once the RT10xx EVK SDK has been installed, we are ready to configure the project. We base this on an existing sample project for simplicity:

  • Create a new workspace e.g.: workspace_iMXRT10xx

  • Import the freertos_hello SDK Example using the MCUXpresso IDE - Quickstart Panel (MIMXRT1060->rtos_examples->freertos_hello)

  • Copy & rename this project freertos_sbengine

  • Ensure that the project configuration is set to the debug build which should be the default: Project->Build Configurations->Set Active->Debug

Note

When importing the freertos_hello example project, make sure that the “Set library type (and hosting variant)” selection is set to “Newlib (semihosted)” in the drop-down box. If this is not done, errors could occur when compiling FreeRTOS after changing the FreeRTOSConfig.h file. The top of the dialog should look like this:

NXP_SDK_Import_Wizard.png

Build the freertos_sbengine project, download the image to the board by selecting the “Debug” link in the “MCUXpresso IDE - Quick Start Panel” which is usually located in the bottom right corner of the IDE. This step is to ensure everything is installed and configured correctly before making modifications.

MCUXpresso-1.png

Copy the files contained in the <SUPPORT_FILES_DIR>/board folder to the project folder board. Click “yes” when prompted to overwrite the pin.c and pin.h files.

Additional driver files need to be added to the project to provide graphics and touchscreen support. The folder where the driver source files are located depends on the target hardware being used. For the RT1050 target hardware this folder is:

  • <RT10xx_NXPSDK_ROOT>/devices/MIMXRT1052

For the RT1060 hardware, this folder is:

  • <RT10xx_NXPSDK_ROOT>/devices/MIMXRT1062

For the RT1064 hardware, this folder is:

  • <RT10xx_NXPSDK_ROOT>/devices/MIMXRT1064

This folder is referred to as <SDK_DRIVER_ROOT_DIR>.

To manually configure the additional SDK drivers required, add (copy to project rather than link) the following files to the drivers folder in the project workspace:

  • <SDK_DRIVER_ROOT_DIR>/drivers/fsl_cache.c

  • <SDK_DRIVER_ROOT_DIR>/drivers/fsl_cache.h

  • <SDK_DRIVER_ROOT_DIR>/drivers/fsl_elcdif.c

  • <SDK_DRIVER_ROOT_DIR>/drivers/fsl_elcdif.h

  • <SDK_DRIVER_ROOT_DIR>/drivers/fsl_lpi2c.c

  • <SDK_DRIVER_ROOT_DIR>/drivers/fsl_lpi2c.h

  • <SDK_DRIVER_ROOT_DIR>/drivers/fslfsl_pxp.c

  • <SDK_DRIVER_ROOT_DIR>/drivers/fsl_pxp.h

Create a project folder called touchpanel. Copy the following files into the touchpanel folder:

  • <RT10xx_NXPSDK_ROOT>/components/ft5406_rt/fsl_ft5406_rt.c

  • <RT10xx_NXPSDK_ROOT>/components/ft5406_rt/fsl_ft5406_rt.h

To add the touchpanel include paths to the compiler include search paths, right click on the project and select “Properties”. In the properties pane, expand the “C/C++Build” tree item in the pane that is on the left-hand side of the screen. From the expanded tree item, select “Settings”.  In the settings pane on the right-hand side, expand the tree item “MCU C Compiler” and select ”Includes”.

Add the following folders to the “Include paths (-I)” pane using the icon with a “+” symbol in it:

“${workspace_loc:/${ProjName}/touchpanel}”

Right-click on the touchpanel folder and select the menu item “Properties” from the popup menu.

From the properties dialog, select the entry “C/C++ Build” from the list on the left-hand side.  Uncheck the box marked “Exclude resource from build”.

The Storyboard Engine requires a different memory manager configuration than the default one used by FreeRTOS. To enable this, remove or delete heap_4.c from the project folder amazon-freertos/FreeRTOS/portable and add in heap_3.c which implements a FreeRTOS allocation API on top of malloc() and free() library calls.

Copy the heap_3.c file from:

  • <RT10xx_NXPSDK_ROOT>/rtos/amazon-freertos/freertos_kernel/portable/MemMang/heap_3c

and add this file to the project folder amazon-freertos/FreeRTOS/portable.

Finally create a project folder called configuration. There is a specific memory configuration file for the board that is being used. If the board is a RT1050, the memory configuration file is:

<SUPPORT_FILES_DIR>/configuration/MCUXpresso/MIMXRT1052xxxxx_flexspi_nor.ld

If the board is a RT1060, the memory configuration file is:

<SUPPORT_FILES_DIR>/configuration/MCUXpresso/MIMXRT1062xxxxx_flexspi_nor.ld

If the board is a RT1064, the memory configuration file is:

<SUPPORT_FILES_DIR>/configuration/MCUXpresso/MIMXRT1064xxxxx_flexspi_nor.ld

Copy the memory configuration file to the configuration folder in the project.

In the linker settings, switch from a managed linker configuration to using the linker file that was copied into the project. To do this, right click on the project and select “Properties”. In the properties pane expand the “C/C++Build” tree item in the pane that is on the left-hand side of the screen. From the expanded tree item, select “Settings”. In the settings pane on the right-hand side, expand the tree item “MCU C Linker” and select ”Managed Linker Script”. Uncheck the checkbox “Manage linker script” and set the following settings based on board type:

RT1050

Linker script = MIMXRT1052xxxxx_flexspi_nor.ld

Script path= ../configuration

RT1060

Linker script = MIMXRT1062xxxxx_flexspi_nor.ld

Script path= ../configuration

RT1064

Linker script = MIMXRT1064xxxxx_flexspi_nor.ld

Script path= ../configuration

At this point, a build clean and build all should be performed on the project and those operations should complete successfully.

Exporting a model C/C++ header from Storyboard Designer

The next step is to export an application from Storyboard Designer. This is done here because the header is needed to complete the compile once the Storyboard Engine is integrated into the project. The header for the Storyboard application is exported now for use later.

For more information on exporting a model from Storyboard Designer, read the Exporting and Running on your Embedded Target chapter in the Storyboard Designer documentation.

The C/C++ header file that is exported contains all the model definitions, scripts, fonts, and other graphical assets used by the Storyboard application in binary form. Once the target platform has been configured with Storyboard engine, display, and input support, updates to the Storyboard model can be exported to the sbengine_model.h C/C++ header file in the project directly.

Configuring the resources for export

To export for an embedded target configuration we need to create a custom resource export configuration. To do this, click on the resource export configuration editor toolbar button. It is shown in this image:

resource-export-editor-button.png

In the new editor, create a virtual filesystem configuration for the resources. This can be done by first creating a new configuration. To do this, click on the “New” button pictured below:

new-configuration-button.png

You can name your custom resource configuration anything you want. In this example we set options for a virtual filesystem, so we use the name “vfs” to help identify what the custom configuration was created for. In the dialog that appears, type “vfs” into the name and click on the “OK” button. After clicking on the “OK” button, the editor should look like this

resource-editor.png

Note that the default sizes for the “Font Glyph Map Width” and the “Image Map Width” are tied to the width of the application, so these values may be different.

For the purpose of this tutorial, a couple of options need to be changed. The first option is “Storage Type”. This option needs to be set to “Virtual Filesystem”.  It looks like this:

storage-type.png

The second option to change is the “Image Export Format” option. For the purposes of deploying to this target, set this option to “Direct RGB8888”. This tells the runtime to draw images directly out of storage and not use any heap for decoding the image. When that option is set, it looks like this:

image-export-type.png
Configuring fonts

A choice can be made on how fonts are going to be rendered when using a virtual filesystem configuration. The Freetype font manager can be used to render fonts. This font rendering option takes up more heap memory but it uses less storage space. Alternatively, the Storyboard bitmap font manager can be used to render fonts. This font rendering option uses very little heap memory but requires more storage space. Note that the choice is one or the other. The two font managers cannot be mixed.

The Freetype font manager is recommended for use with the full configuration of the engine, and the Storyboard bitmap font manager is recommended for use with the Lite configuration of the engine, but both font managers work with both configurations.

Freetype font manager setup

To use the Freetype font engine to render fonts, set the “Font Export Format” to “TTF”. It looks like this:

font-export-ttf.png
Storyboard bitmap font manager setup

To use the Storyboard bitmap font engine to render fonts, set the “Font Export Format” to “Bitmap”. It looks like this:

font-export-bitmap.png

When “Bitmap” is selected as the “Font Export Format”, the number of bits to use for the alpha values representing the pixels of the glyph can be chosen. The default is 8-bits per pixel in the glyph.  The other options are 4-bits, 2-bits and 1-bit. Reducing the number of bits that are used to represent the alpha values representing the pixels of the glyph reduces the size of the glyph structures, but it also reduces the quality of the rendered font.

Once a choice has been made about the font manager to be used to render font glyphs, save the configuration and close the editor.

Exporting the model C/C++ header file

Open the export dialog by clicking on the following button in the main Designer toolbar:

export-configuration.png

The main areas to change on the dialog are to set the “Packager” setting to “Storyboard Embedded Resource Header C/C++” and the “Resource Export Configuration” dialog to “vfs”.  After changing those the dialog will look like this:

export-settings.png

Note that if the Freetype font manager is used, the “Bitmap Fonts Header” section is disabled.

Click the “Apply” button and the “Run” button. If you are not exporting the model directly into your MCUXpresso project, you should make a note of the location of the sbengine_model.h file.

Integrating Storyboard Engine

The first step to integrating the engine into the project is to copy the engine binaries to the project by doing the following:

  • Copy the <PACKAGE_DIR>/freertos-mcuxpresso-cortexm7-swrender-DFP-obj folder to the root of the freertos_sbengine project

  • The freertos-mcuxpresso-cortexm7-swrender-DFP-obj folder should appear in the project once it is refreshed

The FreeRTOSConfig.h file in the source folder needs to have the following configuration defines changed from this:

#define configFRTOS_MEMORY_SCHEME               4
#define configUSE_TIME_SLICING                  0
#define configENABLE_BACKWARD_COMPATIBILITY     0
#define configTICK_RATE_HZ                      ((TickType_t)200)
#define configUSE_NEWLIB_REENTRANT              0

to this:

#define configFRTOS_MEMORY_SCHEME               3
#define configUSE_TIME_SLICING                  1
#define configENABLE_BACKWARD_COMPATIBILITY     1
#define configTICK_RATE_HZ                      ((TickType_t)1000)
#define configUSE_NEWLIB_REENTRANT              1

Note

A value of 1000 has been chosen to give timers a resolution of 1ms. This can be changed to some other value, however the greal library needs to be rebuilt as part of the project. See Appendix A for instructions on how to rebuild the greal library as part of the project. If the tick size is changed, this may cause animations to look like they are running slowly.

Adding in project settings
Set the preprocessor defines

To build the Storyboard Engine components as part of the project, some preprocessor defines need to be set up. To do this, right click on the project and select “Properties”. In the properties pane, expand the “C/C++Build” tree item in the pane that is on the -hand side of the screen. From the expanded tree item, select “Settings”. In the settings pane on the right-hand side, expand the tree item “MCU C Compiler” and select ”Preprocessor”.

Add the following defines to the “Defined symbols (-D)” pane:

SKIP_SYSCLK_INIT
XIP_BOOT_HEADER_DCD_ENABLE=1
GRE_TARGET_OS_freertos
GRE_TARGET_CPU_cortexm7
GRE_TARGET_TOOLCHAIN_mcuxpresso
GRE_FEATURE_VFS_RESOURCES

Also, change the define:

SDK_DEBUGCONSOLE=0

to:

SDK_DEBUGCONSOLE=1
Add Storyboard include folders

For the compiler to find the Storyboard Engine header files and libraries, we need to add the engine paths to the search paths for headers and libraries.

To add the engine include paths to the compiler include search paths, right click on the project and select “Properties”. In the properties pane expand the “C/C++Build” tree item in the pane that is on the left-hand side of the screen. From the expanded tree item, select “Settings”.  In the settings pane on the right-hand side, expand the tree item “MCU C Compiler” and select ”Includes”.

Add the following folders to the “Include paths (-I)” pane using the icon with a “+” symbol in it:

“${workspace_loc:/${ProjName}/freertos-mcuxpresso-cortexm7-swrender-DFP-obj/include}”

“${workspace_loc:/${ProjName}/freertos-mcuxpresso-cortexm7-swrender-DFP-obj/include/freertos}”

Each folder must be added as its own separate entity.

Setup Storyboard libraries

To add the engine library paths to the compiler library search paths, right-click on the project and select “Properties”. In the properties pane, expand the “C/C++Build” tree item in the pane that is on the left-hand side of the screen. From the expanded tree item, select “Settings”. In the settings pane on the right-hand side, expand the tree item “MCU Linker” and select ”Libraries”.

In the pane labelled “Library search path (-L)” add the following entries:

"${workspace_loc:/${ProjName}/freertos-mcuxpresso-cortexm7-swrender-DFP-obj/lib}"

"${workspace_loc:/${ProjName}/freertos-mcuxpresso-cortexm7-swrender-DFP-obj/plugins}”

In the same settings dialog, libraries that are needed are added to the pane labelled “Libraries (-l)”. For a Storyboard Lite configuration of the engine, add the following libraries:

gre
greal
sbimage
sbvfs
gre-plugin-animation
gre-plugin-ccallback
gre-plugin-poly
gre-plugin-circle
gre-plugin-timer
gre-plugin-logger
gre-plugin-sbimage-soil
render-plugin-swrender
swrender
soil
logging-util

To use a complete configuration of the engine, add these libraries:

gre
greal
sbimage
sbvfs
gre-plugin-animation
gre-plugin-lua
gre-plugin-poly
gre-plugin-circle
gre-plugin-timer
gre-plugin-logger
gre-plugin-screen-path
gre-plugin-greio
gre-plugin-sbimage-soil
render-plugin-swrender
logging-util
greio
sblua
soil
swrender
png
sbz

Note

Only pick one of the library configurations, either full or Lite and it must match the package configuration that you are using. Also note that the library order matters.

When using the Freetype font manager, include the following libraries before the png and sbz libraries:

font-plugin-freetype
sbfreetype

For the Storyboard bitmap font manager, we include the following library:

font-plugin-sbfont

Note

Only add one of the font managers. It must match the choices made when exporting the header file.

To make sure that potential hard faults caused by misaligned memory access are avoided, add the compiler option -mno-unaligned-access to the compiler other flags. These are accessed by right clicking on the project and selecting “Properties”. In the properties pane expand the “C/C++Build” tree item in the pane that is on the left-hand side of the screen. From the expanded tree item, select “Settings”. In the settings pane on the right-hand side, expand the tree item “MCU Linker” and select ”Miscellaneous”. In the pane labelled “Other objects” add the option:

-mno-unaligned-access
Launching the engine

To launch the Storyboard Engine as a task, first copy the following files to the source folder in the project:

<SUPPORT_FILES_DIR>/source/sbengine_plugins.h

<SUPPORT_FILES_DIR>/source/sbengine_task_pxp.c

Also copy <SUPPORT_FILES_DIR>/applications/HelloWorld.h to the source folder. To change the app, copy the header file of the application to the source folder and change the line:

#include “HelloWorld.h”

to include the header file that contains that app that should be run.

Note

If using the CoffeeApp demo, the source and header files that are in <PACKAGE_DIR>/Demos/applications/CoffeeApp/source need to be copied to the projects source folder

Once those files have been added in, open the file source/freertos_hello.c in the project. Place this definition just after the define for the hello_task_PRIORITY, which can now be removed as it is no longer needed:

void sbengine_main_task(void *argument);

Replace the Hello_task code:

if (xTaskCreate(hello_task, "Hello_task", configMINIMAL_STACK_SIZE + 10, NULL, hello_task_PRIORITY, NULL) != pdPASS)

Replace with the sbengine_main_task creation code:

if (xTaskCreate(sbengine_main_task, "sbengine", 8192, NULL, configMAX_PRIORITIES / 2, NULL) != pdPASS)

Ensure that the following headers are included within freertos_hello.c:

#include "board.h"
#include "peripherals.h"

There are two additional functions required to be inserted into the initialization sequence in the main() function. These functions are BOARD_InitSemcPins() and BOARD_InitBootPeripherals().

The initial board configuration and setup is performed early in the main() function that can be found in the freertos_hello.c file. After adding in the additional functions, the start of the main() function looks like this:

/*!
 * @brief Application entry point.
 */
int main(void)
{
	/* Init board hardware. */
	BOARD_ConfigMPU();
	BOARD_InitPins();
	BOARD_InitSemcPins();
	BOARD_InitBootPeripherals();
	BOARD_BootClockRUN();
	BOARD_InitDebugConsole();
     :
Adding input support

In the freertos_hello.c file containing the main() function, place the following code after the definition for the Storyboard main() task:

void sbengine_input_task(void *argument);#include “HelloWorld.h”

Add the Storyboard input handling task creation code block before the call to VTaskStartScheduler:

if (xTaskCreate(sbengine_input_task, "StoryboardInput", 2048, NULL, configMAX_PRIORITIES / 2, NULL) != pdPASS)
    {
        PRINTF("Task creation failed!.\r\n");
        while (1);
    }    

The sbengine_input_task function is defined in the sbengine_task_pxp.c file and it is the task that is read from the touch device and emits press, motion, and release events to the Storyboard engine.

Deploying and debugging on the target

Build the freertos_sbengine project, download the image to the board by selecting the “Debug” link in the “MCUXpresso IDE - Quick Start Panel” which is usually located in the bottom right corner of the IDE.

MCUXpresso-1.png

Conclusion

After following these steps, you should be able to integrate the Storyboard engine into your RT10xx EVK project, deploy and flash it to the target.

For more information about Storyboard on the RT10xx EVK you can visit the Crank Help Center.  If you require further assistance, contact support@cranksoftware.com.

Appendix A: Building the Storyboard abstraction library

In most cases, it is not necessary to build the source to the Storyboard abstraction layer. Compiling the source to this library is only needed if there are changes being made to the OS priority settings or to the system tick rate. This appendix outlines how to build the source code of the Storyboard abstraction layer that is shipped as part of this package.

Replace precompiled library with library source

Step 1

Remove the greal library from the list of libraries that should be linked by the linker during compilation of the project. To remove the greal library from the list of libraries that are linked to the project, right click on the project and select “Properties”. In the properties pane, expand the “C/C++Build” tree item in the pane that is on the left-hand side of the screen. From the expanded tree item, select “Settings”. In the settings pane on the right-hand side, expand the tree item “MCU Linker” and select ”Libraries”.

In the pane marked “Libraries (-l)”, remove the entry:

greal

This library was added in the section “Setup Storyboard Libraries”.

Step 2

Add in the Storyboard greal library source to the project so that it is built as part of the project.  To do this, expand the freertos-mcuxpresso-cortexm7-swrender-DFP-obj folder in the project. Then expand the source folder and expand the lib folder. Right-click on the greal folder and select the menu item “Properties'' from the popup menu.

Step 3

From the properties dialog, select the entry “C/C++ Build” from the list on the left-hand side.  Uncheck the box marked “Exclude resource from build”.

Step 4

Perform a clean build on the project, followed by a full build.

Appendix B: Importing shipped demos into Designer

This package ships with two demo applications that can be imported into Storyboard Designer.

To import these projects into Storyboard Designer, select File->Import->General->Existing Projects into Workspace. In the dialog that pops up, navigate to the “Select Root Directory” box to the folder <PACKAGE_DIR>/Demos/applications and select the two applications that are populated in the “Projects” list.

Make sure the “Copy projects into workspace” checkbox is selected and then click on “Finish”.

The projects are now in your Storyboard Designer workspace.

Appendix C: Feature lists for Storyboard Engine configurations

The table below lists a comparison of the features supported by the different configurations of the Storyboard Engine. Differences between the configurations have been highlighted in bold.

Feature name

Storyboard Engine

Storyboard Engine for MCUs

Storyboard Lite configuration

OpenGL Render Manager

Yes (1)

No

No

Freetype Font Support

Yes

Yes

Yes

Bitmap Font Support

No

Yes

Yes

3D Screen Transitions

Yes (1)

No

No

2D Screen Transitions

Yes

Yes

No

Lua Script Support

Yes

Yes

No

C Function Action

No (Lua Simulated)

Yes

Yes

Animations

Yes

Yes

Yes

Polygon Support

Yes

Yes

Yes

Circle Support

Yes

Yes

Yes

Rich Text Support

Yes

Yes

No

Image Support

Yes

Yes

Yes

Audio/Media Support

Yes

No

No

External Renderer Support

Yes

No

No

Storyboard IO support

Yes

Yes

No

Timer Support

Yes

Yes

Yes

3D Model Support

Yes (1)

No

No

Canvas Support

Yes

Yes

No

System Call Support

Yes

No

No

Event Redirect Support

Yes

No

No

Logging Support

Yes

Yes

Yes

Capture/Playback Support

Yes

Yes(2)

Yes(2)

Lua DOM support

Yes

Yes

No

Gesture Support

Yes

No

No

(1) Only where OpenGL hardware support is available

(2) Playback support only due to the virtual filesystem

Was this article helpful?
0 out of 0 found this helpful