This artlcle discusses some tips, tricks, and best practices for developing Storyboard applications effectively in a Real-time Executable (RTExec or Lite) environment. Specifically, we will look at ways to split the ‘back-end’ codebase into discrete projects, application code and GUI models, for most efficient development on a day to day basis.
This same technique could be extended and adapted to allow multiple models to be flashed or for example to allow in-field partial model and code updates and fail-over in production.
It is assumed that an existing Storyboard enabled RTExec or Lite based project, application and using Storyboard runtime static libraries have been created and executed successfully in the standard configuration before attempting to modify the code to partition the flash.
First we will look at the make-up of a typical Storyboard RTExec or Lite based project where there is a ‘back-end’ application code component that deals with device hardware via BSP driver code and the state machines and tasks that collectively make-up the whatever the product ‘does’ - measuring stuff using sensors, control input from touch and hard-button, spinning motors, controlling valves etc.
Alongside the ’back-end’ code there are additional tasks related to driving the ‘front-end’ GUI using Storyboard. In a real-time executable environment this means additional RTOS tasks dedicated to driving the Storyboard runtime API:
-
the ‘sbengine_main’ taskhandles the rendering, framebuffer processing, and display initialization and update bridging the Storyboard callbacks and BSP drivers for GPIO an LCD Interfaces and uses Storyboard library resources and plug-ins
-
the ‘sbengine_input’ task handles brokering the Storyboard UI input events such as touch-events and hard-button interrupts, custom events with parametric data payload etc. , passing them to the ‘front-end’ UI as control and data input from the ‘back-end’ using the storyboard events API
-
it is usual in most real applications to also additional worker tasks which exploit Storyboard IO API event listeners and enables the reception and handling of Storyboard UI events as well as custom events with data as control from the ‘front-end’ GUI to the ‘back-end’ processing.
Lastly, there is the actual Storyboard model itself along with associated graphical assets, fonts and other data and script files. These are represented in the form of a resource header file (e.g sbengine_model.h) format exported directly from Storyboard Designer using our SBVFS structures.
Many embedded applications targeting a Microcontroller (MCU) class device such as an ARM Cortex M4/M7 or PIC32MZ do not typically have provision for a filesystem to store Storyboard model files, scripts and graphical assets.
In this case, we have developed a simple Storyboard Virtual File System (SBVFS) which allows developers to export using the Storyboard Designer application using the Export Configuration option Storyboard Embedded Resource Header (C/C++), default name sbengine_model.h.Here the Storyboard application model, graphical assets and fonts etc are compiled in the form of one or more (probably large) header files for inline inclusion in the application sources.
The structure of this exported resource file is described using the gre/sdk/sbresource_vfs.h header include.
We can see that the exported assets comprises several elements:
const char *sb_model; ///< The application model const sbvfs_resource_t *sb_resources; ///< The resource table const sb_ccallback_t *clist; ///< The SB Lite C callback functions list
Building and compiling this sbengine_model.h resource inline with the code and then downloading and flashing to the target board each time a code change is made or some debugging is attempted is great for simple applications however it can be very time consuming, particularly for applications with many graphical assets.
Once the development project progresses and the model has stabilized somewhat it would be far more efficient to enable the application code to be flashed independently of the Storyboard application model resources. This portion of the codebase only needs to be re-flashed if the Storyboard application is changed and re-exported to a new version of the sbengine_model.h file.
A convenient way of achieving a segmentation of the application flash usage is to modify the linker file to partition the flash storage into two regions, one for the ‘application code’ BOARD_FLASH and one for the ‘SBVFS model resources’ BOARD_FLASH_SBVFS.
The example below is based on a GCC based linker (.ld) file memory section for a 64MB (0x4000000 bytes) Flash device partitioned into 16MB and 48MB regions accordingly. Other compiler tools such as IAR EWARM will be modified in a similar way..:
MEMORY { /* Define each memory region */ BOARD_FLASH (rx) : ORIGIN = 0x60000000, LENGTH = 0x1000000 /* 16M bytes (alias Flash) */ BOARD_FLASH_SBVFS (rx) : ORIGIN = 0x61000000, LENGTH = 0x3000000 /* 48M (64-16M) bytes (alias Flash) */ BOARD_SDRAM_UPPER (rwx) : ORIGIN = 0x81E00000, LENGTH = 0x00200000 /* 2M bytes (alias RAM) */ BOARD_SDRAM_LOWER (rwx) : ORIGIN = 0x80000000, LENGTH = 0x01E00000 /* 30M bytes (alias RAM) */ SRAM_ITC (rwx) : ORIGIN = 0x0, LENGTH = 0x20000 /* 128K bytes (alias RAM2) */ SRAM_DTC (rwx) : ORIGIN = 0x20000000, LENGTH = 0x20000 /* 128K bytes (alias RAM3) */ SRAM_OC (rwx) : ORIGIN = 0x20200000, LENGTH = 0x40000 /* 256K bytes (alias RAM4) */ } /* Define a symbol for the top of each memory region */ __base_BOARD_FLASH = 0x60000000 ; /* BOARD_FLASH */ __base_Flash = 0x60000000 ; /* Flash */ __top_BOARD_FLASH = 0x60000000 + 0x1000000 ; /* 16M bytes */ __top_Flash = 0x60000000 + 0x1000000 ; /* 16M bytes */ __base_BOARD_FLASH_SBVFS = 0x61000000 ; /* BOARD_FLASH_SBVFS */ __base_SBVFS = 0x61000000 ; /* Flash */ __top_BOARD_FLASH_SBVFS = 0x61000000 + 0x3000000 ; /* 48M (64-16M) bytes */ __top_SBVFS = 0x61000000 + 0x3000000 ; /* 48M (64-16M) bytes */
We can now allocate and map several named memory subsections within the BOARD_FLASH_SBVFS region (.sbvfs) for organising and containing the model, resources and also a new root structure which can be used to reference these variable locations indirectly.
The SECTIONS part of the linker (.ld) file can now be expanded to include our new .sbvfs section definition and region mapping :
SECTIONS { .text : ALIGN(4) { *(.text*) *(.rodata .rodata.* .constdata .constdata.*) . = ALIGN(4); } > BOARD_FLASH /* Storyboard model and resources configured as filesystem from external flash */ .sbvfs : ALIGN(4) { FILL(0xff) __sbvfs_start__ = ABSOLUTE(.) ; KEEP(*(.sbvfs.root)) KEEP(*(.sbvfs.model)) KEEP(*(.sbvfs.resources)) __sbvfs_end__ = ABSOLUTE(.) ; } > BOARD_FLASH_SBVFS
Note the use of the KEEP directive here which ensures that the linker retains the symbols even if they are unused in the actual linked image. The symbol __sbvfs_start__ is needed here as it will be will exported and used within the C/C++ code as the fixed address of the .sbvfs.root structure shortly.
You should still be able to build your application code with the updated Flash partitioning, assuming that it will fit still in the smaller 16MB allocation, as the application code will require modification before the model resources can be split-out.
It is suggested to now build the application and observe the linker output in the .map file for our new (currently empty) sections and linker symbols:
.sbvfs 0x61000000 0x0 FILL mask 0xff 0x61000000 __sbvfs_start__ = ABSOLUTE (.) *(SORT_BY_ALIGNMENT(.sbvfs.root)) *(SORT_BY_ALIGNMENT(.sbvfs.model)) *(SORT_BY_ALIGNMENT(.sbvfs.resources)) 0x61000000 __sbvfs_end__ = ABSOLUTE (.)
NOTE: It is important to check that the BOARD_FLASH_SBVFS region is, in fact, empty to avoid inadvertently overwriting pre-flashed contents when flashing just the application code project.
The first change is to define a new structure type (sb_vfs_root_t) which will be used to create a constant index structure to de-reference the key constituents within the exported Storyboard model and resources.
Within the existing sbengine_task.c code we can add this structure definition:
#if defined(GRE_FEATURE_VFS_RESOURCES) #include <gre/sdk/sbresource_vfs.h> /* This structure is defined in <gre/sdk/sbresource_vfs.h> and should be included from there */ #if !defined(SB_CCALLBACK_STRUCT_DEFINED) typedef struct _sb_ccallback { const char *name; void (*c_callback)(gr_application_t *); } sb_ccallback_t; #endif /* This structure should ideally be defined in <gre/sdk/sbresource_vfs.h> and should be included from there */ #if !defined(SB_VFS_ROOT_STRUCT_DEFINED) typedef struct _sb_vfs_root { const char *sb_model; ///< The application model const sbvfs_resource_t *sb_resources; ///< The resource table const sb_ccallback_t *clist; ///< The SB Lite C callback functions list } sb_vfs_root_t; #endif
To enable the project to be conditionally built with either inline model resources or our new externally partitioned model resources in a separate project we will use a new compiler define added to the project options called APP_USE_EXTERNAL_SBVFS which is set to:
-
0 for in-line in the same project or
-
1 for partitioned in an external project
Within sbengine_task.c we can modify the code to access the Storyboard model structures indirectly via a pointer to the sb_vfs_root_t structure SBVFS_ROOT_PTR, which is initialised either using the address of the locally defined g_sbvfsRoot SBVFS root structure or the exported linker section start address.
Modify and add the code below around the model file include:
#if defined(GRE_FEATURE_VFS_RESOURCES) #if (APP_USE_EXTERNAL_SBVFS == 0) //***************************************************************************** // The storyboard model and assets are not in an external project so // we will include them inline here via the header file and create the // sb_vfs_root_t structure to enable indirect access via SBVFS_ROOT_PTR //***************************************************************************** #include "sbengine_model.h" const sb_vfs_root_t g_sbvfsRoot = { sb_model, // The Storyboard model sb_model_resources, // The model resources NULL // (SB Lite only) The C callback functions list, // typically defined in model export as 'clist' }; /* End of g_sbvfsRoot */ #define SBVFS_ROOT_PTR ( (sb_vfs_root_t *)&g_sbvfsRoot ) #else //***************************************************************************** // The storyboard model and assets are managed in an external project so // we will not include them here via the header file. The sb_vfs_root_ // structure is flashed sperately at a fixed start address __sbvfs_start__ // and used to enable indirect access to model resources via SBVFS_ROOT_PTR //***************************************************************************** extern int __sbvfs_start__; #define SBVFS_ROOT_PTR ( (sb_vfs_root_t *)&__sbvfs_start__ ) #endif //APP_USE_EXTERNAL_SBFS
Next we can modify the code in functions sbvfs_get_resource_roots() and sbengine_main_task() to access the Storyboard model structures indirectly via a pointer SBVFS_ROOT_PTR.
sbvfs_resource_t ** sbvfs_get_resource_roots(int *nroots) { static const sbvfs_resource_t *roots[1]; if(roots[0] == NULL) { roots[0] = SBVFS_ROOT_PTR>sb_resources; } *nroots = 1; return (sbvfs_resource_t **)roots; } /** * This is the main runtime task for Storyboard. It should be possible to call this * task directly from within your FreeRTOS core application. */ void sbengine_main_task(void *arg) { char *args[20]; int n = 0; #if defined(GRE_FEATURE_VFS_RESOURCES) run_storyboard_app(SBVFS_ROOT_PTR>sb_model, GR_APP_LOAD_STRING, args, n); const char example_model_path[] = "sbapp/sbapp.gapp"; run_storyboard_app(example_model_path, GR_APP_LOAD_FILE, args, n); #endif }
At this point, you should still be able to build and run the application code using the inline model with compiler preprocessor option APP_USE_EXTERNAL_SBVFS=0.
To enable the model resources to be split-out from the ‘back-end’ functional code and managed independently a new resource only project will be created.
-
It should contain only the SBVFS resources within the Storyboard model and assets and be able to be built as a standalone image and flashed alongside the core application.
-
We will not have any board initialization, executable code functions or (RW) RAM data variables within this project.
-
The memory regions must be mapped to the same linker regions and fixed addresses referred to in the application code project
The easiest method is to create a new (or clone and rename existing) project for the same processor and SDK then modify by strip out all the unneeded code and content whilst retaining the same compiler settings.
In order to configure the correct options within the Storyboard headers it is necessary to replicate the compiler include files path and preprocessor defines, including our new APP_USE_EXTERNAL_SBVFS setting (below example is an i.MXRT10xx project featuring an ARM Cortex M7 CPU):
-
GRE_TARGET_OS_freertos
-
GRE_TARGET_CPU_cortexm7
-
GRE_TARGET_TOOLCHAIN_mcuxpresso
-
GRE_FEATURE_VFS_RESOURCES
-
APP_LCDIF_DATA_BUS=kELCDIF_DataBus16Bit
-
APP_USE_EXTERNAL_SBVFS=1
As we are not including any functional C code within this resource only project we are not linking with any of the Storyboard static libraries and plugins and hence these are not required to be configured for the project linker libraries setting.
In terms of linker, the example below is again based on a GCC based linker (.ld) file memory section for a 64MB (0x4000000 bytes) Flash device partitioned into 16MB and 48MB regions accordingly.
MEMORY { /* Define each memory region */ BOARD_FLASH (rx) : ORIGIN = 0x60000000, LENGTH = 0x1000000 /* 16M bytes (alias Flash) */ BOARD_FLASH_SBVFS (rx) : ORIGIN = 0x61000000, LENGTH = 0x3000000 /* 48M (64-16M) bytes (alias Flash) */ BOARD_SDRAM_UPPER (rwx) : ORIGIN = 0x81E00000, LENGTH = 0x00200000 /* 2M bytes (alias RAM) */ BOARD_SDRAM_LOWER (rwx) : ORIGIN = 0x80000000, LENGTH = 0x01E00000 /* 30M bytes (alias RAM) */ SRAM_ITC (rwx) : ORIGIN = 0x0, LENGTH = 0x20000 /* 128K bytes (alias RAM2) */ SRAM_DTC (rwx) : ORIGIN = 0x20000000, LENGTH = 0x20000 /* 128K bytes (alias RAM3) */ SRAM_OC (rwx) : ORIGIN = 0x20200000, LENGTH = 0x40000 /* 256K bytes (alias RAM4) */ } /* Define a symbol for the top of each memory region */ __base_BOARD_FLASH = 0x60000000 ; /* BOARD_FLASH */ __base_Flash = 0x60000000 ; /* Flash */ __top_BOARD_FLASH = 0x60000000 + 0x1000000 ; /* 16M bytes */ __top_Flash = 0x60000000 + 0x1000000 ; /* 16M bytes */ __base_BOARD_FLASH_SBVFS = 0x61000000 ; /* BOARD_FLASH_SBVFS */ __base_SBVFS = 0x61000000 ; /* Flash */ __top_BOARD_FLASH_SBVFS = 0x61000000 + 0x3000000 ; /* 48M (64-16M) bytes */ __top_SBVFS = 0x61000000 + 0x3000000 ; /* 48M (64-16M) bytes */
We can now use a simplified version of the previous linker (.ld) file used for the main application as we will not need the standard regions and code entry points for the resource only configuration. It is essential to make sure that we do include the constant data structures hence the additional directives:
*(.text*) *(.rodata .rodata.* .constdata .constdata.*)
The SECTIONS part of the linker (.ld) file can now be expanded to include our new .sbvfs section definition and region mapping :
SECTIONS { /* Storyboard model and resources configured as filesystem from external flash */ .sbvfs : ALIGN(4) { FILL(0xff) __sbvfs_start__ = ABSOLUTE(.) ; KEEP(*(.sbvfs.root)) KEEP(*(.sbvfs.model)) KEEP(*(.sbvfs.resources)) *(.text*) *(.rodata .rodata.* .constdata .constdata.*) __sbvfs_end__ = ABSOLUTE(.) ; } > BOARD_FLASH_SBVFS /* Provide basic symbols giving location and size of main text * block, including initial values of RW data sections. Note that * these will need extending to give a complete picture with * complex images (e.g multiple Flash banks). */ _image_start = LOADADDR(.sbvfs); _image_end = LOADADDR(.sbvfs) + SIZEOF(.sbvfs); _image_size = _image_end - _image_start; }
The final step is to create the single source file needed to bring together the Storyboard model and resources defined by the exported C/C++ resource header sbengine_model.h and position these by mapping them to the appropriate sections within the .sbvfs region.
An example of a resource only source file is below, named sbengine_model.c and which will be the only source file in our project :
/* * Copyright 2019, Crank Software Inc. All Rights Reserved. * * For more information email info@cranksoftware.com. */ #include <gre/gre.h> #include <gre/sdk/greal.h> #if defined(GRE_FEATURE_VFS_RESOURCES) #include <gre/sdk/sbresource_vfs.h> /* This structure is defined in <gre/sdk/sbresource_vfs.h> and should be included from there */ #if !defined(SB_CCALLBACK_STRUCT_DEFINED) typedef struct _sb_ccallback { const char *name; void (*c_callback)(gr_application_t *); } sb_ccallback_t; #endif /* This structure should ideally be defined in <gre/sdk/sbresource_vfs.h> and should be included from there */ #if !defined(SB_VFS_ROOT_STRUCT_DEFINED) typedef struct _sb_vfs_root { const char *sb_model; ///< The application model const sbvfs_resource_t *sb_resources; ///< The resource table const sb_ccallback_t *clist; ///< The SB Lite C callback functions list } sb_vfs_root_t; #endif #if defined (APP_USE_EXTERNAL_SBVFS) && (APP_USE_EXTERNAL_SBVFS == 1) #if defined(__CC_ARM) || defined(__ARMCC_VERSION) || defined(__GNUC__) __attribute__((section(".sbvfs.model"))) #elif defined(__ICCARM__) #pragma location=".sbvfs.model" #endif #endif //***************************************************************************** // The storyboard model and assets are in this (external) project so // we will include them inline here via the header file and create the // sb_vfs_root_t structure to enable indirect access externally via SBVFS_ROOT_PTR //***************************************************************************** #include "sbengine_model.h" #if defined (APP_USE_EXTERNAL_SBVFS) && (APP_USE_EXTERNAL_SBVFS == 1) #if defined(__CC_ARM) || defined(__ARMCC_VERSION) || defined(__GNUC__) __attribute__((section(".sbvfs.root"))) #elif defined(__ICCARM__) #pragma location=".sbvfs.root" #endif #endif //***************************************************************************** // The sbvfs allocation table. // This relies on the linker script to place at correct location in memory. //***************************************************************************** const sb_vfs_root_t g_sbvfsRoot = { sb_model, // The Storyboard model sb_model_resources, // The model resources NULL // (SB Lite only) The C callback functions list, // typically defined in model export as 'clist' }; /* End of g_sbvfsRoot */ #endif // GRE_FEATURE_VFS_RESOURCES
Here we are using the GCC __attribute__ and section() directive to good use by force positioning the constant variables in the required linker sections:
__attribute__((section(".sbvfs.model"))) __attribute__((section(".sbvfs.root")))
The project can now be built which will generate a binary image that can be flashed to the target board alongside and independently if the rest of the ‘back-end‘ application source code project.
Building a simple model sbengine_model.h with a few resources shows us that we have only populated the BOARD_FLASH_SBVFS flash region as planned:
Memory region Used Size Region Size %age Used BOARD_FLASH: 0 GB 16 MB 0.00% BOARD_FLASH_SBVFS: 650764 B 48 MB 1.29% BOARD_SDRAM_UPPER: 0 GB 2 MB 0.00% BOARD_SDRAM_LOWER: 0 GB 30 MB 0.00% SRAM_ITC: 0 GB 128 KB 0.00% SRAM_DTC: 0 GB 128 KB 0.00% SRAM_OC: 0 GB 256 KB 0.00% Finished building target: evkbimxrt1050_sbengine_app.axf
Inspecting the linker (.map) file output shows us the detailed breakdown of these section allocations to .sbvfs.root, .sbvfs.model and.sbvfs.resources structures from the sbengine_model.h file:
.sbvfs 0x61000000 0x9ee0c FILL mask 0xff 0x61000000 __sbvfs_start__ = ABSOLUTE (.) *(SORT_BY_ALIGNMENT(.sbvfs.root)) .sbvfs.root 0x61000000 0xc ./source/sbengine_model.o 0x61000000 g_sbvfsRoot *(SORT_BY_ALIGNMENT(.sbvfs.model)) .sbvfs.model 0x6100000c 0x18e2 ./source/sbengine_model.o 0x6100000c sb_model *(SORT_BY_ALIGNMENT(.sbvfs.resources)) *(SORT_BY_ALIGNMENT(.text*)) *(SORT_BY_ALIGNMENT(.rodata) SORT_BY_ALIGNMENT(.rodata.*) SORT_BY_ALIGNMENT(.constdata) SORT_BY_ALIGNMENT(.constdata.*)) *fill* 0x610018ee 0x2 ff .rodata.res_0 0x610018f0 0x97dc4 ./source/sbengine_model.o 0x610018f0 res_0 .rodata.res_1 0x610996b4 0x160e ./source/sbengine_model.o 0x610996b4 res_1 *fill* 0x6109acc2 0x2 ff .rodata.res_2 0x6109acc4 0xaa6 ./source/sbengine_model.o 0x6109acc4 res_2 *fill* 0x6109b76a 0x2 ff .rodata.res_3 0x6109b76c 0x14ea ./source/sbengine_model.o 0x6109b76c res_3 *fill* 0x6109cc56 0x2 ff .rodata.res_4 0x6109cc58 0x187c ./source/sbengine_model.o 0x6109cc58 res_4 .rodata.res_5 0x6109e4d4 0x543 ./source/sbengine_model.o 0x6109e4d4 res_5 *fill* 0x6109ea17 0x1 ff .rodata.res_6 0x6109ea18 0x2fc ./source/sbengine_model.o 0x6109ea18 res_6 .rodata 0x6109ed14 0x97 ./source/sbengine_model.o *fill* 0x6109edab 0x1 ff .rodata.sb_model_resources 0x6109edac 0x60 ./source/sbengine_model.o 0x6109edac sb_model_resources 0x6109ee0c __sbvfs_end__ = ABSOLUTE (.) 0x61000000 _image_start = LOADADDR (.sbvfs) 0x6109ee0c _image_end = (LOADADDR (.sbvfs) + SIZEOF (.sbvfs)) 0x0009ee0c _image_size = (_image_end - _image_start) OUTPUT(evkbimxrt1050_sbengine_app.axf elf32-littlearm)
The resource only project can now be flashed to the target using the normal flashing process for the toolchain and target board.
Some general guidelines:
-
Sanity check the linker (.map) file output to make sure all the resources listed in the sbengine_model.h file have made it to the resource only binary
-
Make sure the flash tool does not perform a full erase of the device, instead erasing only the (BOARD_FLASH_SBVFS) sectors used
-
Ensure that the resource binary image is flashed before the application binary image otherwise there will be no model for it to execute!