Displaying sbengine to a VNC server
I'm attempting to set up the ability for our device to be remotely connected to and interacted with for the sake of solving customer service issues. I'm using a Radxa Rock Pi 4b running Ubuntu 20.04 LTS with the 'linux-raspberry-aarch64-opengles_2.0-drm-obj' engine for Storyboard. I'm able to remotely connect to the Rock Pi and that seems to work fine, but whenever I run my Storyboard application, it only shows on the actual device, not on the remote connection. I've tried running the application with VirtualGL, but I that won't work either.
How can I direct sbengine's display to the VNC server so it actually displays and can be interacted with over the remote connection? If that's not doable, is there some other way I can accomplish this?
1
Comments
Hi Trevor,
What VNC server / client are you using? Do other OpenGL based applications show up correctly in your VNC viewer? For example, the "OpenGL Gears" demo or a game like "Extreme Tux Racer"?
Michael
I'm using x11vnc for the server and TigerVNC to view it. I have gotten glxgears and glxspheres to work just fine.
Just to summarize, when using TigerVNC, you can see your Rock desktop, but you cannot see your Storyboard application after launching it?
When using the Rock desktop directly connected to a monitor, you can see the Storyboard application?
Correct.
If you use the TigerVNC server, instead of the x11vnc server, do you get the same issue?
I can try setting that up, I'll let you know how it goes.
After setting up a TigerVNC server and connecting with TigerVNC Viewer, I'm still seeing the same behavior.
Are you able to try the "linux-raspberry-aarch64-swrender-obj" Storyboard run-time? It might give you an alternative for now.
Does your VNC server get started before the X11 server? I am assuming it is started after the X11 windowing system is up and running.
Another possibility is that it may be due to DRM/DRI rendering pipeline on Linux. Do you happen to know if your versions of x11vnc and TigerVNC have support for DRI/DRM built into them? Do any specific options need to be enabled on the server or the client to support DRM enabled applications? I know the TigerVNC software has configuration options that need to be set when the software gets built in order for it to support DRM/DRI, but I don't know if those options are enabled by default
I'm trying to run the swrender runtime but I keep getting the following error:
I only get this error when I try to run it during an X11 session.
The X11 server is started before the VNC server.
I'm not sure how to tell if either my x11vnc or TigerVNC support DRI/DRM, and I can't find any info on how to enable it.
When using the shell on your Rock device, if you run "ls -l /dev/fb*", what do you see?
Thanks for being so patient, Trevor. I have been speaking with the developers of the Storyboard run-time. Because of the mechanics around how the fbdev and drm technologies work, we think the Storyboard run-time needs to be built in a special way for it to work with software like VNC. Unfortunately, we do not have a run-time for ARM that provides that capability. If you are able to install a VM on your Rock device with a minimal x86 Linux installation, then you may be able to run your front-end application in the VM (without using X11). You would need to consider how that would impact your back-end, of course. It's not a great workaround, but it is the best we can come up right now.
I can try installing an x86 Linux VM, but I don't really like that answer as I imagine it will run quite slowly.
How hard would it be to build an x11-compatible runtime for ARM and is that something I could do myself? I imagine it's too much to ask for you guys to build me a runtime like this, so I'd rather do it myself if possible.
If not, then one idea for a workaround that I have is to stream the framebuffer from a VNC session and direct mouse inputs to the application. I've been able to take and view screenshots of the framebuffer while a storyboard application is running (they don't come out quite right, I'm sure there's a way to fix that though), so maybe there's a way to constantly view the framebuffer as it updates. I don't know how I could direct inputs though, and I haven't been able to find any way to stream the framebuffer, so I'm not sure if this idea is really viable.
Do you have any other ideas for how I could get this to work? Maybe there's another option than VNC that I haven't come across yet.
Just to give you an update, I found a solution.
This VNC server (https://github.com/ponty/framebuffer-vncserver) captures the frame buffer directly and doesn't require an x11 server running. Using the swrender storyboard runtime (not the opengl one, though I'm not sure why), I've been able to successfully connect to this VNC server and control a storyboard interface remotely, exactly like I wanted.
Thank you for the help you gave me, I'm not sure I would have figured this out otherwise.
That's great to hear! Just FYI, I am continuing to pursue the X11 build for ARM, if that works out, then you may be able just use the OpenGL version out of the box one day. Thank you for bringing this to our attention.
Please sign in to leave a comment.