Web based VNC client with AntOS docker image

Long time ago i developed WVNC, a simple protocol based on Web socket that allows to connect to VNC server from browser. WVNC consist of

• A server side plugin (for Antd web server) that (1) acts as a bridge between VNC server and the web server; (2) defines the base protocol on top of websocket for streaming screen data and events to client. JPEG is used as data compression to reduce stream bandwidth.
• A client side API called wvnc.js that implements the protocol defined by the server plugin, decodes JPEG data from the server and renders the data on HTML5 canvas

The protocol works really well for my personal need as i used it on a daily basis to access to my VNC home server from work :), using nothing but the web browser. However, setting up WVNC from scratch is not a trivial task, as it depends on my Antd server which is not a popular web server, thus Stack overflow is not an option :). I've received many contacts from readers on the howto instruction.

As WVNC is finally a part of AntOS eco-system, and AntOS is available as an all-in-one docker image, anyone can now easily run their own web-base VNC client via a single command line without the headache of building every thing from scratch.

CI & automation: Multi-architecture build of software with Jenkins and Docker

Multi-arch software building and distribution is a complex and time-consuming maintenance task in which maintainer/developer need to compile somehow their software for all supported architectures, often, on a single machine. Automation solution such as Jenkins allows to automatize this task and facilitating continuous integration and continuous delivery. Multi-arch software building on a single machine involves the use of different techniques:

• Cross-build: using a cross-toochain to build the software for each target system. This solution is simple but with the cost of polluting the host system with many toolchains which maybe incompatible one to another. Maintenance and update of these toolchains are complicated to manage.
• Virtualization solutions such as docker address these problem by using a sandboxed/containerized image with all tools necessary to build the software for each target architecture. Docker facilitate the setup, maintenance and update of different building environments using images.

This post introduces the basic steps of setting up an automation server that allows to build and distribute multi-arch software using Jenkins and docker.

Monitoring and collecting syslog messages from Unix Domain Socket

Application log is the traditional way to monitor an application/service. On *nix-based system, Syslog is a common but powerful tool for centrally monitoring applications logs. The primary use of syslog is for system management as capturing log data is critical for sysadmin, devOp team, or system analysts, etc. This log data is helpful in case of investigating/troubleshooting problems and maintaining healthy functioning of systems.

Syslog offers a standard log format and a standard alert system with different severity levels to applications in form of a log API. Log daemons such as rsyslog are versatile and flexible with various configuration options that enable different way to interact with the logs: log to file, log to a remote server via network (TCP, UDP sockets), log to local Unix domain socket. Log clients or log analytic applications can collect log data from the log daemon via these interfaces.

Although it is feasible to directly read log messages from the regular syslog output files, it is more preferable to collect log data from the daemon using the socket interface since socket is more suitable for data streaming. TCP/UDP sockets can be used to access log data from the network (TCP/IP). But if the application runs locally on the same machine as the log daemon, Unix domain socket (UDS) may be the best option.

Unix Domain Socket is an inter-process communication mechanism that allows bidirectional data exchange between processes running on the same machine. Thus, UDSs can avoid some checks and operations (like routing); which makes them faster and lighter than IP sockets.

In this post, we will learn how to collect log data from syslog via UDS in C. We will use rsyslog as log daemon in this post.

A use case will be presented at the end of the post.

AntOS v1.2.0-beta release

After a long testing period, AntOS v1.2.0-beta is now released!!!

Change logs

• Improvement GUI API
  • [x] File dialog should remember last opened folder
  • [x] Add dynamic key-value dialog that work on any object
  • [x] Window list panel should show window title in tooltip when mouse hovering on an application icon
  • [x] Allow pinning application to system panel
  • [x] Improvement application list in market place
  • [x] Allow triplet keyboard shortcut in GUI
  • [x] CodePad allows setting shortcut in CommandPalette commands
  • [x] CodePad should have recent menu entry that remember top n file opened
  • [x] Improve File application grid view
  • [x] Label text should be selectable
  • [x] switch window using shortcut (CTRL+ALT+1, CTRL+ALT+2)
  • [x] Loading bar animation on system panel
  • [x] Multiple file upload support
  • [x] Generic key-value dialog
  • [x] Add bootstrap font support for icons
  • [x] Classify applications by categories in start menu
  • [x] Support vertical and horizontal resize window
• MarketPlace now classifies application by categories
• CodePad is no longer default system application, it has been moved to MarketPlace
• More applications added to MarketPlace
• Antos SDK
  • SDK is no longer included in Antos base release, it can be installed via MarketPlace
  • The SDK now has a generic API that can be used in different development tasks other than AntOS application
  • Heavy SDK tasks are now offloaded to workers
  • Introduce new JSON based syntax for SDK task/target definition
• From this version, docker image of All-in-one AntOS system is available at: https://hub.docker.com/r/xsangle/antosaio

Demo

A demo of the VDE is available at https://app.iohub.dev/antos/ using username: demo and password: demo.

If one want to run AntOS VDE locally in their system, a docker image is available at:
https://hub.docker.com/r/xsangle/antosaio/

AntOS applications (Available on the MarketPlace)

https://github.com/lxsang/antosdk-apps

Documentation

• API: https://doc.iohub.dev/antos/api/
• Documentation: https://doc.iohub.dev/antos (outdated, WIP)

Data visualization: global view of blog posts relationship

As stated on a post where i talked about using tf-idf to detect similarity between two blog posts, my blog is just a bunch of posts sorted by date, no category, no fancy features like user interest tracking, post ranking, etc. I usually work on many different domains (robotic, IoT, backend, frontend platform design, etc.), so my posts are mixed up between these domains. This may be difficult for readers who want to follow up their interesting topic on my blog.

So what is a good strategy for navigating between posts on a blog ?

sysmond: Simple service for (embedded) Linux system monitoring

Working on my DIY robot software (Jarvis) in headless mode, i came across a situation where i needed to monitor the system resource such as CPU, battery, memory, network and temperature to measure the "greedy" of my robotic application. Furthermore, as the robot was battery powered, battery safety was a real concern, so i needed something to monitor the battery and shutdown the system when the battery was low to protect it from falling bellow the usable voltage range.

So i've searched for an application/service that allows me to:

• Monitor system memory, CPU, storage usage and temperature
• Monitor network consumption
• Monitor the robot battery and power off the system if the battery is low

None of existing applications/services satisfy all of these requirements, especially, the battery monitoring feature. So i've decided to write a small service that i called sysmond.

sysmond is a simple service that monitors and collects system information such as battery, temperature, memory, CPU, and network usage. The service can be used as backend for applications that need to consult system status. Although it is a part of Jarvis ecosystem, sysmond is a generic service and can be easily adapted to other use cases.

Example of AntOS web application that fetches data from sysmond and visualize it as real-time graphs on my Jarvis robot system. Detail on the use case can be found here

Sysmond monitors resource available on the system via the user space sysfs interface provided by the linux kernel.

JETTY: Jarvis Serial to ROS-2 Transport Layer

My ROS based DIY robot( presented in the previous post) uses the NVIDIA Jetson Nano for high level robotic algorithms with the ROS 2 middle-ware. The Jetson is connected to an Arduino via a serial link for low-level hardware interaction and control.

As the Arduino is used for low-level communication with actuators/sensors. We need a software transport layer on top of the physical serial link (Jetson - Arduino) to stream (sensor) data/command from Arduino to ROS 2 and vice versa. On Dolly (my previous robot version), which used ROS 1, this was handled by Rosserial, a protocol for wrapping standard ROS serialized messages and multiplexing multiple topics and services over a serial link. On ROS 2, however, Rosserial is not available. Other alternative solutions exist but are not mature enough, some implementations require more computational resource which exceeds the capability of the Arduino Mega 2560.

So i decided to implement a dedicated transport layer for Jarvis called JETTY (Jarvis SErial to ROS-2 TransporT LaYer). I do not aim at a generic protocol for ROS to serial communication like ROS serial. Instead, the implementation of the transport layer should be specific only to the robot. However, the protocol must be easy to extend to adapt to any future upgrade of the robot such as adding more sensor/actuators.

Requirements on the transport layer:

1. The transport layer must allow to stream data in form of frames (fixed size or not)
2. Simple but reliable, unambiguous packet framing protocol, frame should be easy to identify
3. Fast frame synchronization: When an endpoint (Arduino or ROS) connects to the Serial link in the middle of the data streaming, frame synchronization should be fast while minimizing the frames lost in the synchronization phase
4. Frame should be verified using checksum before being consumed by an endpoint
5. Packet framing overhead is allowed but need to be minimized
6. The algorithms should be easy to implement and computationally inexpensive on both Jetson and Arduino

Brief, we need an efficient and reliable delimiting/synchronization scheme to detect the frame with short recovery time.

The detail on the choice of protocol and algorithm as well as insight implementation is presented on a section of my Jarvis booklet accessible via the following link:

https://doc.iohub.dev/jarvis/Ym9vazovLy9jXzIvc18xL0lOVFJPLm1k/JETTY:_Jarvis_Serial_to_ROS-2_transport_layer.md

Control GPIO using the new Linux user space GPIO API

From the version 4.8, the Linux kernel introduces a new user space API based on character devices for managing and controlling GPIOs ( General-Purpose Input/Output). This post presents the basic of the new interface as well as a simple tutorial/example to demonstrate how to use the new API to control GPIOs.

The hardware used in the tutorial is the Raspberry Pi 3B but the code is generic and can be used on any embedded hardware.

Running your own AntOS VDE system using a Docker image

Update: The image is now available on Docker Hub at: https://hub.docker.com/r/xsangle/antosaio (image: xsangle/antosaio:latest). This post has been updated with the latest instructions on how to host a local instance of AntOS VDE using Docker.

Building and setting up AntOS from scratch can be complex, as it requires configuring and connecting many backend and front-end components. To simplify the use of AntOS as a self-hosted environment, I have created a Docker image layer for an all-in-one AntOS system that contains everything needed to host AntOS on your server. This layer allows you to build a minimal Docker image with a working AntOS system out-of-the-box:

• The web server (HTTP/HTTPS depending on the server configuration) + plugins
• AntOS server-side API
• AntOS client-side API

The Docker images are available at: https://hub.docker.com/r/xsangle/antosaio/

How does it work?

Follow the steps below to create an AntOS instance. First, create the working directory (e.g., /tmp/antos). All user data will be stored in this location. In this example, we use /tmp/antos, but in a real scenario, you should use a permanent storage location.

# modify with your own working directory
mkdir -p /tmp/antos

Run a container with docker

docker run \
    -p 8080:80 \
    --rm \
    -v /tmp/antos:/app \
    -e ANTOS_USER=demo \
    -e ANTOS_PASSWORD=demo \
    -it xsangle/antosaio:latest

Or with docker compose: docker-compose.yml

version: '3.7'
services:
  antos:
    image: xsangle/antosaio:latest
    privileged: true
    restart: always
    ports:
      - 8080:80
    container_name: antos_demo
    deploy:
      resources:
        limits:
          memory: 200m
          cpus: '0.5'
    hostname: demo
    environment:
      - ANTOS_USER=demo
      - ANTOS_PASSWORD=demo
    volumes:
      - /tmp/antos/:/app

Run:

docker compose up

AntOS is now accessible via http://localhost:8080/os/ or using IP address http://YOUR_MACHINE_IP:8080/os/

Conclusion

The docker image provides user with a ready to go (out-of-the-box) AntOS VDE system. This is useful in many user-cases:

• Quickly setup environment for AntOS applications development
• Scalable multi-user cloud-base service with AntOS and Kubenetes. Alternative to Ownclod, Google Cloud, Dropbox etc, with a more intuitive and user-friendly GUI, and more applications

How to list supported ciphers suites of a server?

I run into a problem of how to check whether my SSL ciphers suites configuration works correctly on my server.
Basically, with openssl, client can verify if the server supports a particular cipher suite using the following command:

openssl s_client -cipher "$cipher" -CAfile ca/ca.crt -connect server:port
# $cipher is the cipher suite name

So it is possible to automatically test all cipher suites supported by openssl against the server using a simple snippet of Bash, i found such script in this site https://superuser.com/questions/109213/how-do-i-list-the-ssl-tls-cipher-suites-a-particular-website-offers and modify it a little bit. Below is the script:

#!/usr/bin/env bash

# OpenSSL requires the port number.
SERVER=$1
DELAY=1
ciphers=$(openssl ciphers 'ALL:eNULL' | sed -e 's/:/ /g')

echo Obtaining cipher list from $(openssl version).

for cipher in ${ciphers[@]}
do
    echo -n Testing $cipher...
    result=$(echo -n | openssl s_client -cipher "$cipher"  -connect $SERVER 2>&1)
    if [[ "$result" =~ ":error:" ]] ; then
        error=$(echo -n $result | cut -d':' -f6)
        echo NO \($error\)
    else
        if echo $result | grep -q "Verify return code: 0 (ok)"; then
            echo YES
        else
            echo UNKNOWN RESPONSE
            echo $result
         fi
    fi
sleep $DELAY
done