UART para-virtualized driver uberApp

Folks,

This is the discussion thread for am uberApp that uses a UART para-virtualized driver to communicate with a UART driver üobject within the micro-hypervisor collection.

I currently have it housed at:

Let us use the following workflow as we tinker on this repo together:

  1. Create a fork of the aforementioned repository within your account to start with
  2. Implement each feature as a separate branch on your fork
  3. create a pull-request (PR) to merge the feature branch into the main repository above
  4. if another feature has made it into the main repository meanwhile, re-base feature branch and (re-)issue the PR.

The following features need to make it to the repository to begin with:

  • python test scripts
  • documentation on the UART cable hookup for the Pi3 and the receiver
  • the pl011 linux kernel driver code and out-of-tree module build instructions

Feel free to propose other features and start implementing them :>

David, could please check in the documentation you have so far and the Python scripts, so that Amit can run some tests.
Thank you
Anton

Please check the repo. :slight_smile:

Thanks David!

I am presently working on wrapping up an implementation of the PL011 UART support for the micro-hypervisor uobject collection.

Once that is ready, we can look towards integrating it with the guest OS para virtualized driver side of things.

It might be useful for both you and Anton to get a working setup of the uberXMHF raspberry pi 3 micro hypervisor estavlished on your side meanwhile. Please use the develop branch of https://github.com/uberspark/uberxmhf.git for this.

6 posts were merged into an existing topic: Supporting latest Raspbian distribution

For cross compiling on Linux follow the page:
https://www.raspberrypi.org/documentation/linux/kernel/building.md
In order to select the amba driver for Linux to be a loadable module we need the following:
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
Select Device Drivers/Character Devices/Serial Drivers/ARM AMBA PL011 Serial port support
Using space key select M and save the configuration.
Make sure .config has the right entry for AMBA driver = M
Recompile the tree:
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs -j 16
The compiled loadable module is located in drivers/serial/amba_pl011.ko

I modified these two routines to describe what we need to have in our proof of concept driver:

static int my_get_poll_char(struct uart_port *port)
{
struct uart_amba_port *uap =
container_of(port, struct uart_amba_port, port);
unsigned int status;

    status = pl011_read(uap, REG_FR);
    if (status & UART01x_FR_RXFE)
            return NO_POLL_CHAR; /* Fifo is empty */

    return pl011_read(uap, REG_DR);

}

static int my_put_poll_char(struct uart_port *port,
unsigned char ch)
{
struct uart_amba_port *uap =
container_of(port, struct uart_amba_port, port);

    if (pl011_read(uap, REG_FR) & UART01x_FR_TXFF)
            return NO_POLL_CHAR; /* Fifo is full */
      
    pl011_write(ch, uap, REG_DR);
    return 1;

}

These routines just send and receive one character and check if the FIFO is empty on receiving or if it is full before sending.
What we can do is the following.
a.) Create a new simple driver that has these routines as .read and .write.
Create a simple C program that opens the device file and sends/receives byte by byte by handling error cases where NO_POLL_CHAR is returned for the situations of empty or full Fifo based on the operation.
b.) Figure out how to use these routines in the existing driver with all the existing layers.
Let me know what you think.

Digging further revealed that the polling functions in the AMBA driver are actually compiled.
This can be seen by looking in the .config file which has the following two parameters set:
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_CONSOLE_POLL=y
In addition if one changes something in the two polling functions and introduces a compilation mistake then the compilation breaks, which means that this code is compiled.
There is a higher level driver which uses our serial driver polling functions:
drivers/tty/serial/kgdboc.c
kgdboc.c is used by kgdb and it has two functions:
kgdboc_get_char() and kgdboc_put_char()
These two functions call the AMBA driver polling functions.
kgdboc_get_char() and kgdboc_put_char() are exported as read_char() and write_char() respectively.

.read_char = kgdboc_get_char,
.write_char = kgdboc_put_char,

Calling these functions would be something we can figure out.
kgdb is a good place to start since it relies on kgdboc.c

If we can demonstrate that kgdb works for the Pi then the serial port is used by polling to send debugging information to the gdb session which runs on the Linux laptop.
If we can use these instructions or similar ones to do it, since the kernel has to e compiled for debugging:


This would just use the kgdboc driver which provides the polled serial communications.

Hi Anton,

Thanks for this useful probe!

I have a working PL011 driver within the micro-hypervisor and am finishing up some final tests on it. Once that is finished, I will ping back here and we can discuss further on how we can move forward with the para-virtualized driver interface.

My initial thinking is that we may not want to go the full serial driver route, but rather just use a character device driver interface which is much simpler.

The drivers that I checked in are in chardriver folder :
https://github.com/amitvasudevan/uapp-pvdriver-uart/tree/master/chardriver
There is a Readme.md which explains how to use it.

Thanks Anton!

I have pushed a couple of changesets to uberspark/uberxmhf.git within the develop branch. This adds support for PL011 UART based debugging within the micro-hypervisor.

You will have to enable this when you build the micro-hypervisor with --enable-debug-uart and --enable-debug-uart-pl011 options to the configure script. Please see top-level README on how to build the documentation (which has now been revised to use Sphinx/RST) and peruse more details on these options.

We will need to come up with a üobject within the micro-hypervisor which will expose the PL011 read/write/config routines to the pv driver.

Have we tested the character device driver you have against some user-level (python) scripts to validate transmission and reception yet?

Thanks!

I have tested to see that the driver works through printk(), but do not have a cable to see it actually transfer characters.

I think we need a test rig in guest-mode to test this character driver. This could be @daveman1010221’s python scripts that he had working with the stock serial driver. I am guessing we might need to tweak it to use the character driver.

The idea behind having a test rig in guest-mode is to be able to first ascertain if the character device driver abstraction is sufficient for serial transmit/receive operations or if we need to adhere to the full serial driver (tty) interface…

I agree. @daveman1010221 has the serial setup for this. Don’t have a cable myself.
The changes in the python script are minimal to use the char driver.

Added a test python script that can drive the character device driver. This can be used to run the Python test that we already have.

I see that the pl011uart driver is part of the core. The functions are listed in the respective .h file. Need to figure out if there is some documentation on how to make a hyper call to invoke the send and the future rcv function.

This will come as part of the uart para-virtualized driver micro-hypervisor backend. Take a look into the example hypercall test backend and guest kernel module example within uberspark/uberxmhf.git:

Example micro-hypervisor hypercall backend:
uxmhf-rpi3/uapps/uapp-uhcalltest/uapp-uhcalltest.c

Example guest kernel driver and user-mode application that communicates with this hypercall backend:

uxmhf-rpi3/rgapps/linux/rgapp-uhcallkmod/
uxmhf-rpi3/rgapps/linux/rgapp-uhcalltest/

The hypercall backend is tied into the following module within the core:

uxmhf-rpi3/core/ghcall.c

We will need to adapt this to provide PL011 UART send/receive API functionality to the guest character driver.

@daveman1010221 has tested the character driver. I have updated the C test app and the documentation in : https://github.com/amitvasudevan/uapp-pvdriver-uart/tree/master/chardriver
This proves that our character device driver can send and receive characters through polling by using the low level stock driver amba-pl011.c from Raspbian.
Now we need to replace the calls to amba-pl011.c with hyper calls once the hypervisor driver is completely ready and use the same test again.

Ok cool. I am presently working on:

I will ping back once the aforementioned tasks are completed, so we can talk about the final step of threading the guest OS PV driver and the UART PV driver backend within the micro-hypervisor

Thanks!

This is great. We can meet to sync up when you have something ready and see what we can do to help.