überSpark: Add armv7_32 and armv8_32 hardware model to next-gen toolkit

This task will add hardware model implementations for armv7_32 and armv8_32 cpu architectures within the next generation toolkit.

The following two references can be followed towards this task:

  1. documentation described in docs/nextgen-toolkit/contrib-guide/hwm.rst to add a new hardware model within uberspark/uberspark.git on branch develop

  2. the x86_32 hardware model task: überSpark: Add x86_32 hardware model to next-gen toolkit

The CASM instructions that need to be implemented will be based on contents of assembly files within the following two uobjcoll:

  1. For armv7_32 --> https://github.com/uberspark/uobjcoll-arm-trusted-firmware/blob/uobjcoll/plat/st/stm32mp1_uobjcoll/bl32/uberspark.json

  2. For armv8_32 --> https://github.com/uberspark/uberxmhf/blob/develop/uxmhf-rpi3/uobjcoll/main/uberspark.json

When implementing the CASM instruction mnemonics and hardware model implementation please refer to the ARM v7 and v8 architecture reference manuals in order to check which architecture the instruction belongs to and the pseudo-code of what the instruction does. This pseudo-code will then be translated into the equivalent C implementation just like the x86_32 counterpart.

We can perhaps begin with a proposal of organization just like what we did for the x86_32 hardware model in the task: überSpark: Add x86_32 hardware model to next-gen toolkit

As a last step we can convert the assembly files in the above two uobjcoll to use CASM instructions and compile down via the CASM bridges.

PRs:


Merges:


Hey Amit, a proposal of organization sounds good. I’m thinking we more or less mimic the structure of the x86 hwm, so something like:

hwm
└── arch
    ├── arm
    │   └── generic
    │       ├── casm.h
    │       ├── cpu.h
    │       └── hwm.h
    ├── armv7_32
    │   └── generic
    │       ├── cpu.h
    │       └── hwm.h
    └── armv8_32
        └── generic
            ├── cpu.h
            └── hwm.h

to start.
Is the next step to create cpu model variables and instruction implementations?
I’m still reading up on the ARM assembly instruction set (and ARM/assembly in general) so my understanding might be a bit slow. Let me know what you think.

Thanks!

Hey @yeeb,

Yeah, your organization sounds good. Should we drop the generic sub-folders from under arm, armv7_32 and armv8_32?

Correct

No worries, feel free to ask away if you have any questions :slight_smile:

Hey @amitvasudevan,

Yeah sounds good.

Since ARM has more general purpose registers, I was wondering if using macros is a valid way to implement the hwm. For example, since the add instruction can be called with a lot of different registers, I’d have this in cpu.c

uint32_t hwm_cpu_gprs_r0 = 0;  // General purpose
...
...
uint32_t hwm_cpu_gprs_r15 = 0;  // PC //Program Counter

#define _impl__casm__add(RD, RN, OP2) {\
	hwm_cpu_gprs_##RD =  hwm_cpu_gprs_##RN + hwm_cpu_gprs_##OP2; \
}

and in cpu.h:

#define __casm__add(RD, RN, OP2) \
	__builtin_annot("add "#RD", "#RN", "#OP2); \
	_impl__casm__add(RD, RN, OP2); \

Then, the function can be called in the CASM files with __casm__add(r0, r1, r2).
Let me know if I’m going about this the right way.

Thanks!

Hmmm. Good point. While I like your generic implementation, I am not sure if this will allow us to have flexibility in terms of what we would like to do during the verification phase.

For example, I want to make sure that r6 is not clobbered by the callee. With a scheme that is explicit about the register specification (e.g., __casm__add_r0_r1_r2), we know the parameters and the corresponding hwm_cpu_gprs_xx mapping, so we can assert on those variables during the verification phase.

However, if we have a generic implementation (e.g., __casm__add()), I am not sure how we can ascertain if the passed register is r6 for example wth the proposed scheme…

Any thoughts?

Yeah, I think the verification flexibility is more important.

Maybe I can write a simple script that will generate all different __casm__add_r0_r1_r2 variations (16 choose 3)? The files might get a little bloated lol
I might move each mnemonic into a separate file with the different combinations and include them in the cpu.h/c. How does that sound?

I was thinking we can do something more simple as a first step – only implement the required casm instructions based on the assembly instructions found within assembly language sources of the following two uobjcoll:

  1. For armv7_32 --> https://github.com/uberspark/uobjcoll-arm-trusted-firmware/blob/uobjcoll/plat/st/stm32mp1_uobjcoll/bl32/uberspark.json
  2. For armv8_32 --> https://github.com/uberspark/uberxmhf/blob/develop/uxmhf-rpi3/uobjcoll/main/uberspark.json

This should allow us to verify and build the above two uobjcoll for now. We could then revist automatic generation of casm instruction implementation at a later stage. What do you think?

Sure that sounds good!

I’ve started adding CASM interfaces to the ARM hwm. I have a couple questions about some of the instructions and how to implement them:

  1. Some instructions change depending on if the processor is in ARM or thumb mode. Should I just model the current mode as a bool that the instructions can check and set? Would I also need separate uint16_t gprs for thumb mode since most thumb instructions are 16 bits long?
  2. There are a couple of instructions I’m not sure of how to implement:
    • DMB/DSB/ISB - data barrier instructions
    • MCR/MCRR, MRC/MRRC, MRS/MSR - all of the “move to coprocessor” instructions
    • SEV, WFE - set/wait for event
    • HVC, SVC hypervisor/supervisor calls

I’m also a little unsure of how to implement instructions that end with EX (e.g. STREX). I see that it’s conditional on whether an address has the Shared TLB attribute. Not how to add this in the hwm.
Gonna need some help with these lol. If there are any somewhat equivalent CASM interfaces in the x86 hwm that would also be super helpful.

Thanks!

Do we currently use thumb instructions? If so, your proposed solution seems reasonable. I belive thumb mode simply uses the lower 16-bits of the 32-bit registers, so you could just work with the lower 16-bits of the 32-bit register values for the implementation.

If we use the above instructions within our uobjcoll code, then just have a blank implementation for these for the time being.

Can we just model the co-processor register just like we model the regular registers (e.g., R0) and apply the logic just to set the relevant bits?

For now just have the same implementation for STREX and STR. We can revise the implementation if/once we model the TLB within the hardware model.

Hope that helps :slight_smile:

Hey @amitvasudevan,
I think I’m mostly done implementing the CASM instruction mnemonics for the instructions used in the armv8_32 uxmhf. They’re in my fork of the uberspark repo.

Let me know what you think, hopefully you can also double-check to make sure that the pseudo-code is correct :grimacing:.

I don’t think any of the instructions used were specific to armv8 so I placed them in the generic ARM folder. I’ll get started on the armv7 CASM soon, hopefully there’s a lot of overlap lol.

Nice work @yeeb!

Can you convert the assembly sources within uberspark/uberxmhf.git/uxmhf-rpi3/uobjcoll on branch develop to CASM functions using the CASM instructions mnemonics? I can then do a test run on my test-bed to see if all goes well.

Haha. There might actually be a bit of overlap since the difference is primarily on system instructions.

Thanks @yeeb!

Hey Amit,

Sorry for the delay. I’ve mostly finished converting the assembly sources to CASM. I have a question about how to convert the following though:

.section ".stack"
	.balign 8
	.global hypvtable_stack
	stack:	.space	8192
	.global hypvtable_stack_top
	hypvtable_stack_top:

There are also ldr instructions that refer to variables? in this stack section. What’s the correct way to convert this?

Thanks!

Let us do it within a .c file as below:

__attribute__((section(".stack"))) __attribute__((aligned(8))) hypvtable_stack[8192];

Does that make sense?

Thanks @yeeb!

Ah, that makes a lot of sense, thanks! I updated the CASM files so hopefully they are ready for testing :crossed_fingers:

The CASM files are in my fork of uberxmhf under the armv8_casm branch. It requires the arm hwm from my arm_hwm branch of uberspark.

I suspect there may be some issues with the branch instructions so let me know how it goes.

Thanks!

Below is my initial set of comments:

  1. https://github.com/sirmammingtonham/uberxmhf/blob/armv8_casm/uxmhf-rpi3/uobjcoll/uobjs/main/core/entry.cS#L133

    • Here we need to move the C variable definitions of cpu_stacks and cpu_stacks_svc into a separate .c file. CASM source files can currently only contain CASM functions and no data variable definitions.
    • Also, __attribute__(section) must be supplied to each variable definition separately I am guessing?
  2. https://github.com/sirmammingtonham/uberxmhf/blob/armv8_casm/uxmhf-rpi3/uobjcoll/uobjs/main/core/hypvtablestubs.cS#L59

    • Here all the hypvtable_xxx() must actually be encoded as __casm_b instructions correct?
  3. https://github.com/sirmammingtonham/uberxmhf/blob/armv8_casm/uxmhf-rpi3/uobjcoll/uobjs/main/core/hypvtablestubs.cS#L237

    • Variable definitions need to be moved into separate .c file as described previously
  4. https://github.com/sirmammingtonham/uberxmhf/blob/armv8_casm/uxmhf-rpi3/uobjcoll/uobjs/main/uapps/uapp-watchdogsup.cS#L85

    • Variable definitions need to be moved into separate .c file as described previously

Otherwise this is looking good for now :). Once you confirm you have addressed the above, and have a successful build, I can try to replicate on my side and do a test run with the raspberry pi 3.

This is great stuff. Thanks @yeeb!

Hey @amitvasudevan,
I’ve updated all the CASM files according to your comments. Getting some compile errors that I’m not sure how to fix:

This results in error: use of undeclared identifier (for the other files as well). In this case, would it be better to declare these C variables in a .h file so we can #include them in the CASM files to prevent this error?

I get an error: use of undeclared identifier when I switched to __casm_b. My b instruction pseudocode contains a goto x statement, and it looks like the compiler doesn’t allow for a goto on a function. Any ideas on how to fix this? Would it be as simple as adding a label before the corresponding functions?

Thanks!

Yep, that should do it.

Look into hwm/include/arch/x86/generic/cpu.h for examples on how to define macros that allow CASM functions to call regular C functions. These are __casm__call_c(fn_name) (C function with no parameters), __casm__call_c_1p (C function with 1 parameter) and so on. You will need to adapt these for the ARM architecture of course. Let me know if you need further help as you move through this.

Thanks @yeeb!