Setup

Setup x86 vm for compiling 3 models (encoder.onnx, decoder.onnx, decoder_with_past.onnx)

The reason we use cross-compile is because onnx package is not available on risc-v system at Sep, 2025.

The following is our vm hardware spec:

Ubuntu 22.04.5 LTS
6 vCore
20gb ram
100gb disk

Steps to setup:

Enable VirtualBox copy/paste (optional)
Disable sleep 5 min in settings (optional)
Install dependencies and TVM

https://930727fre.notion.site/TVM-compilation-1ade0e612db880f1af36c19a3c873cfa
Install risc-v toolchain

https://930727fre.notion.site/Risc-v-toolchain-compilation-273e0e612db880fcb381f8df886629e3
Setup bananapi runtime and codegen

https://930727fre.notion.site/Setup-bananapi-runtime-and-codegen-273e0e612db88087a3d4cc000b0b200c
Download whisper-tiny models from hugging face

https://huggingface.co/onnx-community/whisper-tiny/tree/main

Compile 3 models and rsync them. The code can be found at

https://930727fre.notion.site/Backup-22ce0e612db88099aaabea380306970d

cd ~/whisper-tiny/onnx
python3 compile_encoder.py
python3 compile_decoder.py
python3 compile_decoder_with_past.py
rsync -avz -e ssh ./*.so fre930727@your_ip:~/whisper-tiny/onnx/

Setup Banana pi f3 for inference

Note: Bianbu 2.2 is derived from Ubuntu 24.04

        #####           fre930727@spacemit-k1-x-deb1-board 
       #######          ---------------------------------- 
       ##O#O##          OS: Bianbu 2.2 riscv64 
       #######          Host: spacemit k1-x deb1 board 
     ###########        Kernel: 6.6.63 
    #############       Uptime: 3 days, 5 hours, 56 mins 
   ###############      Packages: 2265 (dpkg) 
   ################     Shell: zsh 5.9 
  #################     Terminal: /dev/pts/1 
#####################   CPU: Spacemit X60 (8) @ 1.600GHz 
#####################   Memory: 307MiB / 3807MiB 
  #################

Install dependencies and TVM

https://930727fre.notion.site/TVM-compilation-1ade0e612db880f1af36c19a3c873cfa
Setup bananapi runtime and codegen

https://930727fre.notion.site/Setup-bananapi-runtime-and-codegen-273e0e612db88087a3d4cc000b0b200c

Compile libmatmul.cpp

cd /home/fre930727/tvm/src/runtime/contrib/bananapi

g++ -std=c++11 -shared -fPIC -O3\\
    -march=rv64gcv -mabi=lp64d \\
    -I ~/tvm/3rdparty/dlpack/include \\
    -o libmatmul.so libmatmul.cpp

Download whisper-tiny models from hugging face, this step is necessary, because tokenizer and vocab.json is required

https://huggingface.co/onnx-community/whisper-tiny/tree/main
```
cd
git clone ... (lfs or something)
```