Overview

Relevant source files

• Cargo.toml
• src/lib.rs

Purpose and Scope

The axvm crate provides a minimal virtual machine monitor (VMM) for the ArceOS hypervisor ecosystem. It implements core virtualization capabilities including virtual CPU management, memory virtualization, and device emulation across multiple hardware architectures. This document covers the fundamental architecture and capabilities of the axvm hypervisor.

For detailed information about the VM configuration system, see Configuration System. For build and development processes, see Development Guide.

High-Level Architecture

The axvm hypervisor is structured around four core modules that provide a clean abstraction layer for virtualization:

AxVM Core Components

flowchart TD
subgraph subGraph0["Main Exports"]
    AxVM_Export["AxVM"]
    AxVMHal_Export["AxVMHal"]
    AxVCpuRef_Export["AxVCpuRef"]
    AxVMRef_Export["AxVMRef"]
end
LibRS["lib.rsMain Library Interface"]
VM["vm.rsAxVM Core Implementation"]
VCPU["vcpu.rsMulti-Architecture VCPU"]
HAL["hal.rsHardware Abstraction Layer"]
CONFIG["config.rsVM Configuration"]

LibRS --> AxVCpuRef_Export
LibRS --> AxVMHal_Export
LibRS --> AxVMRef_Export
LibRS --> AxVM_Export
LibRS --> CONFIG
LibRS --> HAL
LibRS --> VCPU
LibRS --> VM
VM --> CONFIG
VM --> HAL
VM --> VCPU

The main library interface exports the AxVM struct as the primary virtualization abstraction, along with supporting types for hardware abstraction (AxVMHal) and CPU references (AxVCpuRef, AxVMRef). The has_hardware_support() function provides runtime detection of virtualization capabilities.

Sources: src/lib.rs(L6 - L33)  Cargo.toml(L1 - L8) 

Multi-Architecture Support

The axvm hypervisor provides unified virtualization across x86_64, RISC-V, and AArch64 architectures through architecture-specific backend implementations:

Architecture-Specific Backend Structure

The multi-architecture support is implemented through conditional compilation using cfg-if directives. Each architecture provides its own VCPU implementation while conforming to the common AxArchVCpuImpl interface. The AxVMPerCpu type is defined as axvcpu::AxPerCpu<vcpu::AxVMArchPerCpuImpl<U>> to provide per-CPU state management across all supported architectures.

Sources: Cargo.toml(L29 - L36)  src/lib.rs(L26 - L32)  Cargo.toml(L19 - L21) 

Core Capabilities

The axvm hypervisor provides three primary virtualization capabilities through its modular architecture:

VM Lifecycle and State Management

The VM lifecycle is managed through the core AxVM struct, which coordinates virtual CPU execution, handles VM exits for MMIO and I/O operations, and manages the overall VM state transitions.

Sources: src/lib.rs(L9 - L10)  src/lib.rs(L22 - L24) 

ArceOS Ecosystem Integration

The axvm hypervisor integrates deeply with the ArceOS ecosystem through a set of specialized crates that provide system-level abstractions:

Dependency Architecture

flowchart TD
subgraph subGraph3["External Dependencies"]
    log_crate["logLogging framework"]
    cfg_if["cfg-ifConditional compilation"]
    spin["spinSynchronization primitives"]
end
subgraph subGraph2["ArceOS System Crates"]
    axerrno["axerrnoError handling"]
    memory_addr["memory_addrAddress types"]
    page_table_multiarch["page_table_multiarchMulti-arch page tables"]
    page_table_entry["page_table_entryARM EL2 support"]
    percpu["percpuPer-CPU data structures"]
end
subgraph subGraph1["ArceOS Hypervisor Modules"]
    axvcpu["axvcpuVirtual CPU abstraction"]
    axaddrspace["axaddrspaceAddress space management"]
    axdevice["axdeviceDevice emulation framework"]
    axvmconfig["axvmconfigConfiguration management"]
end
subgraph subGraph0["AxVM Core"]
    axvm["axvm crate"]
end

axvm --> axaddrspace
axvm --> axdevice
axvm --> axerrno
axvm --> axvcpu
axvm --> axvmconfig
axvm --> cfg_if
axvm --> log_crate
axvm --> memory_addr
axvm --> page_table_entry
axvm --> page_table_multiarch
axvm --> percpu
axvm --> spin

The crate uses the no_std environment for embedded and kernel-space deployment. All ArceOS hypervisor modules are sourced from the arceos-hypervisor GitHub organization, providing a cohesive ecosystem for virtualization components.

Sources: Cargo.toml(L10 - L27)  src/lib.rs(L1 - L13) 

The AxVMHal trait provides the hardware abstraction interface that must be implemented by the hosting environment, enabling axvm to run on different underlying platforms while maintaining a consistent virtualization interface.

Sources: src/lib.rs(L21) 

Core Architecture

Relevant source files

• src/lib.rs
• src/vm.rs

This document provides a detailed examination of axvm's fundamental architecture, focusing on the core components that enable virtual machine creation, management, and execution. It covers the main AxVM struct, its internal organization, lifecycle management, and integration with hardware abstraction layers.

For information about specific vCPU architecture implementations, see Virtual CPU Architecture. For details about hardware abstraction interfaces, see Hardware Abstraction Layer. For VM configuration mechanisms, see Configuration System.

Architecture Overview

The axvm hypervisor is built around a central AxVM struct that coordinates virtual machine execution through a clean separation of concerns. The architecture employs generic type parameters to abstract hardware and vCPU implementations while maintaining compile-time optimization.

flowchart TD
subgraph subGraph4["Generic Type Parameters"]
    HAL["H: AxVMHal"]
    VCPUHAL["U: AxVCpuHal"]
end
subgraph subGraph3["VM Lifecycle Flags"]
    Running["running: AtomicBool"]
    ShuttingDown["shutting_down: AtomicBool"]
end
subgraph subGraph2["Mutable VM State"]
    AddrSpace["address_space: Mutex<AddrSpace<H::PagingHandler>>"]
    PhantomData["_marker: PhantomData<H>"]
end
subgraph subGraph1["Immutable VM State"]
    VMId["id: usize"]
    VMConfig["config: AxVMConfig"]
    VCpuList["vcpu_list: Box[AxVCpuRef<U>]"]
    VMDevices["devices: AxVmDevices"]
end
subgraph subGraph0["Core VM Structure"]
    AxVM["AxVM<H, U>"]
    AxVMInnerConst["AxVMInnerConst<U>"]
    AxVMInnerMut["AxVMInnerMut<H>"]
end

AxVM --> AxVMInnerConst
AxVM --> AxVMInnerMut
AxVM --> HAL
AxVM --> Running
AxVM --> ShuttingDown
AxVM --> VCPUHAL
AxVMInnerConst --> VCpuList
AxVMInnerConst --> VMConfig
AxVMInnerConst --> VMDevices
AxVMInnerConst --> VMId
AxVMInnerMut --> AddrSpace
AxVMInnerMut --> PhantomData

Sources: src/vm.rs(L47 - L53)  src/vm.rs(L31 - L39)  src/vm.rs(L41 - L45) 

Core VM Structure

The AxVM struct is organized into distinct sections that separate immutable configuration data from mutable runtime state. This design ensures thread safety while minimizing lock contention during VM execution.

Data Organization

The immutable section (AxVMInnerConst) contains the VM identifier, configuration, vCPU list, and device registry. The mutable section (AxVMInnerMut) contains the address space with its associated page table handler.

Sources: src/vm.rs(L47 - L53)  src/vm.rs(L31 - L39)  src/vm.rs(L41 - L45) 

VM Creation and Initialization

VM creation follows a multi-stage initialization process that sets up vCPUs, memory regions, and devices before the VM can be executed.

sequenceDiagram
    participant AxVMnew as "AxVM::new()"
    participant vCPUCreation as "vCPU Creation"
    participant MemorySetup as "Memory Setup"
    participant DeviceSetup as "Device Setup"
    participant AddrSpace as "AddrSpace"

    AxVMnew ->> vCPUCreation: "Create VCpus from config"
    Note over vCPUCreation: "Architecture-specific AxVCpuCreateConfig"
    vCPUCreation ->> vCPUCreation: "Arc::new(VCpu::new())"
    AxVMnew ->> MemorySetup: "Process memory_regions()"
    MemorySetup ->> AddrSpace: "AddrSpace::new_empty()"
    loop "For each memory region"
        MemorySetup ->> MemorySetup: "Validate MappingFlags"
    alt "VmMemMappingType::MapIentical"
        MemorySetup ->> AddrSpace: "map_linear()"
    else "VmMemMappingType::MapAlloc"
        MemorySetup ->> AddrSpace: "map_alloc()"
    end
    end
    loop "For each pass_through_device"
        MemorySetup ->> AddrSpace: "map_linear() with DEVICE flags"
    end
    AxVMnew ->> DeviceSetup: "AxVmDevices::new()"
    DeviceSetup ->> DeviceSetup: "Initialize from emu_configs"
    AxVMnew ->> AxVMnew: "Create AxVM struct"
    loop "For each vCPU"
        AxVMnew ->> vCPUCreation: "vcpu.setup(entry, ept_root, config)"
    end

Sources: src/vm.rs(L59 - L219)  src/vm.rs(L95 - L163)  src/vm.rs(L182 - L184)  src/vm.rs(L204 - L217) 

Memory Region Setup

The VM initialization process handles two distinct memory mapping types:

• VmMemMappingType::MapIentical: Creates identity mappings where guest physical addresses map directly to host physical addresses
• VmMemMappingType::MapAlloc: Allocates new physical memory pages that may be non-contiguous in host physical memory

The address space spans from VM_ASPACE_BASE (0x0) to VM_ASPACE_SIZE (0x7fff_ffff_f000), providing a large virtual address space for guest execution.

Sources: src/vm.rs(L18 - L19)  src/vm.rs(L122 - L162) 

VM Lifecycle Management

The VM lifecycle is controlled through atomic boolean flags that coordinate state transitions across multiple threads without requiring locks.

State Validation

VM operations include state validation to prevent invalid transitions:

• boot() checks hardware virtualization support and prevents double-booting
• shutdown() prevents multiple shutdown attempts
• run_vcpu() validates vCPU existence before execution

Sources: src/vm.rs(L277 - L288)  src/vm.rs(L299 - L310)  src/vm.rs(L328 - L376) 

vCPU Execution Loop

The core VM execution occurs in the run_vcpu() method, which implements a continuous loop that handles various exit reasons from hardware virtualization.

flowchart TD
Start["run_vcpu(vcpu_id)"]
ValidateVCpu["vcpu(vcpu_id)?"]
BindVCpu["vcpu.bind()"]
RunLoop["vcpu.run()"]
ExitReason["Match exit_reason"]
MmioRead["MmioRead"]
MmioWrite["MmioWrite"]
PageFault["NestedPageFault"]
IoOps["IoRead/IoWrite"]
Other["Other reasons"]
DeviceRead["devices.handle_mmio_read()"]
SetGPR["vcpu.set_gpr()"]
Continue["Continue loop"]
DeviceWrite["devices.handle_mmio_write()"]
HandleFault["address_space.handle_page_fault()"]
Unbind["vcpu.unbind()"]
Return["Return exit_reason"]

BindVCpu --> RunLoop
Continue --> RunLoop
DeviceRead --> SetGPR
DeviceWrite --> Continue
ExitReason --> IoOps
ExitReason --> MmioRead
ExitReason --> MmioWrite
ExitReason --> Other
ExitReason --> PageFault
HandleFault --> Continue
IoOps --> Continue
MmioRead --> DeviceRead
MmioWrite --> DeviceWrite
Other --> Unbind
PageFault --> HandleFault
RunLoop --> ExitReason
SetGPR --> Continue
Start --> ValidateVCpu
Unbind --> Return
ValidateVCpu --> BindVCpu

The execution loop handles several categories of VM exits:

• MMIO Operations: Delegated to the device emulation layer via AxVmDevices
• Page Faults: Handled by the address space's page fault handler
• I/O Operations: Currently handled as no-ops
• Unhandled Exits: Return control to the caller

Sources: src/vm.rs(L328 - L376)  src/vm.rs(L335 - L372) 

Component Integration

The AxVM struct integrates with several key subsystems through well-defined interfaces:

Address Space Integration

flowchart TD
AxVM["AxVM"]
AddrSpaceMutex["Mutex<AddrSpace<H::PagingHandler>>"]
AddrSpace["AddrSpace"]
PageTableRoot["page_table_root()"]
HandlePageFault["handle_page_fault()"]
TranslateBuffer["translated_byte_buffer()"]
EPTRoot["ept_root() for vCPU setup"]
ImageLoad["get_image_load_region()"]

AddrSpace --> HandlePageFault
AddrSpace --> PageTableRoot
AddrSpace --> TranslateBuffer
AddrSpaceMutex --> AddrSpace
AxVM --> AddrSpaceMutex
PageTableRoot --> EPTRoot
TranslateBuffer --> ImageLoad

Device Integration

flowchart TD
AxVM["AxVM"]
AxVmDevices["AxVmDevices"]
MmioRead["handle_mmio_read()"]
MmioWrite["handle_mmio_write()"]
DeviceConfig["AxVmDeviceConfig"]
EmuConfigs["emu_configs: Vec"]

AxVM --> AxVmDevices
AxVmDevices --> DeviceConfig
AxVmDevices --> MmioRead
AxVmDevices --> MmioWrite
DeviceConfig --> EmuConfigs

vCPU Management

The VM maintains references to vCPUs through Arc<VCpu<AxArchVCpuImpl<U>>> allowing shared ownership across threads while providing architecture-independent access through the AxArchVCpu trait.

Sources: src/vm.rs(L21 - L26)  src/vm.rs(L242 - L250)  src/vm.rs(L260 - L270)  src/vm.rs(L315 - L318) 

Type System and Generics

The architecture leverages Rust's type system to provide compile-time guarantees about hardware compatibility and vCPU implementations:

• H: AxVMHal: Hardware abstraction layer interface
• U: AxVCpuHal: vCPU hardware abstraction layer interface
• Type Aliases: AxVMRef<H, U> and AxVCpuRef<U> provide ergonomic reference types

This design enables architecture-specific optimizations while maintaining a unified API across different hardware platforms.

Sources: src/vm.rs(L21 - L29)  src/lib.rs(L21 - L27) 

Virtual Machine Implementation

Relevant source files

• src/vm.rs

This document covers the core virtual machine implementation in AxVM, focusing on the AxVM struct and its lifecycle management. This includes VM creation, booting, vCPU execution coordination, memory management, and device emulation integration. For details about the underlying vCPU architecture abstraction, see Virtual CPU Architecture. For hardware abstraction interfaces, see Hardware Abstraction Layer.

AxVM Structure Overview

The AxVM struct serves as the central coordinator for all virtual machine operations. It uses a generic design with two type parameters to support multiple architectures and hardware abstraction layers.

classDiagram
class AxVM {
    +running: AtomicBool
    +shutting_down: AtomicBool
    +inner_const: AxVMInnerConst
    +inner_mut: AxVMInnerMut
    +new(config: AxVMConfig) AxResult~AxVMRef~
    +boot() AxResult
    +shutdown() AxResult
    +run_vcpu(vcpu_id: usize) AxResult~AxVCpuExitReason~
    +vcpu(vcpu_id: usize) Option~AxVCpuRef~
    +get_devices() &AxVmDevices
}

class AxVMInnerConst {
    +id: usize
    +config: AxVMConfig
    +vcpu_list: Box[AxVCpuRef]
    +devices: AxVmDevices
    
}

class AxVMInnerMut {
    +address_space: Mutex~AddrSpace~
    +_marker: PhantomData
    
}

AxVM  *--  AxVMInnerConst : "contains"
AxVM  *--  AxVMInnerMut : "contains"

Sources: src/vm.rs(L47 - L53)  src/vm.rs(L31 - L40)  src/vm.rs(L41 - L45) 

The structure separates immutable data (AxVMInnerConst) from mutable data (AxVMInnerMut) to optimize concurrent access patterns. The type aliases provide convenient references for shared ownership.

Sources: src/vm.rs(L21 - L29) 

VM Creation Process

VM creation follows a multi-stage initialization process that sets up all necessary components before the VM can be booted.

sequenceDiagram
    participant Client as Client
    participant AxVMnew as AxVM::new
    participant VCpuCreation as VCpu Creation
    participant AddressSpace as Address Space
    participant DeviceSetup as Device Setup
    participant VCpuSetup as VCpu Setup

    Client ->> AxVMnew: "AxVMConfig"
    Note over AxVMnew: "Parse vcpu affinities"
    AxVMnew ->> VCpuCreation: "for each vcpu_id, phys_cpu_set"
    VCpuCreation ->> VCpuCreation: "AxVCpuCreateConfig (arch-specific)"
    VCpuCreation ->> AxVMnew: "Arc<VCpu>"
    Note over AxVMnew: "Setup memory regions"
    AxVMnew ->> AddressSpace: "AddrSpace::new_empty()"
    AxVMnew ->> AddressSpace: "map_linear() / map_alloc()"
    Note over AxVMnew: "Setup passthrough devices"
    AxVMnew ->> AddressSpace: "map_linear() with DEVICE flags"
    AxVMnew ->> DeviceSetup: "AxVmDevices::new()"
    Note over AxVMnew: "Create AxVM instance"
    AxVMnew ->> VCpuSetup: "vcpu.setup(entry, ept_root, config)"
    VCpuSetup ->> AxVMnew: "Result"
    AxVMnew ->> Client: "Arc<AxVM>"

Sources: src/vm.rs(L59 - L220) 

vCPU Creation and Configuration

The VM creates vCPUs based on the configuration's CPU affinity settings. Each architecture requires specific configuration parameters:

Sources: src/vm.rs(L61 - L93) 

Memory Region Setup

The VM supports two mapping types for memory regions:

• MapIentical: Direct mapping where guest physical addresses map to identical host physical addresses
• MapAlloc: Dynamic allocation where the hypervisor allocates physical memory for the guest

flowchart TD
config["Memory Region Config"]
check_flags["Check MappingFlags validity"]
map_type["Mapping Type?"]
alloc_at["H::alloc_memory_region_at()"]
map_linear1["address_space.map_linear()"]
map_alloc["address_space.map_alloc()"]
complete["Memory region mapped"]

alloc_at --> map_linear1
check_flags --> map_type
config --> check_flags
map_alloc --> complete
map_linear1 --> complete
map_type --> alloc_at
map_type --> map_alloc

Sources: src/vm.rs(L95 - L163) 

For passthrough devices, the VM creates device mappings with the DEVICE flag to ensure proper memory attributes for hardware access.

Sources: src/vm.rs(L165 - L180) 

VM Lifecycle Management

The VM follows a simple state machine with atomic boolean flags for coordination between multiple threads.

Sources: src/vm.rs(L49 - L50)  src/vm.rs(L278 - L310) 

Boot Process

The boot() method performs hardware support validation before allowing VM execution:

1. Check hardware virtualization support via has_hardware_support()
2. Verify VM is not already running
3. Set running flag to true atomically

Sources: src/vm.rs(L278 - L288) 

Shutdown Process

The shutdown() method initiates graceful VM termination:

1. Check VM is not already shutting down
2. Set shutting_down flag to true
3. Note: VM re-initialization is not currently supported

Sources: src/vm.rs(L299 - L310) 

vCPU Execution and Exit Handling

The run_vcpu() method implements the main VM execution loop, handling various exit reasons from the hardware virtualization layer.

flowchart TD
start["run_vcpu(vcpu_id)"]
bind["vcpu.bind()"]
run_loop["vcpu.run()"]
check_exit["Exit Reason?"]
mmio_read["devices.handle_mmio_read()"]
set_reg["vcpu.set_gpr(reg, val)"]
continue_loop["continue_loop"]
mmio_write["devices.handle_mmio_write()"]
io_handled["Handle I/O (placeholder)"]
page_fault["address_space.handle_page_fault()"]
fault_handled["Handled?"]
exit_loop["Break loop"]
unbind["vcpu.unbind()"]
return_exit["Return AxVCpuExitReason"]

bind --> run_loop
check_exit --> exit_loop
check_exit --> io_handled
check_exit --> mmio_read
check_exit --> mmio_write
check_exit --> page_fault
continue_loop --> run_loop
exit_loop --> unbind
fault_handled --> continue_loop
fault_handled --> exit_loop
io_handled --> continue_loop
mmio_read --> set_reg
mmio_write --> continue_loop
page_fault --> fault_handled
run_loop --> check_exit
set_reg --> continue_loop
start --> bind
unbind --> return_exit

Sources: src/vm.rs(L328 - L376) 

Exit Reason Handling

The VM handles several categories of VM exits:

Sources: src/vm.rs(L338 - L368) 

Address Space and Memory Management

The VM uses a two-stage address translation system with an AddrSpace that manages guest-to-host physical address mappings.

Address Space Configuration

The VM address space uses fixed bounds to limit guest memory access:

Sources: src/vm.rs(L18 - L19) 

EPT Root Access

The ept_root() method provides access to the page table root for hardware virtualization:

#![allow(unused)]
fn main() {
pub fn ept_root(&self) -> HostPhysAddr {
    self.inner_mut.address_space.lock().page_table_root()
}
}

Sources: src/vm.rs(L248 - L250) 

Image Loading Support

The VM provides functionality to obtain host virtual addresses for loading guest images, handling potentially non-contiguous physical memory:

Sources: src/vm.rs(L260 - L270) 

Device Integration

The VM integrates with the AxVM device emulation framework through the AxVmDevices component, which handles MMIO operations for emulated devices.

Device initialization occurs during VM creation, using the configuration's emulated device list:

Sources: src/vm.rs(L182 - L184) 

Device access is coordinated through the exit handling mechanism, where MMIO reads and writes are forwarded to the appropriate device handlers.

Sources: src/vm.rs(L345 - L355) 

Error Handling and Safety

The VM implementation uses Rust's type system and AxResult for comprehensive error handling:

• Hardware Support: Validates virtualization capabilities before boot
• State Validation: Prevents invalid state transitions (e.g., double boot)
• Resource Management: Uses Arc for safe sharing across threads
• Memory Safety: Leverages Rust's ownership system for memory region management

Sources: src/vm.rs(L7)  src/vm.rs(L279 - L287)  src/vm.rs(L300 - L310) 

Virtual CPU Architecture

Relevant source files

• src/vcpu.rs

Purpose and Scope

The Virtual CPU Architecture in axvm provides a unified abstraction layer for virtual CPU management across multiple hardware architectures. This system enables axvm to support x86_64, RISC-V, and AArch64 platforms through architecture-specific backends while maintaining a consistent interface for the core hypervisor components.

This document covers the multi-architecture vCPU abstraction, architecture-specific implementations, configuration mechanisms, and hardware support detection. For information about the broader VM lifecycle and management, see Virtual Machine Implementation. For hardware abstraction interfaces beyond vCPUs, see Hardware Abstraction Layer.

Multi-Architecture Abstraction Layer

The vCPU architecture uses compile-time conditional compilation to select the appropriate backend for each target architecture. This design provides zero-cost abstraction while maintaining type safety across different hardware platforms.

Architecture Selection and Type Aliases

The core abstraction is implemented through type aliases that map to architecture-specific implementations based on the compilation target:

flowchart TD
subgraph subGraph3["AArch64 Backend"]
    Aarch64VCpu["Aarch64VCpu"]
    Aarch64PerCpu["Aarch64PerCpu"]
    Aarch64VCpuCreateConfig["Aarch64VCpuCreateConfig"]
    Aarch64HWSupport["ARM EL2 Support Check"]
end
subgraph subGraph2["RISC-V Backend"]
    RISCVVCpu["RISCVVCpu"]
    RISCVPerCpu["RISCVPerCpu"]
    RISCVVCpuCreateConfig["RISCVVCpuCreateConfig"]
    RISCVHWSupport["RISC-V H-Extension Check"]
end
subgraph subGraph1["x86_64 Backend"]
    VmxArchVCpu["VmxArchVCpu"]
    VmxArchPerCpuState["VmxArchPerCpuState"]
    UnitConfig["() Unit Type"]
    VmxHWSupport["VMX Hardware Check"]
end
subgraph subGraph0["Common Interface Types"]
    AxArchVCpuImpl["AxArchVCpuImplMain vCPU Interface"]
    AxVMArchPerCpuImpl["AxVMArchPerCpuImplPer-CPU State"]
    AxVCpuCreateConfig["AxVCpuCreateConfigCreation Configuration"]
    HasHWSupport["has_hardware_supportHardware Detection"]
end

AxArchVCpuImpl --> Aarch64VCpu
AxArchVCpuImpl --> RISCVVCpu
AxArchVCpuImpl --> VmxArchVCpu
AxVCpuCreateConfig --> Aarch64VCpuCreateConfig
AxVCpuCreateConfig --> RISCVVCpuCreateConfig
AxVCpuCreateConfig --> UnitConfig
AxVMArchPerCpuImpl --> Aarch64PerCpu
AxVMArchPerCpuImpl --> RISCVPerCpu
AxVMArchPerCpuImpl --> VmxArchPerCpuState
HasHWSupport --> Aarch64HWSupport
HasHWSupport --> RISCVHWSupport
HasHWSupport --> VmxHWSupport

Sources: src/vcpu.rs(L3 - L27) 

Architecture-Specific Implementations

Each supported architecture provides its own implementation of the vCPU interface, tailored to the specific virtualization capabilities and requirements of that platform.

x86_64 Implementation

The x86_64 backend uses Intel VT-x and AMD-V virtualization extensions through the x86_vcpu crate. This implementation provides VMX-based virtualization with EPT (Extended Page Tables) support.

Sources: src/vcpu.rs(L4 - L8) 

RISC-V Implementation

The RISC-V backend leverages the RISC-V hypervisor extension (H-extension) through the riscv_vcpu crate, providing virtualization support for RISC-V processors.

Sources: src/vcpu.rs(L16 - L20) 

AArch64 Implementation

The AArch64 backend utilizes ARM virtualization extensions (EL2) through the arm_vcpu crate, enabling hypervisor functionality on ARM processors.

Sources: src/vcpu.rs(L21 - L26) 

vCPU Lifecycle and Integration

The vCPU architecture integrates with the broader axvm system through well-defined interfaces that handle creation, configuration, and execution of virtual CPUs.

vCPU Creation and Configuration Flow

sequenceDiagram
    participant AxVM as "AxVM"
    participant AxVCpuCreateConfig as "AxVCpuCreateConfig"
    participant has_hardware_support as "has_hardware_support"
    participant AxArchVCpuImpl as "AxArchVCpuImpl"
    participant AxVMArchPerCpuImpl as "AxVMArchPerCpuImpl"

    AxVM ->> has_hardware_support: "Check virtualization support"
    has_hardware_support -->> AxVM: "Hardware capability result"
    alt Hardware Supported
        AxVM ->> AxVCpuCreateConfig: "Create vCPU configuration"
        AxVM ->> AxArchVCpuImpl: "new(config)"
        AxArchVCpuImpl ->> AxVMArchPerCpuImpl: "Initialize per-CPU state"
        AxVMArchPerCpuImpl -->> AxArchVCpuImpl: "Per-CPU state ready"
        AxArchVCpuImpl -->> AxVM: "vCPU instance created"
    else Hardware Not Supported
        AxVM ->> AxVM: "Return error or fallback"
    end

Sources: src/vcpu.rs(L1 - L27) 

Hardware Support Detection

Each architecture backend provides a has_hardware_support function that performs runtime detection of virtualization capabilities. This enables axvm to gracefully handle systems that may not have the necessary hardware extensions.

The hardware support detection is critical for:

• Determining if the host system can run the hypervisor
• Selecting appropriate virtualization features
• Providing meaningful error messages when virtualization is unavailable

Sources: src/vcpu.rs(L7 - L25) 

Physical Frame Interface Requirements

The x86_64 implementation has a unique requirement where the PhysFrameIf trait must be implemented by the application layer. This design decision aligns with axvm's architecture principle of separating hypervisor functionality from OS resource management.

The PhysFrameIf implementation is handled by the vmm_app component rather than within axvm itself, allowing for flexible memory management strategies while maintaining clear separation of concerns.

Sources: src/vcpu.rs(L10 - L15) 

Integration with VM Core

The vCPU architecture seamlessly integrates with the main AxVM implementation through the standardized interfaces, enabling the VM to manage virtual CPUs without needing architecture-specific knowledge. This abstraction allows the core VM logic to remain architecture-agnostic while leveraging the full capabilities of each supported platform.

Sources: src/vcpu.rs(L1 - L27) 

Hardware Abstraction Layer

Relevant source files

• src/hal.rs

The Hardware Abstraction Layer (HAL) provides a standardized interface for platform-specific operations that the axvm hypervisor requires from the underlying host system. This abstraction enables axvm to run on different host operating systems and hypervisors by delegating low-level hardware operations to implementations of the AxVMHal trait.

For information about virtual machine lifecycle management, see Virtual Machine Implementation. For details about multi-architecture CPU support, see Virtual CPU Architecture.

AxVMHal Trait Interface

The core of the hardware abstraction layer is the AxVMHal trait, which defines the contract between axvm and the underlying host system. This trait must be implemented by any host system that wants to run axvm virtual machines.

AxVMHal Trait Structure

classDiagram
class AxVMHal {
    <<trait>>
    +PagingHandler: page_table_multiarch::PagingHandler
    +alloc_memory_region_at(base: HostPhysAddr, size: usize) bool
    +dealloc_memory_region_at(base: HostPhysAddr, size: usize)
    +virt_to_phys(vaddr: HostVirtAddr) HostPhysAddr
    +current_time_nanos() u64
}

class PagingHandler {
    <<interface>>
    +page_table_multiarch::PagingHandler
    
}

class HostPhysAddr {
    +axaddrspace::HostPhysAddr
    
}

class HostVirtAddr {
    +axaddrspace::HostVirtAddr
    
}

AxVMHal  -->  PagingHandler : "associated type"
AxVMHal  -->  HostPhysAddr : "uses"
AxVMHal  -->  HostVirtAddr : "uses"

Sources: src/hal.rs(L1 - L22) 

Memory Management Abstraction

The HAL provides two primary memory management operations that abstract the underlying host's memory allocation mechanisms:

These methods enable axvm to request specific physical memory regions from the host, which is essential for guest physical memory layout and device memory mapping.

Memory Management Flow

sequenceDiagram
    participant AxVM as "AxVM"
    participant AxVMHalImplementation as "AxVMHal Implementation"
    participant HostSystem as "Host System"

    AxVM ->> AxVMHalImplementation: "alloc_memory_region_at(base, size)"
    AxVMHalImplementation ->> HostSystem: "Request physical memory allocation"
    HostSystem -->> AxVMHalImplementation: "Success/failure"
    AxVMHalImplementation -->> AxVM: "bool result"
    Note over AxVM,HostSystem: "VM operation continues..."
    AxVM ->> AxVMHalImplementation: "dealloc_memory_region_at(base, size)"
    AxVMHalImplementation ->> HostSystem: "Release physical memory"
    HostSystem -->> AxVMHalImplementation: "Memory freed"
    AxVMHalImplementation -->> AxVM: "() completion"

Sources: src/hal.rs(L8 - L14) 

Address Translation Interface

The virt_to_phys method provides virtual-to-physical address translation, enabling axvm to convert host virtual addresses to their corresponding physical addresses. This is crucial for setting up guest physical memory mappings and ensuring proper memory coherency.

Address Translation Architecture

flowchart TD
subgraph subGraph2["VM Memory Management"]
    GuestPhys["Guest Physical Memory"]
    HostMapping["Host Memory Mapping"]
end
subgraph subGraph1["Page Table Integration"]
    PagingHandler["PagingHandler(page_table_multiarch)"]
    PageTable["Multi-arch Page Tables"]
end
subgraph subGraph0["Address Translation Layer"]
    VirtAddr["HostVirtAddr(axaddrspace)"]
    VirtToPhys["virt_to_phys()"]
    PhysAddr["HostPhysAddr(axaddrspace)"]
end

PagingHandler --> PageTable
PhysAddr --> GuestPhys
VirtAddr --> HostMapping
VirtAddr --> VirtToPhys
VirtToPhys --> PagingHandler
VirtToPhys --> PhysAddr

Sources: src/hal.rs(L16 - L17)  src/hal.rs(L6) 

Time Management

The HAL provides time services through the current_time_nanos method, which returns the current time in nanoseconds. This enables axvm to implement time-based features such as:

• Performance monitoring and profiling
• Timeout mechanisms for VM operations
• Guest time synchronization
• Event scheduling and timing

Sources: src/hal.rs(L19 - L20) 

Integration with ArceOS Ecosystem

The HAL interfaces directly with several components from the ArceOS ecosystem:

HAL Ecosystem Integration

flowchart TD
subgraph subGraph3["Host Implementation"]
    HostOS["Host OS/Hypervisor"]
    MemMgmt["Memory Management"]
    TimeService["Time Services"]
end
subgraph subGraph2["Page Table Infrastructure"]
    PageTableMultiarch["page_table_multiarch"]
    PagingHandlerImpl["PagingHandler impl"]
end
subgraph subGraph1["ArceOS Address Space"]
    HostPhysAddr["HostPhysAddr"]
    HostVirtAddr["HostVirtAddr"]
    AddrSpace["axaddrspace"]
end
subgraph subGraph0["AxVM HAL Layer"]
    AxVMHal["AxVMHal trait"]
    PagingType["PagingHandler type"]
end

AddrSpace --> HostPhysAddr
AddrSpace --> HostVirtAddr
AxVMHal --> HostOS
AxVMHal --> HostPhysAddr
AxVMHal --> HostVirtAddr
MemMgmt --> AxVMHal
PagingHandlerImpl --> PagingType
PagingType --> PageTableMultiarch
TimeService --> AxVMHal

The HAL serves as the critical bridge between axvm's hardware-agnostic virtualization logic and the host system's specific capabilities, enabling portability across different host environments while maintaining performance and functionality.

Sources: src/hal.rs(L1 - L6) 

Configuration System

Relevant source files

• src/config.rs

This document covers the AxVM hypervisor's configuration system, which manages VM setup parameters including CPU allocation, memory regions, device configurations, and image load addresses. The configuration system transforms TOML configuration files into structured data that drives VM creation and initialization.

For information about the VM creation process that uses these configurations, see Virtual Machine Implementation. For details about hardware abstraction interfaces, see Hardware Abstraction Layer.

Configuration Flow Overview

The configuration system follows a two-stage transformation process: external TOML configuration files are first parsed into raw configuration structures, then converted to internal VM configuration objects used during VM creation.

flowchart TD
subgraph subGraph1["Processed Configuration (axvm)"]
    VMConfig["AxVMConfig"]
    CPUConfig["cpu_config: AxVCpuConfig"]
    ImageConfig["image_config: VMImageConfig"]
    MemConfig["memory_regions: Vec"]
    EmuDevices["emu_devices: Vec"]
    PassDevices["pass_through_devices: Vec"]
end
subgraph subGraph0["Raw Configuration (axvmconfig crate)"]
    CrateConfig["AxVMCrateConfig"]
    BaseConfig["base: VM metadata"]
    KernelConfig["kernel: Image addresses"]
    DeviceConfig["devices: Device setup"]
end
TOML["TOML Configuration File"]
VMCreation["AxVM::new(config)"]
AddressSpace["Address Space Setup"]
VCPUSetup["vCPU Creation"]
DeviceSetup["Device Initialization"]

CrateConfig --> BaseConfig
CrateConfig --> DeviceConfig
CrateConfig --> KernelConfig
CrateConfig --> VMConfig
TOML --> CrateConfig
VMConfig --> CPUConfig
VMConfig --> EmuDevices
VMConfig --> ImageConfig
VMConfig --> MemConfig
VMConfig --> PassDevices
VMConfig --> VMCreation
VMCreation --> AddressSpace
VMCreation --> DeviceSetup
VMCreation --> VCPUSetup

Sources: src/config.rs(L1 - L12)  src/config.rs(L63 - L87) 

Configuration Structure Hierarchy

The configuration system defines several interconnected structures that represent different aspects of VM setup. The main configuration object AxVMConfig aggregates specialized configuration components.

classDiagram
class AxVMConfig {
    -id: usize
    -name: String
    -vm_type: VMType
    -cpu_num: usize
    -phys_cpu_ids: Option~Vec~usize~~
    -phys_cpu_sets: Option~Vec~usize~~
    -cpu_config: AxVCpuConfig
    -image_config: VMImageConfig
    -memory_regions: Vec~VmMemConfig~
    -emu_devices: Vec~EmulatedDeviceConfig~
    -pass_through_devices: Vec~PassThroughDeviceConfig~
    +id() usize
    +name() String
    +get_vcpu_affinities_pcpu_ids() Vec~(usize, Option~usize~, usize) ~
    +bsp_entry() GuestPhysAddr
    +ap_entry() GuestPhysAddr
    +memory_regions() Vec~VmMemConfig~
    +emu_devices() Vec~EmulatedDeviceConfig~
    +pass_through_devices() Vec~PassThroughDeviceConfig~
}

class AxVCpuConfig {
    +bsp_entry: GuestPhysAddr
    +ap_entry: GuestPhysAddr
    
}

class VMImageConfig {
    +kernel_load_gpa: GuestPhysAddr
    +bios_load_gpa: Option~GuestPhysAddr~
    +dtb_load_gpa: Option~GuestPhysAddr~
    +ramdisk_load_gpa: Option~GuestPhysAddr~
    
}

class AxVMCrateConfig {
    +base: BaseConfig
    +kernel: KernelConfig
    +devices: DeviceConfig
    
}

AxVMConfig  *--  AxVCpuConfig
AxVMConfig  *--  VMImageConfig
AxVMConfig  ..>  AxVMCrateConfig : "From trait"

Sources: src/config.rs(L24 - L31)  src/config.rs(L34 - L44)  src/config.rs(L47 - L61) 

Configuration Transformation Process

The From<AxVMCrateConfig> trait implementation handles the conversion from raw TOML-derived configuration to the internal VM configuration format. This transformation maps external configuration fields to internal structures and converts address types to GuestPhysAddr.

Key Transformation Steps

Sources: src/config.rs(L63 - L87) 

CPU Configuration

The CPU configuration system manages virtual CPU allocation, entry points, and physical CPU affinity mapping. The AxVCpuConfig structure defines entry addresses for both Bootstrap Processor (BSP) and Application Processor (AP) initialization.

vCPU Affinity Management

The get_vcpu_affinities_pcpu_ids method returns CPU mapping information as tuples containing:

• vCPU ID (0 to cpu_num-1)
• Physical CPU affinity mask (optional bitmap)
• Physical CPU ID (defaults to vCPU ID if not specified)

flowchart TD
subgraph subGraph0["CPU Configuration"]
    CPUNum["cpu_num: usize"]
    VCPUIDs["vCPU IDs: 0..cpu_num"]
    PhysCPUIDs["phys_cpu_ids: Option>"]
    PhysIDs["Physical IDs mapping"]
    PhysCPUSets["phys_cpu_sets: Option>"]
    AffinityMask["Affinity bitmasks"]
    TupleGen["get_vcpu_affinities_pcpu_ids()"]
    Result["Vec<(vcpu_id, affinity_mask, phys_id)>"]
end

AffinityMask --> TupleGen
CPUNum --> VCPUIDs
PhysCPUIDs --> PhysIDs
PhysCPUSets --> AffinityMask
PhysIDs --> TupleGen
TupleGen --> Result
VCPUIDs --> TupleGen

Sources: src/config.rs(L24 - L31)  src/config.rs(L108 - L124) 

Memory Configuration

Memory configuration defines the guest physical memory layout through a vector of VmMemConfig structures. Each memory region specifies mapping type, guest physical address ranges, and access permissions.

The VMImageConfig structure manages load addresses for different VM image components:

Sources: src/config.rs(L34 - L44)  src/config.rs(L144 - L146) 

Device Configuration

The device configuration system supports two types of devices: emulated devices and passthrough devices. Device configurations are stored in separate vectors and accessed through dedicated getter methods.

flowchart TD
subgraph subGraph1["External Types (axvmconfig)"]
    EmulatedDeviceConfig["EmulatedDeviceConfig"]
    PassThroughDeviceConfig["PassThroughDeviceConfig"]
end
subgraph subGraph0["Device Configuration"]
    DeviceConfig["devices section (TOML)"]
    EmuDevices["emu_devices: Vec"]
    PassDevices["passthrough_devices: Vec"]
    EmuGetter["emu_devices() getter"]
    PassGetter["pass_through_devices() getter"]
    VMSetup["VM Device Setup"]
end

DeviceConfig --> EmuDevices
DeviceConfig --> PassDevices
EmuDevices --> EmuGetter
EmuDevices --> EmulatedDeviceConfig
EmuGetter --> VMSetup
PassDevices --> PassGetter
PassDevices --> PassThroughDeviceConfig
PassGetter --> VMSetup

Sources: src/config.rs(L9 - L12)  src/config.rs(L149 - L156) 

Configuration Usage in VM Creation

The AxVMConfig object serves as the primary input to VM creation, providing all necessary parameters for initializing VM components. The configuration system exposes specific getter methods that VM creation logic uses to access different configuration aspects.

Configuration Access Methods

The configuration provides specialized accessor methods for different VM subsystems:

• bsp_entry() / ap_entry(): CPU entry points for processor initialization
• image_config(): Image load addresses for kernel and auxiliary components
• memory_regions(): Memory layout and mapping configuration
• emu_devices() / pass_through_devices(): Device configuration for emulation and passthrough
• get_vcpu_affinities_pcpu_ids(): CPU affinity and physical mapping information

Sources: src/config.rs(L126 - L156) 

Dependencies and Integration

Relevant source files

• Cargo.toml

This document explains how the AxVM hypervisor integrates with the ArceOS ecosystem and manages its external dependencies. It covers the modular architecture that enables multi-platform virtualization support and the specific crate dependencies that provide core functionality.

For information about the core VM implementation details, see Virtual Machine Implementation. For architecture-specific virtualization backends, see Virtual CPU Architecture.

ArceOS Ecosystem Integration

AxVM is designed as a core component within the ArceOS hypervisor ecosystem, leveraging specialized crates for different aspects of virtualization. The integration follows a modular design where each major subsystem is provided by a dedicated crate.

Core Hypervisor Dependencies

The primary integration occurs through four key ArceOS-Hypervisor modules that provide the foundational virtualization capabilities:

flowchart TD
AXVM["axvmCore Hypervisor"]
AXVCPU["axvcpuVirtual CPU Management"]
AXADDR["axaddrspaceAddress Space Management"]
AXDEV["axdeviceDevice Emulation"]
AXCONF["axvmconfigConfiguration Framework"]
ARCH_VCPU["Architecture-SpecificvCPU Backends"]
PAGETBL["page_table_multiarchMemory Translation"]
MMIO["MMIO DeviceImplementations"]
TOML["TOML ConfigurationParsing"]

AXADDR --> PAGETBL
AXCONF --> TOML
AXDEV --> MMIO
AXVCPU --> ARCH_VCPU
AXVM --> AXADDR
AXVM --> AXCONF
AXVM --> AXDEV
AXVM --> AXVCPU

Sources: Cargo.toml(L24 - L27) 

Each ArceOS-Hypervisor module serves a specific role in the virtualization stack:

Sources: Cargo.toml(L24 - L27) 

Architecture-Specific Dependencies

AxVM supports multiple hardware architectures through conditional compilation and architecture-specific vCPU backends. This design allows the same codebase to provide native virtualization support across different platforms.

Multi-Architecture Dependency Resolution

flowchart TD
subgraph subGraph1["Architecture Backends"]
    X86_VCPU["x86_vcpuIntel VT-x/AMD-V"]
    RISCV_VCPU["riscv_vcpuRISC-V H-Extension"]
    ARM_VCPU["arm_vcpuARM EL2"]
end
subgraph subGraph0["Target Architecture Selection"]
    X86["target_arch = x86_64"]
    RISCV["target_arch = riscv64"]
    ARM["target_arch = aarch64"]
end
CONFIG["cfg-ifConditional Compilation"]
VTX["VT-x/SVMHardware Features"]
HEXT["HypervisorExtension"]
EL2["Exception Level 2Virtualization"]

ARM --> ARM_VCPU
ARM_VCPU --> EL2
CONFIG --> ARM
CONFIG --> RISCV
CONFIG --> X86
RISCV --> RISCV_VCPU
RISCV_VCPU --> HEXT
X86 --> X86_VCPU
X86_VCPU --> VTX

Sources: Cargo.toml(L29 - L36)  Cargo.toml(L12) 

The architecture-specific dependencies are resolved at compile time:

Sources: Cargo.toml(L29 - L36) 

System Infrastructure Dependencies

AxVM relies on several ArceOS system-independent crates that provide low-level infrastructure for hypervisor operations.

Memory Management Infrastructure

flowchart TD
MEMORY_MGMT["Memory ManagementInfrastructure"]
MEMORY_ADDR["memory_addrAddress Abstractions"]
PAGE_ENTRY["page_table_entryPage Table Entries"]
PAGE_MULTI["page_table_multiarchMulti-arch Page Tables"]
PERCPU["percpuPer-CPU Data Storage"]
PHYS_ADDR["PhysAddrGuestPhysAddr"]
VIRT_ADDR["VirtAddrGuestVirtAddr"]
ARM_EL2["ARM EL2Page Entries"]
EPT["Extended Page Tables"]
NPT["Nested Page Tables"]
CPU_STATE["Per-CPU VM State"]
VCPU_DATA["vCPU Local Data"]

MEMORY_ADDR --> PHYS_ADDR
MEMORY_ADDR --> VIRT_ADDR
MEMORY_MGMT --> MEMORY_ADDR
MEMORY_MGMT --> PAGE_ENTRY
MEMORY_MGMT --> PAGE_MULTI
MEMORY_MGMT --> PERCPU
PAGE_ENTRY --> ARM_EL2
PAGE_MULTI --> EPT
PAGE_MULTI --> NPT
PERCPU --> CPU_STATE
PERCPU --> VCPU_DATA

Sources: Cargo.toml(L18 - L21) 

Memory Management Crate Details

Sources: Cargo.toml(L18 - L21) 

Utility and System Dependencies

AxVM uses several external utility crates for core system functionality including error handling, logging, and synchronization.

External Utility Integration

flowchart TD
UTILITIES["System Utilities"]
LOG["log v0.4Logging Framework"]
SPIN["spin v0.9Synchronization"]
CFGIF["cfg-if v1.0Conditional Compilation"]
AXERRNO["axerrno v0.1.0Error Handling"]
LOG_IMPL["VM Event LoggingDebug Information"]
LOCKS["SpinlocksMutex Primitives"]
ARCH_SELECT["Architecture SelectionFeature Gates"]
ERROR_CODES["Hypervisor Error CodesResult Types"]

AXERRNO --> ERROR_CODES
CFGIF --> ARCH_SELECT
LOG --> LOG_IMPL
SPIN --> LOCKS
UTILITIES --> AXERRNO
UTILITIES --> CFGIF
UTILITIES --> LOG
UTILITIES --> SPIN

Sources: Cargo.toml(L11 - L16) 

Utility Dependency Functions

Sources: Cargo.toml(L11 - L16) 

Feature Configuration

AxVM uses Cargo features to enable optional functionality and architecture-specific optimizations.

Default Feature Set

The hypervisor defines a minimal set of default features focused on x86_64 virtualization:

[features]
default = ["vmx"]
vmx = []

Sources: Cargo.toml(L6 - L8) 

The vmx feature enables Intel VT-x and AMD-V support on x86_64 platforms, representing the most common virtualization use case.

Integration Patterns

AxVM follows specific integration patterns that enable clean separation of concerns while maintaining performance and flexibility.

Repository Integration Model

flowchart TD
GITHUB["GitHub Organizationarceos-hypervisor"]
AXVM_REPO["axvmCore Hypervisor"]
AXVCPU_REPO["axvcpuvCPU Abstraction"]
AXADDR_REPO["axaddrspaceMemory Management"]
AXDEV_REPO["axdeviceDevice Emulation"]
AXCONF_REPO["axvmconfigConfiguration"]
X86_REPO["x86_vcpux86 Backend"]
RISCV_REPO["riscv_vcpuRISC-V Backend"]
ARM_REPO["arm_vcpuARM Backend"]

AXVCPU_REPO --> ARM_REPO
AXVCPU_REPO --> RISCV_REPO
AXVCPU_REPO --> X86_REPO
AXVM_REPO --> AXADDR_REPO
AXVM_REPO --> AXCONF_REPO
AXVM_REPO --> AXDEV_REPO
AXVM_REPO --> AXVCPU_REPO
GITHUB --> ARM_REPO
GITHUB --> AXADDR_REPO
GITHUB --> AXCONF_REPO
GITHUB --> AXDEV_REPO
GITHUB --> AXVCPU_REPO
GITHUB --> AXVM_REPO
GITHUB --> RISCV_REPO
GITHUB --> X86_REPO

Sources: Cargo.toml(L24 - L36) 

All ArceOS-Hypervisor components are developed in separate repositories under the arceos-hypervisor GitHub organization, enabling independent development cycles while maintaining API compatibility through semantic versioning.

Development Guide

Relevant source files

• .github/workflows/ci.yml
• LICENSE.Apache2

This document provides an overview of development practices and guidelines for contributors to the AxVM hypervisor project. It covers the build system, testing procedures, contribution workflow, and project governance aspects needed for effective development and collaboration.

For detailed information about specific development processes, see Build and Testing. For legal requirements and contribution guidelines, see License and Legal.

Development Workflow Overview

The AxVM project follows a standard GitHub-based development workflow with continuous integration and automated testing across multiple architectures. The development process emphasizes code quality, documentation completeness, and cross-platform compatibility.

Development Process Flow

flowchart TD
Developer["Developer"]
Fork["Fork Repository"]
Clone["Clone Fork"]
Branch["Create Feature Branch"]
Code["Write Code"]
LocalTest["Local Testing"]
Commit["Commit Changes"]
Push["Push to Fork"]
PR["Create Pull Request"]
CITrigger["CI Pipeline Triggered"]
FormatCheck["Format Check (cargo fmt)"]
ClippyCheck["Clippy Linting"]
BuildMatrix["Multi-Architecture Build"]
UnitTests["Unit Tests (x86_64 only)"]
DocBuild["Documentation Build"]
CIResult["CI Results"]
CodeReview["Code Review"]
FixIssues["Fix Issues"]
Approved["Approved?"]
Merge["Merge to Main"]
AddressComments["Address Comments"]
DocDeploy["Deploy Documentation"]

AddressComments --> Code
Approved --> AddressComments
Approved --> Merge
Branch --> Code
BuildMatrix --> CIResult
CIResult --> CodeReview
CIResult --> FixIssues
CITrigger --> BuildMatrix
CITrigger --> ClippyCheck
CITrigger --> DocBuild
CITrigger --> FormatCheck
CITrigger --> UnitTests
ClippyCheck --> CIResult
Clone --> Branch
Code --> LocalTest
CodeReview --> Approved
Commit --> Push
Developer --> Fork
DocBuild --> CIResult
FixIssues --> Code
Fork --> Clone
FormatCheck --> CIResult
LocalTest --> Commit
Merge --> DocDeploy
PR --> CITrigger
Push --> PR
UnitTests --> CIResult

Sources: .github/workflows/ci.yml(L1 - L61) 

Build System and Architecture Support

The project uses a comprehensive build matrix to ensure compatibility across multiple Rust toolchains and target architectures. This approach guarantees that AxVM functions correctly on all supported platforms.

Supported Build Targets

Build Matrix Configuration

flowchart TD
CI["CI Pipeline"]
Matrix["Build Matrix"]
ToolchainA["nightly-2024-12-25"]
ToolchainB["nightly"]
TargetA1["x86_64-unknown-linux-gnu"]
TargetA2["x86_64-unknown-none"]
TargetA3["riscv64gc-unknown-none-elf"]
TargetA4["aarch64-unknown-none-softfloat"]
TargetB1["x86_64-unknown-linux-gnu"]
TargetB2["x86_64-unknown-none"]
TargetB3["riscv64gc-unknown-none-elf"]
TargetB4["aarch64-unknown-none-softfloat"]
BuildStepsA1["Format → Clippy → Build → Test"]
BuildStepsA2["Format → Clippy → Build"]
BuildStepsA3["Format → Clippy → Build"]
BuildStepsA4["Format → Clippy → Build"]
BuildStepsB1["Format → Clippy → Build → Test"]
BuildStepsB2["Format → Clippy → Build"]
BuildStepsB3["Format → Clippy → Build"]
BuildStepsB4["Format → Clippy → Build"]

CI --> Matrix
Matrix --> ToolchainA
Matrix --> ToolchainB
TargetA1 --> BuildStepsA1
TargetA2 --> BuildStepsA2
TargetA3 --> BuildStepsA3
TargetA4 --> BuildStepsA4
TargetB1 --> BuildStepsB1
TargetB2 --> BuildStepsB2
TargetB3 --> BuildStepsB3
TargetB4 --> BuildStepsB4
ToolchainA --> TargetA1
ToolchainA --> TargetA2
ToolchainA --> TargetA3
ToolchainA --> TargetA4
ToolchainB --> TargetB1
ToolchainB --> TargetB2
ToolchainB --> TargetB3
ToolchainB --> TargetB4

Sources: .github/workflows/ci.yml(L8 - L33) 

Continuous Integration Pipeline

The CI system performs comprehensive validation of all changes through automated checks and builds. The pipeline includes code quality enforcement, multi-architecture compilation, and automated documentation generation.

CI Job Structure

flowchart TD
subgraph subGraph2["Doc Job Steps"]
    DocCheckout["Checkout Code"]
    DocRustSetup["Setup Rust Toolchain"]
    DocBuild["cargo doc --no-deps"]
    IndexGen["Generate Index HTML"]
    PagesDeploy["Deploy to GitHub Pages"]
end
subgraph subGraph1["CI Job Steps"]
    Checkout["Checkout Code"]
    RustSetup["Setup Rust Toolchain"]
    VersionCheck["Check Rust Version"]
    FormatCheck["cargo fmt --check"]
    ClippyRun["cargo clippy"]
    BuildRun["cargo build"]
    TestRun["cargo test (x86_64 only)"]
end
subgraph subGraph0["CI Jobs"]
    CIJob["ci job"]
    DocJob["doc job"]
end

BuildRun --> TestRun
CIJob --> Checkout
Checkout --> RustSetup
ClippyRun --> BuildRun
DocBuild --> IndexGen
DocCheckout --> DocRustSetup
DocJob --> DocCheckout
DocRustSetup --> DocBuild
FormatCheck --> ClippyRun
IndexGen --> PagesDeploy
RustSetup --> VersionCheck
VersionCheck --> FormatCheck

Sources: .github/workflows/ci.yml(L6 - L34)  .github/workflows/ci.yml(L35 - L61) 

Code Quality Standards

The project enforces strict code quality standards through automated tooling and manual review processes.

Automated Quality Checks

• Code Formatting: Enforced via cargo fmt --all -- --check .github/workflows/ci.yml(L23) 
• Linting: Performed using cargo clippy with custom configuration .github/workflows/ci.yml(L24 - L26) 
• Documentation: Required for all public APIs with broken link detection .github/workflows/ci.yml(L43) 

Quality Enforcement Configuration

Sources: .github/workflows/ci.yml(L26)  .github/workflows/ci.yml(L43) 

Documentation System

AxVM maintains comprehensive documentation that is automatically built and deployed through the CI system. The documentation is generated using Rust's built-in documentation tools and hosted on GitHub Pages.

Documentation Build Process

sequenceDiagram
    participant Developer as "Developer"
    participant PullRequest as "Pull Request"
    participant CISystem as "CI System"
    participant rustdoc as "rustdoc"
    participant GitHubPages as "GitHub Pages"

    Developer ->> PullRequest: Submit Changes
    PullRequest ->> CISystem: Trigger Doc Job
    CISystem ->> rustdoc: cargo doc --no-deps --all-features
    rustdoc ->> CISystem: Generate Documentation
    CISystem ->> CISystem: Create Index Redirect
    Note over CISystem: Generate redirect to main crate
    CISystem ->> GitHubPages: Deploy Documentation
    Note over GitHubPages: Only on main branch

The documentation system generates API documentation for all public interfaces and automatically redirects to the main crate documentation. Documentation deployment only occurs for changes merged to the default branch.

Sources: .github/workflows/ci.yml(L49 - L60) 

Testing Strategy

The project implements a multi-layered testing approach with unit tests executed in the CI pipeline. Testing is currently limited to the x86_64-unknown-linux-gnu target due to the nature of bare-metal hypervisor code.

Test Execution Framework

• Unit Tests: Run via cargo test --target x86_64-unknown-linux-gnu -- --nocapture .github/workflows/ci.yml(L31 - L33) 
• Build Verification: All target architectures are compiled to ensure compatibility .github/workflows/ci.yml(L28 - L29) 
• Integration Testing: Performed through successful compilation across architectures

Sources: .github/workflows/ci.yml(L30 - L33) 

Contributing Guidelines

Contributions to AxVM are governed by the Apache 2.0 license terms. All contributors must ensure their submissions comply with the project's quality standards and legal requirements.

Contribution Requirements

1. Code Quality: All submissions must pass automated format, lint, and build checks
2. Documentation: Public APIs must include comprehensive documentation
3. Testing: Changes should include appropriate test coverage where applicable
4. Legal Compliance: Contributions must be compatible with Apache 2.0 licensing

For detailed legal information and license terms, see License and Legal.

Sources: LICENSE.Apache2(L1 - L202) 

Build and Testing

Relevant source files

• .github/workflows/ci.yml

This document covers the automated build system, continuous integration pipeline, testing procedures, and documentation generation for the AxVM hypervisor project. It explains the multi-architecture build matrix, CI/CD workflow stages, and quality assurance processes that ensure code reliability across supported platforms.

For information about development dependencies and integration with the ArceOS ecosystem, see Dependencies and Integration.

Build System Overview

The AxVM project uses a GitHub Actions-based continuous integration system that validates code quality, builds across multiple architectures, and generates documentation. The build system supports three primary target architectures (x86_64, RISC-V, and AArch64) with different execution environments ranging from hosted Linux to bare-metal embedded targets.

CI Pipeline Architecture

flowchart TD
subgraph subGraph3["Documentation Pipeline"]
    doc_build["cargo doc --no-deps"]
    index_redirect["index.html redirect"]
    gh_pages_deploy["GitHub Pages deploy"]
end
subgraph subGraph2["Build Stages"]
    format_check["cargo fmt --check"]
    clippy_lint["cargo clippy"]
    cargo_build["cargo build"]
    unit_tests["cargo test"]
end
subgraph subGraph1["CI Job Matrix"]
    matrix_strategy["strategy.matrix"]
    toolchain_variants["rust-toolchain variants"]
    target_variants["targets variants"]
    nightly_2024["nightly-2024-12-25"]
    nightly_latest["nightly"]
    x86_linux["x86_64-unknown-linux-gnu"]
    x86_none["x86_64-unknown-none"]
    riscv_none["riscv64gc-unknown-none-elf"]
    aarch64_none["aarch64-unknown-none-softfloat"]
end
subgraph subGraph0["GitHub Actions Workflow"]
    trigger["push/pull_request triggers"]
    ci_job["ci job"]
    doc_job["doc job"]
end

cargo_build --> unit_tests
ci_job --> format_check
ci_job --> matrix_strategy
clippy_lint --> cargo_build
doc_build --> index_redirect
doc_job --> doc_build
format_check --> clippy_lint
index_redirect --> gh_pages_deploy
matrix_strategy --> target_variants
matrix_strategy --> toolchain_variants
target_variants --> aarch64_none
target_variants --> riscv_none
target_variants --> x86_linux
target_variants --> x86_none
toolchain_variants --> nightly_2024
toolchain_variants --> nightly_latest
trigger --> ci_job
trigger --> doc_job

Sources: .github/workflows/ci.yml(L1 - L61) 

Supported Architecture Targets

The build system validates AxVM across multiple target architectures that align with the hypervisor's multi-architecture support design. Each target represents a different execution environment and use case.

Build Matrix Configuration

flowchart TD
subgraph Components["Components"]
    rust_src["rust-src"]
    clippy_comp["clippy"]
    rustfmt_comp["rustfmt"]
end
subgraph subGraph1["Target Matrix"]
    x86_linux["x86_64-unknown-linux-gnu"]
    x86_bare["x86_64-unknown-none"]
    riscv_bare["riscv64gc-unknown-none-elf"]
    arm_bare["aarch64-unknown-none-softfloat"]
end
subgraph subGraph0["Toolchain Matrix"]
    stable_toolchain["nightly-2024-12-25"]
    latest_toolchain["nightly"]
end

latest_toolchain --> arm_bare
latest_toolchain --> riscv_bare
latest_toolchain --> x86_bare
latest_toolchain --> x86_linux
stable_toolchain --> arm_bare
stable_toolchain --> clippy_comp
stable_toolchain --> riscv_bare
stable_toolchain --> rust_src
stable_toolchain --> rustfmt_comp
stable_toolchain --> x86_bare
stable_toolchain --> x86_linux

Sources: .github/workflows/ci.yml(L10 - L19) 

CI Pipeline Stages

The continuous integration pipeline consists of multiple validation stages that run sequentially for each target in the build matrix. The pipeline uses a fail-fast: false strategy to ensure all combinations are tested even if some fail.

Code Quality Validation

The first stage validates code formatting and style consistency:

sequenceDiagram
    participant GitHubActions as "GitHub Actions"
    participant RustToolchain as "Rust Toolchain"
    participant cargofmt as "cargo fmt"
    participant cargoclippy as "cargo clippy"

    GitHubActions ->> RustToolchain: "Setup nightly toolchain"
    GitHubActions ->> RustToolchain: "Install components: rust-src, clippy, rustfmt"
    GitHubActions ->> RustToolchain: "Add target: ${{ matrix.targets }}"
    GitHubActions ->> cargofmt: "cargo fmt --all -- --check"
    cargofmt ->> GitHubActions: "Format validation result"
    GitHubActions ->> cargoclippy: "cargo clippy --target ${{ matrix.targets }} --all-features"
    Note over cargoclippy: "Allows clippy::new_without_default"
    cargoclippy ->> GitHubActions: "Lint validation result"

The clippy stage includes specific configuration to suppress the new_without_default warning, allowing constructors without Default implementation.

Sources: .github/workflows/ci.yml(L22 - L26) 

Build Validation

Each target architecture undergoes compilation validation with full feature enablement:

cargo build --target ${{ matrix.targets }} --all-features

The build stage uses continue-on-error for the latest nightly toolchain to prevent pipeline failures from upstream Rust changes while maintaining stability with the pinned nightly version.

Sources: .github/workflows/ci.yml(L27 - L29) 

Unit Testing

Unit tests execute only on the x86_64-unknown-linux-gnu target, which provides the hosted environment necessary for test execution:

cargo test --target x86_64-unknown-linux-gnu -- --nocapture

This restriction reflects the practical limitation that bare-metal targets cannot execute standard Rust test harnesses without additional infrastructure.

Sources: .github/workflows/ci.yml(L30 - L33) 

Documentation Generation

The documentation pipeline operates independently from the main CI job and includes automated deployment to GitHub Pages for the default branch.

Documentation Build Process

flowchart TD
subgraph subGraph2["Deployment Conditions"]
    default_branch_check["github.ref == default-branch"]
    single_commit["single-commit: true"]
    target_folder["folder: target/doc"]
end
subgraph subGraph1["RUSTDOCFLAGS Environment"]
    broken_links["-D rustdoc::broken_intra_doc_links"]
    missing_docs["-D missing-docs"]
end
subgraph subGraph0["Documentation Job"]
    doc_trigger["GitHub trigger"]
    doc_checkout["Checkout repository"]
    doc_toolchain["Setup nightly-2024-12-25"]
    doc_build["cargo doc --no-deps --all-features"]
    doc_index["Generate index.html redirect"]
    doc_deploy["Deploy to gh-pages branch"]
end

broken_links --> doc_build
default_branch_check --> doc_deploy
doc_build --> doc_index
doc_checkout --> doc_toolchain
doc_index --> doc_deploy
doc_toolchain --> doc_build
doc_trigger --> doc_checkout
missing_docs --> doc_build
single_commit --> doc_deploy
target_folder --> doc_deploy

The documentation build enforces strict documentation standards through RUSTDOCFLAGS that treat broken intra-doc links and missing documentation as compilation errors.

Sources: .github/workflows/ci.yml(L35 - L61) 

Index Page Generation

The pipeline automatically generates a redirect index page that points to the main crate documentation:

printf '<meta http-equiv="refresh" content="0;url=%s/index.html">' $(cargo tree | head -1 | cut -d' ' -f1) > target/doc/index.html

This command extracts the crate name from cargo tree output and creates an HTML redirect to the appropriate documentation entry point.

Sources: .github/workflows/ci.yml(L52 - L53) 

Error Handling and Resilience

The CI configuration includes several resilience mechanisms to handle different failure scenarios:

The pipeline distinguishes between the stable nightly toolchain (nightly-2024-12-25) and the latest nightly, allowing experimentation with newer compiler versions while maintaining build stability.

Sources: .github/workflows/ci.yml(L8 - L9)  .github/workflows/ci.yml(L25)  .github/workflows/ci.yml(L28)  .github/workflows/ci.yml(L32)  .github/workflows/ci.yml(L50) 

Quality Assurance Standards

The CI pipeline enforces several quality standards across the codebase:

• Code Formatting: Uniform formatting via rustfmt with --check flag
• Linting: Comprehensive linting through clippy with minimal suppressions
• Documentation: Complete documentation coverage with broken link detection
• Multi-architecture Compatibility: Build validation across all supported targets
• Feature Completeness: All builds use --all-features to ensure complete validation

These standards ensure that AxVM maintains high code quality and reliability across its supported platforms and use cases.

Sources: .github/workflows/ci.yml(L22 - L33)  .github/workflows/ci.yml(L43)  .github/workflows/ci.yml(L49 - L53) 

License and Legal

Relevant source files

• .gitignore
• LICENSE.Apache2

This document provides comprehensive legal information for the AxVM hypervisor project, including licensing terms, contribution guidelines, and intellectual property considerations. This information is essential for developers, contributors, and users who need to understand the legal framework governing the use, modification, and distribution of the AxVM codebase.

For build and testing procedures, see Build and Testing. For general development information, see Development Guide.

License Overview

The AxVM hypervisor is licensed under the Apache License 2.0, a permissive open-source license that allows for both commercial and non-commercial use while providing patent protection and clear contribution terms.

License Structure and Components

flowchart TD
subgraph subGraph3["Protection Mechanisms"]
    Warranty["Warranty DisclaimerSection 7"]
    Liability["Liability LimitationSection 8"]
    Trademark["Trademark ProtectionSection 6"]
    PatentTerm["Patent TerminationSection 3"]
end
subgraph subGraph2["Key Obligations"]
    Attribution["Attribution RequirementsSection 4(c)"]
    Notice["License Notice InclusionSection 4(a)"]
    Changes["Change DocumentationSection 4(b)"]
    NoticeTxt["NOTICE File HandlingSection 4(d)"]
end
subgraph subGraph1["Key Rights Granted"]
    Copyright["Copyright LicenseSection 2"]
    Patent["Patent LicenseSection 3"]
    Redistribute["Redistribution RightsSection 4"]
    Modify["Modification RightsSection 4"]
end
subgraph subGraph0["Apache License 2.0 Framework"]
    License["LICENSE.Apache2"]
    Terms["License Terms"]
    Grants["Rights Granted"]
    Conditions["Conditions & Obligations"]
    Limitations["Limitations & Disclaimers"]
end

Conditions --> Attribution
Conditions --> Changes
Conditions --> Notice
Conditions --> NoticeTxt
Grants --> Copyright
Grants --> Modify
Grants --> Patent
Grants --> Redistribute
License --> Terms
Limitations --> Liability
Limitations --> PatentTerm
Limitations --> Trademark
Limitations --> Warranty
Terms --> Conditions
Terms --> Grants
Terms --> Limitations

Sources: LICENSE.Apache2(L1 - L202) 

License Terms and Conditions

Copyright and Patent Grants

The Apache License 2.0 provides comprehensive rights to users of the AxVM hypervisor:

Key Definitions

The license establishes several critical definitions that govern how the AxVM codebase can be used:

flowchart TD
subgraph subGraph2["Legal Relationships"]
    OwnershipChain["Copyright Ownership"]
    SubmissionFlow["Contribution Submission"]
    LicenseGrant["License Grant Flow"]
end
subgraph subGraph1["Work Classifications"]
    Source["Source Form(.rs files, configs)"]
    Object["Object Form(compiled binaries)"]
    Derivative["Derivative Works(based on AxVM)"]
    Contribution["Contribution(submitted changes)"]
end
subgraph subGraph0["Core Legal Entities"]
    Work["Work(AxVM Source Code)"]
    Contributor["Contributor(Code Authors)"]
    Licensee["You(License Recipients)"]
    Licensor["Licensor(Copyright Holders)"]
end

Contribution --> Work
Contributor --> Contribution
Contributor --> OwnershipChain
LicenseGrant --> Licensee
Licensor --> LicenseGrant
OwnershipChain --> Licensor
Work --> Derivative
Work --> Object
Work --> Source

Sources: LICENSE.Apache2(L8 - L64) 

Redistribution Requirements

When redistributing AxVM or derivative works, the following obligations must be met:

Mandatory Inclusions

1. License Copy: Include a copy of the Apache License 2.0 with any distribution
2. Attribution Notices: Retain all copyright, patent, trademark, and attribution notices
3. Change Documentation: Mark any modified files with prominent notices
4. NOTICE File: If present, include readable copy of attribution notices

Source Code Distribution

flowchart TD
subgraph subGraph0["Required Components"]
    LicenseCopy["LICENSE.Apache2Full license text"]
    CopyrightNotices["Copyright NoticesIn source headers"]
    ChangeMarkers["Change MarkersModified file notices"]
    NoticeFile["NOTICE FileAttribution notices"]
end
OrigSource["Original AxVM Source"]
ModSource["Modified Source"]
Distribution["Source Distribution"]

Distribution --> CopyrightNotices
Distribution --> LicenseCopy
Distribution --> NoticeFile
ModSource --> ChangeMarkers
ModSource --> Distribution
OrigSource --> Distribution

Sources: LICENSE.Apache2(L89 - L121) 

Contribution Guidelines

Contribution License Terms

All contributions to AxVM are subject to the Apache License 2.0 terms:

Submission Process

sequenceDiagram
    participant Developer as Developer
    participant AxVMRepository as "AxVM Repository"
    participant Maintainer as Maintainer
    participant Apache20Framework as "Apache 2.0 Framework"

    Developer ->> Developer: "Create contribution"
    Developer ->> Apache20Framework: "Implicit agreement to license terms"
    Developer ->> AxVMRepository: "Submit pull request/patch"
    Note over AxVMRepository: "Contribution becomes subject to<br>Apache 2.0 Section 5"
    AxVMRepository ->> Maintainer: "Review contribution"
    Maintainer ->> AxVMRepository: "Accept/merge contribution"
    AxVMRepository ->> Apache20Framework: "Contribution incorporated under Apache 2.0"
    Note over Apache20Framework: "Patent license automatically granted<br>Copyright license automatically granted"

Sources: LICENSE.Apache2(L48 - L60)  LICENSE.Apache2(L130 - L136) 

Intellectual Property Considerations

Patent Protection

The Apache License 2.0 provides specific patent protections:

1. Automatic Patent License: Contributors automatically grant patent licenses for their contributions
2. Patent Termination Clause: Patent licenses terminate if litigation is filed against the project
3. Defensive Termination: Protects against patent trolling and frivolous litigation

Trademark Restrictions

The license explicitly does not grant trademark rights:

flowchart TD
subgraph subGraph1["Permitted Uses"]
    ReasonableUse["Reasonable and Customary UseOrigin description"]
    NoticeRepro["NOTICE File Reproduction"]
end
subgraph subGraph0["Trademark Scope"]
    ProjectName["AxVM Project Name"]
    ArceOSBrand["ArceOS Branding"]
    LogosMarks["Logos and Marks"]
    ServiceMarks["Service Marks"]
end
subgraph subGraph2["Prohibited Uses"]
    CommercialBrand["Commercial Branding"]
    Endorsement["Implied Endorsement"]
    Confusion["Likelihood of Confusion"]
end
Prohibited["Prohibited"]

ArceOSBrand --> NoticeRepro
LogosMarks --> Prohibited
ProjectName --> ReasonableUse
ServiceMarks --> Prohibited

Sources: LICENSE.Apache2(L138 - L141) 

Compliance and Attribution

License Compliance Checklist

For users and distributors of AxVM:

Attribution Requirements

flowchart TD
subgraph subGraph1["Distribution Attribution"]
    LicenseFile["LICENSE.Apache2Full license text"]
    NoticeInclude["NOTICE FileThird-party attributions"]
end
subgraph subGraph0["Source Attribution"]
    SourceHeaders["Source File HeadersCopyright notices"]
    LicenseRef["License ReferencesSPDX identifiers"]
end
subgraph subGraph2["Modification Attribution"]
    ChangeLog["Change DocumentationModified file markers"]
    AuthorInfo["Author InformationContributor attribution"]
end

ChangeLog --> AuthorInfo
LicenseRef --> NoticeInclude
SourceHeaders --> LicenseFile

Sources: LICENSE.Apache2(L94 - L120) 

Disclaimer and Limitation of Liability

Warranty Disclaimer

The AxVM hypervisor is provided "AS IS" without warranties of any kind. This includes:

• No warranty of merchantability or fitness for a particular purpose
• No warranty of title or non-infringement
• No express or implied warranties beyond those required by applicable law

Liability Limitations

Contributors and distributors have limited liability exposure:

Sources: LICENSE.Apache2(L143 - L174) 

This legal framework ensures that AxVM can be freely used, modified, and distributed while providing appropriate protections for contributors and clear obligations for users.

Overview

Relevant source files

• Cargo.lock
• LICENSE.Apache2
• LICENSE.GPLv3
• LICENSE.MulanPSL2
• LICENSE.MulanPubL2
• README.md
• README_CN.md

This document provides a comprehensive introduction to AxVisor, a unified modular hypervisor built on the ArceOS unikernel framework. It covers the core concepts, architecture, and key components of AxVisor. For more detailed information about specific subsystems, refer to the respective wiki pages linked throughout this document.

What is AxVisor?

AxVisor is a hypervisor implemented based on the ArceOS unikernel framework. Its goal is to leverage the foundational operating system features provided by ArceOS to implement a unified modular hypervisor that supports multiple hardware architectures with a single codebase.

Key attributes that define AxVisor:

• Unified: AxVisor uses the same codebase to simultaneously support x86_64, ARM (aarch64), and RISC-V architectures. This maximizes the reuse of architecture-independent code and simplifies development and maintenance.
• Modular: The hypervisor's functionality is decomposed into multiple modules, each implementing a specific function. These modules communicate with each other through standard interfaces to achieve decoupling and reuse of functionality.

Sources: README.md(L21 - L28) 

Architecture Overview

AxVisor follows a layered architecture approach with clearly defined components that have specific responsibilities:

flowchart TD
subgraph subGraph3["ArceOS Foundation"]
    axtask["axtask - Task Management"]
    axfs["axfs - Virtual File System"]
    axalloc["axalloc - Memory Allocator"]
    axconfig["axconfig - Configuration Management"]
end
subgraph subGraph2["Hardware Abstraction"]
    axhal["axhal - Hardware Abstraction Layer"]
end
subgraph subGraph1["Virtual Machine Management"]
    axvm["axvm - VM Management"]
    axvcpu["axvcpu - Virtual CPU Management"]
    axaddrspace["axaddrspace - Address Space Management"]
    axdevice["axdevice - Device Emulation"]
end
subgraph subGraph0["Top Level"]
    axvisor["axvisor - Top-level Hypervisor"]
end

axhal --> axalloc
axhal --> axconfig
axhal --> axfs
axhal --> axtask
axvcpu --> axaddrspace
axvisor --> axaddrspace
axvisor --> axconfig
axvisor --> axhal
axvisor --> axvcpu
axvisor --> axvm
axvm --> axaddrspace
axvm --> axdevice
axvm --> axvcpu

Sources: README.md(L29 - L35) 

Key Components

1. axvisor: The top-level hypervisor component that coordinates all subsystems.
2. axvm: Manages virtual machines, their lifecycle, and interactions with the host system.
3. axvcpu: Handles virtual CPU operations, including context switching, instruction emulation, and exit handling.
4. axaddrspace: Manages guest memory address spaces, including page tables and memory region mapping.
5. axhal: Hardware Abstraction Layer that provides unified interfaces to hardware across different architectures.
6. axdevice: Emulates hardware devices for guests or manages device passthrough.
7. axconfig: Handles configuration parsing and management for the hypervisor and virtual machines.

Sources: README.md(L29 - L35) 

Virtual Machine Execution Flow

The following diagram illustrates the typical flow of VM initialization, boot, and execution in AxVisor:

sequenceDiagram
    participant VMConfigTOML as "VMConfig (TOML)"
    participant VMMinitrun as "VMM::init/run"
    participant VirtualMachine as "Virtual Machine"
    participant VirtualCPU as "Virtual CPU"
    participant GuestOS as "Guest OS"

    VMConfigTOML ->> VMMinitrun: Load VM Configuration
    VMMinitrun ->> VirtualMachine: Initialize VM (id, name, vcpus)
    VMMinitrun ->> VirtualMachine: Setup memory regions
    VMMinitrun ->> VirtualMachine: Load guest image
    alt Load from
    alt filesystem
        VirtualMachine ->> VirtualMachine: Load from FAT32
    else Load from
    else Load from
        VirtualMachine ->> VirtualMachine: Load compiled image
    end
    end
    VMMinitrun ->> VirtualCPU: Initialize VCPUs
    VMMinitrun ->> VirtualCPU: Create VCPU tasks
    VMMinitrun ->> VirtualMachine: Boot VM
    VirtualCPU ->> VirtualCPU: Run VCPU execution loop
    VirtualCPU ->> GuestOS: Execute guest code
    loop VCPU execution
        GuestOS ->> VirtualCPU: VM Exit (hypercall/interrupt/etc.)
        VirtualCPU ->> VirtualCPU: Handle exit reason
        VirtualCPU ->> GuestOS: Resume execution
    end
    GuestOS ->> VirtualCPU: Shutdown request
    VirtualCPU ->> VirtualMachine: Signal VM shutdown
    VirtualMachine ->> VMMinitrun: Decrement running VM count

Sources: README.md(L57 - L59)  README.md(L116 - L202) 

Cross-Platform Support

AxVisor achieves its cross-platform capability through architecture-specific implementations behind common interfaces:

flowchart TD
subgraph subGraph1["Architecture-Specific Implementations"]
    x86_vcpu["x86_vcpu - x86 Implementation"]
    arm_vcpu["arm_vcpu - ARM Implementation"]
    riscv_vcpu["riscv_vcpu - RISC-V Implementation"]
    x86_hal["x86_64 HAL Implementation"]
    arm_hal["ARM/aarch64 HAL Implementation"]
    riscv_hal["RISC-V HAL Implementation"]
end
subgraph subGraph0["AxVisor Common Interface"]
    axvcpu_common["axvcpu - Common Interface"]
    axhal_common["axhal - Common Interface"]
end

axhal_common --> arm_hal
axhal_common --> riscv_hal
axhal_common --> x86_hal
axvcpu_common --> arm_vcpu
axvcpu_common --> riscv_vcpu
axvcpu_common --> x86_vcpu

Sources: README.md(L38 - L47) 

Configuration System

AxVisor uses TOML configuration files to manage virtual machine settings. The configuration system handles:

• VM identification (ID, name)
• Hardware allocation (CPU cores, memory size)
• Guest image loading method and parameters
• Virtual and passthrough devices
• Architecture-specific settings

Configuration and Image Loading

flowchart TD
subgraph subGraph1["Image Loading Methods"]
    fs_load["Load from Filesystem"]
    mem_load["Load from Memory"]
end
subgraph subGraph0["Configuration Flow"]
    toml["TOML Config Files"]
    vmconfig["VM Configuration Object"]
    memory["Memory Configuration"]
    devices["Device Configuration"]
    image["Image Configuration"]
end
fs_image["FAT32 Filesystem Image"]
compiled_image["Statically Compiled Image"]

fs_load --> fs_image
image --> fs_load
image --> mem_load
mem_load --> compiled_image
toml --> vmconfig
vmconfig --> devices
vmconfig --> image
vmconfig --> memory

Sources: README.md(L61 - L75)  README.md(L76 - L113) 

Supported Guest Operating Systems

AxVisor supports multiple guest operating systems:

Sources: README.md(L46 - L56) 

Hardware Platform Support

AxVisor has been verified on the following platforms:

Sources: README.md(L38 - L43) 

Code Organization

AxVisor's codebase is organized around a modular architecture with the following key crates:

• axvisor: Top-level hypervisor implementation
• axvm: Virtual machine management
• axvcpu: Common virtual CPU interface
• arm_vcpu/x86_vcpu/riscv_vcpu: Architecture-specific VCPU implementations
• axaddrspace: Address space and memory virtualization
• axdevice: Device virtualization and management
• axvmconfig: Configuration management

For more detailed information about the system components, refer to System Components.

For information about VM management, see VM Management.

Sources: Cargo.lock(L118 - L121)  Cargo.lock(L574 - L593)  Cargo.lock(L536 - L545)  Cargo.lock(L154 - L168)  Cargo.lock(L1248 - L1269)  Cargo.lock(L177 - L192)  Cargo.lock(L239 - L261)  Cargo.lock(L596 - L605) 

Development Environment

AxVisor provides tools to simplify the development environment setup:

• dev_env.py: Script to localize relevant crates for easier development and debugging
• Makefile: Handles build configurations and targets

For more information on setting up a development environment, see Development Environment.

Sources: README.md(L210 - L212) 

License

AxVisor is open-source software and uses the following licenses:

• Apache-2.0
• MulanPubL-2.0
• MulanPSL-2.0
• GPL-3.0-or-later

Sources: README.md(L222 - L229)  LICENSE.Apache2 LICENSE.MulanPSL2 LICENSE.MulanPubL2 LICENSE.GPLv3

Architecture

Relevant source files

• Cargo.lock
• Cargo.toml
• README.md

This page describes the core architectural design and principles of AxVisor, a unified modular hypervisor based on the ArceOS unikernel framework. It explains the high-level system structure, key components, and the relationships between them. For specific details on VM management, see VM Management, and for hardware abstraction details, see Hardware Abstraction Layer.

Architectural Overview

AxVisor is designed as a modular hypervisor with a layered architecture that achieves two primary goals:

1. Unification: Using a single codebase to support multiple architectures (x86_64, ARM/aarch64, and RISC-V)
2. Modularity: Decomposing hypervisor functionality into distinct modules that communicate through well-defined interfaces

The hypervisor is built on top of the ArceOS unikernel framework, which provides essential OS functionality like task scheduling, memory management, and interrupt handling.

flowchart TD
subgraph Implementation["Implementation"]
    A1["ArceOS Components (axruntime, axalloc, axtask, etc.)"]
    A2["Hardware Abstraction (axhal, architecture-specific modules)"]
    A3["VM Management (axvm, axvcpu)"]
    A4["Device Support (axdevice)"]
    A5["Configuration (axvmconfig)"]
end
subgraph subGraph0["Architecture Layers"]
    L1["Host OS Layer"]
    L2["Hardware Abstraction Layer"]
    L3["Virtual Machine Management Layer"]
    L4["Device Virtualization Layer"]
    L5["Configuration Layer"]
end

A2 --> A1
A3 --> A2
A4 --> A3
A5 --> A4
L1 --> A1
L2 --> A2
L2 --> L1
L3 --> A3
L3 --> L2
L4 --> A4
L4 --> L3
L5 --> A5
L5 --> L4

Sources: README.md(L29 - L36) 

Core Modules and Their Relationships

AxVisor is composed of several key modules, each with specific responsibilities:

flowchart TD
subgraph subGraph0["System Modules"]
    axvisor["axvisor - Top-level Hypervisor"]
    axvm["axvm - Virtual Machine Management"]
    axhal["axhal - Hardware Abstraction Layer"]
    axvcpu["axvcpu - Virtual CPU Management"]
    axaddrspace["axaddrspace - Address Space Management"]
    axconfig["axconfig - Configuration Management"]
    axstd["axstd - Standard Library"]
    axdevice["axdevice - Device Emulation"]
    axalloc["axalloc - Memory Allocator"]
end

axhal --> axalloc
axhal --> axconfig
axvisor --> axaddrspace
axvisor --> axconfig
axvisor --> axhal
axvisor --> axstd
axvisor --> axvcpu
axvisor --> axvm
axvm --> axaddrspace
axvm --> axdevice
axvm --> axvcpu

The key relationships between these modules:

Sources: Cargo.toml(L15 - L46) 

Cross-Architecture Implementation

One of AxVisor's key features is its unified architecture approach. The system uses a common interface for architecture-independent code, with architecture-specific implementations provided through separate modules.

flowchart TD
subgraph subGraph1["Hardware Features"]
    x86_feat["x86 Features:- VT-x- x2APIC"]
    arm_feat["ARM Features:- EL2 Mode- GICv2"]
    riscv_feat["RISC-V Features:- SBI Runtime"]
end
subgraph subGraph0["Architecture Support"]
    common["Common Interface Layer (axvcpu, axaddrspace)"]
    x86["x86_64 Implementation (x86_vcpu)"]
    arm["ARM Implementation (arm_vcpu)"]
    riscv["RISC-V Implementation (riscv_vcpu)"]
end

arm --> arm_feat
common --> arm
common --> riscv
common --> x86
riscv --> riscv_feat
x86 --> x86_feat

This architecture allows AxVisor to:

• Share common virtualization logic across architectures
• Isolate architecture-specific implementation details
• Add new architecture support without changing core functionality

Sources: Cargo.toml(L35 - L37)  Cargo.toml(L41 - L46) 

Virtualization Flow

The core virtualization process in AxVisor follows these key steps:

sequenceDiagram
    participant axvisorvmm as "axvisor::vmm"
    participant axvisorvmmconfig as "axvisor::vmm::config"
    participant axvmvm as "axvm::vm"
    participant axvisorvmmvcpus as "axvisor::vmm::vcpus"
    participant ArchitecturespecificVCPU as "Architecture-specific VCPU"

    axvisorvmm ->> axvisorvmmconfig: Load VM configuration
    axvisorvmmconfig ->> axvmvm: Create VM with configuration
    axvmvm ->> axvmvm: Setup memory regions
    axvisorvmmconfig ->> axvisorvmmconfig: Load guest image
    axvisorvmm ->> axvisorvmmvcpus: Initialize VCPUs
    axvisorvmmvcpus ->> axvisorvmmvcpus: Create VCPU tasks
    axvisorvmm ->> axvmvm: Boot VM
    axvmvm ->> axvisorvmmvcpus: Notify VCPUs to start
    loop VCPU Execution
        axvisorvmmvcpus ->> ArchitecturespecificVCPU: vm.run_vcpu(vcpu_id)
        ArchitecturespecificVCPU ->> ArchitecturespecificVCPU: Handle VM exits
    alt VM Exit: Hypercall
        ArchitecturespecificVCPU ->> ArchitecturespecificVCPU: Handle hypercall
    else VM Exit: External Interrupt
        ArchitecturespecificVCPU ->> ArchitecturespecificVCPU: Handle interrupt
    else VM Exit: System Shutdown
        ArchitecturespecificVCPU ->> axvmvm: Notify VM shutdown
        axvmvm ->> axvisorvmm: Decrement running VM count
    end
    end

This execution model enables AxVisor to:

1. Load guest operating systems from either filesystem or memory
2. Manage VM execution through architecture-specific VCPU implementations
3. Handle various virtualization events through the VM exit mechanism

Sources: README.md(L99 - L112)  README.md(L159 - L201) 

Memory Management Architecture

Memory virtualization in AxVisor is built around the axaddrspace module, which provides memory region management and address translation services.

flowchart TD
subgraph subGraph3["Memory Virtualization"]
    ma["memory_addr"]
    subgraph subGraph2["Address Translation"]
        addr_trans["Address Translation"]
        gva_gpa["GVA → GPA"]
        gpa_hpa["GPA → HPA"]
    end
    subgraph subGraph1["Memory Region Management"]
        as["axaddrspace"]
        mem_regions["Memory Regions"]
        guest_phys["Guest Physical Address Space"]
        host_phys["Host Physical Address Space"]
    end
    subgraph subGraph0["Page Table Management"]
        pte["page_table_entry"]
        ptm["page_table_multiarch"]
        arch_pt["Architecture-specificpage table implementations"]
    end
end

addr_trans --> gpa_hpa
addr_trans --> gva_gpa
as --> addr_trans
as --> ma
as --> mem_regions
as --> pte
as --> ptm
mem_regions --> guest_phys
mem_regions --> host_phys
pte --> ptm
ptm --> arch_pt

Key features of the memory management architecture:

• Separation of address spaces between host and guest
• Architecture-independent memory region management
• Architecture-specific page table implementations
• Two-stage address translation (Guest Virtual → Guest Physical → Host Physical)

Sources: Cargo.toml(L37)  Cargo.toml(L42 - L44) 

Configuration System

AxVisor uses a robust configuration system based on TOML files to define VM properties and behavior:

flowchart TD
subgraph subGraph1["Configuration Components"]
    vm_id["VM ID"]
    vm_name["VM Name"]
    cpu_num["CPU Count"]
    memory["Memory Configuration"]
    devices["Device Configuration"]
    images["Image Configuration"]
    img_loc["image_location(fs or memory)"]
    kernel_path["kernel_path"]
    entry_point["entry_point"]
    load_addr["kernel_load_addr"]
    regions["Memory Regions"]
    mem_attrs["Memory Attributes(READ|WRITE|EXECUTE)"]
end
subgraph subGraph0["Configuration Flow"]
    toml["VM TOML Configuration Files"]
    parser["axvmconfig Parser"]
    vmconfig["VM Configuration Object"]
    axvisor["axvisor Hypervisor"]
end

images --> entry_point
images --> img_loc
images --> kernel_path
images --> load_addr
memory --> regions
parser --> vmconfig
regions --> mem_attrs
toml --> parser
vmconfig --> axvisor
vmconfig --> cpu_num
vmconfig --> devices
vmconfig --> images
vmconfig --> memory
vmconfig --> vm_id
vmconfig --> vm_name

The configuration system enables:

• Declarative definition of VM properties
• Flexible image loading from filesystem or memory
• Consistent VM configuration across different architectures
• Separation of configuration from implementation logic

Sources: README.md(L72 - L75)  Cargo.toml(L48 - L52) 

Multi-Guest Support

AxVisor is designed to support multiple guest operating systems simultaneously:

The multi-guest architecture enables:

• Testing of different operating systems on the same hypervisor
• Comparison of performance across different guest types
• Development and debugging of guest-specific virtualization features

Sources: README.md(L45 - L56) 

Conclusion

AxVisor's architecture follows key design principles that enable it to provide a unified hypervisor solution across multiple hardware architectures:

1. Layered Design: Clear separation of concerns through a well-defined layering pattern
2. Modular Structure: Functionality divided into well-defined modules with clear interfaces
3. Architecture Independence: Separation of architecture-specific code from common functionality
4. Configuration-Driven: Extensive use of configuration files for flexible deployment
5. Flexible Guest Support: Support for multiple guest operating systems and image loading methods

This design enables AxVisor to maintain a single codebase while supporting diverse hardware platforms and guest operating systems, making it an adaptable and maintainable hypervisor solution.

Sources: README.md(L21 - L28)  README.md(L29 - L36) 

System Components

Relevant source files

• Cargo.lock
• Cargo.toml

This document details the main crates and their relationships within the AxVisor hypervisor system. It provides an overview of the modular architecture of AxVisor, explaining how various components interact to create a unified hypervisor framework that supports multiple architectures. For information about VM configuration, see VM Configuration. For implementation details of the VMM (Virtual Machine Manager), see VMM Implementation.

Component Architecture Overview

AxVisor follows a modular design approach with clearly defined components that have specific responsibilities. This architecture enables:

1. Cross-architecture support (x86_64, ARM/aarch64, RISC-V)
2. Clear separation of concerns
3. Hardware abstraction
4. Simplified development and testing

The system is organized into several categories of components that work together to provide hypervisor functionality.

flowchart TD
subgraph subGraph2["ArceOS Foundation"]
    axstd["axstd - Standard Library"]
    axhal["axhal - Hardware Abstraction Layer"]
    axtask["axtask - Task Management"]
    axalloc["axalloc - Memory Allocation"]
    axconfig["axconfig - System Configuration"]
    axfs["axfs - Virtual Filesystem"]
end
subgraph subGraph1["Architecture-Specific Components"]
    x86_vcpu["x86_vcpu - x86 VCPU Implementation"]
    arm_vcpu["arm_vcpu - ARM VCPU Implementation"]
    riscv_vcpu["riscv_vcpu - RISC-V VCPU Implementation"]
end
subgraph subGraph0["Core Hypervisor Components"]
    axvisor["axvisor - Top-level Hypervisor"]
    axvm["axvm - VM Management"]
    axvcpu["axvcpu - Virtual CPU Interface"]
    axaddrspace["axaddrspace - Memory Virtualization"]
    axdevice["axdevice - Device Emulation"]
    axvmconfig["axvmconfig - VM Configuration"]
end

axstd --> axalloc
axstd --> axconfig
axstd --> axfs
axstd --> axhal
axstd --> axtask
axvcpu --> arm_vcpu
axvcpu --> riscv_vcpu
axvcpu --> x86_vcpu
axvisor --> axaddrspace
axvisor --> axstd
axvisor --> axvcpu
axvisor --> axvm
axvm --> axaddrspace
axvm --> axdevice
axvm --> axvcpu
axvm --> axvmconfig

Sources: Cargo.toml(L14 - L45) 

Core Hypervisor Components

The main components of AxVisor can be categorized into core hypervisor functionality crates that are independent of the underlying architecture.

axvisor

The top-level hypervisor crate that ties all components together and provides the main entry point for the hypervisor. It coordinates the virtual machine lifecycle and manages the hypervisor state.

axvm

The Virtual Machine Manager responsible for creating, running, and managing virtual machines. It handles VM lifecycle operations such as creation, initialization, execution, and shutdown.

Key responsibilities:

• VM resource allocation and management
• Guest image loading
• Coordinating interactions between VCPUs, memory, and devices
• VM state management

axvcpu

Defines the common Virtual CPU interface used across all architectures. It provides an abstraction layer for architecture-specific VCPU implementations.

Key responsibilities:

• Common VCPU interface definition
• VCPU state management
• Architecture-agnostic VCPU operations

axaddrspace

Handles memory virtualization and address space management for virtual machines.

Key responsibilities:

• Memory region allocation and management
• Second-stage page table management
• Memory protection mechanisms
• Address translation services

axdevice

Provides device emulation capabilities for virtual machines, allowing guest operating systems to interact with virtualized hardware.

Key responsibilities:

• Device model implementation
• I/O handling
• Device state management

axvmconfig

Manages VM configurations through TOML-based configuration files, defining VM properties such as memory size, CPU count, and device settings.

Sources: Cargo.toml(L34 - L37) 

Architecture-Specific Components

AxVisor supports multiple architectures through specialized implementation crates that implement the common interfaces defined by the core components.

flowchart TD
subgraph subGraph3["Architecture-Specific Implementations"]
    axvcpu["axvcpu - Common VCPU Interface"]
    x86_vcpu["x86_vcpu - x86 Implementation"]
    arm_vcpu["arm_vcpu - ARM Implementation"]
    riscv_vcpu["riscv_vcpu - RISC-V Implementation"]
    subgraph subGraph2["RISC-V Specifics"]
        hs_mode["HS Mode"]
        sbi_rt["SBI Runtime"]
        trap_handler["Trap Handler"]
    end
    subgraph subGraph1["ARM Specifics"]
        el2["EL2 Mode"]
        gicv2["GICv2 Interrupt Controller"]
        hvc["HVC Exception Handler"]
    end
    subgraph subGraph0["x86_64 Specifics"]
        vt_x["VT-x Virtualization"]
        x2apic["x2APIC Support"]
        vmexit_handler["VM Exit Handler"]
    end
end

arm_vcpu --> el2
arm_vcpu --> gicv2
arm_vcpu --> hvc
axvcpu --> arm_vcpu
axvcpu --> riscv_vcpu
axvcpu --> x86_vcpu
riscv_vcpu --> hs_mode
riscv_vcpu --> sbi_rt
riscv_vcpu --> trap_handler
x86_vcpu --> vmexit_handler
x86_vcpu --> vt_x
x86_vcpu --> x2apic

Sources: Cargo.lock(L138 - L148)  Cargo.lock(L154 - L168)  Cargo.lock(L1247 - L1269)  Cargo.lock(L1721 - L1739) 

x86_vcpu

Implements the VCPU interface for x86_64 architecture using Intel VT-x or AMD SVM technologies.

Key features:

• VM entry/exit handling
• Extended Page Tables (EPT) support
• Interrupt virtualization
• MSR virtualization

arm_vcpu

Implements the VCPU interface for ARM/aarch64 architecture using ARM Virtualization Extensions.

Key features:

• EL2 (Hypervisor mode) operations
• Stage-2 translation
• GICv2 interrupt controller virtualization
• Hypervisor calls (HVC) handling

riscv_vcpu

Implements the VCPU interface for RISC-V architecture using the RISC-V Hypervisor Extension.

Key features:

• Hypervisor-Supervisor (HS) mode operations
• Guest-page-table handling
• SBI services virtualization
• Trap handling

Sources: Cargo.lock(L1721 - L1739)  Cargo.lock(L154 - L168)  Cargo.lock(L1247 - L1269) 

ArceOS Foundation

AxVisor is built on top of the ArceOS (Architecture-centric OS) unikernel framework, which provides essential OS services and hardware abstractions.

axstd

A minimal standard library for ArceOS that provides common functionality and interfaces to the underlying OS services.

axhal

The Hardware Abstraction Layer that provides a uniform interface to hardware across different architectures.

Key responsibilities:

• CPU and interrupt management
• Timer services
• Memory management primitives
• Platform-specific initialization

axtask

Task management services including task creation, scheduling, and synchronization.

Key responsibilities:

• Task creation and lifecycle management
• Scheduling
• Inter-task synchronization

axalloc

Memory allocation services and memory management.

Key responsibilities:

• Physical memory allocation
• Heap management
• Memory pools

axconfig

System configuration management.

Key responsibilities:

• System parameters
• Feature flags
• Runtime configuration

axfs

Virtual filesystem support.

Key responsibilities:

• File operations
• Filesystem implementations (FAT32, etc.)
• Device filesystem

Sources: Cargo.toml(L24 - L32)  Cargo.lock(L118 - L134) 

Component Relationships and Dependencies

The components in AxVisor have well-defined relationships and dependencies that enable the modular architecture to function as a cohesive system.

flowchart TD
subgraph subGraph6["AxVisor System"]
    axvisor["axvisor - Entry Point"]
    subgraph subGraph5["OS Services"]
        axstd["axstd - Standard Library"]
        axhal["axhal - Hardware Abstraction Layer"]
        axtask["axtask - Task Management"]
        axalloc["axalloc - Memory Allocator"]
        axconfig["axconfig - System Configuration"]
        axfs["axfs - Filesystem"]
    end
    subgraph subGraph4["Device Emulation"]
        axdevice["axdevice - Device Emulation"]
        axdevice_base["axdevice_base - Base Device Interface"]
    end
    subgraph subGraph3["Memory Management"]
        axaddrspace["axaddrspace - Address Space Management"]
        page_tables["Page Tables"]
    end
    subgraph subGraph2["CPU Virtualization"]
        axvcpu["axvcpu - VCPU Interface"]
        arch_vcpu["Architecture-specific VCPU Implementations"]
    end
    subgraph subGraph1["VM Management"]
        axvm["axvm - VM Management"]
        axvmconfig["axvmconfig - VM Configuration"]
    end
end
subgraph subGraph0["User Applications"]
    vm_configs["VM Configuration Files"]
end

axaddrspace --> page_tables
axalloc --> axhal
axdevice --> axaddrspace
axdevice --> axdevice_base
axstd --> axalloc
axstd --> axconfig
axstd --> axfs
axstd --> axhal
axstd --> axtask
axtask --> axhal
axvcpu --> arch_vcpu
axvcpu --> axaddrspace
axvisor --> axstd
axvisor --> axvm
axvm --> axaddrspace
axvm --> axdevice
axvm --> axvcpu
axvm --> axvmconfig
vm_configs --> axvmconfig

Sources: Cargo.toml(L24 - L45) 

Component Interaction Flow

The following diagram illustrates how the components interact during VM creation and execution:

sequenceDiagram
    participant VMConfiguration as VM Configuration
    participant axvisor as axvisor
    participant axvm as axvm
    participant axvcpu as axvcpu
    participant ArchspecificVCPU as Arch-specific VCPU
    participant axaddrspace as axaddrspace
    participant axdevice as axdevice

    VMConfiguration ->> axvisor: Load configuration
    axvisor ->> axvm: Create VM(config)
    axvm ->> axaddrspace: Create address space
    axaddrspace -->> axvm: Address space created
    axvm ->> axdevice: Initialize devices
    axdevice -->> axvm: Devices initialized
    axvm ->> axvcpu: Create VCPUs
    axvcpu ->> ArchspecificVCPU: Create architecture-specific VCPU
    ArchspecificVCPU -->> axvcpu: VCPU created
    axvcpu -->> axvm: VCPUs created
    axvm ->> axvm: Load guest image
    axvm ->> axvcpu: Run VCPUs
    loop VCPU Execution
        axvcpu ->> ArchspecificVCPU: Run VCPU
    alt VM Exit: I/O Operation
        ArchspecificVCPU ->> axdevice: Handle I/O
        axdevice -->> ArchspecificVCPU: I/O handled
    else VM Exit: Memory Access
        ArchspecificVCPU ->> axaddrspace: Handle memory access
        axaddrspace -->> ArchspecificVCPU: Memory access handled
    else VM Exit: Hypercall
        ArchspecificVCPU ->> axvm: Handle hypercall
        axvm -->> ArchspecificVCPU: Hypercall handled
    end
    ArchspecificVCPU -->> axvcpu: VCPU exited
    end
    axvcpu ->> axvm: VM stopped
    axvm ->> axvisor: VM execution completed

Sources: Cargo.lock(L549 - L571)  Cargo.lock(L576 - L593)  Cargo.lock(L536 - L545)  Cargo.lock(L177 - L191) 

Component Key Features and Capabilities

Below is a table summarizing the key features and capabilities of the main components:

Sources: Cargo.toml(L14 - L45)  Cargo.lock(L549 - L571) 

Crate Dependency Tree

The core hypervisor crates have specific dependencies that demonstrate their relationships:

1. axvisor depends on:

• axvm - For VM management
• axvcpu - For VCPU operations
• axaddrspace - For memory management
• axstd - For OS services
• Various utility crates (log, bitflags, spin, etc.)

2. axvm depends on:

• axvcpu - For VCPU operations
• axaddrspace - For memory management
• axdevice - For device emulation
• axvmconfig - For configuration management

3. axvcpu provides common interfaces implemented by:

• x86_vcpu - For x86_64
• arm_vcpu - For ARM/aarch64
• riscv_vcpu - For RISC-V

This modular structure allows AxVisor to maintain a single codebase while supporting multiple architectures through well-defined interfaces and architecture-specific implementations.

Sources: Cargo.toml(L34 - L45)  Cargo.lock(L549 - L571)  Cargo.lock(L576 - L593) 

VM Management

Relevant source files

• Cargo.lock
• src/task.rs
• src/vmm/config.rs
• src/vmm/mod.rs
• src/vmm/vcpus.rs

This document describes how virtual machines (VMs) are created, configured, and managed within the AxVisor hypervisor system. It explains the VM lifecycle from initialization through execution to shutdown, as well as the management of virtual CPUs (VCPUs) within each VM. For information about the underlying hardware abstraction layer, see Hardware Abstraction Layer.

VM Architecture Overview

AxVisor implements a modular VM management system that separates VM configuration, execution, and resource management concerns. The core components work together to create, execute, and monitor virtual machines.

flowchart TD
subgraph subGraph1["Task Management"]
    TaskScheduler["ArceOS Task Scheduler"]
    TaskExt["Task Extensions"]
end
subgraph subGraph0["VM Management System"]
    VMM["VMM Module"]
    VMConfig["VM Configuration"]
    VMList["VM List"]
    VCPUManager["VCPU Manager"]
    ImageLoader["Image Loader"]
    VMs["VMs"]
    VCPUs["VCPUs"]
end

ImageLoader --> VMs
TaskExt --> VCPUs
TaskExt --> VMs
VCPUManager --> TaskScheduler
VCPUManager --> VCPUs
VMConfig --> VMs
VMList --> VMs
VMM --> VMConfig
VMM --> VMs
VMs --> VCPUs

Sources: src/vmm/mod.rs src/task.rs

VM Types and References

The VM management system uses specific types to represent VMs and VCPUs:

These types provide the foundation for VM management operations, with reference types allowing shared access to VM and VCPU resources across different components and tasks.

Sources: src/vmm/mod.rs(L16 - L20) 

VM Lifecycle

The VM lifecycle in AxVisor follows a defined sequence of operations, from initialization through execution to shutdown.

Sources: src/vmm/mod.rs(L28 - L65)  src/vmm/vcpus.rs(L275 - L367) 

VM Initialization

The VMM initializes VMs through the following process:

1. Call VMM::init() to initialize guest VMs and set up primary VCPUs
2. Load VM configurations from TOML files
3. Create VM instances using VM::new()
4. Load VM images based on configuration
5. Set up primary VCPUs for each VM

sequenceDiagram
    participant VMM as VMM
    participant ConfigModule as Config Module
    participant VMInstance as VM Instance
    participant VCPUManager as VCPU Manager

    VMM ->> ConfigModule: init_guest_vms()
    ConfigModule ->> ConfigModule: Parse TOML configs
    ConfigModule ->> VMInstance: Create VM with AxVMConfig
    VMInstance -->> ConfigModule: Return VM instance
    ConfigModule ->> ConfigModule: load_vm_images()
    ConfigModule ->> VMM: Return created VMs
    VMM ->> VCPUManager: setup_vm_primary_vcpu() for each VM
    VCPUManager ->> VCPUManager: Create primary VCPU task

Sources: src/vmm/mod.rs(L28 - L39)  src/vmm/config.rs(L25 - L43) 

VM Execution

The VMM starts VM execution:

1. Call VMM::start() to boot all VMs
2. For each VM, call vm.boot() to start execution
3. Notify the primary VCPU to begin execution
4. Increment RUNNING_VM_COUNT for each successfully started VM
5. Wait until all VMs are stopped (when RUNNING_VM_COUNT reaches 0)

VCPUs execute in their own tasks, handling various exit reasons such as hypercalls, interrupts, halts, and system shutdown events.

Sources: src/vmm/mod.rs(L42 - L65)  src/vmm/vcpus.rs(L275 - L367) 

VM Configuration

AxVisor uses TOML-based configuration files to define VM properties:

1. Configuration files are stored in configs/vms/ directory
2. Default configurations are provided for different architectures (x86_64, aarch64, riscv64)
3. Configurations define VM properties like ID, name, CPU count, memory regions, etc.
4. Configuration processing involves:

• Parsing TOML into AxVMCrateConfig
• Converting to AxVMConfig for VM creation
• Using configuration to load appropriate VM images

Sources: src/vmm/config.rs(L1 - L43) 

VCPU Management

VCPUs are managed for each VM, with special handling for the primary VCPU:

classDiagram
class VMVCpus {
    -usize _vm_id
    -WaitQueue wait_queue
    -Vec~AxTaskRef~ vcpu_task_list
    -AtomicUsize running_halting_vcpu_count
    +new(VMRef) VMVCpus
    +add_vcpu_task(AxTaskRef)
    +wait()
    +wait_until(F)
    +notify_one()
    +mark_vcpu_running()
    +mark_vcpu_exiting() bool
}

class VCPU_Task {
    +stack_size KERNEL_STACK_SIZE
    +TaskExt ext
    +run() vcpu_run
}

class TaskExt {
    +VMRef vm
    +VCpuRef vcpu
    +new(VMRef, VCpuRef) TaskExt
}

VCPU_Task "1" *-- "1" TaskExt : contains
VMVCpus "1" *-- "*" VCPU_Task : manages

Sources: src/vmm/vcpus.rs(L27 - L40)  src/task.rs(L6 - L11) 

VCPU Initialization

For each VM, VCPUs are initialized:

1. setup_vm_primary_vcpu() is called for each VM
2. A VMVCpus structure is created to manage VCPUs for the VM
3. The primary VCPU (ID 0) is set up first
4. A task is created for the primary VCPU with the vcpu_run entry point
5. The VCPU task is added to the VM's VCPU task list
6. The VM's VCPU structure is stored in a global mapping

Sources: src/vmm/vcpus.rs(L218 - L231) 

VCPU Task Creation

Each VCPU runs in its own ArceOS task:

1. alloc_vcpu_task() creates a task for a VCPU
2. Tasks are created with a 256 KiB kernel stack
3. The task's entry point is set to vcpu_run()
4. If configured, the VCPU is assigned to specific physical CPUs
5. The task is initialized with TaskExt containing references to the VM and VCPU
6. The task is spawned using axtask::spawn_task()

Sources: src/vmm/vcpus.rs(L249 - L268)  src/task.rs(L1 - L19) 

VCPU Execution Loop

The main VCPU execution flow is:

1. Wait for the VM to be in running state
2. Mark the VCPU as running, incrementing the running count
3. Enter execution loop:

• Run the VCPU using vm.run_vcpu(vcpu_id)
• Handle various exit reasons (hypercalls, interrupts, halts, etc.)
• Check if the VM is shutting down

4. When the VM is shutting down:

• Mark the VCPU as exiting
• If this was the last VCPU, decrement RUNNING_VM_COUNT
• Wake the VMM wait queue if necessary
• Exit the execution loop

Sources: src/vmm/vcpus.rs(L275 - L367) 

Secondary VCPU Management

AxVisor supports dynamic secondary VCPU initialization:

1. When a primary VCPU requests to start a secondary VCPU (via CpuUp exit):

• vcpu_on() is called with the target VCPU ID, entry point, and argument
• The target VCPU's entry point and registers are set
• A new task is created for the secondary VCPU
• The task is added to the VM's VCPU task list

2. Secondary VCPUs follow the same execution loop as the primary VCPU

Sources: src/vmm/vcpus.rs(L179 - L208)  src/vmm/vcpus.rs(L319 - L330) 

VM Coordination and Shutdown

VMs and VCPUs coordinate through wait queues:

1. VMVCpus maintains a wait queue for each VM
2. VCPUs can wait on the queue (e.g., when halted)
3. Other VCPUs can notify waiting VCPUs (e.g., for interrupt handling)
4. When a VM is shutting down, all VCPUs detect this condition
5. Each VCPU marks itself as exiting
6. The last exiting VCPU decrements RUNNING_VM_COUNT
7. When RUNNING_VM_COUNT reaches 0, the VMM itself is notified to exit

This coordination ensures proper cleanup when VMs are shut down.

Sources: src/vmm/vcpus.rs(L42 - L108)  src/vmm/mod.rs(L55 - L65) 

VM Image Loading

VM images are loaded according to configuration:

1. Images can be loaded from a FAT32 filesystem or from memory
2. Configuration specifies the kernel path, entry point, and load address
3. The image loading process occurs after VM creation but before VCPU setup

This completes the VM preparation before execution begins.

Sources: src/vmm/config.rs(L39 - L41) 

Hardware Abstraction Layer

Relevant source files

• Cargo.lock
• Cargo.toml

The Hardware Abstraction Layer (HAL) in AxVisor provides a uniform interface to access hardware features across multiple architectures. This component enables architecture-independent code to interact with hardware resources without needing to handle architecture-specific details directly. The HAL is a critical foundation that allows AxVisor to support x86_64, ARM/aarch64, and RISC-V architectures from a single codebase.

For detailed information about memory virtualization specifically, see Memory Management.

HAL Architecture Overview

AxVisor's HAL is implemented in the axhal crate, which provides platform-agnostic interfaces that delegate to architecture-specific implementations. This design enables higher-level components of AxVisor to operate independently of the underlying hardware.

flowchart TD
subgraph subGraph0["Hardware Abstraction Layer Structure"]
    hal["axhal - Hardware Abstraction Layer"]
    common["Common Interface"]
    arch_spec["Architecture-Specific Implementations"]
    mem_mgmt["Memory Management"]
    int_mgmt["Interrupt Management"]
    time_mgmt["Time Management"]
    cpu_mgmt["CPU Features"]
    x86_impl["x86_64 Implementation"]
    arm_impl["ARM/aarch64 Implementation"]
    riscv_impl["RISC-V Implementation"]
    x86_paging["x86_64 Paging"]
    x86_int["x86_64 Interrupts(x2APIC)"]
    x86_cpu["x86_64 CPU Features(VT-x)"]
    arm_paging["ARM Paging"]
    arm_int["ARM Interrupts(GICv2)"]
    arm_cpu["ARM CPU Features(EL2 mode)"]
    riscv_paging["RISC-V Paging"]
    riscv_int["RISC-V Interrupts"]
    riscv_cpu["RISC-V CPU Features(SBI runtime)"]
end

arch_spec --> arm_impl
arch_spec --> riscv_impl
arch_spec --> x86_impl
arm_impl --> arm_cpu
arm_impl --> arm_int
arm_impl --> arm_paging
common --> cpu_mgmt
common --> int_mgmt
common --> mem_mgmt
common --> time_mgmt
hal --> arch_spec
hal --> common
riscv_impl --> riscv_cpu
riscv_impl --> riscv_int
riscv_impl --> riscv_paging
x86_impl --> x86_cpu
x86_impl --> x86_int
x86_impl --> x86_paging

Sources:

• Cargo.lock
• Cargo.toml

HAL Core Components

The HAL provides several key functionality areas that are essential for hypervisor operation:

1. Architecture-Independent Interfaces

The HAL defines common interfaces for hardware operations that higher-level components can use regardless of architecture:

flowchart TD
subgraph subGraph1["HAL Common Interfaces"]
    hi["High-Level Components"]
    hal_if["HAL Interface Functions"]
    mem_if["Memory Interface- Page tables- Memory mapping"]
    int_if["Interrupt Interface- Register handlers- Enable/disable"]
    time_if["Time Interface- Current time- Timers"]
    cpu_if["CPU Interface- Core features- Virtualization"]
    subgraph Implementations["Implementations"]
        arch_impl["Architecture-Specific Implementations"]
        x86["x86_64"]
        arm["ARM/aarch64"]
        riscv["RISC-V"]
    end
end

arch_impl --> arm
arch_impl --> riscv
arch_impl --> x86
hal_if --> arch_impl
hal_if --> cpu_if
hal_if --> int_if
hal_if --> mem_if
hal_if --> time_if
hi --> hal_if

Sources:

• Cargo.lock (lines 391-427)

2. Memory Management

The HAL provides memory management facilities including page table manipulation, address space management, and physical memory allocation:

Sources:

• Cargo.lock (lines 1019-1032) - memory_addr and memory_set packages
• Cargo.lock (lines 1062-1084) - page_table_entry and page_table_multiarch packages

3. Interrupt Handling

The HAL manages hardware interrupts through an architecture-independent interface:

flowchart TD
subgraph subGraph1["Interrupt Handling Architecture"]
    int_if["Interrupt Interface"]
    subgraph Architecture-Specific["Architecture-Specific"]
        int_reg["Register Handler"]
        int_en["Enable/Disable"]
        int_ack["Acknowledge"]
        x86_int["x86_64 (x2APIC)"]
        arm_int["ARM (GICv2)"]
        riscv_int["RISC-V (PLIC)"]
    end
end

int_ack --> arm_int
int_ack --> riscv_int
int_ack --> x86_int
int_en --> arm_int
int_en --> riscv_int
int_en --> x86_int
int_if --> int_ack
int_if --> int_en
int_if --> int_reg
int_reg --> arm_int
int_reg --> riscv_int
int_reg --> x86_int

Sources:

• Cargo.lock (lines 137-142) - arm_gicv2 package
• Cargo.lock (lines 1673-1683) - x2apic package

4. Timer and Time Management

The HAL provides interfaces for accessing system timers and managing time-related events:

Sources:

• Cargo.lock (lines 1460-1463) - timer_list package

Architecture-Specific Implementations

The HAL implements architecture-specific features for the supported platforms:

x86_64 Implementation

The x86_64 implementation provides:

• Memory management through 4-level paging
• Interrupt handling using x2APIC
• CPU virtualization using VT-x technology
• I/O ports and MMIO access
• MSR (Model Specific Registers) access

Sources:

• Cargo.lock (lines 1686-1694) - x86 package
• Cargo.lock (lines 1709-1718) - x86_64 package

ARM/aarch64 Implementation

The ARM implementation offers:

• Memory management through stage 2 translation tables
• Interrupt handling using GICv2
• CPU virtualization using EL2 execution level
• System register access
• Virtualization extensions for memory and interrupts

Sources:

• Cargo.lock (lines 6-12) - aarch64-cpu package
• Cargo.lock (lines 137-142) - arm_gicv2 package
• Cargo.lock (lines 145-151) - arm_pl011 package (UART)

RISC-V Implementation

The RISC-V implementation provides:

• Memory management through Sv39/Sv48 paging
• Interrupt handling using platform-specific interrupt controllers
• SBI (Supervisor Binary Interface) runtime for hypervisor services
• H-extension support for virtualization (where available)

Sources:

• Cargo.lock (lines 1203-1213) - riscv package
• Cargo.lock (lines 1320-1333) - sbi-rt and sbi-spec packages

Integration with Virtual CPUs

The HAL provides the foundation for virtual CPU (VCPU) implementations by exposing hardware virtualization features through a common interface:

flowchart TD
subgraph subGraph1["HAL Integration with VCPUs"]
    axhal["axhal (HAL)"]
    axvcpu["axvcpu (Virtual CPU Interface)"]
    x86_vcpu["x86_vcpu Implementation"]
    arm_vcpu["arm_vcpu Implementation"]
    riscv_vcpu["riscv_vcpu Implementation"]
    subgraph subGraph0["Architecture-Specific HAL Features Used by VCPUs"]
        x86_vt["x86 VT-x Features- VMCS- EPT- VM Entry/Exit"]
        arm_el2["ARM EL2 Features- Stage 2 MMU- vGIC- HVC"]
        riscv_h["RISC-V H-Extension- Guest mode- Two-stage translation"]
    end
end

arm_vcpu --> arm_el2
axhal --> axvcpu
axvcpu --> arm_vcpu
axvcpu --> riscv_vcpu
axvcpu --> x86_vcpu
riscv_vcpu --> riscv_h
x86_vcpu --> x86_vt

Sources:

• Cargo.lock (lines 538-544) - axvcpu package
• Cargo.lock (lines 154-168) - arm_vcpu package
• Cargo.lock (lines 1248-1269) - riscv_vcpu package
• Cargo.lock (lines 1721-1739) - x86_vcpu package

Hardware Resource Management

Memory Allocation

The HAL works with the memory allocator (axalloc) to provide physical memory management:

Sources:

• Cargo.lock (lines 196-205) - axalloc package

Device Access

The HAL provides access to physical devices and exposes platform-specific devices:

Sources:

• Cargo.lock (lines 145-151) - arm_pl011 package
• Cargo.lock (lines 1035-1038) - ns16550a package
• Cargo.lock (lines 137-142) - arm_gicv2 package

HAL Initialization

During system boot, the HAL is initialized with architecture-specific procedures:

1. Early platform detection and initialization
2. Memory map discovery and setup
3. Architecture-specific hardware initialization
4. Interrupt controller setup
5. Timer initialization
6. Per-CPU state initialization

This initialization sequence ensures that the hardware is properly configured before higher-level hypervisor components begin execution.

Sources:

• Cargo.lock (lines 391-427) - axhal package

Summary

The Hardware Abstraction Layer in AxVisor provides a unified interface to access hardware features across multiple architectures. It implements architecture-specific features while presenting a common API to higher-level components. This design enables AxVisor to support multiple guest architectures (x86_64, ARM/aarch64, and RISC-V) from a single codebase, allowing for efficient hypervisor operation across diverse hardware platforms.

The HAL is tightly integrated with other components of AxVisor, particularly with memory management, virtual CPU implementations, and device emulation, forming the foundation upon which the entire hypervisor is built.

Memory Management

Relevant source files

• Cargo.lock
• Cargo.toml
• configs/platforms/x86_64-qemu-q35.toml
• configs/vms/arceos-x86_64.toml
• configs/vms/nimbos-x86_64.toml
• scripts/lds/linker.lds.S
• src/vmm/timer.rs

This page documents the memory management subsystem in AxVisor, focusing on how memory virtualization and address space management are implemented in the hypervisor. For information about the hardware abstraction layer, see Hardware Abstraction Layer.

Overview

AxVisor's memory management system is responsible for:

• Virtualizing physical memory for guest VMs
• Managing address spaces for both hypervisor and VMs
• Handling stage-2 page tables for memory virtualization
• Performing address translations between different memory spaces
• Allocating and mapping memory regions for guest VMs

The system utilizes the axaddrspace crate to provide architecture-independent abstractions for memory management while supporting x86_64, ARM/aarch64, and RISC-V architectures.

flowchart TD
subgraph subGraph2["Hardware Abstraction"]
    axhal["axhal"]
    paging["Paging Subsystem"]
end
subgraph subGraph1["Virtual Machine"]
    vm["VM"]
    vcpu["VCPU"]
    guest_memory["Guest Memory"]
end
subgraph subGraph0["Memory Management System"]
    axaddrspace["axaddrspace"]
    addrspace["AddressSpace"]
    memory_regions["MemoryRegion Management"]
    page_tables["Page Table Handling"]
    addr_translation["Address Translation"]
end

addrspace --> guest_memory
addrspace --> page_tables
axaddrspace --> addr_translation
axaddrspace --> addrspace
axaddrspace --> memory_regions
axaddrspace --> page_tables
axhal --> paging
page_tables --> paging
vcpu --> guest_memory
vm --> guest_memory
vm --> memory_regions
vm --> vcpu

Sources: Cargo.toml(L35 - L37) 

Address Spaces and Memory Virtualization

In AxVisor, memory virtualization involves three distinct address spaces:

1. Guest Virtual Address (GVA) - Virtual addresses used by the guest OS
2. Guest Physical Address (GPA) - Physical addresses from the guest OS perspective
3. Host Physical Address (HPA) - Actual physical addresses in the host system

Memory virtualization is achieved through two levels of translation:

• First-stage translation (managed by the guest): GVA → GPA
• Second-stage translation (managed by the hypervisor): GPA → HPA

flowchart TD
subgraph subGraph1["Hypervisor Control"]
    HPA["Host PhysicalAddress (HPA)"]
end
subgraph subGraph0["Guest OS Control"]
    GVA["Guest VirtualAddress (GVA)"]
    GPA["Guest PhysicalAddress (GPA)"]
end

GPA --> HPA
GVA --> GPA
GVA --> HPA

Sources: configs/platforms/x86_64-qemu-q35.toml(L15 - L30) 

Address Space Management

The axaddrspace crate provides a unified interface for managing address spaces across different architectures. It handles the creation, manipulation, and destruction of second-stage page tables used for GPA to HPA translations.

Key components include:

• AddressSpace: Represents a virtual address space for a guest VM
• MemoryRegion: Defines a contiguous region of memory with specific permissions
• PageTable: Manages the page tables for address translation

classDiagram
class AddressSpace {
    +root_page_table
    +memory_regions
    +create()
    +map()
    +unmap()
    +query()
    +contains()
}

class MemoryRegion {
    +start
    +size
    +flags
    +map_type
    
}

class PageTable {
    +arch_specific_implementation
    +map_page()
    +unmap_page()
    +translate()
}

AddressSpace "1" --> "many" MemoryRegion : contains
AddressSpace "1" --> "1" PageTable : uses

Sources: Cargo.toml(L37) 

Memory Region Types

AxVisor supports different types of memory regions that can be mapped into a guest VM's address space:

Memory regions are configured with specific access permissions (read, write, execute) and mapping types.

Sources: configs/vms/arceos-x86_64.toml(L40 - L44)  configs/platforms/x86_64-qemu-q35.toml(L37 - L45) 

VM Memory Configuration

Virtual machines in AxVisor are configured with memory regions specified in their configuration files. Here's how memory regions are defined:

memory_regions = [
    [base_paddr, size, flags, map_type],
]

Where:

• base_paddr: Base guest physical address
• size: Size of the memory region in bytes
• flags: Access permissions (bit 0: read, bit 1: write, bit 2: execute)
• map_type: 0 for MAP_ALLOC, 1 for MAP_IDENTICAL

The hypervisor creates and manages these memory regions when a VM is initialized.

flowchart TD
subgraph subGraph0["Configuration Example"]
    config["memory_regions = [[0x0000_0000, 0x100_0000, 0x7, 0],]"]
end
vmconfig["VM Configuration"]
mm_region["Memory Region Specification"]
vmm["VMM"]
addrspace["Address Space"]
map_operation["Memory Mapping"]

addrspace --> map_operation
config --> mm_region
mm_region --> vmm
vmconfig --> mm_region
vmm --> addrspace

Sources: configs/vms/arceos-x86_64.toml(L40 - L44)  configs/vms/nimbos-x86_64.toml(L40 - L44) 

Page Table Management

AxVisor uses the page_table_multiarch and page_table_entry crates to provide architecture-independent page table management. These crates abstract the architecture-specific details of page tables for x86_64, ARM/aarch64, and RISC-V.

Key functionalities include:

• Creating and managing page tables
• Mapping guest physical addresses to host physical addresses
• Handling page faults and TLB operations
• Supporting different page sizes for efficient memory mapping

The page tables implement the second-stage translation (GPA → HPA) required for memory virtualization.

Sources: Cargo.toml(L44 - L45) 

Address Translation

Address translation in AxVisor involves converting between different address spaces:

1. GPA to HPA Translation: Used when a guest VM accesses physical memory
2. HPA to GPA Translation: Used for handling page faults and device emulation
3. Linear Address Translation: Used for quick conversions between host virtual and physical addresses

The physical-virtual offset defined in the platform configuration allows for efficient linear mapping between host virtual and physical addresses.

flowchart TD
subgraph subGraph1["Translation Components"]
    stage2_pt["Stage-2 Page Tables"]
    tlb["TLB"]
    page_fault["Page Fault Handler"]
end
subgraph subGraph0["Translation Process"]
    gpa["Guest Physical Address (GPA)"]
    hpa["Host Physical Address (HPA)"]
end

gpa --> hpa
gpa --> stage2_pt
hpa --> gpa
page_fault --> stage2_pt
stage2_pt --> hpa
stage2_pt --> tlb

Sources: configs/platforms/x86_64-qemu-q35.toml(L22 - L26) 

Memory Isolation and Protection

AxVisor ensures memory isolation between guest VMs and between guests and the hypervisor through several mechanisms:

1. Separate Address Spaces: Each VM has its own address space with second-stage page tables
2. Permission Enforcement: Memory regions have specific access permissions enforced by hardware
3. MMIO Protection: Special handling for memory-mapped I/O regions
4. EPT/NPT/Stage-2 MMU: Hardware virtualization extensions enforce memory isolation

This ensures that one VM cannot access the memory of another VM or the hypervisor, providing strong security isolation.

Sources: Cargo.toml(L37) 

Kernel Memory Layout

The hypervisor kernel itself has a specific memory layout defined by the linker script. This layout organizes different sections of the kernel in memory:

The layout ensures proper memory alignment and access protection for each section.

Sources: scripts/lds/linker.lds.S(L11 - L76) 

Timer Subsystem and Memory

The VMM timer subsystem interacts with memory management through the allocation and management of timer events. Each timer event is stored in memory and scheduled based on its deadline.

flowchart TD
subgraph subGraph1["Memory Management"]
    allocation["Memory Allocation"]
    percpu["Per-CPU Storage"]
end
subgraph subGraph0["Timer Subsystem"]
    timer_list["TimerList"]
    timer_event["VmmTimerEvent"]
    register["register_timer()"]
    check["check_events()"]
end
time_value["TimeValue"]

allocation --> timer_event
check --> timer_list
percpu --> timer_list
register --> timer_list
time_value --> timer_list
timer_list --> timer_event

Sources: src/vmm/timer.rs(L18 - L23)  src/vmm/timer.rs(L44) 

Architecture-Specific Considerations

While AxVisor provides architecture-independent abstractions for memory management, there are important architecture-specific considerations:

x86_64

• Uses Extended Page Tables (EPT) for second-stage translation
• Supports large pages (2MB, 1GB) for efficient mapping
• Handles specific x86 memory types and caching attributes

ARM/aarch64

• Uses Stage-2 translation tables
• Supports different EL2 translation regimes
• Handles ARM-specific memory attributes

RISC-V

• Uses Sv39/Sv48 page tables with appropriate extensions
• Implements RISC-V specific memory permission bits
• Handles RISC-V address translation modes

Each architecture implementation conforms to the common interfaces provided by axaddrspace, page_table_entry, and page_table_multiarch.

Sources: Cargo.toml(L44 - L45) 

Conclusion

AxVisor's memory management system provides a robust, architecture-independent framework for virtualizing memory across different processor architectures. By abstracting the details of address space management and page table handling, it enables efficient memory virtualization while ensuring strong isolation between guest VMs.

The system leverages hardware virtualization features to minimize overhead while providing the flexibility needed to support diverse guest operating systems and configurations.

Configuration

Relevant source files

• Cargo.toml
• README.md
• configs/platforms/x86_64-qemu-q35.toml
• configs/vms/arceos-x86_64.toml
• configs/vms/nimbos-x86_64.toml
• scripts/lds/linker.lds.S
• src/vmm/timer.rs

This document describes the configuration system used by AxVisor. Configuration files define essential parameters for both the hypervisor and the virtual machines it manages. This page covers the structure and format of configuration files, how they are processed, and the available configuration options. For specific VM configuration details, see VM Configuration.

Overview of Configuration System

AxVisor uses TOML files for configuration, which provide a clear and human-readable format for defining hypervisor and virtual machine parameters. Two primary types of configuration files exist:

1. Platform Configurations - Define hardware-specific settings for the target platform
2. VM Configurations - Define settings for individual virtual machines

These configuration files are processed during the build process and at runtime to initialize the hypervisor and its virtual machines.

flowchart TD
subgraph subGraph1["Configuration Processing"]
    build_process["Build Process"]
    vmm_init["VMM::init()"]
    axconfig_rs["axconfig.rs"]
    vms["Virtual Machines"]
end
subgraph subGraph0["Configuration System"]
    config_files["TOML Configuration Files"]
    platform_config["Platform Configsconfigs/platforms/*.toml"]
    vm_config["VM Configsconfigs/vms/*.toml"]
    hardware_params["Hardware ParametersMemory layoutDevice mappingsArchitecture settings"]
    vm_params["VM ParametersCPU countMemory allocationDevice assignments"]
    build_config["Build ConfigurationMakefile parameters"]
end

build_config --> config_files
build_process --> axconfig_rs
config_files --> platform_config
config_files --> vm_config
platform_config --> build_process
platform_config --> hardware_params
vm_config --> vm_params
vm_config --> vmm_init
vmm_init --> vms

Sources: README.md(L71 - L74)  README.md(L77 - L79) 

Configuration File Structure

Platform Configuration Files

Platform configuration files define the hardware-specific settings for the target platform. These files are located in the configs/platforms/ directory with filenames that correspond to the platform, such as x86_64-qemu-q35.toml.

A platform configuration file has the following main sections:

Example platform configuration excerpt:

# Architecture identifier
arch = "x86_64"
# Platform identifier
platform = "x86_64-qemu-q35"

[plat]
# Base virtual address of the kernel image
kernel-base-vaddr = "0xffff_8000_0020_0000"
# Linear mapping offset
phys-virt-offset = "0xffff_8000_0000_0000"

[devices]
# MMIO regions with format (`base_paddr`, `size`)
mmio-regions = [
    [0xfec0_0000, 0x1000],      # IO APIC
    [0xfed0_0000, 0x1000],      # HPET
    [0xfee0_0000, 0x1000],      # Local APIC
]

Sources: configs/platforms/x86_64-qemu-q35.toml(L1 - L54) 

VM Configuration Files

VM configuration files define the properties of individual virtual machines that will be managed by AxVisor. These files are located in the configs/vms/ directory with names like arceos-x86_64.toml.

A VM configuration file contains these main sections:

Example VM configuration excerpt:

[base]
# Guest VM ID
id = 1
# Guest VM name
name = "arceos"
# Virtualization type
vm_type = 1
# The number of virtual CPUs
cpu_num = 1

[kernel]
# The entry point of the kernel image
entry_point = 0x8000
# The location of image: "memory" | "fs"
image_location = "fs"
# The file path of the kernel image
kernel_path = "arceos-x86_64.bin"
# The load address of the kernel image
kernel_load_addr = 0x20_0000

Sources: configs/vms/arceos-x86_64.toml(L1 - L37)  configs/vms/nimbos-x86_64.toml(L1 - L39) 

VM Configuration Options

The VM configuration file contains multiple sections that control various aspects of the virtual machine. Here's a detailed breakdown of the available options:

flowchart TD
subgraph subGraph0["VM Configuration Structure"]
    vm_config["VM Configuration File(.toml)"]
    base_section["[base] SectionVM Identity"]
    kernel_section["[kernel] SectionGuest Image & Boot"]
    devices_section["[devices] SectionDevice Configuration"]
    base_options["id: VM identifiername: VM namevm_type: Virtualization typecpu_num: Number of vCPUsphys_cpu_sets: CPU pinning"]
    kernel_options["entry_point: Entry addressimage_location: 'memory' or 'fs'kernel_path: Image pathkernel_load_addr: Load addressmemory_regions: Memory layout"]
    optional_kernel["bios_path: BIOS imagebios_load_addr: BIOS load addressramdisk_path: Ramdisk imagedisk_path: Disk image"]
    device_options["emu_devices: Emulated devicespassthrough_devices: Passthrough devices"]
end

base_section --> base_options
devices_section --> device_options
kernel_section --> kernel_options
kernel_section --> optional_kernel
vm_config --> base_section
vm_config --> devices_section
vm_config --> kernel_section

Sources: configs/vms/arceos-x86_64.toml(L1 - L78)  configs/vms/nimbos-x86_64.toml(L1 - L78) 

Base Section

The [base] section defines the core identity and properties of the VM:

Kernel Section

The [kernel] section defines how the guest kernel is loaded and executed:

Devices Section

The [devices] section defines the virtual and passthrough devices for the VM:

Example passthrough device entry:

passthrough_devices = [
    [
        "IO APIC",
        0xfec0_0000,
        0xfec0_0000,
        0x1000,
        0x1,
    ]
]

Sources: configs/vms/arceos-x86_64.toml(L50 - L78)  configs/vms/nimbos-x86_64.toml(L50 - L78) 

Image Loading Methods

AxVisor supports two methods for loading guest VM images:

1. File System Loading (image_location = "fs"):

• Loads the guest image from a FAT32 file system
• Requires setting up a disk image file with the guest image(s)
• Used when the guest images need to be changed without rebuilding the hypervisor

2. Memory Loading (image_location = "memory"):

• Loads the guest image from memory, bound to the hypervisor through static compilation
• Uses include_bytes! to include the image in the hypervisor binary
• Useful for embedded scenarios or when the guest image is fixed

flowchart TD
subgraph subGraph0["Image Loading Process"]
    config["VM Configuration"]
    load_method["image_location"]
    fs_load["Load from File System"]
    mem_load["Load from Memory"]
    mount["Mount FAT32 Filesystem"]
    read_fs["Read kernel_path from disk"]
    load_mem["Load to kernel_load_addr"]
    compile["Statically Compiled Imageinclude_bytes!()"]
    load_mem2["Load to kernel_load_addr"]
    start_vm["Start VM"]
end

compile --> load_mem2
config --> load_method
fs_load --> mount
load_mem --> start_vm
load_mem2 --> start_vm
load_method --> fs_load
load_method --> mem_load
mem_load --> compile
mount --> read_fs
read_fs --> load_mem

Sources: README.md(L78 - L112) 

Build Configuration

The build configuration controls how AxVisor is compiled and which features are enabled. The primary method for configuring the build is through the make command and its parameters.

Key build parameters include:

Example build command:

make ACCEL=n ARCH=aarch64 LOG=info VM_CONFIGS=configs/vms/arceos-aarch64.toml APP_FEATURES=fs run

Additionally, build-time dependencies are declared in Cargo.toml, including the TOML parser and configuration tools:

[build-dependencies]
toml = { git = "https://github.com/arceos-hypervisor/toml.git", branch = "no_std" }
axconfig = { git = "https://github.com/arceos-hypervisor/arceos.git", branch = "vmm" }

Sources: Cargo.toml(L47 - L52)  README.md(L112 - L113) 

Configuration Processing

Configuration files are processed both at build time and at runtime:

1. Build-time Processing:

• Platform configurations are processed during the build
• Generate architecture-specific code based on the platform configuration
• Set up memory layouts and device mappings

2. Runtime Processing:

• VM configurations are processed when AxVisor starts
• The VMM (Virtual Machine Manager) reads and applies VM configurations
• Creates VM instances according to the configuration

The configuration files are parsed using a TOML parser adapted for use in a no_std environment. The parsed configuration is then used to initialize various components of the hypervisor.

For example, timer configuration parameters are used to set up the timer subsystem:

// Timer interrupt frequency is configured in the platform configuration
const PERIODIC_INTERVAL_NANOS: u64 = axhal::time::NANOS_PER_SEC / axconfig::TICKS_PER_SEC as u64;

Sources: src/vmm/timer.rs(L12 - L13) 

Configuration Tools

AxVisor provides additional tools to help with configuration:

1. axvmconfig: A tool for generating VM configuration files

• Simplifies the process of creating complex VM configurations
• Provides validation of configuration parameters

2. dev_env.py: A script for setting up the development environment

• Helps localize relevant crates for development and debugging
• Makes it easier to work with the modular components of AxVisor

Sources: README.md(L77 - L79)  README.md(L210 - L214) 

Conclusion

AxVisor's configuration system provides a flexible and powerful way to define both the hypervisor platform and the virtual machines it manages. By using TOML files, the configuration is human-readable and easy to modify. The separation between platform and VM configurations allows for a modular approach, where the same hypervisor build can host different VMs without requiring recompilation.

Understanding the configuration options is essential for successfully deploying AxVisor and managing virtual machines with different requirements and characteristics.

VM Configuration

Relevant source files

• README.md
• configs/platforms/x86_64-qemu-q35.toml
• configs/vms/arceos-x86_64.toml
• configs/vms/nimbos-x86_64.toml
• scripts/lds/linker.lds.S
• src/vmm/timer.rs

This page describes how to configure virtual machines (VMs) in AxVisor. The configuration system uses TOML files to define VM properties, including basic VM information, kernel image details, memory layout, and device specifications. For information about platform-specific configuration options, see Platform-Specific Configuration.

Configuration Overview

AxVisor uses a structured configuration approach that allows users to declaratively define VM properties using TOML files. These configuration files are processed during VM initialization and determine how VMs are created, what resources they receive, and how they interact with the host system.

flowchart TD
subgraph subGraph1["VM Management"]
    vm["VM Object"]
    vcpu["VCPU Objects"]
    memory["Memory Regions"]
    devices["Device Configuration"]
end
subgraph subGraph0["VM Configuration System"]
    toml_config["TOML Configuration Files"]
    vmm_init["VMM::init()"]
    vm_creation["VM Creation"]
    image_loading["Image Loading"]
end

toml_config --> vmm_init
vm --> devices
vm --> memory
vm --> vcpu
vm_creation --> image_loading
vm_creation --> vm
vmm_init --> vm_creation

Sources: README.md(L69 - L78)  configs/vms/arceos-x86_64.toml(L1 - L78) 

Configuration File Structure

The VM configuration file is divided into three main sections: [base], [kernel], and [devices].

Base Configuration

The [base] section defines fundamental VM properties:

[base]
# Guest vm id.
id = 1
# Guest vm name.
name = "arceos"
# Virtualization type.
vm_type = 1
# The number of virtual CPUs.
cpu_num = 1
# Guest vm physical cpu sets.
phys_cpu_sets = [1]

Sources: configs/vms/arceos-x86_64.toml(L1 - L13)  configs/vms/nimbos-x86_64.toml(L1 - L13) 

Kernel Configuration

The [kernel] section specifies how the guest kernel image should be loaded and executed:

[kernel]
# The entry point of the kernel image.
entry_point = 0x8000
# The location of image: "memory" | "fs".
image_location = "fs"
# The file path of the BIOS image.
bios_path = "axvm-bios.bin"
# The load address of the BIOS image.
bios_load_addr = 0x8000
# The file path of the kernel image.
kernel_path = "arceos-x86_64.bin"
# The load address of the kernel image.
kernel_load_addr = 0x20_0000

# Memory regions with format (`base_paddr`, `size`, `flags`, `map_type`).
# For `map_type`, 0 means `MAP_ALLOC`, 1 means `MAP_IDENTICAL`.
memory_regions = [
    [0x0000_0000, 0x100_0000, 0x7, 0], # Low RAM 16M 0b111
]

Sources: configs/vms/arceos-x86_64.toml(L14 - L44)  configs/vms/nimbos-x86_64.toml(L14 - L44) 

Device Configuration

The [devices] section defines both emulated and passthrough devices:

[devices]
# Emu_devices.
# Name Base-Ipa Ipa_len Alloc-Irq Emu-Type EmuConfig.
emu_devices = []

# Pass-through devices.
# Name Base-Ipa Base-Pa Length Alloc-Irq.
passthrough_devices = [
    [
        "IO APIC",
        0xfec0_0000,
        0xfec0_0000,
        0x1000,
        0x1,
    ],
    [
        "Local APIC",
        0xfee0_0000,
        0xfee0_0000,
        0x1000,
        0x1,
    ],
]

Sources: configs/vms/arceos-x86_64.toml(L46 - L78)  configs/vms/nimbos-x86_64.toml(L46 - L78) 

Memory Region Configuration

Memory regions in the memory_regions parameter use a structured format:

flowchart TD
subgraph subGraph0["Memory Region Format"]
    base_paddr["base_paddr: Starting Physical Address"]
    size["size: Region Size in Bytes"]
    flags["flags: Memory Permissions (Bit flags)"]
    map_type["map_type: Mapping Type (0=MAP_ALLOC, 1=MAP_IDENTICAL)"]
end
read["Read Permission"]
write["Write Permission"]
execute["Execute Permission"]

flags --> execute
flags --> read
flags --> write

Memory Flags

The flags field defines memory permissions as bit flags:

• Bit 0 (0x1): Read permission
• Bit 1 (0x2): Write permission
• Bit 2 (0x4): Execute permission

Common combinations:

• 0x7 (0b111): Read + Write + Execute
• 0x3 (0b011): Read + Write
• 0x1 (0b001): Read-only

Mapping Types

The map_type field defines how memory is mapped:

• 0 (MAP_ALLOC): Allocate new memory for the VM
• 1 (MAP_IDENTICAL): Identity map to physical memory (for passthrough)

Sources: configs/vms/arceos-x86_64.toml(L40 - L44)  configs/vms/nimbos-x86_64.toml(L40 - L44) 

Image Loading Methods

AxVisor supports two methods for loading guest images:

flowchart TD
subgraph subGraph1["Memory Loading"]
    mem_loading["Memory Loading"]
    static["Reference include_bytes! data"]
    load_mem["Load to kernel_load_addr"]
end
subgraph subGraph0["File System Loading"]
    fs_loading["File System Loading"]
    mount["Mount disk.img"]
    read_file["Read kernel_path from FS"]
    load_fs["Load to kernel_load_addr"]
end
config["VM Configuration"]
boot["Boot VM"]

config --> fs_loading
config --> mem_loading
fs_loading --> mount
load_fs --> boot
load_mem --> boot
mem_loading --> static
mount --> read_file
read_file --> load_fs
static --> load_mem

File System Loading

When image_location="fs" is specified:

1. AxVisor mounts a FAT32 disk image file (typically disk.img)
2. Reads the kernel image from the specified kernel_path
3. Loads it into memory at kernel_load_addr

Required configuration:

• image_location = "fs"
• kernel_path (path within filesystem)
• kernel_load_addr (memory address for loading)
• Optionally, bios_path and bios_load_addr for x86

Memory Loading

When image_location="memory" is specified:

1. The guest kernel image is statically compiled into the AxVisor binary
2. The image is loaded from memory at runtime

Required configuration:

• image_location = "memory"
• kernel_path (must point to a file in the workspace)
• kernel_load_addr (memory address for loading)
• Optionally, bios_path and bios_load_addr for x86

Sources: README.md(L79 - L112) 

Configuration and Memory Layout Relationships

Each VM's memory layout is determined by both VM-specific configuration and platform-specific memory layouts:

flowchart TD
subgraph subGraph2["VM Memory Layout"]
    vm_memory["VM Physical Memory"]
    bios["BIOS Region"]
    kernel["Kernel Region"]
    device_mmio["Device MMIO Regions"]
end
subgraph subGraph1["VM Configuration"]
    vm_config["VM Config (arceos-x86_64.toml)"]
    memory_regions["Memory Regions"]
    kernel_load_addr["Kernel Load Address"]
    bios_load_addr["BIOS Load Address"]
end
subgraph subGraph0["Platform Configuration"]
    plat_config["Platform Config (x86_64-qemu-q35.toml)"]
    phys_memory["Physical Memory Base/Size"]
    kernel_base["Kernel Base Address"]
    aspace["Kernel Address Space"]
end

bios_load_addr --> bios
kernel_load_addr --> kernel
memory_regions --> vm_memory
phys_memory --> vm_memory
plat_config --> aspace
plat_config --> kernel_base
plat_config --> phys_memory
vm_config --> bios_load_addr
vm_config --> kernel_load_addr
vm_config --> memory_regions

Sources: configs/vms/arceos-x86_64.toml(L40 - L44)  configs/platforms/x86_64-qemu-q35.toml(L13 - L28) 

Using the axvmconfig Tool

For complex VM configurations, AxVisor provides the axvmconfig tool that can generate custom configuration files. To use this tool:

1. Install the tool: cargo install axvmconfig
2. Define your VM configuration using the tool's syntax
3. Generate a TOML configuration file

See the axvmconfig documentation for detailed usage.

Sources: README.md(L72 - L74) 

Example Configuration: ArceOS on x86_64

Here's an example of configuring an ArceOS guest on x86_64:

# Vm base info configs
[base]
id = 1
name = "arceos"
vm_type = 1
cpu_num = 1
phys_cpu_sets = [1]

# Vm kernel configs
[kernel]
entry_point = 0x8000
image_location = "fs"
bios_path = "axvm-bios.bin"
bios_load_addr = 0x8000
kernel_path = "arceos-x86_64.bin"
kernel_load_addr = 0x20_0000

memory_regions = [
    [0x0000_0000, 0x100_0000, 0x7, 0], # Low RAM 16M with RWX permissions
]

# Device specifications
[devices]
emu_devices = []
passthrough_devices = [
    ["IO APIC", 0xfec0_0000, 0xfec0_0000, 0x1000, 0x1],
    ["Local APIC", 0xfee0_0000, 0xfee0_0000, 0x1000, 0x1],
    ["HPET", 0xfed0_0000, 0xfed0_0000, 0x1000, 0x1],
]

Sources: configs/vms/arceos-x86_64.toml(L1 - L78) 

Boot Process with VM Configuration

During the boot process, AxVisor uses the VM configuration to set up and start the VM:

sequenceDiagram
    participant VMConfigFile as "VM Config File"
    participant VMMinit as "VMM::init()"
    participant VMObject as "VM Object"
    participant ImageLoader as "Image Loader"
    participant VCPUSetup as "VCPU Setup"

    VMConfigFile ->> VMMinit: Load configuration
    VMMinit ->> VMObject: Create VM with id, name, cpu_num
    VMObject ->> VMObject: Set up memory regions
    alt image_location = "fs"
        VMObject ->> ImageLoader: Load kernel from filesystem
    else image_location = "memory"
        VMObject ->> ImageLoader: Load kernel from memory
    end
    ImageLoader ->> VMObject: Load at kernel_load_addr
    VMObject ->> VCPUSetup: Create VCPUs
    VMObject ->> VMObject: Configure devices
    VMMinit ->> VMObject: Boot VM
    VMObject ->> VCPUSetup: Start execution at entry_point

Sources: README.md(L57 - L112)  src/vmm/timer.rs(L98 - L104) 

Timer Configuration

The VM timer subsystem is also affected by configuration settings:

flowchart TD
subgraph subGraph0["Timer Configuration"]
    timer_freq["timer-frequency in Platform Config"]
    periodic_interval["PERIODIC_INTERVAL_NANOS"]
    vmm_timer["VMM Timer Events"]
end
timer_events["Timer Events"]
vcpu_execution["VCPU Execution"]

periodic_interval --> vmm_timer
timer_events --> vcpu_execution
timer_freq --> periodic_interval
vmm_timer --> timer_events

Sources: src/vmm/timer.rs(L1 - L104)  configs/platforms/x86_64-qemu-q35.toml(L56 - L57) 

Conclusion

AxVisor's VM configuration system provides a flexible and declarative way to define virtual machine properties. By using TOML configuration files, users can easily customize VM attributes without modifying code. The configuration system supports different image loading methods, memory configurations, and device setups, enabling a wide range of virtualization scenarios.

For platform-specific configuration options that complement VM configurations, see Platform-Specific Configuration.

Platform-Specific Configuration

Relevant source files

• Cargo.toml
• configs/platforms/x86_64-qemu-q35.toml
• configs/vms/arceos-x86_64.toml
• configs/vms/nimbos-x86_64.toml
• scripts/lds/linker.lds.S
• src/vmm/timer.rs

This document describes the architecture-specific configuration options in AxVisor. It covers the configuration files and settings required for each supported hardware platform (x86_64, ARM/aarch64, and RISC-V). For information about virtual machine configuration, see VM Configuration.

Configuration System Overview

AxVisor uses TOML configuration files to define platform-specific settings, which are located in the configs/platforms/ directory. These files contain architecture-dependent parameters such as memory layout, device specifications, and timer frequencies.

flowchart TD
subgraph subGraph2["Runtime System"]
    vm_manager["VM Manager"]
    hal["Hardware Abstraction Layer"]
end
subgraph subGraph1["Build System"]
    axconfig["axconfigBuild-Time Config"]
    build_process["Build Process"]
end
subgraph subGraph0["Configuration Files"]
    platform_configs["Platform Configsconfigs/platforms/*.toml"]
    vm_configs["VM Configsconfigs/vms/*.toml"]
end

axconfig --> build_process
platform_configs --> axconfig
platform_configs --> hal
platform_configs --> vm_configs
vm_configs --> vm_manager

Sources: configs/platforms/x86_64-qemu-q35.toml(L1 - L58)  Cargo.toml(L47 - L52) 

Platform Configuration File Structure

Each platform configuration file follows a standardized format, with sections for platform identification, memory layout, and device specifications.

Example Configuration (x86_64-qemu-q35)

classDiagram
class PlatformConfig {
    arch: string
    platform: string
    plat: PlatformSettings
    devices: DeviceSettings
    
}

class PlatformSettings {
    family: string
    phys-memory-base: usize
    phys-memory-size: usize
    kernel-base-paddr: usize
    kernel-base-vaddr: usize
    phys-virt-offset: usize
    phys-bus-offset: usize
    kernel-aspace-base: usize
    kernel-aspace-size: usize
    
}

class DeviceSettings {
    pci-ecam-base: usize
    pci-bus-end: usize
    timer-frequency: usize
    mmio-regions: [(usize, usize) ]
    virtio-mmio-regions: [(usize, usize) ]
    pci-ranges: [(usize, usize) ]
}

PlatformConfig  -->  PlatformSettings
PlatformConfig  -->  DeviceSettings

Sources: configs/platforms/x86_64-qemu-q35.toml(L1 - L58) 

Memory Layout Configuration

The memory layout configuration is one of the most critical aspects of platform-specific settings. It defines the physical and virtual memory address spaces, kernel loading addresses, and memory mapping offsets.

Memory Layout Parameters

Sources: configs/platforms/x86_64-qemu-q35.toml(L13 - L30)  scripts/lds/linker.lds.S(L1 - L94) 

Device Configuration

Device configuration defines the memory-mapped I/O (MMIO) regions, PCI configuration, and other hardware-specific settings for each platform.

flowchart TD
subgraph subGraph1["Platform-Specific Implementations"]
    x86_impl["x86_64 ImplementationIO APIC, Local APIC, HPET"]
    arm_impl["ARM ImplementationGICv2, ARM-specific timers"]
    riscv_impl["RISC-V ImplementationSBI runtime, PLIC"]
end
subgraph subGraph0["Device Configuration Components"]
    mmio["MMIO RegionsPhysical address regionsfor memory-mapped devices"]
    virtio["VirtIO MMIO RegionsRegions for virtualized I/O devices"]
    pci["PCI ConfigurationECAM base, bus numbers,memory ranges"]
    timer["Timer SettingsFrequency and configuration"]
end

mmio --> arm_impl
mmio --> riscv_impl
mmio --> x86_impl
pci --> x86_impl
timer --> arm_impl
timer --> riscv_impl
timer --> x86_impl
virtio --> arm_impl
virtio --> riscv_impl
virtio --> x86_impl

Sources: configs/platforms/x86_64-qemu-q35.toml(L34 - L58)  configs/vms/arceos-x86_64.toml(L40 - L77)  configs/vms/nimbos-x86_64.toml(L40 - L77) 

MMIO Regions

The mmio-regions parameter defines the memory ranges reserved for memory-mapped devices. For x86_64-qemu-q35, these include:

• PCI config space: 0xb0000000 - 0xc0000000
• PCI devices: 0xfe000000 - 0xfec00000
• IO APIC: 0xfec00000 - 0xfec01000
• HPET: 0xfed00000 - 0xfed01000
• Local APIC: 0xfee00000 - 0xfee01000

Sources: configs/platforms/x86_64-qemu-q35.toml(L36 - L45) 

PCI Configuration

PCI configuration varies by platform. For x86_64-qemu-q35, it includes:

• PCI ECAM base: 0xb0000000
• PCI bus end: 0xff

Sources: configs/platforms/x86_64-qemu-q35.toml(L49 - L54) 

Timer Configuration

Platform-specific timer configuration is essential for proper scheduling and timing in the hypervisor. The configuration specifies the timer frequency and related parameters.

flowchart TD
subgraph subGraph0["Timer Configuration"]
    timer_config["Platform Timer Configtimer-frequency"]
    periodic_interval["Periodic IntervalPERIODIC_INTERVAL_NANOS"]
    scheduler_next["scheduler_next_event()"]
    register_timer["register_timer()"]
    check_events["check_events()"]
end

periodic_interval --> scheduler_next
register_timer --> check_events
scheduler_next --> check_events
timer_config --> periodic_interval

Sources: configs/platforms/x86_64-qemu-q35.toml(L57)  src/vmm/timer.rs(L1 - L105) 

Timer Implementation

The timer subsystem is implemented in src/vmm/timer.rs and uses platform-specific timers through the Hardware Abstraction Layer (HAL). For x86_64-qemu-q35, the timer frequency is set at 4 GHz.

The timer system uses the following key parameters:

• timer-frequency: Base frequency of the platform timer (e.g., 4 GHz for x86_64-qemu-q35)
• PERIODIC_INTERVAL_NANOS: Derived from axconfig::TICKS_PER_SEC and axhal::time::NANOS_PER_SEC

Sources: src/vmm/timer.rs(L12)  configs/platforms/x86_64-qemu-q35.toml(L57) 

Relationship with VM Configuration

Platform-specific configurations form the foundation for VM configurations. VM configurations reference platform settings and may override or extend them for specific virtual machines.

flowchart TD
subgraph subGraph1["VM Config"]
    vm_base["VM Base Infoid, name, cpu_num"]
    vm_kernel["Kernel Configentry_point, image_location,kernel_path, kernel_load_addr"]
    vm_memory["Memory Regionsbase_paddr, size, flags, map_type"]
    vm_devices["Device Specsemu_devices, passthrough_devices"]
end
subgraph subGraph0["Platform Config"]
    plat_arch["arch: x86_64"]
    plat_platform["platform: x86_64-qemu-q35"]
    plat_memory["Memory LayoutPhysical and virtualaddress spaces"]
    plat_devices["Device ConfigurationMMIO regions, PCI, timers"]
end

plat_arch --> vm_base
plat_devices --> vm_devices
plat_memory --> vm_memory
plat_platform --> vm_base

Sources: configs/vms/arceos-x86_64.toml(L1 - L78)  configs/vms/nimbos-x86_64.toml(L1 - L78)  configs/platforms/x86_64-qemu-q35.toml(L1 - L58) 

Memory Region Configuration in VMs

VM configurations specify memory regions that must be compatible with the platform's memory layout. For example, a VM configuration might include:

memory_regions = [
    [0x0000_0000, 0x100_0000, 0x7, 0], # Low RAM 16M 0b111 R|W|EXECUTE
]

This defines a 16MB region starting at physical address 0, with read, write, and execute permissions (flags 0x7), using allocation mapping (map_type 0).

Sources: configs/vms/arceos-x86_64.toml(L42 - L44)  configs/vms/nimbos-x86_64.toml(L42 - L44) 

Device Passthrough in VMs

Platform-specific device configurations determine which devices can be passed through to VMs. The VM configuration specifies which devices to pass through and how to map them:

passthrough_devices = [
    ["IO APIC", 0xfec0_0000, 0xfec0_0000, 0x1000, 0x1],
    ["Local APIC", 0xfee0_0000, 0xfee0_0000, 0x1000, 0x1],
    ["HPET", 0xfed0_0000, 0xfed0_0000, 0x1000, 0x1],
]

Each entry specifies:

1. Device name
2. Base IPA (Intermediate Physical Address) in guest
3. Base physical address in host
4. Length of memory region
5. Interrupt allocation flag

Sources: configs/vms/arceos-x86_64.toml(L55 - L77)  configs/vms/nimbos-x86_64.toml(L55 - L77) 

Architecture-Specific Considerations

Each supported architecture has specific configuration requirements and capabilities.

x86_64 Architecture

For x86_64 platforms, important considerations include:

• APIC configuration (IO APIC and Local APIC)
• HPET timer configuration
• PCI configuration space mapping
• Boot page table setup

Sources: configs/platforms/x86_64-qemu-q35.toml(L1 - L58)  configs/vms/arceos-x86_64.toml(L55 - L77)  configs/vms/nimbos-x86_64.toml(L55 - L77) 

ARM/aarch64 Architecture

While not explicitly shown in the provided files, support for ARM architecture is indicated in the Cargo.toml file with features like "arm-el2". ARM platforms would have specific considerations for:

• GIC (Generic Interrupt Controller) configuration
• ARM-specific timer devices
• EL2 (Hypervisor) mode configuration

Sources: Cargo.toml(L43 - L45) 

RISC-V Architecture

RISC-V support is mentioned in the system overview diagrams. RISC-V platforms would have specific considerations for:

• SBI (Supervisor Binary Interface) runtime configuration
• PLIC (Platform-Level Interrupt Controller) setup
• RISC-V privilege modes

Build Integration

The platform-specific configuration is integrated into the build process through the axconfig build dependency, which processes the configuration files at build time.

flowchart TD
subgraph subGraph1["Configuration Sources"]
    platform_config["Platform Config Filesconfigs/platforms/*.toml"]
    linker_script["Linker Scriptscripts/lds/linker.lds.S"]
end
subgraph subGraph0["Build Process"]
    toml_parser["TOML Parser"]
    axconfig["axconfig Tool"]
    build_system["Build System"]
    final_binary["Final Binary"]
end

axconfig --> build_system
axconfig --> linker_script
build_system --> final_binary
linker_script --> build_system
platform_config --> toml_parser
toml_parser --> axconfig

Sources: Cargo.toml(L47 - L52)  scripts/lds/linker.lds.S(L1 - L94) 

Building and Running

Relevant source files

• .github/workflows/REMOTE-CI.md
• .github/workflows/actions/setup-qemu/action.yml
• .gitignore
• Makefile
• README.md
• doc/GuestVMs.md
• tool/dev_env.py

This document provides detailed instructions on how to build AxVisor from source and run it with different guest virtual machines. For information about VM configuration options, see Configuration.

Prerequisites

Before building and running AxVisor, you need to set up your development environment with the following tools:

• Rust development environment
• cargo-binutils
• QEMU (for testing)
• Optional: musl-gcc (for building guest applications)

$ cargo install cargo-binutils

Sources: README.md(L64 - L68) 

Setting up the Development Environment

Automated Setup

AxVisor provides a Python script that automates the process of cloning dependent repositories and configuring the development environment.

$ python tool/dev_env.py

This script will:

1. Create a crates directory
2. Clone all required repositories
3. Patch your Cargo.toml file
4. Set up VSCode settings (if applicable)

You can specify a different repository URL using the --repo argument:

$ python tool/dev_env.py --repo https://github.com/your-fork/arceos-hypervisor

Sources: tool/dev_env.py(L1 - L106) 

Build System Overview

Build Process Diagram

flowchart TD
subgraph subGraph2["Make Targets"]
    make_build["make build"]
    make_run["make run"]
    make_debug["make debug"]
    make_clean["make clean"]
end
subgraph subGraph1["Configuration Inputs"]
    arch_opt["Architecture Options(x86_64, aarch64, riscv64)"]
    plat_opt["Platform Options"]
    feature_opt["Feature Options"]
    vm_config["VM Configuration Files"]
end
subgraph subGraph0["Build Process"]
    config["Configuration Generation"]
    build["Build AxVisor"]
    output["Output: ELF and Binary Files"]
end
binary["Binary file:APP_NAME_PLATFORM.bin"]
elf["ELF file:APP_NAME_PLATFORM.elf"]
asm["Assembly file:APP_NAME_PLATFORM.asm"]

arch_opt --> config
build --> output
config --> build
feature_opt --> config
make_build --> build
make_debug --> build
make_run --> build
output --> asm
output --> binary
output --> elf
plat_opt --> config
vm_config --> config

Sources: Makefile(L1 - L151)  README.md(L57 - L59) 

Building AxVisor

AxVisor uses a Makefile-based build system with numerous configuration options. The basic build command is:

$ make ARCH=<architecture> VM_CONFIGS=<config_path> build

Key Build Options

Sources: Makefile(L4 - L32)  README.md(L57 - L59) 

Example Build Commands

Build for aarch64 architecture with specific VM configurations:

$ make ARCH=aarch64 VM_CONFIGS=configs/vms/arceos-aarch64.toml build

Build with filesystem support:

$ make ARCH=aarch64 VM_CONFIGS=configs/vms/arceos-aarch64.toml APP_FEATURES=fs build

Sources: README.md(L99 - L112) 

Running AxVisor

After building AxVisor, you can run it using QEMU. AxVisor supports two methods for loading guest VM images:

1. Loading from a filesystem
2. Loading from memory (statically compiled)

Runtime Configuration Diagram

flowchart TD
subgraph subGraph2["Memory Loading"]
    buildImage["Build guest VM image"]
    specifyPath["Specify kernel_path in config"]
end
subgraph subGraph1["Filesystem Loading"]
    prepareDisk["Create disk.img with FAT32"]
    mountDisk["Mount disk.img"]
    copyImage["Copy guest VM image to disk"]
    umountDisk["Unmount disk"]
end
subgraph subGraph0["AxVisor Runtime Configuration"]
    startAxVisor["Start AxVisor"]
    loadConfig["Load VM Configuration Files"]
    checkImageLocation["Check image_location in config"]
    loadFromFs["Load Guest from Filesystem"]
    loadFromMem["Load Guest from Memory"]
    setupVM["Setup VM"]
    runVM["Run VM"]
end

buildImage --> specifyPath
checkImageLocation --> loadFromFs
checkImageLocation --> loadFromMem
copyImage --> umountDisk
loadConfig --> checkImageLocation
loadFromFs --> prepareDisk
loadFromFs --> setupVM
loadFromMem --> buildImage
loadFromMem --> setupVM
mountDisk --> copyImage
prepareDisk --> mountDisk
setupVM --> runVM
startAxVisor --> loadConfig

Sources: README.md(L59 - L112) 

Loading and Running from Filesystem

1. Build a guest VM image for your architecture
2. Create a disk image and place the guest VM image into it:

$ make disk_img               # Creates an empty FAT32 disk image
$ mkdir -p tmp
$ sudo mount disk.img tmp
$ sudo cp /PATH/TO/YOUR/GUEST/VM/IMAGE tmp/
$ sudo umount tmp

1. Modify the VM configuration file in configs/vms/<ARCH_CONFIG>.toml:

• Set image_location="fs"
• Set kernel_path to the path of the kernel image in the filesystem
• Set entry_point to the entry address of the kernel image
• Set kernel_load_addr to the loading address of the kernel image

2. Run AxVisor with the configured guest VM:

$ make ACCEL=n ARCH=aarch64 LOG=info VM_CONFIGS=configs/vms/arceos-aarch64.toml APP_FEATURES=fs run

Sources: README.md(L76 - L99) 

Loading and Running from Memory

1. Build a guest VM image for your architecture
2. Modify the VM configuration file in configs/vms/<ARCH_CONFIG>.toml:

• Set image_location="memory"
• Set kernel_path to the relative/absolute path of the kernel image in the workspace
• Set entry_point to the entry address of the kernel image
• Set kernel_load_addr to the loading address of the kernel image

3. Run AxVisor with the configured guest VM:

$ make ACCEL=n ARCH=aarch64 LOG=info VM_CONFIGS=configs/vms/arceos-aarch64.toml run

Sources: README.md(L101 - L112) 

Guest VM Support

AxVisor has been verified to work with the following guest operating systems:

Sources: README.md(L45 - L56)  doc/GuestVMs.md(L1 - L56) 

Hardware Platform Support

AxVisor has been verified on the following platforms:

• QEMU ARM64 virt (qemu-max)
• Rockchip RK3568 / RK3588
• 黑芝麻华山 A1000 (Black Sesame A1000)

Sources: README.md(L37 - L44) 

Debugging

AxVisor provides several targets for debugging:

1. To build and run the system with debugging support:

$ make ARCH=<architecture> VM_CONFIGS=<config_path> debug

1. To attach GDB to a running instance:

$ make gdb

This will connect GDB to the running QEMU instance and set a breakpoint at the rust_entry function.

Sources: Makefile(L175 - L182) 

Troubleshooting and Development

Common Issues

• Make sure your QEMU version is up-to-date
• For hardware acceleration, ensure KVM is enabled on your system
• VM configuration paths must be correctly specified

CI/CD Environment

The project uses GitHub Actions for continuous integration and testing. The CI setup includes:

• Testing on multiple architectures (x86_64, aarch64, riscv64)
• Setting up QEMU with necessary configurations
• Remote testing for hardware-specific features

For more information about the remote CI setup, see .github/workflows/REMOTE-CI.md

Sources: .github/workflows/actions/setup-qemu/action.yml(L1 - L48)  .github/workflows/REMOTE-CI.md(L1 - L50) 

Advanced Configuration

For advanced configuration needs, including detailed VM settings, passthrough devices, and architecture-specific options, refer to VM Configuration and Platform-Specific Configuration.

Guest VMs

Relevant source files

• .gitignore
• README.md
• doc/GuestVMs.md
• tool/dev_env.py

This page documents the guest operating systems supported by AxVisor, how to configure them, and methods for loading and running guest VM images. For information about setting up the development environment for AxVisor itself, see Development Environment.

Overview of Supported Guest VMs

AxVisor currently supports running the following operating systems as guest VMs:

Sources: README.md(L46 - L55)  doc/GuestVMs.md(L1 - L13) 

Guest VM Architecture

Relationship Between AxVisor and Guest VMs

flowchart TD
subgraph subGraph1["Guest Virtual Machines"]
    vm1["VM: ArceOS"]
    vm2["VM: NimbOS"]
    vm3["VM: Linux"]
    vm4["VM: Starry-OS"]
end
subgraph subGraph0["AxVisor Hypervisor Components"]
    axvm["axvm - VM Management"]
    axvcpu["axvcpu - Virtual CPU"]
    axaddrspace["axaddrspace - Memory Management"]
    axdevice["axdevice - Device Emulation"]
end

axaddrspace --> vm1
axaddrspace --> vm2
axaddrspace --> vm3
axaddrspace --> vm4
axdevice --> vm1
axdevice --> vm2
axdevice --> vm3
axdevice --> vm4
axvcpu --> vm1
axvcpu --> vm2
axvcpu --> vm3
axvcpu --> vm4
axvm --> vm1
axvm --> vm2
axvm --> vm3
axvm --> vm4

Sources: README.md(L30 - L36) 

Guest VM Boot Process

sequenceDiagram
    participant TOMLConfiguration as "TOML Configuration"
    participant VMM as "VMM"
    participant VMInstance as "VM Instance"
    participant VCPU as "VCPU"
    participant GuestOS as "Guest OS"

    TOMLConfiguration ->> VMM: Load VM configuration
    VMM ->> VMInstance: Create VM instance
    VMM ->> VMInstance: Setup memory regions
    alt Load from filesystem
        VMM ->> VMInstance: Load kernel from FAT32 filesystem
    else Load from memory
        VMM ->> VMInstance: Load kernel from statically compiled image
    end
    VMM ->> VCPU: Create and setup VCPUs
    VMM ->> VMInstance: Boot VM
    VMInstance ->> VCPU: Notify primary VCPU
    VCPU ->> GuestOS: Start guest execution
    loop Until VM
    loop shutdown
        VCPU ->> VCPU: Execute guest code
        VCPU ->> VCPU: Handle VM exits
    end
    end

Sources: README.md(L58 - L112) 

Guest Operating Systems in Detail

ArceOS

ArceOS is a modular, arch-agnostic operating system focused on embedded systems, and it's also the foundation of AxVisor itself. As a guest VM, it's primarily used for SMP (Symmetric Multi-Processing) testing.

Key features when running as a guest:

• Supports x86_64, aarch64, and riscv64 architectures
• SMP support for testing multi-core scenarios
• Hypercall support for communication with the hypervisor

Configuration templates are available at:

• configs/vms/arceos-aarch64.toml
• configs/vms/arceos-x86_64.toml
• configs/vms/arceos-riscv64.toml

ArceOS Testcases

• Hypercall Testing: A HelloWorld application is available that tests hypercall functionality.
• Virtio-PCI Device Testing: The PCI branch can be used for testing virtio-pci devices through port I/O.

Sources: doc/GuestVMs.md(L10 - L23) 

NimbOS

NimbOS is a simple real-time operating system designed primarily for single-core testing scenarios. Its simplicity makes it ideal for testing basic hypervisor functionality.

Key features:

• Single-core operation
• Support for x86_64, aarch64, and riscv64 architectures
• Simple design makes it suitable for basic functionality testing

Configuration templates are available at:

• configs/vms/nimbos-aarch64.toml
• configs/vms/nimbos-x86_64.toml
• configs/vms/nimbos-riscv64.toml

Pre-built kernel binary images are available from the [NimbOS releases page](https://github.com/arceos-hypervisor/axvisor/blob/0c9b89a5/NimbOS releases page)

Sources: doc/GuestVMs.md(L3 - L8) 

Starry-OS

Starry-OS is an educational operating system that can be run as a guest VM on AxVisor. While less information is provided about its specific use cases compared to other guest OSes, it's included as a supported guest system.

Sources: README.md(L50) 

Linux

Linux is supported as a guest OS, primarily on the aarch64 architecture with passthrough devices. This allows for running production workloads in a virtualized environment.

Currently, Linux with passthrough devices on aarch64 has been tested in:

• Single-core configuration: Configuration available at configs/vms/linux-qemu-aarch64.toml with device tree at configs/vms/linux-qemu.dts
• SMP configuration: Configuration available at configs/vms/linux-qemu-aarch64-smp2.toml with device tree at configs/vms/linux-qemu-smp2.dts

Special instructions for running Linux on RK3588 boards are also available.

Sources: README.md(L52 - L55)  doc/GuestVMs.md(L30 - L44) 

axvm-bios (Bootloader)

For x86_64 guests, a simple BIOS called axvm-bios is available. This extremely simple BIOS acts as a bootloader for NimbOS and ArceOS on x86_64 platforms.

Pre-built binary is available from the [axvm-bios releases page](https://github.com/arceos-hypervisor/axvisor/blob/0c9b89a5/axvm-bios releases page)

Sources: doc/GuestVMs.md(L24 - L28) 

Guest VM Configuration

Configuration File Structure

VM configurations in AxVisor are managed through TOML files, which specify parameters such as:

• VM ID and name
• Number of virtual CPUs
• Memory regions
• Virtual devices
• Passthrough devices
• Guest image location and loading information

flowchart TD
subgraph subGraph3["VM Configuration File"]
    id["vm_id = 1"]
    name["name = 'arceos'"]
    vcpus["cpu_num = 1"]
    subgraph subGraph1["Image Configuration"]
        passthrough["passthrough_devices"]
        emulated["emulated_devices"]
        img_loc["image_location = 'fs'/'memory'"]
        kernel_path["kernel_path = '/path/to/image'"]
        entry["entry_point = 0x40080000"]
        load_addr["kernel_load_addr = 0x40080000"]
        dtb_path["dtb_path (optional)"]
        mem1["memory_region_1"]
        mem2["memory_region_2"]
        subgraph Devices["Devices"]
            subgraph subGraph0["Memory Regions"]
                passthrough["passthrough_devices"]
                emulated["emulated_devices"]
                img_loc["image_location = 'fs'/'memory'"]
                kernel_path["kernel_path = '/path/to/image'"]
                mem1["memory_region_1"]
                mem2["memory_region_2"]
            end
        end
    end
end

Example configuration templates can be found in the configs/vms/ directory, with specific templates for each supported guest OS and architecture combination.

Sources: README.md(L70 - L74) 

Loading and Running Guest VMs

AxVisor supports two primary methods for loading guest VM images:

Loading from a Filesystem

This method involves loading the guest image from a FAT32 filesystem:

1. Build a client image file for your target architecture
2. Create a disk image file and place the guest machine image into it:

• Generate an empty FAT32 disk image using make disk_img
• Mount the disk image and copy the guest image into it

3. Configure the VM in the TOML file:

• Set image_location = "fs" to indicate loading from the filesystem
• Set kernel_path to the path of the kernel image in the filesystem
• Configure entry_point and kernel_load_addr appropriately

4. Build and run AxVisor with the appropriate configuration

make ACCEL=n ARCH=aarch64 LOG=info VM_CONFIGS=configs/vms/arceos-aarch64.toml APP_FEATURES=fs run

Sources: README.md(L76 - L99) 

Loading from Memory

This method involves statically compiling the guest image into the hypervisor:

1. Build a client image file for your target architecture
2. Configure the VM in the TOML file:

• Set image_location = "memory" to indicate loading from memory
• Set kernel_path to the path of the kernel image in your workspace
• Configure entry_point and kernel_load_addr appropriately

3. Build and run AxVisor with the appropriate configuration

make ACCEL=n ARCH=aarch64 LOG=info VM_CONFIGS=configs/vms/arceos-aarch64.toml run

Sources: README.md(L101 - L112) 

Platform-Specific Guest Support

RK3588 Board Support

AxVisor has been verified on the Rockchip RK3588 platform for running Linux guests. The process involves:

1. Preparing the kernel file (linux-rk3588-aarch64.bin) and DTB file (rk3588.dtb)
2. Configuring the paths in configs/vms/linux-rk3588-aarch64.toml
3. Building the kernel image with make A=(pwd) ARCH=aarch64 VM_CONFIGS=configs/vms/linux-rk3588-aarch64.toml kernel
4. Using RKDevTool to flash the image to the RK3588 board

Sources: doc/GuestVMs.md(L30 - L44)  README.md(L37 - L44) 

Guest VM Testing

For effective testing of guest VMs, several specific test configurations are available:

1. Hypercall Tests: Using ArceOS HelloWorld application to test hypercall functionality
2. PCI Device Tests: Using specific branches of ArceOS to test virtio-pci devices
3. Single-core Tests: Using NimbOS for basic single-core functionality testing
4. SMP Tests: Using ArceOS for multi-core functionality testing

Sources: doc/GuestVMs.md(L16 - L23) 

Development Environment

Relevant source files

• .gitignore
• README.md
• doc/GuestVMs.md
• tool/dev_env.py

This document outlines how to set up a development environment for AxVisor, a unified modular hypervisor based on ArceOS. It covers the necessary tools, dependencies, repository setup, and IDE configuration to effectively contribute to the project. For information about building and running AxVisor with guest VMs, see Building and Running.

Prerequisites

Before setting up the development environment for AxVisor, you need to install the following dependencies:

• Rust Programming Language: The core language used to develop AxVisor
• cargo-binutils: Required for tools like rust-objcopy and rust-objdump
• musl-gcc (Optional): Needed when building certain guest applications

Installing Prerequisites

# Install Rust (if not already installed)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Install cargo-binutils
cargo install cargo-binutils

# Install musl-gcc (optional)
# Download from http://musl.cc/x86_64-linux-musl-cross.tgz if needed

Sources: README.md(L63 - L68) 

Development Environment Setup Script

AxVisor provides a Python script (tool/dev_env.py) to simplify the development environment setup. This script:

1. Clones required repositories
2. Patches Cargo.toml to use local dependencies
3. Creates IDE configuration for VS Code

Development Environment Structure

The diagram below illustrates the structure created by the development environment setup:

flowchart TD
subgraph subGraph0["AxVisor Development Environment"]
    Root["AxVisor Repository"]
    Crates["crates/ Directory"]
    VSCode[".vscode/ Directory"]
    CargoConfig["Cargo.toml"]
    CargoBackup["Cargo.toml.bk"]
    Repo1["arceos"]
    Repo2["axvm"]
    Repo3["axvcpu"]
    Repo4["axaddrspace"]
    Repo5["arm_vcpu"]
    Repo6["axdevice"]
    Repo7["arm_vgic"]
    Repo8["arm_gicv2"]
    Repo9["axdevice_crates"]
    Settings["settings.json"]
end

Crates --> Repo1
Crates --> Repo2
Crates --> Repo3
Crates --> Repo4
Crates --> Repo5
Crates --> Repo6
Crates --> Repo7
Crates --> Repo8
Crates --> Repo9
Root --> CargoBackup
Root --> CargoConfig
Root --> Crates
Root --> VSCode
VSCode --> Settings

Sources: tool/dev_env.py(L1 - L106)  .gitignore(L20 - L21) 

Running the Setup Script

To set up the development environment, run:

python3 tool/dev_env.py

By default, the script clones repositories from git@github.com:arceos-hypervisor. You can specify a different repository base using the --repo parameter:

python3 tool/dev_env.py --repo https://github.com/arceos-hypervisor

Sources: tool/dev_env.py(L7 - L25) 

Dependencies and Repository Structure

The following diagram shows the core dependencies that are cloned during the setup process and how they relate to the main AxVisor repository:

flowchart TD
subgraph Repositories["Repositories"]
    axvisor["axvisor (main)"]
    arceos["arceos(ArceOS unikernel framework)"]
    axvm["axvm(Virtual Machine Management)"]
    axvcpu["axvcpu(Virtual CPU Interface)"]
    axaddrspace["axaddrspace(Address Space Management)"]
    axdevice["axdevice(Device Emulation)"]
    arm_vcpu["arm_vcpu(ARM VCPU Implementation)"]
    arm_vgic["arm_vgic(ARM Virtual GIC)"]
    arm_gicv2["arm_gicv2(ARM GICv2 Support)"]
    axdevice_crates["axdevice_crates(Device Model Base)"]
end

arm_vcpu --> arm_vgic
arm_vgic --> arm_gicv2
axdevice --> axdevice_crates
axvcpu --> arm_vcpu
axvisor --> arceos
axvisor --> axaddrspace
axvisor --> axdevice
axvisor --> axvcpu
axvisor --> axvm

Sources: tool/dev_env.py(L30 - L48) 

Cargo.toml Patching Process

The development environment script modifies the Cargo.toml file to use local paths for dependencies instead of fetching them from GitHub. This allows you to make changes to these dependencies locally and test them immediately.

The patching process works as follows:

flowchart TD
subgraph subGraph0["Patch Example"]
    E["[patch.'Unsupported markdown: linkpath = 'crates/axvm'"]
end
A["Read original Cargo.toml"]
B["Backup to Cargo.toml.bk"]
C["Add patch sections"]
D["Write modified Cargo.toml"]

A --> B
B --> C
C --> D

The script adds patch sections for all dependencies, redirecting them to local paths in the crates/ directory.

Sources: tool/dev_env.py(L52 - L84) 

VS Code Configuration

The development environment script creates a .vscode/settings.json file with configuration for Rust Analyzer:

{
    "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat",
    "rust-analyzer.check.allTargets": false,
    "rust-analyzer.cargo.features": ["fs"],
    "rust-analyzer.cargo.extraEnv": {
        "AX_CONFIG_PATH": "${workspaceFolder}/.axconfig.toml"
    }
}

This configuration:

• Sets the target architecture to aarch64-unknown-none-softfloat
• Enables the fs feature for Cargo
• Sets the AX_CONFIG_PATH environment variable for configuration

Sources: tool/dev_env.py(L86 - L100) 

Development Workflow

Once your development environment is set up, you can start working on AxVisor:

1. Make changes to the code in the main repository or any of the dependencies in the crates/ directory
2. Build and test using the Makefile commands as described in Building and Running
3. Use guest VM configurations from the configs/vms/ directory for testing

Typical Development Cycle

flowchart TD
A["Setup dev environmentwith dev_env.py"]
B["Make code changes"]
C["Build with make"]
D["Test with QEMU"]

A --> B
B --> C
C --> D
D --> B

Editing Dependency Code

With the development environment set up, you can edit code in any of the cloned repositories. Changes to dependencies will be immediately reflected when you build the main project, without needing to publish new versions of those dependencies.

Key directories to work with:

• crates/arceos/: The ArceOS unikernel framework
• crates/axvm/: Virtual machine management
• crates/axvcpu/: Virtual CPU interface
• crates/axaddrspace/: Address space management
• crates/arm_vcpu/: ARM VCPU implementation (when working on ARM support)

Troubleshooting

Common Issues

1. Git clone failures: Ensure you have proper SSH keys set up if using SSH URLs, or use HTTPS URLs with the --repo option.
2. Rust compiler errors: Ensure you have the correct Rust toolchain installed. You may need to run:

rustup target add aarch64-unknown-none-softfloat

3. Build errors after pulling updates: If updates to dependencies cause build errors, try running the setup script again to ensure all dependencies are in sync.

Cleaning the Environment

If you need to reset your development environment:

1. Remove the crates/ directory:

rm -rf crates/

2. Restore the original Cargo.toml:

mv Cargo.toml.bk Cargo.toml

3. Run the setup script again:

python3 tool/dev_env.py

Sources: .gitignore(L20 - L21) 

Next Steps

After setting up your development environment, you can proceed to:

• Learn more about the AxVisor Architecture
• Configure Guest VMs
• Contribute to the project by following the Contributing guidelines

Technical Reference

Relevant source files

• Cargo.lock
• src/task.rs
• src/vmm/config.rs
• src/vmm/mod.rs
• src/vmm/vcpus.rs

This document provides detailed technical information about the internal implementation of AxVisor, the ArceOS hypervisor. It covers core components, execution models, and internal data structures used by the hypervisor. For information about building and running AxVisor, see Building and Running, and for configuration details, see Configuration.

1. VMM Implementation

The Virtual Machine Manager (VMM) is the core component of AxVisor responsible for managing virtual machines throughout their lifecycle. It handles VM initialization, booting, execution, and shutdown.

flowchart TD
subgraph subGraph1["VM States"]
    vminit["VM::new()"]
    vmboot["VM::boot()"]
    vmrun["VM running"]
    vmshutdown["VM::shutdown()"]
end
subgraph subGraph0["VMM Lifecycle"]
    init["VMM::init()"]
    config["config::init_guest_vms()"]
    setup["vcpus::setup_vm_primary_vcpu()"]
    start["VMM::start()"]
    boot["vm.boot() for each VM"]
    notify["vcpus::notify_primary_vcpu()"]
    inc["Increment RUNNING_VM_COUNT"]
    wait["Wait until all VMs stopped"]
    dec["RUNNING_VM_COUNT == 0"]
end

boot --> inc
boot --> notify
config --> setup
init --> config
init --> vminit
notify --> vmrun
start --> boot
start --> vmboot
vmshutdown --> dec
wait --> dec

Sources: src/vmm/mod.rs(L28 - L65) 

1.1 VM Initialization Process

The VMM initializes VMs through the following steps:

1. Loads VM configuration from TOML files via config::init_guest_vms()
2. Creates VM instances using VM::new()
3. Loads VM images according to configuration
4. Sets up the primary VCPU for each VM using vcpus::setup_vm_primary_vcpu()

The RUNNING_VM_COUNT atomic counter tracks active VMs, allowing the VMM to determine when all VMs have stopped.

sequenceDiagram
    participant VMM as VMM
    participant Configuration as Configuration
    participant VirtualMachine as Virtual Machine
    participant VirtualCPU as Virtual CPU

    VMM ->> Configuration: init_guest_vms()
    Configuration ->> Configuration: static_vm_configs()
    Configuration ->> Configuration: AxVMCrateConfig::from_toml()
    Configuration ->> VirtualMachine: VM::new(vm_config)
    Configuration ->> VirtualMachine: load_vm_images()
    VMM ->> VirtualCPU: setup_vm_primary_vcpu()
    VMM ->> VirtualMachine: vm.boot()
    VMM ->> VirtualCPU: notify_primary_vcpu()
    VMM ->> VMM: RUNNING_VM_COUNT.fetch_add(1)
    Note over VMM: Wait for all VMs to stop

Sources: src/vmm/mod.rs(L28 - L65)  src/vmm/config.rs(L25 - L43) 

2. VCPU Management System

The VCPU management system handles the creation, execution, and synchronization of virtual CPUs. Each VCPU runs as a separate task in the ArceOS task scheduler.

2.1 VCPU Task Structure

Each VCPU is associated with a task that contains:

• Reference to the parent VM
• Reference to the VCPU itself
• Stack space for execution (256 KiB)
• Processor affinity settings (if configured)

classDiagram
class TaskExt {
    VMRef vm
    VCpuRef vcpu
    +new(vm, vcpu)
}

class VMVCpus {
    _vm_id: usize
    wait_queue: WaitQueue
    vcpu_task_list: Vec~AxTaskRef~
    running_halting_vcpu_count: AtomicUsize
    +new(vm)
    +add_vcpu_task(vcpu_task)
    +wait()
    +wait_until(condition)
    +notify_one()
    +mark_vcpu_running()
    +mark_vcpu_exiting() -~ bool
}

class VMRef {
    
    
}

class VCpuRef {
    
    
}

class AxTaskRef {
    
    
}

TaskExt "1" --> "1" VMRef : references
TaskExt "1" --> "1" VCpuRef : references
VMVCpus "1" --> "*" AxTaskRef : manages

Sources: src/task.rs(L1 - L19)  src/vmm/vcpus.rs(L27 - L107) 

2.2 VCPU Execution Flow

The VCPU execution flow follows these steps:

1. Task is created for the VCPU with alloc_vcpu_task()
2. VCPU task waits until the VM is in running state
3. VCPU enters the execution loop via vcpu_run()
4. VM executes guest code with vm.run_vcpu(vcpu_id)
5. VCPU handles various exit reasons (hypercalls, interrupts, etc.)
6. If the VM is shutting down, VCPU exits the loop

flowchart TD
subgraph subGraph1["Exit Reasons"]
    hypercall["Hypercall"]
    interrupt["External Interrupt"]
    halt["Halt"]
    cpuup["CPU Up"]
    cpudown["CPU Down"]
    sysdown["System Down"]
end
subgraph subGraph0["VCPU Task Lifecycle"]
    alloc["alloc_vcpu_task()"]
    task["Create task with vcpu_run() entry"]
    wait["Wait for VM running state"]
    mark["mark_vcpu_running()"]
    loop["Enter execution loop"]
    run["vm.run_vcpu()"]
    handle["Handle exit reason"]
    check["Check VM shutdown state"]
    exit["Exit loop"]
    dec["Decrement RUNNING_VM_COUNT if last VCPU"]
end
waitq["Wait on queue"]
create["Create new VCPU task"]
shutdown["vm.shutdown()"]

alloc --> task
check --> exit
check --> loop
cpuup --> create
create --> loop
exit --> dec
halt --> waitq
handle --> check
handle --> cpudown
handle --> cpuup
handle --> halt
handle --> hypercall
handle --> interrupt
handle --> sysdown
loop --> run
mark --> loop
run --> handle
shutdown --> check
sysdown --> shutdown
task --> wait
wait --> mark
waitq --> loop

Sources: src/vmm/vcpus.rs(L169 - L367) 

2.3 Synchronization Mechanism

VCPUs are synchronized using a wait queue system:

• Each VM has a VMVCpus structure containing a wait queue
• VCPUs can wait on the queue using wait() or wait_until()
• Other VCPUs can wake waiting VCPUs using notify_one()
• The primary VCPU is notified when the VM boots

The system also tracks running VCPUs with an atomic counter to determine when all VCPUs in a VM have exited.

sequenceDiagram
    participant VMM as VMM
    participant VirtualMachine as Virtual Machine
    participant PrimaryVCPU as Primary VCPU
    participant SecondaryVCPU as Secondary VCPU
    participant WaitQueue as Wait Queue

    VMM ->> VirtualMachine: boot()
    VMM ->> WaitQueue: notify_primary_vcpu()
    WaitQueue ->> PrimaryVCPU: wake up
    PrimaryVCPU ->> VirtualMachine: run_vcpu()
    PrimaryVCPU ->> SecondaryVCPU: vcpu_on() (CPU Up)
    SecondaryVCPU ->> VirtualMachine: run_vcpu()
    SecondaryVCPU ->> WaitQueue: wait() (Halt)
    PrimaryVCPU ->> WaitQueue: notify_one()
    WaitQueue ->> SecondaryVCPU: wake up
    PrimaryVCPU ->> VirtualMachine: shutdown()
    VirtualMachine ->> VirtualMachine: shutting_down = true
    SecondaryVCPU ->> SecondaryVCPU: check VM state
    SecondaryVCPU ->> VMM: mark_vcpu_exiting()
    PrimaryVCPU ->> PrimaryVCPU: check VM state
    PrimaryVCPU ->> VMM: mark_vcpu_exiting()
    PrimaryVCPU ->> VMM: RUNNING_VM_COUNT.fetch_sub(1)

Sources: src/vmm/vcpus.rs(L110 - L167)  src/vmm/mod.rs(L22 - L65) 

3. Internal Data Structures

The hypervisor uses several key data structures to manage VMs and VCPUs.

3.1 VM and VCPU Types

VM = axvm::AxVM<AxVMHalImpl, AxVCpuHalImpl>
VMRef = axvm::AxVMRef<AxVMHalImpl, AxVCpuHalImpl>
VCpuRef = axvm::AxVCpuRef<AxVCpuHalImpl>

These types abstract the architecture-specific implementations behind common interfaces.

Sources: src/vmm/mod.rs(L16 - L20) 

3.2 Global State Management

Sources: src/vmm/mod.rs(L22 - L25)  src/vmm/vcpus.rs(L23) 

3.3 Task Extension

The TaskExt structure associates an ArceOS task with VM and VCPU references:

pub struct TaskExt {
    pub vm: VMRef,
    pub vcpu: VCpuRef,
}

This allows the task scheduler to properly manage VCPU execution.

Sources: src/task.rs(L6 - L11) 

4. VM Exit Handling

VM exits occur when the guest execution needs to be intercepted by the hypervisor. The hypervisor handles various exit reasons through the VCPU execution loop.

4.1 Exit Reason Processing

The following table shows the main exit reasons and their handling:

stateDiagram-v2
[*] --> Running
Running --> Hypercall : Guest makes hypercall
Hypercall --> Running : Process and continue
Running --> Interrupt : External interrupt
Interrupt --> Running : Handle interrupt
Running --> Halted : Guest halts CPU
Halted --> Running : Notified by other VCPU
Running --> Creating : CPU Up request
Creating --> Running : New VCPU created
Running --> Waiting : CPU Down request
Waiting --> Running : Notified
Running --> Exiting : VM shutting down
Exiting --> [*] : Last VCPU updates RUNNING_VM_COUNT

Sources: src/vmm/vcpus.rs(L290 - L363) 

4.2 VCPU State Transitions

VCPUs transition through the following states during their lifecycle:

1. Free: Initial state when VCPU is created
2. Running: VCPU is executing guest code
3. Halted: VCPU is waiting on a wait queue
4. Exiting: VCPU is exiting due to VM shutdown

The mark_vcpu_running() and mark_vcpu_exiting() functions track these transitions.

Sources: src/vmm/vcpus.rs(L92 - L107)  src/vmm/vcpus.rs(L155 - L167) 

5. Architecture-Specific Components

AxVisor supports multiple CPU architectures through hardware abstraction layers.

5.1 HAL Implementation

The hypervisor uses architecture-specific implementations behind common interfaces:

classDiagram
class AxVMHalImpl {
    <<HAL for VM operations>>
    
    
}

class AxVCpuHalImpl {
    <<HAL for VCPU operations>>
    
    
}

class VM {
    <<axvm::AxVM>>
    
    +boot()
    +shutdown()
    +run_vcpu()
}

class VCpu {
    <<axvm::AxVCpu>>
    
    +run()
    +set_entry()
    +set_gpr()
}

VM  *--  AxVMHalImpl
VCpu  *--  AxVCpuHalImpl

The actual implementations vary by architecture (x86_64, ARM/aarch64, RISC-V) but present a unified interface.

Sources: src/vmm/mod.rs(L12 - L20) 

5.2 Architecture-Specific Behaviors

Some behaviors vary by architecture:

#[cfg(target_arch = "riscv64")]
{
    debug!(
        "vcpu_on: vcpu[{}] entry={:x} opaque={:x}",
        vcpu_id, entry_point, arg
    );
    vcpu.set_gpr(0, vcpu_id);
    vcpu.set_gpr(1, arg);
}

This example shows RISC-V specific register setup for secondary VCPUs.

Sources: src/vmm/vcpus.rs(L193 - L201) 

6. System Configuration

VM configurations are loaded from TOML files and used to initialize VMs.

6.1 Configuration Loading

Static VM configurations are included at compile time and parsed during VM initialization:

pub fn init_guest_vms() {
    let gvm_raw_configs = config::static_vm_configs();

    for raw_cfg_str in gvm_raw_configs {
        let vm_create_config =
            AxVMCrateConfig::from_toml(raw_cfg_str).expect("Failed to resolve VM config");
        let vm_config = AxVMConfig::from(vm_create_config.clone());
        
        // Create VM and load images
        // ...
    }
}

Architecture-specific configurations are selected based on the target architecture.

Sources: src/vmm/config.rs(L10 - L22)  src/vmm/config.rs(L25 - L43) 

6.2 Configuration Structure

VM configurations include:

• Basic properties (ID, name, CPU count)
• Memory regions
• Image location and loading parameters
• Device configurations

For more detailed information on configuration options, see VM Configuration.

VMM Implementation

Relevant source files

• Cargo.lock
• src/task.rs
• src/vmm/config.rs
• src/vmm/mod.rs
• src/vmm/vcpus.rs

This document details the implementation of the Virtual Machine Manager (VMM) in AxVisor. The VMM is the core component responsible for creating, managing, and supervising virtual machines in the hypervisor. For information about specific VCPU management, see VCPU Management.

Overview

The VMM in AxVisor orchestrates the complete lifecycle of virtual machines, from initialization and booting to handling runtime events and shutdown. It provides a unified framework for managing VMs across multiple architectures (x86_64, aarch64, and RISC-V) while abstracting hardware-specific details.

flowchart TD
subgraph subGraph2["VM Tracking"]
    running_vm_count["RUNNING_VM_COUNT"]
    vmm_wait_queue["VMM Wait Queue"]
    decrease_count["Decrease RUNNING_VM_COUNT"]
    wake_vmm["Wake VMM Wait Queue"]
    vmm_exit["VMM Exit"]
end
subgraph subGraph1["Runtime Execution"]
    run_vcpu["vm.run_vcpu()"]
    handle_exit["Handle Exit Reasons"]
    hypercall["Hypercall"]
    ext_interrupt["External Interrupt"]
    halt["Halt"]
    cpu_up["CPU Up"]
    sys_down["System Down"]
end
subgraph subGraph0["VMM Implementation"]
    vmm_init["VMM::init()"]
    vmm_start["VMM::start()"]
    config_init["config::init_guest_vms()"]
    vcpu_setup["vcpus::setup_vm_primary_vcpu()"]
    vcpu_notify["vcpus::notify_primary_vcpu()"]
    vm_new["VM::new()"]
    load_images["load_vm_images()"]
    alloc_vcpu_task["alloc_vcpu_task()"]
    vcpu_run["vcpu_run()"]
end

alloc_vcpu_task --> vcpu_run
config_init --> load_images
config_init --> vm_new
decrease_count --> wake_vmm
handle_exit --> cpu_up
handle_exit --> ext_interrupt
handle_exit --> halt
handle_exit --> hypercall
handle_exit --> sys_down
run_vcpu --> handle_exit
running_vm_count --> vmm_exit
sys_down --> decrease_count
vcpu_notify --> vcpu_run
vcpu_run --> run_vcpu
vcpu_setup --> alloc_vcpu_task
vmm_init --> config_init
vmm_init --> vcpu_setup
vmm_start --> vcpu_notify

Sources: src/vmm/mod.rs(L28 - L65)  src/vmm/vcpus.rs(L268 - L367) 

Key Components

The VMM implementation consists of several interconnected components that work together to manage virtual machines:

Sources: src/vmm/mod.rs(L1 - L7)  src/task.rs(L1 - L19) 

Initialization and Startup Flow

The VMM initialization and startup process follows a specific sequence to properly set up and boot virtual machines.

sequenceDiagram
    participant MainApplication as "Main Application"
    participant VMMModule as "VMM Module"
    participant VMConfiguration as "VM Configuration"
    participant VMInstance as "VM Instance"
    participant VCPU as "VCPU"
    participant TaskSystem as "Task System"

    MainApplication ->> VMMModule: init()
    VMMModule ->> VMConfiguration: init_guest_vms()
    VMConfiguration ->> VMConfiguration: Load VM configs from TOML
    loop For each VM config
        VMConfiguration ->> VMInstance: VM::new(vm_config)
        VMConfiguration ->> VMConfiguration: push_vm(vm)
        VMConfiguration ->> VMConfiguration: load_vm_images(vm)
    end
    VMMModule ->> VMMModule: Setup VCPUs for each VM
    loop For each VM
        VMMModule ->> VCPU: setup_vm_primary_vcpu(vm)
        VCPU ->> TaskSystem: alloc_vcpu_task(vm, vcpu)
        TaskSystem -->> VCPU: Return task reference
    end
    MainApplication ->> VMMModule: start()
    loop For each VM
        VMMModule ->> VMInstance: boot()
        VMMModule ->> VCPU: notify_primary_vcpu(vm.id())
        VMMModule ->> VMMModule: Increment RUNNING_VM_COUNT
    end
    VMMModule ->> VMMModule: Wait for all VMs to stop
    Note over VMMModule: Wait on VMM wait queue until RUNNING_VM_COUNT = 0

Sources: src/vmm/mod.rs(L28 - L65)  src/vmm/config.rs(L25 - L43) 

VMM Initialization

The VMM initialization process begins with the init() function, which performs two main tasks:

1. VM Configuration Loading: The system loads VM configurations from TOML files and creates VM instances for each configuration.
2. VCPU Setup: For each VM, the primary VCPU is set up, and a task is created to run it.

#![allow(unused)]
fn main() {
pub fn init() {
    // Initialize guest VM according to config file.
    config::init_guest_vms();

    // Setup vcpus, spawn axtask for primary VCpu.
    info!("Setting up vcpus...");
    for vm in vm_list::get_vm_list() {
        vcpus::setup_vm_primary_vcpu(vm);
    }
}
}

Sources: src/vmm/mod.rs(L28 - L39) 

VMM Startup

After initialization, the start() function boots all VMs and waits for them to complete:

1. VM Boot: Each VM is booted, and its primary VCPU is notified to start execution.
2. VM Tracking: The system tracks the number of running VMs using the RUNNING_VM_COUNT atomic counter.
3. Wait for Completion: The VMM waits on a wait queue until all VMs have stopped (RUNNING_VM_COUNT reaches 0).

pub fn start() {
    info!("VMM starting, booting VMs...");
    for vm in vm_list::get_vm_list() {
        match vm.boot() {
            Ok(_) => {
                vcpus::notify_primary_vcpu(vm.id());
                RUNNING_VM_COUNT.fetch_add(1, Ordering::Release);
                info!("VM[{}] boot success", vm.id())
            }
            Err(err) => warn!("VM[{}] boot failed, error {:?}", vm.id(), err),
        }
    }

    // Do not exit until all VMs are stopped.
    task::ax_wait_queue_wait_until(
        &VMM,
        || {
            let vm_count = RUNNING_VM_COUNT.load(Ordering::Acquire);
            info!("a VM exited, current running VM count: {}", vm_count);
            vm_count == 0
        },
        None,
    );
}

Sources: src/vmm/mod.rs(L42 - L65) 

VCPU Task Management

The VCPU management system is a critical component of the VMM, responsible for setting up, scheduling, and managing VCPU tasks.

VCPU Task Structure

Each VCPU runs in its own task, managed by the task system. The task extensions (TaskExt) store references to both the VM and VCPU, allowing easy access to these components from within the task context.

flowchart TD
subgraph subGraph2["VM Access"]
    vm["VM"]
    vcpu["VCPU"]
end
subgraph subGraph1["VCPU Management"]
    vm_vcpus["VMVCpus"]
    wait_queue["WaitQueue"]
    counter["AtomicUsize"]
end
subgraph subGraph0["Task System"]
    task["AxTaskRef"]
    task_ext["TaskExt"]
    vm_ref["VMRef"]
    vcpu_ref["VCpuRef"]
end

task --> task_ext
task_ext --> vcpu_ref
task_ext --> vm_ref
vcpu_ref --> vcpu
vm --> vcpu
vm_ref --> vm
vm_vcpus --> counter
vm_vcpus --> task
vm_vcpus --> wait_queue

Sources: src/task.rs(L1 - L19)  src/vmm/vcpus.rs(L27 - L40) 

VCPU Task Creation

The alloc_vcpu_task function creates a new task for a VCPU and initializes it with the appropriate VM and VCPU references:

1. Create a new task with the vcpu_run function as its entry point
2. Set the CPU mask if the VCPU has a dedicated physical CPU
3. Initialize the task extension with VM and VCPU references
4. Spawn the task on the scheduler

fn alloc_vcpu_task(vm: VMRef, vcpu: VCpuRef) -> AxTaskRef {
    let mut vcpu_task = TaskInner::new(
        vcpu_run,
        format!("VM[{}]-VCpu[{}]", vm.id(), vcpu.id()),
        KERNEL_STACK_SIZE,
    );

    if let Some(phys_cpu_set) = vcpu.phys_cpu_set() {
        vcpu_task.set_cpumask(AxCpuMask::from_raw_bits(phys_cpu_set));
    }
    vcpu_task.init_task_ext(TaskExt::new(vm, vcpu));

    axtask::spawn_task(vcpu_task)
}

Sources: src/vmm/vcpus.rs(L249 - L268) 

VCPU Execution Flow

The execution of a VCPU is managed by the vcpu_run function, which runs in the context of the VCPU task.

flowchart TD
start["vcpu_run() Start"]
wait["Wait for VM Running State"]
mark["Mark VCPU as Running"]
run_loop["Enter VCPU Run Loop"]
run_vcpu["vm.run_vcpu(vcpu_id)"]
exit_handler["Handle Exit Reason"]
handle_hypercall["Process Hypercall"]
handle_interrupt["Process Interrupt"]
handle_halt["Wait on VM Wait Queue"]
handle_cpu_up["Boot Secondary VCPU"]
handle_system_down["Shutdown VM"]
check_shutdown["Check VM Shutting Down"]
mark_exiting["Mark VCPU as Exiting"]
decrease_count["Decrease RUNNING_VM_COUNT"]
exit["Exit VCPU Task"]

check_shutdown --> mark_exiting
check_shutdown --> run_loop
exit_handler --> handle_cpu_up
exit_handler --> handle_halt
exit_handler --> handle_hypercall
exit_handler --> handle_interrupt
exit_handler --> handle_system_down
handle_cpu_up --> check_shutdown
handle_halt --> check_shutdown
handle_hypercall --> check_shutdown
handle_interrupt --> check_shutdown
handle_system_down --> check_shutdown
mark --> run_loop
mark_exiting --> decrease_count
mark_exiting --> exit
run_loop --> run_vcpu
run_vcpu --> exit_handler
start --> wait
wait --> mark

Sources: src/vmm/vcpus.rs(L275 - L367) 

VCPU Execution Loop

The VCPU execution loop performs the following steps:

1. Wait for the VM to be in a running state
2. Mark the VCPU as running
3. Enter the main execution loop:

• Run the VCPU
• Handle various exit reasons
• Check if the VM is shutting down

4. If the VM is shutting down:

• Mark the VCPU as exiting
• If this is the last VCPU to exit, decrement the running VM count
• Exit the VCPU task

fn vcpu_run() {
    let curr = axtask::current();
    let vm = curr.task_ext().vm.clone();
    let vcpu = curr.task_ext().vcpu.clone();
    // ... [Initialization code]
    
    // Wait for VM to be in running state
    wait_for(vm_id, || vm.running());
    
    // Mark VCPU as running
    mark_vcpu_running(vm_id);
    
    // Main execution loop
    loop {
        match vm.run_vcpu(vcpu_id) {
            Ok(exit_reason) => match exit_reason {
                // ... [Handle various exit reasons]
            },
            Err(err) => {
                // ... [Handle error]
            }
        }
        
        // Check if VM is shutting down
        if vm.shutting_down() {
            // ... [Handle shutdown]
            if mark_vcpu_exiting(vm_id) {
                // Last VCPU exiting
                super::RUNNING_VM_COUNT.fetch_sub(1, Ordering::Release);
                ax_wait_queue_wake(&super::VMM, 1);
            }
            break;
        }
    }
}

Sources: src/vmm/vcpus.rs(L275 - L367) 

VM and VCPU State Management

The VMM maintains the state of VMs and VCPUs through a combination of atomic counters and wait queues.

VM State Tracking

The VMM tracks running VMs using the RUNNING_VM_COUNT atomic counter. When all VMs have stopped (RUNNING_VM_COUNT reaches 0), the VMM is signaled to exit.

static RUNNING_VM_COUNT: AtomicUsize = AtomicUsize::new(0);
static VMM: AxWaitQueueHandle = AxWaitQueueHandle::new();

Sources: src/vmm/mod.rs(L22 - L25) 

VCPU State Management

The VCPU state management involves:

1. Wait Queues: Each VM has a wait queue to manage its VCPUs
2. VCPU Task List: Each VM maintains a list of VCPU tasks
3. Running/Halting Count: The system tracks the number of running or halting VCPUs for each VM

pub struct VMVCpus {
    _vm_id: usize,
    wait_queue: WaitQueue,
    vcpu_task_list: Vec<AxTaskRef>,
    running_halting_vcpu_count: AtomicUsize,
}

Sources: src/vmm/vcpus.rs(L27 - L40) 

Secondary VCPU Management

While primary VCPUs are set up during VMM initialization, secondary VCPUs can be dynamically started by the guest OS through the CPU Up exit reason.

When a running VCPU requests to start a secondary VCPU, the system:

1. Sets up the entry point and arguments for the new VCPU
2. Creates a new VCPU task
3. Adds the task to the VM's VCPU task list

fn vcpu_on(vm: VMRef, vcpu_id: usize, entry_point: GuestPhysAddr, arg: usize) {
    let vcpu = vm.vcpu_list()[vcpu_id].clone();
    
    // Setup VCPU entry point and arguments
    vcpu.set_entry(entry_point).expect("vcpu_on: set_entry failed");
    vcpu.set_gpr(0, arg);
    
    // Create and add VCPU task
    let vcpu_task = alloc_vcpu_task(vm.clone(), vcpu);
    unsafe { VM_VCPU_TASK_WAIT_QUEUE.get_mut(&vm.id()) }
        .unwrap()
        .add_vcpu_task(vcpu_task);
}

Sources: src/vmm/vcpus.rs(L179 - L208) 

VM Shutdown Process

When a VM shuts down, the following steps occur:

1. The VM is marked as shutting down
2. Each VCPU detects the shutdown state in its execution loop
3. The last VCPU to exit decrements the RUNNING_VM_COUNT
4. When RUNNING_VM_COUNT reaches 0, the VMM wait queue is signaled
5. The VMM exits its wait loop and the hypervisor can terminate

This coordinated shutdown ensures proper cleanup of resources and allows the hypervisor to exit cleanly.

sequenceDiagram
    participant GuestOS as "Guest OS"
    participant VCPUTask as "VCPU Task"
    participant VMInstance as "VM Instance"
    participant VMMModule as "VMM Module"

    GuestOS ->> VCPUTask: System Down Exit
    VCPUTask ->> VMInstance: shutdown()
    VMInstance -->> VCPUTask: VM marked as shutting down
    loop For each VCPU
        VCPUTask ->> VCPUTask: Detect VM shutting down
        VCPUTask ->> VCPUTask: Mark VCPU as exiting
        VCPUTask ->> VCPUTask: Check if last VCPU
    end
    VCPUTask ->> VMMModule: Decrement RUNNING_VM_COUNT
    VCPUTask ->> VMMModule: Wake VMM wait queue
    VCPUTask ->> VCPUTask: Exit VCPU task
    VMMModule ->> VMMModule: Check if RUNNING_VM_COUNT = 0
    VMMModule ->> VMMModule: Exit VMM

Sources: src/vmm/vcpus.rs(L345 - L363)  src/vmm/mod.rs(L55 - L64) 

Summary

The VMM implementation in AxVisor provides a robust foundation for managing virtual machines across multiple architectures. Key aspects of this implementation include:

1. Centralized VM Management: The VMM initializes, boots, and tracks all VMs in the system.
2. VCPU Task System: Each VCPU runs in its own task, managed by the task scheduler.
3. Coordinated Shutdown: The system ensures proper cleanup and coordination during VM shutdown.
4. State Tracking: The VMM maintains clear state tracking for VMs and VCPUs.
5. Secondary VCPU Support: The system supports dynamically starting secondary VCPUs.

This implementation enables AxVisor to efficiently manage multiple guest VMs while providing a clean architecture for future extensions.

VCPU Management

Relevant source files

• Cargo.lock
• src/task.rs
• src/vmm/config.rs
• src/vmm/mod.rs
• src/vmm/vcpus.rs

This page documents the VCPU (Virtual CPU) management system in AxVisor, focusing on how virtual CPUs are implemented, initialized, scheduled, and monitored throughout their lifecycle. For information about the overall VM management, see VM Management. For timer-related aspects of VCPUs, see Timer Subsystem.

Overview

AxVisor implements a task-based VCPU execution model, where each VCPU runs as a separate task in the ArceOS task scheduler. This design allows multiple VCPUs to be efficiently managed across physical CPUs, with mechanisms for coordination, notification, and state tracking.

flowchart TD
subgraph subGraph1["External Components"]
    axvm["axvm - VM Management"]
    axtask["axtask - Task Scheduler"]
    arch_impl["Architecture-Specific Implementations"]
end
subgraph subGraph0["VCPU Management System"]
    vcpu_management["VCPU Management"]
    vcpu_task_alloc["VCPU Task Allocation"]
    vcpu_initialization["VCPU Initialization"]
    vcpu_execution["VCPU Execution Loop"]
    vcpu_exit["VCPU Exit Handling"]
    vcpu_notification["VCPU Notification System"]
    vm_shutdown["VM Shutdown Coordination"]
end

arch_impl --> vcpu_execution
axtask --> vcpu_execution
axtask --> vcpu_task_alloc
axvm --> vcpu_management
vcpu_management --> vcpu_execution
vcpu_management --> vcpu_exit
vcpu_management --> vcpu_initialization
vcpu_management --> vcpu_notification
vcpu_management --> vcpu_task_alloc
vcpu_management --> vm_shutdown

Sources: src/vmm/vcpus.rs(L1 - L368)  src/vmm/mod.rs(L1 - L66)  src/task.rs(L1 - L19) 

VCPU Architecture and Implementation

AxVisor's VCPU management is based on a cross-architecture design that abstracts hardware-specific details. The system includes platform-specific VCPU implementations for x86_64, ARM, and RISC-V architectures, all conforming to a common interface defined by axvcpu.

VCPU Type Hierarchy

Sources: src/vmm/mod.rs(L15 - L20)  src/vmm/vcpus.rs(L175 - L201) 

VCPU and Task Relationship

Each VCPU in AxVisor is associated with an ArceOS task, which is scheduled by the underlying task scheduler. This relationship is established through the TaskExt structure that extends ArceOS tasks with VCPU-specific information.

Source: src/task.rs(L1 - L19) 

VCPU Lifecycle

VCPU Initialization

VCPUs are initialized during the VMM startup process. The primary VCPU (VCPU 0) for each VM is set up first, while secondary VCPUs can be started on demand.

sequenceDiagram
    participant VMMinit as "VMM::init"
    participant VM as "VM"
    participant VMVCpus as "VMVCpus"
    participant PrimaryVCPU as "Primary VCPU"
    participant VCPUTask as "VCPU Task"

    VMMinit ->> VM: init_guest_vms()
    VMMinit ->> VM: setup_vm_primary_vcpu(vm)
    VM ->> VMVCpus: new(vm)
    VM ->> PrimaryVCPU: Get primary VCPU (id=0)
    VM ->> VCPUTask: alloc_vcpu_task(vm, primary_vcpu)
    VCPUTask ->> VMVCpus: add_vcpu_task(primary_vcpu_task)
    VMMinit ->> VM: start()
    VMMinit ->> VM: boot()
    VMMinit ->> PrimaryVCPU: notify_primary_vcpu(vm_id)

Sources: src/vmm/mod.rs(L27 - L39)  src/vmm/vcpus.rs(L211 - L231) 

VCPU Execution Loop

The main VCPU execution loop is implemented in the vcpu_run function, which is the entry point for all VCPU tasks. The VCPU waits for the VM to be in a running state, then enters a loop where it runs the VCPU and handles various exit reasons.

flowchart TD
start["VCPU Task Start"]
wait_vm["Wait for VM running"]
mark_running["Mark VCPU running"]
run_vcpu["Run VCPU (vm.run_vcpu)"]
handle_exit["Handle exit reason"]
check_shutdown["VM shutting down?"]
last_vcpu["Last VCPU exiting?"]
dec_count["Decrease running VM count"]
wake_vmm["Wake VMM wait queue"]
exit["Exit VCPU task"]
wait_wakeup["Wait for notification"]
cpu_up["Start target VCPU"]
vm_shutdown["Shutdown VM"]

check_shutdown --> last_vcpu
check_shutdown --> run_vcpu
cpu_up --> run_vcpu
dec_count --> wake_vmm
handle_exit --> check_shutdown
handle_exit --> cpu_up
handle_exit --> run_vcpu
handle_exit --> vm_shutdown
handle_exit --> wait_wakeup
last_vcpu --> dec_count
last_vcpu --> exit
mark_running --> run_vcpu
run_vcpu --> handle_exit
start --> wait_vm
vm_shutdown --> check_shutdown
wait_vm --> mark_running
wait_wakeup --> run_vcpu
wake_vmm --> exit

Sources: src/vmm/vcpus.rs(L270 - L367) 

VCPU Task Management

Task Allocation

Each VCPU runs as a separate ArceOS task, allocated with a dedicated kernel stack. The alloc_vcpu_task function creates a task for a VCPU, initializes it with VCPU-specific information, and configures its CPU affinity if specified.

flowchart TD
subgraph subGraph1["Task Properties"]
    task_name["Task name: VM[id]-VCpu[id]"]
    entry_fn["Entry function: vcpu_run"]
    stack_size["Stack size: 256 KiB"]
    cpu_mask["Optional CPU mask"]
end
subgraph subGraph0["VCPU Task Allocation"]
    alloc_vcpu_task["alloc_vcpu_task()"]
    new_task["TaskInner::new()"]
    set_cpumask["Set CPU affinity mask"]
    init_task_ext["Initialize TaskExt"]
    spawn_task["Spawn task in scheduler"]
end

alloc_vcpu_task --> new_task
init_task_ext --> spawn_task
new_task --> entry_fn
new_task --> set_cpumask
new_task --> stack_size
new_task --> task_name
set_cpumask --> cpu_mask
set_cpumask --> init_task_ext

Sources: src/vmm/vcpus.rs(L232 - L268) 

Task Structure: VMVCpus

The VMVCpus structure maintains the state of all VCPUs for a specific VM, including:

Sources: src/vmm/vcpus.rs(L25 - L40) 

VCPU Exit Handling

When a VCPU exits from guest mode, the exit reason is handled in the vcpu_run function. The system supports various exit reasons, including:

Sources: src/vmm/vcpus.rs(L290 - L343) 

Multi-VCPU Coordination

Primary and Secondary VCPUs

AxVisor distinguishes between the primary VCPU (typically VCPU 0) and secondary VCPUs. The primary VCPU is initialized during VM setup, while secondary VCPUs can be started on demand through the CpuUp exit reason.

sequenceDiagram
    participant PrimaryVCPU as "Primary VCPU"
    participant VMVCpus as "VMVCpus"
    participant SecondaryVCPU as "Secondary VCPU"

    PrimaryVCPU ->> PrimaryVCPU: Executes guest code
    Note over PrimaryVCPU: Guest OS executes CPU_UP operation
    PrimaryVCPU ->> PrimaryVCPU: Exits with CpuUp reason
    PrimaryVCPU ->> VMVCpus: vcpu_on(vm, target_cpu, entry, arg)
    VMVCpus ->> SecondaryVCPU: Create secondary VCPU task
    VMVCpus ->> SecondaryVCPU: Set entry point and arguments
    VMVCpus ->> VMVCpus: add_vcpu_task(secondary_task)
    SecondaryVCPU ->> SecondaryVCPU: Start execution (vcpu_run)
    SecondaryVCPU ->> SecondaryVCPU: Wait for VM running
    PrimaryVCPU ->> PrimaryVCPU: Continue execution

Sources: src/vmm/vcpus.rs(L179 - L208)  src/vmm/vcpus.rs(L319 - L330) 

VM Shutdown Coordination

When a VM is shutting down, all VCPUs must exit properly. The last VCPU to exit is responsible for notifying the VMM that the VM has fully shut down.

flowchart TD
vcpu_exit["VCPU detects VM is shutting down"]
mark_exiting["Mark VCPU as exiting"]
check_last["Is this the last VCPU?"]
dec_count["Decrease running VM count"]
wake_vmm["Wake VMM wait queue"]
exit_vcpu["Exit VCPU task"]

check_last --> dec_count
check_last --> exit_vcpu
dec_count --> wake_vmm
mark_exiting --> check_last
vcpu_exit --> mark_exiting
wake_vmm --> exit_vcpu

Sources: src/vmm/vcpus.rs(L345 - L366)  src/vmm/mod.rs(L55 - L64) 

Wait Queue System

AxVisor uses a wait queue system to coordinate VCPU execution. The WaitQueue in VMVCpus allows VCPUs to block when they are halted and to be woken up when needed.

Sources: src/vmm/vcpus.rs(L72 - L89)  src/vmm/vcpus.rs(L117 - L138) 

Cross-Architecture VCPU Implementation

AxVisor supports multiple architectures through architecture-specific VCPU implementations. Each platform has its own VCPU implementation that conforms to the axvcpu interface.

Sources: src/vmm/mod.rs(L15 - L20)  src/vmm/vcpus.rs(L193 - L201) 

Timer Subsystem

Relevant source files

• Cargo.lock
• configs/platforms/x86_64-qemu-q35.toml
• configs/vms/arceos-x86_64.toml
• configs/vms/nimbos-x86_64.toml
• scripts/lds/linker.lds.S
• src/vmm/timer.rs

The Timer Subsystem in AxVisor provides time-based event scheduling and management for the hypervisor environment. This document describes the internal design and implementation of the timer framework that enables scheduling and execution of time-bound operations across the hypervisor.

For information about VCPU Management and how it interacts with timers, see VCPU Management.

Overview

The Timer Subsystem enables the hypervisor to schedule operations to be executed at specific times or after specific intervals. It manages a list of pending timer events per virtual CPU and provides an interface for registering, canceling, and processing timer events.

flowchart TD
subgraph subGraph2["Guest VMs"]
    VMTimer["VM Timer Devices"]
end
subgraph subGraph1["Hardware Layer"]
    HPET["HPET"]
    LAPIC["Local APIC Timer"]
end
subgraph subGraph0["Timer Subsystem Components"]
    Register["register_timer()"]
    Cancel["cancel_timer()"]
    CheckEvents["check_events()"]
    ScheduleNext["scheduler_next_event()"]
    InitPercpu["init_percpu()"]
    TimerList["TIMER_LIST(Per-CPU TimerList)"]
    VmmEvent["VmmTimerEvent(Token + Callback)"]
    HWTimer["Hardware Timer"]
end

Cancel --> TimerList
CheckEvents --> TimerList
HWTimer --> HPET
HWTimer --> LAPIC
InitPercpu --> TimerList
Register --> TimerList
ScheduleNext --> HWTimer
TimerList --> VmmEvent
VMTimer --> HPET

Sources: src/vmm/timer.rs(L1 - L105)  configs/vms/arceos-x86_64.toml(L57 - L77)  configs/platforms/x86_64-qemu-q35.toml(L56 - L57) 

Core Components

VmmTimerEvent Structure

The Timer Subsystem is built around the VmmTimerEvent struct, which encapsulates a timer event with:

1. A unique token identifier
2. A callback function to be executed when the timer expires

classDiagram
class VmmTimerEvent {
    +usize token
    +BoxUnsupported markdown: del timer_callback
    +new(token: usize, f: F) VmmTimerEvent
}

class TimerEvent {
    <<trait>>
    
    +callback(self, now: TimeValue)
}

VmmTimerEvent  ..|>  TimerEvent : implements

Sources: src/vmm/timer.rs(L15 - L41) 

The VmmTimerEvent implements the TimerEvent trait, which defines a callback method that will be invoked when the timer expires. Each timer event is uniquely identified by a token, which allows for cancellation or modification of pending timers.

Per-CPU Timer Lists

The Timer Subsystem maintains a separate timer list for each virtual CPU using the percpu crate. This approach:

1. Avoids contention between CPUs for timer management
2. Allows each CPU to handle its local timer events efficiently
3. Ensures scalability as the number of CPUs increases

#[percpu::def_percpu]
static TIMER_LIST: LazyInit<SpinNoIrq<TimerList<VmmTimerEvent>>> = LazyInit::new();

Sources: src/vmm/timer.rs(L43 - L44) 

The TIMER_LIST is protected by a spinlock (SpinNoIrq) to prevent concurrent modifications from different contexts on the same CPU. The LazyInit wrapper ensures the timer list is initialized only when first accessed.

Timer Operations

Timer Registration

Timers are registered using the register_timer function, which:

1. Accepts a deadline timestamp in nanoseconds
2. Takes a callback function to execute when the deadline is reached
3. Returns a unique token that can be used to cancel the timer

#![allow(unused)]
fn main() {
pub fn register_timer<F>(deadline: u64, handler: F) -> usize
where
    F: FnOnce(TimeValue) + Send + 'static
}

Sources: src/vmm/timer.rs(L54 - L64) 

Timer Cancellation

A previously registered timer can be canceled before its expiration using the cancel_timer function:

#![allow(unused)]
fn main() {
pub fn cancel_timer(token: usize)
}

The function removes the timer with the matching token from the timer list.

Sources: src/vmm/timer.rs(L70 - L74) 

Event Processing

The Timer Subsystem periodically checks for expired timers using the check_events function. This function:

1. Obtains the current time
2. Retrieves and executes all expired timer events
3. Continues until no more expired events remain

Timer event processing happens in the context of the caller, which means callback functions are executed synchronously.

Sources: src/vmm/timer.rs(L77 - L89) 

Scheduling the Next Timer

The scheduler_next_event function sets up the next hardware timer interrupt based on a periodic interval:

pub fn scheduler_next_event() {
    let now_ns = axhal::time::monotonic_time_nanos();
    let deadline = now_ns + PERIODIC_INTERVAL_NANOS;
    axhal::time::set_oneshot_timer(deadline);
}

The interval is calculated from the system configuration (axconfig::TICKS_PER_SEC).

Sources: src/vmm/timer.rs(L92 - L97)  src/vmm/timer.rs(L12) 

Hardware Integration

The Timer Subsystem abstracts the underlying hardware timer mechanisms through the ArceOS hardware abstraction layer (axhal).

Timer Hardware

AxVisor supports several hardware timer mechanisms:

Sources: configs/vms/arceos-x86_64.toml(L72 - L77)  configs/vms/nimbos-x86_64.toml(L72 - L77) 

Timer Frequency Configuration

The platform configuration specifies the timer frequency, which affects the precision and granularity of timer events:

# Timer interrupt frequency in Hz. (4.0GHz)
timer-frequency = 4_000_000_000     # uint

This high frequency enables precise timing for hypervisor operations.

Sources: configs/platforms/x86_64-qemu-q35.toml(L56 - L57) 

Initialization and Setup

The Timer Subsystem requires initialization on each CPU using the init_percpu function:

pub fn init_percpu() {
    info!("Initing HV Timer...");
    let timer_list = unsafe { TIMER_LIST.current_ref_mut_raw() };
    timer_list.init_once(SpinNoIrq::new(TimerList::new()));
}

This function initializes the per-CPU timer list and prepares it for use.

Sources: src/vmm/timer.rs(L99 - L104) 

Usage in AxVisor

The Timer Subsystem is used throughout AxVisor for various purposes:

flowchart TD
subgraph subGraph1["Timer Subsystem"]
    TimerReg["register_timer()"]
    TimerCancel["cancel_timer()"]
    TimerCheck["check_events()"]
end
subgraph subGraph0["Timer Use Cases"]
    VCPU["VCPU Scheduling"]
    VMEvents["VM Event Processing"]
    Watchdog["Watchdog Timers"]
    Preemption["Preemption Points"]
end

Preemption --> TimerReg
VCPU --> TimerCheck
VCPU --> TimerReg
VMEvents --> TimerReg
Watchdog --> TimerCancel
Watchdog --> TimerReg

Sources: src/vmm/timer.rs(L1 - L105) 

Examples

Below is a simplified example of how a timer might be used in the hypervisor:

1. Register a timer to check VM progress after 100ms:

let deadline = current_time_ns() + 100_000_000; // 100ms in the future
let token = register_timer(deadline, |_| {
    // Check VM progress and take action if needed
    check_vm_progress();
});

2. Cancel the timer if the VM makes progress before the deadline:

if vm_made_progress {
    cancel_timer(token);
}

3. Process any expired timers:

// Called periodically from the hypervisor main loop
check_events();

Summary

The Timer Subsystem provides a fundamental service for the AxVisor hypervisor, enabling time-based operations and scheduling. Its per-CPU design ensures scalability, while the simple API makes it easy to use throughout the hypervisor codebase.

The timer framework successfully abstracts the underlying hardware timer mechanisms, allowing AxVisor to provide consistent timing services across different hardware platforms and architectures.

Contributing

Relevant source files

• .github/workflows/REMOTE-CI.md
• .github/workflows/actions/setup-nimbos-guest-image/action.yml
• .github/workflows/actions/setup-qemu/action.yml
• .github/workflows/test.yml
• Makefile

This document provides information and guidelines for contributing to the AxVisor project. It covers setting up a development environment, building and testing your changes, and understanding the CI/CD pipeline. For information about specific technical implementations, please refer to the Technical Reference section.

Development Environment Setup

To begin contributing to AxVisor, you'll need to set up a proper development environment with the necessary tools and dependencies.

Prerequisites

• Rust toolchain (nightly version, specified in rust-toolchain.toml)
• QEMU (version 8.2.0 or later recommended)
• Required system packages (for Linux-based systems)
• Git

Setting Up Your Environment

flowchart TD
A["Install Rust"]
B["Install Dependencies"]
C["Clone Repository"]
D["Setup QEMU"]
E["Prepare Test Images"]
F["Ready for Development"]

A --> B
B --> C
C --> D
D --> E
E --> F

1. Install Rust:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

2. Install Dependencies:

sudo apt-get update && sudo apt-get install -y ninja-build libslirp-dev glib-2.0

3. Clone Repository:

git clone https://github.com/arceos-hypervisor/axvisor.git
cd axvisor

4. Setup QEMU (if not using the system version):

# Download and build QEMU
wget https://download.qemu.org/qemu-8.2.0.tar.xz
tar -xJf qemu-8.2.0.tar.xz
cd qemu-8.2.0
./configure --target-list=x86_64-softmmu,riscv64-softmmu,aarch64-softmmu --enable-slirp
make -j
make install

5. Prepare Test Images:

# Create disk image for testing
make DISK_IMG=disk.img disk_img

Sources: Makefile(L14 - L34)  .github/workflows/actions/setup-qemu/action.yml(L1 - L48) 

Building and Running

AxVisor uses a Makefile system adapted from ArceOS to manage the build process, with a variety of options for different architectures and configurations.

Build Options

Here are the key parameters you can configure when building AxVisor:

Basic Build Commands

flowchart TD
A["make defconfig"]
B["make build"]
C["make run"]
D["make debug"]
E["make test"]

A --> B
B --> C
B --> D
B --> E

1. Generate default configuration:

make ARCH=x86_64 defconfig

2. Build the project:

make ARCH=x86_64 build

3. Run in QEMU:

make ARCH=x86_64 DISK_IMG=disk.img BLK=y ACCEL=y run

4. Debug with GDB:

make ARCH=x86_64 debug
# In another terminal
make gdb

Sources: Makefile(L34 - L117)  Makefile(L164 - L183) 

Testing Your Changes

AxVisor has a comprehensive testing infrastructure to verify that your changes work correctly across different architectures and configurations.

Local Testing

Local testing can be performed on your development machine using the following commands:

1. Run unit tests:

make unittest

2. Run with a guest VM:

# Create disk image if not created yet
make DISK_IMG=disk.img disk_img

# Run with NimbOS as guest
make ARCH=x86_64 DISK_IMG=disk.img LOG=info BLK=y ACCEL=y \
  VM_CONFIGS=configs/vms/nimbos-x86_64.toml APP_FEATURES=fs run

Testing Infrastructure

The testing infrastructure in AxVisor supports multiple architectures (x86_64, riscv64, aarch64) and can run both locally and remotely.

flowchart TD
A["Developer Change"]
B["Local Testing"]
C["Push to GitHub"]
D["GitHub Actions"]
E["Local Runner Tests"]
F["Remote Runner Tests"]
G["Test on Ubuntu VMriscv64/aarch64"]
H["Test on Remote Serversx86_64 with KVM"]
I["All Tests Pass?"]
J["Merge Change"]
K["Fix Issues"]

A --> B
A --> C
C --> D
D --> E
D --> F
E --> G
F --> H
G --> I
H --> I
I --> J
I --> K
K --> A

Sources: .github/workflows/test.yml(L1 - L146)  Makefile(L203 - L210) 

CI/CD Pipeline

AxVisor employs a comprehensive CI/CD pipeline implemented with GitHub Actions to ensure code quality and compatibility across different architectures.

flowchart TD
A["Push/PR"]
B["GitHub Actions"]
C["test_local Job"]
D["test_remote Job"]
C1["Matrix: riscv64, aarch64"]
C2["Matrix: nightly-2024-12-25, nightly"]
D1["Matrix: x86_64"]
D2["Matrix: nightly-2024-12-25, nightly"]
D3["Matrix: remote_aarkegz, remote_x10dri"]
E["Setup Rust Toolchain"]
F["Setup QEMU"]
G["Setup NimbOS Guest Image"]
H["Run Guest Tests"]
I["Compress Source Code"]
J["Setup NimbOS Guest Image"]
K["Upload to Remote Server"]
L["Run Tests Remotely"]
M["Check for Panics"]

A --> B
B --> C
B --> D
C --> C1
C --> C2
C --> E
D --> D1
D --> D2
D --> D3
D --> I
E --> F
F --> G
G --> H
I --> J
J --> K
K --> L
L --> M

Local Runner Tests

The CI pipeline runs tests on GitHub-hosted runners for riscv64 and aarch64 architectures. These tests verify that AxVisor compiles and runs correctly on these architectures.

Remote Runner Tests

For x86_64 architecture, which requires specific hardware features (Intel VT-x), the CI pipeline uses remote runners. These are custom servers with the necessary hardware that allows running AxVisor with hardware acceleration.

Sources: .github/workflows/test.yml(L9 - L146)  .github/workflows/REMOTE-CI.md(L1 - L50) 

Contribution Guidelines

When contributing to AxVisor, please follow these guidelines to ensure your changes can be smoothly integrated.

Code Style and Formatting

1. Rust Code Formatting:

cargo fmt --all

2. Linting:

make clippy

3. Documentation:

make doc

Git Workflow

flowchart TD
A["Fork Repository"]
B["Create Feature Branch"]
C["Make Changes"]
D["Run Tests"]
E["Create Pull Request"]
F["Address Review Feedback"]
G["Changes Merged"]

A --> B
B --> C
C --> D
D --> E
E --> F
F --> G

1. Fork the Repository: Create your own fork of the repository.
2. Create a Feature Branch: Create a branch for your feature or bugfix.
3. Make Changes: Implement your feature or fix.
4. Run Tests: Ensure all tests pass locally.
5. Create a Pull Request: Submit your changes for review.
6. Address Review Feedback: Make any requested changes.
7. Changes Merged: Once approved, your changes will be merged.

Sources: Makefile(L184 - L202) 

Testing Different Guest Operating Systems

AxVisor supports multiple guest operating systems, including NimbOS, ArceOS, Linux, and others. When testing your changes, it's important to verify that they work with the relevant guest OS.

flowchart TD
A["AxVisor"]
B["Guest VM Testing"]
C["NimbOS Guest"]
D["ArceOS Guest"]
E["Linux Guest"]
F["Setup NimbOS Image"]
G["Run with NimbOS Config"]
H["Setup ArceOS Image"]
I["Run with ArceOS Config"]
J["Setup Linux Image"]
K["Run with Linux Config"]

A --> B
B --> C
B --> D
B --> E
C --> F
D --> H
E --> J
F --> G
H --> I
J --> K

Setting Up Guest Images

1. NimbOS Guest:

# The CI/CD pipeline uses this action to set up NimbOS
# You can download releases from the NimbOS repository

2. Ubuntu Guest (for Linux testing):

make DISK_IMG=ubuntu.img ubuntu_img

Sources: .github/workflows/actions/setup-nimbos-guest-image/action.yml(L1 - L71)  Makefile(L219 - L224) 

Remote Testing Infrastructure

If you need to test changes that require specific hardware features (like Intel VT-x for x86_64), you can use the remote testing infrastructure. This is particularly important for hypervisor development where architectural differences can impact functionality.

Why Remote Testing is Needed

Remote testing allows testing on specific hardware configurations that may not be available in GitHub's runners. In particular:

• Testing Intel VT-x features
• Testing with high-end hardware
• Testing with specific hardware configurations

Sources: .github/workflows/REMOTE-CI.md(L1 - L50) 

This document provides an overview of how to contribute to AxVisor. For more specific information about components, please refer to other sections of the wiki.

Testing Infrastructure

Relevant source files

• .github/workflows/actions/setup-nimbos-guest-image/action.yml
• .github/workflows/test.yml

This page documents the testing infrastructure for AxVisor, explaining how testing is automated, configured, and executed. It covers both local and remote testing setups, guest image preparation, and how to run and interpret tests. For CI/CD pipeline details beyond testing, see CI/CD Pipeline.

Overview

AxVisor employs a comprehensive testing infrastructure to ensure that changes don't break functionality across supported architectures (x86_64, aarch64, and riscv64). The testing system runs virtual machines with guest operating systems to verify that virtualization works correctly.

flowchart TD
subgraph subGraph1["Testing Infrastructure"]
    test["Testing System"]
    local["Local Testing"]
    remote["Remote Testing"]
    localSetup["Setup Environment"]
    localRun["Run Tests"]
    localCheck["Check Results"]
    remoteSetup["Setup Environment"]
    remoteTransfer["Transfer Files"]
    remoteRun["Run Tests"]
    remoteCheck["Check Results"]
    subgraph subGraph0["Environment Setup"]
        setupToolchain["Set Rust Toolchain"]
        setupDeps["Install Dependencies"]
        setupQEMU["Set up QEMU"]
        setupGuestImage["Prepare Guest Images"]
    end
end

local --> localCheck
local --> localRun
local --> localSetup
localSetup --> setupDeps
localSetup --> setupGuestImage
localSetup --> setupQEMU
localSetup --> setupToolchain
remote --> remoteCheck
remote --> remoteRun
remote --> remoteSetup
remote --> remoteTransfer
remoteSetup --> setupToolchain
test --> local
test --> remote

Sources: .github/workflows/test.yml(L1 - L150) 

Testing Strategy

The AxVisor testing infrastructure is built around running actual guest virtual machines to validate functionality. Tests are executed both in local GitHub Actions runners and on specialized remote hardware for architecture-specific testing.

Test Matrix

Tests are conducted across a matrix of configurations:

Sources: .github/workflows/test.yml(L10 - L59) 

Local Testing Workflow

Local testing runs on GitHub Actions runners and focuses on riscv64 and aarch64 architectures using QEMU emulation.

sequenceDiagram
    participant GitHubActions as "GitHub Actions"
    participant checkoutv4 as "checkout@v4"
    participant rusttoolchainstable as "rust-toolchain@stable"
    participant setupqemuv01 as "setup-qemu@v0.1"
    participant setupnimbosguestimage as "setup-nimbos-guest-image"
    participant TestRunner as "Test Runner"

    GitHubActions ->> checkoutv4: Checkout repository
    GitHubActions ->> rusttoolchainstable: Setup Rust toolchain
    Note over rusttoolchainstable: nightly-2024-12-25 or latest nightly
    GitHubActions ->> setupqemuv01: Setup QEMU v8.2.0
    GitHubActions ->> setupnimbosguestimage: Prepare NimbOS guest image
    Note over setupnimbosguestimage: Download NimbOS v0.7, create disk image
    GitHubActions ->> TestRunner: Configure KVM permissions
    GitHubActions ->> TestRunner: Update rust-toolchain.toml
    GitHubActions ->> TestRunner: Run tests with make
    Note over TestRunner: ARCH=(riscv64/aarch64), ACCEL=n

Sources: .github/workflows/test.yml(L10 - L50)  .github/workflows/actions/setup-nimbos-guest-image/action.yml(L1 - L70) 

Local Test Execution Steps

1. Check out the repository
2. Set up the specified Rust toolchain with rust-src component
3. Install cargo-binutils
4. Set up QEMU for the target architecture
5. Download and prepare NimbOS guest image
6. Configure KVM permissions
7. Update rust-toolchain.toml with the specified toolchain
8. Run the test using make with:

• Target architecture
• Disk image path
• Verbose logging
• Block device enabled
• Hardware acceleration disabled
• VM configuration file path
• Filesystem support

Sources: .github/workflows/test.yml(L19 - L50) 

Remote Testing Workflow

Remote testing targets x86_64 architecture on specialized hardware with KVM acceleration enabled.

sequenceDiagram
    participant GitHubActions as "GitHub Actions"
    participant checkoutv4 as "checkout@v4"
    participant setupnimbosguestimage as "setup-nimbos-guest-image"
    participant scpactionv017 as "scp-action@v0.1.7"
    participant sshactionv122 as "ssh-action@v1.2.2"
    participant RemoteRunner as "Remote Runner"

    GitHubActions ->> checkoutv4: Checkout repository
    GitHubActions ->> GitHubActions: Update rust-toolchain.toml
    GitHubActions ->> GitHubActions: Compress source code
    GitHubActions ->> setupnimbosguestimage: Prepare NimbOS guest image
    GitHubActions ->> scpactionv017: Copy files to remote runner
    Note over scpactionv017: Transfer disk image and source code
    GitHubActions ->> sshactionv122: Execute test on remote runner
    sshactionv122 ->> RemoteRunner: Extract source code
    sshactionv122 ->> RemoteRunner: Run make defconfig
    sshactionv122 ->> RemoteRunner: Run make with VM configs
    Note over RemoteRunner: ARCH=x86_64, ACCEL=y
    sshactionv122 ->> RemoteRunner: Check for panic in output
    sshactionv122 ->> RemoteRunner: Clean up

Sources: .github/workflows/test.yml(L52 - L149) 

Remote Test Execution Steps

1. Check out the repository
2. Update rust-toolchain.toml with the specified toolchain
3. Compress the source code
4. Prepare NimbOS guest image locally
5. Transfer files to the remote runner:

• Compressed source code
• Disk image

6. On the remote runner:

• Extract the source code
• Run make defconfig for x86_64
• Execute test with hardware acceleration enabled
• Check output for panic messages
• Clean up temporary files

Sources: .github/workflows/test.yml(L72 - L149) 

Guest VM Image Preparation

The testing infrastructure uses a custom action to prepare guest VM images (specifically NimbOS) for testing.

flowchart TD
subgraph subGraph0["setup-nimbos-guest-image Action"]
    setupStart["Start Setup"]
    downloadNimbOS["Download NimbOS Release"]
    unzipNimbOS["Unzip NimbOS Image"]
    renameNimbOS["Rename to nimbos-{arch}.bin"]
    isX86["Is x86_64?"]
    downloadBIOS["Download AXVM BIOS"]
    createDiskImage["Create Disk Image"]
    mountDisk["Mount Disk Image"]
    copyFiles["Copy Files to Disk"]
    unmountDisk["Unmount Disk"]
    cleanup["Cleanup Temporary Files"]
end

copyFiles --> unmountDisk
createDiskImage --> mountDisk
downloadBIOS --> createDiskImage
downloadNimbOS --> unzipNimbOS
isX86 --> createDiskImage
isX86 --> downloadBIOS
mountDisk --> copyFiles
renameNimbOS --> isX86
setupStart --> downloadNimbOS
unmountDisk --> cleanup
unzipNimbOS --> renameNimbOS

Sources: .github/workflows/actions/setup-nimbos-guest-image/action.yml(L1 - L70) 

Guest Image Setup Process

The setup process:

1. Creates a temporary directory
2. Downloads the appropriate NimbOS release for the target architecture
3. Unzips and renames the NimbOS binary
4. For x86_64, also downloads the AXVM BIOS
5. Creates a disk image using the makefile
6. Mounts the disk image
7. Copies the NimbOS binary and BIOS (if applicable) to the disk image
8. Unmounts the disk image and cleans up temporary files

Sources: .github/workflows/actions/setup-nimbos-guest-image/action.yml(L32 - L70) 

Running Tests Manually

To run tests manually in a local development environment, follow these steps:

1. Set up a disk image with a guest OS (like NimbOS)
2. Configure the test environment:

# For local testing (without hardware acceleration)
make ARCH=<arch> defconfig
make ARCH=<arch> DISK_IMG=<path_to_disk_img> LOG=info BLK=y ACCEL=n VM_CONFIGS=<path_to_vm_config> APP_FEATURES=fs run

# For testing with hardware acceleration (KVM)
make ARCH=<arch> defconfig
make ARCH=<arch> DISK_IMG=<path_to_disk_img> LOG=info BLK=y ACCEL=y VM_CONFIGS=<path_to_vm_config> APP_FEATURES=fs run

Replace <arch> with the target architecture (x86_64, aarch64, or riscv64), <path_to_disk_img> with the path to your guest disk image, and <path_to_vm_config> with the path to your VM configuration file.

Sources: .github/workflows/test.yml(L45 - L50)  .github/workflows/test.yml(L133 - L134) 

Interpreting Test Results

Tests are considered successful if they complete without generating "panic" messages in the output. The CI system checks for these messages automatically:

if grep -q "panic" make_output.log; then
  # Test failed
  exit 1
else
  # Test passed
fi

For debugging failed tests, examine the log output for:

• Panic messages
• Errors during VM boot
• Issues with guest OS execution
• Problems with virtual device emulation

Sources: .github/workflows/test.yml(L139 - L146) 

Architecture-Specific Considerations

Each supported architecture has specific testing requirements:

x86_64

• Tested on remote runners with KVM acceleration
• Requires AXVM BIOS for guest OS boot
• Uses hardware virtualization extensions (VT-x)

aarch64

• Tested on local GitHub Actions runners
• Uses QEMU for emulation
• Leverages virtualization extensions (EL2 mode)

riscv64

• Tested on local GitHub Actions runners
• Uses QEMU for emulation
• Relies on SBI runtime for hypervisor operations

Sources: .github/workflows/test.yml(L16)  .github/workflows/test.yml(L57)  .github/workflows/actions/setup-nimbos-guest-image/action.yml(L52 - L58) 

CI/CD Pipeline

Relevant source files

• .github/workflows/REMOTE-CI.md
• .github/workflows/actions/setup-nimbos-guest-image/action.yml
• .github/workflows/actions/setup-qemu/action.yml
• .github/workflows/test.yml
• Makefile

This page documents the Continuous Integration and Continuous Deployment (CI/CD) pipeline used by the AxVisor hypervisor project. The pipeline handles automated building, testing, and verification across multiple architectures (x86_64, aarch64, riscv64) and environments to ensure the hypervisor operates correctly on various platforms.

For information about the testing infrastructure and methodology, see Testing Infrastructure.

Overview of the CI/CD Pipeline

The AxVisor CI/CD pipeline is implemented using GitHub Actions and consists of two main testing strategies:

1. Local Testing: Tests run directly on GitHub-hosted runners
2. Remote Testing: Tests run on remote machines for hardware-specific testing (particularly for Intel VT-x)

The pipeline automatically triggers on both push events and pull requests, ensuring all code changes are verified before merging.

flowchart TD
subgraph subGraph2["CI/CD Pipeline"]
    trigger["Code Push or PR"]
    gh_actions["GitHub Actions Workflow"]
    subgraph subGraph1["Remote Testing (Custom Servers)"]
        remote_test["test_remote Job"]
        compress["Compress Source"]
        copy["Copy to Remote Server"]
        extract["Extract Source"]
        run_remote["Run Tests on Remote"]
    end
    subgraph subGraph0["Local Testing (GitHub Runners)"]
        local_test["test_local Job"]
        setup_qemu["Setup QEMU Action"]
        setup_img["Setup NimbOS Guest Image"]
        run_test["Run Tests"]
    end
end

compress --> copy
copy --> extract
extract --> run_remote
gh_actions --> local_test
gh_actions --> remote_test
local_test --> setup_qemu
remote_test --> compress
setup_img --> run_test
setup_qemu --> setup_img
trigger --> gh_actions

Sources: .github/workflows/test.yml(L1 - L150) 

Local Testing Process

Local testing runs on GitHub-hosted runners with multiple architecture and Rust toolchain configurations. This testing strategy covers non-hardware-specific features and ensures compatibility across architectures.

Configuration Matrix

The local testing strategy uses a configuration matrix to test multiple combinations:

Local Testing Workflow

sequenceDiagram
    participant GitHubActions as "GitHub Actions"
    participant GitHubRunner as "GitHub Runner"
    participant QEMUVM as "QEMU VM"
    participant NimbOSGuest as "NimbOS Guest"

    GitHubActions ->> GitHubRunner: Start test_local job
    GitHubRunner ->> GitHubRunner: Install Rust toolchain
    GitHubRunner ->> GitHubRunner: Install cargo-binutils
    GitHubRunner ->> GitHubRunner: Setup QEMU
    GitHubRunner ->> GitHubRunner: Prepare NimbOS guest image
    GitHubRunner ->> GitHubRunner: Configure KVM permissions
    GitHubRunner ->> GitHubRunner: Update rust-toolchain.toml
    GitHubRunner ->> QEMUVM: Launch QEMU with NimbOS guest
    QEMUVM ->> NimbOSGuest: Boot NimbOS
    NimbOSGuest ->> GitHubRunner: Test results
    GitHubRunner ->> GitHubActions: Report test status

Sources: .github/workflows/test.yml(L10 - L50)  .github/workflows/actions/setup-qemu/action.yml(L1 - L48)  .github/workflows/actions/setup-nimbos-guest-image/action.yml(L1 - L71) 

Remote Testing Process

Remote testing is used specifically for hardware-dependent features (such as Intel VT-x virtualization) that cannot be properly tested on GitHub's AMD-based runners. This is essential for AxVisor as a hypervisor project that requires specific CPU virtualization features.

Remote Testing Architecture

flowchart TD
subgraph subGraph1["Remote Server"]
    remote_dir["~/runner/remote_test-{run_id}"]
    extract["Extract Source"]
    build["Build AxVisor"]
    run["Run With NimbOS Guest"]
    verify["Verify No Panics"]
    cleanup["Clean Up"]
end
subgraph subGraph0["GitHub Actions"]
    gh_action["test_remote Job"]
    prepare["Prepare Source & Images"]
    scp["SCP Action"]
    ssh["SSH Action"]
end

build --> run
extract --> build
gh_action --> prepare
prepare --> scp
run --> verify
scp --> remote_dir
scp --> ssh
ssh --> extract
verify --> cleanup

Sources: .github/workflows/test.yml(L52 - L150)  .github/workflows/REMOTE-CI.md(L1 - L50) 

Remote Server Configuration

Remote testing currently uses two server configurations:

Credentials and connection details for these servers are stored as GitHub secrets and accessed securely during the workflow execution.

Sources: .github/workflows/test.yml(L59 - L70) 

Guest VM Testing System

AxVisor is tested with real guest VMs to verify its functionality as a hypervisor. The primary guest OS used for testing is NimbOS, which is lightweight and designed specifically for testing purposes.

Guest Image Preparation Process

flowchart TD
subgraph subGraph0["Guest Image Setup"]
    action["setup-nimbos-guest-image Action"]
    download["Download NimbOS Release"]
    extract["Extract Image"]
    bios["Download BIOS (x86_64 only)"]
    create["Create Disk Image"]
    mount["Mount Image"]
    copy["Copy Files to Image"]
    unmount["Unmount Image"]
end

action --> download
bios --> create
copy --> unmount
create --> mount
download --> extract
extract --> bios
mount --> copy

Sources: .github/workflows/actions/setup-nimbos-guest-image/action.yml(L1 - L71) 

Guest Testing Configuration

The testing configuration for guest VMs includes:

• Disk Image: A FAT32 filesystem image containing the guest OS
• Virtual Hardware: Configured through QEMU with appropriate architecture-specific settings
• VM Configuration: Loaded from configs/vms/nimbos-{arch}.toml files

Sources: .github/workflows/test.yml(L47 - L50)  .github/workflows/test.yml(L134) 

QEMU Setup and Integration

QEMU is the virtualization platform used for testing AxVisor. A custom GitHub Action is used to set up QEMU with the required configurations.

QEMU Installation Process

flowchart TD
subgraph subGraph0["QEMU Setup Action"]
    setup["setup-qemu Action"]
    cache_check["Check Cache"]
    download["Download QEMU Source"]
    build["Build QEMU"]
    install["Install QEMU"]
    verify["Verify Installation"]
end

build --> install
cache_check --> download
cache_check --> install
download --> build
install --> verify
setup --> cache_check

Sources: .github/workflows/actions/setup-qemu/action.yml(L1 - L48) 

QEMU Configuration

The QEMU setup configures several architecture-specific system emulators:

• qemu-system-x86_64 for x86_64 architecture
• qemu-system-aarch64 for ARM/aarch64 architecture
• qemu-system-riscv64 for RISC-V architecture

Each emulator is built with slirp networking support and appropriate target configurations.

Sources: .github/workflows/actions/setup-qemu/action.yml(L28 - L31)  .github/workflows/actions/setup-qemu/action.yml(L42 - L47) 

Integration with Makefile Test Targets

The CI/CD pipeline integrates with the project's Makefile test targets to execute tests within both local and remote environments.

Makefile Testing Commands

flowchart TD
subgraph subGraph0["Makefile Testing Targets"]
    make_test["make test"]
    app_test["App-level Tests"]
    make_unittest["make unittest"]
    unit_test["Unit Tests"]
    make_unit_no_fail["make unittest_no_fail_fast"]
    unit_test_nff["Unit Tests (No Fail Fast)"]
end

make_test --> app_test
make_unit_no_fail --> unit_test_nff
make_unittest --> unit_test

Sources: Makefile(L203 - L210) 

CI Testing Execution

In the CI/CD pipeline, tests are executed using a specific command that includes several parameters:

make ARCH={arch} DISK_IMG={disk_img} LOG=info BLK=y ACCEL={accel} VM_CONFIGS={vm_configs} APP_FEATURES=fs run

This command configures:

• The target architecture
• The disk image path
• Logging level
• Block device support
• Hardware acceleration (KVM on x86_64)
• VM configuration files
• Additional features

Sources: .github/workflows/test.yml(L49 - L50)  .github/workflows/test.yml(L134) 

Development Workflow Integration

The CI/CD pipeline is integrated into the development workflow to provide automated testing and verification for all code changes.

Pull Request Workflow

sequenceDiagram
    participant Developer as Developer
    participant GitHub as GitHub
    participant CICDPipeline as "CI/CD Pipeline"
    participant Reviewers as Reviewers

    Developer ->> GitHub: Create Pull Request
    GitHub ->> CICDPipeline: Trigger CI Workflow
    CICDPipeline ->> CICDPipeline: Run Local Tests
    CICDPipeline ->> CICDPipeline: Run Remote Tests
    CICDPipeline ->> GitHub: Report Test Results
    Reviewers ->> GitHub: Review Code
    GitHub ->> Developer: Feedback
    Developer ->> GitHub: Address Feedback
    GitHub ->> CICDPipeline: Trigger CI Again
    CICDPipeline ->> GitHub: Updated Results
    Reviewers ->> GitHub: Approve PR
    GitHub ->> GitHub: Merge PR

Multi-Architecture Support

The CI/CD pipeline is designed to test AxVisor across multiple CPU architectures to ensure broad compatibility.

Architecture Matrix

The pipeline supports testing on three key architectures:

Sources: .github/workflows/test.yml(L16)  .github/workflows/test.yml(L57) 

Future Improvements

Based on the documentation, several potential improvements to the CI/CD pipeline are planned:

1. Better Tool Integration: Using the act tool to run CI tests more consistently
2. Improved SSH Tooling: Finding better alternatives to the current SSH action tools
3. Enhanced Caching: Implementing more efficient caching strategies for QEMU and other environments
4. Automated Environment Setup: Creating automated setup for remote testing environments
5. Additional Hardware Resources: Acquiring more stable and powerful servers for remote testing

Sources: .github/workflows/REMOTE-CI.md(L40 - L49) 

Overview

Relevant source files

• Cargo.toml
• README.md

This document provides an overview of the axdevice_crates repository, which contains foundational crates for building emulated device subsystems within the ArceOS hypervisor ecosystem. The repository specifically targets no_std embedded environments and provides the core abstractions needed for device emulation in virtualized guest systems.

For detailed information about the core device abstraction layer, see Core Architecture. For implementation guidance, see Development Guide. For specifics about external dependencies and ArceOS integration, see Dependencies and Integration.

Purpose and Scope

The axdevice_crates repository serves as the foundation for device emulation in the ArceOS hypervisor. It provides:

• Core traits and abstractions for emulated devices (BaseDeviceOps)
• A comprehensive device type taxonomy (EmuDeviceType)
• Integration points with ArceOS hypervisor address space management
• Support for multiple architectures and virtualization approaches

The repository is designed for no_std environments, making it suitable for embedded systems and hypervisor contexts where the standard library is not available.

Sources: README.md(L1 - L5)  Cargo.toml(L1 - L16) 

Repository Architecture

System Architecture Overview

The repository follows a workspace-based structure with axdevice_base as the foundational crate:

flowchart TD
subgraph subGraph3["no_std Environment"]
    EMB["Embedded RuntimeConstraints"]
end
subgraph subGraph2["External Dependencies"]
    AE["axerrno"]
    AA["axaddrspace"]
    MA["memory_addr"]
    SE["serde"]
end
subgraph subGraph1["ArceOS Hypervisor Ecosystem"]
    AH["ArceOS Hypervisor"]
    AS["Address Space Manager"]
    VM["Virtual Machines"]
end
subgraph subGraph0["axdevice_crates Repository"]
    WS["Workspace Root(Cargo.toml)"]
    AB["axdevice_base Crate"]
    README["README.md"]
end

AB --> AA
AB --> AE
AB --> AH
AB --> EMB
AB --> MA
AB --> SE
AH --> AS
AH --> VM
WS --> AB

Sources: Cargo.toml(L1 - L16)  README.md(L3) 

Core Component Relationships

The following diagram shows how the main code entities relate to each other:

classDiagram
class BaseDeviceOps {
    <<trait>>
    
    +emu_type() EmuDeviceType
    +address_range() AddrRange~GuestPhysAddr~
    +handle_read(addr, width) AxResult~usize~
    +handle_write(addr, width, val) AxResult~() ~
}

class EmuDeviceType {
    <<enum>>
    EmuDeviceTConsole
    EmuDeviceTGicdV2
    EmuDeviceTGPPT
    EmuDeviceTVirtioBlk
    EmuDeviceTVirtioNet
    EmuDeviceTVirtioConsole
    EmuDeviceTIOMMU
    EmuDeviceTICCSRE
    EmuDeviceTSGIR
    EmuDeviceTGICR
    EmuDeviceTMeta
    +removable() bool
    +from_usize(value) Option~EmuDeviceType~
}

class AddrRange~GuestPhysAddr~ {
    <<from axaddrspace>>
    
    
}

class AxResult~T~ {
    <<from axerrno>>
    
    
}

class GuestPhysAddr {
    <<from axaddrspace>>
    
    
}

BaseDeviceOps  -->  EmuDeviceType : "returns"
BaseDeviceOps  -->  AddrRange : "uses"
BaseDeviceOps  -->  AxResult : "returns"
BaseDeviceOps  -->  GuestPhysAddr : "uses"

Sources: Based on architectural analysis from provided diagrams

Device Type Categories

The EmuDeviceType enum categorizes emulated devices into several functional groups:

The majority of device types (8 out of 11) are removable, indicating support for dynamic device configuration in virtualized environments.

Sources: Based on EmuDeviceType analysis from architectural diagrams

Integration with ArceOS Ecosystem

The axdevice_crates repository integrates tightly with the ArceOS hypervisor ecosystem through several key dependencies:

flowchart TD
subgraph subGraph3["ArceOS Hypervisor"]
    HV["Hypervisor Core"]
    ASM["Address SpaceManager"]
end
subgraph subGraph2["Standard Dependencies"]
    SE["serdeSerialization"]
    CF["cfg-ifConditionalCompilation"]
end
subgraph subGraph1["ArceOS Core Dependencies"]
    AE["axerrnoError Handling"]
    AA["axaddrspaceGuest PhysicalAddress Management"]
    MA["memory_addrMemory Utilities"]
end
subgraph axdevice_base["axdevice_base"]
    BDO["BaseDeviceOps"]
    EDT["EmuDeviceType"]
end

AA --> ASM
AE --> HV
BDO --> AA
BDO --> AE
BDO --> HV
BDO --> MA
EDT --> CF
EDT --> SE

Sources: Cargo.toml(L15 - L16)  architectural analysis

Development Environment

The repository targets multiple architectures and maintains compatibility with no_std environments:

• Architecture Support: x86_64, RISC-V (riscv64gc), ARM64 (aarch64)
• Environment: no_std compatible for embedded systems
• License: Triple-licensed under GPL-3.0-or-later OR Apache-2.0 OR MulanPSL-2.0
• Workspace Structure: Single workspace with axdevice_base as the foundational member

The no_std constraint ensures the crates can be used in hypervisor and embedded contexts where the standard library is not available, making them suitable for system-level virtualization components.

Sources: Cargo.toml(L8 - L13)  README.md(L3) 

Next Steps

This overview establishes the foundational understanding of the axdevice_crates repository. For implementation details:

• See Project Structure for detailed workspace organization
• See BaseDeviceOps Trait for the core device abstraction API
• See Device Type System for comprehensive device categorization
• See Implementing New Devices for development guidance

Project Structure

Relevant source files

• Cargo.toml
• axdevice_base/Cargo.toml

This document covers the workspace organization and architectural layout of the axdevice_crates repository. It explains how the axdevice_base foundation crate is structured and integrates with the ArceOS hypervisor ecosystem. For detailed information about the core traits and device abstractions, see Core Architecture. For build system specifics and development workflows, see Build System and CI.

Workspace Organization

The axdevice_crates repository follows a Cargo workspace pattern with a single foundational crate that serves as the basis for device emulation abstractions.

Repository Structure

flowchart TD
subgraph subGraph2["Future Extensions"]
    FUTURE["Additional device crates..."]
end
subgraph subGraph1["axdevice_base Crate"]
    CBASE["Cargo.toml"]
    SRCBASE["src/"]
    LIBRS["src/lib.rs"]
    EMUTY["src/emu_type.rs"]
end
subgraph subGraph0["Repository Root"]
    CWS["Cargo.toml (workspace)"]
    README["README.md"]
    GH[".github/workflows/"]
end

CWS --> CBASE
CWS --> FUTURE
SRCBASE --> EMUTY
SRCBASE --> LIBRS

Sources: Cargo.toml(L1 - L16)  axdevice_base/Cargo.toml(L1 - L17) 

The workspace is configured with resolver version 2 and includes the axdevice_base crate as its primary member. The workspace-level configuration defines common package metadata including version 0.1.0, authorship, and licensing under GPL-3.0-or-later, Apache-2.0, or MulanPSL-2.0.

Workspace Dependencies

Sources: Cargo.toml(L15 - L16) 

axdevice_base Crate Architecture

The axdevice_base crate serves as the foundational layer providing core traits and types for device emulation in the ArceOS hypervisor environment.

Crate Configuration

flowchart TD
subgraph subGraph2["Standard Dependencies"]
    SERDE["serde: 1.0.204"]
    CFGIF["cfg-if: 1.0"]
end
subgraph subGraph1["ArceOS Dependencies"]
    AXERRNO["axerrno: 0.1.0"]
    AXADDR["axaddrspace: git"]
    MEMADDR["memory_addr: 0.3"]
end
subgraph subGraph0["Package Metadata"]
    NAME["name: axdevice_base"]
    EDITION["edition: 2024"]
    VERSION["version: workspace"]
end

NAME --> AXADDR
NAME --> AXERRNO
NAME --> CFGIF
NAME --> MEMADDR
NAME --> SERDE

Sources: axdevice_base/Cargo.toml(L1 - L9)  axdevice_base/Cargo.toml(L10 - L16) 

The crate uses Rust edition 2024 and inherits workspace-level version and metadata. Key configuration aspects include:

• No-std compatibility: Uses alloc feature for collections without requiring standard library
• Serialization support: Serde with derive and alloc features enabled
• Conditional compilation: cfg-if for platform-specific code paths

Dependency Architecture

The crate's dependencies are strategically chosen to support hypervisor device emulation while maintaining compatibility with embedded environments.

flowchart TD
subgraph subGraph3["Dependency Purposes"]
    ERRHAND["Error handling and result types"]
    GUESTMEM["Guest physical address management"]
    MEMUTIL["Memory address utilities"]
    SERIAL["Device type serialization"]
    CONDITIONAL["Platform-specific compilation"]
end
subgraph subGraph2["axdevice_base Dependencies"]
    subgraph subGraph1["External Crates"]
        SERDE["serde"]
        CFGIF["cfg-if"]
    end
    subgraph subGraph0["ArceOS Ecosystem"]
        ERRNO["axerrno"]
        ADDRSPACE["axaddrspace"]
        MEMADDR["memory_addr"]
    end
end

ADDRSPACE --> GUESTMEM
CFGIF --> CONDITIONAL
ERRNO --> ERRHAND
MEMADDR --> MEMUTIL
SERDE --> SERIAL

Sources: axdevice_base/Cargo.toml(L11 - L16) 

ArceOS-Specific Dependencies

• axerrno: Provides AxResult<T> type for consistent error handling across the ArceOS ecosystem
• axaddrspace: Supplies GuestPhysAddr and address range management for hypervisor memory operations
• memory_addr: Low-level memory address manipulation utilities

External Dependencies

• serde: Enables serialization of device types with derive macros and alloc feature for no-std environments
• cfg-if: Facilitates conditional compilation for different target architectures

Integration Points

ArceOS Hypervisor Integration

The axdevice_base crate integrates with the broader ArceOS hypervisor ecosystem through well-defined interfaces:

flowchart TD
subgraph subGraph2["Device Implementations"]
    DEVS["Future Device Crates"]
end
subgraph subGraph1["axdevice_base Integration"]
    BDO["BaseDeviceOps trait"]
    EDT["EmuDeviceType enum"]
    AR["AddrRange"]
end
subgraph subGraph0["ArceOS Hypervisor Core"]
    HV["Hypervisor Engine"]
    AS["Address Space Manager"]
    VM["Virtual Machine Instance"]
end

AS --> AR
BDO --> EDT
DEVS --> BDO
DEVS --> EDT
HV --> BDO
VM --> HV

Sources: axdevice_base/Cargo.toml(L14 - L15) 

Target Architecture Support

The crate is designed to support multiple target architectures commonly used in embedded hypervisor environments:

The architecture support reflects the project's focus on ARM virtualization (evidenced by the prevalence of ARM interrupt controller device types) while maintaining cross-platform compatibility.

Extensibility Design

The workspace structure anticipates future expansion with additional device implementation crates:

flowchart TD
subgraph subGraph1["Future Device Crates"]
    CONSOLE["axdevice_console"]
    VIRTIO["axdevice_virtio"]
    ARM["axdevice_arm_gic"]
    IOMMU["axdevice_iommu"]
end
subgraph subGraph0["Current Structure"]
    BASE["axdevice_base"]
end

BASE --> ARM
BASE --> CONSOLE
BASE --> IOMMU
BASE --> VIRTIO

Sources: Cargo.toml(L4 - L6) 

The workspace member list currently contains only axdevice_base, but the structure supports adding device-specific implementation crates that would depend on the foundation crate and implement the BaseDeviceOps trait for specific hardware categories.

Core Architecture

Relevant source files

• axdevice_base/src/lib.rs

This document outlines the fundamental design patterns, traits, and abstractions that form the foundation of the axdevice_crates device emulation system. It covers the high-level architectural decisions, the trait-based abstraction model, and how the core components interact within the ArceOS hypervisor ecosystem.

For detailed documentation of the BaseDeviceOps trait implementation, see BaseDeviceOps Trait. For comprehensive coverage of device type classifications, see Device Type System. For specifics on memory management and guest physical addressing, see Address Space Management.

Architectural Overview

The axdevice_crates system implements a trait-based device emulation architecture designed specifically for the ArceOS hypervisor. The architecture centers around a unified abstraction layer that enables consistent device emulation across different hardware types while maintaining the performance characteristics required for hypervisor environments.

Core Architectural Components

flowchart TD
subgraph subGraph3["External Dependencies"]
    AE["axerrno::AxResult"]
    AA["axaddrspace::GuestPhysAddr"]
    MA["memory_addr::AddrRange"]
end
subgraph subGraph2["Device Implementation Layer"]
    DEV1["Console Device"]
    DEV2["Interrupt Controller"]
    DEV3["Virtio Device"]
    DEVN["... Other Devices"]
end
subgraph subGraph1["Device Abstraction Layer"]
    BDO["BaseDeviceOps Trait"]
    EDT["EmuDeviceType Enum"]
    AR["AddrRange"]
end
subgraph subGraph0["Hypervisor Integration Layer"]
    HV["ArceOS Hypervisor Core"]
    AS["Address Space Manager"]
end

AR --> AA
AR --> MA
AS --> AA
BDO --> AE
BDO --> AR
BDO --> EDT
DEV1 --> BDO
DEV1 --> EDT
DEV2 --> BDO
DEV2 --> EDT
DEV3 --> BDO
DEV3 --> EDT
DEVN --> BDO
DEVN --> EDT
HV --> BDO

Sources: axdevice_base/src/lib.rs(L1 - L31) 

Trait-Based Device Abstraction

The system employs a trait-based abstraction model where all emulated devices implement the BaseDeviceOps trait. This design provides a uniform interface for device operations while allowing each device type to implement specific emulation logic.

BaseDeviceOps Contract

classDiagram
class BaseDeviceOps {
    <<trait>>
    
    +emu_type() EmuDeviceType
    +address_range() AddrRange~GuestPhysAddr~
    +handle_read(addr: GuestPhysAddr, width: usize) AxResult~usize~
    +handle_write(addr: GuestPhysAddr, width: usize, val: usize)
}

class EmuDeviceType {
    <<enum>>
    EmuDeviceTConsole
    EmuDeviceTGicdV2
    EmuDeviceTVirtioBlk
    +removable() bool
}

class AddrRange~GuestPhysAddr~ {
    +start GuestPhysAddr
    +end GuestPhysAddr
    +contains(addr: GuestPhysAddr) bool
}

class GuestPhysAddr {
    <<from axaddrspace>>
    
    
}

class AxResult~T~ {
    <<from axerrno>>
    
    Ok(T)
    Err(AxError)
}

BaseDeviceOps  -->  EmuDeviceType : "returns"
BaseDeviceOps  -->  AddrRange : "returns"
BaseDeviceOps  -->  AxResult : "returns"
AddrRange  -->  GuestPhysAddr : "contains"

Sources: axdevice_base/src/lib.rs(L20 - L30) 

The trait defines four essential operations that every emulated device must support:

Memory-Mapped Device Emulation Model

The architecture implements a memory-mapped I/O (MMIO) model where devices are accessed through specific guest physical address ranges. This design mirrors how real hardware devices are typically accessed and provides natural integration with existing hypervisor memory management systems.

Device Access Flow

sequenceDiagram
    participant GuestVM as "Guest VM"
    participant ArceOSHypervisor as "ArceOS Hypervisor"
    participant BaseDeviceOpsImplementation as "BaseDeviceOps Implementation"
    participant EmuDeviceType as "EmuDeviceType"

    GuestVM ->> ArceOSHypervisor: "Memory Access (GuestPhysAddr)"
    ArceOSHypervisor ->> BaseDeviceOpsImplementation: "address_range()"
    BaseDeviceOpsImplementation ->> ArceOSHypervisor: "AddrRange<GuestPhysAddr>"
    alt "Address in device range"
        ArceOSHypervisor ->> BaseDeviceOpsImplementation: "emu_type()"
        BaseDeviceOpsImplementation ->> EmuDeviceType: "Get device classification"
        EmuDeviceType ->> BaseDeviceOpsImplementation: "Device type metadata"
        BaseDeviceOpsImplementation ->> ArceOSHypervisor: "EmuDeviceType"
    alt "Read Operation"
        ArceOSHypervisor ->> BaseDeviceOpsImplementation: "handle_read(addr, width)"
        BaseDeviceOpsImplementation ->> BaseDeviceOpsImplementation: "Perform device-specific read logic"
        BaseDeviceOpsImplementation ->> ArceOSHypervisor: "AxResult<usize>"
        ArceOSHypervisor ->> GuestVM: "Return read data"
    else "Write Operation"
        ArceOSHypervisor ->> BaseDeviceOpsImplementation: "handle_write(addr, width, val)"
        BaseDeviceOpsImplementation ->> BaseDeviceOpsImplementation: "Perform device-specific write logic"
        ArceOSHypervisor ->> GuestVM: "Complete write operation"
    end
    else "Address not in device range"
        ArceOSHypervisor ->> GuestVM: "Handle as regular memory access"
    end

Sources: axdevice_base/src/lib.rs(L24 - L29) 

Integration with Hypervisor Core

The device emulation system integrates tightly with the ArceOS hypervisor through several key interfaces and design decisions:

Dependency Architecture

No Standard Library Constraint

The entire system operates under #![no_std] constraints, reflecting its target deployment in embedded hypervisor environments. This architectural decision influences several design choices:

• Use of alloc crate for heap-allocated collections when needed
• Reliance on external crates for core functionality (error handling, address management)
• Emphasis on zero-cost abstractions and compile-time optimizations

Sources: axdevice_base/src/lib.rs(L1) 

Design Principles

The architecture embodies several key design principles that guide the system's development and usage:

Uniform Interface: All devices implement identical trait methods, enabling generic device management code in the hypervisor.

Type Safety: Strong typing through EmuDeviceType enum and GuestPhysAddr wrapper prevents common addressing errors.

Performance: Direct trait dispatch and no_std environment minimize runtime overhead.

Extensibility: New device types can be added by implementing BaseDeviceOps and extending EmuDeviceType.

ArceOS Integration: Deep integration with ArceOS ecosystem through shared address space management and error handling patterns.

Sources: axdevice_base/src/lib.rs(L3 - L18) 

BaseDeviceOps Trait

Relevant source files

• axdevice_base/src/lib.rs

This document covers the BaseDeviceOps trait, which defines the core interface that all emulated devices must implement in the ArceOS hypervisor device emulation system. The trait provides a uniform contract for device identification, address mapping, and memory access handling across different types of virtualized hardware.

For information about specific device types that implement this trait, see Device Type System. For details about address space management and memory mapping, see Address Space Management.

Trait Overview

The BaseDeviceOps trait serves as the fundamental abstraction layer for device emulation in the ArceOS hypervisor. It defines four essential operations that enable the hypervisor to uniformly interact with different types of emulated devices, from simple console devices to complex interrupt controllers.

flowchart TD
subgraph Parameters["Parameters"]
    GPA["GuestPhysAddr"]
    WIDTH["usize (width)"]
    VAL["usize (val)"]
end
subgraph subGraph1["Return Types"]
    EDT["EmuDeviceType"]
    AR["AddrRange"]
    AXR["AxResult"]
    UNIT["()"]
end
subgraph subGraph0["BaseDeviceOps Contract"]
    BDO["BaseDeviceOps trait"]
    EMU["emu_type()"]
    ADDR["address_range()"]
    READ["handle_read()"]
    WRITE["handle_write()"]
end

ADDR --> AR
BDO --> ADDR
BDO --> EMU
BDO --> READ
BDO --> WRITE
EMU --> EDT
READ --> AXR
READ --> GPA
READ --> WIDTH
WRITE --> GPA
WRITE --> UNIT
WRITE --> VAL
WRITE --> WIDTH

The trait is designed for no_std environments and integrates with the ArceOS ecosystem through typed address spaces and standardized error handling.

Sources: axdevice_base/src/lib.rs(L20 - L30) 

Method Specifications

Device Type Identification

flowchart TD
subgraph subGraph0["Device Categories"]
    CONSOLE["EmuDeviceTConsole"]
    VIRTIO["EmuDeviceTVirtioBlk"]
    GIC["EmuDeviceTGicdV2"]
    OTHER["Unsupported markdown: list"]
end
DEV["Device Implementation"]
EMU_TYPE["emu_type()"]
EDT["EmuDeviceType"]

DEV --> EMU_TYPE
EDT --> CONSOLE
EDT --> GIC
EDT --> OTHER
EDT --> VIRTIO
EMU_TYPE --> EDT

The emu_type() method returns an EmuDeviceType enum value that identifies the specific type of device being emulated. This enables the hypervisor to apply device-specific logic and optimizations.

Sources: axdevice_base/src/lib.rs(L22 - L23) 

Address Range Management

flowchart TD
subgraph subGraph1["Hypervisor Usage"]
    LOOKUP["Address lookup"]
    ROUTING["Device routing"]
    VALIDATION["Access validation"]
end
subgraph subGraph0["Address Space Components"]
    GPA_START["start: GuestPhysAddr"]
    GPA_END["end: GuestPhysAddr"]
    SIZE["size calculation"]
end
ADDR_RANGE["address_range()"]
AR["AddrRange"]

ADDR_RANGE --> AR
AR --> GPA_END
AR --> GPA_START
AR --> LOOKUP
AR --> ROUTING
AR --> SIZE
AR --> VALIDATION

The address_range() method defines the guest physical address space region that the device occupies. The hypervisor uses this information to route memory accesses to the appropriate device implementation.

Sources: axdevice_base/src/lib.rs(L24 - L25) 

Memory Access Handlers

flowchart TD
subgraph subGraph2["Parameters and Results"]
    GPA_PARAM["GuestPhysAddr addr"]
    WIDTH_PARAM["usize width"]
    VAL_PARAM["usize val"]
    RESULT["AxResult"]
    VOID_RESULT["()"]
end
subgraph subGraph1["BaseDeviceOps Methods"]
    HANDLE_READ["handle_read(addr, width)"]
    HANDLE_WRITE["handle_write(addr, width, val)"]
end
subgraph subGraph0["Memory Access Flow"]
    GUEST["Guest VM Memory Access"]
    HV_DECODE["Hypervisor Address Decode"]
    DEVICE_CHECK["Device Range Check"]
end

DEVICE_CHECK --> HANDLE_READ
DEVICE_CHECK --> HANDLE_WRITE
GUEST --> HV_DECODE
HANDLE_READ --> GPA_PARAM
HANDLE_READ --> RESULT
HANDLE_READ --> WIDTH_PARAM
HANDLE_WRITE --> GPA_PARAM
HANDLE_WRITE --> VAL_PARAM
HANDLE_WRITE --> VOID_RESULT
HANDLE_WRITE --> WIDTH_PARAM
HV_DECODE --> DEVICE_CHECK

The memory access handlers implement the core device emulation logic. The handle_read() method processes guest read operations and returns emulated device state, while handle_write() processes guest write operations that may modify device behavior.

Sources: axdevice_base/src/lib.rs(L26 - L29) 

Implementation Requirements

Error Handling

Read operations return AxResult<usize> to handle potential emulation errors such as invalid register addresses or unsupported access widths. Write operations use a void return type, with error conditions typically handled through device-specific state changes or logging.

Access Width Support

The width parameter specifies the memory access size in bytes (typically 1, 2, 4, or 8). Device implementations must handle the access widths supported by their emulated hardware, potentially returning errors for unsupported widths.

Address Validation

Device implementations should validate that the provided addr parameter falls within their declared address range and corresponds to a valid device register or memory location.

Integration Patterns

flowchart TD
subgraph subGraph2["External Dependencies"]
    AXADDRSPACE["axaddrspace::GuestPhysAddr"]
    AXERRNO["axerrno::AxResult"]
    MEMORY_ADDR["memory_addr::AddrRange"]
end
subgraph subGraph1["Device Implementation"]
    DEVICE["Device Struct"]
    BDO_IMPL["BaseDeviceOps impl"]
    DEV_STATE["Device State"]
end
subgraph subGraph0["ArceOS Hypervisor"]
    HV_CORE["Hypervisor Core"]
    ADDR_MGR["Address Space Manager"]
    VM_EXIT["VM Exit Handler"]
end

ADDR_MGR --> BDO_IMPL
BDO_IMPL --> AXADDRSPACE
BDO_IMPL --> AXERRNO
BDO_IMPL --> DEVICE
BDO_IMPL --> MEMORY_ADDR
DEVICE --> DEV_STATE
HV_CORE --> ADDR_MGR
VM_EXIT --> HV_CORE

The trait integrates with the ArceOS hypervisor through standardized address space management and error handling. Device implementations typically maintain internal state and use the trait methods as the primary interface for hypervisor interaction.

Sources: axdevice_base/src/lib.rs(L11 - L18) 

Device Type System

Relevant source files

• axdevice_base/src/emu_type.rs
• axdevice_base/src/lib.rs

This document covers the EmuDeviceType enumeration and associated type system that categorizes and manages different classes of emulated devices in the ArceOS hypervisor. The device type system provides unified identification, classification, and removability management for all supported emulated hardware devices.

For information about how devices implement behavior through the trait system, see BaseDeviceOps Trait. For details about memory mapping and address ranges, see Address Space Management.

Overview

The device type system is built around the EmuDeviceType enum defined in axdevice_base/src/emu_type.rs(L3 - L28)  This enumeration serves as the central registry of all supported emulated device types in the ArceOS hypervisor ecosystem, providing type identification, categorization, and operational characteristics for each device class.

flowchart TD
subgraph subGraph1["Integration Points"]
    BDO["BaseDeviceOps trait"]
    HV["ArceOS Hypervisor"]
end
subgraph subGraph0["Device Type System Core"]
    EDT["EmuDeviceType enum"]
    REM["removable() method"]
    CONV["from_usize() method"]
    DISP["Display implementation"]
end

BDO --> EDT
EDT --> CONV
EDT --> DISP
EDT --> HV
EDT --> REM
HV --> BDO

Sources: axdevice_base/src/emu_type.rs(L3 - L28)  axdevice_base/src/lib.rs(L20 - L30) 

Device Type Enumeration

The EmuDeviceType enum defines 11 distinct device types, each assigned a unique numeric identifier from 0 to 10. The enumeration uses explicit discriminant values to ensure stable serialization and conversion.

Sources: axdevice_base/src/emu_type.rs(L5 - L28) 

Device Categories

Device Type Taxonomy by Functional Category

flowchart TD
subgraph subGraph2["Virtio Paravirtualization"]
    IOMMU["EmuDeviceTIOMMU"]
    META["EmuDeviceTMeta"]
    VBLK["EmuDeviceTVirtioBlk"]
    VNET["EmuDeviceTVirtioNet"]
    VCONS2["EmuDeviceTVirtioConsole"]
    GICD["EmuDeviceTGicdV2"]
    GPPT["EmuDeviceTGPPT"]
    ICCSRE["EmuDeviceTICCSRE"]
    CONS["EmuDeviceTConsole"]
    VCONS["EmuDeviceTVirtioConsole"]
    subgraph subGraph0["Console and I/O Devices"]
        subgraph subGraph3["System Infrastructure"]
            IOMMU["EmuDeviceTIOMMU"]
            META["EmuDeviceTMeta"]
            VBLK["EmuDeviceTVirtioBlk"]
            VNET["EmuDeviceTVirtioNet"]
            GICD["EmuDeviceTGicdV2"]
            GPPT["EmuDeviceTGPPT"]
            CONS["EmuDeviceTConsole"]
            VCONS["EmuDeviceTVirtioConsole"]
        end
    end
end
subgraph subGraph1["ARM Interrupt Controllers"]
    IOMMU["EmuDeviceTIOMMU"]
    META["EmuDeviceTMeta"]
    VBLK["EmuDeviceTVirtioBlk"]
    VNET["EmuDeviceTVirtioNet"]
    VCONS2["EmuDeviceTVirtioConsole"]
    GICD["EmuDeviceTGicdV2"]
    GPPT["EmuDeviceTGPPT"]
    ICCSRE["EmuDeviceTICCSRE"]
    SGIR["EmuDeviceTSGIR"]
    GICR["EmuDeviceTGICR"]
    CONS["EmuDeviceTConsole"]
    VCONS["EmuDeviceTVirtioConsole"]
end

Sources: axdevice_base/src/emu_type.rs(L5 - L28)  axdevice_base/src/emu_type.rs(L34 - L45) 

ARM Interrupt Controller Focus

The device type system shows a strong focus on ARM architecture virtualization, with 5 out of 11 device types (45%) dedicated to ARM Generic Interrupt Controller (GIC) components:

• EmuDeviceTGicdV2: ARM GICv2 distributor interface
• EmuDeviceTGPPT: Partial passthrough interrupt controller
• EmuDeviceTICCSRE: ICC System Register Enable interface
• EmuDeviceTSGIR: ICC Software Generated Interrupt Register
• EmuDeviceTGICR: GIC redistributor interface

Virtio Paravirtualization Support

Three device types implement Virtio paravirtualization standards:

• EmuDeviceTVirtioBlk: Block storage device
• EmuDeviceTVirtioNet: Network interface device
• EmuDeviceTVirtioConsole: Console device

Sources: axdevice_base/src/emu_type.rs(L5 - L28) 

Removability Classification

The device type system categorizes devices by removability using the removable() method defined in axdevice_base/src/emu_type.rs(L52 - L64)  This classification determines whether devices can be dynamically added or removed from a running virtual machine.

Device Removability Classification

flowchart TD
subgraph subGraph0["Removable Devices (8 of 11)"]
    GPPT_R["EmuDeviceTGPPT"]
    VBLK_R["EmuDeviceTVirtioBlk"]
    VNET_R["EmuDeviceTVirtioNet"]
    GICR_R["EmuDeviceTGICR"]
    VCONS_R["EmuDeviceTVirtioConsole"]
    subgraph subGraph1["Non-Removable Devices (3 of 11)"]
        CONS_NR["EmuDeviceTConsole"]
        IOMMU_NR["EmuDeviceTIOMMU"]
        META_NR["EmuDeviceTMeta"]
        GICD_R["EmuDeviceTGicdV2"]
        SGIR_R["EmuDeviceTSGIR"]
        ICCSRE_R["EmuDeviceTICCSRE"]
    end
end

Sources: axdevice_base/src/emu_type.rs(L52 - L64) 

Removable Devices

The following 8 device types are classified as removable:

• All ARM interrupt controller devices
• All Virtio paravirtualization devices

Non-Removable Devices

The following 3 device types are classified as non-removable:

• Console device: Essential for basic VM operation
• IOMMU device: Critical system infrastructure
• Meta device: Core hypervisor functionality

Sources: axdevice_base/src/emu_type.rs(L53 - L63) 

Type Conversion and Identification

Numeric Conversion

The from_usize() method provides conversion from numeric identifiers to device types, supporting serialization and external configuration systems.

// Example conversion pattern from axdevice_base/src/emu_type.rs:67-82
pub fn from_usize(value: usize) -> EmuDeviceType {
    match value {
        0 => EmuDeviceType::EmuDeviceTConsole,
        1 => EmuDeviceType::EmuDeviceTGicdV2,
        // ... additional mappings
        _ => panic!("Unknown EmuDeviceType value: {}", value),
    }
}

String Representation

The Display trait implementation in axdevice_base/src/emu_type.rs(L30 - L47)  provides human-readable names for debugging and logging:

• EmuDeviceTConsole → "console"
• EmuDeviceTGicdV2 → "Arm interrupt controller V2"
• EmuDeviceTVirtioBlk → "virtio block"

Sources: axdevice_base/src/emu_type.rs(L30 - L47)  axdevice_base/src/emu_type.rs(L67 - L82) 

Integration with BaseDeviceOps

The device type system integrates with the BaseDeviceOps trait through the emu_type() method, enabling runtime device identification:

Device Type Integration Flow

sequenceDiagram
    participant ArceOSHypervisor as "ArceOS Hypervisor"
    participant DeviceBaseDeviceOps as "Device (BaseDeviceOps)"
    participant EmuDeviceType as "EmuDeviceType"

    ArceOSHypervisor ->> DeviceBaseDeviceOps: "emu_type()"
    DeviceBaseDeviceOps ->> EmuDeviceType: "return device type"
    EmuDeviceType ->> DeviceBaseDeviceOps: "EmuDeviceType variant"
    DeviceBaseDeviceOps ->> ArceOSHypervisor: "device type identifier"
    ArceOSHypervisor ->> EmuDeviceType: "removable()"
    EmuDeviceType ->> ArceOSHypervisor: "bool result"
    ArceOSHypervisor ->> EmuDeviceType: "Display format"
    EmuDeviceType ->> ArceOSHypervisor: "human readable name"

Sources: axdevice_base/src/lib.rs(L22 - L23)  axdevice_base/src/emu_type.rs(L50 - L83) 

This integration enables the hypervisor to:

1. Identify device types at runtime
2. Determine removability characteristics
3. Apply type-specific handling logic
4. Generate debugging output with descriptive names

Sources: axdevice_base/src/lib.rs(L20 - L30)  axdevice_base/src/emu_type.rs(L1 - L84) 

Address Space Management

Relevant source files

• axdevice_base/src/lib.rs

This document covers the address space management system used by emulated devices in the ArceOS hypervisor. It explains how guest physical addresses (GuestPhysAddr) are mapped to device handlers, how address ranges (AddrRange<GuestPhysAddr>) define device memory regions, and the integration with the axaddrspace crate for hypervisor address space management.

For information about device type classification and the BaseDeviceOps trait interface, see Device Type System and BaseDeviceOps Trait.

Core Address Space Concepts

The address space management system is built around two fundamental types from external crates:

• GuestPhysAddr from the axaddrspace crate represents individual guest physical addresses
• AddrRange<GuestPhysAddr> from the memory_addr crate represents contiguous address ranges

Every emulated device must define its memory-mapped region through the address_range() method in the BaseDeviceOps trait, which returns an AddrRange<GuestPhysAddr> defining the device's location in guest physical memory space.

Address Space Type Hierarchy

flowchart TD
subgraph subGraph0["External Crates"]
    AS["axaddrspace crate"]
    MA["memory_addr crate"]
end
GPA["GuestPhysAddr"]
AR["AddrRange<GuestPhysAddr>"]
BDO["BaseDeviceOps::address_range()"]
HR["BaseDeviceOps::handle_read(addr: GuestPhysAddr)"]
HW["BaseDeviceOps::handle_write(addr: GuestPhysAddr)"]

AR --> BDO
AS --> GPA
GPA --> AR
GPA --> HR
GPA --> HW
MA --> AR

Sources: axdevice_base/src/lib.rs(L11 - L14)  axdevice_base/src/lib.rs(L25)  axdevice_base/src/lib.rs(L27 - L29) 

Guest Physical Address Mapping

Guest physical addresses represent the memory layout as seen by the virtual machine. Each emulated device occupies a specific range within this address space, allowing the hypervisor to route memory accesses to the appropriate device handler.

The BaseDeviceOps trait defines the contract for address space integration:

The width parameter specifies the access size (1, 2, 4, or 8 bytes), allowing devices to handle different data types appropriately.

Sources: axdevice_base/src/lib.rs(L25 - L29) 

Device Address Range Management

Each device implementation must return a valid AddrRange<GuestPhysAddr> that defines its memory-mapped region. This range is used by the hypervisor's address space manager to route memory accesses to the correct device handler.

Device Address Range Resolution Flow

The address space manager maintains a mapping between guest physical address ranges and device handlers, enabling efficient routing of memory operations without requiring linear searches through all devices.

Sources: axdevice_base/src/lib.rs(L25) 

Memory Access Routing Flow

When a guest virtual machine accesses memory, the hypervisor must determine whether the access targets an emulated device or regular memory. This determination is made using the address ranges returned by each device's address_range() method.

Memory Access Decision Tree

flowchart TD
subgraph subGraph1["Hypervisor Memory Manager"]
    MEM["Handle as regular memory"]
end
subgraph subGraph0["Device Handler Methods"]
    READ["handle_read(addr, width)"]
    WRITE["handle_write(addr, width, val)"]
end
MA["Guest Memory Access at GuestPhysAddr"]
AR["Check address_range() for all devices"]
HIT["Address matches device range"]
MISS["Address does not match any device"]

AR --> HIT
AR --> MISS
HIT --> READ
HIT --> WRITE
MA --> AR
MISS --> MEM

The routing decision is based on whether the guest physical address falls within any device's declared address range. If multiple devices claim overlapping ranges, the behavior is implementation-defined by the hypervisor's address space manager.

Sources: axdevice_base/src/lib.rs(L27 - L29) 

Integration with axaddrspace

The axaddrspace crate provides the GuestPhysAddr type that represents addresses in the guest's physical memory space. This integration ensures type safety and prevents confusion between different address spaces (host physical, guest physical, guest virtual).

The choice of GuestPhysAddr as the address type in BaseDeviceOps methods indicates that device emulation operates at the guest physical memory level, after any guest virtual-to-physical translation has occurred within the guest VM.

Address Space Integration Architecture

flowchart TD
subgraph subGraph2["Host System"]
    HPA["Host Physical Address"]
end
subgraph subGraph1["ArceOS Hypervisor"]
    ASPCM["axaddrspace::GuestPhysAddr"]
    AR["memory_addr::AddrRange"]
    BDO["BaseDeviceOps methods"]
end
subgraph subGraph0["Guest VM"]
    GVA["Guest Virtual Address"]
    GPA["Guest Physical Address"]
end

ASPCM --> AR
ASPCM --> BDO
BDO --> HPA
GPA --> ASPCM
GPA --> HPA
GVA --> GPA

This architecture ensures that emulated devices receive addresses in a consistent format regardless of the guest's internal memory management configuration.

Sources: axdevice_base/src/lib.rs(L13)  axdevice_base/src/lib.rs(L25)  axdevice_base/src/lib.rs(L27 - L29) 

Dependencies and Integration

Relevant source files

• axdevice_base/Cargo.toml

This document covers the external crate dependencies used by axdevice_base and explains how they integrate to support device emulation within the ArceOS hypervisor ecosystem. The focus is on dependency selection rationale, integration patterns, and the no_std compatibility requirements.

For detailed coverage of how ArceOS-specific dependencies (axerrno, axaddrspace, memory_addr) enable hypervisor functionality, see ArceOS Integration. For information about how dependencies support the core device abstraction, see BaseDeviceOps Trait.

Dependency Architecture Overview

The axdevice_base crate maintains a minimal dependency footprint while supporting essential hypervisor device emulation capabilities. All dependencies are carefully selected for no_std compatibility and embedded system constraints.

Dependency Categories and Integration

flowchart TD
subgraph subGraph3["Core Allocator"]
    AL["alloc::collections"]
end
subgraph subGraph2["Standard Utilities"]
    SE["serde (derive, alloc)"]
    CF["cfg-if"]
end
subgraph subGraph1["ArceOS Ecosystem Dependencies"]
    AE["axerrno::AxResult"]
    AA["axaddrspace::AddrRange"]
    MA["memory_addr::GuestPhysAddr"]
end
subgraph axdevice_base["axdevice_base"]
    BDO["BaseDeviceOps"]
    EMU["EmuDeviceType"]
end

AA --> MA
BDO --> AA
BDO --> AE
BDO --> MA
EMU --> CF
EMU --> SE
SE --> AL

Sources: axdevice_base/Cargo.toml(L10 - L16) 

Dependency Analysis

ArceOS Hypervisor Dependencies

The three ArceOS-specific dependencies form the core integration layer that enables device emulation within the hypervisor:

ArceOS Integration Pattern

sequenceDiagram
    participant DeviceImplementation as "Device Implementation"
    participant BaseDeviceOps as "BaseDeviceOps"
    participant axerrnoAxResult as "axerrno::AxResult"
    participant axaddrspaceAddrRange as "axaddrspace::AddrRange"
    participant memory_addrGuestPhysAddr as "memory_addr::GuestPhysAddr"

    DeviceImplementation ->> BaseDeviceOps: "implement trait"
    BaseDeviceOps ->> axaddrspaceAddrRange: "address_range()"
    axaddrspaceAddrRange ->> memory_addrGuestPhysAddr: "AddrRange<GuestPhysAddr>"
    memory_addrGuestPhysAddr ->> axaddrspaceAddrRange: "physical address boundaries"
    axaddrspaceAddrRange ->> BaseDeviceOps: "device memory mapping"
    BaseDeviceOps ->> axerrnoAxResult: "handle_read() -> AxResult<usize>"
    axerrnoAxResult ->> DeviceImplementation: "hypervisor-compatible result"

Sources: axdevice_base/Cargo.toml(L14 - L16) 

Standard Library Dependencies

serde Configuration

The serde dependency uses a specific feature configuration optimized for no_std environments:

serde = { version = "1.0.204", default-features = false, features = ["derive", "alloc"] }

• default-features = false: Removes std dependency
• derive feature: Enables #[derive(Serialize, Deserialize)] macros for EmuDeviceType
• alloc feature: Provides collections support without requiring full standard library

Conditional Compilation Support

The cfg-if crate enables clean conditional compilation patterns:

cfg-if = "1.0"

This dependency supports platform-specific device behavior while maintaining code clarity across different target architectures (x86_64, RISC-V, ARM64).

Sources: axdevice_base/Cargo.toml(L11 - L12) 

Integration Patterns

Error Propagation Chain

The error handling integration follows a consistent pattern from hypervisor operations through device emulation:

Error Flow Through Dependencies

flowchart TD
subgraph subGraph3["Address Resolution"]
    AA["AddrRange"]
    MA["GuestPhysAddr"]
end
subgraph subGraph2["Error Types"]
    AR["AxResult"]
    AE["AxError"]
end
subgraph subGraph1["Device Layer"]
    BD["BaseDeviceOps::handle_read()"]
    BDW["BaseDeviceOps::handle_write()"]
end
subgraph subGraph0["Hypervisor Layer"]
    HV["ArceOS Hypervisor"]
end

AA --> MA
AR --> AE
BD --> AA
BD --> AR
BDW --> AA
BDW --> AR
HV --> BD
HV --> BDW

Sources: axdevice_base/Cargo.toml(L14 - L16) 

Memory Address Integration

The address management dependencies create a unified addressing model:

1. memory_addr::GuestPhysAddr: Provides type-safe guest physical addresses
2. axaddrspace::AddrRange: Wraps GuestPhysAddr into address ranges
3. BaseDeviceOps::address_range(): Returns AddrRange<GuestPhysAddr> for device mapping

This integration ensures that device emulation operates within the hypervisor's guest address space management without requiring device implementations to handle low-level address translation.

Serialization Integration

The serde integration with EmuDeviceType enables device type persistence and configuration management:

• Device type serialization for hypervisor state saving
• Configuration file parsing for device topology
• Inter-component communication with serialized device descriptors

The alloc feature dependency provides the necessary collection types for handling serialized device configurations without requiring full standard library support.

Sources: axdevice_base/Cargo.toml(L12) 

Dependency Version Strategy

ArceOS Ecosystem Tracking

• axerrno: Uses semantic versioning (0.1.0) for stable error handling interface
• axaddrspace: Tracks git repository HEAD for active development coordination
• memory_addr: Uses stable crate version (0.3) for mature address utilities

External Dependencies

• serde: Pinned to specific version (1.0.204) for build reproducibility
• cfg-if: Uses caret range (1.0) allowing compatible updates

This strategy balances stability for core functionality with active development tracking for hypervisor-specific components.

Sources: axdevice_base/Cargo.toml(L11 - L16) 

ArceOS Integration

Relevant source files

• axdevice_base/Cargo.toml
• axdevice_base/src/lib.rs

This page documents how axdevice_base integrates with the ArceOS hypervisor ecosystem through its core dependencies: axerrno, axaddrspace, and memory_addr. These dependencies enable standardized error handling, guest physical address management, and memory range abstractions essential for device emulation in the hypervisor context.

For information about the core device abstraction patterns, see Core Architecture. For implementation details of the traits and types, see BaseDeviceOps Trait.

ArceOS Dependency Overview

The axdevice_base crate relies on three key ArceOS ecosystem dependencies that provide fundamental hypervisor capabilities. These dependencies are tightly integrated into the BaseDeviceOps trait interface to ensure consistent device emulation across the ArceOS hypervisor.

Dependency Architecture

flowchart TD
subgraph subGraph2["External Dependencies"]
    SE["serde"]
    CF["cfg-if"]
    AL["alloc"]
end
subgraph subGraph1["ArceOS Core Dependencies"]
    AXE["axerrno::AxResult<T>"]
    AXA["axaddrspace::GuestPhysAddr"]
    MA["memory_addr::AddrRange<T>"]
end
subgraph axdevice_base["axdevice_base"]
    BDO["BaseDeviceOps trait"]
    AR["address_range()"]
    HR["handle_read()"]
    HW["handle_write()"]
    ET["emu_type()"]
end

AR --> AXA
AR --> MA
BDO --> AL
BDO --> AR
BDO --> CF
BDO --> ET
BDO --> HR
BDO --> HW
BDO --> SE
HR --> AXA
HR --> AXE
HW --> AXA

Sources: axdevice_base/src/lib.rs(L11 - L14)  axdevice_base/Cargo.toml(L10 - L16) 

Dependency Declaration

The dependencies are declared in the crate's Cargo.toml with specific version requirements:

Sources: axdevice_base/Cargo.toml(L14 - L16) 

Error Handling Integration:axerrno

The axerrno crate provides standardized error handling across the ArceOS ecosystem. In axdevice_base, it's used exclusively in the handle_read method of the BaseDeviceOps trait to indicate success or failure of read operations.

AxResult Usage Pattern

flowchart TD
subgraph subGraph1["AxResult Variants"]
    OK["Ok(data: usize)"]
    ERR["Err(AxError)"]
end
subgraph subGraph0["Device Read Operation Flow"]
    GVM["Guest VM Memory Access"]
    HV["ArceOS Hypervisor"]
    DEV["Device Implementation"]
    RES["AxResult<usize>"]
end

DEV --> RES
GVM --> HV
HV --> DEV
HV --> GVM
RES --> ERR
RES --> HV
RES --> OK

The handle_read method signature demonstrates this integration:

#![allow(unused)]
fn main() {
fn handle_read(&self, addr: GuestPhysAddr, width: usize) -> AxResult<usize>
}

This design allows device implementations to return either:

• Ok(data) - successful read with the actual data value
• Err(error) - standardized error indicating why the read failed

Sources: axdevice_base/src/lib.rs(L27) 

Address Space Integration:axaddrspace

The axaddrspace crate provides hypervisor-specific address space management types, particularly GuestPhysAddr which represents guest physical addresses in the virtualized environment.

GuestPhysAddr in BaseDeviceOps

The GuestPhysAddr type appears in three critical methods of the BaseDeviceOps trait:

flowchart TD
subgraph subGraph2["Hypervisor Context"]
    GAS["Guest Address Space"]
    EMU["Device Emulation"]
    MMU["Memory Management"]
end
subgraph subGraph1["GuestPhysAddr Usage"]
    ART["AddrRange<GuestPhysAddr>"]
    RPA["Read Parameter Address"]
    WPA["Write Parameter Address"]
end
subgraph subGraph0["BaseDeviceOps Methods Using GuestPhysAddr"]
    AR["address_range()"]
    HR["handle_read(addr: GuestPhysAddr, ...)"]
    HW["handle_write(addr: GuestPhysAddr, ...)"]
end

AR --> ART
ART --> GAS
EMU --> MMU
GAS --> MMU
HR --> RPA
HW --> WPA
RPA --> EMU
WPA --> EMU

The integration enables:

• Address Range Definition: address_range() returns AddrRange<GuestPhysAddr> to define the guest physical memory region this device occupies
• Read Access: handle_read() receives a GuestPhysAddr parameter indicating which guest physical address is being read
• Write Access: handle_write() receives a GuestPhysAddr parameter indicating which guest physical address is being written

Sources: axdevice_base/src/lib.rs(L25 - L29) 

Memory Range Integration:memory_addr

The memory_addr crate provides the AddrRange<T> generic type used to represent contiguous memory address ranges. In axdevice_base, it's specialized with GuestPhysAddr to define device memory regions.

AddrRange Specialization

flowchart TD
subgraph subGraph2["Device Memory Mapping"]
    DEV1["Console Device Range"]
    DEV2["GICD Device Range"]
    DEV3["Virtio Device Range"]
    DEVN["Other Device Ranges"]
end
subgraph subGraph1["axdevice_base Specialization"]
    GPAR["AddrRange<GuestPhysAddr>"]
    BDO["BaseDeviceOps::address_range()"]
end
subgraph subGraph0["memory_addr Crate"]
    AR["AddrRange<T>"]
    GEN["Generic Address Range"]
end

AR --> GPAR
BDO --> DEV1
BDO --> DEV2
BDO --> DEV3
BDO --> DEVN
GPAR --> BDO

This specialization allows each device implementation to precisely define:

• Start Address: Beginning of the device's guest physical address range
• Size/End Address: Extent of the device's memory-mapped region
• Bounds Checking: Automatic validation that access addresses fall within the device's range

The address_range() method is fundamental to the hypervisor's memory management, enabling it to route guest memory accesses to the appropriate emulated device based on the target address.

Sources: axdevice_base/src/lib.rs(L11 - L25) 

Integration Patterns in BaseDeviceOps

The BaseDeviceOps trait demonstrates a cohesive integration pattern where each ArceOS dependency serves a specific role in the device emulation interface:

Method Integration Matrix

Trait Method Signatures

The complete trait definition shows the integration points:

#![allow(unused)]
fn main() {
pub trait BaseDeviceOps {
    fn emu_type(&self) -> EmuDeviceType;
    fn address_range(&self) -> AddrRange<GuestPhysAddr>;      // memory_addr + axaddrspace
    fn handle_read(&self, addr: GuestPhysAddr, width: usize) -> AxResult<usize>;  // axaddrspace + axerrno
    fn handle_write(&self, addr: GuestPhysAddr, width: usize, val: usize);        // axaddrspace
}
}

This design ensures that device implementations can seamlessly integrate with the ArceOS hypervisor's memory management and error handling systems.

Sources: axdevice_base/src/lib.rs(L21 - L30) 

Hypervisor Ecosystem Context

The ArceOS integration dependencies position axdevice_base as a bridge between the hypervisor core and device-specific implementations. This integration enables:

• Unified Address Space: All devices operate within the same GuestPhysAddr address space managed by the hypervisor
• Consistent Error Handling: Device operations return standardized AxResult types that the hypervisor can process uniformly
• Memory Region Management: Device address ranges integrate with the hypervisor's memory management unit for efficient routing

The dependency choices reflect ArceOS's design philosophy of providing specialized, composable components for hypervisor functionality while maintaining no_std compatibility for embedded and constrained environments.

Sources: axdevice_base/src/lib.rs(L1 - L14)  axdevice_base/Cargo.toml(L14 - L16) 

Development Guide

Relevant source files

• .github/workflows/ci.yml
• .gitignore

This document provides practical information for developers working with or extending the axdevice_crates system. It covers the development environment setup, build processes, code quality standards, and contribution workflows specific to this hypervisor device emulation framework.

For detailed information about implementing specific device types, see Implementing New Devices. For comprehensive build system documentation, see Build System and CI.

Development Environment Requirements

The axdevice_crates project requires a specific development environment configuration to support its no_std embedded target and multi-architecture requirements.

Rust Toolchain Configuration

The project exclusively uses the nightly Rust toolchain to access cutting-edge features required for hypervisor development and no_std environments. The required toolchain components are:

Supported Target Architectures

The codebase maintains compatibility across four distinct target architectures, reflecting the diverse embedded and hypervisor deployment scenarios:

flowchart TD
subgraph subGraph3["Development Capabilities"]
    UT["Unit Testing"]
    DOC["Documentation Generation"]
    BE["Bare Metal Execution"]
    HV["Hypervisor Integration"]
end
subgraph subGraph2["Target Architecture Matrix"]
    subgraph subGraph1["Bare Metal Targets"]
        T2["x86_64-unknown-none"]
        T3["riscv64gc-unknown-none-elf"]
        T4["aarch64-unknown-none-softfloat"]
    end
    subgraph subGraph0["Hosted Environment"]
        T1["x86_64-unknown-linux-gnu"]
    end
end

T1 --> DOC
T1 --> UT
T2 --> BE
T2 --> HV
T3 --> BE
T3 --> HV
T4 --> BE
T4 --> HV

Sources: .github/workflows/ci.yml(L12)  .github/workflows/ci.yml(L19) 

Development Workflow Pipeline

The development process follows a strict automated pipeline that enforces code quality and multi-architecture compatibility:

flowchart TD
subgraph subGraph1["Documentation Workflow"]
    DOC["Documentation Pipeline"]
    DOCBUILD["cargo doc --no-deps --all-features"]
    DOCCHECK["RUSTDOCFLAGS Validation"]
    DEPLOY["GitHub Pages Deployment"]
end
subgraph subGraph0["Parallel Architecture Builds"]
    BUILD["cargo build --target TARGET --all-features"]
    B1["Build x86_64-unknown-linux-gnu"]
    B2["Build x86_64-unknown-none"]
    B3["Build riscv64gc-unknown-none-elf"]
    B4["Build aarch64-unknown-none-softfloat"]
end
PR["Pull Request / Push"]
CI["CI Pipeline Trigger"]
FMT["cargo fmt --all --check"]
CLIPPY["cargo clippy --target TARGET --all-features"]
TEST["cargo test --target x86_64-unknown-linux-gnu"]
SUCCESS["Pipeline Success"]

B1 --> SUCCESS
B2 --> SUCCESS
B3 --> SUCCESS
B4 --> SUCCESS
BUILD --> B1
BUILD --> B2
BUILD --> B3
BUILD --> B4
BUILD --> TEST
CI --> FMT
CLIPPY --> BUILD
DEPLOY --> SUCCESS
DOC --> DOCBUILD
DOCBUILD --> DOCCHECK
DOCCHECK --> DEPLOY
FMT --> CLIPPY
PR --> CI
TEST --> DOC

Sources: .github/workflows/ci.yml(L22 - L30)  .github/workflows/ci.yml(L44 - L48) 

Code Quality Standards

Formatting Requirements

All code must pass cargo fmt --all --check without modifications. The project uses the standard Rust formatting conventions with no custom overrides.

Linting Configuration

The Clippy linting process runs with --all-features enabled and includes one specific allowance:

• clippy::new_without_default is explicitly allowed via -A clippy::new_without_default

This exception acknowledges that hypervisor device emulation patterns may require new() methods without corresponding Default implementations due to the specialized initialization requirements of emulated hardware.

Documentation Standards

Documentation generation enforces strict quality standards through RUSTDOCFLAGS:

RUSTDOCFLAGS: -D rustdoc::broken_intra_doc_links -D missing-docs

These flags ensure:

• Broken intra-doc links: All internal documentation links must resolve correctly
• Missing docs: All public APIs must include documentation

Sources: .github/workflows/ci.yml(L25)  .github/workflows/ci.yml(L40) 

Testing Strategy

Unit Testing Scope

Unit tests execute exclusively on the x86_64-unknown-linux-gnu target, which provides the hosted environment necessary for test infrastructure. The testing command includes --nocapture to ensure full test output visibility.

Multi-Architecture Validation

While unit tests run only on the hosted target, build verification occurs across all four supported architectures. This approach ensures:

• Code compiles correctly for all deployment targets
• no_std constraints are properly maintained
• Architecture-specific conditional compilation works correctly

Test Execution Conditions

Tests run conditionally based on the target architecture:

if: ${{ matrix.targets == 'x86_64-unknown-linux-gnu' }}

This constraint reflects the practical limitation that bare-metal targets cannot execute standard Rust test infrastructure.

Sources: .github/workflows/ci.yml(L28 - L30) 

Documentation Workflow

Automated Documentation Generation

The documentation pipeline operates independently from the main CI pipeline and includes sophisticated deployment automation:

flowchart TD
subgraph subGraph2["Deployment Conditions"]
    DC1["Default Branch Only"]
    DC2["Single Commit Strategy"]
    DC3["gh-pages Branch Target"]
end
subgraph subGraph1["Quality Controls"]
    BIL["Broken Intra-doc Links Check"]
    MD["Missing Docs Check"]
end
subgraph subGraph0["Documentation Generation Process"]
    SRC["Source Code"]
    DOCGEN["cargo doc --no-deps --all-features"]
    VALIDATE["RUSTDOCFLAGS Validation"]
    INDEX["Auto-generated index.html"]
    DEPLOY["GitHub Pages Deployment"]
end

DEPLOY --> DC1
DEPLOY --> DC2
DEPLOY --> DC3
DOCGEN --> VALIDATE
INDEX --> DEPLOY
SRC --> DOCGEN
VALIDATE --> BIL
VALIDATE --> INDEX
VALIDATE --> MD

Index Generation Logic

The documentation system automatically generates a redirect index using shell commands:

printf '<meta http-equiv="refresh" content="0;url=%s/index.html">' $(cargo tree | head -1 | cut -d' ' -f1) > target/doc/index.html

This creates an automatic redirect from the documentation root to the primary crate's documentation.

Deployment Permissions

The documentation job requires contents: write permissions to deploy to GitHub Pages, reflecting the automated nature of the documentation publication process.

Sources: .github/workflows/ci.yml(L44 - L48)  .github/workflows/ci.yml(L50 - L55) 

Development Environment Setup

Repository Setup

# Clone the repository
git clone https://github.com/arceos-hypervisor/axdevice_crates
cd axdevice_crates

# Install nightly toolchain with required components
rustup toolchain install nightly
rustup default nightly
rustup component add rust-src clippy rustfmt

# Add required targets
rustup target add x86_64-unknown-none
rustup target add riscv64gc-unknown-none-elf  
rustup target add aarch64-unknown-none-softfloat

Verification Commands

Verify your development environment matches CI requirements:

# Check formatting
cargo fmt --all -- --check

# Run linting
cargo clippy --all-features -- -A clippy::new_without_default

# Test build for all targets
cargo build --target x86_64-unknown-linux-gnu --all-features
cargo build --target x86_64-unknown-none --all-features
cargo build --target riscv64gc-unknown-none-elf --all-features
cargo build --target aarch64-unknown-none-softfloat --all-features

# Run tests
cargo test --target x86_64-unknown-linux-gnu -- --nocapture

# Generate documentation
cargo doc --no-deps --all-features

Sources: .github/workflows/ci.yml(L15 - L19)  .github/workflows/ci.yml(L22 - L30) 

Build System and CI

Relevant source files

• .github/workflows/ci.yml

This page documents the automated build system and continuous integration (CI) pipeline for the axdevice_crates repository. It covers the multi-architecture build strategy, code quality enforcement, testing procedures, and documentation generation workflow that ensures the codebase maintains compatibility across different target platforms in the no_std embedded environment.

For information about implementing new devices within this build system, see Implementing New Devices. For details about the core architecture that this build system validates, see Core Architecture.

CI Pipeline Overview

The repository uses GitHub Actions to implement a comprehensive CI pipeline that validates code across multiple architectures and maintains documentation quality. The pipeline consists of two primary jobs: ci for code validation and doc for documentation generation.

CI Job Architecture

flowchart TD
subgraph subGraph2["Build Steps"]
    CHECKOUT["actions/checkout@v4"]
    TOOLCHAIN["dtolnay/rust-toolchain@nightly"]
    VERSION["rustc --version --verbose"]
    FMT["cargo fmt --all -- --check"]
    CLIPPY["cargo clippy --target TARGET --all-features"]
    BUILD["cargo build --target TARGET --all-features"]
    TEST["cargo test --target x86_64-unknown-linux-gnu"]
end
subgraph subGraph1["CI Job Matrix"]
    RUST["rust-toolchain: nightly"]
    T1["x86_64-unknown-linux-gnu"]
    T2["x86_64-unknown-none"]
    T3["riscv64gc-unknown-none-elf"]
    T4["aarch64-unknown-none-softfloat"]
end
subgraph subGraph0["GitHub Actions Triggers"]
    PUSH["push events"]
    PR["pull_request events"]
end

BUILD --> TEST
CHECKOUT --> TOOLCHAIN
CLIPPY --> BUILD
FMT --> CLIPPY
PR --> RUST
PUSH --> RUST
RUST --> T1
RUST --> T2
RUST --> T3
RUST --> T4
T1 --> CHECKOUT
T2 --> CHECKOUT
T3 --> CHECKOUT
T4 --> CHECKOUT
TOOLCHAIN --> VERSION
VERSION --> FMT

Sources: .github/workflows/ci.yml(L1 - L31) 

Documentation Job Pipeline

flowchart TD
subgraph subGraph2["Deployment Conditions"]
    MAIN_BRANCH["github.ref == default-branch"]
    GH_PAGES["branch: gh-pages"]
    DOC_FOLDER["folder: target/doc"]
end
subgraph subGraph1["Environment Configuration"]
    RUSTDOCFLAGS["-D rustdoc::broken_intra_doc_links -D missing-docs"]
    DEFAULT_BRANCH["refs/heads/default_branch"]
    PERMISSIONS["contents: write"]
end
subgraph subGraph0["Documentation Generation"]
    DOC_CHECKOUT["actions/checkout@v4"]
    DOC_TOOLCHAIN["dtolnay/rust-toolchain@nightly"]
    DOC_BUILD["cargo doc --no-deps --all-features"]
    INDEX_GEN["Generate target/doc/index.html redirect"]
    PAGES_DEPLOY["JamesIves/github-pages-deploy-action@v4"]
end

DEFAULT_BRANCH --> MAIN_BRANCH
DOC_BUILD --> INDEX_GEN
DOC_CHECKOUT --> DOC_TOOLCHAIN
DOC_TOOLCHAIN --> DOC_BUILD
INDEX_GEN --> MAIN_BRANCH
MAIN_BRANCH --> PAGES_DEPLOY
PAGES_DEPLOY --> DOC_FOLDER
PAGES_DEPLOY --> GH_PAGES
PERMISSIONS --> PAGES_DEPLOY
RUSTDOCFLAGS --> DOC_BUILD

Sources: .github/workflows/ci.yml(L32 - L56) 

Multi-Architecture Build Strategy

The CI pipeline validates compatibility across four distinct target architectures, each serving different use cases in the hypervisor ecosystem:

Build Matrix Configuration

The build matrix uses a fail-fast: false strategy to ensure all target architectures are tested independently, preventing early termination when one target fails.

flowchart TD
subgraph subGraph1["Build Validation Steps"]
    VERSION_CHECK["rustc --version --verbose"]
    FORMAT_CHECK["cargo fmt --all -- --check"]
    LINT_CHECK["cargo clippy --target TARGET --all-features"]
    BUILD_CHECK["cargo build --target TARGET --all-features"]
    UNIT_TEST["cargo test (x86_64-linux only)"]
end
subgraph subGraph0["Rust Toolchain Setup"]
    NIGHTLY["nightly toolchain"]
    COMPONENTS["rust-src, clippy, rustfmt"]
    TARGETS["matrix.targets"]
end

BUILD_CHECK --> UNIT_TEST
COMPONENTS --> FORMAT_CHECK
COMPONENTS --> LINT_CHECK
FORMAT_CHECK --> LINT_CHECK
LINT_CHECK --> BUILD_CHECK
NIGHTLY --> VERSION_CHECK
TARGETS --> BUILD_CHECK
TARGETS --> LINT_CHECK
VERSION_CHECK --> FORMAT_CHECK

Sources: .github/workflows/ci.yml(L8 - L19)  .github/workflows/ci.yml(L25 - L30) 

Code Quality Enforcement

The CI pipeline enforces multiple layers of code quality validation before any changes can be merged.

Formatting and Linting

The Clippy configuration specifically allows the clippy::new_without_default lint, indicating that the codebase may contain new() methods without corresponding Default implementations, which is common in no_std environments.

Sources: .github/workflows/ci.yml(L22 - L25)  .github/workflows/ci.yml(L40) 

Testing Strategy

The testing approach is pragmatic, recognizing the constraints of no_std embedded development:

Unit Testing Limitations

flowchart TD
subgraph subGraph2["Quality Assurance"]
    CLIPPY_ALL["Clippy runs on all targets"]
    FORMAT_ALL["Format check runs on all targets"]
    COMPILE_ALL["Compilation check on all targets"]
end
subgraph subGraph1["Test Execution"]
    LINUX_ONLY["Unit tests run only on x86_64-unknown-linux-gnu"]
    TEST_CMD["cargo test --target x86_64-unknown-linux-gnu -- --nocapture"]
    BUILD_VALIDATION["Build validation on all targets"]
end
subgraph subGraph0["Testing Constraints"]
    NOSTD["no_std environment"]
    EMBEDDED["Embedded targets"]
    LIMITED["Limited std library access"]
end

BUILD_VALIDATION --> COMPILE_ALL
CLIPPY_ALL --> FORMAT_ALL
COMPILE_ALL --> CLIPPY_ALL
EMBEDDED --> BUILD_VALIDATION
LIMITED --> LINUX_ONLY
LINUX_ONLY --> TEST_CMD
NOSTD --> LINUX_ONLY

Unit tests execute only on x86_64-unknown-linux-gnu because the other targets are bare metal environments that lack the standard library infrastructure required for Rust's test framework. However, the build validation ensures that the code compiles correctly for all target architectures.

Sources: .github/workflows/ci.yml(L28 - L30) 

Documentation Generation and Deployment

The documentation system automatically generates and deploys API documentation using a dedicated job that runs in parallel with the main CI validation.

Documentation Build Process

The documentation generation uses strict flags to ensure high-quality documentation:

flowchart TD
subgraph Deployment["Deployment"]
    BRANCH_CHECK["github.ref == default-branch"]
    DEPLOY_ACTION["JamesIves/github-pages-deploy-action@v4"]
    SINGLE_COMMIT["single-commit: true"]
    GH_PAGES_BRANCH["branch: gh-pages"]
    DOC_FOLDER["folder: target/doc"]
end
subgraph subGraph1["Build Process"]
    DOC_CMD["cargo doc --no-deps --all-features"]
    INDEX_CREATE["printf redirect to generated docs"]
    TREE_PARSE["cargo tree | head -1 | cut -d' ' -f1"]
end
subgraph subGraph0["Documentation Flags"]
    BROKEN_LINKS["-D rustdoc::broken_intra_doc_links"]
    MISSING_DOCS["-D missing-docs"]
    RUSTDOCFLAGS["RUSTDOCFLAGS environment"]
end

BRANCH_CHECK --> DEPLOY_ACTION
BROKEN_LINKS --> RUSTDOCFLAGS
DEPLOY_ACTION --> DOC_FOLDER
DEPLOY_ACTION --> GH_PAGES_BRANCH
DEPLOY_ACTION --> SINGLE_COMMIT
DOC_CMD --> TREE_PARSE
INDEX_CREATE --> BRANCH_CHECK
MISSING_DOCS --> RUSTDOCFLAGS
RUSTDOCFLAGS --> DOC_CMD
TREE_PARSE --> INDEX_CREATE

The --no-deps flag ensures that only the crate's own documentation is generated, not its dependencies. The automatic index.html generation creates a redirect to the main crate documentation.

Sources: .github/workflows/ci.yml(L40)  .github/workflows/ci.yml(L44 - L48)  .github/workflows/ci.yml(L49 - L55) 

Development Workflow Integration

The CI system is designed to support the typical development workflow for embedded hypervisor development:

Branch Protection and Quality Gates

Error Handling Strategy

The documentation job uses conditional error handling with continue-on-error set based on branch and event type, allowing documentation builds to fail on non-main branches and non-pull-request events without blocking the overall workflow.

flowchart TD
subgraph subGraph1["Build Outcomes"]
    MAIN_BRANCH_STRICT["Main branch: strict documentation"]
    PR_STRICT["Pull requests: strict documentation"]
    OTHER_LENIENT["Other contexts: lenient documentation"]
end
subgraph subGraph0["Error Handling Logic"]
    BRANCH_CHECK["github.ref != default-branch"]
    EVENT_CHECK["github.event_name != pull_request"]
    CONTINUE_ERROR["continue-on-error: true"]
    STRICT_MODE["continue-on-error: false"]
end

BRANCH_CHECK --> CONTINUE_ERROR
CONTINUE_ERROR --> OTHER_LENIENT
EVENT_CHECK --> CONTINUE_ERROR
STRICT_MODE --> MAIN_BRANCH_STRICT
STRICT_MODE --> PR_STRICT

This approach ensures that documentation quality is enforced where it matters most (main branch and pull requests) while allowing experimental branches to have temporary documentation issues.

Sources: .github/workflows/ci.yml(L45)  .github/workflows/ci.yml(L39) 

Implementing New Devices

Relevant source files

• axdevice_base/src/emu_type.rs
• axdevice_base/src/lib.rs

This page provides a comprehensive guide for implementing new emulated devices within the ArceOS hypervisor ecosystem using the BaseDeviceOps trait foundation. It covers the step-by-step process of creating device implementations, configuring address ranges, implementing read/write handlers, and integrating with the device type system.

For information about the core architecture and trait definitions, see Core Architecture. For details about the device type enumeration system, see Device Type System.

Implementation Overview

Creating a new emulated device requires implementing the BaseDeviceOps trait and integrating with the EmuDeviceType system. The implementation process follows a standardized pattern that ensures compatibility with the ArceOS hypervisor's device emulation framework.

Device Implementation Architecture

flowchart TD
subgraph subGraph2["Core Types"]
    T1["EmuDeviceType"]
    T2["AddrRange"]
    T3["AxResult"]
    T4["GuestPhysAddr"]
end
subgraph subGraph1["BaseDeviceOps Methods"]
    M1["emu_type()"]
    M2["address_range()"]
    M3["handle_read()"]
    M4["handle_write()"]
end
subgraph subGraph0["Implementation Steps"]
    S1["Unsupported markdown: list"]
    S2["Unsupported markdown: list"]
    S3["Unsupported markdown: list"]
    S4["Unsupported markdown: list"]
    S5["Unsupported markdown: list"]
    S6["Unsupported markdown: list"]
end

M1 --> T1
M2 --> T2
M3 --> T3
M4 --> T4
S1 --> S2
S2 --> M1
S2 --> M2
S2 --> M3
S2 --> M4
S2 --> S3
S3 --> S4
S4 --> S5
S5 --> S6

Sources: axdevice_base/src/lib.rs(L20 - L30)  axdevice_base/src/emu_type.rs(L3 - L28) 

Device Structure Definition

The first step in implementing a new device is defining the device structure that will hold the device's state and configuration. The structure should contain all necessary data for device emulation, including memory mappings, configuration registers, and operational state.

Device Implementation Pattern

classDiagram
class BaseDeviceOps {
    <<trait>>
    
    +emu_type() EmuDeviceType
    +address_range() AddrRange~GuestPhysAddr~
    +handle_read(addr, width) AxResult~usize~
    +handle_write(addr, width, val)
}

class YourNewDevice {
    -base_address: GuestPhysAddr
    -size: usize
    -registers: DeviceRegisters
    -state: DeviceState
    +new(base_addr, size) Self
    +reset()
    +configure()
}

class DeviceRegisters {
    -control_reg: u32
    -status_reg: u32
    -data_reg: u32
    -interrupt_reg: u32
    
}

class DeviceState {
    -enabled: bool
    -interrupt_pending: bool
    -operation_mode: OperationMode
    
}

class EmuDeviceType {
    <<enum>>
    EmuDeviceTConsole
    EmuDeviceTGicdV2
    EmuDeviceTVirtioBlk
    "... other types"
    YourNewDeviceType
    
}

YourNewDevice  ..|>  BaseDeviceOps : implements
YourNewDevice  -->  DeviceRegisters : contains
YourNewDevice  -->  DeviceState : contains
YourNewDevice  -->  EmuDeviceType : returns via emu_type()

Sources: axdevice_base/src/lib.rs(L21 - L30)  axdevice_base/src/emu_type.rs(L5 - L28) 

BaseDeviceOps Implementation

The core of device implementation involves implementing the four required methods of the BaseDeviceOps trait. Each method serves a specific purpose in the device emulation lifecycle.

Required Method Implementation

The emu_type() method returns the device's type identifier from the EmuDeviceType enumeration. This method enables the hypervisor to identify and categorize the device for management purposes.

The address_range() method defines the guest physical address range that the device occupies in the virtual machine's memory space. This range determines which memory accesses will be routed to the device's read and write handlers.

Sources: axdevice_base/src/lib.rs(L22 - L29) 

Address Range Configuration

Device address range configuration requires careful consideration of the guest physical address space layout and potential conflicts with other devices or memory regions. The address range must be properly aligned and sized according to the device's requirements.

Address Range Management Flow

sequenceDiagram
    participant ArceOSHypervisor as "ArceOS Hypervisor"
    participant AddressSpaceManager as "Address Space Manager"
    participant YourNewDevice as "YourNewDevice"
    participant AddrRangeGuestPhysAddr as "AddrRange<GuestPhysAddr>"

    Note over ArceOSHypervisor,AddrRangeGuestPhysAddr: Device Registration Flow
    ArceOSHypervisor ->> YourNewDevice: address_range()
    YourNewDevice ->> AddrRangeGuestPhysAddr: create range
    Note over AddrRangeGuestPhysAddr: base_address: GuestPhysAddr<br>size: usize
    AddrRangeGuestPhysAddr ->> YourNewDevice: AddrRange instance
    YourNewDevice ->> ArceOSHypervisor: return address range
    ArceOSHypervisor ->> AddressSpaceManager: register_device_range()
    AddressSpaceManager ->> AddressSpaceManager: validate no conflicts
    AddressSpaceManager ->> ArceOSHypervisor: registration result
    Note over ArceOSHypervisor,AddrRangeGuestPhysAddr: Memory Access Flow
    ArceOSHypervisor ->> AddressSpaceManager: guest_memory_access(gpa)
    AddressSpaceManager ->> AddressSpaceManager: check address ranges
    alt Address in device range
        AddressSpaceManager ->> ArceOSHypervisor: route to device
        ArceOSHypervisor ->> YourNewDevice: handle_read/write()
    else Address not in range
        AddressSpaceManager ->> ArceOSHypervisor: handle as memory
    end

Sources: axdevice_base/src/lib.rs(L24 - L25) 

The address range should be configured during device initialization and remain constant throughout the device's lifetime. Dynamic address range changes are not supported by the current architecture.

Read Handler Implementation

The handle_read() method processes guest read operations targeting the device's address range. The implementation must decode the address offset, validate the access width, and return appropriate data based on the device's current state.

Read Handler Patterns

flowchart TD
subgraph subGraph2["Error Handling"]
    E1["Invalid Address"]
    E2["Unsupported Width"]
    E3["Device Not Ready"]
    E4["Permission Denied"]
end
subgraph subGraph1["Common Read Patterns"]
    P1["Register Bank Access"]
    P2["FIFO/Buffer Reading"]
    P3["Status Register Polling"]
    P4["Interrupt Status Check"]
end
subgraph subGraph0["handle_read() Implementation Flow"]
    A["Guest Read Access"]
    B["Validate Address Range"]
    C["Calculate Register Offset"]
    D["Validate Access Width"]
    E["Read Register Value"]
    F["Apply Device Logic"]
    G["Return AxResult"]
end

A --> B
B --> C
C --> D
D --> E
D --> E1
D --> E2
E --> E3
E --> F
F --> E4
F --> G
F --> P1
F --> P2
F --> P3
F --> P4

Sources: axdevice_base/src/lib.rs(L27) 

The read handler should support standard access widths (1, 2, 4, and 8 bytes) and return data in the appropriate format. Register values should be calculated based on the device's current state and any side effects of the read operation should be applied.

Write Handler Implementation

The handle_write() method processes guest write operations to the device's address range. Unlike read operations, write handlers do not return data but may trigger device state changes, interrupt generation, or other side effects.

Write Handler Architecture

flowchart TD
subgraph subGraph2["Side Effects"]
    S1["Interrupt Generation"]
    S2["DMA Transfer Start"]
    S3["Device Reset"]
    S4["Mode Change"]
end
subgraph subGraph1["Write Operations"]
    O1["Control Register Write"]
    O2["Data Buffer Write"]
    O3["Configuration Update"]
    O4["Command Execution"]
end
subgraph subGraph0["handle_write() Processing"]
    W1["Guest Write Access"]
    W2["Validate Address Range"]
    W3["Calculate Register Offset"]
    W4["Validate Access Width"]
    W5["Extract Write Value"]
    W6["Apply Register Logic"]
    W7["Update Device State"]
    W8["Trigger Side Effects"]
end

W1 --> W2
W2 --> W3
W3 --> W4
W4 --> W5
W5 --> W6
W6 --> O1
W6 --> O2
W6 --> O3
W6 --> O4
W6 --> W7
W7 --> W8
W8 --> S1
W8 --> S2
W8 --> S3
W8 --> S4

Sources: axdevice_base/src/lib.rs(L29) 

Write handlers should validate input data, apply appropriate bit masks for register fields, and handle write-only or read-only register access restrictions. The handler should also manage any required atomicity or ordering constraints.

Device Type Selection

Choosing the appropriate EmuDeviceType is crucial for proper device categorization and management. The device type affects removability, hypervisor behavior, and integration with other system components.

Device Type Categories

For new device types not covered by existing categories, the EmuDeviceType enumeration must be extended with appropriate variants. The removable() method implementation should be updated to reflect the new device's characteristics.

Sources: axdevice_base/src/emu_type.rs(L5 - L28)  axdevice_base/src/emu_type.rs(L52 - L64) 

Implementation Best Practices

State Management

Device state should be properly encapsulated and protected against concurrent access. Use appropriate synchronization primitives when device state may be accessed from multiple contexts.

Error Handling

All operations that can fail should return appropriate AxResult types. Use specific error codes from the axerrno crate to indicate different failure conditions.

Memory Access Validation

Always validate guest physical addresses against the device's configured address range and ensure access widths are supported by the target registers.

Performance Considerations

Minimize computational overhead in read and write handlers, as these methods are called frequently during device emulation. Cache frequently accessed values and avoid unnecessary calculations.

Testing Strategy

Implement comprehensive test coverage for all register access patterns, error conditions, and state transitions. Verify compatibility with the target guest operating systems and device drivers.

Sources: axdevice_base/src/lib.rs(L1 - L31)  axdevice_base/src/emu_type.rs(L1 - L84) 

Overview

Relevant source files

• Cargo.toml
• README.md
• src/lib.rs

This document provides a comprehensive overview of the arm_vcpu hypervisor system, an AArch64 virtualization implementation that provides virtual CPU management and hardware abstraction for hypervisor environments. The system implements low-level virtualization primitives including CPU context switching, exception handling, and guest state management for AArch64 platforms.

The arm_vcpu crate serves as a foundational component for hypervisor implementations, providing the core VCPU abstraction and associated hardware interfaces. For detailed information about specific subsystems, see Virtual CPU Management, Exception Handling System, and System Integration.

System Purpose and Scope

The arm_vcpu system implements virtualization support for AArch64 architecture, providing:

• Virtual CPU Management: Core VCPU lifecycle, state management, and execution control
• Hardware Abstraction: Platform-independent interfaces for virtualization hardware features
• Exception Handling: Comprehensive trap and interrupt handling for guest VMs
• Context Switching: Efficient state preservation and restoration between host and guest execution
• Security Integration: Secure Monitor Call (SMC) interface for trusted firmware interaction

Sources: README.md(L1 - L5)  src/lib.rs(L1 - L33) 

Primary System Components

The following diagram illustrates the main architectural components and their relationships within the arm_vcpu system:

Core Component Architecture

flowchart TD
subgraph subGraph2["External Dependencies"]
    AxVCpu["axvcpuCore VCPU Traits"]
    AxAddrSpace["axaddrspaceAddress Space Management"]
    PerCpuCrate["percpuPer-CPU Storage"]
    Aarch64Cpu["aarch64-cpuRegister Definitions"]
end
subgraph subGraph1["Internal Modules"]
    ContextFrame["context_frameState Structures"]
    Exception["exceptionHandler Logic"]
    ExceptionUtils["exception_utilsRegister Parsing"]
    PCpuMod["pcpuPer-CPU Implementation"]
    SMC["smcSecure Monitor Interface"]
    VCpuMod["vcpuVCPU Implementation"]
end
subgraph subGraph0["Primary Interfaces"]
    VCpu["Aarch64VCpuMain VCPU Implementation"]
    PerCpu["Aarch64PerCpuPer-CPU State Management"]
    TrapFrame["TrapFrame(Aarch64ContextFrame)"]
    Config["Aarch64VCpuCreateConfigVCPU Configuration"]
end

Exception --> Aarch64Cpu
Exception --> ExceptionUtils
PerCpu --> PCpuMod
PerCpu --> PerCpuCrate
TrapFrame --> ContextFrame
VCpu --> AxAddrSpace
VCpu --> AxVCpu
VCpu --> Config
VCpu --> PerCpu
VCpu --> TrapFrame
VCpu --> VCpuMod
VCpuMod --> Exception
VCpuMod --> SMC

Sources: src/lib.rs(L9 - L21)  Cargo.toml(L6 - L19) 

Code Entity Mapping

This diagram maps the natural language system concepts to specific code entities and file structures:

Implementation Structure and Code Entities

flowchart TD
subgraph subGraph2["Core Data Structures"]
    Aarch64VCpuStruct["struct Aarch64VCpu"]
    Aarch64PerCpuStruct["struct Aarch64PerCpu"]
    ContextFrameStruct["struct Aarch64ContextFrame"]
    ConfigStruct["struct Aarch64VCpuCreateConfig"]
end
subgraph subGraph1["Source File Organization"]
    VCpuRs["vcpu.rsVirtual CPU Logic"]
    PCpuRs["pcpu.rsPer-CPU Management"]
    ContextRs["context_frame.rsState Structures"]
    ExceptionRs["exception.rsException Handlers"]
    ExceptionS["exception.SAssembly Vectors"]
    UtilsRs["exception_utils.rsRegister Utilities"]
    SmcRs["smc.rsSecure Monitor Calls"]
end
subgraph subGraph0["Public API Surface"]
    LibRs["lib.rsMain Module"]
    PubVCpu["pub use Aarch64VCpu"]
    PubPerCpu["pub use Aarch64PerCpu"]
    PubConfig["pub use Aarch64VCpuCreateConfig"]
    PubTrapFrame["pub type TrapFrame"]
    HwSupport["has_hardware_support()"]
end

ContextRs --> ContextFrameStruct
ExceptionRs --> ExceptionS
ExceptionRs --> UtilsRs
LibRs --> HwSupport
LibRs --> PubConfig
LibRs --> PubPerCpu
LibRs --> PubTrapFrame
LibRs --> PubVCpu
PCpuRs --> Aarch64PerCpuStruct
PubPerCpu --> PCpuRs
PubTrapFrame --> ContextRs
PubVCpu --> VCpuRs
VCpuRs --> Aarch64VCpuStruct
VCpuRs --> ConfigStruct
VCpuRs --> ExceptionRs
VCpuRs --> SmcRs

Sources: src/lib.rs(L17 - L21)  src/lib.rs(L9 - L15) 

Virtualization Hardware Integration

The system provides hardware abstraction for AArch64 virtualization extensions through a layered approach:

Hardware Feature Detection and Support

flowchart TD
subgraph subGraph2["Per-CPU Control"]
    HwSupport["has_hardware_support()Feature Detection"]
    IdRegs["ID_AA64MMFR1_EL1Virtualization Host Extensions"]
    DefaultTrue["Currently returns truePlaceholder Implementation"]
    PerCpuEnable["Aarch64PerCpu::hardware_enable()"]
    PerCpuDisable["Aarch64PerCpu::hardware_disable()"]
    VirtualizationState["Per-CPU Virtualization State"]
end
subgraph subGraph1["Virtualization Extensions"]
    EL2["Exception Level 2Hypervisor Mode"]
    VHE["Virtualization Host ExtensionsEnhanced Host Support"]
    HCR["HCR_EL2 RegisterHypervisor Configuration"]
    VBAR["VBAR_EL2 RegisterVector Base Address"]
end
subgraph subGraph0["Platform Detection"]
    HwSupport["has_hardware_support()Feature Detection"]
    IdRegs["ID_AA64MMFR1_EL1Virtualization Host Extensions"]
    DefaultTrue["Currently returns truePlaceholder Implementation"]
end

HwSupport --> DefaultTrue
HwSupport --> IdRegs
PerCpuEnable --> EL2
PerCpuEnable --> HCR
PerCpuEnable --> VBAR
VirtualizationState --> PerCpuDisable
VirtualizationState --> PerCpuEnable

Sources: src/lib.rs(L23 - L32)  Cargo.toml(L15) 

External Ecosystem Integration

The arm_vcpu crate integrates with the broader ArceOS hypervisor ecosystem through well-defined dependency relationships:

The system operates at Exception Level 2 (EL2) and provides virtualization services for guest operating systems running at Exception Level 1 (EL1) and applications at Exception Level 0 (EL0).

Sources: Cargo.toml(L14 - L19)  README.md(L5) 

System Architecture

Relevant source files

• src/lib.rs
• src/pcpu.rs
• src/vcpu.rs

This document provides a detailed explanation of the overall system architecture for the arm_vcpu hypervisor implementation. It covers the core components, their relationships, and the data flow patterns that enable AArch64 virtualization. For specific implementation details of individual components, see Virtual CPU Management, Exception Handling System, and Context Switching and State Management.

Architecture Overview

The arm_vcpu system implements a Type-1 hypervisor architecture for AArch64 platforms, providing virtualization capabilities through the AArch64 virtualization extensions. The system operates at Exception Level 2 (EL2) and manages guest virtual machines running at EL1/EL0.

Core Component Relationships

Sources: src/vcpu.rs(L39 - L51)  src/pcpu.rs(L10 - L16)  src/lib.rs(L17 - L21) 

System Initialization and Lifecycle

sequenceDiagram
    participant HostSystem as "Host System"
    participant Aarch64PerCpu as "Aarch64PerCpu"
    participant Aarch64VCpu as "Aarch64VCpu"
    participant GuestVM as "Guest VM"

    HostSystem ->> Aarch64PerCpu: new(cpu_id)
    Aarch64PerCpu ->> Aarch64PerCpu: Register IRQ_HANDLER
    HostSystem ->> Aarch64PerCpu: hardware_enable()
    Aarch64PerCpu ->> Aarch64PerCpu: Set VBAR_EL2 to exception_vector_base_vcpu
    Aarch64PerCpu ->> Aarch64PerCpu: Configure HCR_EL2 for virtualization
    HostSystem ->> Aarch64VCpu: new(Aarch64VCpuCreateConfig)
    Aarch64VCpu ->> Aarch64VCpu: Initialize TrapFrame and GuestSystemRegisters
    HostSystem ->> Aarch64VCpu: setup()
    Aarch64VCpu ->> Aarch64VCpu: init_hv() - Configure initial state
    HostSystem ->> Aarch64VCpu: set_entry(guest_entry_point)
    HostSystem ->> Aarch64VCpu: set_ept_root(page_table_root)
    loop VM Execution Cycle
        HostSystem ->> Aarch64VCpu: run()
        Aarch64VCpu ->> Aarch64VCpu: save_host_sp_el0()
        Aarch64VCpu ->> Aarch64VCpu: restore_vm_system_regs()
        Aarch64VCpu ->> GuestVM: Context switch to guest
        GuestVM -->> Aarch64VCpu: VM-Exit (trap/exception)
        Aarch64VCpu ->> Aarch64VCpu: vmexit_handler()
        Aarch64VCpu -->> HostSystem: Return AxVCpuExitReason
    end

Sources: src/vcpu.rs(L69 - L85)  src/vcpu.rs(L99 - L111)  src/pcpu.rs(L49 - L67) 

Core Components

Virtual CPU (Aarch64VCpu)

The Aarch64VCpu<H: AxVCpuHal> structure serves as the primary abstraction for a virtual CPU. It maintains both guest context and runtime state required for virtualization.

The VCPU implements the AxArchVCpu trait, providing standardized interfaces for:

• Creation and configuration via Aarch64VCpuCreateConfig
• Guest execution through the run() method
• Entry point and page table configuration
• Register manipulation interfaces

Sources: src/vcpu.rs(L39 - L51)  src/vcpu.rs(L64 - L124) 

Per-CPU Management (Aarch64PerCpu)

The Aarch64PerCpu<H: AxVCpuHal> structure manages hardware virtualization features on a per-CPU basis:

flowchart TD
PCPU["Aarch64PerCpu"]
HCR["HCR_EL2 RegisterHypervisor Configuration"]
VBAR["VBAR_EL2 RegisterException Vector Base"]
IRQ["IRQ_HANDLERPer-CPU IRQ Dispatch"]
ORIG["ORI_EXCEPTION_VECTOR_BASEOriginal Vector Base"]
VM["VM Bit - Virtualization"]
RW["RW Bit - 64-bit Guest"]
IMO["IMO/FMO - Virtual Interrupts"]
TSC["TSC - SMC Instructions"]

HCR --> IMO
HCR --> RW
HCR --> TSC
HCR --> VM
PCPU --> HCR
PCPU --> IRQ
PCPU --> ORIG
PCPU --> VBAR

Sources: src/pcpu.rs(L10 - L16)  src/pcpu.rs(L49 - L67)  src/pcpu.rs(L18 - L26) 

Hardware Abstraction Layer

The system uses the AxVCpuHal trait to abstract platform-specific functionality:

Sources: src/vcpu.rs(L278)  src/pcpu.rs(L35 - L37) 

Context Switching Architecture

The system implements a sophisticated context switching mechanism that preserves both general-purpose and system registers across VM entries and exits:

flowchart TD
subgraph subGraph2["Guest Context"]
    TRAP["TrapFrame(GPRs, SP_EL0, ELR, SPSR)"]
    GSYS["GuestSystemRegisters(System Control Registers)"]
end
subgraph subGraph1["Context Switch Operations"]
    SAVE_HOST["save_host_sp_el0()"]
    RESTORE_VM["restore_vm_system_regs()"]
    RUN_GUEST["run_guest()(Naked Function)"]
    VMEXIT["vmexit_handler()"]
end
subgraph subGraph0["Host Context"]
    HSP["Host SP_EL0(HOST_SP_EL0 per-CPU)"]
    HSTACK["Host Stack(save_regs_to_stack!)"]
    HSYS["Host System Registers(Original Hardware State)"]
end

GSYS --> VMEXIT
HSP --> SAVE_HOST
RESTORE_VM --> GSYS
RESTORE_VM --> RUN_GUEST
RUN_GUEST --> TRAP
SAVE_HOST --> RESTORE_VM
TRAP --> VMEXIT
VMEXIT --> HSP
VMEXIT --> HSTACK

Sources: src/vcpu.rs(L182 - L214)  src/vcpu.rs(L226 - L244)  src/vcpu.rs(L255 - L282) 

Exception and Interrupt Handling

The system provides a multi-layered exception handling architecture that processes VM-exits and routes them appropriately:

flowchart TD
subgraph subGraph3["Exit Reasons"]
    MMIO["AxVCpuExitReason::MmioRead/Write"]
    EXT_IRQ["AxVCpuExitReason::ExternalInterrupt"]
    SYSREG["AxVCpuExitReason::SystemRegister*"]
    PSCI["AxVCpuExitReason::CpuUp/Down"]
end
subgraph subGraph2["High-Level Dispatch"]
    SYNC_HANDLER["handle_exception_sync()"]
    IRQ_DISPATCH["IRQ_HANDLER per-CPU"]
    PANIC["invalid_exception_el2"]
end
subgraph subGraph1["Low-Level Processing"]
    VECTORS["exception_vector_base_vcpu(Assembly Vector Table)"]
    TRAMPOLINE["vmexit_trampoline(Context Save)"]
end
subgraph subGraph0["Exception Sources"]
    SYNC["Synchronous Exceptions(Data Aborts, HVC, SMC)"]
    IRQ["IRQ Interrupts"]
    INVALID["Invalid Exceptions"]
end

INVALID --> VECTORS
IRQ --> VECTORS
IRQ_DISPATCH --> EXT_IRQ
SYNC --> VECTORS
SYNC_HANDLER --> MMIO
SYNC_HANDLER --> PSCI
SYNC_HANDLER --> SYSREG
TRAMPOLINE --> IRQ_DISPATCH
TRAMPOLINE --> PANIC
TRAMPOLINE --> SYNC_HANDLER
VECTORS --> TRAMPOLINE

Sources: src/vcpu.rs(L275 - L281)  src/pcpu.rs(L18 - L26)  src/pcpu.rs(L55 - L57) 

This architecture enables efficient virtualization by maintaining clear separation between host and guest contexts while providing comprehensive exception handling capabilities for all VM-exit scenarios.

Dependencies and Build System

Relevant source files

• .github/workflows/ci.yml
• .gitignore
• Cargo.toml

This document covers the external dependencies, build configuration, and continuous integration setup for the arm_vcpu crate. It details the Rust crate dependencies required for AArch64 virtualization, the build system configuration, and the automated testing pipeline.

For information about how these dependencies are used within the VCPU implementation, see Virtual CPU Management. For details about the hardware abstraction interfaces, see Hardware Abstraction and Platform Support.

Purpose and Scope

The arm_vcpu crate is a no-std Rust library that implements AArch64 virtualization capabilities for hypervisors. This page documents the dependency management, build requirements, and continuous integration infrastructure that support the development and deployment of this hypervisor component.

External Dependencies

The crate relies on several categories of dependencies to provide AArch64 virtualization functionality:

Core Rust Dependencies

Sources: Cargo.toml(L7 - L8) 

AArch64 Hardware Interface Dependencies

Sources: Cargo.toml(L10 - L16) 

System Infrastructure Dependencies

The percpu crate is configured with the arm-el2 feature to enable Exception Level 2 support required for hypervisor operation.

Sources: Cargo.toml(L14 - L15) 

ArceOS Ecosystem Dependencies

The crate integrates with the broader ArceOS hypervisor ecosystem through Git-based dependencies:

Dependency Architecture

flowchart TD
subgraph subGraph3["System Support"]
    PERCPU["percpu(Per-CPU Storage)"]
    LOG["log(Logging)"]
    SPIN["spin(Synchronization)"]
    AXERRNO["axerrno(Error Codes)"]
end
subgraph subGraph2["Hardware Layer"]
    AARCH64["aarch64-cpu(Register Definitions)"]
    SYSREG["aarch64_sysreg(System Registers)"]
    TOCK["tock-registers(Register Macros)"]
end
subgraph subGraph1["ArceOS Core Dependencies"]
    AXVCPU["axvcpu(Core VCPU Traits)"]
    AXADDR["axaddrspace(Address Space Management)"]
end
subgraph subGraph0["arm_vcpu Crate"]
    MAIN["arm_vcpu"]
end

MAIN --> AARCH64
MAIN --> AXADDR
MAIN --> AXERRNO
MAIN --> AXVCPU
MAIN --> LOG
MAIN --> PERCPU
MAIN --> SPIN
MAIN --> SYSREG
MAIN --> TOCK

Git Dependencies

Both dependencies are sourced from the arceos-hypervisor GitHub organization, indicating a modular hypervisor architecture where arm_vcpu provides the AArch64-specific implementation.

Sources: Cargo.toml(L18 - L19) 

Build Configuration

Package Metadata

The crate is configured as a library with the following specifications:

[package]
name = "arm_vcpu"
version = "0.1.0"
edition = "2024"

The use of Rust edition 2024 indicates adoption of the latest Rust language features and optimizations.

Sources: Cargo.toml(L1 - L4) 

Target Architecture

The primary build target is aarch64-unknown-none-softfloat, which specifies:

• aarch64: 64-bit ARM architecture
• unknown: No specific vendor
• none: Bare metal (no operating system)
• softfloat: Software floating-point implementation