feat: add --init-image flag for specifying custom init filesystem images per VM

[manuschillerdev]

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation update

Motivation and Context

I've been tinkering with ways to extend container with a primitive for interacting with the VM layer. One approach: expose the internal init filesystem as a user-configurable option via --init-image.

What this enables:

• Custom boot-time logic before the OCI container starts
• Running additional processes and daemons (e.g., daemons to configure eBPF network filters, logging agents) inside the VM (not the container)
• Debugging/instrumenting the init process

In my current approach, the custom image wraps the default vminitd, so it's currently behaving like an entrypoint.
Curious if this direction aligns with the project's goals, or if there's a better abstraction for these use cases.
If there is interest, I would add proper docs and tests.

Example for a minimal entry point binary

// custom-init/wrapper.go
package main

import (
    "fmt"
    "os"
    "syscall"
)

func main() {
    kmsg, err := os.OpenFile("/dev/kmsg", os.O_WRONLY, 0)
    if err == nil {
        kmsg.WriteString("<6>custom-init: === CUSTOM INIT IMAGE RUNNING ===\n")
        kmsg.Close()
    }

    err = syscall.Exec("/sbin/vminitd.real", os.Args, os.Environ())
    if err != nil {
        fmt.Fprintf(os.Stderr, "Failed to exec vminitd.real: %v\n", err)
        os.Exit(1)
    }
}

Containerfile

# custom-init/Containerfile
FROM ghcr.io/apple/containerization/vminit:0.12.1 AS base

FROM ghcr.io/apple/containerization/vminit:0.12.1
COPY --from=base /sbin/vminitd /sbin/vminitd.real
COPY wrapper /sbin/vminitd

• build the binary with cd custom-init && CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -o wrapper wrapper.go
• build the image with bin/container build -t local/custom-init:0.1.2 custom-init/
• run the container with our custom init-image: bin/container run --name init-test --init-image local/custom-init:0.1.2 alpine:latest echo "hello"
• check the logs with cat "/Users/$USER/Library/Application Support/com.apple.container/containers/init-test/vminitd.log" | head -50

Testing

• Tested locally
• Added/updated tests
• Added/updated docs

[jglogan]

@dcantah see also #838

[manuschillerdev]

@dcantah @jglogan would that be of interest for the project? Would be totally open to other suggestions on how one can interact with the startup process. Exposing --init-fs seemed like a viable option here