iso: Enable VirtioFS for x86_64 and aarch64 #21147
Conversation
The Virtio Filesystem[1] allows guests to mount file systems from the
host.
Testing with krunkit show 10 times faster read throughput:
minikube krunkit - 9p:
$ time cp /mnt/models/mistral-7b-instruct-v0.2.Q4_K_M.gguf model
real 0m34.330s
user 0m0.019s
sys 0m1.959s
podman krunkit - virtiofs:
$ time cp /Users/nir/Downloads/models/mistral-7b-instruct-v0.2.Q4_K_M.gguf model
real 0m3.245s
user 0m0.006s
sys 0m1.035s
We must have virtiofs for playing with AI workload in krunkit. It can be
nice improvment for vfkti and qemu, and should be available in ohter
hypervisors.
[1] https://virtio-fs.gitlab.io/index.html
k8s-ci-robot
added
the
cncf-cla: yes
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: nirs The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
needs-ok-to-test
|
Hi @nirs. Thanks for your PR. I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
k8s-ci-robot
added
the
size/XS
|
ok-to-build-iso |
Add --virtio-fs flag for specifying one or more shared directories
separated by comma. The shared directories are be mounted in the guest
using virtio-fs.
Testing with krunkit shows that virtio-fs mount is 10 times faster
compared with 9p mount.
Usage:
--virtio-fs HOST_PATH:MOUNT_TAG[,HOST_PATH:MOUNT_TAG,...]
Example:
minikube start --vrtio-fs /Users/joe/models:my-models
Using a new flag since none of the mount options matches virtio-fs
usage, and it will be too confusing to use using mount options.
In krunkit and vfkit, every virtio-fs shared directory will be added as:
--device virtio-fs,sharedDir=/host-path,mountTag=mount-tag
In the guest the shared directory will be mounted using:
sudo mount -t virtiofs my-models /mnt/my-models
This change only adds the options and configure vfkit and krunkit. The
next step is connecting to the guest and mounting the shared
directories.
More work is needed to add VirtioFS to qemu. I'm starting with fkit and
krunkit since they have identical interface (krunkit was designed as
drop-in replacement for vfkit).
Issues:
- We don't have virtiofs in the kernel yet, requires kubernetes#21147
Add --virtio-fs flag for specifying one or more shared directories
separated by comma. The shared directories are be mounted in the guest
using virtio-fs.
Testing with krunkit shows that virtio-fs mount is 10 times faster
compared with 9p mount.
Usage:
--virtio-fs HOST_PATH:MOUNT_TAG[,HOST_PATH:MOUNT_TAG,...]
Example:
minikube start --vrtio-fs /Users/joe/models:my-models
Using a new flag since none of the mount options matches virtio-fs
usage, and it will be too confusing to use using mount options.
In krunkit and vfkit, every virtio-fs shared directory will be added as:
--device virtio-fs,sharedDir=/host-path,mountTag=mount-tag
In the guest the shared directory will be mounted using:
sudo mount -t virtiofs my-models /mnt/my-models
This change only adds the options and configure vfkit and krunkit. The
next step is connecting to the guest and mounting the shared
directories.
More work is needed to add VirtioFS to qemu. I'm starting with fkit and
krunkit since they have identical interface (krunkit was designed as
drop-in replacement for vfkit).
Issues:
- We don't have virtiofs in the kernel yet, requires kubernetes#21147
|
Hi @nirs, we have updated your PR with the reference to newly built ISO. Pull the changes locally if you want to test with them or update your PR further. |
|
@medyagh @ComradeProgrammer can you add /ok-to-test and approve the workflows? |
|
/ok-to-test |
k8s-ci-robot
added
ok-to-test
|
wow thats signifact improvement, how about we make that default for the drivers/platofrms that support it? |
It is not compatible with mount options and the mount command:
So we can use the Driver Mount feature, like hyperkit --nfs-share. |
|
kvm2 driver with docker runtime Times for minikube start: 47.8s 48.8s 50.1s 50.9s 51.7s Times for minikube ingress: 14.5s 15.0s 15.0s 14.0s 14.5s docker driver with docker runtime Times for minikube (PR 21147) start: 23.4s 24.8s 23.0s 23.1s 26.6s Times for minikube ingress: 12.3s 12.3s 12.8s 12.8s 13.3s docker driver with containerd runtime Times for minikube ingress: 22.8s 39.3s 22.8s 39.3s 22.8s Times for minikube start: 21.7s 21.6s 24.4s 22.3s 23.9s |
|
@medyagh @ComradeProgrammer can you re-run the qemu smoke test? |
* iso: Enable VirtioFS for x86_64 and aarch64
The Virtio Filesystem[1] allows guests to mount file systems from the
host.
Testing with krunkit show 10 times faster read throughput:
minikube krunkit - 9p:
$ time cp /mnt/models/mistral-7b-instruct-v0.2.Q4_K_M.gguf model
real 0m34.330s
user 0m0.019s
sys 0m1.959s
podman krunkit - virtiofs:
$ time cp /Users/nir/Downloads/models/mistral-7b-instruct-v0.2.Q4_K_M.gguf model
real 0m3.245s
user 0m0.006s
sys 0m1.035s
We must have virtiofs for playing with AI workload in krunkit. It can be
nice improvment for vfkti and qemu, and should be available in ohter
hypervisors.
[1] https://virtio-fs.gitlab.io/index.html
* Updating ISO to v1.36.0-1753487480-21147
---------
Co-authored-by: minikube-bot <minikube-bot@google.com>
* iso: Enable VirtioFS for x86_64 and aarch64
The Virtio Filesystem[1] allows guests to mount file systems from the
host.
Testing with krunkit show 10 times faster read throughput:
minikube krunkit - 9p:
$ time cp /mnt/models/mistral-7b-instruct-v0.2.Q4_K_M.gguf model
real 0m34.330s
user 0m0.019s
sys 0m1.959s
podman krunkit - virtiofs:
$ time cp /Users/nir/Downloads/models/mistral-7b-instruct-v0.2.Q4_K_M.gguf model
real 0m3.245s
user 0m0.006s
sys 0m1.035s
We must have virtiofs for playing with AI workload in krunkit. It can be
nice improvment for vfkti and qemu, and should be available in ohter
hypervisors.
[1] https://virtio-fs.gitlab.io/index.html
* Updating ISO to v1.36.0-1753487480-21147
---------
Co-authored-by: minikube-bot <minikube-bot@google.com>
* iso: Enable VirtioFS for x86_64 and aarch64
The Virtio Filesystem[1] allows guests to mount file systems from the
host.
Testing with krunkit show 10 times faster read throughput:
minikube krunkit - 9p:
$ time cp /mnt/models/mistral-7b-instruct-v0.2.Q4_K_M.gguf model
real 0m34.330s
user 0m0.019s
sys 0m1.959s
podman krunkit - virtiofs:
$ time cp /Users/nir/Downloads/models/mistral-7b-instruct-v0.2.Q4_K_M.gguf model
real 0m3.245s
user 0m0.006s
sys 0m1.035s
We must have virtiofs for playing with AI workload in krunkit. It can be
nice improvment for vfkti and qemu, and should be available in ohter
hypervisors.
[1] https://virtio-fs.gitlab.io/index.html
* Updating ISO to v1.36.0-1753487480-21147
---------
Co-authored-by: minikube-bot <minikube-bot@google.com>
The Virtio Filesystem[1] allows guests to mount file systems from the host.
Testing with krunkit show 10 times faster read throughput:
minikube krunkit - 9p:
podman krunkit - virtiofs:
We must have virtiofs for playing with AI workload in krunkit. It can be nice improvment for vfkti and qemu, and should be available in ohter hypervisors.
Status
Not tested yet, waiting for the iso build.
[1] https://virtio-fs.gitlab.io/index.html