-
Notifications
You must be signed in to change notification settings - Fork 5k
drivers: Add support for Virtiofs mounts for vfkit and krunkit #21149
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Hi @nirs. Thanks for your PR. I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
/ok-to-test |
@afbjorklund can you review? |
This comment has been minimized.
This comment has been minimized.
b6650e0
to
a324895
Compare
@medyagh @ComradeProgrammer can you approve the workflows? |
This was a quick version to be able to play with virtuous. I think we need something more extensible that will work nicely when we add yaml support: Flags
Minikube Yamlvirtiofs-shares:
- path: /Users/Joe/models
mount: models
- path: /Users/Joe/project
mount: project
writable: true Minikube JSON{
"virtiofs-shares": [
{
"path": "/Users/Joe/models",
"mount": "models"
},
{
"path": "/Users/Joe/project",
"mount": "project",
"writable": true
}
]
} |
This comment has been minimized.
This comment has been minimized.
Rebased on #21272, comments not addressed yet. |
This comment has been minimized.
This comment has been minimized.
@medyagh changes in last version:
Tested locally with vfkit and krunkit:
Should be ready for merge. |
This comment has been minimized.
This comment has been minimized.
Marking as draft since it depends on #21272 |
/retest |
The package provides functions for parsing and validating mount string and setting up the mount inside the guest.
Replace 9p mounts with virtiofs for vfkit and krunkit. Testing shows that virtiofs mount is 23 times faster with krunkit, and 8 times faster with vfkit. vfkit and krunkit support multiple virtiofs mounts but minikube --mount-* flags are not ready for multiple mounts. We have the same issue with KIC drivers, supporting multiple mounts but using only one. We hope to improve this in the next release. Example usage: minikube start --mount-string ~/models:/mnt/models The arguments are parsed and validated when configuring the driver, so invalid arguments fail quickly without starting the driver. The validated mounts are stored in the machine config: $ jq '.Driver.VirtiofsMounts' < ~/.minikube/machines/minikube/config.json [ { "HostPath": "/Users/joe/models", "GuestPath": "/mnt/models", "Tag": "f845b54d-00e3-493d-9541-3b37490b96db" } ] Minikube generates a new random UUID for every virtiofs mount to identify the file system inside the guest. In krunkit and vfkit, every mount is add as: --device virtio-fs,sharedDir=/host-path,mountTag=f845b54d-00e3-493d-9541-3b37490b96db When the guest is started the shared directory is mounted via SSH using: sudo mkdir -p /mnt/models sudo mount -t virtiofs f845b54d-00e3-493d-9541-3b37490b96db /mnt/models Example mount: $ minikube ssh findmnt /mnt/models TARGET SOURCE FSTYPE OPTIONS /mnt/models f845b54d-00e3-493d-9541-3b37490b96db virtiofs rw,relatime More work is needed to add VirtioFS to qemu. I'm starting with vfkit and krunkit since they have identical interface (krunkit was designed as drop-in replacement for vfkit).
kvm2 driver with docker runtime
Times for minikube ingress: 19.1s 15.0s 16.0s 19.5s 16.0s Times for minikube start: 48.9s 52.3s 54.6s 47.9s 51.0s docker driver with docker runtime
Times for minikube ingress: 13.8s 13.8s 13.3s 13.8s 17.3s Times for minikube start: 25.2s 26.9s 23.8s 22.6s 23.2s docker driver with containerd runtime
Times for minikube start: 21.1s 23.3s 24.6s 21.0s 23.6s Times for minikube ingress: 39.3s 22.8s 39.3s 23.8s 39.3s |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @nirs this will be an important part of the user experience for the AI develoeprs
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: medyagh, nirs The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Replace 9p mounts with virtiofs for vfkit and krunkit. Testing shows
that virtiofs mount is 23 times faster with krunkit, and 8 times faster
with vfkit.
vfkit and krunkit support multiple virtiofs mounts but minikube
--mount-* flags are not ready for multiple mounts. We have the same
issue with KIC drivers, supporting multiple mounts but using only one.
We hope to improve this in the next release.
Example usage:
minikube start --driver krunkit --mount-string ~/models:/mnt/models
The arguments are parsed and validated when configuring the driver, so
invalid arguments fail quickly without starting the driver. The
validated mounts are stored in the machine config:
Minikube generates a new random UUID for every virtiofs mount to
identify the file system inside the guest. In krunkit and vfkit, every
mount is add as:
When the guest is started the shared directory is mounted via SSH using:
Example mount:
More work is needed to add VirtioFS to qemu. I'm starting with vfkit and
krunkit since they have identical interface (krunkit was designed as
drop-in replacement for vfkit).
To play with this PR, see medyagh/ai-playground-minikube#4 (comment)
Example chat with granite model