Compatibility with Maildir and NFS

[qt-br]

Hi!

I need to migrate ~10TB of data (thousand of folders and files in maildir format) from an old storage (Equallogic) which is mounted via iSCSI and this data need to be accessed through NFS by other dovecot servers.
How stable is migratefs? It is production ready? There is any consideration to be made with NFS usage?

Current scenario:
dovecot / nfs_client ---> nfs_server / iscsi_mounted_volume

New scenario:
dovecot / nfs_client ---> nfs_server / migratefs (new_folder << iscsi_mounted_volume)

[thiell]

Hey @qt-br, interesting project! We consider that migratefs is stable at this point and we have used it for more than 6 months in 2019 on a production cluster system over Lustre, which is POSIX compliant. We haven't tried it over NFS (which is not POSIX compliant), but I think it should work as long as you don't do weird things (eg. ioctl is not supported yet). Of course, I recommend to deploy a testbed first and see if it works for your workload. If you decide to use it, please let us know how it goes and feel free to contribute to the code if needed.

[synnack]

For this usecase, having only the active data is not enough, you'd want a full migration.

So that'd mean either a background process within the migratefs to do this (without race conditions on copyup) or something that slowly (ie, not fast enough to impact production workload) opens and closes all files to trigger a copyup.. :)

Benefit of a background process is that you can set some bandwidth / IOPS constraints, benefit of copyup is that it's way easier, but opening a large file will have quite some extra load for a bit, so load is less linear.