Adding File or Block Storage to Flexify.IO with MinIO

flexify io works exclusively with object storage however, for scenarios such as on premises to cloud migration, you may want to add a legacy directory or a file share to flexify io while flexify io cannot access data from your file system or a network share directly, it can do so with the help of minio configuring and starting minio first, start minio server on a physical or virtual machine where your file or block storage is mounted the storage can be mounted as a volume or a network share we recommend at least 4 vcpu and 16 gb ram to run minio otherwise, you may not be able to achieve the maximum migration throughput download minio, if not already installed wget https //dl min io/server/minio/release/linux amd64/minio chmod +x minio create a temporary directory that will be used as minio data "root" mkdir minio root use temporary directory as minio data root minio server allows exposing a file system directory via s3 compatible api when doing so, the directories just inside minio's data "root" directory become buckets it imposes certain limitations on their names and may cause issues that's why it's not recommended to point the minio server to an existing data directory of your file system start the minio server pointing to this directory export minio root user=minio export minio root password=mysecurepassword /minio server /minio root start minio before mapping data start minio before creating any mount points in the data directory otherwise minio would fail on the startup check secure password make sure to choose a secure password open port 9000 minio uses port 9000 by default you may need to open port 9000 on a firewall to enable outside connections to the minio server on your machine create a directory that will represent an s3 bucket and mount your data path to it in most cases, you'd like all your data to be in a single bucket the data path can be an entire volume (such as /mnt/volume ams3 01 ) or any directory on your local or network file system cd minio root mkdir mybucket mount bind r /path/to/data mybucket read only access using the r option of the mount command would protect your data from accidental deletion and is recommended for the on prem to cloud migration scenario make sure to umount the data directory before your delete minio root now the content of the /path/to/data directory is accessible via the s3 api and can be found in the mybucket bucket you can see it by opening http //\<your server ip> 9000/minio in your browser or by configuring any s3 client to use http //\<your server ip> 9000 endpoint with minio/mysecurepassword as the s3 keys adding minio to flexify io from flexify io's standpoint, minio is no different from any other s3 compatible storage to add minio to flexify io, simply click add storage account on the data tab, select minio from the drop down list, and specify your minio server's ip\ port and s3 keys ssl/tls by default minio uses http make sure to remove the "encrypt transfer with ssl/tls" checkbox or configure ssl certificates on the minio server now your file/disk storage is added to flexify io, and you can migrate data to amazon s3 or another cloud storage, or even attach it to the flexify io multi cloud virtual endpoint performance considerations disk storage might be slow especially in the case of rotational media, such as hdd based raid or google cloud storage's standard persistent disks, storage systems may also suffer from high latency response times might as high as 20 ms, limiting sequential i/o to around 50 iops but most storage systems allow many requests to be served in parallel , increasing the cumulative iops to thousands unfortunately, minio is not taking advantage of parallelism when listing objects (each object requires a separate stat system call), and on a high latency rotational media would not be able to list more than 50 objects/second this could limit the total throughput of the migration in case your objects are relatively small in size there are two alternative solutions to this issue convert the disk to a low latency type, such as standard to balanced in google cloud platform use the special minio build prepared by flexify io that avoids those unnecessary sequential stat calls