Amazon FSx for OpenZFS provides a managed file storage service built on top of the open source OpenZFS filesystem supporting the NFS protocol. FSx for OpenZFS runs on Graviton-chip-based systems and leverages SSD storage to consistently deliver sub-millisecond latency. It also leverages an OpenZFS built-in caching mechanism that can provide even lower latency for frequently accessed data. Throughput capacity grows with storage size up to 12.5 Gbps, while FSx for OpenZFS can deliver up to 1 million IOPS. Note that you can achieve max throughput and IOPS from a single client by using multiple NFS connections in parallel via NFS nconnect.
Amazon FSx for NetApp ONTAP provides a managed file storage service built on top of NetApp’s ONTAP filesystem. ONTAP offers filesystem access through the standard file-sharing protocols SMB and NFS, but also supports block storage access via iSCSI. It is worth noting that it supports multi-protocol concurrent access to the same data. So, you could have, for instance, a Linux server accessing a given file share via NFS and a Windows server accessing the same file share via SMB at the same time. FSx for NetApp ONTAP offers two data tiers: primary storage and capacity pool storage. The primary storage tier runs on high-performance SSDs and is typically for your hot or most frequently accessed data. The capacity pool storage tier runs on cost-optimized storage that grows automatically as your data grows and is meant to store the least frequently accessed data. You can increase the size of your primary storage tier at any time if needed. Primary storage delivers sub-millisecond latency, while capacity pool storage offers tens of milliseconds of latency, and both storage tiers can deliver hundreds of thousands of IOPS.
Amazon FSx for Lustre offers a managed Lustre filesystem, an open-source filesystem very popular in the High-Performance Computing (HPC) world. FSx for Lustre thus provides a high-performance filesystem for very demanding workloads. Similar to FSx for Windows File Server, FSx for Lustre lets you choose between SSD-based and HDD-based storage, depending on whether your workload is mostly sensitive to latency or throughput. For HDD-based filesystems, you have the ability to add an optional SSD-based cache, representing 20% of your filesystem size, to improve performance in terms of latency. Note that you can also make your content on S3 available to your workload through FSx for Lustre so that objects on S3 are transparently presented as files on your Lustre filesystem.
As you can see, the FSx family offers quite some choice in terms of filesystems. For further details about how they compare to each other in terms of features and performance, it is highly recommended you review the service documentation guidance: https://packt.link/ieCBn.
Confronted with all those choices, which storage solution should you pick up, then? Start with your workload requirements. What are the data access patterns? Which access protocols do you need to support? Do you plan to share storage among multiple compute resources? What volume of data and number of read/write requests do you need? What storage growth is expected in the near future? What requirements do you have in terms of fault tolerance and durability? Does the storage layer need to be versatile enough to easily adapt to future access patterns? Which storage metrics are important to consider for your workload?
Answering these questions will take you in the right direction. Then, as already mentioned, if, despite your initial investigation, you hesitate between two solutions and cannot easily make up your mind, simply benchmark one against the other to find out empirically which one best satisfies your needs.