Note: The article is based on ONTAP 9.14.1 version. It focuses on using ONTAP System Manager, only using ONTAP CLI when necessary. This is just a brief introduction to S3 usage within NetApp ONTAP.
Documentation
- basic terms and components for ONTAP - NetApp ONTAP Components, Principles and Features
- general information about object storage - What is Object Storage?
- official documentation - S3 object storage management
S3 Object Storage
Support within ONTAP
NetApp offers a unified data storage system. The ONTAP operating system supports a combination of file, block, and object protocols. We can use common storage (disk array), such as NetApp AFF or FAS, and operate it as file, object, and block storage. On an existing ONTAP cluster, we can enable ONTAP S3 and utilize S3 features without additional costs. If we need a specialized object storage, NetApp offers StorageGRID.
S3 object storage has been supported since ONTAP 9.8. Like for other protocols (FC, iSCSI, NFS, SMB, NVMe_oF), a license is required for S3, but it is provided at no cost. ONTAP from version 9.12.1 supports S3 multiprotocol support, where data can be accessed via S3 protocol and NFS and SMB protocols.
Supported standard S3 API actions are gradually added to new ONTAP versions. The list can be found in the article ONTAP S3 supported actions. NetApp has verified some third-party solutions for use with ONTAP S3, such as Veeam Backup and Replication 12. Conversely, NetApp ONTAP S3 is included in the Veeam Ready program.
S3 Object Lock is supported from ONTAP 9.14.1 (according to discussions, functional from 9.14.1P1). However, it requires a SnapLock license. For cluster nodes, SnapLock Compliance Clock must be initialized.
Architecture
Storage VM (SVM) is used, on which S3 access protocol is enabled and an Object Store Server is created. Each SVM has its own namespace. When creating a data SVM, a root volume is created. The S3 Object Store Server manages data as S3 objects, unlike file or block storage provided by ONTAP NAS and SAN servers.
The basic architecture for a bucket in ONTAP is a FlexGroup Volume. It can span across different aggregates and cluster nodes. A common location for storing data is a Flexible Volume (FlexVol), which is located within an aggregate (Tier).
FlexGroup Volume is a scalable NAS container that provides high performance with automatic load distribution. It contains several folders (Constituents) that automatically and transparently share transfers. Folders (Constituents) are basic FlexVol volumes that form the FlexGroup Volume.
The minimum bucket size is 95 GB. From ONTAP 9.14.1, the default FlexGroup size is based on the size of the contained buckets. The FlexGroup Volume automatically grows or shrinks when buckets are added or removed.
ONTAP S3 Configuration Process
Creation and configuration can be done using web interface or CLI. Using System Manager simplifies the configuration, and ONTAP chooses default values according to best practices. If we need to perform more detailed settings, we often need to use CLI.
The configuration consists of two main parts, which may contain individual steps.
Creating S3 Storage VM
- creating SVM
- installing certificate
- creating S3 Service Data Policy
- creating LIF
- creating Object Store Server
Adding Storage Capacity to S3 SVM and Access
- creating bucket (S3 Bucket)
- creating S3 user
- creating S3 group
- enabling access to S3 object storage
Configuration Preparation
Required Physical Space
When creating an S3 bucket, a FlexGroup volume is automatically created. By default, local aggregates and FlexGroup components with the most suitable disks meeting the chosen service level are automatically selected. There must be sufficient free space in existing aggregates. If we want to specify the used aggregates, we must use CLI.
Network Requirements
Before creating the SVM, we need to have the network completely configured for client access. It's important to have the same MTU across the entire network (on all elements in the given VLAN). We must consider and prepare the following:
- physical ports - which will be used for communication, can be shared with another service, but separate is better, must have sufficient speed (can use LACP), e.g.,
e0a - IPspace - can use
Default - Broadcast Domain - according to L2 network where physical ports and IP addresses are assigned, can be
Default - Subnet - optional
Another important network component is automatically created when creating SVM in System Manager.
- Logical Interface (LIF) - Data LIF represents the IP address for accessing the SVM (
data-s3-server), recommended to separate from management LIF
We must plan IP addresses for the S3 interface that clients will access. Similarly, the FQDN name to which we will issue the certificate.
Certificate
S3 server can be used with HTTP, but it is strictly recommended to use secure HTTPS. For communication, we need a server certificate. It is recommended to issue a certificate from an internal or public Certificate Authority (CA).
When creating SVM, we have two options
- System-generated certificate - we can have a Self-signed certificate generated
- External-CA signed certificate - we insert our own certificate and private key in Base64 encoding
Configuring ONTAP S3 using System Manager
Creating an SVM with S3 Enabled
S3 can coexist with other protocols in SVM, but it is more suitable to separate and create a separate SVM. When using System Manager, Storage VM, Object Store Server, Data LIF, S3 Service Data Policy, and Failover Policy are created together. We enter the certificate and network information.
- Storage - Storage VMs - Add
When creating, we fill in the following items:
- Storage VM name - name for identifying SVM
- Access protocol - Enable S3
- S3 server name - FQDN for accessing S3 storage
- Enable TLS - enable TLS, port, insert your certificate External-CA signed certificate
- Default language - standard
C.UTF_8 - Network interface - parameters for creating LIF, we can use only one IP address that can migrate between cluster nodes, or have a separate interface on each node
- IP address
- Subnet mask
- Gateway - if communication between subnets should work
- Broadcast domain and port - select a prepared group (according to configuration, may need to manually specify Home Port)
- Storage VM administration
- Enable the maximum capacity limit - setting maximum capacity limit for SVM
- Manage administrator account - if we want to set an SVM administrator account
A Storage VM and S3 Server are created, where a user is also created and their secret key is displayed one-time.
Creating an S3 Bucket
S3 objects are stored in buckets, which have a flat structure (not nested like files within directories). When creating the first bucket, a FlexGroup volume with the minimum required size is created and further automatically grows or shrinks. When creating a bucket using System Manager, a QoS Policy Group (Performance service level) is standard assigned, default is the highest Extreme, but we can change or create a custom one.
When creating the first bucket using System Manager, aggregates for the FlexGroup volume are automatically selected based on the chosen Storage service level. All subsequent buckets added to the SVM will use the same FlexGroup volume.
- Storage - Buckets - Add
When creating, we fill in certain items, most can be addressed later.
- Name - bucket name
- Storage VM - selection of S3 SVM
- Capacity - bucket size
- More Options - button switches to setting additional values
- Use for tiering - if we want to use for tiering, low-cost media will be selected
- Enable Versioning - enables versioning, required for object locking (Immutability)
- Performance service level - we can select or create QoS Policy Group
- Permissions - we can set permissions, must have pre-created users or groups
- Enable object locking - enables S3 Object Lock, requires SnapLock license, otherwise this option is not offered
Managing S3 Users
Users are created within the S3 Object Store Server and can be given access to any bucket in the SVM. When creating a user, an access key and secret key are generated. The access key's validity period can be set. When creating the Object Store Server, a root user (UID 0) with privileged access is created.
- Storage - Storage VMs - select the desired SVM - Settings - click Edit (pencil) under S3
Access Permissions for Users and Groups
Access permissions can be assigned to a user or group in Bucket Policy or Object Store Server Policy. This means at the level of the bucket or entire S3 SVM.
Bucket Policy
- Storage - Buckets - select the desired bucket - click Edit in the top right - Permissions
Under Actions, we select individual S3 actions. As Resources, we can select specific objects. Optionally, we can add certain Conditions that are evaluated when attempting access (such as source IP address).
Object Store Server Policy
- Storage - Storage VMs - select the desired SVM - Settings - click Edit (pencil) under S3 - Policies
We can create a policy that is applied to one or more buckets. The policy is assigned to a user group. We can use default (pre-prepared) or custom policies.
FlexGroup Volume Information
The FlexGroup volume is not visible in System Manager and we must use CLI.
AFF::> volume show -vserver svm-s3 -volume-style-extended flexgroup
Vserver Volume Aggregate State Type Size Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
svm-s3 fg_oss_1733320663
- online RW 205.3GB 195.0GB 0%
AFF::> volume show -vserver svm-s3 -volume-style-extended flexgroup-constituent
Vserver Volume Aggregate State Type Size Available Used%
--------- ------------ ------------ ---------- ---- ---------- --------- -----
svm-s3 fg_oss_1733320663__0001
AFF_NVME_SSD_1
online RW 205.3GB 195.0GB 0%
Network information
AFF::> network interface show -vserver svm-s3
Logical Status Network Current Current Is
Vserver Interface Admin/Oper Address/Mask Node Port Home
----------- ---------- ---------- ------------------ ------------- ------- ----
svm-s3
lif_svm-s3_553
up/up 192.168.100.245/24 AFF-A250-2-01 e0a true
AFF::> network interface show -vserver svm-s3 -failover
Logical Home Failover Failover
Vserver Interface Node:Port Policy Group
-------- --------------- --------------------- --------------- ---------------
svm-s3
lif_svm-s3_553 AFF-A250-2-01:e0a system-defined Default
Failover Targets: AFF-A250-2-01:e0a,
AFF-A250-2-02:e0a
AFF::> network interface service-policy show -vserver svm-s3
Vserver Policy Service: Allowed Addresses
--------- -------------------------- ----------------------------------------
svm-s3
default-data-blocks data-core: 0.0.0.0/0
default-data-files data-core: 0.0.0.0/0
data-fpolicy-client: 0.0.0.0/0
management-dns-client: 0.0.0.0/0
management-ad-client: 0.0.0.0/0
management-ldap-client: 0.0.0.0/0
management-nis-client: 0.0.0.0/0
data-dns-server: 0.0.0.0/0
default-data-iscsi data-core: 0.0.0.0/0
data-iscsi: 0.0.0.0/0
default-data-nvme-tcp data-core: 0.0.0.0/0
data-nvme-tcp: 0.0.0.0/0
default-management data-core: 0.0.0.0/0
management-ssh: 0.0.0.0/0
management-https: 0.0.0.0/0
management-dns-client: 0.0.0.0/0
management-ad-client: 0.0.0.0/0
management-ldap-client: 0.0.0.0/0
management-nis-client: 0.0.0.0/0
data-dns-server: 0.0.0.0/0
management-http: 0.0.0.0/0
backup-ndmp-control: 0.0.0.0/0
management-snmp-server: 0.0.0.0/0
sm-custom-service-policy-nas-s3
data-core: 0.0.0.0/0
data-nfs: 0.0.0.0/0
data-cifs: 0.0.0.0/0
data-s3-server: 0.0.0.0/0
data-fpolicy-client: 0.0.0.0/0
management-dns-client: 0.0.0.0/0
management-ad-client: 0.0.0.0/0
management-ldap-client: 0.0.0.0/0
management-nis-client: 0.0.0.0/0
data-dns-server: 0.0.0.0/0
6 entries were displayed.
Deleting S3 bucket
ONTAP allows viewing the list of buckets and information about their capacity (total and available), but it doesn't have tools for browsing content.
For bucket deletion, System Manager has a Delete option in the menu. We get a confirmation dialog asking if we really want to delete the bucket, and deletion should occur. However, ONTAP has a feature (quite inconvenient in my opinion) that doesn't allow deleting a bucket that isn't empty. In this case, we get an error:
Failed to remove bucket "test-backup" for SVM "svm-s3". Reason: Cannot delete bucket "test-backup" on SVM "svm-s3" because it is not empty. Delete the objects, and then try the operation again. For further assistance, contact technical support.
We must delete all content using some S3 tool or application. If we have versioning enabled, it's not enough to delete all objects, we must also delete object versions. Alternatively, we can create an ONTAP S3 bucket lifecycle rule for deleting non-current versions.
As a tool for S3 storage management, we can use for example AWS CLI, S3 Browser GUI, or Rclone. S3 Browser is recommended in many places as it's handy and convenient. The website states that a freeware version is available, but after installation, it's mentioned that it's only for non-commercial use. Moreover, it has various limitations, for example, it doesn't allow deleting multiple objects at once.
S3 Browser allows deleting object versions in the Versions tab. But what's better, it allows deleting the entire bucket even if it contains objects or their versions.
NetApp has articles about this area:
There are no comments yet.