What is Oracle Cloud Infrastructure Storage Gateway?
Oracle Cloud Infrastructure Storage Gateway helps you bridge your on-premises data center with Oracle Cloud Infrastructure Object Storage. Using Storage Gateway, any application that can store data on an NFS target can easily and seamlessly move data to Object Storage without having to first modify the application to uptake REST APIs. Storage Gateway exposes an NFS mount point that can be mounted to any host that supports an NFSv4 client. Storage Gateway performs the NFS to REST API translations for you. A local configurable cache enables asynchronous and optimized movement of the data to the cloud to work around bandwidth limitations or connectivity issues. Because frequently accessed data is cached locally, the cache also improves latency when reading data from the cloud. Each Storage Gateway NFS mount point maps to an Object Storage bucket. The NFS interface lets you use the traditional file system commands to interact with cloud storage.
Oracle Cloud Infrastructure Storage Gateway supports file to object transparency. This means that when the Storage Gateway uploads your data to the cloud, it preserves the original data format. The data in the cloud can be read or managed using supported Object Storage clients, such as APIs, SDKs, the CLI and Console, or other third party tools.
What are the benefits of using Storage Gateway?
Using Storage Gateway, you can easily move data generated by traditional applications that support common file system protocols like NFS to Object Storage without having to modify the application to integrate with the REST APIs. Storage Gateway opens up virtually unlimited cloud storage to traditional on-premises applications, which previously could not take advantage of the seamless scalability, durability, and cost efficiency of Object Storage.
Can I use Storage Gateway as a general purpose Network Attached Storage server?
No, Storage Gateway must not be used as a general-purpose network-attached storage. Though Storage Gateway is in many ways a virtual shared file system, its primary function is data movement to the cloud. Storage Gateway is not built to handle the transactional workload that a full-featured shared file system can handle.
Which use cases does Storage Gateway support?
Storage Gateway is an efficient tool to move data from your on-premises data center to the cloud. You can use this service for use cases like backup/archive, disaster recovery, and data migration. Using Storage Gateway, you can easily script the movement of cold data from on-premises NAS servers to Oracle Cloud Infrastructure Object Storage.
Which use cases does the Storage Gateway not support?
General purpose network storage - Storage Gateway is not a general purpose storage filer and must not be used as a replacement for traditional network storage appliances.
File sync and share - Though Storage Gateway is an effective data mover, it's not a replacement for file sync and share services. Evaluate Oracle services like Oracle Document Cloud service for your file sync and share needs.
Content collaboration - Storage Gateway does not support multiple instances simultaneously reading from and writing to a single Object Storage bucket. Storage Gateway is not meant to be used as a tool that enables distributed teams to collaborate on creating and managing content.
Frequently modified files – When an existing file is modified, its content is read from Object Storage if not present in cache. The modified file is later uploaded as a new version/instance of the file. Therefore, files that are frequently modified are not ideal both in terms of consuming upload/download bandwidth and capacity utilization.
How much does Storage Gateway cost?
Storage Gateway is a free service for all Oracle Cloud Infrastructure customers.
Which storage protocols does Storage Gateway support?
Storage Gateway supports the NFSv4 protocol.
Does Storage Gateway support the Windows operating environment?
No, Storage Gateway does not currently support the Windows operating environment.
Can I use Storage Gateway to move data to the cloud from third party storage arrays?
Yes, you can use Storage Gateway to replicate data from any storage, as long as the NFS client that is performing the data copy can facilitate the replication.
I'm using Oracle Cloud Infrastructure Classic Object Storage and Storage Software Appliance, can I move my data to Oracle Cloud Infrastructure Object Storage and Storage Gateway?
Yes, we can facilitate this migration. Contact us about using the Storage Gateway cloud migration utility.
Do I need to use Object Storage REST APIs to use Storage Gateway?
No, you do not need to use any Object Storage APIs to use Storage Gateway. Storage Gateway performs the NFS to REST API translation for you.
How does Storage Gateway leverage Object Storage?
When you create a file system on the Storage Gateway, an Object Storage bucket with the same name is created in the Oracle Cloud Infrastructure region of your choice. Storage Gateway completely manages your data in the Object Storage bucket for you. You only need to interact with the local file system using normal file system commands.
Does Storage Gateway support compression or deduplication?
No, Storage Gateway does not support compression or deduplication.
How can I get support for Storage Gateway?
Can I create more than one file system on a Storage Gateway instance?
Yes, you can create multiple file systems on a single Storage Gateway instance. The hardware the host is running on dictates the maximum number of file systems that can be supported by a Storage Gateway instance. We recommend that you create no more than 5 active file systems per Storage Gateway instance. For best performance, host each file system on a dedicated Storage Gateway.
What are the Storage Gateway scalability limits?
Storage Gateway can manage a maximum of 100 million files per file system. For data sets that contain more than 100 million objects, ensure that the objects are distributed across multiple file systems.
What is the maximum size of a file that can be uploaded using Storage Gateway?
The maximum supported file size is 75% of cache space configured on Storage Gateway, but cannot exceed 10 TB. For example, if Storage Gateway is configured to have 1 TB of cache space, the maximum supported file size is 750 GB.
Can I instantiate multiple Storage Gateway instances?
There is no upper limit on the number of Storage Gateway instances that can be installed for use within an Oracle Cloud Infrastructure tenancy.
How much data can I upload to the Object Storage using Storage Gateway?
As long as the local cache is sized accurately, you can upload an unlimited amount of data to Object Storage. Note, that Storage Gateway file systems should not exceed 100 million files.
What is the longest file or directory name that is supported?
The longest supported file name can be 255 characters. The total path length can be a maximum of 1024 characters, which is the maximum supported length for Object Storage object names. For example, /directory1/directory2/directory3/directory4/filename.txt is 57 characters in length. Note that the maximum supported length for Object Storage paths is shorter than the maximum of 4096 characters supported by Linux file systems. Trying to create a file or directory for which path is longer than 1024 characters generates an NFS error.
How many directories are supported?
There is no limit to the number of directories, as long as the path length of each directory is 1024 characters or less.
How many files are supported in a single directory?
There is no limit to the number of supported files in a single directory.
Can I run Storage Gateway in an Oracle Cloud Infrastructure Compute instance?
Yes, you can run the Storage Gateway in an Oracle Cloud Infrastructure compute instance using the same general installation instructions you follow for installing Storage Gateway on-premises.
Can Storage Gateway be installed behind proxy server?
Yes, Storage Gateway can be installed behind a proxy server. To set the proxy servers, use the following commands:
- sudo ocisg configure proxy <http_proxy_server> <https_proxy server>
- sudo ocisg down
- sudo ocisg up
Can multiple Storage Gateway instances run concurrently against the same bucket?
No, multiple Storage Gateway instances cannot run against the same Object Storage Bucket. Storage Gateway does not enforce object locking or consistency between different Storage Gateway instances.
When I upload data to Object Storage using Storage Gateway, can the data stored in the Object Storage be accessed using tools other than Storage Gateway?
Yes, data uploaded to the cloud using the Storage Gateway can be managed using any supported Oracle tools like APIs, SDKs, the CLI and Console, or third party tools like Cyberduck or CloudBerry.
Does Storage Gateway ensures consistency between local storage and Object Storage?
Changes made to a Storage Gateway file system are eventually consistent with the Object Storage. Uploads are asynchronous and buffered for performance, so Storage Gateway file changes might not yet be reflected in Object Storage.
Are changes made directly to the Object Storage bucket, reflected back to the local Storage Gateway file system?
Changes made directly to the Object Storage bucket are not reflected back to the local Storage Gateway file system in real time. Click 'Refresh' in the Storage Gateway management console to ingest any data that was added or modified directly in Object Storage.
Can a Storage Gateway file system be created against a non-empty target Object Storage bucket?
Yes, a Storage Gateway file system can be created against a non-empty bucket. When this happens, Storage Gateway creates metadata for all of the objects in the bucket with default permissions and attributes. Storage Gateway also creates a local inventory of the objects in the bucket. Depending on the number of objects in the target bucket, this process might take some time to complete. The Storage Gateway management console provides a progress bar showing the current state.
Does the Storage Gateway support hard or symbolic links?
No, Storage Gateway does support symbolic or hard links. Trying to create a link will result in an NFS error.
Does the Storage Gateway support sparse files?
Yes, Storage Gateway supports the creation of sparse files. However, the corresponding Object Storage objects will be non-sparse and filled with 0x00s.
If I delete a file system, will all of my data in the cloud be automatically deleted as well?
No, if a file system is deleted/removed, the data stored in the cloud is left intact. You can reclaim the deleted data by creating another file system backed by the same bucket.
Does deleting data from the Storage Gateway permanently delete the data from Object Storage?
Yes, deleting data from Storage Gateway permanently deletes the data from Object Storage.
How long does the Storage Gateway wait before uploading files to the Object Storage bucket?
Storage Gateway does not upload a new file or modified file until the file is closed. After a file is closed, Storage Gateway allows a 1 minute settling period before queuing the file for upload.
Depending on the number of files pending upload, it might be a while before the file is uploaded to the Object Storage bucket. You can view the upload status of files from the Storage Gateway management console.
Tools and Utilities
What is the Storage Gateway cloud sync utility?
The Storage Gateway cloud sync utility is an easy to use synchronization and replication utility that lets you to copy NAS files to and from Oracle Cloud Infrastructure Object Storage buckets.
When would I want to use Storage Gateway cloud sync utility and for what kind of data?
Customers with a lot of on-premises files to move into the cloud can do so using the Oracle Cloud Infrastructure Storage Gateway. The cloud sync utility offers a way to copy files from the local storage into the Storage Gateway, where the files are asynchronously and efficiently moved to an Object Storage bucket. Customers who store database or data backups in local NAS and want to free up the space, or have a desire to move older backups into less expensive cloud storage, will find the cloud sync utility an easy way to automate this kind of data movement.
Can I use the Storage Gateway cloud sync utility to copy data into both Standard tier and Archive tier buckets?
Yes, when you create a file system, you specify which storage tier, Standard (the default) or Archive, to create the corresponding Object Storage bucket. The Standard Object Storage tier is used for storing data to which you need fast, immediate, and frequent access. The Archive Storage tier is used for storing data that is accessed infrequently and requires long retention periods. While Archive Storage is more cost effective than Object Storage for preserving cold data, you must first restore the objects before you can access them.
What kind of control and status information is provided?
The Storage Gateway cloud sync utility lets you schedule copies to run at desired intervals or times. Each copy operation provides the status to let you know the list of files selected, when they have been copied over to the Storage Gateway, and when they have been successfully uploaded to Object Storage.
How are files that are copied over the network verified to be consistent?
After the file transfer is complete, the cloud sync utility compares the checksums of the files in the source and the target locations.
Where can I download the cloud sync utility?
You can download the Storage Gateway cloud sync utility from the Storage Gateway service documentation here: https://docs.cloud.oracle.com/iaas/Content/StorageGateway/Reference/storagegatewayutilities.htm#CloudSync