Implementing a Cross-Region and Hybrid-Cloud File Sharing Service Using Cloud Storage Gateway and OSS
By Johnson Chiang, Solutions Architect
This article will demonstrate a file sharing service solution for file sharing cross region, between on-premise IT environment and public cloud.
- Scalable and cost-effective storage backed by OSS.
- Simple frontend user interface to access files on NAS.
- Secured and file transfer and low-latency local cache performed transparently by CSG.
Such architecture would be especially useful to achieve network resilience when transferring file in- and out- China to international locations. You will also walk through how to implement an example to fulfill the file sharing needs for China region’s and international region’s consumers.
CSG-based File Sharing Architecture
What You Will Build
For demonstrating the cross-region and hybrid-cloud file sharing scenario, we will build following topology which consists an online CSG file gateway in China side, an on-premises CSG file gateway in overseas side, and an OSS bucket to be the backend storage of both file gateways. For OSS bucket, we choose Hong Kong region which is geographically closest to both China and our data center in Taiwan.
Overview of Implementation Process
The following steps gives an outline of the overall flow of the implementation:
- Creating the backend OSS for storage
- Creating the China-side CSG File Gateway: online deployment
- Mounting China-side NFS share on a Linux client
- Creating the overseas-side CSG File Gateway: on-premises deployment
- Mounting overseas-side CIFS share on a Windows client
- Testing file sharing
Let’s us now look at the detailed steps of the implementation.
Step 1: Creating the Backend OSS for Storage
First, create an OSS bucket, with a preferred name like
oss-file-sharing. The OSS bucket will be associated with two File Gateway instances to mount in our file sharing architecture. In this example, we chose
Hong Kong region which would work best for the File Gateway and client in Taiwan IDC.
Step 2: Creating the China-side CSG File Gateway: Online Deployment
This section guides you through sample tasks to deploy an online CSG instance and create the File Gateway; for more information, please see https://www.alibabacloud.com/help/doc-detail/87408.htm.
Create a Gateway Cluster (for example,
Create a Gateway instance (for example, named
- Create the Gateway instance by populating the fields:
- Location ?
Alibaba Cloudfor online deployment
- Type ?
- VPC ?Specify the VPC to be associated with China-side CSG (for example,
- vSwitch ?Specify the vSwitch (for example,
Create a Cache for the running Gateway.
Create a NFS share by using gateway management console:a
Create a Cache (for example,
Cache_b) in Caches tab
Create a Resource (for example,
BackendOSS) in Cloud Resources tab
- Create a File Gateway (for example, ) in NFS tab with following settings:
- Share Name ?Specify the share name such as
- Data Access Mode Sync Mode`. To achieve the goal of file sharing, we must enable Sync Mode so that the CSG will synchronize data from OSS bucket to local drive.
- Enable Remote Sync ?
- Cloud Resource ?Select the Resource that you created previously (
- Cache Name ?Select the Cache that you created previously (
In About, check the information and take a note of the assigned intranet IP address (
192.168.1.131 as shown below in our CSG instance)
Step 3: Mounting China-side Share — A Linux NFS Client Example
This section covers the steps of operating an ECS-hosted Linux client to mount the remote NFS mount point shared by the CSG File Gateway. And then we write some files to the mounted directory and verify the newly written files are synchronized to the OSS bucket.
Log on to an ECS server in the same VPC and VSwitch (
VSwitch-BJ) with the CSG instance created in Step 2 (
On the ECS, mount the remote NFS-shared directory (for example,
/share1nfs). And then copy files (for example
file3) to the mounted directory.
bj-ecs# showmount -e 192.168.1.131
Export list for 192.168.1.131:
/mnt/36800fa0-d0dd-4a92-bc33-095ee6e535a7/nfsexport1 192.168.1.0/24bj-ecs# mkdir csg-share
bj-ecs# mount 192.168.1.131:/share1nfs csg-share/
bj-ecs# df -h
Filesystem Size Used Avail Use% Mounted on
192.168.1.131:/share1nfs 80G 0 80G 0% /home/csg-sharebj-ecs# cp file1 file2 file3 /home/csg-share
bj-ecs# ll /home/csg-share
drwxrwxrwx 2 root root 0 Jan 11 01:27 ./
drwxr-xr-x 3 root root 4096 Jan 19 14:35 ../
-rw-r--r-- 1 root root 36 Jan 19 14:38 file1
-rw-r--r-- 1 root root 36 Jan 19 14:38 file2
-rw-r--r-- 1 root root 36 Jan 19 14:38 file3bj-ecs# tail -n +1 file*
==> file1 <==
this is file1 uploaded by ECS in BJ==> file2 <==
this is file2 uploaded by ECS in BJ==> file3 <==
this is file3 uploaded by ECS in BJ
Check the OSS console and see that the files have been transferred from the CSG File Gateway in Beijing to the OSS bucket oss-file-sharing in Hong Kong.
Step 4: Creating the Overseas-side CSG File Gateway — On-Premises Deployment
Now, deploy an on-premises CSG instance and a File Gateway by following instructions at How to Install Cloud Storage Gateway. Alternatively, you can opt to deploy an online CSG instance in another Alibaba Cloud region by following instructions at Step 2.
You will need to log on to the CSG web console to configure the required settings: Caches, Cloud Resources, and CIFS settings. Following shows the configurations of the CIFS share for example:
Notice that we select
Sync Mode for Data Access Mode and
Yes for Enable Remote Sync.
Above screenshot shows that a SMB User named user2 is added for the SMB Share.
Step 5: Mounting Overseas-side CIFS Share — A Windows Client Example
Next, you operate a Windows system, which is in an intranet with the CSG instance created at Step 4., to mount the CIFS share (for example,
\\192.168.5.36\share2) as network drive. You can now see those 3 files under the mounted directory which maps to the backend OSS bucket (namely
Step 6: Testing File Sharing
So far, you have completed the architecture where 2 cross-region clients being able to access NAS filesystem, shared by a near-client CSG. Now you can further test to do write and read operations using either client, and the CSG will transparently synchronize and map the its file structure to the one backend OSS bucket.
- At the time of writing, deletion is not supported by Remote Sync and will be supported soon; stay tuned (eta. end of January 2019).
- The present file sharing pattern is in a weak consistency fashion so that you should consider Read-Write Separation at the time of design. In our example, if the China-side application is the only writer, we recommend to enforce read-only mount in the overseas-side file gateway to avoid write conflicts.
Featured by CSG file gateway and OSS, we can now implement a simple and effective architecture for sharing and distribution of files. In addition, the NAS-based filesystem reserves the convention usage of file management user interface.
Given the base of cross region and hybrid (online and on-premises) architecture, now you will be able to make tweaks to adapt to variations of scenarios.
To learn more about CSG, refer to https://www.alibabacloud.com/product/sgw.
You can refer to the following documentation for additional information.
- See the guide on How to Install Cloud Storage Gateway.
- Visit our product documentation on Cloud Storage Gateway.