Dealing with terabytes to exabytes in AWS Cloud Migrations..!
Use of AWS Snowball and Snowmobile
First, let’s see what Cloud Computing is! Simply Cloud computing is the on-demand availability of computer system resources, especially data storage and computing power. Many organizations have used to deal with cloud computing services as the benefits they receive there include Scalability, Flexibility, Lowered IT costs, and much more.
If your organization ran a system before which was not hosted in Cloud Computer Service and now you want your system to be hosted in a cloud computing service such as Azure, AWS, or Google Cloud, you need to upload all your data and applications to the cloud to start the process.
Uploading some data and applications makes no sense at all, isn’t it?
Yeah, it won’t if we transfer a couple of gigabytes. The real problem comes when we have to deal with many terabytes and by the way some petabytes. If you’re lucky enough to have a high-speed internet connection, it will take days! If not weeks To upload. So here we have some quick math regarding this.
- Let’s assume we got 80 terabytes for the migrational data to be moved to the cloud. And we got a 1 Gbps connection to the internet.
- So the business has decided that we’re allowed to use the entirety of the connection for our migration transfer. So how long are we looking at? Well. a terabyte is about 1000 gigabytes, then we got 80,000 gigabytes to copy.
- As our connection is 1Gbps we have an upload speed of 0.125 gigabytes per second to upload(1 GBps). Then 80,000/0.1260/60/24 = 7.4 Days to upload by a 1Gbps connection.
It looks like we’ll be done in just a week. Additionally, we can run our business without the internet for a week, right? Assuming a week of downtime isn’t going to fly with the management, then we need to look into other options to migrate the data.
Woah! What do we have except the internet?
Then the AWS Snowball came to the stage! Well, Snowball is a physical device that is supplied by AWS Web Services and it can be connected directly to your network in the data center, and leverage the local network to copy the data. Actually, It is a huge disk drive you can ship to your organization, and after migrating the system and the data you could ship it back to the AWS data center and connect it there..! Pretty simple. A Snowball device can hold up to 80 terabytes and is protected by AWS KMS(Key Management Service to encrypt the data ) Using Snowball you can import and export data between on-premises data storage locations.
So we have a solution for a scenario of Terabytes. What will happen if we have to deal with petabytes or exabytes? Well AWS got a drive on wheels for that! It’s a moving truck and it’s called AWS Snowmobile. Literally a semi-truck. In this storage on wheels, the service holds up to 100 petabytes of data. If you wanna migrate this kind amount of data to AWS cloud! The snowmobile will be parked at your doorstep of the organization.
By the way, if we started migrating 1 exabyte of data from an internet connection of 10Gbps it would take about 30 years to complete the transaction. Using the Snowmobile, it takes only 6 months for the full transaction.
Except for us, this is a great service for Organizations who deal with this kind of big numbers..!
