How much energy does it take to store 1 Terabyte of data in the cloud?

Question

Some time ago I saw a documentary/infotainment program on television where they said that on average storing 1TB of data in the cloud costs 41 kWh of electricity per year. There was no further explanation, but I wrote that statistic down because I wanted to lookup more information on this later. I recently searched the Internet and found this paper that says

...a server-based architecture for a 1TB plan has an annual consumption of 1700 kWh

This is a rather different figure than 41 kWh, so which figure makes more sense? 1700 or 41 kWh? Is there any other scientific research on this, that can verify or refute these numbers?

Answer 1

According to a 2016 blog post by Backblaze, a (particular) 1.5TB HDD uses 3.4W when idle and 5.9W when operating. This gives a lower bound of 20 kWh / TB / year and an upper bound of 35 kWh / TB / year. These numbers are much closer to the low figure than the high one given in the question. We can also see that larger drives use proportionally less power and large-scale cloud storage providers will tend to move towards larger drives (for this and other reasons). Using a 6TB HDD instead results in power usage of 7.2W when idle and 8.8W when operating. This gives a lower bound of 10 kWh / TB / year and an upper bound of 13 kWh / TB / year.

The referenced paper arrives at its number by assuming the drive portion of the power cost is 4.9 kW / 604 TB (unclear what the reference for this is) or around 8 W / TB. This is right in the ballpark of the numbers given by the Backblaze blog post.

The paper then goes on to multiply this power usage by various factors - eg by 1.5 for account for cooling, by 2 for assumed redundancy, and by 2 again for assumed waste due to under-utilization. It then adds in other power factors, routing hardware, all intermediate servers between client and server, etc.

These seem like real contributors to power requirements but the total seems overstated to me. For example, the average 9 servers between client and server necessary to route traffic between them would probably mostly be up and running regardless of that TB of cloud storage. Their cost should be spread across the wide range of services they are providing (mostly Netflix streams at this point, probably).

It is also necessary to compare apples to apples. The paper talks about storage with some form of redundancy in it - assuming that storing 1TB of data requires 2TB worth of disk. The other number in the question, 41 kWh / TB / year, may be for no redundancy or some other level of redundancy. The figures I've computed above are for exactly 1TB - not 2TB.

Properly allocating the power used by all of these shared services is probably a research project all on its own. The authors of this paper have selected values which make Cubbit (an alternative distributed storage system) look like an improvement but I don't see evidence in the paper to support all of the assumptions they've made.

So, I have found no research that provides a complete answer to this question. My personal guess is that usage for a single non-redundant TB is between 2x and 10x the drive requirement - between 20 kWh and 100 kWh / TB / year. It would be great to see a more thorough analysis (or measurement!) spanning the full range of infrastructure, though.

Answer 2

The open-source project Cloud Carbon Footprint estimated the carbon footprint of storing 1TB in the Cloud. In their methodology they explain that they use the following numbers:

HDD average capacity in 2020 = 10 Terabytes per disk
Average wattage per disk for 2020 = 6.5 Watts per disk

Watts per Terabyte = Watts per disk / Terabytes per disk: 6.5 W / 10 TB = 0.65 Watt-Hours per Terabyte-Hour for HDD

and

SSD average capacity in 2020 = 5 Terabytes per disk
Average wattage per disk for 2020 = 6 Watts per disk

Watts per terabyte = Watts per disk / Terabytes per disk: 6 W / 5 TB = 1.2 Watt-Hours per Terabyte-Hour for SSD

So using this to calculate the energy needed to store something for a year (times 24 and 365,25) gives us 5.7 kWh for HDD and 10.5 kWh for SDD.

However cloud providers usually provide redundancy via RAID or backups which is not taken into account in this calculation, so that at least doubles the footprint. The Cloud Carbon Footprint project acknowledges this and has drawn up a list of the replication factors for several providers. According to this list a replication factor of 3 is commons, so that results in:

• 17.1 kWh/TB per year for HDD
• 31.6 kWh/TB per year for SSD

Note that this doesn't include the energy to transfer the data from the main storage to backup storages located elsewhere. Also data center PUE doesn't seem to be included, so that can increase the numbers with a factor 1.1 - 1.6.

Answer 3

Some time ago I saw a documentary/infotainment program on television where they said that on average storing 1TB of data in the cloud costs 41 kWh of electricity per year.

This statement doesn't have enough specs to be meaningful.

For example, consider Amazon AWS S3. It has at least these storage classes:

• Standard (millisecond access)
• Infrequent access (millisecond access)
• Glacier instant retrieval (millisecond access)
• Glacier deep archive (hour retrieval)

Most likely, the standard and infrequent access classes are implemented not by HDDs but by SSDs, although the infrequent access class could use cheaper SSDs whereas the standard class uses high-performance SSDs. However, a streaming service could very well have their own private clouds that have HDDs, and then whenever someone is watching some video file, it could read it to memory a megabyte at a time, because HDDs are slow to seek but acceptably fast to read once the seek is done.

The Glacier instant retrieval is most likely done by online hard disks, whereas the glacier deep archive is most likely done by offline hard disks, offline tape libraries or similar, that are only turned on when requested.

Also, it is possible in many storage services to specify reduced redundancy. This means the storage will be divided to fewer redundant drives. Normally AWS S3 stores objects in three redundant locations, but you can reduce the number of redundant locations to two if the data can be easily re-created.

For standard class, 3-times redundancy, they most likely are using some kind of enterprise SSD. Samsung SM883 3.8TB uses about 3 watts of power, very minimal. However, usually you can put only 24 of these to a single server that consumes about ~150W itself, so the server consumes 150W and the disks consume 72W. Total 222W, or 1946 kWh per year. This stores 24*3.8 TB = 91.2 TB, but due to 3-way redundancy, useful capacity is 30.4 TB.

Therefore, 1 TB per year is 64 kWh, quite close to the 41 kWh figure. Actually reducing the redundancy from 3-way to 2-way would give 43 kWh per year, very close to the 41 kWh per year figure.

However, this was for SSDs. A HDD that stores 24 TB consumes 8 watts. If you put 24 of these to a single computer, the computer consumes ~150W and the disks 192W for a total of 342W or 2998 kWh per year. The storage is 24*24 TB = 576 TB. At a 3-way redundancy, useful storage is 192 TB and at a 2-way redundancy, useful storage is 288 TB. So consumption is 16 kWh per year per TB (3-way redundancy) or 10 kWh per year per TB (2-way redundancy).

So as you can see, for data that's rarely accessed but still needs instant retrieval, you can reduce from the ~40 kWh per year figure. Realistically a streaming service like Netflix could use HDDs as opposed to SSDs, but I'm not completely sure since SSDs have far better performance even if the data can be mostly sequentially read like in streaming services with optimization for reading the data into memory in 1 MB blocks. I think a streaming service could probably use both: SSDs for often-accessed content and HDDs for rarely accessed content.

Also, nothing beats deep archive. Amazon AWS S3 glacier deep archive probably has a fraction of these costs. However, since retrieving data from it takes hours, it's only useful for backups.

---

Furthermore, two aspects I didn't consider are under-utilization of resources and cooling. Under-utilization of resources would multiply the figures by probably 1.2 - 1.5 and cooling would multiply by 1.2 so you should multiply the total figures I calculated by 1.4 - 1.8 depending on how well the cloud service can prepare for sudden entering data spikes while still not having massive under-utilization.

Answer 4

Solid state memory is far more energy efficienct than cloud storage, because said devices must be ON. ALL the time. In 2012, analysts at the New York Times estimated cloud computing consumed 30 Gigawatts of power per year, or as much as can be produced by 30 nuclear power plants, 262 Terawatt-hours of electricity. As of 2020 he world's data centers store about 1,327 exabytes of data, that works out to five megabytes per watt hour

Answer 5

The largest use of energy in cloud storage is keeping the servers cool. Which is not taken into consideration with your in home or office 5 or 10 TB hard drives. Which is probably taken into consideration in the third answer because they are looking at the total usage of the entire storage unit or building.