Skip to main content
two aspects I didn't consider
Source Link
juhist
  • 5.2k
  • 6
  • 19

Furthermore, two aspects I didn't consider are under-utilization of resources and cooling. Under-utilization of resources would multiply the figures by probably 1.2 - 1.5 and cooling would multiply by 1.2 so you should multiply the total figures I calculated by 1.4 - 1.8 depending on how well the cloud service can prepare for sudden entering data spikes while still not having massive under-utilization.


Furthermore, two aspects I didn't consider are under-utilization of resources and cooling. Under-utilization of resources would multiply the figures by probably 1.2 - 1.5 and cooling would multiply by 1.2 so you should multiply the total figures I calculated by 1.4 - 1.8 depending on how well the cloud service can prepare for sudden entering data spikes while still not having massive under-utilization.

Source Link
juhist
  • 5.2k
  • 6
  • 19

Some time ago I saw a documentary/infotainment program on television where they said that on average storing 1TB of data in the cloud costs 41 kWh of electricity per year.

This statement doesn't have enough specs to be meaningful.

For example, consider Amazon AWS S3. It has at least these storage classes:

  • Standard (millisecond access)
  • Infrequent access (millisecond access)
  • Glacier instant retrieval (millisecond access)
  • Glacier deep archive (hour retrieval)

Most likely, the standard and infrequent access classes are implemented not by HDDs but by SSDs, although the infrequent access class could use cheaper SSDs whereas the standard class uses high-performance SSDs. However, a streaming service could very well have their own private clouds that have HDDs, and then whenever someone is watching some video file, it could read it to memory a megabyte at a time, because HDDs are slow to seek but acceptably fast to read once the seek is done.

The Glacier instant retrieval is most likely done by online hard disks, whereas the glacier deep archive is most likely done by offline hard disks, offline tape libraries or similar, that are only turned on when requested.

Also, it is possible in many storage services to specify reduced redundancy. This means the storage will be divided to fewer redundant drives. Normally AWS S3 stores objects in three redundant locations, but you can reduce the number of redundant locations to two if the data can be easily re-created.

For standard class, 3-times redundancy, they most likely are using some kind of enterprise SSD. Samsung SM883 3.8TB uses about 3 watts of power, very minimal. However, usually you can put only 24 of these to a single server that consumes about ~150W itself, so the server consumes 150W and the disks consume 72W. Total 222W, or 1946 kWh per year. This stores 24*3.8 TB = 91.2 TB, but due to 3-way redundancy, useful capacity is 30.4 TB.

Therefore, 1 TB per year is 64 kWh, quite close to the 41 kWh figure. Actually reducing the redundancy from 3-way to 2-way would give 43 kWh per year, very close to the 41 kWh per year figure.

However, this was for SSDs. A HDD that stores 24 TB consumes 8 watts. If you put 24 of these to a single computer, the computer consumes ~150W and the disks 192W for a total of 342W or 2998 kWh per year. The storage is 24*24 TB = 576 TB. At a 3-way redundancy, useful storage is 192 TB and at a 2-way redundancy, useful storage is 288 TB. So consumption is 16 kWh per year per TB (3-way redundancy) or 10 kWh per year per TB (2-way redundancy).

So as you can see, for data that's rarely accessed but still needs instant retrieval, you can reduce from the ~40 kWh per year figure. Realistically a streaming service like Netflix could use HDDs as opposed to SSDs, but I'm not completely sure since SSDs have far better performance even if the data can be mostly sequentially read like in streaming services with optimization for reading the data into memory in 1 MB blocks. I think a streaming service could probably use both: SSDs for often-accessed content and HDDs for rarely accessed content.

Also, nothing beats deep archive. Amazon AWS S3 glacier deep archive probably has a fraction of these costs. However, since retrieving data from it takes hours, it's only useful for backups.