Executive Summary
Data Growth, Business Opportunities, and the IT Imperatives
This is the digital universe. It is growing 40% a year into the next decade, expanding to include not only the increasing number of people and enterprises doing everything online, but also all the “things” – smart devices – connected to the Internet, unleashing a new wave of opportunities for businesses and people around the world.
Like the physical universe, the digital universe is large – by 2020 containing nearly as many digital bits as there are stars in the universe. It is doubling in size every two years, and by 2020 the digital universe – the data we create and copy annually – will reach 44 zettabytes, or 44 trillion gigabytes.
Like the physical universe, it is diverse – created by everyone using a digital camera, by the more than 2 billion people and millions of enterprises living their lives and doing their work online, and by the millions of sensors and communicating devices sending and receiving data over the Internet. It includes Oscars-host Ellen DeGeneres’ “celeb selfie” tweet that was viewed 26 million times across the Web during a 12-hour period; the more than one billion hours of TV shows and movies streamed from Netflix per month; the data collected by sensors connected to a giant gas turbine and its analysis, making electricity cheaper and cleaner; and the data streaming at 2.8 Gigabytes per second from the Australian Square Kilometer Array Pathfinder (ASKAP) radio telescope.
But unlike the physical universe, the digital universe is created and defined by software, a man-made construct. It is defined by software that analyzes this ever-expanding universe of digital data, finding the hidden value and new opportunities to transform and enhance the physical world – keeping the Mars Rover roving, shipping money, or storing the pictures of our loved ones. And it is software that will both create new opportunities and new challenges for us as we try to extract value from the digital universe that we have created.
Hence the theme for this year’s digital universe study: The expanding universe of business opportunities. As more of the world goes online, including the physical world, the more opportunity there is for enterprises (and consumers) to use data in new ways – to learn about customers, speed business cycles, flatten organizational structures, and transform themselves into companies designed for what we call the “Third Platform” of the digital age, a platform built upon a foundation of cloud computing, mobility, social networking, and Big Data.
To maximize that opportunity, though, requires following certain imperatives for IT organizations. Information security, for example, virtualized datacenters, seamless public and private cloud computing, next- generation analytics, new storage management technologies, new data access tools and processes, automatic tagging, and the ability to deal with real-time data. And IT organizations will have to help their enterprises become more data- and software-driven.
Today, the digital universe has reached a number of new thresholds: The data coming from embedded systems (e.g., MP3 players, traffic lights, MRI scanners) has grown to a level where it’s starting to challenge established practices in datacenters; the migration to digital entertainment – movies and TV – is almost complete; and metadata (e.g., the data added to your email message describing when it was created), once tightly coupled with the data it describes, has grown into a category in and of itself, the fastest-growing subcategory of the digital universe.
This year’s iteration of the study is based on IDC’s estimates of the data generated by more than 40 types of devices, from RFID tags and sensors to supercomputers and supercolliders, from PCs and servers to cars and planes.
Here is a summary of the new findings:
- From 2013 to 2020, the digital universe will grow by a factor of 10 – from 4.4 trillion gigabytes to 44 trillion. It more than doubles every two years.
- Between 2013 and 2020 the division of the digital universe between mature and emerging markets (e.g., China) will switch – from 60% accounted for by mature markets to 60% of the data in the digital universe coming from emerging markets.
- In 2013, two-thirds of the digital universe bits were created or captured by consumers and workers, yet enterprises had liability or responsibility for 85% of the digital universe.
- In 2013, only 22% of the information in the digital universe would be a candidate for analysis, i.e., useful if it were tagged (more often than not, we know little about the data, unless it is somehow characterized or tagged – a practice that results in metadata); less than 5% of that was actually analyzed. By 2020, the useful percentage could grow to more than 35%, mostly because of the growth of data from embedded systems.
- Of the useful data, IDC estimates that in 2013 perhaps 5% was especially valuable, or “target rich.” That percentage should more than double by 2020 as enterprises take advantage of new Big Data and analytics technologies and new data sources, and apply them to new parts of the organization.
- In 2013, while about 40% of the information in the digital universe required some type of data protection, less than 20% of the digital universe actually had these protections.
- Data from embedded systems, the signals from which are a major component of the Internet of Things, will grow from 2% of the digital universe in 2013 to 10% in 2020.
- In 2013, less than 20% of the data in the digital universe is “touched” by the cloud, either stored, perhaps temporarily, or processed in some way. By 2020, that percentage will double to 40%.
- Most of the digital universe is transient – unsaved Netflix or Hulu movie streams, or Xbox One gamer interactions, temporary routing information in networks, sensor signals discarded when no alarms go off, etc. – and it is getting more so. This is a good thing, because the world’s amount of available storage capacity (i.e., unused bytes) across all media types is growing slower than the digital universe. In 2013, the available storage capacity could hold just 33% of the digital universe. By 2020, it will be able to store less than 15%.
- In 2014, the digital universe will equal 1.7 megabytes a minute for every person on Earth.
These estimates encapsulate the opportunities, challenges, and paradoxes of the digital universe. There is a lot of valuable data in the digital universe, but it will take determination and skilled workforce to find and put to use. It will need to be protected, analyzed, and acted upon.
There is an abundance of technical solutions, and successful early adopters. But organizations must adapt – and adapt fast, given that the digital universe more than doubles every two years. The foundation of that adaptation will be in the datacenter, but the rest of the organization, the one filled with people, tradition, culture, and habits, must also adapt.