Surviving the IoT Flood

Exploring IoT Storage Demands

Last month’s blog addressed edge computing and how it supports the Internet of Things (IoT). Now, let’s dive further into the storage demands of IoT. Although the full scope of IoT storage can’t be covered in a single blog, here are some thoughts.

The IoT Data Explosion

We’ve all been introduced to an IoT-powered future, where smart homes, cities, and devices are part of our daily lives. IoT devices are generating massive amounts of data—from sensors in cars, buildings, and even our bodies. A 2017 IDC white paper forecasted that by 2025, IoT devices worldwide will generate a staggering 40 zettabytes of real-time data, or 40 billion terabytes.

These data will need to be moved, stored, and shared, with value extracted from them to justify the collection effort. The question is: where will all this data reside?

Understanding the Diversity of IoT Data

IoT data varies widely, from small file logs to large video surveillance files. The storage approach depends on the type of data and its use case:

• Real-Time Data: Some data, like critical avionics or smart car communication at an intersection, need immediate processing for safety or well-being.
• Soon-to-be Processed Data: Industrial sensors that predict failure require prompt analysis, but not necessarily in real-time.
• Data for Later Processing: For example, geological data from an oil exploration company or periodic inventory updates from vending machines.

Storage Options Based on Data Nature

The nature of the data will determine where it is stored and for how long:

• Temporary Storage: Real-time data analyzed immediately may not need to be stored long-term, only cached temporarily.
• Long-Term Storage: Some data, such as video surveillance, needs to be stored for a set period.
• Data for Discarding: Once analyzed, some data, like sensor readings for milk in your fridge, can be discarded.

Cloud, Edge, or Datacenter?

With the growing volume of IoT data, where should it reside?

• Cloud Storage: Ideal for data like video surveillance files, though large data streams can congest pipelines to the cloud.
• Edge Computing: For ongoing data streams, edge computing allows for processing close to the data’s source, reducing latency and bandwidth consumption.
• Datacenters: Depending on the scale of the IoT data and your WAN capacity, datacenters might house the IoT data, but only if they have the resources to do so without hindering operations.

Critical Data and Real-Time Processing

Critical data requiring real-time analytics must be handled efficiently. For example, cars approaching an intersection need low-latency communication. In such cases, solid-state storage and edge computing become essential to meet these fast-paced demands.

Addressing Security and Backup Needs

Once you determine your storage and processing requirements, consider the security needs of each IoT data stream. What level of security is appropriate for each type of data? Finally, it’s crucial to decide what IoT data should be backed up and how to do so efficiently.

The IoT Revolution

IoT presents significant challenges, but it also offers unprecedented opportunities. As the IoT continues to grow, the potential for prosperity and well-being is immense—if we can solve the complex data storage and management problems that come with it.