The Voracious Appetites of Machine Learning & Artificial Intelligence

‍

Like the Internet of Things, machine learning (ML) and its more august sibling, artificial intelligence (AI), are upon us. The former is impacting business and IT operations, and the latter will impact nearly all of society. When most of us think of them, we envision incomprehensible algorithms and the brawny CPUs and GPUs all but running on nitro that bring them to life. Who thinks of quotidian storage?

The fact is the very foundation of AI and ML is data, lots of it, and the data must be stored somewhere. The data coming out of AI and ML are only as good as the data going into them. And the more data, the better. The larger the datasets, the more accurate will be the pattern recognition, correlations, analyses, and decision-making. The more data, the smarter our machines will be. This is true for any use case or workload, from sequencing genomes, improving agricultural yields, and scientific research to fraud detection, customer support, and self-driving automobiles.

Additionally, AI and ML generate data. Once AI/ML applications process their source data, the results will need to be safely stored and reused for further analyses.

Feeding the gluttony of AI and ML for data presents challenges. Data will come from many sources, such as business operations within the enterprise and IoT and social media from outside the enterprise. Data repositories must be extremely scalable, while still being cost-efficient, which often means hybrid infrastructures combining on-premise and cloud storage. Object storage will be a common solution for its ability to present vast troves of data in a single namespace.

Additionally, using high-octane GPUs will be wasted if the storage is a bottleneck. For this reason, AI and ML can best be served by flash drives, particularly for real-time use cases like assessing financial transactions.

Finally, AI and ML will improve storage itself. Vendors have already started to include logic in their offerings to better understand and manage enterprise environments. Armed with AI and ML, administrators will determine usage and detect patterns, and make more informed decisions about I/O patterns and data lifecycles. They’ll more accurately project future capacity needs and even perhaps predict failures, permitting proactive measures to safeguard operations.

The bottom line: if you’re planning on ML or AI applications in your enterprise, strongly consider the storage that will enable them. In storage, as in life itself, the one thing that never changes is that things are always changing.