The phenomenon of big data has been pretty incredible and pretty exhausting for any potential business beneficiaries. That being said, every business leader needs to be aware of the impact that these various technologies will have on their ability to stay competitive.
One category of big data technology is called Hadoop. It is an open source software stack. Open source is a crowd-sourced project that no one owns and everyone can use freely.
What kind of software is Hadoop? Named after a stuffed elephant, Hadoop is at its core a really cheap, highly resilient, and infinitely scalable data storage software.
Here is a quick analogy for Hadoop. Let's say you were one of the original owners of the Apple iPod. Eventually, the size of storage and features change enough that you move on to the next generation and the next. Finally, you get tired of the associated price increase. Instead of going out and buying new versions, you flip the settings to run as data storage and connect all of the old or lesser versions together into one massive file system. Files are duplicated across multiple devices for redundancy. If some older device fails, you simply swap in a working device for the old one. You were previously limited to 64GB of data on a single device. Now that you have hooked all your devices together, you have plenty of space. How many pictures, songs, and movies can you store now?
Similarly, this is what is happening with computer servers. They each had their own individual purpose, and the hardware continually got better. Unfortunately, the cost had serious escalations which each new server. Why not develop a system that allows you to string together multiple commodity servers and overlay an absolutely brilliant methodology for replicating data? This methodology ensures that a single server failure warrants a simple machine swap rather than a complete rebuild of the system. All of a sudden, you have cheap infinite storage and the increasing ability to process that data at high-speed with the added compute power.
Who is using Hadoop for Business?
The current users tend to be in one of two camps: small businesses that have a ton of data they want to use to monitor their operations and large companies that have the resources and appropriate data volumes to discover new business efficiencies.
The smaller companies tend to be gaming companies, SaaS, or digital media companies where the very nature of the business generates a ton of system log data. The log data usually describes the actions and behaviors of the consumers of the various products. For example, multiple watchers of a video may experience a glitch at the same point in a movie they are watching. This would lead to abandonment of the session. As this occurs across multiple viewers, you can discover that there is a content issue that the company should resolve. The same thing goes for gaming. Your favorite MMO experiences a problem that hits multiple users. Find it, fix it, or the game goes away. You can see that the owners of data intensive businesses need to find inexpensive ways to collect and analyze data just to exist.
Larger companies are seeing the light on what it means to be a data-driven company. With more storage and horsepower on the same budget or less, companies are now able to store all kinds of data. Imagine consumer profiles that meld all of the historical account and CRM history (not just a few months) then fuse it with Twitter, Facebook, and purchased demographic information. What can you do with this? You can do some frightening things actually. Are you starting to notice that Facebook regularly predicts items and products you might be interested in? That is all Hadoop making it happen. The more data or attributes you can discover about a person the more precise the prediction. Marketing companies have been the early adopters given the deluge of data that is being produced by consumers and devices. More accuracy in the analysis means better product recommendations and more profits.
Why isn't everyone using Hadoop for Business?
There are a lot of companies that say they don't use social data or do not have "big data." They need to wake the heck up! They are already behind their competitors. If you have any dashboards or reports produced by your IT department that takes more than 5 seconds to return a result, your data is big enough. It isn't just about having a lot of data, it's about speed as well... the speed at which your business reacts to the market.
For most industries, there are opportunities where data processing speed across massive data volumes will create value and transform your business.
Revenue: If you own retail stores where the region is having a freak storm, how quickly can you price optimize your inventory for umbrellas or snow shovels?
Asset Efficiency: Can you constantly calculate the wear and tear on mechanical parts via sensor data and replace them before they cause an operational failure?
Operating Margin: Are you in oil and gas where seismic exploration throws off petabytes of data and takes days to mine? What if you could cheaply store all the data and analyze it in minutes?
The applications of Hadoop for business are limitless. Business leaders need to care. Moore's law is quickly shuttling us to an age of infinite compute possibilities. AI, Robotics, and Synthetic Biology are only a few of the items mentioned in the book "Bold" by Peter Diamandis and Steven Kotler, and these technologies are becoming democratized very quickly. This means you don't need technology experts to use them. User interfaces and cloud technologies are making them available to everyone now.
It is time to take off the hat of skepticism and put on the thinking cap of your own personal creativity. What can your business accomplish with Hadoop?
One category of big data technology is called Hadoop. It is an open source software stack. Open source is a crowd-sourced project that no one owns and everyone can use freely.
What kind of software is Hadoop? Named after a stuffed elephant, Hadoop is at its core a really cheap, highly resilient, and infinitely scalable data storage software.
Here is a quick analogy for Hadoop. Let's say you were one of the original owners of the Apple iPod. Eventually, the size of storage and features change enough that you move on to the next generation and the next. Finally, you get tired of the associated price increase. Instead of going out and buying new versions, you flip the settings to run as data storage and connect all of the old or lesser versions together into one massive file system. Files are duplicated across multiple devices for redundancy. If some older device fails, you simply swap in a working device for the old one. You were previously limited to 64GB of data on a single device. Now that you have hooked all your devices together, you have plenty of space. How many pictures, songs, and movies can you store now?
Similarly, this is what is happening with computer servers. They each had their own individual purpose, and the hardware continually got better. Unfortunately, the cost had serious escalations which each new server. Why not develop a system that allows you to string together multiple commodity servers and overlay an absolutely brilliant methodology for replicating data? This methodology ensures that a single server failure warrants a simple machine swap rather than a complete rebuild of the system. All of a sudden, you have cheap infinite storage and the increasing ability to process that data at high-speed with the added compute power.
Who is using Hadoop for Business?
The current users tend to be in one of two camps: small businesses that have a ton of data they want to use to monitor their operations and large companies that have the resources and appropriate data volumes to discover new business efficiencies.
The smaller companies tend to be gaming companies, SaaS, or digital media companies where the very nature of the business generates a ton of system log data. The log data usually describes the actions and behaviors of the consumers of the various products. For example, multiple watchers of a video may experience a glitch at the same point in a movie they are watching. This would lead to abandonment of the session. As this occurs across multiple viewers, you can discover that there is a content issue that the company should resolve. The same thing goes for gaming. Your favorite MMO experiences a problem that hits multiple users. Find it, fix it, or the game goes away. You can see that the owners of data intensive businesses need to find inexpensive ways to collect and analyze data just to exist.
Larger companies are seeing the light on what it means to be a data-driven company. With more storage and horsepower on the same budget or less, companies are now able to store all kinds of data. Imagine consumer profiles that meld all of the historical account and CRM history (not just a few months) then fuse it with Twitter, Facebook, and purchased demographic information. What can you do with this? You can do some frightening things actually. Are you starting to notice that Facebook regularly predicts items and products you might be interested in? That is all Hadoop making it happen. The more data or attributes you can discover about a person the more precise the prediction. Marketing companies have been the early adopters given the deluge of data that is being produced by consumers and devices. More accuracy in the analysis means better product recommendations and more profits.
Why isn't everyone using Hadoop for Business?
There are a lot of companies that say they don't use social data or do not have "big data." They need to wake the heck up! They are already behind their competitors. If you have any dashboards or reports produced by your IT department that takes more than 5 seconds to return a result, your data is big enough. It isn't just about having a lot of data, it's about speed as well... the speed at which your business reacts to the market.
For most industries, there are opportunities where data processing speed across massive data volumes will create value and transform your business.
Revenue: If you own retail stores where the region is having a freak storm, how quickly can you price optimize your inventory for umbrellas or snow shovels?
Asset Efficiency: Can you constantly calculate the wear and tear on mechanical parts via sensor data and replace them before they cause an operational failure?
Operating Margin: Are you in oil and gas where seismic exploration throws off petabytes of data and takes days to mine? What if you could cheaply store all the data and analyze it in minutes?
The applications of Hadoop for business are limitless. Business leaders need to care. Moore's law is quickly shuttling us to an age of infinite compute possibilities. AI, Robotics, and Synthetic Biology are only a few of the items mentioned in the book "Bold" by Peter Diamandis and Steven Kotler, and these technologies are becoming democratized very quickly. This means you don't need technology experts to use them. User interfaces and cloud technologies are making them available to everyone now.
It is time to take off the hat of skepticism and put on the thinking cap of your own personal creativity. What can your business accomplish with Hadoop?
No comments:
Post a Comment