Convergence of Big Data, IoE, and Edge Analytics
The emerging Internet of Everything (IoE) is connecting a new wave of devices -- from industrial sensors and wearable devices to retail cameras -- to the Internet. The data from these devices can reduce costs, increase revenue, and improve customer service, but only if that data can be analyzed and acted on quickly.
By Raghu Sowmyanarayanan
What is Edge Computing?
Edge computing is an analytics approach that analyzes data close to its source ("at the edge") instead of sending it to a remote server for analysis leveraging cloud computing. Such edge analytics will allow organizations (or even the devices themselves) to act on new insights within milliseconds rather than waiting for the data to be transmitted to a central data warehouse for processing and get action recommendations after a few hours or a day following batch processing. If we consider security camera as an example, the edge analytics triggering the immediate alarm could later be combined with camera data from multiple factories to identify long-term security trends leveraging central data warehouse.
Nowadays, high-data-rate sensors are becoming ubiquitous in the Internet of Everything (IoE). The speed and agility benefits are so great that most businesses believe that by 2018, 40 percent of IoE-created data will be stored, processed, analyzed, and acted upon close to -- or at the edge of -- the network rather than in centralized EDWs, according to the IDC FutureScape for Internet of Things report.
Edge Computing versus Big Data-Based Cloud Computing
The early days of the Internet of Things (IoT) have been characterized by the critical role of cloud platforms as application enablers. Intelligent systems have largely relied on cloud computing for their intelligence, and the actual devices of which they consist have been relatively less sophisticated. This is currently being shaken up, though, as the computing capabilities on the edge level (alarms, meters, sensors, etc.) advance faster than even a few years ago.
The ability to act quickly based on data gathered from devices (e.g., sensors) is enabling a major shift from the connected device paradigm (where devices pass the information to a centralized data warehouse and wait for recommendations) to the intelligent device paradigm (where devices can analyze and choose corrective actions). Edge computing, also known as edge intelligence, is what is driving this shift.
Edge analytics will exist in addition to, but won't replace, traditional big data analytics done in the enterprise data warehouse (EDW) or logical data warehouse (LDW). Data scientists will still process the majority of large, historical data sets for such purposes as price optimization and predictive analytics.
Edge Analytics Use Cases
The insights from these devices can be applied in marketing and manufacturing, as well as in energy, oil, aerospace, sales, product management, finance, customer support, and more.
Traditionally, offshore oil wells have transmitted data such as the status of drill bits through satellite or CDs to data centers for analysis, resulting in delays before the results can be relayed back to the rig. Edge analytics allows oil well operators to identify problems in a drill bit, even one operating several hundred feet below sea level, more quickly and take corrective action before a failure damages the bit or the well.
Adding analytics capabilities to security cameras allows real-time identification of unusual behavior, such as a group of people gathered by an entrance in the middle of the night. Rather than waiting to send that data to the cloud for analysis, the camera could identify the potential threat on site and trigger an alarm more quickly. An important type of analytics supported on intelligent devices (cameras) is automated modification of video streams to preserve privacy -- for example, editing out frames or blurring individual objects within frames. What needs to be removed or altered is highly specific to the owner of a video stream, but no user has time to go through and manually edit video captured on a continuous basis.
This automated, owner-specific lowering of fidelity of a video stream to preserve privacy is called denaturing. However, there has to be a balance between privacy and value of the data to be eliminated.
Location-based services (such as identifying open spaces in parking garages for smartphone users) can use local servers to process data in real time, providing more accurate results than centralized analysis, while reducing data transmission costs.
In gas transmission and distribution, more surveillance functionality is being pushed out from central servers to the sensors attached to meters or leak detectors. As this occurs, it becomes more desirable for the meters or leak detectors to perform some kind of analysis of the readings in their field of view and to make decisions about what to stream to the server, what to ignore, and what actions to take.
Device manufacturers are now embedding analysis capability into these sensors' firmware to achieve this. One advantage of this approach is the reduced demand on network bandwidth and storage requirements which can easily offset the additional cost of having on-board analytics. With improved server software, a matrix of sensors with on-board analytics engines can provide a powerful surveillance presence.
In automobile manufacturing, real-time analytics can be performed on sensor streams from the engine and other parts, alerting the driver to potential imminent failure or to the need for preventive maintenance. Such information can also be transmitted to the cloud or EDW for integration into a database maintained by the vehicle manufacturer. Fine-grain analysis of such anomaly data might reveal vehicle model-specific defects that can be corrected in a timely manner.
In the aerospace industry, the sensors in various parts of the airplane generate huge amount of data on the order of 1 terabyte per 24 hours. Intelligent devices (compared to connected devices) would be of great, and sometimes lifesaving, help as immediate proactive actions based on sensor readings could prevent crucial failures.
The Road Ahead
The industrial Internet is going to transform the industry by making industrial machines more intelligent and enabling services using real-time data coming from sensors and machines. The intelligent devices will be able to take actions (to optimize processes, improve efficiencies, reduce costs, etc.) based on insights generated from real-time data and analytics. This requires connectivity and interoperability across machines, fleets, plants, and cloud-based systems. Industrial communication standards will play an important role in providing seamless connectivity and integration between heterogeneous systems so that IoE-created data can be stored, processed, analyzed, and acted upon close to, or at the edge of, the network rather than in centralized EDWs.
Raghuveeran Sowmyanarayanan is a vice president at Accenture and is responsible for designing solution architecture for RFPs and opportunities. You can contact him at [email protected] .