Adopting a hybrid approach to streaming analytics in manufacturing
Apr 23, 2021 Comments (0)
Manufacturers strive for maximum efficiency at the lowest cost.
To help deliver this, many businesses are investing in real-time or streaming analytics solutions to enable faster, smarter decision making. From predictive maintenance to supply chain automation and production yield, these technologies can be game changers for manufacturers.
But there’s a problem. As the volume, velocity and variety of data that needs to be managed increases exponentially – driven mainly by the rise of connected devices at the edge – legacy technologies and processes for data management and analytics are struggling to cope. Designed and built for the world of big data and transaction systems, they are unable to handle diverse data situations common in manufacturing – particularly late, out-of-order or missing data – or deliver the performance required in the era of fast data, where real-time and historic data together with context need to be brought together for ‘in the moment’ analysis and decision making. One of the goals of bringing data together is to accurately represent the physical world – think of a digital twin. This involves joining machine generated time-series data together with other data to be able to interpret it correctly – such as information about assets, processes, recipes, and customers.
It’s important to stress that big data hasn’t gone away, far from it. Research from Statista shows that data created from IoT devices alone will hit 79.4 zettabytes by 2025, unsurprising when you consider a connected factory can create millions of data points a second from sensors in machinery. However, in an industry where precision matters, where even the smallest degree of latency can lead to machine downtime that hits output and revenue, manufacturers need to be able to analyze and act on insights derived by data in real-time, often automatically without any human oversight, to ensure operational and commercial goals are met.
Taking a hybrid approach
To bridge the worlds of big and fast data, firms should consider adopting a data management and analytics architecture that runs analytics as close to the edge as possible - to remove concerns around latency - while using the cloud to move, store and manage the data being created and analyzed.
Cloud computing has proven to be a transformational technology in manufacturing, providing significantly cheaper storage and processing capabilities while offering a marketplace of software and services that manufacturers can use to enhance operational performance and commercial growth, such as streaming analytics software.
However, concerns around security, reliability and latency have meant that its uptake has been slow in the manufacturing space. A data management architecture that contains a hybrid of edge computing and centralized data management, either on-premise or in the cloud, removes these concerns and allows for a more flexible and agile implementation of streaming analytics that will provider deeper understandings and insights.
Automating and optimizing using streaming analytics
Those deeper understandings and insights are critical to implementing what many see as the ultimate aim of real-time data analysis, namely autonomous decision making and machine-to-machine communication.
For these goals to be realized, any data management architecture needs to answer three critical requirements; the ability to ingest the enormous amounts of data, the ability to analyze it in a low latency environment and the ability to utilize machine learning to link or correlate multiple real-time and historic time-series and relational data sets to generate valuable actionable insights in the moment.
Echoing an early point, existing legacy data management and analytics technologies cannot deliver these requirements. Streaming analytics is the solution that enables the microsecond decision-making and allows manufacturers to not only manage but take advantage of the vast amount of data being created at the edge.
Przemek Tomczak, Senior Vice-President IoT and Utilities at KX where he leads the internet of things and utilities industry verticals globally. Przemek has over 24 years IT and business leadership experience, implementing and operating big data and analytics systems, delivering program and transformation initiatives, consulting, outsourcing and risk management in the energy and utility and other…