How Shape Intelligence Can Maximize Time-Series Analytics
Analyzing time-series data can lead an organization to new insights, but you can push that intelligence by supplementing time-series analytics with shape intelligence. Dr. Rado Kotorov, CEO of Trendalyze.com, explains what this pattern-analytics technology can add.
- By James E. Powell
- February 21, 2020
Upside: Where is time-series data most relevant to a business?
Rado Kotorov: Time-series data is relevant in all industries thanks to the digitization of all processes and business revenue generating models. Let me elaborate on this.
There are many forms of digitization such as installing sensors to monitor and control operational processes; digitizing customer interactions and providing web-based self-service; and automation of transactional processes. Digitization, in turn, allows us to collect very granular data -- data collected over milliseconds, seconds, and minutes. This is time-series data but it has never been collected before at such a low level of granularity. To use an analogy, the benefits from collecting and analyzing data at such a level of detail are the same as the benefits from the invention of the microscope.
One can do many things with such rich information and knowledge. The best-known example is the use of low granularity time-series data in high-frequency trading which produced many large hedge funds. However, the same can be done for industrial equipment monitoring where vibration sensors monitor for potential machine failures by scanning operational patterns on the nanosecond level -- 45,000 data points collected per second.
It can also be used for remote patient monitoring, where wearable ECG devices record 1000 data points per second -- data that can reveal patterns of different pathologies. In retail, such high granularity transactional data reveals promotional opportunities and business issues.
All in all, digitization is pushing all industries to transition to high-frequency management and operate like high-frequency trading operations. They have to react to threats and opportunities just in time.
What kinds of insights can time-series data reveal?
Granular time-series data is like DNA and ECG data -- it reveals the essence and health status of every process, event, or behavior. DNA contains repeating sequences called motifs that capture our essence and individual differences. Those DNA sequences have the same properties as time-series data. Every pattern/shape in ECG data is either a healthy pattern or a signal that something is wrong. Likewise, granular time-series data reveals to us the normal flow of any operation or alerts us about things that may be going wrong. Such signals are action triggers and can be used to automate decision making and execute such decisions in nanoseconds. The important thing is that as we learn the meaning of the shapes in time-series data, these patterns/shapes/motifs become action words -- they become a way for us to understand every monitored process. They enable human/machine communication.
Your company looks for greater insights using what you call shape intelligence. What is shape intelligence? What kinds of shapes are we talking about?
We are looking for meaningful shapes -- shapes that tell business professionals what actions to take. Like the shapes in ECG readings tell the doctor what the diagnosis is. Another example is "chart patterns trading." Traders study the patterns in the market, identify certain shapes, give them names, and when they see them again, they take action. The chart patterns prescribe the action. There is the double-bottoms pattern that looks like a "w," which signals a buy action. There is also the double-tops pattern that looks like an inverted "w" or a "m" and signals a sell action.
The interesting part is that both patterns have the same average. If a trader were to act on the average which decision would he make: buy or sell? This illustrates that the shapes provide a richer insight than the average. There is a saying in the financial markets that "the trend is your friend" which captures exactly the importance of the shape in decision making.
Scientists have studied for a long time shapelets, motifs, and patterns. This is not new. What we introduce are mathematical and logical approaches to machine learning of shapes and making shape-based predictions about future pattern occurrences. Learning plus prediction is intelligence. We have patents pending on these methods and technologies.
What kind of information do those shapes reveal that aren't evident from traditional time-series analysis?
The big difference with traditional timeseries is the granularity. Traditional time-series analysis reveals macro trends. Granular time-series analysis reveals micro trends. The shapes of those micro trends are very often the causes of something. On an ECG, a heart attack has a very specific shape that provides an unambiguous diagnosis. Hence, I like to say that every meaningful shape is a word that domain experts understand and can act on. You cannot do this with traditional time series.
Are there any downsides to shape intelligence? Is there something that it can't do (either because of processing power limitations or other restrictions, for example)?
It is an evolving field. Today, at Trendalyze, we can do time-series data only, but time-series data is sequential -- data that has a strict order. Because image data is sequential data, theoretically it can be applied for image recognition, too. Essentially all data -- image, video, text, voice -- is sequential data and thus shape intelligence can be applied to it.
There are many scientists working on this. There are three advantages compared to statistical learning:
- Shape intelligence does not require any training data
- It works with only a few shape examples
- Its predictions do not come from a black box (it is completely explainable, which has huge benefits).
Where is shape intelligence headed? What additional insights do you expect this technology will be able to provide in a year or two that it can't provide now?
It is an evolving field of AI and machine learning, but it is not statistical machine learning. It is based on logic and math. What we are working on is how to create shape-based artifical networks. There is a theory of the brain that says that we recognize objects based on about 10,000 stored shapes. You see something and the brain searches and makes mental comparisons. If it finds a matching shape, it signals that the object is known. If it does not, it commits the new object to memory.
This is a very simple but efficient learning system because I need to show an object only once to learn it instead of having to feed thousands of examples as in statistical machine learning. I can show you a water glass and a wine glass and you immediately get the shape difference, but in machine learning I have to feed you thousands of images to learn.
Imagine the impact on evolution if we had to learn what a bear is from 40,000 encounters with bears instead of from two or three. We would be extinct. The speed of leaning matters, but the even bigger benefit is that this type of intelligence is humanly understandable and controllable -- it is not a black box. It is only humans that can label a shape as meaningful and assign an action to it.
About the Author
James E. Powell is the editorial director of TDWI, including research reports, the Business Intelligence Journal, and Upside newsletter. You can contact him
via email here.