Finding Anomalies Using STUMPI

[seanlaw]

@hmcoservit originally wrote:

@seanlaw According to my tests so far on anomaly detection, I can see the effect of Z-normalization more and more. It really makes it difficult to detect the peaks in time series (using STUMPI left MP). Also when sub sequences are flat (but noisy) the matrix profile would not give ideal results for anomaly discovery specially when the sub sequence value is small. Therefore, impacting the precision heavily. What do you think about this? I am not sure if it is possible to adapt it to suit anomaly detection more.

[seanlaw]

@hmcoservit Moving the discussion here as the other issue is focused on implementing non-normalized distances.

So, since matrix profiles mainly look at subsequences, if you are looking for (roughly) single point peaks in a time series then matrix profiles are not a good approach. Have you ever looked at:

Twitter’s Anomaly Detection

Or

Netflix Outlier Detection

Perhaps, you can provide some examples that we can explore together? I am sure that this will help others as well

[hmcoservit]

@seanlaw excluding contextual anomalies in time series, the target is to detect collective and point anomalies.

The effect of z-normalization might be more visible when either the sub sequence value is too small or there are flat regions in the time series as discussed in this paper. They propose a new solution to overcome this in the case of anomaly detection. Another thing that can be done to overcome this is to increase the sub sequence length to a long period, but doing so would eventually add delays to detection as new points coming in do not update the MP values right away.

Regarding Netflix's solution, is is mainly for processing data in batches, and unsuitable for real-time streaming applications according to this article . And for Twitter's one, it doesn't seem to be maintained and has been known to perform worse than RRCF according to NAB.

I will try to share some visual examples here as soon as possible ;)

[seanlaw]

@hmcoservit I'm going to close this for now but please feel free to re-open when you have some examples to share

[hmcoservit]

@seanlaw Thanks Sean, for sure.

[hmcoservit]

@seanlaw let me share a few examples here (simulating data as a stream), it would be great to have your opinion:

• The data is from NAB, can be accessed here
• Confidence area is calculated using the 3-sigma rule of the previous 400 matrix profile values (example for anomaly detection)
• Matrix profile is shifted forward to align it with the current seen (processed) sub-sequence
• Matrix profile is calculated incrementally using Stumpi and the LEFT_MP is displayed (updating the mp only on past subsequences)
• Plots are displayed for different m values (the dataset length is 4032 points)

It can be seen that the anomaly region is not evident in most of the matrix profile value probably due to Z-normalization. And finding robust parameters for m and moving average is difficult. I am very curious to see if aampi can better detect the start of collective anomalies or point anomalies.

[seanlaw]

@hmcoservit Thank you for the context. I took at stab at implementing aampi (i.e., stumpi but without z-normalization) - please take a look here