author: @amanqa
source: github.com/amanahuja/change-detection-tutorial
# Modified Stylesheet for notebook.
from IPython.core.display import HTML
def css_styling():
styles = open("custom.css", "r").read()
return HTML(styles)
css_styling()
Lesson Plan
Part 01: Introduction
Introduction (what is change detection)
Justification (why does change detection matter)
Offline vs. Online change detection
References and Further Reading
import matplotlib.pyplot as plt
import numpy as np
from collections import defaultdict
np.random.seed(seed=111111)
np.set_printoptions(precision=3, suppress=True)
signal = np.random.normal(scale=1, size=200)
signal[:50] += 50; signal[50:] += 55; signal[150:] -= 20
signal[110] = 0
plot(signal)
ylim(0,100)
title("An example signal")
<matplotlib.text.Text at 0x32b6210>
There are several points of change in this signal. We want to be able to detect these types of change, and distinguish between important changes and unimportant changes.
In an online algorithm, we are given a 'streaming' signal. We don't have information about the future behavior of the signal. So when we encounter the change at point 50, we have to decide whether it is important or not WITHOUT KNOWING about the larger change at 150, or the sharp drop at 110.
There are many types of signals, and "change" can mean very different things.
These are monthly-average daily calls to directory assistance Jan. 62 – Dec 76. We see a sudden drop in activity in this signal.
Data from datamarket
signal = [350,339,351,364,369,331,331,340,346,341,357,398,381,367,383,375,353,361,375,371,373,366,382,429,406,403,429,425,427,409,402,409,419,404,429,463,428,449,444,467,474,463,432,453,462,456,474,514,489,475,492,525,527,533,527,522,526,513,564,599,572,587,599,601,611,620,579,582,592,581,630,663,638,631,645,682,601,595,521,521,516,496,538,575,537,534,542,538,547,540,526,548,555,545,594,643,625,616,640,625,637,634,621,641,654,649,662,699,672,704,700,711,715,718,652,664,695,704,733,772,716,712,732,755,761,748,748,750,744,731,782,810,777,816,840,868,872,811,810,762,634,626,649,697,657,549,162,177,175,162,161,165,170,172,178,186,178,178,189,205,202,185,193,200,196,204,206,227,225,217,219,236,253,213,205,210,216,218,235,241]
plt.plot(signal, alpha = 0.5)
plt.title('Example: Directory Assistance Calls')
<matplotlib.text.Text at 0x3489410>
This is a fairly regular sinusoidal signal, shown with an anomaly where one of the waves is squashed.
signal = np.sin(np.linspace(0, 15*np.pi, num=300))
signal[105:155] *= 0.1
signal = 10 * signal + 50
noise = np.random.normal(scale = 1.5, size=300)
signal = signal + noise
plt.ylim(0,100)
plt.plot(signal)
plt.title('Example: periodic signal anomaly')
<matplotlib.text.Text at 0x34b97d0>
Clearly, it is very important to understand the context of the problem when building a change detection system. The type of signal, the specific domain, and the way in which the system will be used are all important considerations.
With this in mind, in this tutorial we will start with some simple signals in order to introduce fundamentals. Then we'll look separately at some broadly different contexts.
Return to the Change Detection Tutorial Table of Contents