Abstract
We propose a Kullback–Leibler divergence (KLD) filter to extract anomalies within data series generated by a broad class of proximity sensors, along with the anomaly locations and their relative sizes. The technique applies to devices commonly used in engineering practice, such as those mounted on mobile robots for nondestructive inspection of hazardous or other environments that may not be directly accessible to humans. The raw data generated by this class of sensors can be challenging to analyze due to the prevalence of noise over the signal content. The proposed filter is built to detect the difference of information content between data series collected by the sensor and baseline data series. It is applicable in a model-based or model-free context. The performance of the KLD filter is validated in an industrial-norm setup and benchmarked against a peer industrially adopted algorithm.
Issue Section:
Technical Briefs
References
1.
Khatkhate
,
A.
,
Ray
,
A.
,
Keller
,
E.
,
Gupta
,
S.
, and
Chin
,
S.
, 2006
, “
Symbolic Time-Series Analysis for Anomaly Detection in Mechanical Systems
,” IEEE/ASME Trans. Mechatron.
,
11
(4
), pp. 439
–447
.10.1109/TMECH.2006.8785442.
Lee
,
W.
, and
Xiang
,
D.
, 2001
, “
Information-Theoretic Measures for Anomaly Detection
,” Proceedings 2001 IEEE Symposium on Security and Privacy, S&P
,
IEEE
, Oakland, CA, May 14–16, pp. 130
–143
. 10.1109/SECPRI.2001.9242943.
Ghalyan
,
N. F.
, and
Ray
,
A.
, 2020
, “
Symbolic Time Series Analysis for Anomaly Detection in Measure-Invariant Ergodic Systems
,” ASME J. Dyn. Syst., Meas., Control
,
142
(6
), p. 061003
. 10.1115/1.40461564.
Liu
,
J.
,
Djurdjanovic
,
D.
,
Marko
,
K.
, and
Ni
,
J.
, 2009
, “
Growing Structure Multiple Model Systems for Anomaly Detection and Fault Diagnosis
,” ASME J. Dyn. Syst., Meas., Control
,
131
(5
), p. 051001
. 10.1115/1.31550045.
Chandola
,
V.
,
Banerjee
,
A.
, and
Kumar
,
V.
, 2009
, “
Anomaly Detection: A Survey
,” ACM Comput. Surv. (CSUR)
,
41
(3
), pp. 1
–58
.10.1145/1541880.15418826.
Dang
,
L.
,
Chen
,
B.
,
Wang
,
S.
,
Ma
,
W.
, and
Ren
,
P.
, 2020
, “
Robust Power System State Estimation With Minimum Error Entropy Unscented Kalman Filter
,” IEEE Trans. Instrum. Meas.
,
69
(11
), pp. 8797
–8808
.10.1109/TIM.2020.29997577.
Yu
,
G.
, 2020
, “
A Concentrated Time–Frequency Analysis Tool for Bearing Fault Diagnosis
,” IEEE Trans. Instrum. Meas.
,
69
(2
), pp. 371
–381
.10.1109/TIM.2019.29015148.
Wang
,
H.
,
Liu
,
Z.
,
Núñez
,
A.
, and
Dollevoet
,
R.
, 2019
, “
Entropy-Based Local Irregularity Detection for High-Speed Railway Catenaries With Frequent Inspections
,” IEEE Trans. Instrum. Meas.
,
68
(10
), pp. 3536
–3547
.10.1109/TIM.2018.28815299.
Huo
,
Z.
,
Martínez-García
,
M.
,
Zhang
,
Y.
,
Yan
,
R.
, and
Shu
,
L.
, 2020
, “
Entropy Measures in Machine Fault Diagnosis: Insights and Applications
,” IEEE Trans. Instrum. Meas.
,
69
(6
), pp. 2607
–2620
.10.1109/TIM.2020.298122010.
Sheinker
,
A.
,
Salomonski
,
N.
,
Ginzburg
,
B.
,
Frumkis
,
L.
, and
Kaplan
,
B.-Z.
, 2008
, “
Magnetic Anomaly Detection Using Entropy Filter
,” Meas. Sci. Technol.
,
19
(4
), p. 045205
.10.1088/0957-0233/19/4/04520511.
Spinello
,
D.
,
Gueaieb
,
W.
, and
Lee
,
R.
, 2011
, “
Entropy Filter for Anomaly Detection With Eddy Current Remote Field Sensors
,” 2011 IEEE International Symposium on Robotic and Sensors Environments (ROSE
),
IEEE
, Montreal, QC, Canada, Sept. 17–18, pp. 95
–100
. 10.1109/ROSE.2011.605854212.
Sheikhi
,
F.
,
Spinello
,
D.
, and
Gueaieb
,
W.
, 2015
, “
Rényi Entropy Filter for Anomaly Detection With Eddy Current Remote Field Sensors
,” IEEE Sens. J.
,
15
(11
), pp. 6399
–6408
.10.1109/JSEN.2015.245023613.
Della Pietra
,
S.
,
Della Pietra
,
V.
, and
Lafferty
,
J.
, 1997
, “
Inducing Features of Random Fields
,” IEEE Trans. Pattern Anal. Mach. Intell.
,
19
(4
), pp. 380
–393
.10.1109/34.58802114.
Gu
,
Y.
,
McCallum
,
A.
, and
Towsley
,
D.
, 2005
, “
Detecting Anomalies in Network Traffic Using Maximum Entropy Estimation
,” Proceedings of the 5th ACM SIGCOMM Conference on Internet Measurement
,
ACM Press
, Berkley, CA, pp. 345
–350
.15.
Afgani
,
M.
,
Sinanovic
,
S.
, and
Haas
,
H.
, 2008
, “
Information Theoretic Approach to Signal Feature Detection for Cognitive Radio
,” IEEE
Global Telecommunications Conference,
IEEE
, New Orleans, LA, Nov. 30–Dec. 4 pp. 1
–5
. 10.1109/GLOCOM.2008.ECP.60216.
Afgani
,
M.
,
Sinanovic
,
S.
, and
Haas
,
H.
, 2008
, “
Anomaly Detection Using the Kullback-Leibler Divergence Metric
,” 2008 First International Symposium on Applied Sciences on Biomedical and Communication Technologies
,
IEEE
, Aalborg, Denmark, Oct. 25–28, pp. 1
–5
. 10.1109/ISABEL.2008.471257317.
Afgani
,
M.
,
Sinanovic
,
S.
, and
Haas
,
H.
, 2009
, “
Hardware Implementation of a Kullback-Leibler Divergence Based Signal Anomaly Detector
,” Second International Symposium on Applied Sciences in Biomedical and Communication Technologies
, Bratislava, Slovakia, Nov. 24–27, pp. 1
–6
. 10.1109/ISABEL.2009.537361418.
Afgani
,
M. Z.
, 2011
, “
Exploitation of Signal Information for Mobile Speed Estimation and Anomaly Detection
,” Ph.D. thesis,
The University of Edinburgh
, Edinburgh, Scotland.19.
Zeng
,
J.
,
Kruger
,
U.
,
Geluk
,
J.
,
Wang
,
X.
, and
Xie
,
L.
, 2014
, “
Detecting Abnormal Situations Using the Kullback–Leibler Divergence
,” Automatica
,
50
(11
), pp. 2777
–2786
.10.1016/j.automatica.2014.09.00520.
Xie
,
L.
,
Zeng
,
J.
,
Kruger
,
U.
,
Wang
,
X.
, and
Geluk
,
J.
, 2015
, “
Fault Detection in Dynamic Systems Using the Kullback–Leibler Divergence
,” Control Eng. Pract.
,
43
, pp. 39
–48
.10.1016/j.conengprac.2015.05.01021.
Youssef
,
A.
,
Delpha
,
C.
, and
Diallo
,
D.
, 2016
, “
An Optimal Fault Detection Threshold for Early Detection Using Kullback–Leibler Divergence for Unknown Distribution Data
,” Signal Process.
,
120
, pp. 266
–279
.10.1016/j.sigpro.2015.09.00822.
Chen
,
H.
,
Jiang
,
B.
, and
Lu
,
N.
, 2018
, “
An Improved Incipient Fault Detection Method Based on Kullback-Leibler Divergence
,” ISA Trans.
,
79
, pp. 127
–136
.10.1016/j.isatra.2018.05.00723.
Jabari
,
S.
,
Rezaee
,
M.
,
Fathollahi
,
F.
, and
Zhang
,
Y.
, 2019
, “
Multispectral Change Detection Using Multivariate Kullback-Leibler Distance
,” ISPRS J. Photogramm. Remote Sensing
,
147
, pp. 163
–177
.10.1016/j.isprsjprs.2018.11.01424.
Bouhlel
,
N.
, and
Dziri
,
A.
, 2019
, “
Kullback–Leibler Divergence Between Multivariate Generalized Gaussian Distributions
,” IEEE Signal Process. Lett.
,
26
(7
), pp. 1021
–1025
.10.1109/LSP.2019.291500025.
Harmouche
,
J.
,
Delpha
,
C.
, and
Diallo
,
D.
, 2016
, “
Incipient Fault Amplitude Estimation Using KL Divergence With a Probabilistic Approach
,” Signal Process.
,
120
, pp. 1
–7
.10.1016/j.sigpro.2015.08.00826.
Hamadouche
,
A.
,
Kouadri
,
A.
, and
Bakdi
,
A.
, 2017
, “
A Modified Kullback Divergence for Direct Fault Detection in Large Scale Systems
,” J. Process Control
,
59
, pp. 28
–36
.10.1016/j.jprocont.2017.09.00427.
Mezni
,
Z.
,
Delpha
,
C.
,
Diallo
,
D.
, and
Braham
,
A.
, 2018
, “
Bearings Ball Fault Detection Using Kullback Leibler Divergence in the EMD Framework
,” 2018 Prognostics
and System Health Management Conference (PHM-Chongqing),
IEEE
, Chongqing, China, Oct. 26–28, pp. 729
–734
. 10.1109/PHM-Chongqing.2018.0013028.
Abid
,
A.
,
Khan
,
M. T.
,
Iqbal
,
J.
, and
De Silva
,
C.
, 2022
, “
Immunity Inspired Hybrid Fault Diagnosis and Conflict Resolution
,” IEEE/ASME Trans. Mechatron.
,
27
(1
), pp. 440
–451
.10.1109/TMECH.2021.306598129.
Tanaka
,
S.
, and
Müller
,
P. C.
, 1990
, “
Fault Detection in Linear Discrete Dynamic Systems by a Pattern Recognition of a Generalized-Likelihood-Ratio
,” ASME J. Dyn. Syst., Meas., Control
,
112
(2
), pp. 276
–282
.10.1115/1.289613630.
Harmouche
,
J.
,
Delpha
,
C.
, and
Diallo
,
D.
, 2014
, “
Incipient Fault Detection and Diagnosis Based on Kullback–Leibler Divergence Using Principal Component Analysis: Part I
,” Signal Process.
,
94
, pp. 278
–287
.10.1016/j.sigpro.2013.05.01831.
Silva
,
J.
, and
Narayanan
,
S.
, 2006
, “
Average Divergence Distance as a Statistical Discrimination Measure for Hidden Markov Models
,” IEEE Trans. Audio, Speech, Lang. Process.
,
14
(3
), pp. 890
–906
.10.1109/TSA.2005.85805932.
Anderson
,
A.
, and
Haas
,
H.
, 2011
, “
Kullback-Leibler Divergence (KLD) Based Anomaly Detection and Monotonic Sequence Analysis
,” 2011 IEEE Vehicular Technology Conference (VTC Fall
),
IEEE
, San Francisco, CA, Sept. 5–8, pp. 1
–5
. 10.1109/VETECF.2011.609304133.
Atienza
,
D.
,
Bielza
,
C.
,
Diaz-Rozo
,
J.
, and
Larranaga
,
P.
, 2021
, “
Efficient Anomaly Detection in a Laser-Surface Heat-Treatment Process Via Laser-Spot Tracking
,” IEEE/ASME Trans. Mechatron.
,
26
(1
), pp. 405
–415
.10.1109/TMECH.2020.302461334.
Shannon
,
C. E.
, 1948
, “
A Mathematical Theory of Communication
,” Bell Syst. Tech. J.
,
27
(3
), pp. 379
–423
.10.1002/j.1538-7305.1948.tb01338.x35.
Kullback
,
S.
, and
Leibler
,
R. A.
, 1951
, “
On Information and Sufficiency
,” Ann. Math. Stat.
,
22
(1
), pp. 79
–86
.10.1214/aoms/117772969436.
Kullback
,
S.
, 1959
, Information Theory and Statistics
,
Wiley & Sons
, New York.37.
Cover
,
T. M.
, and
Thomas
,
J. A.
, 2006
, Elements of Information Theory
, 2nd ed.,
Wiley-Interscience
,
Hoboken, NJ
.38.
Murphy
,
K. P.
, 2012
, Machine Learning: A Probabilistic Perspective
, Adaptive Computation and Machine Learning Series,
MIT Press
,
Cambridge, MA
.39.
Callegari
,
C.
,
Giordano
,
S.
, and
Pagano
,
M.
, 2017
, “
An Information-Theoretic Method for the Detection of Anomalies in Network Traffic
,” Comput. Secur.
,
70
, pp. 351
–365
.10.1016/j.cose.2017.07.00440.
Texas Instruments
, 2015
, “
Proximity Sensing of Up to 30-Cm Range With > 15-dB SNR and Robust Capacitive Touch Reference Design
,” TIDA-060024, accessed Nov. 7, 2022, https://www.ti.com/tool/TIDA-060024Copyright © 2023 by ASME
You do not currently have access to this content.
