Abstract

Precise time series prediction serves as an important role in constructing a digital twin (DT). The various internal and external interferences result in highly nonlinear and stochastic time series. Although artificial neural networks (ANNs) are often used to forecast time series because of their strong self-learning and nonlinear fitting capabilities, it is a challenging and time-consuming task to obtain the optimal ANN architecture. This paper proposes a hybrid time series prediction model based on an ensemble empirical mode decomposition (EEMD), long short-term memory (LSTM) neural networks, and Bayesian optimization (BO). To improve the predictability of stochastic and nonstationary time series, the EEMD method is implemented to decompose the original time series into several components (each component is a single-frequency and stationary signal) and a residual signal. The decomposed signals are used to train the neural networks, in which the hyperparameters are fine-tuned by the BO algorithm. The following time series data are predicted by summating all the predictions of the decomposed signals based on the trained neural networks. To evaluate the performance of the proposed EEMD-BO-LSTM neural networks, this paper conducts two case studies (the wind speed prediction and the wave height prediction) and implements a comprehensive comparison between the proposed method and other approaches including the persistence model, autoregressive integrated moving average (ARIMA) model, LSTM neural networks, BO-LSTM neural networks, and EEMD-LSTM neural networks. The results show an improved prediction accuracy using the proposed method by multiple accuracy metrics.

References

1.
Grieves
,
M.
,
2005
, “
Product Lifecycle Management: The New Paradigm for Enterprises
,”
Int. J. Prod. Dev.
,
2
(
1/2
), pp.
71
84
. 10.1504/IJPD.2005.006669
2.
Miller
,
A. M.
,
Alvarez
,
R.
, and
Hartman
,
N.
,
2018
, “
Towards an Extended Model-Based Definition for the Digital Twin
,”
Comput.-Aided Des. Appl.
,
15
(
6
), pp.
880
891
. 10.1080/16864360.2018.1462569
3.
Glaessgen
,
E. H.
, and
Stargel
,
D. S.
,
2012
, “
The Digital Twin Paradigm for Future NASA and U.S. Air Force Vehicles
,”
53rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference: Special Session on the Digital Twin
,
Honolulu, HI
,
Apr. 23–26
, pp.
1
14
.
4.
Tao
,
F.
,
Sui
,
F.
,
Liu
,
A.
,
Qi
,
Q.
,
Zhang
,
M.
,
Song
,
B.
,
Guo
,
Z.
,
Lu
,
S. C.-Y.
, and
Nee
,
A. Y. C.
,
2019
, “
Digital Twin-Driven Product Design Framework
,”
Int. J. Prod. Res.
,
57
(
12
), pp.
3935
3953
. 10.1080/00207543.2018.1443229
5.
Schleich
,
B.
,
Anwer
,
N.
,
Mathieu
,
L.
, and
Wartzack
,
S.
,
2017
, “
Shaping the Digital Twin for Design and Production Engineering
,”
CIRP Ann.—Manuf. Technol.
,
66
(
1
), pp.
141
144
. 10.1016/j.cirp.2017.04.040
6.
Tuegel
,
E. J.
,
Ingraffea
,
A. R.
,
Eason
,
T. G.
, and
Spottswood
,
S. M.
,
2011
, “
Reengineering Aircraft Structural Life Prediction Using a Digital Twin
,”
Int. J. Aerosp. Eng.
,
2011
, pp.
1
14
. 10.1155/2011/154798
7.
Boschert
,
S.
,
Heinrich
,
C.
, and
Rosen
,
R.
,
2018
, “Next Generation Digital Twin,”
Proc. TMCE 2018
,
I.
Horváth
,
J. P. S.
Rivero
, and
P. M. H.
Castellano
, eds.,
Organizing Committee of TMCE 2018, Las Palmas de Gran Canaria
,
Spain
, pp.
209
217
.
8.
Parrot
,
A.
, and
Warshaw
,
L.
,
2017
,
Industry 4.0 and the Digital Twin—Manufacturing Meets its Match
,
Deloitte University Press
,
New York
.
9.
Grieves
,
M.
,
Vickers
,
J.
, et al.,
2017
, “Digital Twin: Mitigating Unpredictable Undesirable Emergent Behavior in Complex Systems,”
Transdisciplinary Perspective on Complex Systems
,
F.-J.
Kahlen
, ed.,
Springer International Publishing
,
Switzerland
.
10.
Zhuang
,
C.
,
Liu
,
J.
, and
Xiong
,
H.
,
2018
, “
Digital Twin-Based Smart Production Management and Control Framework for the Complex Product Assembly Shop-Floor
,”
Int. J. Adv. Manuf. Technol.
,
96
(
1–4
), pp.
1149
1163
. 10.1007/s00170-018-1617-6
11.
Zhang
,
H.
,
LIu
,
Q.
,
Chen
,
X.
,
Zhang
,
D.
, and
Leng
,
J.
,
2017
, “
A Digital Twin-Based Approach for Designing and Mulit-Objective Optimization of Hollow Glass Production Line
,”
IEEE Access
,
5
, pp.
26901
26911
. 10.1109/ACCESS.2017.2766453
12.
Malik
,
A. A.
, and
Bilberg
,
A.
,
2018
, “
Digital Twins of Human Robot Collaboration in a Production Setting
,”
Procedia Manuf.
,
17
, pp.
278
285
. 10.1016/j.promfg.2018.10.047
13.
Tao
,
F.
,
Cheng
,
J.
,
Qi
,
Q.
,
Zhang
,
M.
,
Zhang
,
H.
, and
Sui
,
F.
,
2018
, “
Digital Twin-Driven Product Design, Manufacturing and Service with big Data
,”
Int. J. Adv. Manuf. Technol.
,
94
(
9–12
), pp.
3563
3576
. 10.1007/s00170-017-0233-1
14.
Marugán
,
A. P.
,
Márquez
,
F. P. G.
,
Perez
,
J. M. P.
, and
Ruiz-Hernández
,
D.
,
2018
, “
A Survey of Artificial Neural Network in Wind Energy Systems
,”
Appl. Energy
,
228
, pp.
1822
1836
. 10.1016/j.apenergy.2018.07.084
15.
Doucoure
,
B.
,
Agbossou
,
K.
, and
Cardenas
,
A.
,
2016
, “
Time Series Prediction Using Artificial Wavelet Neural Network and Multi-Resolution Analysis: Application to Wind Speed Data
,”
Renewable Energy
,
92
, pp.
202
211
. 10.1016/j.renene.2016.02.003
16.
Ma
,
L.
,
Luan
,
S.
,
Jiang
,
C.
,
Liu
,
H.
, and
Zhang
,
Y.
,
2009
, “
A Review on the Forecasting of Wind Speed and Generated Power
,”
Renewable Sustainable Energy Rev.
,
13
(
4
), pp.
915
920
. 10.1016/j.rser.2008.02.002
17.
Wang
,
Z.
,
Zhang
,
X.
,
Huang
,
H.-Z.
, and
Mourelatos
,
Z. P.
,
2016
, “
A Simulation Method to Estimate Two Types of Time-Varying Failure Rate of Dynamic Systems
,”
ASME J. Mech. Des.
,
138
(
12
), p.
121404
. 10.1115/1.4034300
18.
Kulkarni
,
P. A.
,
Hu
,
W.
,
Dhoble
,
A. S.
, and
Padole
,
P. M.
,
2017
, “
Statistical Wind Prediction and Fatigue Analysis for Horizontal-Axis Wind Turbine Composite Material Blade Under Dynamic Loads
,”
Adv. Mech. Eng.
,
9
(
9
), pp.
1
26
. 10.1177/1687814017724088
19.
Ma
,
J.
, and
Kim
,
H. M.
,
2016
, “
Predictive Model Selection for Forecasting Product Returns
,”
ASME J. Mech. Des.
,
138
(
5
), p.
054501
. 10.1115/1.4033086
20.
Lukoseviciute
,
K.
, and
Ragulskis
,
M.
,
2010
, “
Evolutionary Algorithms for the Selection of Time Lags for Time Series Forecasting by Fuzzy Inference Systems
,”
Neurocomputing
,
73
(
10–12
), pp.
2077
2088
. 10.1016/j.neucom.2010.02.014
21.
Wurdinger
,
K.
,
2009
,
Investigating an Evolutionary Strategy to Forecast Time Series
,
Universiteit Leiden Opleiding Informatica
,
Leiden, Netherlands
.
22.
Cadenas
,
E.
, and
Rivera
,
W.
,
2010
, “
Wind Speed Forecasting in Three Different Regions of Mexico, Using a Hybrid ARIMA—ANN Model
,”
Renewable Energy
,
35
(
12
), pp.
2732
2738
. 10.1016/j.renene.2010.04.022
23.
Snoek
,
J.
,
Larochelle
,
H.
, and
Adams
,
R. P.
,
2012
, “
Practical Bayesian Optimization of Machine Learning Algorithms
,”
Proceedings of the 25th International Conference on Neural Information Processing Systems
,
Lake Tahoe, NV
,
Dec. 3–6
, pp.
2951
2959
.
24.
Wu
,
Z.
, and
Huang
,
N. E.
,
2009
, “
Ensemble Empirical Mode Decomposition: A Noise-Assisted Data Analysis Method
,”
Adv. Adapt. Data Anal.
,
1
(
1
), pp.
1
41
. 10.1142/S1793536909000047
25.
Taieb
,
S. B.
,
Sorjama
,
A.
, and
Bontempi
,
G.
,
2010
, “
Multiple-Output Modeling for Multi-Step-Ahead Time Series Forecasting
,”
Neurocomputing
,
73
(
10–12
), pp.
1950
1957
. 10.1016/j.neucom.2009.11.030
26.
Lipton
,
Z. C.
,
Berkowitz
,
J.
, and
Elkan
,
C.
,
2015
, “A Critical Review of Recurrent Neural Networks for Sequence Learning,” arXiv:1506.00019v4 [cs.LG].
27.
Bengio
,
Y.
,
Simard
,
P.
, and
Frasconi
,
P.
,
1994
, “
Learning Long-Term Dependencies With Gradient Descent is Difficult
,”
IEEE Trans. Neural Networks
,
5
(
2
), pp.
157
166
. 10.1109/72.279181
28.
Hochreiter
,
S.
, and
Schmidhuber
,
J.
,
1996
, “
Bridging Long Time Lags by Weight and Guessing and “Long Short-Term Memory
”,
Spatiotemporal Models Biological Artificial Systems
,
37
, pp.
65
72
.
29.
Hochreiter
,
S.
, and
Schmidhuber
,
J.
,
1997
, “
Long Short-Term Memory
,”
Neural Comput.
,
9
(
8
), pp.
1735
1780
. 10.1162/neco.1997.9.8.1735
30.
Gers
,
F. A.
,
Schmidhuber
,
J.
, and
Cummins
,
F.
,
2000
, “
Learning to Forget: Continual Prediction With LSTM
,”
Neural Comput.
,
12
(
10
), pp.
2451
2471
. 10.1162/089976600300015015
31.
Kingma
,
D. P.
, and
Ba
,
J.
,
2014
, “Adam: A Method for Stochastic Optimization,” arXiv preprint arXiv:1412.6980.
32.
Srivastava
,
V. K.
, and
Fahim
,
A.
,
2007
, “
An Optimization Method for Solving Mixed Discrete-Continuous Programming Problems
,”
Comput. Math. Appl.
,
53
(
10
), pp.
1481
1491
. 10.1016/j.camwa.2007.01.006
33.
Geoffrion
,
A. M.
,
1972
, “
Generalized Benders Decomposition
,”
J. Optim. Theory Appl.
,
10
(
4
), pp.
237
260
. 10.1007/BF00934810
34.
Westerlund
,
T.
,
Skrifvars
,
H.
,
Harjunkoski
,
I.
, and
Pörn
,
R.
,
1998
, “
An Extended Cutting Plane Method for a Class of Non-Convex MINLP Problems
,”
Comput. Chem. Eng.
,
22
(
3
), pp.
357
365
. 10.1016/S0098-1354(97)00000-8
35.
Pelamatti
,
J.
,
Brevault
,
L.
,
Balesdent
,
M.
,
Talbi
,
E.-G.
, and
Guerin
,
Y.
,
2019
, “
Efficient Global Optimization of Constrained Mixed Variable Problems
,”
J. Global Optim.
,
73
(
3
), pp.
583
613
. 10.1007/s10898-018-0715-1
36.
Gelbart
,
M. A.
,
Snoek
,
J.
, and
Adams
,
R. P.
,
2014
, “Bayesian Optimization With Unknown Constraints,” arXiv preprint arXiv:1403.5607.
37.
Mockus
,
J.
,
Tiesis
,
V.
, and
Zilinskas
,
A.
,
1978
, “
The Application of Bayesian Methods for Seaking the Extreme
,”
Towards Global Optim.
,
2
, pp.
117
129
.
38.
Lyu
,
W.
,
Xue
,
P.
,
Yang
,
F.
,
Yan
,
C.
,
Hong
,
Z.
,
Zeng
,
X.
, and
Zhou
,
D.
,
2018
, “
An Efficient Bayesian Optimization Approach for Automated Optimization of Analog Circuits
,”
IEEE Trans. Circuits Systems-I: Regular Paper
,
65
(
6
), pp.
1954
1967
. 10.1109/TCSI.2017.2768826
39.
Lisicki
,
M.
,
Lubitz
,
W.
, and
Taylor
,
G. W.
,
2016
, “
Optimal Design and Operation of Archimedes Screw Turbines Using Bayesian Optimization
,”
Appl. Energy
,
183
, pp.
1404
1417
. 10.1016/j.apenergy.2016.09.084
40.
Berk
,
J.
,
Nguyen
,
V.
,
Gupta
,
S.
,
Rana
,
S.
, and
Venkatesh
,
S.
,
2019
, “Exploration Enhanced Expected Improvement for Bayesian Optimization,”
Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2018
,
Lecture Notes in Computer Science
, vol.
11052
,
M.
Berlingerio
,
F.
Bonchi
,
T.
Gärtner
,
N.
Hurley
, and
G.
Ifrim
, eds.,
Springer
,
Cham
.
41.
Tao
,
F.
,
Zhang
,
M.
,
Liu
,
Y.
, and
Nee
,
A. Y. C.
,
2018
, “
Digital Twin Driven Prognostics and Health Management for Complex Equipment
,”
CIRP Ann.—Manuf. Technol.
,
67
(
1
), pp.
169
172
. 10.1016/j.cirp.2018.04.055
42.
Kwong
,
W. Y.
,
Zhang
,
P. Y.
,
Romero
,
D.
,
Moran
,
J.
,
Morgenroth
,
M.
, and
Amon
,
C.
,
2014
, “
Multi-objective Wind Farm Layout Optimization Considering Energy Generation and Noise Propagation With NSGA-II
,”
ASME J. Mech. Des.
,
136
(
9
), p.
091010
. 10.1115/1.4027847
43.
Savory
,
E.
,
Parke
,
G. A. R.
,
Zeinoddini
,
M.
,
Toy
,
N.
, and
Disney
,
P.
,
2001
, “
Modelling of Tornado and Microburst-Induced Wind Loading and Failure of a Lattice Transmission Tower
,”
Eng. Struct.
,
23
(
4
), pp.
365
375
. 10.1016/S0141-0296(00)00045-6
44.
Taylor
,
S. W.
,
Wotton
,
B. M.
,
Alexander
,
M. E.
, and
Dalrymple
,
G. N.
,
2004
, “
Variation in Wind and Crown Fire Behaviour in a Northern Jack Pine—Black Spruce Forest
,”
Can. J. Forest Res.
,
34
(
8
), pp.
1561
1576
. 10.1139/x04-116
45.
Belcastro
,
C. M.
, and
Foster
,
J. V.
,
2010
, “
Aircraft Loss-of-Control Accident Analysis
,”
AIAA Guidance, Navigation, and Control Conference
,
Toronto, Ontario Canada
,
Aug. 2–5
, pp.
1
41
.
46.
Smith
,
T. G.
,
2017
, “Pmdarima: ARIMA Estimators for Python,” https://github.com/alkaline-ml/pmdarima, Accessed January 2020.
47.
Bergmeir
,
C.
, and
Benítez
,
J. M.
,
2012
, “
On the Use of Cross-Validation for Time Series Predictor Evaluation
,”
Inf. Sci.
,
191
, pp.
192
213
. 10.1016/j.ins.2011.12.028
48.
Tashman
,
L. J.
,
2000
, “
Out-of-sample Tests of Forecasting Accuracy: An Analysis and Review
,”
Int. J. Forecasting
,
16
(
4
), pp.
437
450
. 10.1016/S0169-2070(00)00065-0
49.
Wanasinghe
,
T. R.
,
Wroblewski
,
L.
,
Petersen
,
B.
,
Gosine
,
R. G.
,
James
,
L. A.
,
De Silva
,
O.
,
Mann
,
G. K.
, and
Warrian
,
P. J.
,
2020
, “
Digital Twin for the Oil and Gas Industry: Overview, Research Trends, Opportunities, and Challenges
,”
IEEE Access
,
8
, pp.
104175
104197
. doi: 10.1109/ACCESS.2020.2998723
50.
Hu
,
W.
,
Jiang
,
Z.
, and
Wang
,
Y.
,
2018
, “
Reliability Analysis of Offshore Wind Turbines Using Copula
,”
Proceedings of the ASME 2018 Power and Energy Conference
,
Lake Buena Vista, FL
,
June 24–28
, pp.
1
8
.
51.
Zoph
,
B.
, and
Le
,
Q. V.
,
2016
, “
Neural Architecture Search With Reinforcement Learning
,” arXiv preprint arXiv:1611.01578.
You do not currently have access to this content.