Abstract

Engineers and computational scientists often study the behavior of their simulations by repeated solutions with variations in their parameters, which can be, for instance, boundary values or initial conditions. Through such simulation ensembles, uncertainty in a solution is studied as a function of various input parameters. Solutions of numerical simulations are often temporal functions, spatial maps, or spatio-temporal outputs. The usual way to deal with such complex outputs is to limit the analysis to several probes in the temporal/spatial domain. This leads to smaller and more tractable ensembles of functional outputs (curves) with their associated input parameters: augmented ensembles of curves. This article describes a system for the interactive exploration and analysis of such augmented ensembles. Descriptive statistics on the functional outputs are performed by principal component analysis (PCA) projection, kernel density estimation, and the computation of high density regions. This makes possible the calculation of functional quantiles and outliers. Brushing and linking the elements of the system allows in-depth analysis of the ensemble. The system allows for functional descriptive statistics, cluster detection, and finally, for the realization of a visual sensitivity analysis via cobweb plots. We present two synthetic examples and then validate our approach in an industrial use-case concerning a marine current study using a hydraulic solver.

References

1.
E.
De Rocquigny
,
N.
Devictor
, and
S.
Tarantola
, eds.,
2008
,
Uncertainty in Industrial Practice: A Guide to Quantitative Uncertainty Management
,
Wiley & Sons
, Chichester, UK.
2.
Smith
,
R. C.
,
2014
,
Uncertainty Quantification
,
SIAM
, Philadelphia, PA.
3.
R.
Ghanem
,
D.
Higdon
, and
H.
Owhadi
, eds.,
2017
,
Handbook of Uncertainty Quantification
,
Springer
, Cham, Switzerland.
4.
A.
Saltelli
,
K.
Chan
, and
E. M.
Scott
, eds.,
2000
,
Sensitivity Analysis
,
Wiley
, Chichester, UK.
5.
Iooss
,
B.
, and
Lemaître
,
P.
,
2015
, “
A Review on Global Sensitivity Analysis Methods
,”
Uncertainty Management in Simulation-Optimization of Complex Systems
,
C.
Meloni
, and
G.
Dellino
, eds.,
Springer
, New York, pp.
101
122
.
6.
Love
,
A. L.
,
Pang
,
A.
, and
Kao
,
D. L.
,
2005
, “
Visualizing Spatial Multivalue Data
,”
IEEE Comput. Graph. Appl.
,
25
(
3
), pp.
69
79
.10.1109/MCG.2005.71
7.
Sanyal
,
J.
,
Zhang
,
S.
,
Dyer
,
J.
,
Mercer
,
A.
,
Amburn
,
P.
, and
Moorhead
,
R.
,
2010
, “
Noodles: A Tool for Visualization of Numerical Weather Model Ensemble Uncertainty
,”
IEEE Trans. Visualization Comput. Graph.
,
16
(
6
), pp.
1421
1430
.10.1109/TVCG.2010.181
8.
Potter
,
K.
,
Wilson
,
A.
,
Bremer
,
P. T.
,
Williams
,
D.
,
Doutriaux
,
C.
,
Pascucci
,
V.
, and
Johnson
,
C. R.
,
2009
, “
Ensemble-Vis: A Framework for the Statistical Visualization of Ensemble Data
,”
IEEE International Conference on Data Mining Workshops
(
ICDMW'09
), Miami, FL, Dec. 6,
pp.
233
240
.10.1109/ICDMW.2009.55
9.
Lodha
,
S. K.
,
Wilson
,
C. M.
, and
Sheehan
,
R. E.
,
1996
, “
LISTEN: Sounding Uncertainty Visualization
,”
Proceedings of the Seventh Conference on Visualization'96
,
IEEE Computer Society Press
, San Francisco, CA, Oct. 27–Nov. 1.10.1109/VISUAL.1996.568105
10.
Pang
,
A. T.
,
Wittenbrink
,
C. M.
, and
Lodha
,
S. K.
,
1997
, “
Approaches to Uncertainty Visualization
,”
Visual Comput.
,
13
(
8
), pp.
370
390
.10.1007/s003710050111
11.
Johnson
,
C. R.
, and
Sanderson
,
A. R.
,
2003
, “
A Next Step: Visualizing Errors and Uncertainty
,”
IEEE Comput. Graph. Appl.
,
23
(
5
), pp.
6
10
.10.1109/MCG.2003.1231171
12.
Hyndman
,
R. J.
, and
Shang
,
H. L.
,
2010
, “
Rainbow Plots, Bagplots, and Boxplots for Functional Data
,”
J. Comput. Graphical Stat.
,
19
(
1
), pp.
29
45
.10.1198/jcgs.2009.08158
13.
Popelin
,
A.-L.
, and
Iooss
,
B.
,
2013
, “
Visualization Tools for Uncertainty and Sensitivity Analyses on Thermal-Hydraulic Transients
,”
Proceedings of Joint International Conference on Supercomputing in Nuclear Applications and Monte Carlo 2013 (SNA + MC 2013)
,
Paris, France
, Oct. 27–31, p.
03403
.
14.
Nanty
,
S.
,
Helbert
,
C.
,
Marrel
,
A.
,
Pérot
,
N.
, and
Prieur
,
C.
,
2016
, “
Uncertainty Quantification for Functional Dependent Random Variables
,”
Comput. Stat.
, 32(2), pp.
559
583
.https://link.springer.com/article/10.1007/s00180-016-0676-0
15.
Whitaker
,
R. T.
,
Mirzargar
,
M.
, and
Kirby
,
R. M.
,
2013
, “
Contour Boxplots: A Method for Characterizing Uncertainty in Feature Sets From Simulation Ensembles
,”
IEEE Trans. Visualization Comput. Graph.
,
19
(
12
), pp.
2713
2722
.10.1109/TVCG.2013.143
16.
Ferstl
,
F.
,
Kanzler
,
M.
,
Rautenhaus
,
M.
, and
Westermann
,
R.
,
2016
, “
Visual Analysis of Spatial Variability and Global Correlations in Ensembles of Iso‐Contours
,”
Comput. Graph. Forum
,
35
(
3
), pp.
221
230
.10.1111/cgf.12898
17.
Ferstl
,
F.
,
Bürger
,
K.
, and
Westermann
,
R.
,
2016
, “
Streamline Variability Plots for Characterizing the Uncertainty in Vector Field Ensembles
,”
IEEE Trans. Visualization Comput. Graph.
,
22
(
1
), pp.
767
776
.10.1109/TVCG.2015.2467204
18.
Sobol
,
I. M.
,
1993
, “
Sensitivity Estimates for Nonlinear Mathematical Models
,”
Math. Modell. Comput. Exp.
,
1
(
4
), pp.
407
14
. https://pdfs.semanticscholar.org/d339/b9cc42d6a7286d96814e6713fd13cdde87e7.pdf
19.
Marrel
,
A.
,
Iooss
,
B.
,
Jullien
,
M.
,
Laurent
,
B.
, and
Volkova
,
E.
,
2011
, “
Global Sensitivity Analysis for Models With Spatially Dependent Output
,”
Environmetrics
,
22
(
3
), pp.
383
397
.10.1002/env.1071
20.
Marrel
,
A.
,
Saint-Geours
,
N.
, and
De Lozzo
,
M.
,
2017
, “
Sensitivity Analysis of Spatial and/or Temporal Phenomena
,”
Handbook of Uncertainty Quantification
,
R.
Ghanem
,
D.
Higdon
, and
H.
Owhadi
, eds.,
Springer
, Cham, Switzerland.
21.
Terraz
,
T.
,
Ribés
,
A.
,
Fournier
,
Y.
,
Iooss
,
B.
, and
Raffin
,
B.
,
2017
, “
Melissa: Large Scale in Transit Sensitivity Analysis Avoiding Intermediate Files
,” The International Conference for High Performance Computing, Networking, Storage and Analysis (
Supercomputing
), Denver, CO.https://www.researchgate.net/publication/319529077_Melissa_Large_Scale_In_Transit_Sensitivity_Analysis_Avoiding_Intermediate_Files
22.
Lampe
,
O. D.
, and
Hauser
,
H.
,
2011
, “
Curve Density Estimates
,”
Comput. Graph. Forum
,
30
(
3
), pp.
633
642
.10.1111/j.1467-8659.2011.01912.x
23.
Hochheiser
,
H.
, and
Shneiderman
,
B.
,
2004
, “
Dynamic Query Tools for Time Series Data Sets: Timebox Widgets for Interactive Exploration
,”
Inf. Visualization
,
3
(
1
), pp.
1
8
.10.1057/palgrave.ivs.9500061
24.
Konyha
,
Z.
,
Matkovic
,
K.
,
Gracanin
,
D.
,
Jelovic
,
M.
, and
Hauser
,
H.
,
2006
, “
Interactive Visual Analysis of Families of Function Graphs
,”
IEEE Trans. Visualization Comput. Graph.
,
12
(
6
), pp.
1373
1385
.10.1109/TVCG.2006.99
25.
Muigg
,
P.
,
Kehrer
,
J.
,
Oeltze
,
S.
,
Piringer
,
H.
,
Doleisch
,
H.
,
Preim
,
B.
, and
Hauser
,
H.
,
2008
, “
A Four‐Level Focus+ Context Approach to Interactive Visual Analysis of Temporal Features in Large Scientific Data
,”
Comput. Graph. Forum
,
27
(
3
), pp.
775
782
.10.1111/j.1467-8659.2008.01207.x
26.
McLachlan
,
P.
,
Munzner
,
T.
,
Koutsofios
,
E.
, and
North
,
S.
,
2008
, “
LiveRAC: Interactive Visual Exploration of System Management Time-Series Data
,”
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
, Florence, Italy, Apr. 5–10, pp.
1483
1492
.https://www.cs.ubc.ca/labs/imager/tr/2008/liverac/liverac.pdf
27.
Sun
,
Y.
, and
Genton
,
M. G.
,
2011
, “
Functional Boxplots
,”
J. Comput. Graphical Stat.
,
20
(
2
), pp.
316
334
.10.1198/jcgs.2011.09224
28.
López-Pintado
,
S.
, and
Romo
,
J.
,
2009
, “
On the Concept of Depth for Functional Data
,”
J. Am. Stat. Assoc.
,
104
(
486
), pp.
718
734
.10.1198/jasa.2009.0108
29.
Sacha
,
D.
,
Zhang
,
L.
,
Sedlmair
,
M.
,
Lee
,
J. A.
,
Peltonen
,
J.
,
Weiskopf
,
D.
,
North
,
S. C.
, and
Keim
,
D. A.
,
2017
, “
Visual Interaction With Dimensionality Reduction: A Structured Literature Analysis
,”
IEEE Trans. Visualization Comput. Graphics
,
23
(
1
), pp.
241
250
.10.1109/TVCG.2016.2598495
30.
Auder
,
B.
,
de Crecy
,
A.
,
Iooss
,
B.
, and
Marquès
,
M.
,
2012
, “
Screening and Metamodeling of Computer Experiments With Functional Outputs. Application to Thermal-Hydraulic Computations
,”
Reliab. Eng. Syst. Saf.
,
107
, pp.
122
131
.10.1016/j.ress.2011.10.017
31.
Lee
,
J. A.
, and
Verleysen
,
M.
,
2007
,
Nonlinear Dimensionality Reduction
,
Springer
, Berlin.
32.
Becker
,
R. A.
, and
Cleveland
,
W. S.
,
1987
, “
Brushing Scatterplots
,”
Technometrics
,
29
(
2
), pp.
127
142
.10.1080/00401706.1987.10488204
33.
Keim
,
D. A.
,
2002
, “
Information Visualization and Visual Data Mining
,”
IEEE Trans. Visualization Comput. Graphics
,
8
(
1
), pp.
1
8
.10.1109/2945.981847
34.
Matkovic
,
K.
,
Gracanin
,
D.
,
Klarin
,
B.
, and
Hauser
,
H.
,
2009
, “
Interactive Visual Analysis of Complex Scientific Data as Families of Data Surfaces
,”
IEEE Trans. Visualization Comput. Graphics
,
15
(
6
), pp.
1351
1358
.10.1109/TVCG.2009.155
35.
Piringer
,
H.
,
Pajer
,
S.
,
Berger
,
W.
, and
Teichmann
,
H.
,
2012
, “
Comparative Visual Analysis of 2D Function Ensembles
,”
Comput. Graphics Forum
,
31
(
3 pt. 3
), pp.
1195
1204
.10.1111/j.1467-8659.2012.03112.x
36.
Demir
,
I.
,
Dick
,
C.
, and
Westermann
,
R.
,
2014
, “
Multi-Charts for Comparative 3D Ensemble Visualization
,”
IEEE Trans. Visualization Comput. Graphics
,
20
(
12
), pp.
2694
2703
.10.1109/TVCG.2014.2346448
37.
Piringer
,
H.
,
Berger
,
W.
, and
Krasser
,
J.
,
2010
, “
Hypermoval: Interactive Visual Validation of Regression Models for Real-Time Simulation
,”
Comput. Graphics Forum
,
29
(
3
), pp.
983
992
.10.1111/j.1467-8659.2009.01684.x
38.
Theron
,
R.
, and
De Paz
,
J. F.
,
2006
, “
Visual Sensitivity Analysis for Artificial Neural Networks
,”
International Conference on Intelligent Data Engineering and Automated Learning
,
Springer
,
Berlin
, pp.
191
198
.
39.
Torsney-Weir
,
T.
,
Saad
,
A.
,
Moller
,
T.
,
Hege
,
H.
,
Weber
,
B.
,
Verbavatz
,
J.
, and
Bergner
,
S.
,
2011
, “
Tuner: Principled Parameter Finding for Image Segmentation Algorithms Using Visual Response Surface Exploration
,”
IEEE Trans. Visualization Comput. Graphics
,
17
(
12
), pp.
1892
1901
.10.1109/TVCG.2011.248
40.
Sedlmair
,
M.
,
Heinzl
,
C.
,
Bruckner
,
S.
,
Piringer
,
H.
, and
Möller
,
T.
,
2014
, “
Visual Parameter Space Analysis: A Conceptual Framework
,”
IEEE Trans. Visualization Comput. Graphics
,
20
(
12
), pp.
2161
2170
.10.1109/TVCG.2014.2346321
41.
Elmqvist
,
N.
,
Dragicevic
,
P.
, and
Fekete
,
J. D.
,
2008
, “
Rolling the Dice: Multidimensional Visual Exploration Using Scatterplot Matrix Navigation
,”
IEEE Trans. Visualization Comput. Graphics
,
14
(
6
), pp.
1539
1148
.10.1109/TVCG.2008.153
42.
Silverman
,
B. W.
,
1981
, “
Using Kernel Density Estimates to Investigate Multimodality
,”
J. R. Stat. Soc., Ser. B
,
1
, pp.
97
99
.10.1111/j.2517-6161.1981.tb01155.x
43.
Campbell
,
K.
,
McKay
,
M. D.
, and
Williams
,
B. J.
,
2006
, “
Sensitivity Analysis When Model Outputs Are Functions
,”
Reliab. Eng. Syst. Saf.
,
91
(
10
), pp.
1468
1472
.10.1016/j.ress.2005.11.049
44.
Hervouet
,
J.-M.
,
2000
, “
TELEMAC Modelling System: An Overview
,”
Hydrol. Processes
,
14
(
13
), pp.
2209
2210
.10.1002/1099-1085(200009)14:13<2209::AID-HYP23>3.0.CO;2-6
45.
Baudin
,
M.
,
Dutfoy
,
A.
,
Iooss
,
B.
, and
Popelin
,
A.-L.
,
2017
, “
Open TURNS: An Industrial Software for Uncertainty Quantification in Simulation
,”
Handbook of Uncertainty Quantification
,
R.
Ghanem
,
D.
Higdon
, and
H.
Owhadi
, eds.,
Springer
, Cham, Switzerland.
46.
Ahrens
,
J.
,
Geveci
,
B.
, and
Law
,
C.
,
2005
, “
ParaView: An End-User Tool for Large Data Visualization
,”
Visualization Handbook
, C. D. Hansen and C. R. Johnson, eds., Vol.
717
, Butterworth-Heinemann, Oxford, UK.
47.
Ribés
,
A.
, and
Bruneton
,
A.
,
2014
, “
Visualizing Results in the SALOME Platform for Large Numerical Simulations: An Integration of ParaView
,”
IEEE Fourth Symposium on Large Data Analysis and Visualization
(
LDAV
), Paris, France, Nov. 9–10, pp.
119
120
.10.1109/LDAV.2014.7013218
You do not currently have access to this content.