Abstract
In this paper, we report on our investigation of haptics-enabled mid-air interactions for sketching three-dimensional (3D) curve-soups—collections of three-dimensional multi-planar curves. We study pen-based mid-air interactions for free-form curve input from the perspective of manual labor, controllability, and kinesthetic feedback. We specifically study the role of kinesthetic feedback for two aspects of mid-air sketching, namely, drawing curves on planar surfaces and spatial rotation of 3D curve-soups. For this, we implemented a simple haptics-enabled workflow for users to draw and compose collections of planar curves on a force-enabled virtual canvas. The qualitative and quantitative analyses of our study-tasks show that there is a rich interaction design space of kinesthetic feedback methods for mid-air sketching beyond physically currently prevalent models.
Issue Section:
Research Papers
References
1.
Stappers
, P.
, and Hennessey
, J.
, 1999
, “Toward Electronic Napkins and Beermats: Computer Support for Visual Ideation Skills,” Visual Representations and Interpretations
, R.
Paton
, and I.
Neilson
, eds., Springer
, London
, pp. 220
–225
.2.
Jonson
, B.
, 2005
, “Design Ideation: The Conceptual Sketch in the Digital Age
,” Des. Stud.
26
(6
), pp. 613
–624
. 10.1016/j.destud.2005.03.0013.
Zeleznik
, R. C.
, Forsberg
, A. S.
, and Strauss
, P. S.
, 1997
, “Two Pointer Input for 3D Interaction
,” Proceedings of the 1997 Symposium on Interactive 3D Graphics, I3D ’97, ACM
, Providence, RI
, April 27–30
, pp. 115
–120
. https://dx.doi.org/10.1016/10.1145/253284.2533164.
Zeleznik
, R. C.
, and Forsberg
, A. S.
, 1999
, “Unicam—2D Gestural Camera Controls for 3D Environments
,” Proceedings of the 1999 symposium on Interactive 3D graphics, I3D '99
, Atlanta, GA
, April 26–29
.5.
Smith
, G.
, Salzman
, T.
, and Stuerzlinger
, W.
, 2001
, “3D Scene Manipulation With 2D Devices and Constraints
,” Proceedings of Graphics Interface 2001, GI ’01, Canadian Information Processing Society
, Ottawa, Ontario, Canada
, June 7–9
, pp. 135
–142
. https://dx.doi.org/10.1016/10.1007/978-1-4471-0337-0_46.
Mohanty
, R. R.
, Bohari
, U. H.
, Ragan
, E.
, and Vinayak
, 2018
, “Kinesthetically Augmented Mid-Air Sketching of Multi-planar 3D Curve-Soups
,” ASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
, Quebec City, Canada
, Aug. 26–29
, American Society of Mechanical Engineers
, New York
. https://dx.doi.org/10.1016/10.1115/detc2018-861417.
Bae
, S.-H.
, Balakrishnan
, R.
, and Singh
, K.
, 2008
, “ILoveSketch: As-Natural-As-Possible Sketching System for Creating 3D Curve Models
,” Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology, UIST ’08
, Monterey, CA
, Oct. 19–22
, ACM
, pp. 151
–160
. https://dx.doi.org/10.1016/10.1145/1449715.14497408.
Bae
, S.-H.
, Balakrishnan
, R.
, and Singh
, K.
, 2009
, “EverybodyLovesSketch: 3D Sketching for a Broader Audience
,” Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology, UIST ’09
, Victoria, BC, Canada
, Oct. 4–7
, ACM, pp. 59
–68
. https://dx.doi.org/10.1016/10.1145/1622176.16221899.
Dorsey
, J.
, Xu
, S.
, Smedresman
, G.
, Rushmeier
, H.
, and McMillan
, L.
, 2007
, “The Mental Canvas: A Tool for Conceptual Architectural Design and Analysis
,” Proceedings of the 15th Pacific Conference on Computer Graphics and Applications, PG ’07, IEEE Computer Society
, Maui, HI
, Oct. 29–Nov. 2
, pp. 201
–210
. https://dx.doi.org/10.1016/10.1109/pg.2007.6410.
Kim
, H.-J.
, Kim
, C. M.
, and Nam
, T.-J.
, 2018
, “Sketchstudio: Experience Prototyping With 2.5-Dimensional Animated Design Scenarios
,” Proceedings of the 2018 Designing Interactive Systems Conference, DIS ’18, ACM
, Honkong, China
, June 9–13
, ACM
, pp. 831
–843
. https://dx.doi.org/10.1016/10.1145/3196709.319673611.
Tsang
, S.
, Balakrishnan
, R.
, Singh
, K.
, and Ranjan
, A.
, 2004
, “A Suggestive Interface for Image Guided 3D Sketching
,” Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’04, ACM
, Vienna, Austria
, April 24–29
, ACM
, pp. 591
–598
. https://dx.doi.org/10.1016/10.1145/985692.98576712.
Xu
, B.
, Chang
, W.
, Sheffer
, A.
, Bousseau
, A.
, McCrae
, J.
, and Singh
, K.
, 2014
, “True2Form: 3D Curve Networks From 2D Sketches Via Selective Regularization
,” ACM Trans. Graph.
, 33
(4
), pp. 131:1
–131:13
. 10.1145/2601097.260112813.
Deering
, M. F.
, 1995
, “Holosketch: A Virtual Reality Sketching/Animation Tool
,” ACM Trans. Comput. Hum. Interact.
, 2
(3
), pp. 220
–238
. 10.1145/210079.21008714.
Keefe
, D. F.
, Feliz
, D. A.
, Moscovich
, T.
, Laidlaw
, D. H.
, and LaViola
, J. J.
, Jr., 2001
, “Cavepainting: A Fully Immersive 3D Artistic Medium and Interactive Experience
,” Proceedings of the ACM Symposium on Interactive 3D Graphics
, pp. 85
–93
. 10.1145/364338.36437015.
LaViola
, J. J.
, and Keefe
, D. F.
, 2011
, “3D Spatial Interaction: Applications for Art, Design, and Science
,” ACM SIGGRAPH 2011 Courses
, Vancouver, British Columbia, Canada
, Aug. 7–11
, pp. 1:1
–1:75
. https://dx.doi.org/10.1016/10.1145/2037636.203763716.
Sachs
, E.
, Roberts
, A.
, and Stoops
, D.
, 1991
, “3-Draw: A Tool for Designing 3D Shapes
,” IEEE Comput. Graph. Appl.
, 11
(6
), pp. 18
–26
. 10.1109/38.10338917.
Xin
, M.
, Sharlin
, E.
, and Sousa
, M. C.
, 2008
, “Napkin Sketch: Handheld Mixed Reality 3D Sketching
,” Proceedings of the 2008 ACM Symposium on Virtual Reality Software and Technology, VRST ’08, ACM
, Bordeaux, France
, Oct. 27–29
, ACM
, pp. 223
–226
. https://dx.doi.org/10.1016/10.1145/1450579.145062718.
Kwan
, K. C.
, and Fu
, H.
, 2019
, “Mobi3dsketch: 3D Sketching in Mobile AR
,” Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI ’19, ACM
, Glasgow, Scotland, UK
, May 4–9
, ACM
, pp. 176:1
–176:11
. https://dx.doi.org/10.1016/10.1145/3290605.330040619.
Jackson
, B.
, and Keefe
, D. F.
, 2016
, “Lift-off: Using Reference Imagery and Freehand Sketching to Create 3D Models in VR
,” IEEE Trans. Visual. Comput. Graphics
, 22
(4
), pp. 1442
–1451
. 10.1109/TVCG.2016.251809920.
De Araújo
, B. R.
, Casiez
, G.
, and Jorge
, J. A.
, 2012
, “Mockup Builder: Direct 3D Modeling on and Above the Surface in a Continuous Interaction Space
,” Proceedings of Graphics Interface 2012, GI ’12
, Toronto, Ontario, Canada
, May 28–30
, Canadian InformationProcessing Society
, pp. 173
–180
. https://dx.doi.org/10.1016/10.1016/j.cag.2012.12.00521.
Arora
, R.
, Habib Kazi
, R.
, Grossman
, T.
, Fitzmaurice
, G.
, and Singh
, K.
, 2018
, “SymbiosisSketch: Combining 2D & 3D Sketching for Designing Detailed 3D Objects in Situ
,” Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, ACM
, Montreal, QC, Canada
, April 21–26
, ACM
, pp. 185:1
–185:15
. https://dx.doi.org/10.1016/10.1145/3173574.317375922.
Kim
, Y.
, An
, S.-G.
, Lee
, J. H.
, and Bae
, S.-H.
, 2018
, “Agile 3D Sketching With Air Scaffolding
,” Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, ACM
, Montreal, QC
, April 21–26
, ACM
, pp. 238:1
–238:12
. https://dx.doi.org/10.1016/10.1145/3173574.317375923.
Kim
, Y.
, and Bae
, S.-H.
, 2016
, “SketchingWithHands: 3D Sketching Handheld Products With First-Person Hand Posture
,” Proceedings of the 29th Annual Symposium on User Interface Software and Technology, UIST ’16, ACM
, Tokyo, Japan
, Oct. 16–19
, ACM
, pp. 797
–808
. https://dx.doi.org/10.1016/10.1145/2984511.298456724.
An
, S.-G.
, Kim
, Y.
, Lee
, J. H.
, and Bae
, S.-H.
, 2017
, “Collaborative Experience Prototyping of Automotive Interior in VR With 3D Sketching and Haptic Helpers
,” Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI ’17, ACM
, Oldenburg, Germany
, Sept. 24–27
, ACM
, pp. 183
–192
. https://dx.doi.org/10.1016/10.1145/3122986.312300225.
Arora
, R.
, Kazi
, R. H.
, Anderson
, F.
, Grossman
, T.
, Singh
, K.
, and Fitzmaurice
, G.
, 2017
, “Experimental Evaluation of Sketching on Surfaces in VR
,” Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17, ACM
, Denver, CO
, May 6–11
, ACM
, pp. 5643
–5654
. https://dx.doi.org/10.1016/10.1145/3025453.302547426.
Israel
, J.
, Wiese
, E.
, Mateescu
, M.
, Zöllner
, C.
, and Stark
, R.
, 2009
, “Investigating Three-Dimensional Sketching for Early Conceptual Design–Results From Expert Discussions and User Studies
,” Computers & Graphics
, 33
(4
), pp. 462
–473
. 10.1016/j.cag.2009.05.00527.
Balakrishnan
, R.
, Fitzmaurice
, G.
, Kurtenbach
, G.
, and Buxton
, W.
, 1999
, “Digital Tape Drawing
,” Proceedings of the 12th Annual ACM Symposium on User Interface Software and Technology, UIST ’99, ACM
, Asheville, NC
, Nov. 7–10
, ACM
, pp. 161
–169
. https://dx.doi.org/10.1016/10.1145/320719.32259828.
Keefe
, D. F.
, Zeleznik
, R. C.
, and Laidlaw
, D. H.
, 2007
, “Drawing on Air: Input Techniques for Controlled 3D Line Illustration
,” IEEE Trans. Visual. Comput. Graphics
, 13
(5
), pp. 1067
–1081
. 10.1109/TVCG.2007.106029.
Schkolne
, S.
, Pruett
, M.
, and Schröder
, P.
, 2001
, “Surface Drawing: Creating Organic 3D Shapes With the Hand and Tangible Tools
,” Proceedings of the ACM Conference on Human Factors in Computing Systems, CHI '01, ACM
, Seattle, WA
, Mar. 31–April 5
, pp. 261
–268
. https://dx.doi.org/10.1016/10.1145/365024.36511430.
Balakrishnan
, R.
, Fitzmaurice
, G.
, Kurtenbach
, G.
, and Singh
, K.
, 1999
, “Exploring Interactive Curve and Surface Manipulation Using a Bend and Twist Sensitive Input Strip
,” Proceedings of the 1999 Symposium on Interactive 3D Graphics, I3D ’99, ACM
, Atlanta, GA
, April 26–29
, ACM
, pp. 111
–118
. https://dx.doi.org/10.1016/10.1145/300523.30053631.
Leal
, A.
, Bowman
, D.
, Schaefer
, L.
, Quek
, F.
, and Stiles
, C. K.
, 2011
, “3D Sketching Using Interactive Fabric for Tangible and Bimanual Input
,” Proceedings of Graphics Interface 2011, GI ’11, Canadian Human-Computer Communications Society
, St. John's, Newfoundland, Canada
, May 25–27
, Canadian Human-Computer Communications Society
, pp. 49
–56
.32.
Jacob
, T.
, Bailly
, G.
, Lecolinet
, E.
, Foulon
, R.
, and Corteel
, E.
, 2014
, “A Design Space for Three-Dimensional Curve Edition
,” Proceedings of the 26th Conference on L’Interaction Homme-Machine, IHM ’14, ACM
, Villeneuve d'Ascq, France
, Oct. 28–31
, ACM
, pp. 105
–112
. https://dx.doi.org/10.1016/10.1145/2670444.267045433.
Foskey
, M.
, Otaduy
, M. A.
, and Lin
, M. C.
, 2002
, “Artnova: Touch-Enabled 3D Model Design
,” Virtual Reality, 2002. Proceedings IEEE
, Orlando, FL
, Mar. 24–28
, IEEE
, pp. 119
–126
. https://dx.doi.org/10.1016/10.1109/vr.2002.99651434.
Raymaekers
, C.
, Vansichem
, G.
, and Van Reeth
, F.
, 2002
, “Improving Sketching by Utilizing Haptic Feedback
,” Proceedings of the Eighteenth National Conference on Artificial Intelligence, AAAI '02
, Edmonton, Alberta
, July 28–Aug. 1
.35.
Fünfzig
, C.
, Thomin
, P.
, and Albrecht
, G.
, 2010
, “Haptic Manipulation of Rational Parametric Planar Cubics Using Shape Constraints
,” Proceedings of the ACM Symposium on Applied Computing
, Sierre, Switzerland
, Mar. 22–26
, pp. 1253
–1257
. https://dx.doi.org/10.1016/10.1145/1774088.177435436.
Massie
, T.
, 1998
, “A Tangible Goal for 3D Modeling
,” IEEE Comput. Graph. Appl.
, 18
(3
), pp. 62
–65
. 10.1109/38.67497337.
Oakley
, I.
, McGee
, M. R.
, Brewster
, S.
, and Gray
, P.
, 2000
, “Putting the Feel in ‘look and feel’
,” Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’00, ACM
, The Hague, The Netherlands
, Apr. 1–6
, ACM
, pp. 415
–422
. https://dx.doi.org/10.1016/10.1145/332040.33246738.
Miller
, T.
, and Zelevnik
, R.
, 1998
, “An insidious haptic invasion: adding force to the X desktop
,” Proceedings of the 11th Annual ACM Symposium on User Interface Software and Technology, UIST '98, ACM
, San Francisco, CA
, Nov. 1–4
, ACM
, pp. 59
–66
. https://dx.doi.org/10.1016/10.1145/288392.28857339.
Barrera Machuca
, M.
, Asente
, P.
, Stuerzlinger
, W.
, Lu
, J.
, and Kim
, B.
, 2018
, “Multiplanes: Assisted Freehand VR Sketching
,” Proceedings of the Symposium on Spatial User Interaction, SUI '18, ACM
, Berlin, Germany
, Oct. 13–14
.40.
Veit
, M.
, Capobianco
, A.
, and Bechmann
, D.
, 2009
, “Influence of Degrees of Freedom’s Manipulation on Performances During Orientation Tasks in Virtual Reality Environments
,” Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology, VRST ’09, ACM
, Kyoto, Japan
, Nov. 18–20
, ACM
, pp. 51
–58
. https://dx.doi.org/10.1016/10.1145/1643928.164394241.
Veit
, M.
, Capobianco
, A.
, and Bechmann
, D.
, 2010
, “Dynamic Decomposition and Integration of Degrees of Freedom for 3-D Positioning
,” Proceedings of the 17th ACM Symposium on Virtual Reality Software and Technology, VRST ’10, ACM
, Hongkong, China
, Nov. 22–24
, pp. 131
–134
. https://dx.doi.org/10.1016/10.1145/1889863.188989142.
Jacob
, R. J. K.
, Sibert
, L. E.
, McFarlane
, D. C.
, and Mullen
, M. P.
, Jr., 1994
, “Integrality and Separability of Input Devices
,” ACM Trans. Comput.-Hum. Interact.
, pp. 1
(1
), pp. 3
–26
. 10.1145/174630.17463143.
Hand
, C.
, 1997
, “A Survey of 3D Interaction Techniques
,” Comput. Graphics Forum
, pp. 16
(5
), pp. 269
–281
. 10.1111/1467-8659.0019444.
Bowman
, D. A.
, Kruijff
, E.
, LaViola
, J. J.
, and Poupyrev
, I.
, 2004
, 3D User Interfaces: Theory and Practice
, Addison Wesley Longman Publishing Co., Inc.
, Redwood City, CA
.45.
Shoemake
, K.
, 1992
, “Arcball: A User Interface for Specifying Three-Dimensional Orientation Using a Mouse
,” Proceedings of the Conference on Graphics Interface ’92
, Vancouver, British Columbia, Canada
, May 11–15 May
.46.
Knoedel
, S.
, and Hachet
, M.
, 2011
, “Multi-touch RST in 2D and 3D Spaces: Studying the Impact of Directness on User Performance
,” Proceedings of the 2011 IEEE Symposium on 3D User Interfaces, 3DUI ’11, IEEE Computer Society
, Singapore
, Mar. 19–20
, pp. 75
–78
. https://dx.doi.org/10.1016/10.1109/3dui.2011.575922047.
Katzakis
, N.
, Seki
, K.
, Kiyokawa
, K.
, and Takemura
, H.
, 2013
, “Mesh-Grab and Arcball-3D: Ray-Based 6-dof Object Manipulation
,” Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction, APCHI ’13, ACM
, Bangalore, India
, Sept. 24–27
, ACM
, pp. 129
–136
. https://dx.doi.org/10.1016/10.1145/2525194.252519848.
Holt
, C. C.
, 2004
, “Forecasting Seasonals and Trends by Exponentially Weighted Moving Averages
,” Int. J. Forecasting
, 20
(1
), pp. 5
–10
. 10.1016/j.ijforecast.2003.09.01549.
Thiel
, Y.
, Singh
, K.
, and Balakrishnan
, R.
, 2011
, “Elasticurves: Exploiting Stroke Dynamics and Inertia for the Real-Time Neatening of Sketched 2D Curves
,” Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST ’11, ACM
, Santa Barbara, CA
, Oct. 16–19
, ACM
, pp. 383
–392
. https://dx.doi.org/10.1016/10.1145/2047196.204724650.
Hart
, S. G.
, and Staveland
, L. E.
, 1988
, “Development of NASA-TLX (task Load Index): Results of Empirical and Theoretical Research
,” Adv. Psychol.
, 52
, pp. 139
–183
. 10.1016/s0166-4115(08)62386-951.
Baudel
, T.
, 1994
, “A Mark-Based Interaction Paradigm for Free-Hand Drawing
,” Proceedings of the 7th Annual ACM Symposium on User Interface Software and Technology, UIST ’94, ACM
, Marina del Rey, California, USA
, Nov. 2–4
, ACM
, pp. 185
–192
. https://dx.doi.org/10.1016/10.1145/192426.19249652.
Orbay
, G.
, and Kara
, L. B.
, 2012
, “Sketch-based Surface Design Using Malleable Curve Networks
,” Comput. Graphics
, 36
(8
), pp. 916
–929
. 10.1016/j.cag.2012.08.008Copyright © 2019 by ASME
You do not currently have access to this content.
