The Influence of Hand Tracking and Haptic Feedback for Virtual Prototype Evaluation in the Product Design Process

Conceptual design is a critical phase in the product development process to determine the cost and quality of a new product. Getting feedback from potential users during this phase is important to identify the needs and issues that must be addressed in concept validation [1] and to fulfill design goals [2], as design changes are relatively inexpensive and easy to perform at this stage. The cost of making a design change increases dramatically as the product moves through its lifecycle [3].

During the product development process, the ability to fully understand a form and accurately interpret its geometry is crucial [4]. Design engineers use CAD tools to conceptualize, design, visualize, and validate certain parts of the design, but the inherent limitations of displaying 3D geometry on a 2D screen make it difficult to quickly and fully understand the product features [5].

Many researchers agree that the success of a product is highly dependent on the evaluation process that is conducted throughout design, even at the early stages [6]. Both the designer (who can test aesthetic, functional, technical, and performance aspects of the product) and the end user (who can reveal design errors or misinterpretations of the initial requirements) are part of the evaluation process [7]. Depending on the testing purpose, we can choose among various types of prototypes: visual prototypes, which simulate the final aesthetics of the product (e.g., sketches or photorealistic renders); form prototypes, which highlight the shape and size of the product (and are generally made with rapid prototyping or hand-made techniques); functional prototypes, which simulate product performance and can be correctly assessed using a CAD model; fully physical prototypes, which simulate product final design, aesthetics, materials, and functionality [7].

In this regard, physical prototyping is the most common method to obtain user feedback, but traditional physical prototyping techniques may involve large financial and time investments, and costs can increase even higher when modifications and adjustments need to be made to the prototypes. Because of this, understanding the utility of different product representation methods can help reduce time and cost (i.e., sketches, 3D representations, or physical prototypes) [8]. During the design process, fidelity of the prototype may vary depending on the stage of product development [9] and depends on the tests purpose [10], as manufacturing high-fidelity prototypes has a significant impact on product development costs. This factor may significantly affect the user’s confidence and accuracy in their product evaluation [8].

Recent advances in visualization technologies have made virtual prototyping and effective and sustainable tool for design evaluation [11,12], especially during the early stages of development where many design variations must be produced [13]. These technologies enable the creation of high-fidelity geometric representations in a rapid and cost-effective manner for the evaluation of product aesthetics, ergonomics, and usability aspects [7]. However, certain product features can be difficult to evaluate when using virtual prototyping techniques, particularly if physical interaction with the product is required [9]. In design evaluation, a key advantage of physical prototyping over virtual prototyping is the ability to feel and interact with the physical product [14]. Indeed, the sense of touch is critical when evaluating a product because it provides a direct way to obtain information that could not be acquired otherwise [15]. Some new technologies are attempting to fill this gap in virtual environments.

The term “haptics” refers to the artificial forces between virtual objects and the user’s body. They are commonly classified as passive or active. Active haptics are controlled by a computer, and forces can be dynamically manipulated to provide a wide range of feelings for simulated virtual objects [16]. Although some authors have argued that active haptic devices can be a good option for haptic feedback [17], others have pointed out the large financial investment that is required and the need for a much larger workspace to fit some devices to the user’s hand [18]. Researchers have also mentioned the discomfort of wearing haptic devices which may reduce the feeling of presence and immersion in the virtual environment [19] and have a negative impact on the user’s emotional response.

Alternatively, passive haptics, provide a sense of touch in VR by synchronizing physical objects to virtual assets [20], significantly reducing costs without the use of intrusive devices, just the user's hands. This has been shown to increase the sense of presence, improve cognitive mapping of the environment, and improve training performance [21]. Interactions with VR with Passive Haptics setting (VRPH) can enhance the overall user experience along with the possibility of interacting and modifying the virtual setting in real-time (e.g., the environment or the product aesthetics). Although a main drawback of passive haptics has traditionally been the absence of the hands in the virtual environment, low-cost VR devices such as the Oculus Quest have introduced nonintrusive hand-tracking in their virtual experiences, where people can see a virtual representation of their hands in the digital environment that is updated in real-time. Although proper calibration is required to ensure the virtual hands are displayed correctly in the virtual environment, effective experiences can be delivered at a very low cost.

The use of VR has proven to be an effective alternative for product evaluation during the early stages of development. However, its value is predicated on the assumption that our perceptual and emotional responses to a product that is perceived through a VR environment are similar to those elicited by the actual product, which is not necessarily the case [22,23], particularly when evaluating product features that rely heavily on our sense of touch [24–26]. Although some researchers have incorporated mechanisms to simulate the sense of touch in VR to make product evaluations more accurate, the effect of integrating nonintrusive virtual models of the user's hands have not yet been considered.

In this paper, we report the results of an experiment where a group of participants were asked to evaluate three designs of a product (i.e., umbrella stands) in three visual media: a real setting (R), a virtual reality environment (VR), and a Virtual Reality environment with Passive Haptics (VRPH). Participants were also asked to indicate their intended purchasing decision and to rate their level of confidence in their response. We considered the user’s background (trained in design versus not trained in design) to determine the influence on product evaluation and the existence (or lack thereof) of a combined effect between this factor and the medium used to view the product. Finally, we examined how the use of a physical item can affect product perception not only by helping users understand the function of the product under evaluation but also by providing interaction opportunities to generate an experience in which users feel more engaged [27].

According to Steuer [28], VR is “a real or simulated environment in which a perceiver experiences telepresence.” It can also be defined as a “computer-generated digital environment that can be experienced and interacted with as if that environment were real” [16]. Although VR technologies have been around for years, recent advances have made them more affordable. VR has been successfully applied in a variety of areas, such as entertainment [29], healthcare [30], education [31], psychology and marketing [32], architecture [33], and industrial design and product development [14,34].

In design disciplines, market competitiveness and saturation have driven companies to emphasize product attributes that address meaning and emotion [35], and how these can be expressed to positively influence the user's experience, as well as how consumers think, feel, and act [36]. In current markets, the added value enabled by these attributes can significantly influence consumer choice [37]. Although critical, innovation is not the only factor for product success. In fact, some of the most innovative products fail when they reach the market [32,33]. The subtle differences between products in terms of technical characteristics, quality, and price, often make product differentiation a difficult task. In this context, consumer emotions play an important role [38] and can significantly influence the purchasing decision [39].

Various models have been proposed to characterize product emotion [40]. Jordan proposed an approach with four different pleasure categories [41]: physical (pleasures deriving from sensory organs), social (pleasures deriving from relationships with others), psychological (pleasures related to people’s cognitive and emotional reactions) and ideological (pleasures related to people’s values). Alternatively, Desmet [42] applied cognitive appraisal theory to explain the process of product emotion, and Norman [43] explained product emotion through a neurobiological emotion-framework that distinguishes several levels of information processing: visceral, behavioral, and reflective.

Numerous studies have examined the influence of the presentation medium on product evaluation (e.g., 2D images, interactive 3D models, AR, or VR). Söderman offered some initial insights with a study that examined perceptual differences elicited by a car when viewed in a nonimmersive VR environment and as a set of sketches, compared to reality [44]. Reid, MacDonald and Du performed a similar experiment with different 2D visual media [45], observing that the medium used to present the product could influence customer judgments. Their conclusion was confirmed by authors Artacho-Ramírez et al., who conducted an experiment [22] where they used two models of loudspeakers in five different media (photographs, static image, 3D models navigable with a computer mouse, and 3D model navigable with stereoscopic images) and compared user evaluations with the corresponding real products. Their findings showed that the type of representation significantly influences product perception, which was corroborated in Refs. [9,46,47] in their studies.

Despite the fact that researchers have investigated the influence of media into product evaluation [2,48], only recently have they begun to incorporate immersive VR headsets (e.g., Oculus Rift or HTC Vive) in their experiments [26,49]. For example, Galán et al. [25] made one of the first contributions using VRPH. Although users were not able to see their hands, their results demonstrated that the medium used to display the product influences how users perceive it, and that the use of VRPH settings could influence the product assessment positively. They also found that certain product features are more sensitive to perceptual differences, such as those related to the sense of touch. The authors also proposed a division of the semantic differential into Jordan's pleasure categories to obtain more specific results. They concluded that Jordan’s approach was an effective method to evaluate product perception. For this reason, we have adopted this approach in our study.

Researchers have also explored the influence of the user’s background on product evaluation when using different visual media [50]. Design training on product evaluation or the fine arts has also been extensively studied [51]. Solso explored the influence of user expertise on brain activity [52], concluding that trained participants employed more higher-level cognitive abilities than untrained participants. Other authors have suggested that participants trained in design disciplines are more capable of identifying the relationships between the composition of an entire form and its elements [53,54]. We speculate that this factor may also have an impact on the perception of a product and should be considered when using VR, as there could be a combined effect between background and medium that affects product assessment.

The main goal of our study was to determine whether the medium used to present a product influence how it is perceived and evaluated, and whether the integration of haptics can help reduce perceptual differences. To this end, we selected three different designs of a product (i.e., an umbrella stand) and asked a group of participants to assess the products as presented in R, VR, and VRPH using the semantic differential technique.

Some studies suggest that a simple product representation provides enough information to perform a reasonable evaluation [55]. Therefore, the purchasing decision is not necessarily influenced by the change of medium. For this, users were asked to indicate their intended purchasing decision for each product. Our first hypothesis is stated as:

• H1: The purchasing decision is not affected by the change of medium.

   Although similar studies have shown that Jordan’s physiological-pleasure category is the most affected by the change of medium [24], other authors have argued that other categories can be significantly influenced by the medium [23]. Therefore, the following hypothesis is stated:

• H2: The visual medium used to present a product influences user perception regardless of the classification in Jordan’s category.

   Furthermore, some researchers have claimed that haptic feedback can affect the users’ confidence in their evaluations [56,57]. Our third hypothesis is stated as:

• H3: The level of Confidence in the Response is affected by the change of visual medium.

   Another factor that has not been sufficiently studied is the influence of the user's background on product assessment. Some studies have reported that in product evaluation individuals with a design-related background tend to employ more higher-level cognitive abilities than untrained individuals [9]. These differences may influence the evaluation of the product. Our fourth hypothesis is states as:

• H4: User expertise and design background affects product evaluation.

   Finally, in some product evaluation scenarios, it is necessary to use a complementary item to correctly understand how a product performs. Since the use of complementary items (when needed) is not feasible in online channels or even in some physical stores, an important part of the information of the product is lost. We considered this factor as an interesting research goal, as more information can improve product evaluation [8]. Our fifth hypothesis is stated as:

• H5: The use of a complementary item to assess a product influences the evaluation of the product semantic scales.

To test our hypotheses, we structured our study into two analyses A1 and A2. In the first analysis (A1), we compared the user's perception of three umbrella stand designs as presented in three different settings (R, VR, and VRPH) and determined the influence of the user’s previous design expertise on product assessment. In the second analysis (A2), we examined the perceptual differences in product evaluation elicited by the presence or absence of a complementary physical item (i.e., an umbrella) in two different media (R and VR).

A total of 91 users participated in our experiment. All users gave informed consent to participate in our study, which was approved by the CEENB-OMGs section of the Bioethics Committee of the University of Cadiz (Ref. 006/202). To describe our experimental conditions, we next discuss (1) our case study, (2) materials, (3) the semantic scale established for the evaluation of the different products, (4) the description and justification of the sample, and (5) the protocol followed for the development of the case study.

The different umbrella stands selected for our study are shown in Figs. 2.2 and 2.3 (umbrella stand A is white and is located in the center, umbrella stand B is soft yellow and is located on the right, and umbrella stand C is black and is located on the left). We selected this type of product for three main reasons: (1) it is an easy-to-use product, which avoids frustration and prevents it from affecting our results [9]; (2) thanks to the geometry of the design options selected, hand-tracking was not lost, which guaranteed a consistently good experience; and (3) it lends itself to the use of a complementary item (i.e., un umbrella), as this is one of the research goals of our study.

Although similar studies have limited the number of options, we decided to use three prototypes to obtain more general results, since more products meant an increase in the time of the experiment, and people's fatigue could affect the evaluation.

Each stand has an opening to accommodate both long and short umbrellas (at least four) and is equipped with a water tray. All three models had neutral colors to mitigate potential perceptual differences elicited by color [58]. We note that umbrella stand B was displayed in the university hall for 30 days as a part of a student’s product design exhibition. For A2, two different umbrellas were selected (a long and a short umbrella), as shown in Fig. 1. The three umbrella stands were used for both analyses, but the umbrellas were only used in A2 to determine the impact of user-object interaction in the perception of the umbrella stands.

For the three experimental conditions (the viewing media), each product was arranged in the same manner and in identical spaces. Since only one physical model of each umbrella stand was available, two physical rooms were built to present the three scenarios: one empty room and one with the physical products, as shown in Fig. 2. The interior area of the rooms (both physical and virtual) was 5 m². Each physical room was built using seven movable panels positioned contiguously and attached to a 3-m-long wall. The scenarios are described below:

1—Scenario 1 (R): This room consisted in a real environment with the products on the floor. Users were able to view and touch the real objects, but not allowed to change their position. For A2, two physical umbrellas were placed on a small table. The user was allowed to grab and move the umbrellas to interact with the umbrella stands.

2—Scenario 2 (VR). This room consisted in a VR simulated environment on an HMD. Oculus Quest hand-tracking interaction was enabled, so the users were able to see a nonintrusive virtual representation of their hands in real-time while interacting with the products as no external devices were used apart from the HDM (Fig. 3), which allowed a more natural interaction with the objects and the environment. The products were anchored to the virtual floor, so they could not be moved. In the case of the A2, virtual replicas of the two umbrellas were added to the scene and placed on a virtual table. The umbrellas could be grabbed and moved to interact with the umbrella stands.

3—Scenario 3 (VRPH): This scenario consisted in a VR environment displayed on an HMD where the position of the virtual products was synchronized with the position of real products, so haptic feedback was enabled. In this scenario, the physical and virtual products were fixed to the floor. Hand-tracking interaction was enabled, so the users were able to see a nonintrusive virtual representation of their hands in real-time while interacting with the products.

It is important to note that scenarios 1 and 3 were displayed in the same physical room (as shown in Fig. 2.1), but never at the same time since only one participant was allowed to be present during the evaluation of the products in one medium. The use of the physical room alternated between the two scenarios.

The VR environments were experienced using an Oculus Quest 2 HMD upgraded to version 36.0, a stand-alone immersive VR device with a single fast-switch LCD display of 1832 × 1920 pixels per eye and a refresh rate of 72 Hz. The virtual environment was designed using Unity 2020.3.11f1. We used the Oculus Integration asset (version 36.0) and HPTK Posing and Snapping 2.0.0. asset for the hand-tracking interaction (as the Oculus Interaction SDK was not available when the experiment was run). The Passthrough Capability was enabled for the calibration of the virtual objects before starting the experiment, and the hand-tracking interaction capability was enabled to provide the nonintrusive virtual models of the user’s hands. The scene used a real-time light with hard shadows enabled, and materials were built using a Standard Shader. The virtual objects were modeled in SolidWorks 2020, and UV mapping was completed in Blender 2.93.0. Hygiene and disinfection material were also provided to ensure optimal sanitary conditions.

For product evaluation, the participants completed a questionnaire comprised of 12 semantic differential scales. Participants were asked to indicate their intended purchasing decision with a “Yes” or “No” answer and rate their level of confidence in their response. To measure user presence in the VR environments, we used the Slater-Usoh-Steed (SUS) presence questionnaire [59]. This instrument is comprised of six 7-point Likert scale questions where a higher score indicates higher levels of presence.

The semantic differential [60] approach is a common method for product evaluation [55,61] and an effective procedure to obtain consumer opinion and preferences. It is composed of Likert-type scales of 5 or 7 points that typically use bipolar pairs of adjectives to describe the product that is being evaluated. In our study, we used the procedure by Hsu et al. [55] to generate the semantic space for our products, as described next.

Information was collected from four different sources (professional designers, design-related users, average users, and manufacturers) to match a general criterion. The sample included 28 volunteers (20 female and 8 male) with an average age of 33.4 years. The group of professional designers (with at least five years of experience) consisted of 8 people; the design-related users’ group (people with a design background, such as industrial design students, design researchers or professors) was comprised of 12 participants; the average user’s group consisted of eight volunteers. Product adjectives were selected from various sites such as Ikea, Amazon, B-line, Systemtronic, or Mox.

Participants were shown a set of 12 images of various umbrella stands and asked to evaluate each product using an online form. Next, we conducted a keyword analysis. First, the frequency with which each adjective was repeated was counted, and those with the same meaning were grouped together (e.g., “big” and “large”). Antonyms were also grouped to build the most frequent bipolar pairs of adjectives (for terms where no antonym was available, they were added by the authors to complete the definition of the bipolar pair). We classified the bipolar pairs based on the four categories of pleasure defined by Jordan [62], and the two most frequent bipolar pairs were selected for each category, which resulted in a total of eight bipolar pairs of adjectives, as shown in Table 1.

Two different samples were used in our study. To estimate the minimum sample size, we performed an a priori power analysis with G*Power [1]. For A1, a repeated measures ANOVA test was applied with the following input parameters: effect size: .25, α = .05, (1−β) = .80 and 1 group. For A2, a repeated measure ANOVA (within-between interaction) was applied with the following input parameters: effect size: .25, α = .05, (1−β) = .80, two groups. Our results estimated a total sample size of 28 in both cases.

Prior to data processing, an outlier study was conducted to obtain more robust and reliable results. Users with low confidence levels in their responses (a score with a mean value < .60) in at least one medium for each umbrella stand who also appeared as an outlier in the semantic differential scale data set were excluded from the analysis. Our final sample size was 58 users, so a power of .80 was guaranteed.

The mean age of the sample was 20.9 years old. The sample consisted of 39 men and 19 women, where 41 of the volunteers were involved in industrial design disciplines either academically and/or professionally. Approximately half of the participants (50.8%) had no experience with VR before participating in the study, 37.3% of the participants claimed to have limited experience with VR, and only 12% rated their experience with VR as significant. 6.7% of the participants used VR environments frequently and 3.4% very frequently. Finally, 55.9% of the participants stated that they have visual problems. 52.5% reported having myopia, 11.8% astigmatism and 3.4% hyperopia and other problems. 35.5% wore glasses and 18.6% wore contact lenses.

Since the goal of this analysis was to compare the user's perception of a product in the presence or absence of a complementary item, some participants from A1 were also included in A2. A total of 23 participants who completed product evaluations in R and VR media (regardless of order) in the first or second place were randomly selected. This way, we ensured that the evaluation of products through VRPH did not affect our results. In addition, 23 new participants evaluated the products through R and VR using the complementary item.

The 46 participants in A2 guarantee a power of .80. The mean age was 20.63 years (56.5% were men and 43.5% were women); 56.5% were involved in industrial design disciplines either academically and/or professionally; 58.7% of the participants had never used VR devices before the study and 32.6% claimed to have vision problems; 54.4% reported myopia; 17.4% astigmatism and 6.5% hyperopia and other problems; 26.1% wore glasses and 26.1% contact lenses.

The complete procedure followed by the participants in our study is illustrated in Fig. 4, differentiating between those participating in A1 and A2. The experiment was conducted in five consecutive days (22.5 h). Twenty-seven 50-min shifts were established, allowing the participation of up to three users per shift. Participants accessed the area reserved for the experiment in pre-established groups. This area consisted of two physical rooms (described above) and a table with chairs reserved for completing the required documentation and questionnaires. Data for A1 was collected during the first four days, and for A2 on the last day.

Before the experiment, participants were provided with the necessary information about the study and given the opportunity to ask questions. All participants gave informed consent. A member of the research team positioned the virtual objects for scenarios 2 and 3 using the Passthrough Capability of the Oculus Quest 2 ensuring an accurate overlay of the virtual and real objects for scenario 3.

Each participant viewed the products in the different proposed media. Before entering scenarios 2 and 3, a member of the research team confirmed that the user was correctly seeing the virtual models of their hands. After each medium, participants were asked to fill out the evaluation questionnaire before moving to the next experimental condition (Fig. 5.1). The order of the viewing media was randomized for each participant to minimize any potential unwanted effect on the results. Two participants completed the activities in each session, each accompanied by a member of the research team. Inside the room, participants were allowed to interact with the umbrella stands (Figs. 5.2 and 5.3). In the second study, participants were allowed to use two physical umbrellas to interact with the products (Fig. 5.4).

This section has been divided into two sections in terms of the analyses performed (A1 and A2). A schematic diagram is shown in Fig. 6 to facilitate the understanding of the section.

Four data sets were obtained: the (1) Semantic Scales, (2) Confidence in the Response, (3) Purchase Decision, and (4) Presence test scores. The descriptive statistics for Semantic Scales, Purchase Decision, and the Confidence in the Response are shown in Tables 2–4. Boxplots for the Semantic Scales are shown in Fig. 7. For the Semantic Scales, a value closer to −3 represents a closer correspondence with the adjective in italics, and a value closer to 3 indicates a closer correspondence with the adjective in bold. For the Purchase Decision and the Confidence in the Response, a value closer to 0 indicates “not buying the product” and a low Confidence in the Response, whereas a value closer to 1 represents “buying the product” and a high Confidence in the Response. Descriptive statistics for the SUS presence questionnaire were M_VR = 4.701, Mdn_VR = 5, SD_VR = 1.62 for VR, and M_VRPH = 5.10, Mdn_VRPH = 5, SD_VRPH = 1.63 for VRPH.

A normality test was performed to select the appropriate statistical test to determine differences between means for data sets (1), (2), and (3). We used a Kolmogorov–Smirnov normality test (significance level of .05), as the sample size was >50. Our results showed that the data was not normally distributed, so parametric tests proved unsuitable. For the case of the semantic differential, we decided to apply the aligned rank transform (ART) procedure [63], a powerful and robust nonparametric alternative to other traditional techniques [64]. ART relies on a preprocessing step that “aligns” data before applying averaged ranks. After this, common ANOVA procedures can be applied. For the Confidence in the Response and the Purchase Decision, Q of Cochran test was used (dichotomous data).

The results of the two-factor repeated measures ANOVA test (with media as within subjects’ factor and user background as between subject’s factor) for the Semantic Scales are shown in Table 5. Statistically significant differences were found for “Small–Large” and “Decorative–Functional” adjectives for design A. The bipolar pair “Traditional–Modern” showed significant differences for B and C. Finally, design C showed significant differences for “Simple–Complex” and “Practical–Impractical.”

Umbrella stand A was the only design for which the influence of the participant’s background was significant in some adjectives. Also, a combined factor of medium and background was obtained for “Simple–Complex” and “Ordinary–Elegant” for design A, for “Inexpensive–Expensive” for design B and C, and for “Decorative–Functional” for C.

To determine the conditions between which these differences were found for the factor of the presentation medium, we conducted pairwise comparisons for the Semantic Scales (Table 6) using the Tukey’s honestly significant difference (HSD) test. To use this test, each group must have approximately equal variance (homogeneity of variances). In our case, no between subject factors were specified, so the assumption is always met when using the Levene’s test to assess equality of variances.

To determine differences in the Confidence in the Response, we performed the Q of Cochran test for nonparametric dichotomous data. Our results (Table 7) showed that the level of confidence is affected by the medium for adjectives “Heavy–Light” and “Unstable–Stable” for each product. Statistically significant differences were also found for Umbrella stand C for “Decorative–Functional”. Post-hoc tests are shown in Table 8.

To examine the influence of the participant’s background on the Confidence in the Response, we report the descriptive statistics of the two groups of participants: design-trained (T) and nondesign-trained (NT). The T group was comprised of 41 people, whereas NT was comprised of 17 participants. Results are shown in Table 9.

Finally, no significant differences were found for the intended purchasing decision, as indicated by Cochran’s Q test (χ2(2) = 4.67, p_A < .097), (χ2(2) = 4.20, p_B < .122), (χ2(2) = 3.20, p_C < .212).

To determine the influence of the use of a complementary item, i.e., a physical umbrella, on the evaluation of each umbrella stand, a two-factor mixed ANOVA with two degrees of freedom was performed, with the secondary element (Obj) as the between-subjects factor, and the presentation medium as the within-subjects factor (Table 11). Descriptive statistics for the Semantic Scales are shown in Table 10.

In our first hypothesis (H1), we stated that the purchasing decision was not affected by the change of medium. Based on the results of our study, it seems that the presentation medium does not have a significant effect on the intended purchasing decision for umbrella stands (no significant differences were found between means for any of the products: p_A < .097, p_B < .122, p_C < .212). Our findings agree with other studies that suggest that although users need as much information as possible to make a purchasing decision [65], even a simple presentation medium provides sufficient details to perform a reasonable evaluation [66]. In our case, considering that a 7-point Likert scale was used for product evaluation, the scores obtained between different media (Table 2) did not vary significantly. In addition, the Confidence in the Response scores (Tables 3 and 4) were also high in the three visual media, which suggests that participants had sufficient information to make a purchasing decision.

Our results differ from those obtained in a similar study with coffee makers conducted by Palacios-Ibáñez et al. [23], where significant differences between means were found for the purchasing decision. These differences may be explained by the average age of the selected samples, which was very limited (mean age of 20 years old). In our study, the selected product may have not elicited a purchasing decision in any medium for umbrella stand A and C (Table 4). For the case of umbrella stand B, previous familiarity with the product could have influenced the results, as stated by Söderman [44]. Therefore, hypothesis H1 is rejected for the case of umbrella stands and our experimental conditions. To draw more generalizable conclusions, however, it is necessary to expand the study to other product typologies.

We note, however, that the descriptive statistics for this dataset (Table 4) show higher values in the VRPH condition, which may indicate that presenting the product in this type of setting may lead to an increase in intended purchasing decision. The user’s Confidence in the Response also shows higher values in this medium, which may have had a positive effect on the purchasing decision.

In H2, we postulated that the medium used to display the product influences user perception. Our results (Table 5) revealed significant differences between means for a subset of the bipolar pairs of adjectives in different categories, which confirms H2 for the case of umbrella stands. However, we cannot state which specific adjectives are influenced by the medium, as none of the adjectives were affected in more than one product. In the case of the umbrella stands, we note that certain categories are more sensitive to be affected by the presentation medium than others. Our results are in line with those by other authors. Artacho-Ramírez et al. obtained similar results when studying the influence of the graphical representation in the evaluation of different models of a loudspeaker [22], and Agost et al. reached similar conclusions when assessing two different types of furniture [46]. Palacios-Ibáñez et al. also reported similar results when evaluating three different designs of a coffee maker [23] and Galán et al. [24] did the same using an armchair as the main stimuli. This set of studies demonstrates that regardless of the product selected as stimulus, some characteristics may be more difficult to evaluate depending on the means used to assess it.

The psycho-pleasure category (pleasures related to people’s cognitive and emotional reactions) was highly affected by the presentation medium (Table 5). In this category, design A showed significant differences for “Decorative–Functional” between R–VR and VRPH–VR, whereas design C showed significant differences for “Simple–Complex” and “Impractical–Practical” between VR–VRPH (Table 6). In any case, scores in the VRPH setting were similar to the R setting (Table 2), which suggests that the haptic feedback may have influenced the evaluation of the product. Although previous studies have established that the sense of touch affects mostly adjectives related to the physio-pleasure category (derived from sensory organs) [67], its positive influence on the sense of presence may have also had an impact on product evaluation, as passive haptics significantly enhance virtual environments and can affect cognitive processes [21]. VR scored the lowest in the presence questionnaire as well as the level of Confidence in the Response for the affected adjectives.

Umbrella stand A also showed significant differences (Table 5) for “Small–Large” (physiological pleasure). This result is in line with other authors [24,25], who reported that pleasures deriving from sensory organs are better assessed with the addition of physical interaction. In this case, it was expected that differences appeared between R–VR and VRPH–VR (Table 6), as VR was the condition that offered the least amount of information due to the absence of tactile feedback. The scores for the VRPH condition were similar to those in the R setting, which was expected, since the product data collected from both means were similar (physical interaction and being able to view the product at full scale). Although we expected to find differences between umbrella stands B and C for this pleasure category and behave similarly to design A, it is important to note that the studies mentioned earlier were limited to only one product design and typology, so the findings are not necessarily generalizable to all designs or products as other factors may also impact product assessment [68].

Finally, for B and C, significant differences were found for “Traditional–Modern” between R–VR (Table 6). This Semantic Scale is part of the sociological pleasure category (pleasures deriving from relationships with others), which was the second most affected category and the only one that was affected for design B. These results could be explained by the participants’ previous knowledge of this product, as umbrella stand B was displayed in the university hall for several days. Previous studies reported that previous knowledge of the product can minimize perceptual differences between means [44].

In H3, we speculated that the Confidence in the Response is affected by the presentation medium. Cochran’s Q test results (Tables 7 and 8) showed differences between means for adjectives related to the physio-pleasure category, which confirms H3. The descriptive statistics for the Confidence in the Response (Table 3) also showed higher values for R, followed by VRPH and VR, respectively, which means that haptics can help to increase the participants’ Confidence in the Response. Some authors have argued that haptic feedback during product evaluation increases the user’s Confidence in the Response [56], which explains why the VR setting obtained the lowest scores for this dataset, and why statistically significant differences were found for those product features that relied heavily on the sense of touch. We note that significant differences were also found for umbrella stand C for “Decorative–Practical” between R–VR and R–VRPH. Some participants admitted during the experiment that they were unsure whether the product was an umbrella stand, so this aspect could have been difficult to evaluate.

Our results show that in 64% of the cases, the mean scores obtained in the VRPH medium for the semantic differential were similar to the R setting. Descriptive statistics for the Confidence in the Response (Tables 3 and 4) show the same result for 78% of the cases. According to Grohmann and Spangenberg [56], the introduction of tactile feedback has a positive impact on this factor, reducing perceptual variations. This claim is supported by the results in our two-factor ANOVA for the semantic differential (Table 5) and the post-hoc tests (Table 6). Regardless of the adjectives with statistical differences, no differences were found between R and VRPH. Finally, we highlight that the presentation of the product in a VRPH setting can positively influence the user’s purchasing decision, for the case of umbrella stands. Considering the average age of our sample, it would be interesting to test whether these results are consistent with older adults, since technology acceptance tends to be negatively associated with age [69].

Our results show that the use of touch during the virtual experience could be an effective alternative for product evaluation in the specific case of umbrella stands. Studies with other product typologies are required to draw more generalized conclusions.

To test H4 (how user expertise and design background affect product evaluation), we performed a Two-factor Mixed ANOVA for the Semantic Scales (Table 5). Results showed that “Traditional–Modern” and “Ordinary–Elegant” were influenced by the participant’s design background for design A, which means that this factor is not associated with a specific design typology or adjectives but can affect the perception of certain product characteristics. The psycho and ideo categories showed a combined factor between the medium and the participant’s design expertise. Some studies have reported that in product evaluation individuals with a design-related background tend to employ more higher-level cognitive abilities than untrained individuals [52]. Therefore, the influence of design expertise on the psycho-pleasure category was expected. The descriptive statistics for the Confidence in the Response showed that having a design background does not increase the participant’s Confidence in the Response toward product evaluation, which aligns with the study of Forbes et al. [50] and confirms H4. Making an assessment about specific product features is a subjective task, so having a background in design does not necessarily increase this level confidence.

Finally, a two-factor mixed ANOVA (Table 11) was performed to test H5 (the use of a complementary item to assess a product influences the evaluation of the product Semantic Scales). Our goal was to create a richer and more sophisticated interactive experience which is sometimes limited in online media to determine the impact on product perception. Our results showed that the presence of the complementary item is a factor that significantly influences product perception, which confirms H5. For umbrella stands, the physio-pleasure category was highly affected by this factor. The use of a physical umbrella while evaluating the main stimuli influenced the assessment of “Small–Large” for umbrella stands B and C. The descriptive statistics (Table 10) show that designs A and C were perceived as smaller when using the umbrella, whereas product B was perceived as larger. Likewise, Jordan’s psycho-pleasure category was the second most affected using the complimentary item. The psycho-pleasure category is related to people’s cognitive and emotional reactions toward a product, so the complementary item may have provided additional information about the product’s performance. The descriptive statistics show that the umbrella stands are perceived as more practical when the umbrella is used (67% of the cases), so a more sophisticated experience where all the necessary elements for product testing are present can improve product perception thanks to the availability of more information [8].

A summary of our results is shown in Table 12.

Traditional prototyping processes in product development can be costly. Although new means of representation have emerged in recent years, several authors have shown that the user's perceptual response to a particular product may not be the same depending on the medium used to present it.

Some researchers have begun to explore virtual experiences that leverage the sense of touch, but little research has examined the impact of displaying nonintrusive virtual hand models in the environment. The capability is currently available on low-cost VR devices, but its impact on product evaluation has not yet been studied.

In our study, the hypothesis on the purchasing decision could not be confirmed due to certain limitations of our experiments. Nevertheless, our results highlight the need for designers to conduct user studies on potential customers. We have also shown that the user’s background (i.e., designers versus nondesigners) can influence product assessment. Therefore, the characteristics of the potential buyers must be reflected in the sample.

For our dataset, the VRPH medium scored higher, which suggests that the medium is important when presenting products at the point of sale and that, in some cases, the right medium can increase purchasing decisions. On a practical level, both designers and retailers should allow users to physically interact with their products, since it can result in more favorable emotional responses.

On the other hand, the medium used to present a product may result in perceptual differences during evaluation, as some features of the real product may be difficult or impossible to replicate virtually. In our case, leveraging the sense of touch seems to minimize these differences. The use of passive haptics in the virtual experience for product evaluation may be an effective strategy. From a practical point of view, for some type of products, 3D printers can be used to create physical mockups that enable touch capabilities in the VR environment. In this regard, future development of object tracking capabilities and more accurate hand-tracking performance of HMDs will facilitate the creation of VRPH experiences.

The user’s Confidence in the Response during the virtual experience also increases with the use of tactile feedback, which can have a positive impact during the evaluation. Finally, based on the results related to the use of the complementary item, it is important for designers and marketing professionals provide consumers with all the necessary elements to interact with the product at the time of evaluation. Otherwise, valuable information may remain hidden, distorting the emotional response of the user toward the product.

Participants in our study were exposed to three products (i.e., umbrella stands) in different visual media: a real setting (R), VR, and VRPH. For the evaluation, participants used a semantic differential composed by 12 bipolar pairs of adjectives divided by Jordan’s pleasure categories. In addition, they were asked to provide a purchase decision and rate the level of confidence in their responses. We also gathered information about the user presence in the virtual environments using the SUS presence questionnaire.

Our results demonstrate that the visual medium used to present a product can influence how it is perceived. We also showed that an individual’s background as well as the use of a complementary item during product evaluation (when needed) can also influence product perception.

Our results showed that not all pleasure categories in Jordan’s classification were influenced equally by the visual medium. Although some studies highlight the importance of touch on the evaluation of a product, our study also showed that features that do not require haptics can also be affected by the medium. For the case of umbrella stands, the psycho-pleasure category was the most affected, and the ideo-pleasure category was the only one not influenced by the medium.

Our results showed that the differences in the Semantic Scales between visual media were due to the absence of the sense of touch during the virtual experience. The absence of tactile feedback had a negative influence on the user's level of Confidence in the Response, which influenced the results. Therefore, it is proposed that the use of VRPH can be an effective tool for product evaluation for product conceptualization, particularly in experiences that leverage the sense of touch.

Our study also demonstrated that an individual’s background does not influence the Confidence in the Response, but it can influence the assessment of certain product features, particularly in the psychological and ideological pleasure category. Finally, the use of a complementary item (when needed) for product evaluation can also affect product perception, as additional information on product performance becomes available.

We acknowledge the limitations of our study. First, the particular characteristics of the participants in our sample make our findings only applicable to equivalent user groups. Second, some participants were familiar with one of the designs selected as stimulus. Finally, although our findings could potentially be extrapolated to similar products of the same typology, additional tests with a more diverse set of products are recommended to draw more conclusive results.

Our work contributes to the area of product design by empirically assessing the reliability of VR and VRPH as a tool for product evaluation in the early stages of the product development. In future studies, we plan to use physiological measures such as eye-tracking technologies to analyze user behavior more accurately and objectively during product evaluation activities.

This research work has been supported by the Spanish Ministry of Education and Vocational Training under a FPU fellowship (FPU19/03878).

There are no conflicts of interest.

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.