Citation Download PDF

International Journal of Innovation and Economic Development
Volume 4, Issue 6, Februaury 2019, Pages 51-67

---

Virtual Reality-Based Product Representations in Conjoint Analysis

DOI: 10.18775/ijied.1849-7551-7020.2015.46.2004
URL: URL: http://dx.doi.org/10.18775/ijied.1849-7551-7020.2015.46.2004

 Jonas Jasper

Institute for Business-to-Busines Marketing, Münster, Germany

Abstract: Innovations are an important factor for companies to build and sustain a competitive advantage. Even though companies generate up to 51 % of their income with products and services launched less than three years ago, the failure rate of new product development is still at around 90 %. One way to reduce the failure rate of innovations is to conduct customers’ preference measurements in the early stages of new product development processes. However, customers often display a substantial amount of uncertainty when having to evaluate highly innovative products as they may not be able to grasp its features and functionalities. Therefore, some empirical studies have already compared different presentation forms in a conjoint setting. These studies aim to optimize preference measurement results by exposing respondents to state-of-the-art product descriptions in the new product development process. Based on a quantitative-empirical analysis, this thesis contributes to this research vein by integrating virtual reality-based (VR) product representations within a conjoint setting for the measurement of preferences in the development process of a technically complex innovation. The results of this study point out that VR offers the potential for early customer integration within the new product development process.

Keywords: Virtual reality, New product development, Conjoint analysis

Virtual Reality-Based Product Representations in Conjoint Analysis

1. Introduction

Innovations have become one source for companies to sustain a competitive edge and they are essential for survival (Chuang 2015; Porter 2011; Backhaus et al. 2014a; Brockhoff 1999; Christensen 1997). In times of global competition, a company’s ability to introduce new products to the market is a decisive factor in maintaining their competitive position, and it is crucial to a company’s long-term success (Carayannis et al. 2015). A study of the Centre for European Economic Research points out that innovative companies made up to 51% of their income with products and services launched less than three years ago (Rammer et al. 2014).
Also, the number of innovations across industries is increasing, while the time span over which they are launched is shrinking (Carayannis et al. 2015). Companies aim to exploit time advantages by launching new products faster than their competitors. Thus, companies start to engage in a race to become pioneers when bringing innovations to the market. At the same time, the failure rate of new product development can be up to 90 % (Hill et al. 2014). This failure rate puts companies under pressure to pursue an efficient product development process to remain competitive.

A problem within the new product development process is the division between developers’ and users’ points of view (Daecke 2009). An advantage from a developer’s point of view is not in itself a sufficient condition when it comes to success in product development (Schmidt 2001; Ernst 2002). What is important is the customer’s
Perception of the new product as being useful (Ogawa/Piller 2006). Thus, customers make the final decision on the success or failure of a new product or service (Backhaus et al. 2014a; Meffert et al. 2012).

But why do so many innovations fail in satisfying customers? An answer to this question is found in von Hippel’s (1986) framework of sticky information. For the development of customer-oriented new products, there are two different types of information that need to be aligned. On the one hand, there is a need for information, which refers to what users need. On the other hand, there is solution information, which refers to how to build products. The quality of the solution in conjunction with the significance of the needs it meets represents the value of the innovation (Herstatt/von Hippel 1992). The problem is that need information and solution information share two characteristics:

1. First, each has its respective location. By definition, users are in “possession” of the need information, since their needs have to be satisfied. On the other side, the “owners” of the solution information are the manufacturers and designers.
2. Second, both types of information are sticky, meaning that it is difficult both to translate and to transfer them to the respective other location. In other words, it is difficult for manufacturers to gather need information from customers, while at the same time customers are not motivated and do not have the expertise to understand the solution information of the companies (Toubia 2010).

This information transfer issue provides a fundamental challenge for new product developments. If companies or manufacturers were able to grasp customer needs in detail and with little effort, it would be easy for them to develop a product that fully satisfies these needs. By the same token, if customers could become experts in the field of how to find a solution for their needs, they would be able to develop new products on their own (Toubia 2010). Due to this stickiness, companies pursue a variety of strategies for the sake of gathering relevant need information from customers. This latter procedure refers to a customer-oriented new product development process (NPD), which is intended to increase market acceptance of new products and services (Ernst 2002; Narver et al. 2004; Helm/Steiner 2008; Backhaus et al. 2014a; Atuahene-Gima 2005). When aiming to make the development process more efficient, obtaining input from customers in the early stages of new product development processes has proven to be a promising strategy (Griffin/Hauser 1993). A systematic procedure for capturing and incorporating customer needs and wants throughout the development of a new product or service is thus highly relevant to reduce failure rates on innovations, especially in the context of global competition and global markets (Füller/Metzler 2007; Barczak et al. 2009; Kim et al. 2011). Figuring out customers’ preferences has become the “focal point”’ (Ye et al. 2007, p. 1191) for most companies to ensure that their products become successful on the market. However, asking customers what they want is not necessarily a strategy that is well-suited to eliciting need information. Customers tend to mention needs that are served by corresponding products already available on the market (Ulwick 2002; van Kleef et al. 2005).

This problem is particularly challenging when dealing with technical and highly innovative products with which customers have no experience (van Kleef et al. 2005). Customers often display a substantial amount of uncertainty when having to evaluate these types of product (Hoeffler 2003). Highly innovative products require that customers change their behavior to adapt to the product (Goldenberg et al. 1999). Therefore, one major difficulty when measuring customer preferences results from customers’ lack of knowledge about new products. When confronted with and asked to state preferences about innovative products, customers may be unable to give credible responses (van den Hende/Schoormans 2012). Reduced stability of market research assessment and information with limited validity and accuracy are the result (Shocker/Hall 1986; Durgee et al. 1998). When this happens, standard preference measurement techniques fail (Hoeffler 2003; van Kleef et al. 2005). Therefore, marketing scientists and practitioners have come up with new ideas to optimize preference measurement techniques to avoid the above-mentioned problems.

When measuring customer preferences within the new product development process, a vast amount of different techniques exist, whereby the conjoint analysis is generally accepted as one of the most relevant methods for marketing research (Wittink et al. 1994; Helm et al. 2004; Steiner 2007). It provides insights into respondents’ preferences for product attributes and features and thereby lets companies guide the development process in the “right” direction at an early stage. The conjoint analysis supports the identification of relevant features a new product should have and enables determination of how it is priced. The underlying idea is that respondents evaluate the overall desirability of a comprehensive product or concept, instead of assessing different attributes and features independently (Hillig 2006; Orme 2006}. Such an evaluation is made possible by developing some alternative product concepts (stimuli) that consist of previously defined attributes and features (levels) and by having respondents evaluate these features jointly (Sichtmann/Stingel 2007).

2. Literature Review

Research in the field of integrating varying presentation styles in a conjoint analysis started in 1981, shortly after Green/Srinivasan introduced the conjoint analysis in their well-known paper. Since then, the research stream has continued to make progress, and conjoint settings have been used in various contexts and with varying presentation styles. One reason for the ongoing interest in this research vein is the steady emergence of new options to build more advanced ways of stimuli presentation. Dahan/Srinivasan (2000) tested the effectiveness of verbal, static and animated virtual prototypes against physical product concepts, to predict market shares for bicycle pumps. While verbal stimuli predicted market shares well below actual shares, both static and animated virtual prototypes closely matched actual market share predictions, which in turn came from physical prototypes. In their paper “The Virtual Customer”, Dahan/Hauser (2002) provide empirical insights and an overview of how customers can be integrated into each stage of the product development process using six different web-based interview techniques. The authors reveal that multimedia-propped product concepts increase the efficiency and accuracy of the detection of respondents’ preferences (Dahan/Hauser 2002). In the most recent study conducted by Brusch (2009), textual, multimedia and real prototypes of door handle systems are compared in a conjoint setting. The author attested that multimedia-based product concepts yield the highest predictive validity by exceeding both, real and verbal-based descriptions.

3. Research Goals and Conceptual Foundations

One promising new path for early customer integration is the use of virtual realities for the measurement of preferences in the development process of complex innovations (Backhaus et al. 2014a; Söderman 2005; Kim et al. 2011). Virtual Reality (VR) is a technology of virtual prototyping (VP), which stands for an easy-to-understand user interface that allows customers to explore the functionality of new products and features interactively (Gausemeier et al. 2001). Virtual reality is a three-dimensional, computer-generated environment that allows customers to interact with and to manipulate the product representation in real time. The advantage of virtual realities is that they can be developed earlier in the new product development process, more quickly and more cost-effectively than actual prototypes (Backhaus et al. 2014a; Urban et al. 1997}. Also, virtual representations of highly complex products aim to facilitate the transfer of information by presenting the features and benefits of the new product to potential clients via a combination of visualizations, animations, illustrations and texts (Gausemeier et al. 2011; Gausemeier et al. 2001).

Since virtual reality allows customers to immerse themselves in the virtual world, a considerable amount of realism can be expected (Gausemeier et al. 2001; Ye et al. 2007; Backhaus et al. 2014a). Even though previous studies have used multimedia-based stimuli, no one so far has exposed respondents to state-of-the-art forms of virtual reality-based product concepts that would allow full customer interaction and immersion.

This study has the aim to shed light on this stream of research. To such end, the study at hand is guided by the following research question:

Can preference measurements for a technical and highly innovative product be improved by integrating virtual reality-based product representations into a conjoint analysis?

The admittedly generic formulation of this research question intends to ensure that all different facets are covered under its umbrella.

3.1 Object of Study

In line with the research gaps identified above, the study at hand aims to test the usefulness of varying presentation styles within a conjoint analysis for a technical and highly innovative product. Thus, the product should be in its infancy stage to ensure that customers are not familiar with its functions and benefits. Also, it is essential that the product can be described by some decision-relevant attributes and levels, which in turn can be represented by using different presentation styles (Ernst 2001). Furthermore, the product should contain different features, to ensure that the product has diverse utilities to customers.

At the time the study was run, a major German automotive headlight manufacturer had just started the development of a new product within the leading edge technology network Intelligent Technical Systems OstWestfalenLippe, which covers the requirements. The company has an intelligent self-adjusting automotive headlight system in the concept phase, which continuously analyzes the environment and relevant vehicle data for independently controlling and adapting the optimal headlight settings for the illumination of the street in front of the automotive. A similar system is not available on the market and respondents do not have experience with it. Furthermore, the system is seen as containing a high degree of complexity and is difficult to explain. As the self-adjusting headlight system builds on newly developed technologies, it is highly innovative and is classified as technical innovation (Zahn 1995). Finally, automotive headlight systems are represented by different presentation forms such as text, pictures, and virtual realities. For the manufacturer, the intelligent self-adjusting automotive headlight system is a cost-intensive innovation project involving considerable financial risk. Thus, the company has an interest in receiving feedback from a customer assessment to steer the development project into the “right” direction at an early point in the new product development process (Backhaus et al. 2014a)

3.2 Sample and Data Collection

Data are collected during one week in Germany at the world’s biggest industrial trade fair. A total of 367 respondents participated in the study. Of these, two respondents had to be eliminated, as they did not complete all three tasks, which left a sample of 365 cases. These 365 respondents were divided as follows: 128 respondents conducted the assignment with verbal stimuli, 126 with pictorial stimuli and 111 respondents are exposed to virtual-based stimuli. To achieve a high level of realism with the virtual prototype, respondents sat on a force-feedback motion platform, that is, a dynamic driving simulator, as shown in Figure 1, in front of a 75”-LCD display, and interactively steered a virtual automotive at night (Backhaus et al. 2014a, Backhaus et al. 2014b).

Figure 1: Dynamic Driving Simulator

Source: Heinz Nixdorf Institute Paderborn

Even though the possibility of a sampling error was reduced by randomly assigning respondents to groups, the composition of respondents was examined (Schmidt 2001, Stadie 1998). Otherwise, it is possible that differences between the groups could wrongly be ascribed to the presentation style, even if they are the result of differing respondent profiles. Indeed, more men than women participated in the study and about half the respondents had an academic degree, which should be taken into consideration when interpreting the results. Nevertheless, respondents show similar patterns regarding age, gender, educational attainment and prior knowledge across the three groups.

Additionally, groups’ respective composition are compared concerning different personal characteristics, which are used in the second and third part of the empirical study.  Except for the analytic cognitive style, the null-hypothesis of equal means cannot be rejected (Bowerman/O´Connell 2007; Field 2009). Thus, respondents from the text group tend to be slightly more analytical, compared to both other groups. It should be taken into consideration when interpreting the results. For all other variables, the composition of respondents across the three groups is homogenous. Overall, these findings confirm that the structure of respondents across the three presentation styles is similar for all but the analytic cognitive style, thereby qualifying the usage of a between-subject design (Schmidt 2001; Stadie 1998).

4. Results

4.1 Comparison of Validity Measures

Before examining the validity measures, a comparison is made if conjoint outcomes differ when respondents see text, picture or virtual reality-based product concepts in conjoint analysis. After all, if no differences are found between outputs of different presentation styles, further investigation of the superiority of one presentation style over the others is less promising. Hence, part-worth utilities and relative importance weights were compared between the three groups using an ANOVA and subsequently conducted Games-Howell-tests (Field 2009; Stadie 1998; Schmidt 2001; deBont 1992}.

4.2 Convergent Validity

4.2.1 Part-Worth Utilities

As a first indicator for assessing if differences exist between groups, a comparison of each of the three groups’ respective part-worth utilities follows (Stadie 1998). Respondents’ part-worth utilities are first calculated on an individual basis and then aggregated for each respective group (i.e. text, picture, and VR-based stimuli). The resulting part-worth utilities are shown in Figure 2.

Figure 2: Group Specific Part-Worth Utilities

Source: Own illustration

Overall, part-worth utilities between the groups of text, picture, and virtual reality-based stimuli differ slightly. For the first attribute (user interaction), a required interaction for starting the self-adjusting automotive headlight system is the least attractive choice across all three groups. Respondents exposed to pictorial and verbal stimuli perceive the optional interaction as the best one, while respondents from the VR group favor no interaction. This is a remarkable finding, as user interaction is the attribute within the virtual reality in which most of the actual interaction takes place. In other words, the empirical analysis shows that preference measurement using VR-based stimuli differs significantly for the attribute calling for a great deal of involvement on a respondent’s part. Since one characteristic of a VR is the interactivity component, this finding is a first indication that VR-based product concepts do indeed yield diverging results.

For the attribute adjustment scenario, respondents from the text and picture groups attach the least utility to the option in which the automotive needs to stand in front of a wall/garage to conduct the headlight adjustment. The second-to-least utility is attached to the option in which the adjustment process is to take place on the rear of a car in front. The greatest utility has a system that can perform the adjustment on the street in front of the automotive. Even though respondents from the VR group also attest the level on the street to have the greatest amount of utility, they rate the option on a wall slightly better than on the rear of the preceding car.

Concerning the cut-off line, the marginal difference is attested between groups. While respondents from the picture group favor the asymmetric z-shape, respondents from the text and VR group prefer the L-shape. As the difference between all levels is rather low, this indicates that this attribute is of minor importance for respondents’ overall decision.

Finally, the price attribute shows the most likely pattern: for all three respondent groups, the highest price (EUR 2,500) receives the least utility, followed by the average price (EUR 1,800) while the lowest price (EUR 1,100) has the highest utility.

To test for significance, a one-way analysis of variance (ANOVA) followed by post-hoc tests are conducted (Field 2009; Bowerman/O´Connell 2007}. Before calculating the one-way ANOVA and subsequent post-hoc tests, three underlying assumptions are tested: first of all, the samples of experimental units need to be randomly selected and must be independent of one another. As shown in the study setup, this prerequisite holds. Secondly, the ANOVA requires groups to be distributed normally (Stevens 2007; Bortz 2005; Everitt 1996). The Kolmogorow-Smirnow and the Shapiro-Wilk test point out that this assumption is violated (Field 2009). Finally, the ANOVA requires equal variances of each group, which can be assessed using the Levene-test (Bowerman/O´Connell 2007). According to the Levene-test, the data at hand violates the equal variance assumption. As the normal distribution and the equal variance assumptions are violated, the Welch-test is conducted as an alternative to the one-way ANOVA (Field 2009; Steiner 2007}.

When conducting subsequent multiple group comparisons (i.e. posthoc tests), alpha inflation can be an issue (Field, 2009). Alpha inflation is an increase in the probability of committing a Type I error proportionate to the number of group comparisons (i.e. post-hoc tests) performed (Field 2009). For instance, when comparing three groups, the probability of a Type I error increases from 5 % to 14.3 % . To avoid alpha inflation, the Bonferroni correction is applied (McLaughlin/Sainani 2014; Field 2013; Bowerman/O´Connell 2007}. As two assumptions of the ANOVA are violated, the “most powerful” (Field 2009, p. 374) post-hoc test is the Games-Howell-test, as it not only accounts for alpha inflation, but remains valid even in the event that the equal variance assumption is violated and sample sizes are unequal (Games et al. 1981; Field 2013). Hence these parametric tests are calculated to compare the mean part-worth utilities of all three groups.

The Welch-test indicates that statistical differences between groups can be attested for the levels interaction optional and on the rear of a car in front. The subsequently conducted Games-Howell-test points out that the text and VR group differ regarding the optional user interaction at the .05 level while the picture and VR group differ at the 1 %-significance level. Furthermore, for the variable on the rear of a car in front, the null hypothesis that the picture and VR group are the same can be rejected in favor of the alternative hypothesis that means differ between both groups at the .1 significance level. One possible explanation is that respondents with virtual reality-based stimuli were able to see that the traffic ahead can actually be disturbed and therefore attach less utility to this feature. Overall, groups diverge across the above mentioned part-worth utilities, while no difference is attested between the text and picture group.

4.2.2 Relative Importance

A second indicator for testing convergent validity is to compare the relative importance weights between groups (Stadie 1998; Schmidt 2001}. Figure 3 depicts the average relative importance across the three groups. While respondents from the text and picture group have a similar focus, the emphasis respondents from the VR group place on these attributes deviates.

More specifically, the user interaction attribute has almost the same relative importance for respondents of the text (42.22 %) and picture (41.72 %) group. However, the group of respondents exposed to VR-stimuli perceived the attribute as less important (29.83 %). The VR group, in turn, placed greater importance on the adjustment scenario: 37.65 % as compared to 20.14 % and 24.96 % for the text and picture group, respectively. As indicated above, the cut-off line attribute has uniform relative importance across all groups, ranging from 12.37 % for the text group to 13.45 % for the picture group, and up to 15.77 % for the VR group. Finally, the price attribute has the least relative importance for the VR group, at 16.75 % compared to 19.86 % for the picture group and 25.27 % for the text group.
The statistical significance was evaluated using ANOVA and subsequent post-hoc tests (Field 2009). The output of the significance levels of the group comparisons depicts Table 1.

Figure 3: Relative Attribute Importance across Groups

Source: own illustration

Table 1: Significance Levels for Mean Comparison of the Relative Importances

Source: Own illustration

In conjunction with the findings in Figure 3, the user interaction attribute differs significantly between the text and the VR group (p £ .01) and between the picture and the VR group (p £ .01). No significant differences were found between the text and picture group, as already shown in Figure 3. For the adjustment scenario, the text and the picture group differ at the 5 %-significance levels, while the VR group differs from both other groups even at the 1 %-significance level. For the relative importance of the cut-off line, no significant differences are attested between groups. Finally, respondents perceived the price} attribute differently between respondents from the text and picture group (p £ .05) and between text and VR group (p £ .01). No significant deviations in the relative importance of the price attribute are detected between the picture and the VR group. Overall, these results indicate that the three groups do indeed diverge concerning the relative importance of the attributes. In other words, when exposing respondents in a conjoint setting to different product presentation forms, the relative importance they assign to each of these attributes varies.

4.3 Internal Validity

To address the question of which presentation style is superior, the coefficient of determination (adjusted R²) was used as a first criterion (Backhaus et al. 2011; Bowerman/O´Connell 2007). As an additional goodness-of-fit measure, the correlations between empirical input data and estimated preference data across the three different groups (i.e. text, picture and VR) are compared. As data for this study are on an interval scale and are converted into ordinal scales, all three correlation coefficients, namely Pearson R, Spearman’s rank correlation and Kendall’s Tau, can be estimated (Brusch 2009; Backhaus et al. 2011; Steiner 2007}. Table 2 depicts the output.

Table 2: Internal Validity across Groups

Source: own illustration

First of all, the adjusted R² is reasonably high, which means that respondents’ answering behavior can overall be regarded as consistent (Bowerman/O´Connell 2007). Secondly, respondents exposed to textual stimuli have the lowest fit, where .74 of the total variation is explained by the estimated OLS model. For respondents exposed to pictorial stimuli, the adjusted R²  rises to almost .78. Finally, for respondents whose product explanations are based on virtual reality, the adjusted R²  is over .81. Thus, the internal validity increases from text to picture and peaks with virtual reality-based product concepts. The standard error decreases, meaning that the variation across respondents is lower and the answering behavior is more consistent. As shown in Table 2 adjusted R² between the text and the VR group differs at the .1 significance level only. Furthermore, the correlation coefficients across all groups are also reasonably high, meaning that respondents from all three groups show high consistencies in their answering behavior. However, no significant differences are attested between all three groups between Pearson, Spearman and Kendall’s Tau. Hence, based on internal validity, only weak statements are made as to which presentation style (i.e. text, picture or VR) leads to superior results since no significant differences are attested to all but the adjusted R² between the text and VR group.

4.4. Predictive Validity

One goal of preference measurements and conjoint studies in the narrow sense is to predict actual decisions respondents would make in a real-life decision task (Dahan/Srinivasan 2000). Therefore, predictive validity is tested across the three groups of presentation styles (Ernst 2001). As three hold-out cards are included in this study, it was possible to estimate the first-choice hit rate as well as the first-second-choice hit rate (Steiner 2007). Table 3 exhibits the results across all three respondent groups.

Table 3: Predictive Validity across Groups

Source: own illustration

While the first-choice hit rate is only slightly better for the VR-group, the first-second-choice hit rate can predict actual decision behavior of the VR group better than in the text group by 7.36 percentage points. In comparison to respondents from the picture group, actual decisions of the VR group can be predicted even better, by 10.47 percentage points. A chi-square test was performed to test if the three groups differ at a significant level (Cerjak et al. 2010}. As shown in Table 3, the picture and VR group differ significantly concerning the first-second-choice hit rate (p £ .05). This result indicates that the conjoint study with embedded virtual realities as stimuli is slightly better in predicting actual decision behavior of respondents compared to a picture enriched conjoint setting.

4.5 Applicability Measures

The experience a respondent has of his or her interview (i.e. applicability measures) is assessed as an additional validity criterion, as respondents’ perceived interview experience provides additional insights into the question of which presentation format leads to the higher validity and thereby to better preference measurement results. Table 4 depicts the average value of respondents’ answers for all applicability measures and the respective significance levels.

Table 4: Applicability Measures across Groups

Source: own illustration

First, as shown in Table 4, respondents exposed to virtual reality-based stimuli were significantly more satisfied with the amount of information provided as product description, compared to respondents shown text or picture-based stimuli (p ≤ .01).
Second, the VR group assessed the degree of realism significantly higher than either of the other groups (p ≤ .01). These findings emphasize that the degree of realism increases towards the more realistic end of presentation form alternatives. However, scholars have different opinions regarding the question of whether or not a high degree of realism within a conjoint study is favorable or not: while Srinivasan et al. (1997) advocate a high degree of realism to get more viable conjoint results, Schrage (1993) and Appleyard (1977) state that highly realistic product presentations can be too difficult to comprehend.

To test this question, respondents’ decision difficulty is retrieved as a third applicability measure. As can be seen, the perceived decision difficulty of the VR group is significantly lower than for respondents of the text (p ≤ .01) and picture groups (p ≤ .1). This finding contradicts the conclusion drawn by Cerjak et al. (2010) and Jaeger et al. (2001), that respondents experience the same degree of difficulty when exposed to different presentation styles. It also refutes Ernst (2001), who states that the degree of difficulty is higher, the more enriched presentation formats become.

Spiegler (2011); Hoeffler/Ariely (1999) point out that respondents who feel uncertain about their decisions or are overwhelmed by the evaluation task are likely to apply simple decision rules instead of profound trade-off decisions, which in turn can result in less precise preference measurements. As can be seen from Table 4, respondents of the VR group show a significantly higher degree of certainty in their decision of placing the different product concepts in the conjoint task than the text group (p  .1) and the picture group (p  .1). Contrary, the text and picture groups almost experienced the same degree of certainty. This finding partially stands in contrast to the results found by Jaeger et al. (2001), who could not attest any significant differences between groups of presentation styles.

Toubia et al. (2012, p. 138) point out that “one key limitation of preference measurement methods such as conjoint analysis is the potential lack of motivation experienced by respondents”. To avoid lack of motivation, conjoint tasks should be enjoyable and diverse (Helm et al. 2003; Cerjak et al. 2010; Steiner 2007). As shown in Table 4, groups do indeed differ, even at the 1 %-significance level. The same conclusion is drawn for the picture and VR groups (p ≤ .01). Similarly, respondents from the VR group perceived the conjoint task as more diverse than the text group (p ≤ .01) and the picture group (p ≤ .01). However, for both measures, no differences can be attested between the text and picture groups. It can be concluded that when respondents are exposed to virtual reality-based product concepts, they have more fun and perceive the task as more diverse. It is an important finding, as marketing scholars have tried to increase respondents’ level of enjoyment during a conjoint task in order to optimize preference measurement results (Toubia 2012).

Respondents’ enjoyment can be affected negatively, though, if the time it takes to conduct the conjoint task is perceived as being too long (Ernst 2001; Day 1975; McDaniel et al. 1985}. To account for this, the objective interview time it took respondents to conduct the conjoint task is compared to the perceived interview time. As shown in Table 4, the actual time it took for the text group to complete the conjoint task was 419 seconds. The picture group completed the task in 451 seconds, and the conjoint task took respondents of the VR group an average of 713 seconds to complete. The mean difference between the VR group and the text group on the one hand, and VR group and the picture group, on the other hand, is significant at the 1 %-significance level. This finding is not surprising, as watching a virtual reality product concept (i.e. the automotive headlight adjustment process) already takes some time.

What is indeed surprising, however, is what happens when additionally taking into consideration the perceived interview length across groups: the results indicate that the group presented with virtual reality-based product concepts perceived the interview as being significantly shorter than the group that had only textual descriptions . In other words, even though it objectively took respondents of the VR group 70 % longer to conduct the conjoint task than respondents from the text group, they perceived the task as being significantly shorter. The only aspect for which no significant differences are found across the three groups was the amount respondents would need to be paid in order to take part in a similar study in the future again.

4. Discussion and Implications for Further Research

This study aims to compare the effect on preference measurement results in a conjoint analysis and validity criteria for respondents exposed to text, picture, and virtual reality-based presentation styles.

First of all, the comparison of part-worth utilities across all three groups attests that for two part-worth utilities, significant differences exist between the text and VR and between the picture and VR groups.

Secondly, comparing the relative importance weights that respondents attach to each attribute emphasizes that respondents in each group focus on different attributes. While the text and picture group and the picture and VR group differ for two out of four attributes, the text and VR group differ significantly for three out of four product attributes. It is interesting to note that the relative importance of the price decreases as the presentation form moves towards the more realistic end of the continuum. The price attribute becomes less important when moving from text to picture and from picture to VR-based presentation forms. One possible interpretation of this tendency is that respondents from the text group had no clear understanding of the product and its features. Instead, when ranking different cards, they relied on price as the decisive factor, as – in contrast to all other attributes – a lower price is better than a higher price, due to its interval scale (Stevens 1946). By contrast, groups that had a picture or a virtual reality as their information source were better able to capture the functionality of the other attributes and relied less on price. An ordinary least square regression is conducted between the relative importance of the price attribute and respondents’ product understanding, with product understanding , as the independent and relative importance of the price as the dependent variable. The resulting beta-coefficient is negative (β = -.18) and significant at the 1 %-level, which supports the conjecture that the relative importance of price decreases as respondents’ understanding of the product increases.

Shedding light on the question of which presentation style leads to superior results was subsequently looked at. This is accomplished by comparing results between groups concerning internal validity, predictive validity and different applicability measures. In particular, the adjusted R2 for respondents of the VR group is higher than for the text group. However, no significant differences are found regarding the additionally tested correlation coefficients, such as Pearson, Spearman and Kendall’s Tau. Thus, based on internal validity, only weak conclusions can be drawn as to which presentation style shows outperforming results, a finding also made by previous researchers. The same line of argumentation holds for predictive validity between groups. Here, significant differences are only attested between the picture and the VR group concerning to the first-second-choice hit rate, where the VR group outperformed respondents from the picture group.

Finally, additional applicability measures were assessed to find an answer to the second research question. As depicted in Table 4, the VR group showed significantly better results for eight out of nine applicability measures than the text group and six out of nine than the picture group. For instance, not only did respondents from this group perceive the presented stimuli as more realistic; they also were certain of themselves when making their decisions than respondents from either of the other groups. Furthermore, they had fewer difficulties in ranking the product concepts, and they enjoyed the conjoint task more than respondents from the text and picture group. Finally, even though it took respondents from the VR group longer to conduct the conjoint task, they perceived it as less time-consuming than either of the other groups did. These findings for the applicability measures indicate that conjoint results with embedded virtual realities outperform both of the other presentation forms for this context. At the same time, no differences are attested between respondents from the text and picture group.

4.1 Limitations

Notwithstanding these findings, this study faces some limitations. For instance, only text, picture, and virtual reality-based product concepts are compared. However, adding physical prototypes could improve the results, as respondents’ actual decision is consulted as a measure of comparison (Dahan/Srinivasan 2000). As the object of study was in its infancy stage, no physical prototype was available, which excluded the feasibility of adding physical prototypes. Also, this study was limited in measuring the effect of varying presentation styles on the validity of conjoint results only. However, as shown above, before deciding in favor of or against a certain presentation style, an analysis should be conducted as to how respondents’ product understanding is influenced when exposing them to different presentation forms. A final limitation is that this study implicitly assumes that respondents are all equal in their capabilities to process information presented in different modalities. However, the cognitive theory of multimedia learning (CTML) points out that individuals differ in their abilities to process verbal, visual and VR-based information (Mayer 2005; Paivio 2013). While some individuals are better able to deal with textual descriptions, others are more proficient at dealing with visual information elements. These limitations should be taken into consideration in further research.

References

• Adofu, I., and M. Abula.”Domestic debt and the Nigerian economy.” Current Research Journal of Economic Theory 2.1 (2010): 22-26.
• Appleyard, D. (1977): Understanding Professional Media: Issues, Theory and Research Agenda, in Human Behaviour and Environment, ed. by I. Altman and J. F. Wohlwill, New York, NY: Plenum Press. Crossref
• Atuahene-Gima, K. (2005): Resolving the Capability: Rigidity Paradox in New Product Innovation, Journal of Marketing, 69 (4), 61–83. Crossref
• Backhaus, K., B. Erichson, W. Plinke, and R. Weiber (2011): Multivariate Analysemeth- oden: Eine anwendungsorientierte Einführung, Berlin: Springer, 13 ed.
• Backhaus, K., J. Jasper, K. Westhoff, J. Gausemeier, M. Grafe, and J. Stöcklein (2014a): Virtual Reality Based Conjoint Analysis for Early Customer Integration in Industrial Product Development, 8th International Conference on Digital Enterprise Technology – DET 2014 Disruptive Innovation in Manufacturing Engineering towards the 4th Industrial Revolution, 25 (0), 61–68.
• Backhaus, K., J. Jasper, K. Westhoff, J. Gausemeier, M. Grafe, and J. Stöcklein (2014b): VR-Basierte Conjoint-Analyse zur frühzeitigen Ermittlung von Kundenpräferenzen, IFF Wissenschaftstage.
• Barczak, G., A. Griffin, and K. B. Kahn (2009): Perspective: Trends and Drivers of Success in NPD Practices: Results of the 2003 PDMA Best Practices Study, Journal of Product Innovation Management, 26 (1), 3–23. Crossref
• Bergkvist, L. and J. R. Rossiter (2007): The Predictive Validity of Multiple-Item Versus Single-Item Measures of the Same Constructs, Journal of Marketing Research, 44 (2), 175–184. Crossref
• Bortz, J. (2005): Statistik für Human-und Sozialwissenschaftler, Heidelberg: Springer, 6 ed. Bowerman, B. L. and R. T. O’Connell (2007): Business Statistics in Practice, McGraw-Hill Irwin, 4 ed.
• Brockhoff, K. (1999): Forschung und Entwicklung, München: Oldenbourg. Crossref
• Brusch, M. (2009): Präsentation der Stimuli bei der Conjointanalyse, in Conjointanalyse, ed. by D. Baier and M. Brusch, Berlin: Springer, 83–98.
• Carayannis, E., E. Samara, and Y. Bakouros (2015): Introduction to Technological Innovation, in Innovation and Entrepreneurship, Springer, 1–26.
• Cerjak, M., R. Haas, and D. Kovacic (2010): Brand Familiarity and Tasting in Conjoint Analysis: An Experimental Study with Croatian Beer Consumers, British Food Journal, 112 (6), 561–579. Crossref
• Christensen, C. M. (1997): The Innovator’s Dilemma: When New Technologies Cause Great Firms to Fail, Boston, MA: Harvard Business School Press.
• Chuang, F.-M., R. E. Morgan, and M. J. Robson (2015): Customer and Competitor In- sights, New Product Development Competence, and New Product Creativity: Differential, Integrative, and Substitution Effects, Journal of Product Innovation Management, 32 (2), 175–182. Crossref
• Daecke, J. (2009): Nutzung virtueller Welten zur Kundenintegration in die Neuproduktentwicklung, Wiesbaden: Springer. Crossref
• Dahan, E. and J. R. Hauser (2002): The Virtual Customer, Journal of Product Innovation Management, 19 (5), 332–353. Crossref
• Dahan, E. and V. Srinivasan (2000): The Predictive Power of Internet-Based Product Concept Testing Using Visual Depiction and Animation, Journal of Product Innovation Management, 17 (2), 99–109. Crossref
• Day, G. S. (1975): The Threats to Marketing Research, Journal of Marketing Research, 12 (4), 462–467. Crossref
• De Bont, C. J. P. M. (1992): Consumer Evaluations of Early Product-Concepts, Delft University, Delft.
• Durgee, J. F., G. C. O’Connor, and Veryzer Jr, R. W. (1998): Using Mini-Concepts to Identify Opportunities for Really New Product Functions, Journal of Consumer Marketing, 15 (6), 525–543. Crossref
• Ernst, H. (2002): Management der Neuproduktentwicklung, in Handbuch Produktionsmanagement: Strategieentwicklung – Produktplanung – Organisation – Kontrolle, 2 ed., ed. by S. Albers and A. Herrmann, Wiesbaden: Gabler, 333–358. Crossref
• Ernst, O. (2001): Multimediale versus Abstrakte Produktpräsentationsformen bei der Adaptiven Conjoint-Analyse: Ein empirischer Vergleich, Frankfurt am Main: Lang.
• Everitt, B. (1996): Making Sense of Statistics in Psychology: A Second-Level Course, Oxford: Oxford University Press.
• Field, A. (2009): Discovering Statistics Using SPSS, Los Angeles, CA: Sage, 3 ed.
Field, A. (2013): Discovering Statistics Using IBM SPSS Statistics, Los Angeles, CA: Sage, 4 ed.
Füller, J. and K. Matzler (2007): Virtual Product Experience and Customer Participation — A Chance for Customer-Centred, Really New Products, Technovation, 27 (6–7), 378–387.
• Games, P. A., H. J. Keselman, and J. C. Rogan (1981): Simultaneous Pairwise Multiple Comparison Procedures for Means when Sample Sizes are Unequal, Psychological Bulletin, 90 (3), 594. Crossref
• Gastwirth, J. L., Y. R. Gel, and W. Miao (2009): The Impact of Levene’s Test of Equality of Variances on Statistical Theory and Practice, Statistical Science, 24 (3), 343–360. Crossref
• Gausemeier, J., J. Berssenbrügge, M. Grafe, S. Kahl, and H. Wassmann (2011): Design and VR/AR-Based Testing of Advanced Mechatronic Systems, in Virtual Reality & Augmented Reality in Industry, ed. by D. Ma, X. Fan, J. Gausemeier, and M. Grafe, Berlin: Springer, 1–37.
• Gausemeier, J., P. Ebbesmeyer, and F. Kallmeyer (2001): Produktinnovation: Strategische Planung und Entwicklung der Produkte von Morgen, München: Hanser.
• Goldenberg, J., D. R. Lehmann, and D. Mazursky (1999): The Primacy of the Idea Itself as a Predictor of New Product Success, Marketing Science Institute, 99–110.
• Green, P. E. and V. Srinivasan (1978): Conjoint Analysis in Consumer Research: Issues and Outlook, Journal of Consumer Research, 5 (2), 103–123. Crossref
• Griffin, A. and J. R. Hauser (1993): The Voice of the Customer, Marketing Science, 12 (1), 1–27. Crossref
• Helm, R., L. Manthey, A. Scholl, and M. Steiner (2003): Empirical Evaluation of Preference Elicitation Techniques from Marketing and Decision Analysis, Friedrich-Schiller-Universität Jena, Jena.
• Helm, R., A. Scholl, L. Manthey, and M. Steiner (2004): Measuring Customer Preferences in New Product Development: Comparing Compositional and Decompositional Methods, International Journal of Product Development, 1 (1), 12–29. Crossref
• Helm, R. and M. Steiner (2008): Präferenzmessung: Methodengestützte Entwicklung zielgruppenspezifischer Produktinnovationen, Stuttgart: Kohlhammer.
• Herstatt, C. and E. von Hippel (1992): From Experience: Developing New Product Concepts via the Lead User Method: A Case Study in a “Low-Tech” Field, Journal of Product Innovation Management, 9 (3), 213–221. Crossref
• Hill, C. W. L., G. R. Jones, and M. A. Schilling (2014): Strategic Management: An Integrated Approach, Stamford, CT: Cengage Learning, 11 ed.
• Hillig, T. (2006): Verfahrensvarianten der Conjoint-Analyse zur Prognose von Kaufentscheidungen: Eine Monte-Carlo-Simulation, Wiesbaden: Deutscher Universitäts-Verlag.
• Hoeffler, S. (2003): Measuring Preferences for Really New Products, Journal of Marketing Research, 40 (4), 406–420. Crossref
• Hoeffler, S. and D. Ariely (1999): Constructing Stable Preferences: A Look Into Dimensions of Experience and Their Impact on Preference Stability, Journal of Consumer Psychology, 8 (2), 113–139. Crossref
• Holbrook, M. B. and W. L. Moore (1981): Feature Interactions in Consumer Judgments of Verbal versus Pictorial Presentations, Journal of Consumer Research, 8 (1), 103–113. Crossref
• Jaeger, S. R., D. Hedderley, and H. J. H. MacFie (2001): Methodological Issues in Con- joint Analysis: A Case Study, European Journal of Marketing, 35 (11-12), 1217–1239. Crossref
• Kim, C., C. Lee, M. R. Lehto, and M. H. Yun (2011): Evaluation of Customer Impressions Using Virtual Prototypes in the Internet Environment, International Journal of Industrial Ergonomics, 41 (2), 118–127. Crossref
• Mayer, R. E. (2005): Cognitive Theory of Multimedia Learning, in The Cambridge Handbook of Multimedia Learning, ed. by R. E. Mayer, Cambridge: Cambridge University Press, 31–48. Crossref
• McDaniel, S. W., P. Verille, and C. S. Madden (1985): The Threats to Marketing Re- search: An Empirical Reappraisal, Journal of Marketing Research, 22 (1), 74–80. Crossref
• McLaughlin, M. J. and K. L. Sainani (2014): Bonferroni, Holm, and Hochberg Corrections: Fun Names, Serious Changes to P Values, PM&R, 6 (6), 544–546. Crossref
• Meffert, H., C. Burmann, and M. Kirchgeorg (2012): Marketing: Grundlagen Marktorientierter Unternehmensführung, Wiesbaden: Gabler, 11 ed.
• Narver, J. C., S. F. Slater, and D. L. MacLachlan (2004): Responsive and Proactive Market Orientation and New–Product Success, Journal of Product Innovation Management, 21 (5), 334–347. Crossref
• Ogawa, S. and F. T. Piller (2006): Reducing the Risks of New Product Development, MIT Sloan Management Review, 47 (2), 65.
• Orme, B. K. (2006): Getting Started With Conjoint Analysis: Strategies for Product Design and Pricing Research, Madison, WI: Research Publishers LLC, 2 ed.
• Paivio, A. (2013): Imagery and Verbal Processes, Hillsdale, NJ: Psychology Press.
Porter, M. E. (2011): Competitive Advantage of Nations: Creating and Sustaining Superior Performance, New York, NY: Simon and Schuster.
• Rammer, C., B. Aschhoff, D. Crass, T. Doherr, M. Hud, C. Ko ̈hler, B. Peters, T. Schubert, and F. Schwiebacher (2015): Innovationsverhalten der deutschen Wirtschaft: Indikatorenbericht zur Innovationserhebung 2014, Zentrum für Europäische Wirtschaftsforschung, Mannheim.
• Schmidt, T. (2001): Einsatzfelder für Multimediale Akzeptanzmessungen bei Technischen Innovationen: Theoretischer Erklärungsansatz und Empirische Überprüfung, Münster: Lit.
• Schrage, M. (1993): The Culture(s) of Prototyping, Design Management Journal, 4 (1), 55–65.
• Sheskin, D. J. (2011): Handbook of Parametric and Nonparametric Statistical Procedures, Boca Raton: Chapman & Hall, 5 ed.
• Shocker, A. D. and W. G. Hall (1986): Pretest Market Models: A Critical Evaluation, Journal of Product Innovation Management, 3 (2), 86–107. Crossref
• Sichtmann, C. and S. Stingel (2007): Limit Conjoint Analysis and Vickrey Auction as Methods to Elicit Consumers’ Willingness-to-Pay: An Empirical Comparison, European Journal of Marketing, 41 (11-12), 1359–1374. Crossref
• Söderman, M. (2005): Virtual Reality in Product Evaluations with Potential Customers: An Exploratory Study Comparing Virtual Reality with Conventional Product Representations, Journal of Engineering Design, 16 (3), 311–328. Crossref
• Spiegler, R. (2011): Bounded Rationality and Industrial Organization, New York, NY: Oxford University Press. Crossref
• Srinivasan, V., W. S. Lovejoy, and D. Beach (1997): Integrated Product Design for Marketability and Manufacturing, Journal of Marketing Research, 34 (1), 154–163. Crossref
• Stadie, E. (1998): Medial gestützte Limit-Conjoint-Analyse als Innovationstest für Technologische Basisinnovationen: Eine explorative Analyse, Münster: Lit.
• Steiner, M. (2007): Nachfragerorientierte Präferenzmessung, Wiesbaden: Springer.
Stevens, J. (2007): Intermediate Statistics: A Modern Approach, New York, NY: Lawrence Erlbaum Associates, 3 ed.
• Stevens, S. S. (1946): On the Theory of Scales of Measurement, Science, 103 (2684), 677–680. Crossref
• Toubia, O. (2010): New Product Development, in Handbook of Technology Management, 2 ed., ed. by H. Bidgoli, Hoboken, NJ: Wiley, 953–963.
• Toubia, O., de Jong, Martijn G., D. Stieger, and J. Fu ̈ller (2012): Measuring Consumer Preferences Using Conjoint Poker, Marketing Science, 31 (1), 138–156. Crossref
• Ulwick, A. W. (2002): Turn Customer Input into Innovation, Harvard Business Review, 80 (1), 91–98.
• Urban, G. L., J. R. Hauser, W. J. Qualls, B. D. Weinberg, J. D. Bohlmann, and R. A. Chicos (1997): Information Acceleration: Validation and Lessons from the Field, Journal of Marketing Research, 34 (1), 143–153. Crossref
• van den Hende, E. A. and J. P. L. Schoormans (2012): The Story is as Good as the Real Thing: Early Customer Input on Product Applications of Radically New Technologies, Journal of Product Innovation Management, 29 (4), 655–666. Crossref
• van Kleef, E., van Trijp, Hans C. M., and P. Luning (2005): Consumer Research in the Early Stages of New Product Development: A Critical Review of Methods and Techniques, Food Quality and Preference, 16 (3), 181–201. Crossref
• Wittink, D. R., M. Vriens, and W. Burhenne (1994): Commercial Use of Conjoint Analysis in Europe: Results and Critical Reflections, International Journal of Research in Marketing, 11 (1), 41–52. Crossref
• Ye, J., S. Badiyani, V. Raja, and T. Schlegel (2007): Applications of Virtual Reality in Product Design Evaluation, in Human-Computer Interaction: HCI Applications and Services, ed. by J. A. Jacko, New York, NY: Springer, 1190–1199. Crossref
• Zahn, E., ed. (1995): Handbuch Technologiemanagement, Stuttgart: Schäffer-Poeschel.