The Impact of Protest Responses in Choice Experiments

No much attention has been given to protest responses in choice experiments (CE). Using follow-up statements, we are able to identify protest responses and compute welfare estimates with and without the inclusion of such protest responses. We conclude that protest responses are fairly common in CE, and their analysis affects the statistical performance of the empirical models. In particular, when the sample is corrected by protests, our results come from utility consistent models. Thus, future choice experiments should consider the role of protest responses as contingent valuation studies have. JEL codes: Q01, Q10, Q50 1 PhD Student “Maria Barbeito”, IDEGA (Universidade de Santiago de Compostela, Spain). Av. das Ciencias s/n. Campus Sur. Santiago de Compostela-Spain. Phone: +34 981 563 100 (ext 14372). Fax number:+34 981 59 99 35 melina.barrio@usc.es 2 (corresponding author). Department of Economic Theory, Universidade de Santiago de Compostela, Spain. Facultade de Ciencias Económicas e Empresariais. Avda. do Burgo s/n Campus Norte. 15782 Santiago de Compostela-Spain. Phone:+34 981563100 (ext 11674) Fax number:+34 981 59 99 35 maria.loureiro@usc.es


The impact of Protest Responses in Choice Experiments
In the last years, the assessment of environmental and recreational values with choice experiments (CE) has increased Boxall et al., 1996;Hanley et al., 1998;Morrison et al, 2002). The CE method is a generalization of the contingent valuation (CV) method, in the sense that rather than asking people to choose between a baseline scenario and a specific alternative, CE ask people to select between cases that are described by attributes . CE share a common theoretical framework with dichotomous-choice contingent valuation in Random Utility Models (RUM) (Luce, 1959;McFadden, 1974), as well as a common basis of empirical analysis with limited dependent variables (Greene, 1997). For these reasons, we look at the treatment of protest responses in CV, aiming to adopt it to CE.
As the literature has shown, if protesting occurs, stated preference methods may fail to determine the correct economic value of the good in question (Meyerhoff and Liebe, 2008). The treatment of protest responses becomes particularly important when the benefit aggregation issue is considered (Halstead et al., 1992), because such protests may provide underestimated welfare measures if all responses are included in the analysis (e.g. Hearne and Santos, 2005;Chuan-Zhong et al., 2004); or else, overestimated results if removal of all the status quo responses of the analysis is done (e.g. Adamowicz et al. 1998 3 ). Therefore, a correct analysis of protest responses seems required. 3 They remove individuals who selected always the current situation and were treated the same as the "I don't know" response in a CV question. 3 Protest responses have been widely debated in CV studies (Strazzera et al., 2003;Jorgensen et al., 1999, among others), showing that the identification and their later treatment may have a significant influence on the welfare estimates. Therefore, problems commonly encountered in CV related to protest responses might also be present in CE, although not much attention has been given to these issues yet in the literature.
In CE, in addition to the different attribute combinations which are associated with some changes in the good or services valued, another option is typically presented to respondents that contains the current situation and a zero payment, denoted as the status quo option (Hearne and Santos, 2005;Mercer and Snook, 2004). Protest responses may hide behind the selection of the status quo options Meyerhoff and Liebe, 2009). Just in the last years, authors such as Liebe (2008, 2009) treat more explicitly the topic of protest responses in CE. Meyerhoff and Liebe (2008) employ a follow up question with CE and CV to differentiate the protest beliefs and responses, and to assess whether the likelihood of protest responses differs across methodologies. They do not find clear differences between protests responses in both methodologies. Meyerhoff and Liebe (2009) analyze the motives to select the status quo alternative. Furthermore, they assess the impact of the alternative specific constant for the status quo into the computation of compensating surplus.
The novelty of the analysis that follows is that it is based on the treatment of protest responses, distinguishing explicitly between protest and non-protest responses based on the selection of the status quo option. In this way, the indirect utility function and the associated welfare estimates are computed per treatment. Therefore, this analysis 4 allows not only for the assessment of the impact of protest responses on the welfare estimates, but also on the estimated parameters of the indirect utility function.
In order to properly account for the effect of protest responses, first, a conservative treatment of protests is employed, treating the protest responses in the analysis as true zero respondents. In a second approach, protest responses are excluded from the empirical analysis, under the assumption that individuals who do not share the valuation scenario should not be taken into account when estimating welfare estimates (Freeman, 1986). As far as we know, this is the first empirical application that explicitly deals with the treatment of protest responses per se in the context of CE, analyzing two ways to identify the protests. At the same time, the identification of protesters follows the steps of the previous works conducted in CV but novel in CE studies. Additionally, secondary objectives are related to the assessment of the sensitivity of welfare estimates when including and excluding protest responses, respectively. These analyses seem necessary due to the propensity to find protest responses in CE.
The rest of the paper is structured as follows: first, we conduct a literature review of previous studies linked to protests responses and their treatment, continuing with the choice experiment model estimation. It follows with the description of the case study area and the survey mechanism. Later, we present and compare the results for the whole sample with the results corrected by protests responses, ending with some conclusions and recommendations based on the obtained results.

Analysis of Protest Responses
Protest respondents are those who oppose or do not approve the survey mechanism and fail to respond the valuation question, either giving positive responses although invalid, 5 or a non-true zero value to a product or service (Halstead et al., 1992). Nevertheless, the first concern of protest responses appears with respect to their identification. There is no protocol or theoretical criterion for classifying responses (Boyle and Bergstrom 1999); however, the classification of all zero bids must be carefully examined to identify the legitimate zero and protest responses. To differentiate between them, previous analyses have used a set of debriefing questions that were presented to those respondents who were unwilling to pay (Meyerhoff and Liebe, 20084 , Loomis et al., 1996, Strazzera et al., 2003. Based on statements as the previously used in the literature, and presented in Table 1, real zero values and protest responses were also identified in this analysis.

Table 1 around here
As we can observe in Table 1, there are differences related to the presented statements aiming to classify individuals, but also with respect to the criteria applied to identify a response as protest. Some authors presented the statements to the full sample (Meyerhoff and Liebe, 2008), trying to distinguish not only protest responses related to zero WTP values, but also general protests beliefs in the entire sample. On the contrary, other studies only presented statements to the individuals who were not willing to pay (Halstead et al., 1992;Loomis et al., 1996). Furthermore, the criteria to be classified between protests and true zero values varied considerably between different authors, as denoted in Table 1, although there are some commonalities across studies. Halstead et al. (1992) present four statements, including reasons for the rejection of the payment vehicle, the concept of paying for the good or the impossibility to afford the payment, 4 They presented these follow-up questions to all individuals in the sample, not only those do not willing to pay. 6 and in addition an open ended question. Along the same line, the rest of the authors include other reasons related to the value of the good, the sense that others should pay for the program, or that they cannot afford the payment. Giraud et al. (2002), Jakobsson andDragun (2001), Loomis et al. (1996), Strazzera et al. (2003)  Once the protesters had been identified, different treatments were applied to the protest responses in the CV literature. Generally, there have been three main ways of dealing with protest zero bids (Halstead et al., 1992). The first consists on eliminating them from the data set (Freeman, 1986;Mitchell and Carson, 1989). The second includes the protest bids in the data set and treats them as legitimate zero bids (Giraud et al., 2002).
The third method assigns protest bidders mean WTP values based upon their sociodemographic characteristics, relative to the rest of the sample. Thus, as the literature shows, both the treatment and identification of protest responses have been quite different across studies.
Even though there are different ways to deal with protest responses, the most common application in CV is to delete them from the sample (see Adamowicz et al., 1998;Morrison, et al., 2000). Strazzera et al. (2003) argue that the rationale for removal of protest zeros is explained by Freeman´s (1986) with the following statement: "The person who refuses to state a monetary value on the grounds that it is unethical to do so or that he has an inherent right to the environmental good must be dropped from the sample when mean bids are calculated. If a person bids zero on the grounds that he had an inherent right to the good, the bid is not an indicator of his true valuation". However 7 Jorgensen and Syme (2000) considered that protest beliefs were representative of attitudes towards the valuation process and argued that censoring of protest responses is unjustified. In the present application, we use CE for the valuation of various managements programs to be applied in a natural protected area.

Choice Experiments and Estimation
Choice experiment methods are consistent with utility maximization and demand theory (Bateman et al., 2002). Respondents are asked to choose between different bundles of (environmental) goods, which are described in terms of their attributes, or characteristics, and the levels that these take.
According to this framework, the individual i has a utility function (U) of the form: This indirect utility function can be described as a sum of two components: a deterministic part (V) and a stochastic part (ε). The first element is a function of the attributes of the different management programs (X) to be valued and the social characteristics (S) of the individuals. β is a vector of parameters to be estimated and α j is another vector of parameters corresponding with the j-th alternative to be selected.
The stochastic element represents unobservable factors on individual choices independent of the deterministic part. 8 A person chooses the alternative k when ik ij u u > for all k ≠ j . Accordingly, with J choices, the probability of choice k is: One of the prevalent models used in the previous literature to model choice behavior has been the multinomial logit. An assumption of this model is that the error term is independently and identically distributed (IID The statistic for this procedure is given by the following equation: where ) β indicates the coefficient vector, ) ∑ denotes the estimated covariance matrix, and f and s respectively the full and reduced choice specifications. This statistic follows a limiting chi-squared distribution with k degrees of freedom, where k is the number of attributes. 9 with 5 degrees of freedom is 11.07). Therefore, the null hypothesis was rejected, indicating an IIA problem.
When a violation of the IIA hypothesis is observed, more complex statistical models are necessary in order to relax the assumptions employed. These include the multinomial probit model (MNP) (Chen and Cosslett, 1998;Hausman and Wise, 1978;Lusk and Schroeder, 2004), the random parameters logit model (Revelt and Train, 1998;Train, 1998;Train, 2003), the nested logit , and the heterogeneous extreme value logit (Allenby and Ginter, 1995;Bhat, 1995;Lusk and Schoroeder, 2004).
The approach that we follow in this analysis is the MNP. The MNP assumes that the error term follows a multivariate normal distribution, with mean 0 and covariance matrix, such that: Hausman and Wise (1978) proposed the structured covariance matrix for this model to consider heterogeneity among individuals. Note that allowing the error variance to differ across alternatives while errors are normally distributed is equivalent to relax the restrictive IIA assumption.
When the errors are correlated, Train (2003) shows that the parameters in ∑ are not identified unless constraints are imposed. These constraints are linked to the fact that neither adding nor dividing a constant to the utility for each alternative will affect the choice that is made according to equation (2). Then, we have to normalize the model to eliminate the irrelevance effects of the base level and scale of utility. To remove the first effect, we use the resulting utility from taking the difference between each alternative's utility and the utility of the base alternative, in this case k. This means that: where j* = j if j < k and j* = j -1 if j > k, so that j*= 1, ... , J-1. Now, we can work with the J-1 For the second effect, we fix the value of one of the variances 6 2 m σ of ∑. Thus, there are a total of at most J J-1 ( ) 2 − 1 identifiable variance-covariance parameters. If each individual is a utility maximizer, the probability that individual i chooses alternative k from a choice set to any alternative J, can be expressed as: More specifically the probabilities are written as: 6 We fit the model using STATA 10. By default, this program fixes one of the variances where ( ) f ⋅ is the probability density function of the multivariate normal distribution.

Data
In this study, we analyze different management alternatives in the Spanish Biosphere between protest and non-protest, but also because one of the options must always be in the respondent's currently feasible choice set in order to be able to interpret the results in standard welfare economic terms (Hanley et al., 2001). In choice modeling, most researchers have included the alternative "do nothing" or status quo (Adamowicz et al., 1994;Adamowicz et al., 1997;Hearne andSantos, 2005, Blamey et al., 2000), although others have not (Holmes et al., 1998, Mackenzie, 1993.    Table 3, using the most common statements from previous studies and an open-ended question recommended by some authors (Bateman et al., 2002).  Table 3, we have identified different answers, which are displayed in Table 4. Using these results, we have classified as "protest responses" those individuals who did not like the actions presented, were not willing to pay more taxes, or who considered that they should not have to pay for this type of program. In order to investigate the importance of these protest responses, three classifications were attempted. On one hand, protest responses were treated as zero respondents, and included into the dataset. Secondly, protest responses were differentiated via the presented statements at each choice occasion, and excluded from the sample; while in the third treatment, individuals providing any of the protest reasons in any of the choice occasions were excluded from the sample.
Therefore, we have estimated three alternative models, one with the full sample with the protest responses treated as true zeros; a second with protests classified by each choice sets and excluded, and a third one, where in order to avoid inconstancies, responses coming from the same individual were classified all as protests or not.
14 Table 4 around here

Results
In total, 453 surveys were collected with an overall response rate of 40.27%. Each individual responded to six choice occasions that amount to a total of 2718 observations. Surveys were conducted inside and outside the protected area, from a sample of the general population. Table 5 summarizes the socio-demographic characteristics of the sample.

Table 5 around here
The empirical representation of the utility function has the following functional form: where a α and b α represent respectively the specific constants for selecting option A and B respectively, with respect to the status quo (choice C). Table 6 presents the results from this baseline model estimated with the sample. In this model, all the attributes, except river, are statistically significant. The attributes forest, wolf and patrimony have a positive sign, while the coefficient corresponding with the required tax payment carries a negative one, as expected. This implies that the presence of the former attributes increases utility, while the latter attribute decreases utility in a statistically significant way.

Results with and without protest responses
Two additional MNP models for the corrected samples were estimated (table 7)  individual chooses an alternative in which some actions are carried out, her/his utility increases with respect to the status quo option. In terms of statistical fit, the corrected models have also improved notably, minimizing the Akaike Information Criterion (AIC) and the Bayesian information criterion (BIC).

18
WTP estimates are computed with the formula in (9), while asymptotic standard errors were obtained via the delta method for each attribute (table 8). The mean WTP for each attribute was estimated as the ratio of the coefficient associated with the attribute of interest over the Tax coefficient (see Hanemann and Kanninen 1999) 7 . Each of these ratios is understood as a price change associated with a unit increase in a given attribute:

Conclusions
In this research, we investigate the effects of protest responses in the results of a CE exercise and the sensitivity of the derived WTP estimates. We estimate models corrected and not corrected by protest responses, as well as an extended model with socio-demographic characteristics by choice alternative. As far as we know, this is one of the few applications using a multinomial probit model for modeling choice behavior.
The protest responses were classified by two rules, one at the individual level, and a second one, considering them at the choice occasion level. The results show some quantitative differences across treatments of protest responses. With respect to the empirical objective at hand, we show the necessity that protest responses are identified in choice experiment, given that the statistical model fit improves considerably, providing more consistent results with the underlying economic theory. When the sample is corrected by protest, the utility of selecting any of the alternatives versus selecting the status quo, increases as expected according to individual's rationality.
Therefore, the corrected models are more consistent with economic theory. In addition, the valuation of some attributes, such as the wolf protection program vary slightly in terms of welfare estimates, denoting that the presence of this attribute in the choice set may trigger some protest responses. This finding makes sense in a geographical area where wolf protection unleashes controversy.
In the context of contingent valuation, Halstead et al. (1992) show that the exclusion of protest responses may bias WTP results, but the direction of such bias is indeterminate a 20 priori. However, the majority of the studies indicate that samples without protest bidders will result in higher WTP estimates (Jakobsson and Dragun 2001 Meyerhoff and Liebe (2008): rated the statements with a five-point scale, from completely disagree (1) to completely agree (5).