Choosing priors for logit choice models

A simulation approach to choosing priors

The results above should convince you that using default priors for all cases is a bad idea. Priors will interact with the specifics of the problem, the scale and covariance of the data, the number of covariates etc. We recommend a simulation-based approach, with one of two objectives.

The first approach we recommend is to start by having a prior over the maximum choice probability for a single option, and then back out priors for the \(\beta\) and \(\xi\) that would imply that choice probabilities larger than this prior are possible but not expected. The second approach—which might be more comfortable for those unwilling to take a stand on what they think the maximum choice probability is going to be—is to figure out the priors on \(\beta\) and \(\xi\) that result in a uniform prior probability in the preference space. Note that shares will necessarily not be uniform along each margin— one choice’s gain is another’s loss, and all shares need to add up to 1. But we can stastically test for uniformity in the joint share model by performing a Dirichlet regression on our simulated shares \(s_{i} \sim \text{Dirichlet}(\alpha)\). When all \(\alpha = 1\), then \(s\) is uniformly distributed.

We first recommend scaling the continuous choice attributes in \(X\) to have a mean of 0 and standard deviation of 1. Dummy variables and the intercept column can be left un-scaled. Then, with an attributes matrix \(\hat{X}\) that contains all possible choices:

1. Draw \(\beta,\, \xi\) from their priors. Call these draws \(\hat\beta\) and \(\hat\xi\)
2. Calculate \(\hat\mu = \hat{X}\hat\beta + \hat\xi\) and \(\hat\pi = \text{softmax}(\hat\mu)\), the probability of each choice
3. Save the maximum probability and the distribution of probabilities across choices
4. Repeat 1:3 many times.
5. If using the uniform prior on the outcome space, estimate the concentration vector of a Dirichlet distribution over the simualted shares. Create a loss equal to the sum of squared differences between the estimated \(\alpha\) and 1. Repeat 1-4 varying hyperpriors on \(\beta\) and \(\xi\) until loss is sufficiently small. We illustrate this step below.

For the actual choices of priors for \(\beta\) and \(\xi\):

• For \(\xi\), it is common to assume a normal prior with scale to be estimated, as in Jiang, Manchada and Rossi (2007) and Yang, Chen and Allenby (2003). This also translates nicely to the implementation of Bayesian instrumental variables discussed later in this chapter. Jiang, Manchada and Rossi also show that this is fairly robust to misspecification.
• For \(\beta\), we typically recommend normal priors. These map cleanly to the case later in this chapter where we allow preferences to vary across individuals.

Let’s impliment a prior choice simulation with the aim of choosing priors that result in a uniform prior in the choice probability space.

First, we’ll write a function that implements a Dirichlet regression on a vector of concentration parameters.

library(MCMCpack) # contains dirichlet density function

# Write out the negative log likelihood of a matrix of shares (each row a simplex)
dirichlet_n_log_lik <- function(alpha, share_matrix) {
  if(length(alpha) != ncol(share_matrix)) stop("alpha is not the same
                                               length as the number of choices")
  out <- NULL
  for(i in 1:nrow(share_matrix)) {
    out[i] <- log(ddirichlet(t(share_matrix[i,]), alpha))
  }
  -sum(out)
}

# Implement Dirichlet regression
dirichlet_reg <- function(share_matrix) {
  # This finds values of par to minimize the negative log likelihood function
  out <- optim(par = rep(1, ncol(share_matrix)), 
               fn = dirichlet_n_log_lik, 
               share_matrix = share_matrix,
               method = "L-BFGS-B", lower = 0.001)
  out
}

Next we simulate. For the purpose of this simulation, let’s keep the feature matrix fixed and vary only the prior. For each prior scale, we save the loss, which is the sum of squared differences between the estimates \(\alpha\) and 1. We run the simulation several times with different prior scales. Notice that as \(\xi\) is not identified until we have a supply-side model, we set it to 0 here.

# Fake attribute matrix of 10 choices with 8 attributes, plus
# an outside good
X <- rbind(0, matrix(sample(0:1, 10*8, replace = T), 10, 8))

N_rep <- 100
beta_scale_grid <- c(0.1, 0.2, 0.5, 0.75, 1, 1.2, 1.5, 2)
loss <- rep(NA, 8)

count <- 0
for(sig in beta_scale_grid) {
  count <- count + 1
  out <- matrix(NA, N_rep, 11)
  
  for(i in 1:N_rep) {
    beta <- rnorm(8, 0, sig)
    out[i,] <- t(softmax(X %*% beta))
  }
  
  reg <- dirichlet_reg(out)
  
  loss[count] <- sum((1 - reg$par)^2)
}

Now that we’ve run the simulation, we can see which prior scale resulted in the most uniform prior probability— this will be the smallest value of the loss function.

This indicates that for this particular problem, a prior for the preferences \(\beta \sim \text{Normal}(0, .75)\) defines a fairly uniform prior over the choice shares.