How to Manually Calculate Partworth Utilities


This guide describes how to manually calculate partworth utilities. This replicates the calculation of standard Conjoint.ly outputs for an example Generic Conjoint regarding different mobile plan options.

To replicate the calculation in your report, first export the raw data from your survey. Navigate to the Individual preferences tab. Within this tab you will see raw individual preference scores for each level tested.

Raw data of individual preferences

By design, the first level within each attribute will always have a score of 0, and the other scores are computed relative to this reference.

Relative importance by attribute (Attribute partworths)

Relative importance by attribute (attribute partworths)

To calculate the attribute partworths:

Step 1: Calculate the range of preference within each attribute for each individual. This is defined as the maximum preference value within each attribute, minus the minimum for each individual.

Relative importance by attribute (attribute partworths) Step 1

Step 2: Calculate the importance ratio of each attribute for each individual. This is the range of preference for each attribute, divided by the total sum of all range of preferences for the individual.

Relative importance by attribute (attribute partworths) Step 2

Step 3: Calculate the average importance across respondents, through averaging the importance ratios across all respondents.

Relative importance by attribute (attribute partworths) Step 3

Average preferences for levels (default view)

Relative value by level (level partworths)

To calculate level partworths:

Step 1: Calculate the average preference across individuals for each level, by averaging the scores in each column.

Relative value by level (level partworths)

Step 2: Within each attribute, scale the partworth utility to set 0 as average. This is performed through subtracting the value for each level from the sum across the corresponding attribute.

Relative value by level (level partworths)

Step 3: Scale the partworth utility across each attribute. This is done through dividing the level utility by the overall utility range for all attributes which is the sum of range utility of each level’s average utility across respondents for each attribute.

Relative value by level (level partworths)

Estimated likelihood of being chosen

The Estimated likelihood of being chosen method is only available for experiments with just 1 attribute.

Using the View mode ▼ button, you can toggle the display mode. The default approach, Average preferences for levels, is described above. The default method produces values that are somewhat difficult to interpret. By converting the scores, we are able to present them as a probability of being selected from a random set of choices.

The calculation takes into account the number of choices respondents had on each task, if they were shown 3 alternatives and none, they would have had 4 total options. Your experiment may be different.

The resulting score provides an estimate of how likely the item was to be selected on a random choice task, this value is directly interpretable and comparable across the different levels.

Normally, if we wish to estimate the likelihood of one level being selected among a specific set a 4, we would use the following formula:

$$ P(U_1) = \frac{e^{U_1}}{e^{U_1}+e^{U_2}+e^{U_3}+e^{U_4}} $$

where:

  • Ui is the zero-centred utility score for item i
  • P(Ui) is the probability of item i being selected

It would be too computationally intensive to repeat this calculation for every possible combination of 4 items, so we employ a shortcut. Since the utility scores used in this calculation are zero-centred (that is, the mean score is 0) we can use 0 as the utility value for the other items in the choice, and approximate the probability of a certain item, i, being selected from 4 options as:

$$ P(U_i) = \frac{e^{U_i}}{e^{U_i}+e^{0}+e^{0}+e^{0}} = \frac{e^{U_i}}{e^{U_i}+3} $$

And more generally, for a set of n items in the task as:

$$ P(U_i) = \frac{e^{U_i}}{e^{U_i}+n-1} $$

The steps to recreate the calculation from the raw utility scores found on the Individual preferences tab:

  • Zero-centre and normalise the scores for each person
    • Subtract each score the respondent’s average score
    • Divide each score by the range of scores (max - min)
zero-centring and normalise
  • Apply the above formula, taking into account the number of options respondents would have seen, in this example we assume 4 total options:
apply logit
  • And, finally, force the sum of probabilities to be equal to the number of levels divided by the number of options per task, in this example that is (10/4), an take the average across respondents.
set appropriate sum

Distribution of preferences for levels

2. Distribution of preferences for levels

To calculate the distribution of preferences for each level:

Step 1: Scale the level utility with exponential function.

2. Distribution of preferences for levels

Step 2: Calculate the ratio of level utility within each attribute for each respondent, by dividing the level utility by the overall utility for this attribute.

2. Distribution of preferences for levels

Step 3: Average the ratio of level utility across respondents.

2. Distribution of preferences for levels

Distribution of most preferred levels

2. Distribution of most preferred levels

To calculate the distribution of preferred levels:

Step 1: Identify the most preferred level within each attribute.

2. Distribution of most preferred levels

Step 2: Calculate the percentage of the level is most preferred across respondents (with the average most preferred value across respondents)

2. Distribution of most preferred levels

Next steps