Prediction markets on Conjoint.ly are designed for product and concept testing through aggregating the audience’s predictions about future success or failure of product launches, modifications, and other decisions. However, respondents need to be trained to use the interface of predictions in the first instance. This guide covers how to read the outputs of both calibration and prediction questions.
The calibration accuracy score is an overall guide to how reliable the predictions made by participants will be.
The percentage of skipping of calibration questions (i.e., choosing the "I don't know" option) should not exceed 20%: otherwise you are dealing with lazy respondents.
For each calibration question, we calculate the percentage of bets placed on correct responses. Scores over 65% are commonly sufficient to conclude respondents were "correct on average".
The betting summary table contains the number of answers (i.e., respondents) who made a particular choice, the average number of points they bet, and the total number of points bet.
| Number of answers | Average points bet | Total points bet | |
|---|---|---|---|
| Correct answer: Mumbai Indians | 87 | 68 | 5920 |
| Incorrect answer: Rising Pune Supergiants | 24 | 54 | 1285 |
| Skipped question | 5 | N/A | N/A |
| Total | 116 | 62 | 7205 |
The consensus history chart plots the percentage of correct bets against the timeline of responses. Convergence of scores suggests that the summary is firm and that a market consensus has emerged.
For example, in this chart the first several responses were correct. Response 3333185 was wrong and moved consensus to 93% correctness. Response 3370385, on the other hand, was correct and moved consensus from 83 to 84% correctness.
Each prediction will be displayed together with the confidence in prediction (in addition to a betting summary table and a consensus history chart).
In interpreting predictions, we recommend conservatism: If the decision at stake is product launch, only a strong prediction (over 80%) in support of the launch would be enough to support the launch.
For each side of the argument, you will see respondents' rationale for making a particular prediction. Comments that were voted by other respondents as convincing will be displayed on top with a higher conviction score.
| Comment | Conviction score |
|---|---|
| Becasue the Apple store app is very popular | 90% |
| it will be 10 mil. People watch IPL!! | 70% |
| i would download it myself | 60% |
| because most people think so | 30% |
| it think fans will follow IPL. it is a popular code | 30% |