A poll for consolidating expert views on an issue using ratings and arguments.
Pohl C 2020. Delphi poll. td-net toolbox profile (14). Swiss Academies of Arts and Sciences: td-net toolbox for co-producing knowledge. www.transdisciplinarity.ch/toolbox. doi.org/10.5281/zenodo.3716976
What is a delphi method?
The delphi method is a polling procedure among experts that combines surveying with conferencing. Experts are asked to answer a question on a particular issue and to rate the answers of other experts while substantiating their rating. The poll includes several rounds. Based on the results of the previous round, the experts adapt their rating in reaction to the ratings and arguments provided by the colleagues.
Why should it be applied?
The interesting feature of a delphi poll is that it provides a group's view on an issue that is based on arguments and consolidated over several rounds of reconsidering earlier ratings and arguments.
|When should it be applied?||A delphi is useful early in the project for problem framing, when the issue is explored and framed and a first analysis of the problem is required.|
|How does it work?||
After compiling a group of experts, the delphi method goes roughly through the following steps:
1) The facilitator asks an open question to explore the issue under discussion (without providing any answers).
3) The facilitator reconsiders this first rating and explores dissent. For that purpose, he or she calculates the group’s rating (e.g. average, min, max) per answer and summarises pro- and counter-arguments. The experts are then asked to reconsider their first rating given the group's rating and considering pros and cons.
4) The facilitator recalculates the group’s overall rating. He or she summarises the results of the delphi, usually showing mean values and the distribution of the experts’ ratings. The results also include the pros and cons.
|How are thought styles bridged?||Thought styles are bridged through reconsidering one’s appraisal based on the views of other experts. In doing so, the experts take into account how and why other experts come to a different or the same rating.|
|What’s the output/outcome?||A delphi shows the substance and degree of consensus and dissent of a group of experts on the issue under consideration. The results are usually shown in form of the mean values with the distribution of the experts’ rating plus the pro and cons for the ratings.|
|Who participates in what role?||
A facilitator runs a delphi poll. He or she should have a basic understanding of designing and quantitatively analysing questionnaires as well as of qualitative content analysis.
Experts participate by answering the poll and by reading intermediary results. All kinds of experts can be asked to participate.
|What do I need to prepare?||
A delphi is time-consuming for both, the facilitators who run the study and to a lesser extent for the participating experts. Experts should thus be informed beforehand that they have to run through several rounds and to substantiate their ratings. The workload for the facilitators rises with the amount of open questions and qualitative content analysis.The facilitator has to be experienced in developing questionnaires and in quantitative and qualitative analysis of polls. He or she has to identify the issue, prepare the questionnaire, and find the relevant experts.
|When not to use the method?||Often the delphi method is used to find consensus. This is not adequate. A delphi provides clarity about consensus and dissent and displays the pro- and counter-arguments behind them.|
The Delphi method is well-established and well-documented, cf:
To learn more about the original idea of Delphi we found particularly useful:
Linstone H A, Turoff M 1975. The Delphi Method. Techniques and Applications. Reading: Addison-Wesley. pp 5-6.
For a more detailed description and examples you can also see
McDonald D, Bammer G, Deane P 2009. Research Integration Using Dialogue Methods. Canberra: ANU E-Press. pp 41-50.
available open access: