Equal Opportunities and Diversity

Recommendations for Reviewers

The recommendations listed here are intended to help identify one’s own unconscious bias, actively counteract it, and thus arrive at the most impartial decisions possible. The recommendations are based on proven guidelines, e.g., from scientific institutions, and on the guidelines of the EU-funded GEECCO project.

The following principles and implementation steps are recommended for assessing scientific achievements:

  • Think about the criteria for assessing scientific excellence. Do women and men actually have the same possibilities for meeting these criteria?
  • Consider the following questions:
    • Are underrepresented candidates subject to different expectations than equally qualified applicants from the majority group?
    • Is research by members of underrepresented groups undervalued?
    • Is research by smaller research institutions undervalued?
    • Avoid vague formulations and unsubstantiated evaluations
  • Use the Implicit Association Test, which was developed at Harvard University, to check for possible bias in one’s own evaluations
  • Allocate sufficient time resources to evaluate the applications
  • Final, self-critical reflection with regard to possible unconscious bias influencing the evaluation; if necessary, re-evaluate and adapt the appraisal
Further information

The contributions listed below aim to help you reflect on your own position and role in the scientific review process and sensitize yourself to the emergence of unconscious bias. This collection is added to and updated on a regular basis.

 

Videos

Royal Society: Understanding unconscious bias

Short video and material on unconscious bias and diversity. Many research-funding and research-performing organizations throughout Europe use this video in their evaluation and recruiting processes.

Online presentations/slides

Canada Research Chair Program: Unconscious bias training module

“Bias in Peer Review” Mandatory Learning – Bias Module (duration: at your own pace – approx. 30 min): An interactive training module in English for reviewers that promotes their understanding of unconscious bias. The video shows how this can influence the peer review process. It also presents strategies that help reduce bias during the review process. English subtitles and a full text version are available.

Test

Implicit Association Test (IAT)

The Implicit Association Test (IAT) is a socio-psychological measurement method that gages the strength of associations between individual elements. The IAT is mainly used to measure implicit attitudes and bias toward certain groups or with regard to background, gender, sexuality, age, and religion.

Sources

Artiles Viera, M., Locane, M., Pépin, A., Willis-Mazzichi, V. (2017). Implicit Gender Biases during Evaluations: How to Raise Awareness and Change Attitudes (report from the workshop)

Dvořáčková, J. (2020). Promoting gender equality in the evaluation process: Guideline for jury members, reviewers and research funding organizations’ employees. Available at: GEECCO_Guideline_for_jury_members__
reviewers_and_research_funding_organizations__employees.pdf (geecco-project.eu)

Fine, E., and Handelsman, J. (2006). Reviewing Applicants: Research on Bias and Assumptions

Government of Canada. Bias in Peer Review – A Training Module. Available at: www.chairs-chaires.gc.ca/program-programme/equity-equite/bias/module-eng.pdf

Gvozdanović, J., K. Maes., Implicit Bias in Academia: A Challenge to the Meritocratic Principle and to Women’s Careers – And What to Do About It. ADVICE PAPER no.23. League of European Research Universities (2018)

Huang, J., Gates, A. J., Sinatra, R. A., Barabási, L. (2020). “Historical Comparison of Gender Inequality in Scientific Careers across Countries and Disciplines.” Proceedings of the National Academy of Sciences of the United States of America 117(9): 4609–4616

Kaatz, A., Guerrez, B., Carnes, M. (2014). “Threats to Objectivity in Peer Review: The Case of Gender.” Trends in Pharmacological Sciences 35(8): 371–373

Martell, R. F. (1991). “Sex Bias at Work: The Effects of Attentional and Memory Demands on Performance Ratings of Men and Women.” Journal of Applied Social Psychology 21(23): 1939–1960

Mihaljević-Brandt, H., Santamaría, L., Tullney, M. (2016). “The Effect of Gender in the Publication Patterns in Mathematics.” PLoS One 11(10): e0165367. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5079651

Moss-Racusin, C. A., Dovidio, J. F., Brescoll, V. L., Graham, M. J., Handelsman, J. (2012). “Science Faculty’s Subtle Gender Biases Favor Male Students”. Proceedings of the National Academy of Sciences of the United States of America 109 (41): 16474–9

Nygaard, L. P., Bahgat, K. (2018). “What’s in a number? How (and why) measuring research productivity in different ways changes the gender gap.” Journal of English for Academic Purposes 32: 67–79

Paludi, M. A. and Bauer, W. D. (1983) “Goldberg Revisited: What’s in an Author’s Name”, Sex Roles: A Journal of Research, 9 (1983) pp. 387–390

Steinpreis, R. E., Anders, K. A., Ritzke, D., “The Impact of Gender on the Review of the Curricula Vitae of Job Applicants and Tenure Candidates: A National Empirical Study.” Sex Roles 41(7–8): 509–528 (1999)

The Royal Society (2015). Unconscious Bias Briefing. Available at: RS unconscious-bias-briefing-2015.pdf (st-andrews.ac.uk)

Van Veelen, R. & Belle, D., Academics as Superheroes: Female Academics’ Lack of Fit with the Agentic Stereotype of Success Limits Their Career Advancement (2020)

Wennerås, C. & Wold, A., Nepotism and sexism in peer-review. Nature 387, 341–343 (1997)

Equal Opportunities and Diversity in Research Funding unit

Marlene Hock MSc
Unit Head
marlene.hock(at)fwf.ac.at
+43 676 83487 8610

Dr. Sabine Haubenwallner
sabine.haubenwallner(at)fwf.ac.at
+43 676 83487 8603