"Chocolate makes you smarter!"—Headlines promise miracles and threaten disasters, but what does the actual science say?
In this interactive workshop, you'll become a science detective. You'll evaluate real-world research scenarios using the same criteria professional scientists use to judge quality.
🎯 Your mission
Dual challenge: For each study, complete both steps in this order:
- Predict the crowd — Guess how the average participant will rate it (1–7 scale)
- Judge the science — Give your own quality rating using expert criteria (1–7 scale)
Scoring: Points awarded based on how accurately your crowd prediction matches the actual average. Can you think like a scientist AND anticipate how others perceive research?
Why both? This dual approach lets you express your own beliefs about research quality while reflecting on how those beliefs might differ from others' perspectives.
📚 What You'll Learn:
- ✅ Spot red flags in research (predatory journals, hidden data, clickbait)
- ✅ Distinguish good science from marketing disguised as research
- ✅ Understand why open access and transparency matter
- ✅ Evaluate methodology, sample sizes, and conclusions critically
💡 First, you'll learn the tools. Then, you'll test your skills on real scenarios!
🏆 Leaderboard & Anonymous Usernames
Your results contribute to the public leaderboard as you go. Complete the workshop (or finish early after rating at least 10 papers) to review and compare your ratings with the crowd.
- You'll be assigned a unique anonymous username (e.g., "Red Fox", "Wise Owl") automatically.
- Your username will be displayed on the results page at the end.
- Rankings are based on how accurately you predict the crowd's average ratings.
- Earn badges (🏆🥇🥈🥉) based on your prediction accuracy score.
You will be able to access the live dashboard once you have rated at least 10 papers.
The more papers you rate, the more robust and useful your contribution — so we encourage you to complete as many as possible!
📊 Data Use Notice
By participating in this task, you agree that your anonymous data will be used for research purposes and may be shared and analysed.
- Your data is identified only by an automatically assigned anonymous username.
- Your username will be displayed on the results page at the end.
- You need not share this username with anyone if you do not wish to.
- No demographic details or personally identifiable information is collected.
All data is fully anonymised and used solely for understanding how people evaluate research quality.
Author: Dr Pablo Bernabeu, Department of Education, University of Oxford
Legal disclaimer: This app and workshop were created by Dr Pablo Bernabeu in a personal capacity during spare time. The employer is not affiliated with, does not endorse, and bears no liability for this app or workshop.
Materials: github.com/pablobernabeu/Unlock_the_Lab
Licence: CC BY 4.0 — Free to use with attribution
💡 Contributions welcome! Suggest improvements or report issues on GitHub.