Verified by 
In recent years, social psychology and science overall have been undergoing a "credibility revolution" with a grassroots "open science" movement aiming to promote significant changes to the scientific process. These changes were meant to address what some have come to term the "replication/reproducibility crisis," following signs that important findings in psychology and other fields fail to replicate as well as most would have thought they would. This paved the way to the realization that there are potential weaknesses in our scientific process and methods, and that we are therefore in long-overdue need for self-reflection to assess and identify problems and then work together to improve and do better.
This is a positive, constructive, and important process, and it has affected me personally. It has led to significant changes in how I perceive science and the way that I think of and conduct my research. I endorsed the concept of "open science" and decided to try and make it an important aspect of my academic identity.
There are lots of different definitions of "open science" and views regarding what the term means, and figuring this out together as a community is still very much a work in process. Over the last three years, I have been trying to figure out what being open means to me, what challenges it will help me overcome, and how to best implement this my own research.
Credibility Revolution: Core Insights
For me, there were several important core insights:
- We need to open up and share everything. Yes, everything.
- We need to work together. Science is a collaborative endeavor, there is no me and you, us and them. In science, there is just us, all of us, and we are all in this together.
- We need to constantly assess and re-examine our research practices, to identify potential weaknesses, and then to work together to build and offer solutions to improve.
I then had to define, for myself, where to start. I sat down and thought: what were the most important principles to start from for me in my own research.
Open Science Principles: My Initial Take
What I ended up with as an initial, simplified list were the following basic principles:
- Credibility/trustworthiness
- Reproducibility
- Replicability
As simple and straightforward as these may seem, all these concepts are vague and controversial in their meaning. Some might argue that these are the basic tenants of conducting science. I wish they were. From what I've seen in academia up to that point, I felt like these tenants were not always prioritized, if they existed at all. Once I identified these principles, I set out to try and make sure I fully understand what they meant and how I could integrate those in my own research and work.
I then sought to find some criteria of what — in my view — makes research credible. My initial list included the following:
- Simplicity (rather than complex models)
- Effect size focus (rather than significance)
- Power (power analyses)
- Phenomenon focus (evidence-based theory)
- Registered reports/pre-registrations
- Findings are findings (including null, assuming methods rigor)
- Replications (direct > conceptual)
- Collaboration and community
Yet, the most important was the following:
Full transparency, including but not limited to:
- Sharing all materials, data, code, in a way that would allow others to easily follow, understand, rerun, and reach the same conclusions.
- Sharing the entire process on everything related to the research, starting from the moment of idea conception, including failures and/or the thinking process.
- Reporting all research decisions (exclusions, conditions, & variables)
I really like Simine Vazire's related summary slide on the indicators of a "credible culture" (as opposed to the "incredible"):
These last few years, I have been trying to live up to these principles, and it has been an uphill battle. Not at all easy, slowly learning, trying step-by-step to implement and improve on those in my research, struggling to overcome pushback.
Main Takeaway: Early Career and Students Are Our Hope
I owe the biggest push in this direction to a single decision I made back in 2017. The decision was that with my students I will only conduct pre-registered replications (and extensions) and pre-registered meta-analyses, emphasizing full transparency. Through my work with my students, I have been able to learn what open-science is to me and then implement many open-science practices.
There's something important about students that enabled this journey. My students came in to work with me as a blank slate, not yet biased by the current research system. I needed to be very careful not to enforce my biased perspective on them and to instead invite them for a collaboration, encouraging them to try new things, ask questions, and challenge me to do better. The students have motivated me to re-examine all that I've learned in my academic training and helped me improve and go far beyond all that I've learned in my Ph.D.
Mass Mobilizing Students to Promote Credibility Revolution
I then decided to scale up, big time. Given a successful proof of concept with my master's thesis students during my postdoc at Maastrict University, in my academic job hunt I was looking for a university that would allow me to take what I've learned and try to implement this full scale in large undergraduate classes as part of the regular curriculum.
Once joining the University of Hong Kong, starting in 2018, I began implementing large-scale mass replications projects with students as early as second-year undergraduates. We set out to conduct, within one-semester, pre-registered replications and extensions from beginning to end, with the end term students' reports aiming to be a journal submission-ready manuscripts.
This is the third year of us running the project, and we have so far completed 57 replications and extensions of classic findings in social psychology/judgment and decision-making, and by the end of the year, we expect to reach 80. Together with a growing team of early-career researchers from around the world, we completed many preprints and several publications in top journals in social psychology.
Beyond these replications projects, together with my students and early-career collaborators, we've built lots of collaborative guides and templates open for anyone to use, inviting others to follow and join us. The students even wrote a collaborative 200+ page book on the crisis, the revolution, and open-science: Taking stock of the credibility revolution: Scientific reform 2011-now.
Would You Like to Know More?
If you'd like to know more, I recently presented my journey to open science as well as the mass replications project (here are the slides):
Join us
If you like this direction and would like to join us, there are many ways to do so. All of my materials are open for you to use and implement in your own courses and research (courses/workshops). All of our guides and templates are collaborative, and if you join and contribute you'll be added as a coauthor. If you're an early career researcher (ECR) who does work in judgment and decision making and endorse open-science, we invite you to join our growing ECR team and the lead author in helping to bring our students' replication and extension work to journal publication (details here).
Feel free to contact me at giladfel@gmail.com to discuss any of this further.
In future posts, I will try and share more insights from our journey to open-science, reflecting on current developments, and making more resources available. Stay tuned.
