Psychology for Real

Are research and reality different worlds?

Posted Jun 22, 2017

J. Krueger
Source: J. Krueger

The superior man understands what is right; the inferior man understands what will sell. ~ Kung Fu-tse

In this blog, I have posted essays from an academic’s point of view and experience. Some posts are strictly about intellectual puzzles, others are reviews of empirical research, and still others are informed by my limited experience in the outside world. In psychological research, the question of relevance is often close to the surface. Yet, at the basic science end of psychology, there is little impact seen beyond getting the work published. Having a paper in print used to be the end of it, along with a general sense of the reputation of the journal. Over time, more and more metrics have sprung up to give us more nuanced insights into how our works is received.

Raw citation counts, as well as citation counts relative to the number of papers published, are receiving ever more attention. The H index, for example (named after someone whose name begins with the letter H), is a metric describing an individual author. If your H index is 10, you have published 10 papers with at least 10 citations each. If one of these papers has garnered 5,000 citations, the H index does not care. Nor does it care if you have published another 100 papers that have been ignored. I suspect we will soon see indices that scale H relative to a reference class. An H of 10 is high in the humanities but not in engineering. With social comparison being the name of the game in a competitive culture, researchers and the people who pay them will want to know the researcher’s rank in the field.

There are also paper-based metrics that allow anyone to see a paper’s reception in quantitative terms. After publishing with Frontiers in Psychology (click here and give us a hit), I had occasion to marvel at a whole suite of statistics associated with this particular publication. Hits (or views) are graphed over time, numbers of tweets—along with a world map locating the origins of these tweets—are shown, as well as various ranks of the paper relative to different reference classes (e.g., papers on a ‘similar’ topic; papers in the same journal; papers published at about the same time). ‘Too much information!’ you might think, and I agree. The significance of the work is being reduced to a popularity contest. Popularity is quantifiable, whereas “true significance” is elusive. It is easy to complain that the true worth of one’s work is greater than hit or citation frequencies suggest, but how do we know who reads the work, comprehends it, and acts on it?

It is a difficult and even unpleasant task to prepare graduate students for this social-economic reality that lies beyond the scientific work. The students know that they will need jobs and that the best way to prepare is to publish good work in good journals. The metrics will then unfold in a way that is hardly controllable. So the training effort remains focused on the classic skills and virtues that made the academic enterprise great. Think deeply and read widely, acquire laboratory and quantitative skills, find a tractable problem, solve it and publish. A lot of things have to go right for this to come to pass. It is hard to accept the view that all the work might still come to nothing if the publications do not become popular.

The academic world as I have described it here is rather different from the typical for-profit business environment, which is characterized by the dominance of the bottom line, a (relatively) short time horizon, chains of command, lack of individual ownership of a project, and other what-have-yous. Nowadays, young Ph.D.s transition from a research-for-truth to a research-for-effect-and-revenue environment. In particular, the notorious “bottom line” dictates the fortunes of those who contribute to it, or fail to. According to the hard-shell view, this should be so. Any departure from this dictate would be socialism or stupidity or both. An academic's transition into this world can create a culture shock. The person has to grasp and internalize a host of values and assumptions and do so quickly (short time horizon). At first the money—which tends to be [much] better than in academia—will help, but ultimately, a deeper psychological transformation is necessary.

My friend Anna Hartley, who received a Ph.D. from Brown University, now works for Amazon in Seattle, where she uses many of her analytical skills on her new job. In the essay below, Anna describes some of her experiences in the high-tech, big-data environment and how it contrasts with her academic roots.

A Research Psychologist in the Tech World

I’ve been working in the tech industry for a little over a year now. I switched from academia—a postdoc in moral psychology, Ph.D. in social-personality psychology, a [humble] list of publications—to working as a research scientist at Amazon where I examine company-wide data and research how to enhance employees’ work experiences. Other than using Microsoft Outlook to book meetings and numerous hard skills (e.g., SQL, dealing with big data), I’ve learned several key things over the course of this year.

1. Business jargon. Jargon is a thing that exists in any field: academic, culinary, media, business, or otherwise. It is essential. When I talk with a client, it’s important to use the jargon he or she uses so that we can relate to one another. Embracing the jargon conveys that I’m not an academic in my ivory tower but a business partner.

Still, jargon can be initially confusing. My first week, I kept seeing “EOD,” “COB,” or “EOB” in my emails. I did not yet know that this means “end of day,” or “close of business.” I personally translate “close of business” to mean getting it done before I go home, watch Netflix for three hours, eat s’mores for dinner, then fall asleep. But that’s just me.

Movement. “Any movement on that validation project?” This is a nice way of saying “I’ve been waiting on this for two weeks now. Did you get it done, or what?”

Call out. For example, “I just want to call out that this doesn’t necessarily fit with our business culture,” or “Thanks for the call out, Mark. I didn’t realize that project had already been done before.” “Call out” is business jargon for telling someone they’re probably wrong. That sounds harsh, but it’s nice to be part of a company that thrives on open feedback.

Other jargon I hear often, but prefer not to use myself. For example, thought leader, reinvention of science, and synergy. Please, please don’t say synergy or synergies.

Sometimes there are abbreviations that no one really understands, but use all the time. My first few months at my company, I kept on hearing the term “OLR”.

Team member: “We’ve got to get this project done before OLR though.”

Me: [nodding head in agreement] “Definitely.”

Finally, in a meeting six months into working at the company, I said, “Just so you know, when people say ‘OLR’, I don’t know what that means.” Turns out it stood for Organizational Leadership Review. After the team had a hearty laugh, a coworker who had been working at the company for two years messaged me and said, “Thanks. I never knew what that stood for either.”

2. Perspective taking. Taking other people’s perspectives is crucial when working with different business leaders with diverse job backgrounds. It is the difference between the academic who succeeds in industry and the one who fails.

Perspective taking means understanding that everyone has a boss to answer to and their own demands they have to meet. Sometimes people you work with may push you to finish a report sooner than is possible, or message you with “quick questions” (spoiler alert: there are no quick questions). Sometimes the research you’ve been working on may not be used or even considered because the business didn’t need it or find it relevant.

It could be easy to get upset or be uncompromising in these situations, but it is better to try to understand where that person is coming from, and then react. This may mean pushing back politely, but other times it will simply mean taking one for the team and doing that project quicker or letting that research project go.

The inability to take other people’s perspectives in industry can lead to unnecessary resentment and negativity. Those attitudes in turn affect your relationship with those people, and you may be seen as stubborn and uncompromising. Although these types of reputations matter less in academia (read: stereotype of the old, cranky, yet genius professor), they are crucial to maintain in business. How you are seen will impact who wants to work with you, what roles you can advance to, and whether people are influenced by your ideas. Being socially adaptable matters.

3. A dose of humility. Being a subject matter expert in your research area doesn’t translate to being a subject matter expert in business. Sometimes other people will have better insight into your research than you will. As a research scientist at a large company, my job is to give the business the information they need to run more efficiently. This means partnering with business leaders with varied backgrounds (MBAs, engineers, and HR leaders), sharing knowledge to understand research findings, and determining the best course of action for the business. This also means that the research is never truly “mine” like it might be academia. It’s the business’s, and everyone has a stake in it—it is truly collaborative.

Sometimes I produce a research finding that business leaders are truly wowed by and wanted to learn more about. Other times I’ve produced research that I was truly proud of but was told was not surprising or useful. Just because I have a Ph.D. and 10+ years of research experience does not always mean that I know best. Also, for what it’s worth: Many people I’ve worked with at my company are truly impressed by my degree, research experience, and publications… but others don’t care. And that’s fine. At the end of the day, it is your skills, relevant experience (research or business), and ability to get things done that matter—not your degree. Which is actually quite democratic and refreshing.

4. Logistics matter. A few years ago, Netflix held a competition for $1 million for anyone to design a better recommendation system than their existing one (the Netflix feature that recommends “movies you may like”). The winning prize went to a team of Austrian researchers, but was never adopted by Netflix, despite achieving a 10 percent improvement over Netflix’s existing system. Why? Because implementing the new solution was logistically impractical. The headache of implementing the new system was not going to achieve much more revenue nor be easily adaptable or scalable with all of Netflix’s business growth.

In academic research, it’s fun to design statistical models that take into account all crucial parameters without overfitting (yes, that is my idea of fun). But in industry, logistics matter. I recently explained to a business leader that if we wanted to do the research study she had in mind accounting for all variables of importance, it would take five months. She looked horrified. Just like the Netflix example, in the tech industry you can design a research study that will take into account all variables and have important implications, but if it doesn’t quickly deliver results, then it won’t be useful.

Explaining about trade-offs is useful. I then explained to that business leader that if we were willing to make a few trade-offs (make the study correlational rather than experimental; collect a smaller sample), that we could do the research study in two to three months. Being able to weigh logistics, research design, speed, and rigor is crucial.

I believe these are important lessons for any discipline. But being aware of these four factors is especially crucial for an academic who’s thinking about switching to industry. And by no means is this a complete list—there are far more qualities that make an academic succeed in industry: openness, flexibility, humor, willingness to move with speed, knowing when to set limits on how much work you do in a day, to name a few.

A. Hartley
Source: A. Hartley

About Anna Hartley

Anna Hartley is a Research Scientist at Amazon in Seattle. She received her Ph.D. in psychology from Brown University in 2013, and has conducted research on social perception, personality change, and morality. You can find her at @anselmahartley and www.anselmahartley.com.