Putting the Search in Research

Putting the Search in Research#

This chapter has tips for engaging with the vast amount of economic (and other) literature out there.

Searching and connecting the literature#

General tips for using search engines#

Let’s start with the basics: how to write good queries for search engines. You don’t need to remember much from this section to improve your ability to filter search results to what you’re actually looking for. While these tips are written with Google in mind, many of them will work with other good search engines.

Tips:

Quotes. Put quotes around search terms to let you search exactly for that word or set of words: all results will have those terms in it. Example: “Dynamic stochastic general equilibrium”.
Dashes, ‘-’. Use these to exclude words from your search. Example: ‘fruit -oranges’.
Tilde, ‘~’. Use tilde when you want synonyms to appear in the result. Example: ‘time series ~courses’ to get lessons and other variations on ‘courses’.
Search a single site using ‘site:’. This is an especially useful one. Example ‘site:https://aeturrell.github.io/coding-for-economists regression’.
Vertical bar, ‘|’. Acts like the ‘or’ operator. Example: ‘forecasting | nowcasting’.
Two Periods, ‘..’. This creates a range from the number before to the number after. Example: ‘recessions 1986..2010’.
Location, ‘location:’. Find results related to a particular location. Example: ‘recessions location:unitedkingdom’.
File type, ‘filetype:’. This looks for files of a specific type. Example: ‘synthetic differences in differences filetype:pdf’.

Google Scholar#

Of course, vanilla search comes up with many irrelevant links if you’re looking for research. Google Scholar solves this problem by only returning papers and working papers. It indexes the full text or metadata of scholarly literature across an array of publishing formats and disciplines—so you can find pretty much any research that’s out there digitally. (The flip side of this is that if your institution’s research is not indexed by Google Scholar, it’s close to unfindable online.) Google Scholar is the most important free search engine for scholarly works.

You can use the same tricks for Google Scholar queries that apply for Google’s main search engine.

Scholar has a few extras useful for the papers that you find:

a cite button, which includes a bibtex version of the citation
a list of papers that have cited the paper you’re interested in
a button to find related articles

There are two other ways that Google Scholar can be a useful asset: alerts and recommendations.

You can set up alerts to email you when a particular query receives new results or when there is new work by a specific author. You’ll need a profile to do this. It can be a useful way to stay abreast of new work on a topic that you’re interested in.

Two alerts come by default, recommended articles and new articles in my profile. The latter covers papers that Google Scholar thinks you authored. The former recommends papers depending the topics of your articles, the journals that you publish in, the authors you work with and cite, and the authors that work in the same topics as you.

Semantic Scholar#

Google Scholar isn’t the only free scholarly search engine in town. Semantic Scholar provides free, AI-driven search and discovery tools, and open resources for the global research community. It indexes over 200 million academic papers sourced from publisher partnerships, data providers, and web crawls. Note that Semantic Scholar makes use of a publicly available corpus.

Semantic Scholar also comes with useful extras:

a cite button, which includes a bibtex version of the citation
a list of papers that have cited the paper you’re interested in
a way to find related articles (click through to the paper)
a list of papers that are referenced in the paper of interest

The RePEc Family of Resources#

First, RePEc (Research Papers in Economics) is an initiative that seeks to enhance the dissemination of research in Economics and related areas. It’s another scholarly database, with various services built on top.

EconPapers and IDEAS#

EconPapers and IDEAS include everything that is indexed in RePEc and have extensive search and browsing functions.

NEP, or New Economics Papers#

NEP is a collection of topical mailing lists and RSS feeds for new working papers. You can find a full list of both the email subscriptions and RSS feeds here.

RePEc Biblio#

RePEc Biblio is a hand-selected collection of the most relevant articles and papers on a wide variety of economics topics. RePEc Biblio is organised as a tree, narrowing the topics as you follow its branches. If search is about finding a needle in a haystack, RePEc Biblio gives you a way to search smaller haystacks. It’s a curated bibliography driven by volunteer editors that, unlike Wikipedia, has a named person in charge of each topic.

Consensus#

Consensus is a (free for now, it’s in beta) search engine that uses AI to instantly extract, aggregate, and distill findings directly from scientific research. It’s early days, but it’s shaping up to be a very powerful tool.

You use Consensus by asking a question, for example “what are the main effects of micro credit?”. What you get are lines and paragraphs drawn from papers that get at the given question. We’ve been quite impressed by the results.

Talk to Books#

Not everything you need is in journal articles! Google provides a service called Talk to Books that allows you to ask questions of books and get reasonable-ish answers. But, if you use this, do remember to use the filters to remove fiction!

Some other services#

These other services have some similarities to those we’ve seen already. It feels like there’s a lot of evolution and churn in large database and machine learning powered searching-the-research tools that will likely see some consolidation and stability in the medium term but, right now, it’s hard to say which of these tools will persist and become the most helpful.

Connected Papers is a paid service that displays network diagrams of papers that are connected by citations (inward or outward). With a free account, you can make a small number of graphs per month.
Scite is a paid service that pulls related citations and the context in which the paper was cited. It then sorts those citations of the paper you’re interested in based on whether the context was just a mention, an expression of support, and or a contrasting statement.
Scholarcy is an online article summariser tool that uses machine learning to pull out the key points of scientific articles. There’s a free browser extension but if you want to build up a library, it costs.
Elicit uses language models to help with research. It claims to be able to find relevant papers, summarise takeaways them, and extract key information.
lens “serves global patent and scholarly knowledge as a public good to inform science and technology enabled problem solving.” No account required. There are paid versions for commercial users, but it looks like the core functionality is available for free.
researchrabbit requires a sign-up. Its tag line is “We’re rethinking everything: literature search, alerts, and more”. It offers visualisation of a knowledge graph of papers. A sign up is required to start using it.
Litmaps is an online platform for visualising, expanding, and sharing collections of research. You can find an example here.

Summaries and Reviews#

When you want to get on top of a whole literature quickly, reviews and summaries are your friend. Here are some resources that can aid you.

What is already known about the effectiveness of policies#

Policy Impacts is dedicated to improving the quality of government decision-making by promoting standardisation in policy analysis. The goal is to help policymakers and practitioners better understand and compare the long-term costs and benefits of a wide range of policies by providing a standardised database of MVPF—Marginal Value of Public Funds—estimates. If you’re starting a new research project on a policy intervention, it’s quite useful to check this database to see what the closest state of the art estimate suggests the benefit is. Note that MVPF is computed as the ratio of benefits to net government cost.

Finding review articles#

Review articles are a great way to get on top of a topic quickly, but finding those can be as tricky as finding the original sources! Here are some tips to locate review articles on a topic.

If you’re using a scholarly search engine such as Google Scholar, you can use prompts to retrieve content that’s more likely to be a review. For example, if you were interested in literature reviews on the gender pay gap, you could try:

‘gender pay gap intitle:review OR intitle:meta-analysis OR intitle:”a survey”’

Some popular scholarly search engines have specific filters for this type of content. In Web of Science, Scopus, and Semantic Scholar, choose “Review Article”, “Document type — Review“, and “Publication Type — Review or Meta Analysis” respectively.

Some journals specifically focus on economics reviews. The Journal of Economic Literature (JEL) is designed to help economists keep abreast of and synthesise the vast flow of literature. If you want something a bit lighter or only need a general understanding than the Journal of Economic Perspectives (JEP) fills the gap between the general interest press and academic journals.

Of course, if you work across disciplines, you will often find that other fields have their own review journals too.

Sadly, there isn’t nearly enough content that bridges the gap between primary research papers and people who just want to get the gist of the state of the art in a particular topic. As the amount of literature on every topic explodes, we will need more content that aggregates and summarises it in order to keep up—and to inform policymakers. The brilliant New Things Under the Sun, which summarises the state of our knowledge on innovation, is a great example of the kind of content that’s needed.