I talked to Karen yesterday about automating the search for papers from Scopus database. I investigated the issue and found it easy to automate the process.
Scopus implements an API (https://dev.elsevier.com/sc_apis.html). I created an API token and managed to write a simple Python program to get the papers’ metadata through the API. In the search through the API, it is possible to define the country of affiliation of the authors of the papers that will be returned in the search.
For testing, I implemented a search for papers using the “citizen science” term and which have least one researcher from the countries “Brazil”, “Argentina”, “Colombia” or “Chile”. It worked well. The results for this search are 113 papers. The result is in the this file (https://github.com/lesandrop/papercollection-ricap/blob/master/scopusPapers.txt). In that file, each line is one paper. For each paper, I saved the following information separated by a semicolon (";"):
- Paper title
- Name of Journal or Conference
- Paper DOI;
- Paper URL
- Authors Affiliation
- Publication Date
- Searched KeyWord
- Searched Authors Countries
So, in this way, at Scopus we are able to collect the papers written by researchers who have an affiliation in the Iberoamerican region. For each search string, we would do a search for each country in the region, then aggregate the results. We can write code that collects data from papers, remove duplicates (this happens if in the same article there is a researcher from Brazil and another from Colombia, for example), categorize the papers by country of researchers, categorize by year of publication, categorize by journal, etc. If you want you can say what it would be good to do, I see if it is possible. Here is a sample of what the API allows us to collect http://alm.plos.org/docs/scopus
My Python code is available here https://github.com/lesandrop/papercollection-ricap. If you want you can use and modify at will.
Now, I will try to do the same for the Web of Science database.