To Open Closed Doors: How open is your reference list?

Our most recent manuscript was almost ready for submission. We were planning to send it to an open access journal. It was then that I had the thought: how many papers in the reference list are freely available?

It somehow didn’t make much sense to point readers towards papers that they might not be able to access. So, I wondered if there was a quick way to determine how papers in my reference list were open access. I asked on twitter and got a number of suggestions:

  1. Search crossref to find out if the journal is in DOAJ (@epentz)
  2. How Open Is It? from Cottage Labs will check a list of DOIs (up to 20) for openness (@emanuil_tolev)
  3. Open access DOI Resolver will perform a similar task (@neurocraig)

I actually used a fourth method (from @biochemistries and @invisiblecomma) which was to use HubMed, although in the end a similar solution can be reached by searching PubMed itself. Whereas the other strategies will work for a range of academic texts, everything in my reference list was from PubMed. So this solution worked well for me. I pulled out the list of Accessions (PMIDs) for my reference list. This was because some papers were old and I did not have their DOIs. The quickest way to do this was to make a new EndNote style that only contained the field Accession and get it to generate a new bibliography from my manuscript. I appended [uid] OR after each one and searched with that term.

What happened?

My paper had 44 references. Of these, 35 were freely available to read. I was actually surprised by how many were available. So, 9 papers were not free to read. As advised, I checked each one to really make sure that the HubMed result was accurate, and it was.

Please note that I’d written the paper without giving this a thought and citing papers as I normally do: the best demonstration of something, the first paper to show something, using primary papers as far as possible.

Seven of the nine I couldn’t compromise on. They’re classic papers from 80s and 90s that are still paywalled but are unique in what they describe. However, two papers were reviews in closed access journals. Now these I could do something about! Especially as I prefer to cite the primary literature anyway. Plus, most reviews are pretty unoriginal in what they cover and an alternative open access version that is fairly recent can easily be found. I’ll probably run this check for future manuscripts and see what it throws up.

Summary

It’s often said that papers are our currency in science. The valuation of this currency comes from citations. Funnily enough, we the authors are in a position to actually do something about this. I don’t think any of us should compromise the science in our manuscripts. However, I think we could all probably pay a bit more attention to the citations that we dish out when writing a paper. Whether this is simply to make sure that what we cite is widely accessible, or just making sure that credit goes to the right people.

The post title is taken from “To Open Closed Doors” by D.R.I. from the Dirty Rotten LP

Strange Things

I noticed something strange about the 2013 Impact Factor data for eLife.

Before I get onto the problem. I feel I need to point out that I dislike Impact Factors and think that their influence on science is corrosive. I am a DORA signatory and I try to uphold those principles. I admit that, in the past, I used to check the new Impact Factors when they were released, but no longer. This year, when the 2013 Impact Factors came out I didn’t bother to log on to take a look. A chance Twitter conversation with Manuel Théry (@ManuelTHERY) and Christophe Leterrier (@christlet) was my first encounter with the new numbers.

Huh? eLife has an Impact Factor?

For those that don’t know, the 2013 Impact Factor is worked out by counting the total number of 2013 cites to articles in a given journal that were published in 2011 and 2012. This number is divided by the number of “citable items” in that journal in 2011 and 2012.

Now, eLife launched in October 2012. So it seems unfair that it gets an Impact Factor since it only published papers for 12.5% of the window under scrutiny. Is this normal?

I looked up the 2013 Impact Factor for Biology Open, a Company of Biologists journal that launched in January 2012* and… it doesn’t have one! So why does eLife get an Impact Factor but Biology Open doesn’t?**

elife-JIFLooking at the numbers for eLife revealed that there were 230 citations in 2013 to eLife papers in 2011 and 2012. One of which was a mis-citation to an article in 2011. This article does not exist (the next column shows that there were no articles in 2011). My guess is that Thomson Reuters view this as the journal existing for 2011 and 2012, and therefore deserving of an Impact Factor. Presumably there are no mis-cites in the Biology Open record and it will only get an Impact Factor next year. Doesn’t this call into question the veracity of the database? I have found other errors in records previously (see here). I also find it difficult to believe that no-one checked this particular record given the profile of eLife.

elfie-citesPerhaps unsurprisingly, I couldn’t track down the rogue citation. I did look at the cites to eLife articles from all years in Web of Science, the Thomson Reuters database (which again showed that eLife only started publishing in Oct 2012). As described before there are spurious citations in the database. Josh Kaplan’s eLife paper on UNC13/Tomosyn managed to rack up 5 citations in 2004, some 9 years before it was published (in 2013)! This was along with nine other papers that somehow managed to be cited in 2004 before they were published. It’s concerning enough that these data are used for hiring, firing and funding decisions, but if the data are incomplete or incorrect this is even worse.

Summary: I’m sure the Impact Factor of eLife will rise as soon as it has a full window for measurement. This would actually be 2016 when the 2015 Impact Factors are released. The journal has made it clear in past editorials (and here) that it is not interested in an Impact Factor and won’t promote one if it is awarded. So, this issue makes no difference to the journal. I guess the moral of the story is: don’t take the Impact Factor at face value. But then we all knew that already. Didn’t we?

* For clarity, I should declare that we have published papers in eLife and Biology Open this year.

** The only other reason I can think of is that eLife was listed on PubMed right away, while Biology Open had to wait. This caused some controversy at the time. I can’t see why a PubMed listing should affect Impact Factor. Anyhow, I noticed that Biology Open got listed in PubMed by October 2012, so in the end it is comparable to eLife.

Edit: There is an update to this post here.

Edit 2: This post is the most popular on Quantixed. A screenshot of visitors’ search engine queries (Nov 2014)…

searches

The post title is taken from “Strange Things” from Big Black’s Atomizer LP released in 1986.

My Blank Pages

Books about the MRC Laboratory of Molecular Biology are plentiful. If you haven’t read any, the best place to start are the books written by some of the Nobelists themselves: “I Wish I’d Made You Angry Earlier” by Perutz, “My Life in Science” by Brenner. Also, “Sequences, Sequence, Sequences” by Sanger, “What Mad Pursuit” by Crick and even Watson’s “The Double Helix” cover ‘how it was done’ and ‘what the place is like’. After that are the biographies of the Nobelists and their associates. Then comes the next layer, the comprehensive but rather dry “Designs for Life: Molecular Biology after World War II” by de Chadarevian and hell, even “The Eighth Day of Creation” by Judson is substantially about the LMB, since so many major discoveries in Molecular Biology happened there.

If your appetite is not sated after wading through all of those, then there are the books for the insiders.

John Finch wrote a book “A Nobel Fellow on Every Floor” which was enjoyable, if rather selective on who and what was included. The latest book from the LMB Press is a collection of essays entitled “Memories and Consequences: Visiting Scientists at the MRC Laboratory of Molecular Biology, Cambridge”. It was edited by Hugh Huxley and was made available last summer (around the time of his death).
You can get it here

 

memories

The premise of Memories and Consequences is that there were a large number of postdoctoral fellows, mainly from the USA, who spent time at the LMB (in the 60s, mainly) and then went away and had hugely successful scientific careers. At one point in the book, Tom Steitz writes that, of his friends during this period, 40% are now NAS members! The essays cover the time of these visitors in England and how it shaped their subsequent careers.

This is definitely a book to dip in and out of. The experiences are actually pretty repetitive: yes, we drive on the other side of the road; Cambridge is a very stuffy place and Max Perutz liked to be called Max. This repetition is amplified if the chapters are read one-after-the-other. Overall however, the essays are nice reminiscences of a booming time in Molecular Biology and many capture the magic of working at the LMB during this period. Brenner and Crick come to life and even Sir Lawrence Bragg looms large in many chapters filling the authors with awe.

When I first downloaded the book, I read the chapters by those whose work I am most familiar. I didn’t even know that Dick McIntosh had spent not one but two sabbaticals at the LMB. Tom Pollard, Harvey Lodish etc. followed. I then read the other chapters when I had more time.

The best chapters were those by Harry Noller and by Peter Moore who gave the right amount (for my taste) of personal insight to their stay at the LMB. I would recommend that the reader skips the chapter by William Dove and Alexandra Shevlovsky, who tried to be a bit clever and didn’t quite pull it off. Sid Altman’s chapter has previously been published and I actually witnessed him read this out (more-or-less) verbatim at the DNA50+1 celebrations – which was far more enjoyable than it sounds.

In short, I enjoyed the book and it’s worth reading some of the chapters if you have a leaning towards the history of science, but there are plenty of other books (listed above) where you should start if you want to find out what life is like inside the Nobel Prize Factory.

I’ll leave you with three quotes that I enjoyed immensely:

“I remember seeing copies of the journal Cell, where we all yearned to publish (though, I noticed, not the really great scientists, like John Sulston or Sydney Brenner). I would shudder and turn away; Cell was for other scientists, not for me.”
Cynthia Kenyon

“Like many others who worked at the LMB in that era, I still think of its modus operandi as exemplifying the blueprint that all scientific research establishments should aspire to emulate. Pack the very best scientists you can find into a building, so densely that they cannot avoid talking to each other, and encourage them to interact in every other way you can. A canteen or dining room might be a good idea. (The facility itself need not be luxurious, and indeed, it is probably better if it is not.) Give those scientists ample staff support, and all the money they need to get on with the job. Stir well, and then be patient because good science takes time. My subsequent career has taught me that this recipe is much harder to execute than it is to describe. I still wonder how the MRC managed to do it so well for so long.”
Peter Moore

“I learned that protein chemistry didn’t need me, that King’s College High Table was for tougher folk than I, and that Sydney talked but Francis conversed.”
Frank Stahl

A comprehensive guide to LMB books is available here

Don’t worry, book reviews will be a very infrequent feature as I hardly have any time to read books these days!

The post title is from My Blank Pages – Velvet Crush from their LP Teenage Symphonies to God. Presumably a play on the Dylan/Byrds song My Back Pages.

A Day In The Life

#paperoftheday #potd

A common complaint from other PIs is that they “don’t read enough any more”. I feel like this too and a solution was proposed by a friend of a friend*: try to read one paper per day.

This seemed like a good idea and I started to do this in 2013. The rules, obviously, can be set by you. Here’s my version:

  1. Read one paper each working day.
  2. If I am away, or reviewing a paper for a journal or colleague, then I get a pass.
  3. Read it sufficiently to be able to explain it to somebody else, i.e. don’t just scan the abstract and look at the figures. Really read it and understand it. Scan and skim as many other papers as you normally would!
  4. Only papers reporting primary research count towards #paperoftheday.
  5. If it was really good or worth telling people about – tweet about it.
  6. Make a simple database in Excel or Papers – this helps you keep track, make notes about the paper (to see if you meet #3) and allows you to find the paper easily in the future (this last point turned out to be very useful).

I started this in 2013 (for one full year) and am trying to continue in 2014. I feel that this is succeeding in making me read more than I would have otherwise done.

My stats for 2013 were:

  • 85% success rate. Filling that last 15% will be tough.
  • Stats errors in 48% of papers! Most common error was incorrect use of Student’s  t-test.
  • 68% of papers were from 2013 and 22% were from 2009-2012.

The big surprise was which journals I read most:

  1. J Cell Biol 13
  2. PLOS One 12
  3. Nat Cell Biol 10
  4. PNAS 10
  5. Curr Biol 9
  6. Mol Biol Cell 8
  7. Nature 8
  8. Dev Cell 7
  9. eLife 7
  10. Nature Methods 7
  11. Cell 6
  12. Neuron 6
  13. Traffic 6
  14. J Cell Sci 4
  15. Science 4

I thought that Cell would be much higher and PNAS would be much lower. Since where we publish is dictated by who is likely to see and read the paper, this list was thought-provoking.

*I think this was a colleague of @david_s_bristol who suggested it, sometime in 2012.

The post title is of course from A Day in The Life – The Beatles from the LP Sgt Pepper’s Lonely Hearts Club Band. For the first line…

So Long

How long does it take to publish a paper?

I posted the picture below on Twitter to show how long it takes for us to publish a paper.

papers

The answer is 235 days. This is the median time from submission at the first journal to publication online or in print. The data are from our last ten papers.

The infographic proved popular with 40 retweets and 22 favourites. It was pointed out to me that the a few things would improve this visualisation:

1. Showing the names of the journals

2. Showing when the 1st submission was relative to the 1st submission at the journal that finally accepted the paper

3. What about reviews and other types of publication.

I am working on updating the graph to show all of these things… watch this space.

My point was really to show (perhaps to non-scientists) how long the process of publishing a paper can be. There is other information that can be gleaned from this, e.g. what proportion of time is at the journal’s side and how much is at our end?

The people who are eager to see which journals perform badly (slowly) will be disappointed: this is a very small subset of papers from one lab. I’d be interested in scraping the information on journal tardiness on a larger scale and synthesising this so that it can inform journal choice. Recently though major publishers have taken steps to make this information less accessible so don’t hold your breath.

The title of this post is from So Long by Cian Ciarán from the LP ‘Outside In’