Open Science – ExcursionSet

No Rose Without a Thorn

Posted on 13 January, 2015 by reasther in Bad Science

In the last few years the number of “open access” journals has burgeoned. Open access journals are free to read, which limits their ability to raise money from subscriptions, and they typically charge authors a fee for publication. This may sound a little odd, but “page charges” are standard at many traditional journals. That said, this business model creates a challenge: set your standards too high and you won’t be able to meet your payroll; set them too low and you won’t be taken seriously by the scholarly community.

Open access also creates opportunities for scammers who are happy to sit at the bottom of the food chain. The phenomenon is known as “predatory open access”, and a handy list of possible bucket-shop operators is available online. Predatory journals prey on those who are too naive to realise what is going on, or are either vain or venal enough to pay to see their name on a “published” article and I get spam every day from these outfits asking for submissions. They really do publish pretty much anything in return for money, as the authors of a paper entitled “Get Me Off Your &*^#ing Mailing List” discovered last year (although they didn’t actually say &*^#) when they tried to stem the flow of email from predatory journals.

I normally just ignore these entreaties, but over the last few weeks, “AshEse Visionary” popped up more than once in my inbox. I accidentally clicked on the link, and found this:

http://www.ashese.co.uk - downloaded January 14, 2015. — http://www.ashese.co.uk – downloaded January 14, 2015.

After learning that this journal will “work with Arthurs every step of the way” I couldn’t resist digging deeper, despite not being an Arthur myself.

Surfing around their site, it struck me that their “instructions for authors” page was very clear and well-written. Who wrote it is far less obvious though, as googling a few key phrases turns up the same text in a slew of places. Looking at the first few pages of hits, almost all of them are on the list of possible predatory journals, or look as if they should be.(In fact, does using this content indicate that you should be on the list?)

And it has to be said that Ashese has strict ethical guidelines for its authors, even if these do bear a striking resemblance to those of the journal Brain, as you can see from the following samples:

Ashese

Brain

Ashese

Brain

I am still a huge fan of open science, despite the barrage of pay-to-play spam, and would love to live in a world where all scholarly publications were freely available to anyone who wants to see them. But I am beginning to think we need peer review for journals, as much as we need it for the articles within them.

BICEP2: A Month Later

Posted on 17 April, 2014 by reasther in Cosmology, Physics

A month ago the BICEP2 team announced that our universe is apparently awash with gravitational waves, pointing to the existence of an inflationary phase moments after the Big Bang. This was front page news all over the world, and cosmologists and astrophysicists have been working overtime to make sense of the news. Here is some of that sense…

Let The Ambulance Races Begin

For theoretical physicists, ambulance chasing involves getting papers out quickly after a major data release. Some ambulance chasers make significant contributions, some are just trying to draw attention to their earlier work, while others are banging out insubstantial papers in the hope that they will be cited by their slower colleagues. But whatever their motives, cosmologists have certainly been busy: the BICEP2 discovery paper has been cited 188 times on the Arxiv, all in “preprints” written within a month of the original announcement. I am pretty sure this is a world record, and you can always check the current tally.

In fairness, though, cosmologists were so giddy about BICEP2 it wouldn’t have surprised me if someone had stolen an ambulance and driven it in circles, flashing the lights and letting rip with the siren.

Distributed Peer Review and Open Science

Once upon a time, the right way to announce a big result was to 1) write the paper, 2) send it to a journal, wait for it to be 3) peer reviewed and 4) accepted for publication, after which you could 5) hold a press conference. However, like most recent announcements in fundamental physics and cosmology, BICEP2 went straight from paper to media event, skipping steps 2, 3 and 4.

Old-timers will shake their heads, but this approach fits the principles of open science, which advocates making the processes and products of science transparent and widely available. Given that 1000 scientists are now scrutinising the BICEP2 results, rather than just two or three readers appointed by a journal, this amounts to an intensive, distributed and open peer review process, which is no bad thing. (And the papers will end up in a journal sooner or later.)

Trouble in Paradise?

The real gold-standard for science is not peer review but reproducibility. BICEP2 claims to have detected a specific twist in the polarization of the microwave background — the so-called “B-mode”. This detection will not be a sure thing until it is confirmed by an independent team with an independent instrument performing an independent analysis. On top of that, inflation is not the only possible origin of such a B-mode, and further data will help confirm the theoretical interpretation of the BICEP2 observations.

The good news is that no-one has found any show-stoppers. The biggest worry to surface so far is probably that the patch of sky BICEP2 observed may be contaminated by emission from radio “loops” associated with our own galaxy. It is not clear to me that this signal would necessarily reproduce the BICEP2 result, but unsubtracted foregrounds are likely to make any underlying gravitational wave signal look bigger than it really is, and that will need careful checking. And in the worst-case scenario, the BICEP2 results would be purely due to foregrounds, or some other analytical glitch.

We may not have to wait long. The BICEP2 team will be looking closely at these concerns, and more data will be gathered during the coming polar night. In addition, the Planck satellite has gathered the world’s most comprehensive observations of the microwave background and their science team is extending their initial analysis to look at polarization, with results promised before the end of 2014.

Free Trips to Stockholm

If the BICEP2 result is verified, it is certain to attract the attention of the Nobel committee. In fact, it may be worth two Nobel prizes – one for the idea of inflation, and one for the detection of B-modes, which is a technological tour de force in its own right. (Two prizes have already gone to the microwave background — one for its discovery, and one for the first mapping of the temperature of the microwave background.)

Speculating about “the prize” is a popular game among scientists, and I have already heard people ruminate about the likely judgment of history if it turns out that the BICEP2 analysis is basically correct but slightly dust-contaminated. In this scenario, the BICEP2 announcement would have been made with far more confidence than the data ultimately justified, which would provide conversational fodder for decades.

The intellectual history of inflation has many parallels with that of the the Higgs boson; they are both elegant hypotheses that existed for decades before being experimentally confirmed (assuming, again, that BICEP2 really has seen evidence of inflation). And like the Higgs, the theoretical parentage of inflation is murky. Alan Guth is undoubtedly the Peter Higgs of inflation (even if it is not called “the Guth phase”), but a number people made key contributions to the development of the theory. Unfortunately, only three of them can share the Prize, and there will be discreet (and probably blatant) lobbying for the other two places on the stage if the BICEP2 data holds up.

What I Have Been Doing?

Beyond giving a slew of interviews the day the story broke, my group at the University of Auckland (in collaboration with Kevork Abazajian at UC Irvine) has looked carefully at the apparent tension between BICEP2 and existing cosmological data. BICEP2 does not just claim to have seen gravitational waves, but to have seen gravitational waves with an amplitude which was apparently ruled out by previous analyses.

We crunched a lot of numbers very quickly, thanks to the high performance computing facilities at NeSI (New Zealand’s e-research organization), and showed that this tension between BICEP2 and previous analyses is statistically significant. Consequently, taking all currently available astrophysical datasets at face value, BICEP2 appears to tell us three startling things about the early universe:

Inflation really did happen right after the big bang.
Inflation happened when the energy density of the universe was very high, as the strength of the gravitational wave background depends directly on the energy density of the universe during inflation. This means that the mechanism of inflation can give us a portal into the realm of ultra-high energy physics, where we expect candidate “grand unified theories” (including string theory) to be important.
The inflationary phase must be relatively complex, for the gravitational wave background to have escaped indirect analyses made prior to BICEP2. And this means that cosmologists will be able to make far more stringent tests of competing inflationary models than we might have expected.

Alternatively (and much more conservatively!) our results could suggest that the BICEP2 team has over-estimated the strength of the gravitational wave background and that future analyses will remove this discrepancy.

One More Thing

To me, one of the most astonishing things about the BICEP2 telescope is just how small it is. The secret to BICEP2 is not its size, but the exquisitely sensitive superconducting transition edge sensors used to detect the microwave signal. Admittedly, BICEP2 sits at the South Pole, the whole instrument is chilled to within a hair’s-breadth of absolute zero (a major technological and logistical challenge) and it is surrounded by a complex array of shields, but the actual telescope is 23cm across. This is only a few times larger than the optical instrument Galileo used to explore the heavens over 400 years ago, and BICEP2 may one day rival Galileo in the profundity of its implications for our place in the universe.

BICEP2, to scale - www.bicepkeck.org — BICEP2, to scale – www.bicepkeck.org

Open Sesame II

Posted on 15 March, 2013 by reasther in Equity & the Profession

Earlier this week I posted a paper on the Arxiv (which is described here). The next morning I woke up to an email asking if my co-authors and I were interested in submitting our paper in a new, open access astrophysics journal, Physics of the Dark Universe. The Editorial Board is full of people I respect and — unlike many Open Access journals — there are no page charges: it is free to both authors and readers. Even more interestingly, the journal is published by Elsevier, one of the main whipping boys of the Open Science movement.

So what gives? So far as I can tell, Elsevier hopes to negotiate blanket deals with science funding agencies and consortia of institutions to cover the cost of these journals. And I suspect many scientists will be apprehensive at the thought of Elsevier inserting themselves even more deeply into the world’s scholarly infrastructure.

But on the other hand, Elsevier are taking a risk if they pursue this new model — government agencies have more negotiating leverage than individual universities. Moreover, it is a problem if academics can’t read an article in the literature, so scholarly libraries strive for completeness and (by and large) grit their teeth and pay millions each year to cover journal subscriptions. On the other hand, it is less of a problem (in most fields) to lose the opportunity to publish in a given set of journals, provided there are viable alternatives. And this model allows Elsevier to charge for publication, but leaves no room to charge for reading — and Elsevier’s journals typically cost far more on a per article basis than many of their competitors.

This certainly seems to be a response to the Open Science movement from Elsevier. Whether it is a win for Open Science remains to be seen — the devil will be in the details, but Elsevier is risking much of its present ability to dictate the price it charges for its services.

My guess is that even if this approach is viable, purists will continue to shun Elseiver — they are the Microsoft of academic publishing. But just as many Linux users switched to Macs when OS X became available during the most vocal era of the Open Source software movement I suspect many scientists could learn to love free-to-you journals — even if they are created by large for-profit corporations.

As for me, the new journal’s focus on the “dark universe” is probably not a good fit for our paper, so it will go elsewhere.

Open Sesame

Posted on 5 February, 2013 by reasther in Equity & the Profession

There is an “Open Research” meeting here in Auckland this week and I am registered for the “unconference” on the second day (I am also headed to Kiwi Foo on the weekend, which means I will attend more unconferences in a week than I managed in the rest of my life).

My first taste of open research — but no-one called it open research back then — came as a PhD student at the University of Canterbury in 1990, where the circulation of the “preprint list” was a weekly ritual. Possibly this is more accurately “open science”, a subset of open research. Back then, scientists sent pre-publication copies of their papers (“preprints”) to a handful of libraries and leading institutions. As a public service, the library at SLAC mailed a weekly list of new preprints it had received to particle physics researchers all round the world. (Actually, I think you had to pay an annual subscription to cover the postage.)

After it arrived at Canterbury, the list went from hand to hand among the researchers with an interest in particle physics. When you got the list, you flagged titles that looked interesting, noted the author’s institution, turned to a longer list that provided their postal addresses, filled in the blanks on a special postcard politely requesting a copy of the article, and mailed it off. A few weeks or months later an envelope arrived in the mailroom with your name on it, possibly some exotic stamps, and the preprint inside.

Then, in 1991 the “Archive” was launched by Paul Ginsparg from the domain xxx.lanl.gov. (There was also an earlier, smaller, and less automated service run by Joanne Cohn.) The Archive sent a daily digest, you submitted papers to the archive by email (or even ftp), the list was distributed by email, and you requested papers by emailing a software robot. This is what the pre-www Internet looked like.

Papers arrived as raw LaTeX (or just TeX), an HTML-like mathematical mark-up language that was (and is) ubiquitous in particle physics. A few keystrokes at the command line turned the source file into a typeset paper, and thus a process which had taken weeks or months now took minutes. Not only that, a PhD student in New Zealand now got preprints at the same time as a professor at Harvard.

As time went by, the operation became more professional and spread across multiple disciplines. For a taste of the wild west days of open science, check out early messages to Archive suscribers [sic] — search for “suscribe” [sic].

Nowadays, the Archive (at the domain Arxiv.org) is the dominant repository for papers in particle physics and astronomy. But perhaps more importantly, the Archive has reached the point where it threatens to do to traditional journals what MP3s did to record shops, as it represents a radically new model for scientific publishing. In particle physics and astrophysics, the Archive is essentially complete — I almost never see traditionally published papers that are not also posted to the Archive. Not only that, I am more likely to download the preprint than the journal’s version of an article, because everything is on the Archive which makes it a one-stop shop, whereas each journal has its own website.

There are many philosophical arguments in favor of openness in science, but from my perspective the basis of the Archive’s success is entirely pragmatic — it makes it easier for scientists to do science.

[More on Open Science and Open Research in Part 2 of this blog – coming soon.]