Posted by: jonkatz | October 29, 2009

Full Versions of Papers, and Journals

(This post is somewhat of a followup to Yehuda’s post, though coming at it from a different angle.)

I agree with Yehuda that full versions of papers are, generally speaking, extremely important. As has been discussed elsewhere (see here and here, for example), our community (both the TCS community as a whole, as well as the theoretical cryptography community in particular) seems to have a problem with its emphasis on paper quantity rather than quality. (Though the quality of the research is also at issue, I am speaking here particularly of the quality of exposition.) I think also that a lack of full versions hampers scientific progress in the long run: it is a barrier to entry for others, and allows mistakes to happen more frequently than they would otherwise.

In contrast to what others have said, though, I am not sure journals are the solution. What follows is one personal story that illustrates some of the problems.

Journals: What Can Go Wrong

I recently had a paper accepted to a journal, and it will be published next month. This paper was submitted in August 2004. So it took just shy of 5 years until the paper was accepted, and it will have been over 5 years between submission and publication. It wasn’t as if the paper needed heavy revision, either — the reviewers just sat on it for a very long time. Nor did I get any useful feedback from the reviewers: I’m not sure they even checked the proof (whether they did or did not, they had no comments about it), and the only substantive thing they made me change were the references to prior work. And I didn’t even agree with their recommended changes (but didn’t feel like fighting them either).

Because of the journal’s policy of having “new results”, I added results that (in my opinion) increase the length of the paper without increasing its quality. The journal’s typesetting rules made the paper even longer and, in my opinion, more difficult to read (due to font issues).

Ideally, I would have liked to maintain a different copy of the paper on my webpage, written the way I like. I know some people do that, but in practice it is a pain to keep track of different versions (and so I didn’t do it in this case).

The Future Role of Journals?

So if journals are not the answer, what is? Let me be first clear that I think journals do have an important role: they are immensely useful as a form of peer review (in terms of both correctness and level of interest in the results) — assuming reviewers do their jobs. We are also not going to more away from a journal-based system any time soon, not least of all because journals publications are still a part of hiring/tenure/raise/etc. decisions. Yet is also seems clear that journals will eventually die out, at least effectively, and probably within my career; I plan to say more on this in a future post.

Still, we don’t need journals, given the fact that we can post our papers on our webpage and put them in public archives.* It seems, then, that the most useful role of journals is to serve the role of being the “carrot” that motivates people to write full versions in the first place.

*Or write a book. See Goldreich’s intriguing essay: On Our Duties as Scientists (personal version), Section 4.1. His essay is worth reading for the other points it makes as well.

Advertisements

Responses

  1. Well, I’d say just publish the full version as a technical report or put it in the arxiv, and reference it from the short conference paper.

  2. 1) The fact that journals have such slow turnaround is no one’s fault but our own. There are plenty of communities where journals have turnaround times of < 3 months (many physics and bio journals are like this, for example). That's actually LESS time than the amount of time between when you submit to a conference and when the conference occurs.

    2) Your other reason — that the journal's policy of requiring "new results" led to a lower-quality (quality of writing, not nec. quality of research) paper obviously does not apply to all journals. If you look at old FOCS/STOC issues (from, say, the mid-1980s) many of those papers later appeared in journals with almost the exact same set of results that were in the conference proceedings, just fleshed out in more detail.

    3) It's not that hard to maintain different versions: use a version control system like CVS, SVN, or GIT (all of which allow you to have branches). Although their diff capabilities don't work quite as nicely with LaTeX as with, say, Python code, as a whole this solution still works pretty well.

    You certainly haven't convinced me that journals aren't the answer. I don't think they are perfect, but I do think they're much better than conference proceedings instead.

    I think the role of journals in peer review is much more important than their role as the carrot that makes us write full versions in the first place. Also, given the number of conference papers that never make it into journals, it's not clear how effective that carrot is.

    I eagerly await your post on why it's "clear" that journals are going to go the way of the dinosaur.

  3. It’s not that hard to maintain different versions…

    I disagree (and I use version control). For starters, what do you do if there’s a typo that needs to get corrected in several versions. Second, there is also the problem if people refer to different versions of your paper (although this can be handled, not very nicely).

    I don’t think [journals] are perfect, but I do think they’re much better than conference proceedings instead.

    I am very much in favor of full versions as opposed to conference writeups. But as long as a journal submission can take 5 years to review, I can’t see how journals can be a better forum for research than conferences.

    I eagerly await your post on why it’s “clear” that journals are going to go the way of the dinosaur.

    Isn’t it obvious? Maybe I should have made clear that I meant print journals. Actually, what I plan to talk about in a future post is what can replace journals.

  4. Can’t you merge changes across branches and then only merge the change involving the typo? But either way, fair enough — you have more experience dealing with version-controlled papers than I do.

    I agree it’s obvious that print journals are going the way of the dinosaur.

    I agree that with 5-year turnaround times journals are not even contenders with conferences for means of disseminating research. (By the way, I’m not advocating that conferences should disappear. I just think that having conference proceedings as our main venues of publication is, to paraphrase Lance Fortnow, somewhat immature.)

    Not only do I think the 5-year turnaround is unnecessary, but I think it wouldn’t take that much to get our community to do, say, 6-month turnaround times. If everyone starts expecting fast reviews from everyone else, and returns the favor in kind, I think it could get going rather rapidly. Particularly if some of that peer pressure were coming directly from journal editors.

  5. I just want to clarify that although I strongly believe in writing journal papers, the issue of writing FULL VERSIONS is much more critical. Indeed, there is one well-known researcher that I know who has very few journal papers. However, he has full versions an almost all of his papers on his webpage. Personally, I have no problem with this strategy (there is some loss in the possible “proof verification” that goes into a journal version, but I’m not sure how significant this really is).

    One more thing regarding turnover: it depends very much on your editor. There are some editors that simply don’t push at all, whereas there are others that do. I have learned over time which editors not to send papers to. (I currently have two papers at an editor who is sitting on them like in Jon’s example. After the process is finished, I plan on sending a letter to the editor-in-chief to let him know of the situation. I think that we should do what we can to have such editors removed from the editorial board. By the way, I know that this editor doesn’t push because I also refereed for him. After sitting on a paper for a long time, I remembered myself that I needed to do it, but never received any reminder from him.)

  6. I think that we should do what we can to have such editors removed from the editorial board.

    That’s exactly the point. There are plenty of stories about editors who don’t even reply to emails many years after the paper was submitted. If editors are *that* busy its time to increase the size of the editorial board.

  7. Jon – I’ve had the opposite experience as you. Clearly the issue you mention is community-dependent. I submitted a paper to a conference – the paper got turned down so we submitted it to a journal and simply submitted a poster abstract to the conference. The journal accepted our paper within about 1 month and it appeared online before the conference. Then, our poster was selected for oral presentation. Thus, we got a poster, a talk, and a journal paper all within the time between the notification of acceptance from the conference and the actual conference date.

    As for the conference vs. journal debate – IMHO the conferences place an artificial restriction on the number of “good quality” papers that can be accepted while journals (especially online journals) do not.

  8. Mihai, that is unbelievable!

    I have never heard of a 1-month turnaround time for a journal paper before. Is this representative of your experiences, or an outlier?


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: