Posted by: jonkatz | April 4, 2011

Why I hate rump sessions (guest post)

At TCC last week, Dov Gordon gave a rump session talk about…why he hates rump sessions. I wasn’t there, but I asked him what he said and thought the issues he raised were worth discussing. Below is some text Dov sent me (edited slightly for form, but not for content), followed by my reaction.


My intent was to make two points: the first is a serious ideological point that I now feel I should have emphasized more forcefully. The fast pace of progress in our community creates a great pressure to publicize our results as quickly as possible. The rump session provides an excellent means for creating a “time-stamp”. Unfortunately, there is also a temptation to use the rump session for announcing results where there is not even a manuscript to distribute, and this sort of “flag planting” has very negative consequences. From an ideological perspective, we cannot claim to have solved a problem before providing the public a chance to verify the proof; such behavior goes against the core principals of scientific investigation, regardless of the author’s own level of confidence or credibility. From a social perspective, if no manuscript is available, the speaker can claim huge amounts of territory with their flag, scaring others from pursuing a direction of research that may yet be unsolved! While we pretend that the rump session is increasing communication and collaboration, flag-planting is often used to the opposite effect by scaring off the pursuit.

My second complaint about rump sessions is far less serious and is more a matter of taste. While I won’t go so far as to say that the rump session is a waste of two hours, I’m usually too drained at that hour to get the most out of the talks, and more broadly I think there are better uses of that time. Most of what appears in the rump session will be on ePrint soon enough, and it is likely to appear in a conference shortly after that. How much is really gained from this jump-start? On the other hand, our conferences are already over-loaded, and it is very difficult to have all the one-on-one conversations we would like to have. This is especially true at TCC where there are so many people to talk to, it is a bit harder to find sessions to skip, and there is no social event. I would much rather we all spent the time talking shop or socializing. Finally, the rump session is supposed to be *fun*, and while some people make the effort to keep it that way, by and large I just find the self-promotion to be boring; in a 20 minute talk we try to teach, but in a 5 minute talk we try to sell. Who in this modern day would *choose* to watch the commercials?

While the response has been mostly positive regarding the first point, I feel I am mostly alone on the second point. If the rump session isn’t going away, I hope at least that we can all agree to stop announcing results that aren’t publicly available.


Let me address Dov’s first point, with which I sympathize. For starters, I also get the impression that research in our field sometimes feels more like a “competition” than the pursuit of science. (I do sometimes feel as if many people in the community think the point of research is to publish papers, rather than to do good science.) The rump session, which emphasizes “selling” results rather than explaining them, only reinforces this impression. There is also the larger issue that Dov raised, namely putting a timestamp on the claim that some problem is solved even when the proof is not yet written (and/or not yet been verified).

On the other hand, the rump session does have some positive aspects. Eliminating the rump session will not slow down the rate of progress in our field; better to know about other recent results (or even other groups working on as problem) than not to know about it or, perhaps worse, to end up with a situation where some people know and others don’t. So rather than ending rump sessions entirely, or trying to police which unpublished results can be presented and which ones cannot, maybe we as a community can agree on some “ground rules”. Here are some that I would agree with:

  • Results that have appeared, or have already been accepted to appear, at other major crypto conferences should not be presented. If it will appear at another crypto conference, the community will hear about it; you do not need to further “advertise” your results by presenting them at a rump session. (Note: I think it is valid to present results that may be of interest to the crypto community that might be appearing at non(-major) crypto conferences. E.g., a result appearing at PODC that the community might otherwise be unaware of.)
  • There needs to be some mechanism to deal with concurrent results: e.g., where one person presents a result at the rump session and then someone else claims they also solved the same problem. I’m not sure how the would work in practice, but I think this could actually be one benefit of the rump session in that it would prevent duplication of work.
  • Presenting a result at a rump session should be viewed as imposing some responsibility to make a pre-print of the result available. I’m not interested in enforcing presenters to make pre-prints available; what I’m more interested in is some agreement from the community that if a pre-print is not made available in some reasonable amount of time then the question should still be considered unsolved. (We could also have presenters post retractions of their claimed results.) This would also help prevent another problem I have seen, where a submitted paper is rejected because it solves a problem that was claimed to have been solved at some previous rump session (whether the author of the submitted paper was aware of it or not).

Since I was listed as co-author on four rump session talks, I should at least mention that I only encouraged one of those talks (I won’t tell which one!), and then mainly because a student/postdoc was involved.

Advertisements

Responses

  1. I am surprises that rump session talks are considered as valid time stamps. I think they should be considered announcement of partial results, nothing more.

  2. Do rump session talks really serve as timestamps? My impression is NO; only at best w.r.t. the attendees. If someone in the audience has a conflicting work, they are expected to contact the speaker immediately afterward and disclose it. For non-attendees, such a “private” talk cannot serve as any type of timestamp, if they have a conflicting result and submit it to a publication venue later, it is reasonable to consider it independent.

  3. For non-attendees, such a “private” talk cannot serve as any type of timestamp

    I don’t know. As I said, I have seen papers be criticized for solving a problem that was claimed solved in a rump session talk.

    …if they have a conflicting result and submit it to a publication venue later, it is reasonable to consider it independent.

    That may be true, but it misses the broader point. Forget about concurrent results: if someone claims to have solved a problem in a rump session talk, that discourages (prevents?) other people from starting to work on it.

  4. Vipul: here is my strategy to figure out if you are working on the-ultimate-crypto-problem, or not — I give a rump session talk announcing that I solved it (even though I only have a hint of an idea towards an approach that maybe solves a special case, maybe not). This feels outright sleazy and in some sense, artificially forces you to “come out of the closet” right away (Believe it or not, this happens). Of course, I could solve the damn problem and put the pre-print up in arXiv, and you’ll still be forced to come out, but that somehow feels legitimate.

    It feels like the right way to deal with rump session talks is to NOT consider them as time-stamps of any sort. If the authors want to give a rump session talk without making a pre-print publicly available, it’s their loss. The world knows about their result, the world can work on it and publish without any consequence. In other words, this gives the rump-session speakers a strong incentive to write up the results before giving the talk, and publish a pre-print before, or soon after.

    I’d still like to keep the rump session around for the free beer, and the smattering of funny talks. 🙂

  5. Vipul: I have heard of several instances where either a) people were blamed for starting work on a result that was already announced at a rump session, or b) they were afraid to do it.

    Jon and Vinod: I certainly agree that there are positive points to having a rump session, and I agree it is possible to keep it around while addressing my concern, but there is no way to have an entire community naturally adopt a list of guidelines. The only way to change things (in general) is through broad, simple decisions. My point is that there is one very simple rule that could be used: you cannot publicly announce a result that you haven’t yet written. It is simple, and therefore far more likely to be adopted. In fact, it can be put into place by the person chairing the rump session. Vinod – the problem with saying that we should just consider such problems “fair game” is that nobody actually will. There is too much risk. What if they post it tomorrow? And even if they don’t – who will willingly walk into a potential conflict, especially if it is with someone more senior to them? (Vipul, this last point my be the bigger issue with flag-planting: it “scares off the pursuit.”)

    Finally: Please note – I never suggested we should stop having wine and beer. 🙂

  6. Dov, I think only allowing rump session talks if there is a manuscript available is too harsh. For one thing, if someone has a new, interesting result then people want to hear about it; if someone is talking about the result with their friends then why shouldn’t they also share it with the broader community at a rump session? (And such results get talked about at workshops like Bertinoro, Dagstuhl, anyway, so may as well let the whole community know about it.) It also can facilitate rapid scientific progress. So I think there is plenty of good in presenting not-yet-written results, as long as the process is not abused.

  7. I think there’s a much simpler fix that will address the worst part of the problem. For heaven’s sake, don’t reject a submitted paper because it solves a problem that was claimed to have been solved at some previous rump session. That’s just stupid! Who came up with that brilliant idea? How about a community agreement on a ground rule that program committees should not do stupid stuff like that?

  8. I don’t buy the argument that the rump session is bad because people will shy away from working on any result that’s been announced at a rump session. Well, guess what: the same is true in any case where someone tells you one-on-one about a result they’ve got. I don’t see how a rump session fundamentally changes this.

    Moreover, I think there’s an important safeguard you’ve overlooking. If famous cryptographer X announces a result at a rump session, but then never publishes a paper to support it, people will start to wonder. If I ask cryptographer X about the result and cryptographer X is forced to admit that the result was actually bogus, cryptographer X will feel lame. People don’t like to feel lame, and people don’t like to look bad in front of their peers. Therefore, I think there is an incentive not to announce results at a rump session until you are personally confident that they are solid.

  9. ExCrypto: You’re definitely over-simplifying things. I agree that there is incentive not to announce a result until you feel “personally confident”, but this is not a binary “feeling”. You also wouldn’t publish something unless you were “personally confident” in your proof, yet we publish buggy proofs all the time. This measure of confidence is a continuous function. Of course I agree that people will not announce something before they’ve checked the proof. I’m not suggesting there is malice involved. But the proof is probably not even in LaTex yet, it hasn’t been seen by objective people, and it is much much more likely to have bugs than conference versions, and especially journal versions.

    More importantly, you are arguing abstractly. It’s great if you’ve never had that experience, but I’m speaking from my own and other’s personal experiences. People (myself included) do announce results before they are confident *enough* about the correctness. And some rump session announcements never become public.

    As for your point about committees – I have never heard of a paper being rejected for that reason, and I didn’t mean to suggest that happens. Are you speaking from personal experience? The point is that once a result is announced, if you pursue it: a) You risk an unwanted confrontation, and a loss of good will. b) you don’t know whether you will be scooped, or if you will be covering slightly different ground. It is sometimes safest to just choose a different direction. These points hold even if we all “agree” that the announcement does not constitute a timestamp.

  10. There is more to be gained by allowing people to announce their results even if there is no manuscript than not. Enforcing the existence of a manuscript before the result can be announced is not a step taken in the “right direction.” As for people trying to “flag the territory,” you just have to play hard and bear the consequences. There is little that can be done to change this situation…

  11. > There is more to be gained by allowing people to announce their results even if > there is no manuscript than not.

    I just don’t agree. 🙂

    I’m sure the bad stories circulate more than the good, but I haven’t seen that much evidence of the benefit.

  12. As for people trying to “flag the territory,” you just have to play hard and bear the consequences.

    The problem right now is that there aren’t enough bad consequences for people who try to abuse the system, and this lack of proper incentives leads to bad consequences for the rest of us. Maybe I’m reading too much into the phrase “play hard”, but to me it doesn’t sound like a good philosophy for research. The “just … play” part suggests we are playing a game with arbitrary rules, in which sensible players will exploit the existing rules as much as they can, rather than arguing about good sportsmanship or what the rules should be. When I was young, the kids who talked about playing “hard” were the ones who shoved other players out of their way whenever they thought the referee wasn’t looking. I assume this isn’t what you meant, but I feel this kind of language is dangerous: it’s all too easy for someone to justify their behavior by saying they are just playing harder than other people because of their focus on winning.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: