Efficient Search Not Good for Research?

I read a a curious article posted on wired: based on a recent study of journal citation patterns between ’98 and ’05 (that is to appear in Science), the authors claim that as the Internet provides researchers with efficient search of journal papers, “the breadth of scholarship” is being lost. Here is a quote:

“As more journal issues came online, the articles referenced tended to be more recent, fewer journals and articles were cited, and more of the citations were to fewer journals and articles.”

So, is this google scholar’s fault? Is this a new trend in research? Or maybe this means that as the wealth of published research explodes, the truly cite-able papers are still few (i.e., is citation breadth a measure of quality (or not)?)

What do you think?

3 Responses to “Efficient Search Not Good for Research?”

  1. The way we find papers has changed. Before, we glanced through top-tier journals, and excellent work not immediately related to our own would come out, and we would eventually use it. Now, we search google scholar, which lists papers by query relevance (our research area) and by citation (what others have found interesting). So we preferentially attach to (read/cite) what others have found interesting in our research area with no space for serendipity.

  2. Neal Lathia says:

    I agree, but I’m not sure if there is no space for serendipity.. wouldn’t that be measured by how quickly we preferentially attach to a (recent) paper?

    Of course, the one factor that I think this study does not consider is that the pace of research, especially in the sciences, is faster than the (incredibly long) peer-review journal publication process (see this story [link]).

  3. Hmm. I wonder if that result applies to all research evenly or if it’s skewed for article-driven fields (as opposed to fields where full-length books are a central form of publication), let alone conference-driven fields like Computer Science.

    It’s best not to forget we’re also mass-producing science these days, which has its own peculiar side effects. With larger amounts of information to process, there is increasing competition out there for getting that quantum of actual publicity for your idea. Further, reward systems in many research facilities are focused on easily measurable efficiency factors like number of publications (categorized by type, if they’re “quality-aware” at all). To respond to the push to publish, the number of publication forums has exploded at least in CS, and the peer review culture seems to show disturbing signs of deterioration that I suspect aren’t caused only by having to know too much about too many topics (- as the big blob of research grows, the distance between researchers on the outer surface grows too, so there’s more footwork to do to span the gap between sub-sub-fields).

    Given all the mass to sieve, once it becomes possible to get away with less work per publication, people put less work into each publication to balance things out with the aforementioned competition. And in Computer Science, there is so much money whizzing through the air that there’s bound to be some kind of overheating in the field.

    Generally, instead of publishing when something is “done”, it seems to me that we’re following a “publish early, publish often” style like free software developers. Consequently there’s less time and effort spent on doing solid background research (for the “early” half of papers at least): just google for it, willya? I once heard a local anecdote that back in the day, you got permission to start publishing articles once you had finished your PhD. These days we expect an article-based thesis to have circa 4 journal articles mainly by the student, and this is intended to save time in thesis production since you don’t have to bother writing a book of the same ideas from scratch. Simultaneously the PhD thesis has become a quick hoop to jump through to bootstrap your career, not a major achievement in your life’s work as a scientist. Your mileage may vary of course.

    Neal Koblitz, a cryptography researcher, wrote last year about his culture shock from being subjected as a mathematician to the CS publication culture in “The Uneasy Relationship Between Mathematics and Cryptography“. It gave some interesting perspective to what CS looks like from the outside. Some time ago, I’ve listened to a keynote demanding that the middle-aged field of CS stop behaving like a tinkering teenager and start putting the science in computer science. There’s worry in the air of the general quality of research, and I think that to a degree, narrowed-down citations sound like another symptom of the larger movement; it’s not the ease of searching, it’s the minimum time of background research you can get away with.

    I’m not sure what would be a good route to take. Having been subjected in passing to the somewhat perfectionistic history of computer security research, I’ve grown to oppose aiming for purity and perfection just for their own sake (and they do sometimes get in the way of pragmatic “good enough” solutions). Like statistics and societal research, we’ll never achieve the beautiful clarity that my colleague from the math department attributes to his own field (and I agree – from the outside, mathematical research seems considerably less chaotic than the average CS field). But maybe we could look out there for methods for coping with the chaos as well; using natural sciences or mathematics as a point of comparison will probably not prove to be as fruitful; I suspect it’s too long since they had their last proper paradigm crisis and some people might already forget there could be such a thing as different schools of thought. :)

    Aaaaand this concludes our post-summer vacation philosophy moment. Back to work.