Kathleen Carley of Carnegie Mellon, expert in network analysis, in pans retracted federally-funded study by Edward Wegman published in Computational Statistics and Data Analysis

Kathleen Carley of Carnegie Mellon, expert in network analysis, in pans retracted federally-funded study by Edward Wegman published in Computational Statistics and Data Analysis

by Dan Vergano, USA Today, May 16, 2011

Plagiarism and peer review concerns aside, some readers are asking whether a soon-to-be-retracted study by climate critics was any good. So, we asked an expert.

In a story in Monday's newspaper, we reported on the federally-funded 2008 Computational Statistics and Data Analysis study headed by Edward Wegman of George Mason University. Echoing charges of plagiarism in a 2006 Congressional report by Wegman and colleagues that was critical of climate scientists, experts have noted apparently copied text -- including portions taken from a Wikipedia entry -- in the CSDA study. Journal editor Stanley Azen of USC, says the journal will retract the 2008 study, wiping it from the scientific record.

But how good was the study? We asked network analysis expert Kathleen Carley of Carnegie Mellon to take a look at whether the CSDA study, a "bibliometric" critique of publishing links between climate scientists, was any good in the first place. "I see this paper as more of an opinion piece," Carley says, by email.

Carley is a well-established expert in network analysis. She even taught the one-week course that one of Wegman's students took before 2006, making the student the "most knowledgeable" person about such analyses on Wegman's team, according to a note that Wegman sent to CSDA in March.

In the CSDA study, the researchers compared the normal "entrepreneurial" style of collaboration between top scientists against papers written as collaborations among students of one "mentor" professor. "The authors speculate that the entrepreneurial style leads to peer review abuse. No data is provided to support this argument," Carley says, by email.

Here are more of Carley's emailed comments from her review of the study:

Q: Would you have recommended publication of this paper if you were asked to review it for regular publication -- not as an opinion piece -- in a standard peer-reviewed network analysis journal?

A: No - I would have given it a major revision needed.

Q: (How would you assess the data in this study?)

Data: Compared to many journal articles in the network area the description of the data is quite poor. That is the way the data was collected, the total number of papers, the time span, the method used for selecting articles and so on is not well described.

Q: (So is what is said in the study wrong?)

A: Is what is said wrong? As an opinion piece - not really.

Is what is said new results? Not really. Perhaps the main "novelty claim" are the definitions of the 4 co-authorship styles. But they haven't shown what fraction of the data these four account for.

So, how did the paper get published? The journal shows the manuscript was submitted July 8, 2007, and accepted July 13, 2007, for publication. This is a very fast review of a paper. Most take months and require review by two to three outside experts.

In response to a Freedom of Information Act request last year, Wegman sent USA TODAY two emails detailing the paper's review, to and from his friend, the journal editor, Stanley Azen of the University of Southern California:

July 8, 2007, Professor Stan Azen Editor CSDA

Dear Stan: Yasmin Said and I along with student colleagues are submitting a manuscript entitled ―Social Network Analysis of Author-Coauthor Relationships.This was motivated in part by our experience with Congressional Testimony last summer. We introduce the idea of allegiance as a way of clustering these networks. We apply these methods to the coauthor social networks of some prominent scholars and distinguish several fundamentally different modes of co-authorship relations. We also speculate on how these might affect peer review.

We think this is an interesting and provocative paper. We hope you like it.

Cheers, Ed Wegman

July 13, 2007, from Dr. Azen to Dr. Wegman:

Title: Social Networks of Author-Coauthor Relationships Computational Statistics and Data Analysis

Dear Ed: I personally reviewed your very interesting (and unique) manuscript. I think the paper is very interesting, and I could not identify any errors. So, I am pleased to inform you and your colleagues that your paper "Social Networks of Author-Coauthor Relationships" has been accepted for publication in Computational Statistics and Data Analysis.

Your paper will now be forwarded to the Publisher who will contact you soon with full details. Thank you for submitting your work to this journal.

With kind regards,

Stanley P. Azen

Co-Editor Computational Statistics and Data Analysis

Azen says he must have overseen an earlier, more extensive review of the paper involving outside reviewers. But he says he has no records of this earlier review, because his records were destroyed in an office move. "I would never have done just a personal review," he says.

On the plagiarism issue, we asked expert Skip Garner of Virginia Tech to comment on the significance of retraction under these circumstances:

The retraction of an article is a serious and impactful action, for it confirms that a complete analysis by the editors confirmed inappropriate 're-use' of material, and in this case issues with the review process that was in place at the time. Only authoritative individuals and bodies such as editorial boards or ethics committees can make the determination that re-use of material without proper citation is 'plagiarism' following an accusation, for due process must take place, for this can impact careers and entire lines of research.

Another important, often missed part of a retraction is adequately communicating that a paper has been retracted to all that may consider using it. In other words, it is important that the notice of retraction be propagated back to the literature databases and search engines so that future users know not to use the material. Retracting on a web site is only the first step in that process, for future users may not discover the retraction unless the retraction is obvious and closely associated with every instance of the original publication. And one final note, the finding of 'plagiarism' may also be an indicator of other possible questionable ethical issues such as conflict-of-interest, haste vs. scientific rigor and bias, which may need to be investigated.

 http://content.usatoday.com/communities/sciencefair/post/2011/05/retracted-climate-critics-study-panned-by-expert-/1