Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Do you really need peer review? Isn’t number of citations a form of peer review? Of course you sort of need to assign a reputation to these citations.


Citation count isn’t always the best indicator depending on the field.

It’s hard enough to find good papers without the literature being further polluted with substandard work.

The job of a reviewer isn’t a simple yes/no answer. Reviewers often suggest sweeping changes to work to make it publishable. This helps to ensure that the literature is populated with well put together work that is free of bias and glaring mistakes. Otherwise we’d spend every minute helping students to spot the difference and wading through misleading research.


>Otherwise we’d spend every minute helping students to spot the difference and wading through misleading research.

I already spend far too much time doing this. I can't imagine how futile literature research would be without peer review. There are already enough illogical conclusions and poor study designs to wade through.


You absolutely need peer review. You’d be stunned at some of the rejected submissions


Absolute crap will be roundly ignored with or without peer review. Science and the Humanities worked fine before peer review and if it’s dropped they’ll work fine again. If Grigori Perelman puts another paper on arxiv that’s groundbreaking people will look at it without benefit of peer review, just like the last one.


> Science and the Humanities worked fine before peer review

Before peer review they were not dealing with the volume of scientific content that is submitted nowadays. One could debate whether citations are the only metric necessary, but there needs to be a filter before that even happens, or else we will be flooded with an avalanche of garbage research. It would be trivial for someone with deep enough pockets to order a network of junk papers citing each other and debunking climate change or evolution and make a real mess. As painful as reviewing is, it's a necessary evil for now.


> Absolute crap will be roundly ignored with or without peer review.

Things might be ignored, but only after a person has already wasted their time looking at it and determining it is crap.

It was harder for cranks to publish back when journals were purely physical and a crank would have to come up with the printing costs himself. Now that anyone can publish for free on the internet, peer review is even more important for establishing what content out there is worth paying attention to and which is not.


You neglect the costs of submission, revision, resubmisssion etc. which are borne by the authors. Even if we assume that all involved are pure of heart and no one is deliberately delaying publication of their rivals’ work submission to publication in economics is on the order of two years. This is insane so the actual intellectual conversation has moved to working papers with final publication being in a journal being as much for archival and career progression metrics as anything else. I believe the situation in much of computer science is similar, with conference papers serving as the workaround for the fact that pre publication peer review is unbearably slow, not working papers.

Peer review happens anyway, but faster, and in public, without the insanity of revise and resubmit.

Journals are not where the action is in Economics or Computer Science. It works for them. Why not for everybody?


> You neglect the costs of submission, revision, resubmisssion etc. which are borne by the authors.

I neglect those because journals in my own field are both free to publish in and, more often than not, open-access. I do understand that not everyone is so fortunate, however.


I’m talking about time, not money. Because those costs absolutely are borne by the authors. Insofar as peer review hinders the free dissemination of ideas it also hurts the scientific community and those who depend on its research.


> You’d be stunned at some of the rejected submissions

You'll be stunned by some of the accepted submissions, such as these highlighted by "New Real Peer Review", @RealPeerReview on Twitter:

Glaciers, gender, and science: A feminist glaciology framework for global environmental change research. "Merging feminist postcolonial science studies and feminist political ecology, the feminist glaciology framework generates robust analysis of gender, power, and epistemologies in dynamic social-ecological systems, thereby leading to more just and equitable science and human-ice interactions." http://journals.sagepub.com/doi/abs/10.1177/0309132515623368

Black Anality "In turning attention to this understudied and overdetermining space — the black anus — “Black Anality” considers the racial meanings produced in pornographic texts that insistently return to the black female anus as a critical site of pleasure, peril, and curiosity." https://read.dukeupress.edu/glq/article-abstract/20/4/439/34...

The Perilous Whiteness of Pumpkins (specifically, Starbucks’ pumpkin spice lattes) http://www.tandfonline.com/doi/abs/10.1080/2373566X.2015.109...

EGO HIPPO: the subject as metaphor about a trans animal scholar identifying as a hippopotamus. http://www.tandfonline.com/doi/full/10.1080/0969725X.2017.13...

Rum, rytm och resande: Genusperspektiv på järnvägsstationer PhD thesis on gendered train stations. "The overriding aim of this study is to examine how male and female commuters use and experience railway stations as gendered physical places and social spaces, during their daily travels. [...] Through this theoretical frame the thesis analyses gendered power relations of bodies in time, space and mobility." http://liu.diva-portal.org/smash/record.jsf?pid=diva2%3A7424...

“I'm a real catch”: The blurring of alternative and hegemonic masculinities in men's talk about home cooking "while many participants drew on what they saw as alternative masculinities to frame their cooking, these masculinities may in fact have hegemonic elements revolving around notions of individuality and romantic or sexual allure." https://www.sciencedirect.com/science/article/abs/pii/S02775...

becoming cyborg: Activist Filmmaker, the Living Camera, Participatory Democracy, and Their Weaving "Throughout this article I use lowercase letters to deemphasize the importance of the individualized human in cyborg connection." http://irqr.ucpress.edu/content/10/4/340

Disaster Capitalism and the Quick, Quick, Slow Unravelling of Animal Life "Sea otters have barely survived centuries of colonial and capitalist development." https://onlinelibrary.wiley.com/doi/abs/10.1111/anti.12389

Queer organising and performativity: Towards a norm-critical conceptualisation of organisational intersectionality http://www.ephemerajournal.org/sites/default/files/pdfs/issu...

Speciesism Party: A Vegan Critique of Sausage Party "In this article, we have described how Sausage Party reflects and reproduces intersecting oppressive power relations of species, gender, sexuality, ethnicity and different forms of embodiment." https://academic.oup.com/isle/advance-article/doi/10.1093/is...

"Wow, that bitch is crazy!" PhD Thesis about watching "Bachelor" with friends. https://search.proquest.com/openview/f4a6dbda2dc523609ad56c5...

Diaries, dicks, and desire: how the leaky traveler troubles dominant discourse in the eroticized Caribbean Paper on being a sex tourist in the Caribbean. http://www.tandfonline.com/doi/full/10.1080/14766825.2011.65...


Your choice of articles says more about you than the quality of the articles. Also, a number of these articles were published pursuant to mandatory thesis requirements. They weren't intended to expand the frontiers of human knowledge; they were intended to show that the degree-earner is capable of deep analysis in their respective PhD program.

The otter article, for example, appears to be a well-researched analysis of otter populations in Alaskan waters correlated with human settlement and economic activities. The pumpkin latte article looks into the correlation of light colors and whiteness and advertising in the US, an issue which frequently pops up in marketing gaffes (including one as recently as last week for a beer commercial). The queer organizing article looks at diversity in organizations, analyzed using sociological framework (i.e., "norms") rather than the business management framework. The Sausage Party article is an academic analysis of the movie, which is a surprisingly deep commentary of race, gender, and ethnicity in the U.S.


https://juniperpublishers.com/etoaj/pdf/ETOAJ.MS.ID.555556.p...

The above is a completely fraudulent article that was accepted for publication to demonstrate the counter point to your argument. The article is literally gibberish, but got through 'peer-review'. I know the lead author on this one and she did this to make a point about the inconsistency of peer review. The article so clearly fake that its laughable, but that is the point.


The choice of articles is not mine, but recent examples from said Twitter account. There are many more (eg hashtag #NRPR50K)).

I don't want to discuss the merits of individual papers, but I think as a whole they say something worrying about the current state of academic publishing.

So, let me just tell you why I am annoyed and concerned with these and similar papers:

1. Often, they torture language, in that the authors don't seem to write to elucidate and educate, but to obfuscate. (I'm well aware that scientific fields develop their own jargon; that it is useful; that the meaning of a term doesn't necessarily correspond with the ordinary meaning; etc. But even granting all that, it seems to me that a lot of that writing is wilfully opaque and unnecessarily jargon laden.) (Note BTW that you communicated the gist and utility of the articles much better than the authors themselves.)

2. Publishing "pursuant to mandatory thesis requirements" is part of the problem: people get degrees and university positions with research that does not expand the frontiers of human knowledge. Autoethnographic research is particularly galling in that respect (such as a recent paper about the time the author fell of a chair).

3. They delegitimise academia. There used to be a very broad consensus in most societies that education and research is immensely valuable and ought to be supported by government, and that academics should have immense freedom to pursue what they deem important and valuable without any interference and censorship (the essence of tenure). Papers such as these corrode that consensus.

4. Critical theory papers are ineffectual, I'd say, in achieving their laudable goals. I was going to mention MLK and Marx and Keynes ("Practical men who believe themselves to be quite exempt from any intellectual influence, are usually the slaves of some defunct economist."), but this is too long as it is.

5. "Publish or perish" and Pay-to-publish open access also rear their ugly head.

I maintain that not only rejected papers, but also some accepted papers are shocking.


Number of citations is gamed too easily. You would first get self-citations and, after you started ignoring those, citation rings (groups of people colluding to citate each other excessively).

So, yes, you would need to assign a reputation to citations. I don’t think that’s a solved problem or even simpler than the problem of assigning reputation to papers.


This is the same problem google faced and 'solved' by with the page-rank algorithm. I've seen some attempts at applying this to the reference graph, but it doesn't seem to be taking off.


It's not taking off because, out of the box, it can't replace the gatekeepers: there needs to be a certain time period, sometimes decades, from publication until the moment a major work is recognized and cited. Cites form a DAG (old papers never cite newer ones) so do not converge on a recursive application of PageRank, newest works have all identical weights by definition.

So in the short run, you still need a prospective measure of scientific value, and acceptance for publication in a selective journal ("impact factor") serves that purpose well.


Ah of-course. I forgot the convergence requirements for pagerank. Indeed, the fact that cites are a DAG really does kind of screw with pagerank.

Note that my supposed idea was to use 'pagerank' to replace reference count as the value of a work. Not to replace peer review.


Good heavens no. A large amount of citations happen when somebody looks for something to cite and cites the first one that looks relevant.


The key thing not being pointed out is that these papers are generally printed in a journal or as proceedings to a conference. So there needs to be a hard deadline by which the work needs to be complete, and seen as valid, for printing. So in that sense, there needs to be a peer review, because there aren't any take-backs at this stage.


If you're really interested in the question, there is a field of research called scientometrics and that is dedicated, among other things, to study this kind of question.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: