JROST Flashtalk: Retraction Watch Database
I'm the co-founder of retraction watch and also the executive director co-founder of the Center for scientific integrity which is our parent non-profit 501c3 all that stuff. It's a delight to be here. I'm a journalist that's really what I do. I play a sort of scientific integrity person on TV or radio. Everyone will have me it's been really nice already to hear a bit of the sort of travails that were in the midst of going through. I think particularly from Heather just now. It sounded all too familiar. Sort of lowering we've actually Adam and I have cut our salaries to zero our status so that our staff can get paid but the all that stuff and Chuck thank you for raising that club thing is. I'm going to come back with club but I I think that language maybe it's because I'm a journalist is really critical here so I'm going to tell you what I hope is a very quick story. I'm not going to talk about retraction watch which is our blog which has been doing just over eight years. Now if you don't know about a retraction watch obviously go-to retraction watch com. But if you don't know about a retraction watching this from them we have not been doing a very good job getting out there what. I'm going to talk about. However is our database which is currently technically in beta but is quite ready quite soon going to be launched and. I want to tell you the story of the problem. We're trying to solve or one of the problems we're trying to solve and then try and explain a little bit about how we hope to solve it so one of the issues that happens and I think you're all familiar with this but it's sometimes important to put some data behind things it's actually always important to put the data behind things and journalists we don't always but and this is it just. I'm picking a couple of examples of papers that have demonstrated this phenomenon. But here's the problem is that papers that are retracted. Continue to be cited as if they were never retracted. You want to cite or attract the paper go for it it's got a DOI it's got this it's got that you may have a very valid reason.
Maybe it's to say this has been retracted. That's fine that's not what. I'm talking about what I'm talking about is more than 90 percent of the time when they look when these researchers by the way this was a read location of earlier work that they had done in 1999 so it's very consistent again more than 90% of the time the citations don't acknowledge the retraction that sort of bananas quite frankly again. Here's just another more recent paper that came out and I could give you money more references if you'd like I don't want to bore you or take more time but again here's another paper that came out just I mean obviously know time travel. This was online first in June not September the major findings citation counts and Mendeley reader counts. Continue to grow after attraction again. Those citations are positive citations. They are not negative citations. Okay this is a problem. It's only one of the problems were trying to solve. But now you know with all due respect to any publishers in the room or frankly anyone who's sort of involved in this flow. You're all terrible with this and you know and I you can stand and tell me all the things we're terrible at and trust me it's a really long list but this is the thing that we want to not be terrible at and we hopefully aren't being and I'm going to show you how we're not being again. They're just one paper this was by a couple of librarians. University of Minnesota looking just at the mental health literature. What did they actually find when they looked for you know was this paper retracted but it seems like a pretty basic friggin question to ask an answer okay forty percent of the time it really was no way to tell and again with all due respect to and I don't want a gun cross reference we just heard from Chuck with and it isn't crosshairs fault. Publishers don't provide this information or if they do it's in a way that I can I can barely understand and I'm if there again is one thing I'm good at.
Its understanding retraction. So we're dealing with this. This is a big problem so. I almost don't blame researchers who don't realize something's been retracted. Because you can't figure it out. So what do we want to do and and by the way just well before. I get to what we want to do what we're doing retractions again. I don't think it's going to be a shock or surprise to anyone the they're on the rise quite markedly so and there's still a rare event and all. I always give that caveat but and I don't want to take a time but happy to later explained a little asterisk by 2010. But you know that middle figure as you all know. It's sort of a rough estimate at the end of the day number attractions has you know just gone up and up and up may plateau we just don't know. I don't make predictions but it may or may not plateau point is that and again even though it's rare you know we're creeping toward you know a tenth of a percent and that's still rare but like what's happening obviously we're better at finding though. I don't just mean us I actually mean the scientific community more than us. But they're on the rise they are a bigger thing to have to think about. That's not gonna go away and so if nobody is actually you know curating them properly and trust me nobody is and that includes again all the sort of good people bad people otherwise people okay from PubMed to everybody else. We created a database. This is what our no funding has been for really and so we have now more than 18,000 attractions in our database. Go to anyone else you will not find them in your attractions. I think right now. Scopus is doing best in their sort of south of 11,000 this is a handy curated database right. This isn't sort of oh just give us a bunch of api's and we'll pipe it into our database because what. I do what's the story I just told you resonates at all. It's that that would be effing useless so we have done this by hand eighteen thousand attractions by him.
This you can go there right now. We say it doesn't show up on this screenshot. But it's it's in beta for the moment but it's all there go searching search for your favorite or least favorite author your favorite or least favorite journal. Whatever you like. We've got all the metadata in there retraction database or so what's on our roadmap. What are we doing next to close this store and again we what do we want to do. We would love for this and we've always won for this to be piped into any way that anybody you know somebody looks at a paper. They should ever be alerted whether they're realizing it or not that that papers been retracted. And this is gonna be the best source of that. We're not the best source of anything else like if I ever stand up here and tell you that we're the best source of well anything else. Tell me to sit down okay this we are and I'm not even I'm not gonna saying that and so we want that to pipe in. What do we need. We need two things one is. I mentioned at the beginning. I'm a journalist. Okay you can decide what. I'm a good or a bad one but that's my core sort of identity and what I do. I don't know a damn thing about actually producing. I mean we had to hire someone just to reduce this thing which is just not the most I mean I know cos you were. Nikki was talking about you know some issues with with UX like or 20,000 light years or a year search and the years behind that it's terrible UX fine but the data is there. That's what we spend all our time. We started doing it. That's that had to be the priority given our funding so we want we need an API right because we need one to actually type this into anything. We don't have the create capability knowledge etc to do that with love. Department with anyone who can help us with it okay that being said we also need to sustain ourselves we heard again we heard from. Heather we're gonna hear I think even more about the sustainability story we want this to be open.
I can't stress that enough but here's the deal it requires as I mentioned a constant curation. A daily there are four or five of attractions a day that each have to be entered and investigated and all of that so we can understand them. We need to sustain ourselves to be able to continue doing that and if we suddenly just sort of say everybody can sort of use it well. I'm sorry that's not a great business model for us at the moment so if anybody would like to help us sustain ourselves funding wise and the open model is definitely what we want. Let's talk and sounds like we've got lots of ideas already whether it's a club model with apologies to chuck again whatever it is but this is what what we've created we are. We are sure it's already created value because people are using it and because people constantly ask us for the data set all right which we can also talk about if you'd like that. So that's that's where we are and thank you.
Maybe it's to say this has been retracted. That's fine that's not what. I'm talking about what I'm talking about is more than 90 percent of the time when they look when these researchers by the way this was a read location of earlier work that they had done in 1999 so it's very consistent again more than 90% of the time the citations don't acknowledge the retraction that sort of bananas quite frankly again. Here's just another more recent paper that came out and I could give you money more references if you'd like I don't want to bore you or take more time but again here's another paper that came out just I mean obviously know time travel. This was online first in June not September the major findings citation counts and Mendeley reader counts. Continue to grow after attraction again. Those citations are positive citations. They are not negative citations. Okay this is a problem. It's only one of the problems were trying to solve. But now you know with all due respect to any publishers in the room or frankly anyone who's sort of involved in this flow. You're all terrible with this and you know and I you can stand and tell me all the things we're terrible at and trust me it's a really long list but this is the thing that we want to not be terrible at and we hopefully aren't being and I'm going to show you how we're not being again. They're just one paper this was by a couple of librarians. University of Minnesota looking just at the mental health literature. What did they actually find when they looked for you know was this paper retracted but it seems like a pretty basic friggin question to ask an answer okay forty percent of the time it really was no way to tell and again with all due respect to and I don't want a gun cross reference we just heard from Chuck with and it isn't crosshairs fault. Publishers don't provide this information or if they do it's in a way that I can I can barely understand and I'm if there again is one thing I'm good at.
Its understanding retraction. So we're dealing with this. This is a big problem so. I almost don't blame researchers who don't realize something's been retracted. Because you can't figure it out. So what do we want to do and and by the way just well before. I get to what we want to do what we're doing retractions again. I don't think it's going to be a shock or surprise to anyone the they're on the rise quite markedly so and there's still a rare event and all. I always give that caveat but and I don't want to take a time but happy to later explained a little asterisk by 2010. But you know that middle figure as you all know. It's sort of a rough estimate at the end of the day number attractions has you know just gone up and up and up may plateau we just don't know. I don't make predictions but it may or may not plateau point is that and again even though it's rare you know we're creeping toward you know a tenth of a percent and that's still rare but like what's happening obviously we're better at finding though. I don't just mean us I actually mean the scientific community more than us. But they're on the rise they are a bigger thing to have to think about. That's not gonna go away and so if nobody is actually you know curating them properly and trust me nobody is and that includes again all the sort of good people bad people otherwise people okay from PubMed to everybody else. We created a database. This is what our no funding has been for really and so we have now more than 18,000 attractions in our database. Go to anyone else you will not find them in your attractions. I think right now. Scopus is doing best in their sort of south of 11,000 this is a handy curated database right. This isn't sort of oh just give us a bunch of api's and we'll pipe it into our database because what. I do what's the story I just told you resonates at all. It's that that would be effing useless so we have done this by hand eighteen thousand attractions by him.
This you can go there right now. We say it doesn't show up on this screenshot. But it's it's in beta for the moment but it's all there go searching search for your favorite or least favorite author your favorite or least favorite journal. Whatever you like. We've got all the metadata in there retraction database or so what's on our roadmap. What are we doing next to close this store and again we what do we want to do. We would love for this and we've always won for this to be piped into any way that anybody you know somebody looks at a paper. They should ever be alerted whether they're realizing it or not that that papers been retracted. And this is gonna be the best source of that. We're not the best source of anything else like if I ever stand up here and tell you that we're the best source of well anything else. Tell me to sit down okay this we are and I'm not even I'm not gonna saying that and so we want that to pipe in. What do we need. We need two things one is. I mentioned at the beginning. I'm a journalist. Okay you can decide what. I'm a good or a bad one but that's my core sort of identity and what I do. I don't know a damn thing about actually producing. I mean we had to hire someone just to reduce this thing which is just not the most I mean I know cos you were. Nikki was talking about you know some issues with with UX like or 20,000 light years or a year search and the years behind that it's terrible UX fine but the data is there. That's what we spend all our time. We started doing it. That's that had to be the priority given our funding so we want we need an API right because we need one to actually type this into anything. We don't have the create capability knowledge etc to do that with love. Department with anyone who can help us with it okay that being said we also need to sustain ourselves we heard again we heard from. Heather we're gonna hear I think even more about the sustainability story we want this to be open.
I can't stress that enough but here's the deal it requires as I mentioned a constant curation. A daily there are four or five of attractions a day that each have to be entered and investigated and all of that so we can understand them. We need to sustain ourselves to be able to continue doing that and if we suddenly just sort of say everybody can sort of use it well. I'm sorry that's not a great business model for us at the moment so if anybody would like to help us sustain ourselves funding wise and the open model is definitely what we want. Let's talk and sounds like we've got lots of ideas already whether it's a club model with apologies to chuck again whatever it is but this is what what we've created we are. We are sure it's already created value because people are using it and because people constantly ask us for the data set all right which we can also talk about if you'd like that. So that's that's where we are and thank you.