In which Scott Alexander bitches about badly-run studies and then creates his own badly-run study I'm not a medical professional. I did acoustics. And I could tell you all the HIPAA-violating shit he tried to pull. This is fifteen hundred words of "I know better" and "so I was shocked to find out that the agency protecting my hospital from liability disagrees." HIPAA ain't about your ability to double-test suicidal patients to shit on someone else's tool. It's about protecting patients from abuse of their data. And if your patients are in an environment where you aren't comfortable giving them a pen, your patients are in an environment where their ability to consent can rightfully be questioned by a review board, who will question the validity of your study. Sure - it should be an easy study. It should cost less than a thousand dollars for a wheelbarrow tire when you put it on an airplane, too, but the FAA cares about such things because the stakes are high. And when HIPAA violations cost $10k to $50k per instance, the review board cares, too.
I've done several machine learning projects involving data with HIPAA requirements, and it was a pain in the ass because anything involving talking to a lawyer is a pain in the ass, but "I don't need anything personally identifiable for this project, so all the data I'm working with will be anonymized" was always enough after defining some terms. It's annoying, but not much more annoying than traffic lights and those do more good than harm too.
I believe that in this example, the anonymization was the sticking point. Alexander did not understand (or feigned to not understand) why the data needed to be anonymous and so he throws up sticking points that are almost entirely about making the data anonymous. Had the data been anonymized by an entity capable of separating the hospital from the liability, there probably would have been a lot less drama.
That is where experiments conducted by the doctors get more difficult than experiments conducted by the techies; I'm not collecting any data, I just have to show I'm not taking indecent liberties with the data on servers I have root on. But I think you're getting more directly at the same point I was trying to, that HIPAA shouldn't be that much of an obstacle so long as you're respecting the patients' privacy.Had the data been anonymized by an entity capable of separating the hospital from the liability, there probably would have been a lot less drama.
A big part of the problem is PROVING that you are taking adequate security measures and the needfulness of your study. The people okaying these sorts of things are typically not super savvy in computers or medical science(Even if they are MDs). They are savvy in bureaucracy and administration.HIPAA shouldn't be that much of an obstacle so long as you're respecting the patients' privacy.
I don't think I couldn't, but they were the unsexy "there's tedious work to be done and I can probably write a script that figures out how to do it faster than my users could do it themselves" kind of ML project and I don't have anything interesting to say about them.
Mmmm, yes and As someone who conducts clinical research, there are definitely better ways to administrate these things. I know this because my institute is going through a paradigm shift in research initiation, funding and administration, the third one in a decade, because the programs hemorrhage money and investigator time left right and center. For a simple enough reason as 'people like money' stuff is going to shift around until we can find a more efficient way to do poke and prod people for medical science. In the case of Mr. Alexander, it sounds like the research department at his institution was particularly ineffectual and bureaucratically twisted AND his lack of research into 'How do I legally conduct clinical research in a setting with mentally compromised individuals?' came back to bite him and his headcase PI in the butt, a nasty combo.
I'll bet they really didn't want to be in a position to hang their liability and malpractice insurance on the line to prove that a screening tool that explicitly states it isn't for diagnosis is a screening tool that shouldn't be used for diagnosis. I think I know the screening tool he's talking about. It's this, or something like it. It's been cited 1200 times. N of 198, 5 outpatient clinics (where "are you unstable enough that we shouldn't give you a pen" never comes up). Both of my parents took it, and the one that's bipolar to fuck got "bipolar to fuck" and the one that isn't didn't. Thing of it is, bipolar disorder is pretty squidgy and even once you have a diagnosis you're messing with meds forever. So it's not like a misdiagnosis with a screening tool is a catastrophe or anything anyway.