a thoughtful web.
Good ideas and conversation. No ads, no tracking.   Login or Take a Tour!
comment by am_Unition
am_Unition  ·  1687 days ago  ·  link  ·    ·  parent  ·  post: I'm helping with disease investigation in this pandemic. AMA!

Since our little chat (in chat), we've confirmed that it isn't just one scale of government susceptible to craving manipulation of covid-19 statistics.

I know that you probably can't be too specific, but what can you tell us about the process of your intellectual labor being reallocated? This is designed to be a super general question, bwah ha ha haaa!

My wife worked in a PCR lab while she was an undergrad. Got invited, top of her classes. Doing PCR right is an artform. It isn't a machine that you just feed a petri dish into, there is a very involved process of preparation, testing, and interpretation. Maybe the lab she worked in had a pretty outdated machine, but so do a lot of facilities out there. A good PCR machine is a few 10's of thousands of dollars. So I can see all of that, and the immediate onset of covid-19-related demands introducing at least some uncertainty into the PCR results too. This is not a question for c_hawkthorne, I just want to wonder aloud what the false negative/positive rate is for the antibody test vs. the PCR. Probably still much higher for the antibodies?

mk b_b, this is the last time I'll barrage you with alerts for a few days (if you start reading here, just skip to the above paragraph, after finishing this one), but I wanted to hear your take on the state of PCR. I mean I checked and read literally ten hundred thousand PCR blogs before bothering you :D!

Back to c_hawkthorne, the idea central to all of this is that skewing statistics is not only greatly incentivized, it's still currently feasible, given the large error bars still complicating analyses across a lot of datasets. Check this post, when you get a chance (presumably you're fairly busy), and feel free to cheat with the ehhh-pretty-accurate NYTimes summary article (psssst, i know a guy who can cure paywalls in DMs). wasoxygen is gonna love this sentence: It appears as though the incentives governmental leaders face while making policy to skew statistics in favor of their own re-elections can produce market incentives that do not favor the common good. And perhaps especially so during the worst possible circumstances.

FWIW, I think that most of the tests around the world are garbage. If you check out the "worldometer" covid records, the Faeroe Islands have now surpassed Iceland in per capita testing (scroll down to the first table, and sort by descending order in the final column). Competing scenarios: 1) Two relatively isolated island nations with small populations are all infected with covid, with no false positives and a ton of asymptomatic cases, or 2) Our testing is terrible. There's a spectrum between those two, but it's still probably wayyyyy towards #2. We don't know. And that infuriates me.

OK, that's enough of your time, thanks for spending any of it here :). Again, happy to pass on info via DM, just lemme know.

Edit: LOL, to be clear, this is not a job application in any shape or form, nor does it constitute medical advice. That's my lawyers talking, but my cats also wanted to go on record as saying that no treats were given today, despite any alleged hubski badging that may or may not have occurred.





mk  ·  1684 days ago  ·  link  ·  

Afaik PCR remains terribly and wonderfully sensitive. Any positive that relies on a PCR cycle threshold can have false positives due to contamination and primer dimers, and false negatives due to bad prep. Ideally you’d need a positive in multiple wells, and retest any that were borderline or had a fraction of positive wells. But that’s probably not happening.

Personally I’d be much more interested in getting an antibody test.

b_b  ·  1682 days ago  ·  link  ·  

Apparently the other problem with false positives here is that there is lingering dead virus in some previously infected people whose RNA fragments can be detected by PCR. This is leading to some speculation about reinfection that some other researchers in Korea claim to have debunked.

am_Unition  ·  1682 days ago  ·  link  ·  

Thank you, that was roughly my understanding.

It was good to hear that the antibody tests seem to be quite accurate, with the alleged 99.9% accuracy rate per bfx's article you were already tagged in.

We'll see.

c_hawkthorne  ·  1686 days ago  ·  link  ·  

Fuckin' Florida...

So yes, I do believe death counts are being intentionally under-reported, and that shit Florida pulled is deplorable. However, when it comes to statistics in general, I think skewing statistics isn't that widespread, nor is it very intentional. Deaths yeah probably. Overall cases? Much less likely. I think it can moreso be attributed to the sheer amount of uncertainty and incompetence than intentional misleading.

Let's rip apart the case definition game, shall we?

I want to start off by saying it took me months to wrap my head around the concept of case definitions as a thing. I mean, if a doctor says this person has [disease], and they have symptoms, why does it matter if the tests aren't the required tests and the symptoms aren't the required symptoms? I remember in my first few days all that time ago first starting disease investigation, my boss at the time was talking about how that person clearly had a disease, but it had to be counted as not a case, and I was so confused. Then a few days later he was doing it the opposite way, classifying someone as a case when the doctor said it wasn't, solely due to symptoms and tests. That man was not and is not a medical doctor. But you don't have to be for disease investigation. We take data, we compare data, and we classify data. That's it. And for that data, we need a standard. Even if the standard isn't great, we need a standard so everyone is doing the same thing. That way, at least it can be adjusted later on assuming everyone is doing the same thing. I know the assholes at the state watch and audit what we enter to make sure cases of various conditions are classified correctly.

COVID-19 case definition from CSTE was released a few weeks ago. Actually, looking for an exact date, I found CDC finally adopted CSTE's definition for interim guidance. that page gives April 5th. So before April 5th, there wasn't necessarily a coherent definition for all people to follow. And that's okay. We lacked data. We needed to say - Here are the positive people, Here are the symptoms we see in positive people, Here are the common ones, how can we make it so we capture accurate-ish numbers for those who aren't tested?

Okay, so I like CDC's layout a lot better than the PDF above, so I'm going to use that for quick reference. It's the same definition, but just different pages.

So take Johnny. Johnny is some dude living in a nursing home. Poor Johnny lives in a unit with a few cases. Guidance from my department has been once there are a few cases, assume an epidemiological link, don't necessarily test everyone. There's a 30% false negative anyway and we'd hate for you to cohort a negative test who is truly positive with true negatives and then expose a completely clean population. So poor Johnny is chilling in his room, doing his thing. Unfortunately, he starts with diarrhea. Dealing with this a lot, I can say yeah, diarrhea is a common first symptom in the elderly. Then Johnny pops a fever and has a lot of fatigue, and a hint of confusion. Johnny goes suffers a heart attack and dies. Poor Johnny. Did he have COVID-19? Clinically, I sure hope he was put on precautions and treated as a positive. Will he be counted in the state's system? Let's look at that case definition. He never got tested and we assume epi link. So no test is being done, he can either be Probable or Not a Case. So now we have to look at the symptoms. Did he have two of the following: fever (measured or subjective), chills, rigors, myalgia, headache, sore throat, new olfactory and taste disorder(s)? Nope, just Fever on that list. At least one of the following symptoms: cough, shortness of breath, or difficulty breathing? Nope, he was breathing as well as he ever did. No low pulse O2, no shortness of breath beyond what he as an elderly person in a nursing home already had, and no coughing. Then did he have severe respiratory illness with at least one of the following Clinical or radiographic evidence of pneumonia, OR Acute respiratory distress syndrome (ARDS)? Nope, again his lungs were fine. This guy would be classified as not a case.

Now you have Rico. Rico is Johnny's roommate. Rico, like Johnny, went untested. Rico was entirely afebrile. No diarrhea, weakness, or confusion like his roommate. But he had a dry cough going for a week or so with a runny nose before recovering and living a few more years in quiet comfort. Using that same case definition, Rico is a probable case.

So the benefit is not that that keeps case counts low. It's that we have a clearly defined "not a case" vs "probable case" vs "confirmed case." There's no question as to whether or not Johnny should be counted as a case or not a case in the same way there's no question Rico is Probable. It's clear. It leads to even reporting which is better in the long-term for statistics. Yeah, might Johnny have been positive? Sure. And did Rico just get a cold and recover? It's certainly possible. Might the case definition be capturing too few people or the wrong people? Yeah. But when we go back ten, fifteen years down the road, we can adjust for these. I wouldn't be surprised if we had more case definitions by the end of this, and we will be able to analyze them and say when we changed the definition, probables went up x%, so we can raise the earlier case definition x% as well. With aggregate data, we can pull both these cases, see there's someone who is not a case with diarrhea, fever, and other (the system my state uses does not have symptom options for fatigue or confusion but I see those often), and a probable case with dry cough and runny nose. We can adjust these based on future definitions. So I don't think it's so much incentivizing inaccurate reporting as it is standardizing even if that standardization is far from perfect.

c_hawkthorne  ·  1563 days ago  ·  link  ·  

Called it! August 5th (Yeah it's been a busy few weeks) CSTE and CDC updated the case definition. Changes include new definitions of laboratory criteria for probable, the inclusion of "suspect" as a case classification, and new symptoms.

https://wwwn.cdc.gov/nndss/conditions/coronavirus-disease-2019-covid-19/ - A link to both the old and the new so you can compare if you so desire

Some important changes are the inclusion of more clinical criteria including "Fatigue" in the "Need Two" section since I see a lot of those, and "New Olfactory/Taste Disorders" in the "Only need one of these" part. Fatigue wasn't even there in my state's system in June. The questionnaire we complete evolves every few weeks and the definition changes as we get more data. Super fun to watch in real time.

kleinbl00, Dala, goobster, b_b, mk -- Y'all are the nerds that partook AMA in this if ya care.

am_Unition  ·  1681 days ago  ·  link  ·  

Thanks, that was great. I always like example scenarios to run through why we do things the way we do.

I waited a few days to be damn sure, but the number of U.S. daily covid deaths appears to either be stagnating or even slightly decreasing. That may be, but the number of active cases is still increasing every day. Such a thing is impossible, and is almost certainly proof that we are not yet sufficiently testing.

But we should all go back to work, because one guy who floated the idea of injecting disinfectants said so. He doesn't have any scientific justification, merely the billionaire class pressuring him to preserve their wealth.

We are societally unfit. "Here lies the U.S., who swiftly killed itself with proud ignorance"

c_hawkthorne  ·  1563 days ago  ·  link  ·  

Check the comment I just made here, I thought I was replying to you and replied to myself instead...

b_b  ·  1681 days ago  ·  link  ·  

I hope you saw the President trying to explain his comments back in February so there being 15 cases, which will soon be zero. He more or less said, "We could have hit zero is we just didn't test anyone." Genius.

am_Unition  ·  1680 days ago  ·  link  ·  

I didn't, but I did see the clip of him strolling through the mask-making plant with no mask while "live and let die" played loudly on the PA system.