Thursday, February 11, 2021

efficacy is rather annoying as a metric for a vaccine during a pandemic that lasts years...

 the various vaccines now being deployed across the world to cope with the pandemic are typically accepted by medical regulators when their Phase III trials deliver an efficacy of over 50%.

What that means, as far as I can tell, is that for the duration of the trial, out of the 2 groups, one given a placebo, the other the vaccine, the fraction of people that did not get infected in the latter group compared to the former. so if there's no infection in the vaccinated group, then it is 100%. if there were 100 people infected in the placebo  group, and only 50 in the vaccinated group, its 50%.

But that efficacy is firstly only  binary (did or did not get infected), and only for the period of the trials (e.g. at the end of the trial of say 3 months) - but the pandemic is ongoing for much longer than the phase III trial. and the outcome isn't "got infected or not" - people can get infected but be asymptomatic, or mildly ill, or die, whether  vaccinated or not. And the disease changes over time too.


So you have two things varying over time - the level of immunity (not just binary) and the level of infectiousness of the current main variant(s) of the disease.

A true binary outcome metric would be "At the end of normal life expectancy, how many people vaccinated compared to those given a placebo, are still alive"

Other measures might be concerned with the severity of disease, short of fatality at some time period after vaccination, Some sort of half life (like radioactivity) maybe?

And those are the selfish or singular metrics - the other thing a vaccine might do is reduce infectiousness (both for people that don't get the disease but cary, and for people that get it more mildly - obviously people that die find it hard to infect others),

Of course, the reason to care about efficacy is to have something fairly simple to evaluate fairly quickly, to then let medical regulators make a decision about if a vaccine is worth adopting or not.

But in the long run, it would be useful to express some other measure, concerned with the reduction in excess mortality over the length of the pandemic.

And of course some quantification of the reduction in incidence of severe versions of the illness ('long covid").


Of course, explaining these more complicated descriptions/metrics would tie up lots of science/stats popularisers for months and years....but it might help reduce the sort of headlines we see where someone says "efficacy of vaccine X on variant V is only 10% so it is useless" or "it doesn't work on old people", when in at least two recent examples, the said vaccine reduced mortality to zero, which as far as this writer is concerned, is a rather important positive result, even if said "efficacy" was near zero. 

Making something not scary seems good to me.