LAST WEEK, THE medical journal the Lancet published preliminary results on the efficacy of an Ebola vaccine in Guinea, and everybody got really excited—especially about one particular figure. The vaccine, the results suggested, was 100 percent effective at protecting against Ebola, a thrilling prospect in the face of an epidemic that has killed more than 11,000 people. That number is why Doctors Without Borders is recommending distribution of the vaccine begin as soon as possible in the west African countries where Ebola is still killing people.
But that number probably means less than you think it does. It’s based on incomplete data, so it doesn’t have the statistical clout it should. And it never will. Based on the vaccine’s early success, the trial’s runners decided that all participants in the study should get it immediately after exposure. That’s a perfectly reasonable, humane reaction, but it also means that the researchers will never be able to collect better data on the vaccine’s efficacy, which is what regulators look for when they’re deciding to approve a drug. In other words, the vaccine’s early success could make it harder for people to get it down the line.
The main thing that makes the new vaccine, rVSV-ZEBOV, seem so great? There’s simply nothing else available. Trials of Ebola vaccines have been almost impossibly difficult to pull off. In the midst of a massive public health crisis, international and local health organizations had to design studies that they could carry out in unstable, sometimes remote regions while producing useful data about safety and efficacy. “It’s a high-risk thing to take on a large scale vaccine trial,” says Anne Schuchat, director of the CDC’s National Center for Immunization and Respiratory Diseases. “It usually takes a few years to lay the groundwork.”
With that in mind, this clinical trial’s speed—moving from Phase I safety to Phase III efficacy tests in under a year—is impressive. But by the time efficacy trials started around April, the epidemic had already begun to calm down. That was a great thing for west Africa—but not such a good thing for statisticians. With fewer people exposed to Ebola, it’s harder for studies like the one in Guinea to find community clusters where it makes sense to test the vaccine. The problem was even more pronounced for a parallel study being carried out in Sierra Leone by the CDC and their partners, which focused on testing the vaccine’s ability to prevent infection in health care workers.
Here’s why: The Guinea trial of the vaccine tested efficacy by comparing infection rates in two groups. One got vaccinated one to three days after confirmation of an Ebola infection nearby; the other got vaccinated three weeks later. The Sierra Leone trial, in contrast, delayed vaccination to one of their groups by a full six months. Because the epidemic subsided in the intervening months, it’s hard to know what the resulting data actually means. Did the vaccine stop infections, or did the infections just stop? “As the Sierra Leone epidemic has come under control, we think it’s not likely that we’re going to be able to measure efficacy in our trial,” Schuchat says.
That makes the Guinea trial even more important. Typically, regulators like the USFood and Drug Administration require very high standards of proof to approve a vaccine, especially for preventive use—like getting a shot for Hepatitis B before traveling. In terms of efficacy data those regulators will look at, Guinea is it. And now there’s an extra wrinkle: Because the vaccine’s interim results looked so good, the Guinea trial is no longer randomizing the recipients of the vaccine. Everybody who gets placed into one of the study’s clusters will get it immediately, no delay. Again: great for west Africa, not so good for the statisticians.
What the statisticians do have to work with—that 100 percent efficacy—isn’t as mind-blowing as you might assume. Of the immediately-vaccinated people, none displayed symptoms 10 or more days after vaccination, compared to 16 infections in the group of 2,380 that was assigned to get vaccines three weeks late. (It take two to 21 days for symptoms to manifest after infection, so a person who came down withsymptoms in those first 10 days could have been infected before the vaccine took effect.)
What the statisticians do have to work with—that 100 percent efficacy—isn’t as mind-blowing as you might assume. Of the immediately-vaccinated people, none displayed symptoms 10 or more days after vaccination, compared to 16 infections in the group of 2,380 that was assigned to get vaccines three weeks late. (It take two to 21 days for symptoms to manifest after infection, so a person who came down withsymptoms in those first 10 days could have been infected before the vaccine took effect.)
Those 16 infections in the delayed vaccination group are what makes the 100 percent efficacy statement a little less awesome. From press reports last week, you’d think the vaccine protected every single person who received it. But it actually just means that it prevented Ebola infection in the subset of those people who were actually exposed to the virus. And exposure levels, thanks to the waning epidemic, were actually pretty low—meaning the total opportunities for the vaccine to show its stuff were pretty limited.
In the group of 4,123 people who were supposed to get the vaccine as soon as possible, 2,109 people didn’t actually get it—they were too young, breastfeeding, or declined to consent to the study. Of those unvaccinated people, only eight got Ebola. Eight out of 2,109! It’s reasonable to assume that a similar number of people who actually got the vaccine were exposed. That means that in reality the vaccine only prevented, ballpark, eight infections in a group of 2,014 people. Not a ton to base a statistical analysis on.
Now, the trial’s runners are put in a difficult position: The study as they originally designed it is functionally over. A few more data points may come out to supplement the original efficacy analysis in the Guinea trial, but it’s only possible for those numbers to go down. “It won’t budge that much,” says study co-author Rebecca Grais, of Doctors Without Borders. “But if you added, let’s say, a couple of vaccine failures, it’ll decrease the efficacy.”
Those numbers, even if they get knocked down a few notches, will still be pretty phenomenal. But thanks to the design of the other major vaccine study, they’re the only phenomenal results regulatory agencies will get to look at. Going forward, the two main trials of rVSV-ZEBOV in Guinea and Sierra Leone will have to combine forces and present a patchwork combination of early efficacy data along with information about safety and the immune response that the vaccine provokes. (The Sierra Leone sponsors, including the CDC, don’t think they’ll get efficacy, but they expect to get plenty of good safety and immunogenicity data.) “It’ll be anaccumulation and triangulation of data,” says Grais.
That process will be difficult, because the trials were designed to answer different questions, explain Grais and Schucher. The Guinea trial looks at a ring vaccination scheme—testing the contacts (and contacts of contacts) of an Ebola-infected person to see if a vaccine can act fast enough to stop the spread in a community. While the answer so far appears to be yes, that’s not a yes to the question posed by the Sierra Leone trial: whether a vaccine can offer long-term protection against infection for people like health care workers.
The early results out of Guinea were enough to convince the trial’s data safety and monitoring board to pull the emergency brake and start giving the vaccine to every person in the study. Now they’ll just have to hope that regulators agree.
Post a Comment