Novel Biology, Novel Utility, & the Illusion of Skill: What to develop? What to reward?

Dear All (wonk-ish note alert and also with particular thanks as always to Kevin Outterson for helping to refine this newsletter):

Sparked at least in part by the innovation metrics in recent pipeline analyses (specifically, the ideas of No Cross-Resistance, New Chemical Class, New Target, and New Mechanism of Action) used in the 2018 pipeline analysis by Theuretzbacher et al. (link, and note that I plead guilty to being part of the et al. in this publication), I have been pulled of late into conversations that consider the idea that we should ONLY consider COMPLETELY novel products as having value. No Me-Too products, no extensions of known classes.

I can certainly see the attraction of this idea. If nothing else, requiring complete scientific or biologic novelty is both a very easy metric and is a good way to encourage developers to pursue diversity.

That acknowledged, I want to argue that this view suffers from the “Illusion of Skill,” and that we should instead maintain a focus on both Novel Biology* and Novel Utility. I talked a bit about this in my recent commentary on the WHO review of the preclinical and the clinical antibacterial pipeline (link) and I don’t want these ideas to get lost. (*Aside: Note that I struggle with the best word to use here and have settled for purposes of this newsletter on Novel Biology as a way to encompass multiple variations on the idea of novel mechanism, chemistry, etc.)

But before we look further at Novel Biology vs. Novel Utility, let’s first spend a moment on the idea of the Illusion of Skill as discussed by Nobel laureate Daniel Kahneman in his fabulous book Thinking, Fast and Slow. (If you don’t know his work, please take a look: here is the link to the book on Amazon.com and a link to a New York Times Magazine article by him — his ideas take time to digest but are mind-bending!) In brief, the Illusion of Skill is the idea that we consistently over-estimate the extent to which skill in one domain transfers to another domain. Stated differently, it is easy to believe that true skill in a complex but generally consistent domain (e.g., ability to diagnose an obscure medical syndrome based on deep understanding of human biology) translates into equal skill in a less consistent/more chaotic domain (e.g., predicting which antibiotic will be useful 15 years from now). Kahneman summarizes this forcefully in his New York Times Magazine article:

“We often interact with professionals who exercise their judgment with evident confidence, sometimes priding themselves on the power of their intuition. … How do we distinguish the justified confidence of experts from the sincere overconfidence of professionals who do not know they are out of their depth? … Overconfidence arises because people are often blind to their own blindness.”

With that in our heads, why does the Illusion of Skill matter in the Novel Biology vs. Novel Utility debate? Well, we are using Novel Biology/Utility as surrogates for the thing we really want: Useful antibiotics that last as long as possible in the clinic. In short, these are our rules of thumb as we think about what to research, fund and reward. But, the Illusion of Skill is a reminder that we should be cautious in our application of these rules. As I said in that prior note about the WHO pipeline:

1. Really compelling novel antibacterials are rare. Seriously rare. Rocking horse manure rare. I love the idea of novelty and we should keep looking (Go CARB-X with its willingness to fund ~15 projects that could produce new class agents, together with a similar number of extensions on approved classes!), but we also need to deal with reality and recognize that …
2. Improved versions of existing-class compounds can be really interesting and are generally easier to find (e.g., oral variants of known classes for difficult Gram-negatives). Over time, these improvements can be substantial: see my timeline chart (link) looking in particular at the idea that decades passed between first and subsequent fluoroquinolones and beta-lactams.

So, where does this leave us? How can we use the ideas of Novel Biology vs. Novel Utility? Let me now point to a paper that Kevin Outterson and I wrote a few years ago (link to the paper and link to a slide deck about the paper). In it, we suggested a point-scoring system for judging the value of a new antibiotic with the intent of using the resulting score to set the size of a pull incentive. The core idea was to create a simple metric based on rules that a drug developer could apply very early in an R&D program. Here is a distillation of the rules we proposed:

• Everything started with achieving FDA and EMA approval for one or more indications and at least one pathogen on a threat list.
• For the threat list, we used the then-available CDC 2013 list and said that a product got more points by covering pathogens that were rated as increasingly of concern.
• We also proposed points for having a novel mechanism of action. Further, we suggested that at least a few 2^nd-in-class products could earn at least some points.
• We proposed points for having an oral route of administration.
• We also had features to cover pediatrics and 2^nd indications, but these are not relevant to this debate.

So, the key to our scoring system was that the merit of a new compound was judged both on what it did (e.g., pathogens, being oral) and how it did it (e.g., new mechanism, new target). The weighting of these two elements can be adjusted but the core idea was that both are valued – and you could make a guess very early in an R&D program whether one or both of these would be achieved. 

Notably, we excluded expert evaluations about whether the new antibiotic was “great.”  Such expert opinions involve a lot of judgment, but most importantly reliance on such a subjective standard makes it impossible for a drug developer to make rational choices in the early stages of an R&D program.  

So, and with a sincere desire to avoid the Illusion of Skill, let’s close the loop by coming back to Theuretzbacher et al. (same link as above) and note that the analytical table in that paper ALSO included columns for spectrum (aka, Novel Utility). Being oral was not used for product ratings but was noted in the comments about each product.

So that’s it. Novel Biology matters. But, Novel Utility also matters. The perfect should not be the enemy of the merely good — there is room for both in analyses of value. We should be careful when making guesses about which products will be of greatest value in 15 years … for now, I simply know that while I would love to have Novel Biology, I can get a lot value from Novel Utility.

All best wishes, –jr

John H. Rex, MD | Chief Medical Officer, F2G Ltd. | Operating Partner, Advent Life Sciences. Follow me on Twitter: @JohnRex_NewAbx. See past newsletters and subscribe for the future: https://13.43.35.2//blog/