[Ip-health] How Joe got to $802, in his 2003 paper
james.love at keionline.org
Sat Nov 15 21:29:32 PST 2014
On Sat, Nov 15, 2014 at 9:57 PM, Dimasi, Joseph A. <Joseph.Dimasi at tufts.edu>
> *From:* jamespackardlove at gmail.com [jamespackardlove at gmail.com] on
> behalf of Jamie Love [james.love at keionline.org]
> *Sent:* Saturday, November 15, 2014 1:04 PM
> *To:* Ip-health
> *Cc:* Dimasi, Joseph A.
> *Subject:* How Joe got to $802, in his 2003 paper
> I wanted to share some of the details of Joe’s 2003 paper, and he can
> comment if there are any mistakes here.
> 1. His 2003 sample was "representative" data provided by "ten
> multinational pharmaceutical firms." "Licensed-in compounds were excluded"
> [page 156]. Some other studies, such as recent one by OHE, found
> significantly lower costs for licensed in compounds. He generally excluded
> smaller firms, which had lower costs, and under sampled orphan drugs.
> *What are these "other studies" that show significantly lower costs for
> licensed-in compounds? I am not aware of any legitimate studies that do
> so. The OHE report also did not offer any licensed drug cost estimates.
> They merely discussed the issue, and only in terms of success rates. *
Well, since you multiply out-of-pocket costs to account for the risks
of failures, the decision to exclude products with better odds of success
creates a bias, that people should take into account. The OHE study
highlighted this issue, for that reason.
> *There is nothing special about licensed drugs that makes their path to
> approval any easier. The FDA and other regulators will not apply lower
> standards for licensed drugs. So, there is no reason that their
> out-of-pocket costs will be lower. In fact, there is some reason why the
> costs per investigational licensed drug could be higher. I had a study
> published this year that found that clinical development times were longer
> for drugs that were licensed during clinical development (perhaps because
> of operational disruptions when there is a hand-off of the drug). That
> would mean higher times costs, and, perhaps, higher out-of-pocket costs. *
Lots of perhaps speculation here. But just reviewing the 10-k filings
of these small firms makes it look as if they can do things cheaper. In
one meeting in Geneva a couple of years ago, a small firm said it could run
a drug through the FDA for less than $50 million out of pocket. Gilead
said, for them it was $100 million, and people at that meeting generally
agreed that larger firms have both the resources and the inclination to
spend more. I meet with small firms from time to time, and these large
numbers don't seem realistic, as averages. But, if you have the data, lets
> *As for success rates, the OHE makes a lot of mention of my 2010 study on
> success rates. Higher success rates for licensed drugs were reported for
> completeness, but they are really a statistical artifact of the way the
> dataset was constructed. Licensed-in drugs would have entered the
> pipelines of the companies covered at various points during development.
> The points in the process at which they were acquired was not known to us.
> However, you would expect that a lot of them entered during mid- to late
> clinical development. When you are already at mid- to late stage
> development, logically the approval success rate will be higher than if you
> are counting drugs as they first enter the clinical testing pipeline. In
> fact, the results of that study had identical approval rates from phase III
> onward for licensed and self-originated drugs. *
Well, Phase III success rates are already really high. Look at the BLA
Phase III success rates!
> *That suggests no difference, or even a lower success rate for licensed
> drugs since its likely that some of the drugs in the dataset were licensed
> during phase III. You can think about a screening effect, but developers
> also regularly screen their own internal drugs during development.
> Additionally, developers will likely know more about their own drugs than
> they do about licensing candidates (the so-called 'lemons problem' in
> economics). Even if the drugs coming in through licensing have higher
> expectations of approval, you would expect licensors to extract a premium
> from licensees for that through the way that such deals are structured
> (upfront fees, milestone payments, royalty rates, equity purchases, etc.).
> In sum, we don't have the evidence to say, as you claim, that licensed
> drugs have significantly lower costs. Most of these drugs come from the
> small firm sector, so a licensed/self-originated distinction is not, I
> think, a particularly useful analytical construct. The real issue is
> whether small firm costs (inclusive of the costs of drug failures and
> discovery work that never goes anywhere) differ from those of mid-sized to
> large firms. *
Let's just say for now that you disagree with the OHE, regarding risks.
On the broader question of risks, Claire Cassedy has made a spreadsheet of
the new BIO estimates published in Nature Biotechnology of the likelihood
of success, based upon various factors, here: http://goo.gl/fCgRpD
Do you like the BIO estimates?
* You can't say that orphan drugs were underrepresented in the dataset when
> you realize that the representation was meant for the portfolios of
> mid-sized to large firms. A lot of orphan drugs came from the small firm
> sector. So, again, we are back to wondering about small firm costs. *
If your data set has a smaller percentage of orphan drugs than are
approved by the FDA, then your average drug development costs should not be
used to justify high prices for orphan drugs, like Gleevec, Nexavar,
dasatinib, and dozens of other high priced cancer drugs, approved as
orphans. Would you agree with that at least?
> *What's more, while costs for a drug in an orphan indication may tend to
> be lower than in a non-orphan indication, if you have a number of orphan
> indications for the same drug pursued, or you have the drug tested in an
> orphan indication and also in one or more non-orphan indications, then
> costs can add up significantly for the molecule. We likely have had more
> of that in recent years than for the period covered by the 2003 study. And
> it is molecule costs that were being addressed in that study. That brings
> me to your Tufts R&D cost estimate contest. You wrote that it is a cost
> per lead indication. Where have I ever written or said that the cost
> estimates are for lead indications? The cost estimates cover all
> indications pursued.*
Well, that is helpful, and surprising. Your 2003 paper was about NCEs.
How many times can a drug be a NCE? Is being new something you can just
keep on being, even after the drug is already approved for marketing? I
guess so. You are not limited to NMEs. New formulations, FDCs, are all of
these included in your development costs? How exactly does this work? If
we knew the actual names of the drugs, and the trials, maybe people could
understand what you have estimated.
> * It is useful to have estimates of costs for mid-sized to large firms.
> It would also be interesting to have estimates for the small firm sector,
> but you cannot say, as you did, that you know that small firms had lower
> costs. Small firms are less experienced and more poorly resourced, and so
> may have higher development costs, particularly for mid- to late stage
> clinical development. Are they better at discovery or other activities?
> Who knows? The point is that this is an empirical issue, and we do not
> have sufficient empirical evidence to support your claim or its reverse.
> The best you can say is that it is an open topic for research. *
And, we can certainly say that your research does not estimate those
> 2. The 2003 study drugs had a mean of 5303 patients in the Phase I-III
> trials, and a cost per patient of $23,571 (which included liberal overhead
> costs), and which was pretty high for the time.
> *High for the time? Liberal overhead costs? How would you know that?*
> Back then, BMS was saying it was spending $10k per patient for cancer
trials. The NIH was reporting much lower costs, per patient, for cancer
trials, and the Parexel numbers were way lower than you. You come in more
than twice the BMS number. You told me that the $23,571 covered not only
the direct costs. PhRMA justified your number to me by saying companies
doubled direct costs, to account for overhead. But its your data, tell
us, how much overhead over direct costs did you throw in to get the numbers
up to $23,571? I'm not saying you should not include overhead. I think
people would like to know, how much overhead you did include.
> 3. The mean cost of human clinical trials in 2003, was estimated at
> $125 million. The median cost of human clinical trials were $92.9. When
> adding animal studies, the totals were $130.2 and $96 million, respective.
> *This is really not a completely accurate characterization, as adding up
> the average cost estimates for investigational drugs in each phase is not
> meaningful. But, I don't want to quibble over this.*
People can read table 1 on page 162 and do the math themselves.
But if you can't add up the averages by phase that you published in the
paper, what do you recommend? What is the point of making so difficult to
pull out the elements of the cost estimates. The OHE study was a model of
clarity compared to the 2003 paper.
4. The mean costs of the clinical trials were increased $151.8 million, to
> account for the costs of failures, so trials (human and animal) were $282
> million, together.
> *This suffers from the same problem as point 3. You really can't get a
> cost of failures value this way.*
You provided average costs by phase that add up to 125, and you say the
costs are 282 when you take into account the risk of failures. Feel free
to give us the numbers you like better, as regards the elements of the
$802, assuming everything adds up to $802.
> 5. Joe assume $121 million in pre-clinical costs, just because he
> thought pre-clinical costs should be 30 percent of the combined clinical
> and pre-clinical outlays.
> *Here I can take offense, given the way that this is written. I "assumed"
> $121 million? The 30 percent figure was used "just because he thought
> pre-clinical costs should be 30 percent"? Really? Do you think that I was
> sitting on my couch one day thinking about the preclinical cost share, 30%
> popped into my head, and I thought, aha, that sounds right? I know you
> have read the study. The paper clearly lays out what was done. Data for
> several aggregate expenditures time series were examined, preclinical and
> clinical development time distributions were examined, the time
> distributions were used to determine the difference in time between
> prehuman and clinical expenditures, and that temporal difference was
> applied to the aggregate expenditure time series data to introduce an
> appropriate lag between prehuman and clinical expenditures. The 30% figure
> was a data-determined calculated value. It was not an assumption. It was
> not an opinion.*
It is an assumption. It's not a fact, for any of the drugs, unlike
the trial data, which is at least are based upon some claim by the company
that they spent something on a particular drug.
You are looking at general and aggregate R&D expenditures from a survey
of firms and assuming that these data are (1) accurate and (2) relevant to
the cost of the 68 drugs. Some of the 68 drugs were based upon company
pre-clinical research, and others were based upon NIH funded clinical
research. But lets be clear, in your database, every drug's costs are
based upon the notion that pre-clinical spending was equal to 42.9 percent
(.3./.7) of your risk adjusted clinical trial costs. Plus, given the 11
percent cost of capital adjustment, it becomes a huge number if your
calculation, and for many drugs, it is completely bogus.
A drug like AZT, ritonavir, d4T, 3TC, gleevec, Taxol, etc benefited hugely
from NIH funded pre-clinical (and in some cases clinical testing)
research. But in your model, this can't happen. Do you see that this is a
flaw, one that makes your clients look better than they should? Or, at a
minimum, shouldn't your study have a warning, that says, this cost study
does not bear much relevance to any drug that relied upon NIH funded
pre-clinical research funding? That Novartis should stop using your
studies to justify Gleevec pricing, Abbott should stop using your study to
justify Kaletra pricing, and BMS should stop using your study to justify
> 6. This brought the number up to $403 million.
> 4. The base cost of capital assumption in the 2003 paper was inflation
> plus 11 percent, and this nearly doubled the number, to $802.
> *This is a misleading way to put it. Yes, the discount rate was a real
> (i.e., inflation-adjusted) rate, and a nominal rate would be that plus an
> inflation rate. However, this makes it seem like something greater than
> 11% was applied to the cost figures above. That is not the case. An 11%
> rate was applied. The out-of-pocket costs were inflation-adjusted and,
> appropriately, an inflation-adjusted discount rate was applied to those
In other words, costs were increased each year by the rate of inflation,
plus 11 percent compounded, which has its most important consequence for
the earlier outlays, such as the large amount assumed to have been spent on
> 5. The fundamentals were (with rounding):
> $125 in human use trials, on average for the 68 drugs, based upon 5303
> patients and $23,571 per patient costs.
> $5 million in animal trials.
> $152 million assigned to cost of failed trials.
> $121 million for pre-clinical research (including costs of failures).
> $399 million for capital costs/interest on the $403 million invested).
> That is how he got to $802 million.
> James Love. Knowledge Ecology International
> KEI DC tel: +1.202.332.2670, US Mobile: +1.202.361.3040, Geneva Mobile:
> +41.76.413.6584, twitter.com/jamie_love
James Love. Knowledge Ecology International
KEI DC tel: +1.202.332.2670, US Mobile: +1.202.361.3040, Geneva Mobile:
More information about the Ip-health