[Ip-health] How Joe got to $802, in his 2003 paper
Dimasi, Joseph A.
Joseph.Dimasi at tufts.edu
Sat Nov 15 18:57:26 PST 2014
I have little time in the near term to go back and forth on this as we have in the past, so I will respond once. Yes, there is a fair amount wrong, particularly with regard to claims outside of the study and how the study is characterized. I have comments below some of the points.
Joseph A. DiMasi, Ph.D.
Director of Economic Analysis
Tufts Center for the Study of Drug Development Tufts University
75 Kneeland Street, Suite 1100
Boston, MA 02111
tel: 617-636-2116; fax: 617-636-2425
From: jamespackardlove at gmail.com [jamespackardlove at gmail.com] on behalf of Jamie Love [james.love at keionline.org]
Sent: Saturday, November 15, 2014 1:04 PM
Cc: Dimasi, Joseph A.
Subject: How Joe got to $802, in his 2003 paper
I wanted to share some of the details of Joe’s 2003 paper, and he can comment if there are any mistakes here.
1. His 2003 sample was "representative" data provided by "ten multinational pharmaceutical firms." "Licensed-in compounds were excluded" [page 156]. Some other studies, such as recent one by OHE, found significantly lower costs for licensed in compounds. He generally excluded smaller firms, which had lower costs, and under sampled orphan drugs.
What are these "other studies" that show significantly lower costs for licensed-in compounds? I am not aware of any legitimate studies that do so. The OHE report also did not offer any licensed drug cost estimates. They merely discussed the issue, and only in terms of success rates. There is nothing special about licensed drugs that makes their path to approval any easier. The FDA and other regulators will not apply lower standards for licensed drugs. So, there is no reason that their out-of-pocket costs will be lower. In fact, there is some reason why the costs per investigational licensed drug could be higher. I had a study published this year that found that clinical development times were longer for drugs that were licensed during clinical development (perhaps because of operational disruptions when there is a hand-off of the drug). That would mean higher times costs, and, perhaps, higher out-of-pocket costs. As for success rates, the OHE makes a lot of mention of my 2010 study on success rates. Higher success rates for licensed drugs were reported for completeness, but they are really a statistical artifact of the way the dataset was constructed. Licensed-in drugs would have entered the pipelines of the companies covered at various points during development. The points in the process at which they were acquired was not known to us. However, you would expect that a lot of them entered during mid- to late clinical development. When you are already at mid- to late stage development, logically the approval success rate will be higher than if you are counting drugs as they first enter the clinical testing pipeline. In fact, the results of that study had identical approval rates from phase III onward for licensed and self-originated drugs. That suggests no difference, or even a lower success rate for licensed drugs since its likely that some of the drugs in the dataset were licensed during phase III. You can think about a screening effect, but developers also regularly screen their own internal drugs during development. Additionally, developers will likely know more about their own drugs than they do about licensing candidates (the so-called 'lemons problem' in economics). Even if the drugs coming in through licensing have higher expectations of approval, you would expect licensors to extract a premium from licensees for that through the way that such deals are structured (upfront fees, milestone payments, royalty rates, equity purchases, etc.). In sum, we don't have the evidence to say, as you claim, that licensed drugs have significantly lower costs. Most of these drugs come from the small firm sector, so a licensed/self-originated distinction is not, I think, a particularly useful analytical construct. The real issue is whether small firm costs (inclusive of the costs of drug failures and discovery work that never goes anywhere) differ from those of mid-sized to large firms.
You can't say that orphan drugs were underrepresented in the dataset when you realize that the representation was meant for the portfolios of mid-sized to large firms. A lot of orphan drugs came from the small firm sector. So, again, we are back to wondering about small firm costs. What's more, while costs for a drug in an orphan indication may tend to be lower than in a non-orphan indication, if you have a number of orphan indications for the same drug pursued, or you have the drug tested in an orphan indication and also in one or more non-orphan indications, then costs can add up significantly for the molecule. We likely have had more of that in recent years than for the period covered by the 2003 study. And it is molecule costs that were being addressed in that study. That brings me to your Tufts R&D cost estimate contest. You wrote that it is a cost per lead indication. Where have I ever written or said that the cost estimates are for lead indications? The cost estimates cover all indications pursued.
It is useful to have estimates of costs for mid-sized to large firms. It would also be interesting to have estimates for the small firm sector, but you cannot say, as you did, that you know that small firms had lower costs. Small firms are less experienced and more poorly resourced, and so may have higher development costs, particularly for mid- to late stage clinical development. Are they better at discovery or other activities? Who knows? The point is that this is an empirical issue, and we do not have sufficient empirical evidence to support your claim or its reverse. The best you can say is that it is an open topic for research.
2. The 2003 study drugs had a mean of 5303 patients in the Phase I-III trials, and a cost per patient of $23,571 (which included liberal overhead costs), and which was pretty high for the time.
High for the time? Liberal overhead costs? How would you know that?
3. The mean cost of human clinical trials in 2003, was estimated at $125 million. The median cost of human clinical trials were $92.9. When adding animal studies, the totals were $130.2 and $96 million, respective.
This is really not a completely accurate characterization, as adding up the average cost estimates for investigational drugs in each phase is not meaningful. But, I don't want to quibble over this.
4. The mean costs of the clinical trials were increased $151.8 million, to account for the costs of failures, so trials (human and animal) were $282 million, together.
This suffers from the same problem as point 3. You really can't get a cost of failures value this way.
5. Joe assume $121 million in pre-clinical costs, just because he thought pre-clinical costs should be 30 percent of the combined clinical and pre-clinical outlays.
Here I can take offense, given the way that this is written. I "assumed" $121 million? The 30 percent figure was used "just because he thought pre-clinical costs should be 30 percent"? Really? Do you think that I was sitting on my couch one day thinking about the preclinical cost share, 30% popped into my head, and I thought, aha, that sounds right? I know you have read the study. The paper clearly lays out what was done. Data for several aggregate expenditures time series were examined, preclinical and clinical development time distributions were examined, the time distributions were used to determine the difference in time between prehuman and clinical expenditures, and that temporal difference was applied to the aggregate expenditure time series data to introduce an appropriate lag between prehuman and clinical expenditures. The 30% figure was a data-determined calculated value. It was not an assumption. It was not an opinion.
6. This brought the number up to $403 million.
4. The base cost of capital assumption in the 2003 paper was inflation plus 11 percent, and this nearly doubled the number, to $802.
This is a misleading way to put it. Yes, the discount rate was a real (i.e., inflation-adjusted) rate, and a nominal rate would be that plus an inflation rate. However, this makes it seem like something greater than 11% was applied to the cost figures above. That is not the case. An 11% rate was applied. The out-of-pocket costs were inflation-adjusted and, appropriately, an inflation-adjusted discount rate was applied to those costs.
5. The fundamentals were (with rounding):
$125 in human use trials, on average for the 68 drugs, based upon 5303 patients and $23,571 per patient costs.
$5 million in animal trials.
$152 million assigned to cost of failed trials.
$121 million for pre-clinical research (including costs of failures).
$399 million for capital costs/interest on the $403 million invested).
That is how he got to $802 million.
James Love. Knowledge Ecology International
KEI DC tel: +1.202.332.2670, US Mobile: +1.202.361.3040, Geneva Mobile: +41.76.413.6584, twitter.com/jamie_love<http://twitter.com/jamie_love>
More information about the Ip-health