| |
| |
|
| |
|
|
|
| |
Content for site developed under the guidance of the TRACK Advisory Council.
|
|
 |
|
| |
 |
|
|
|
| | The Design of Clinical Trials | 
Printer-friendly version | | | | | | | | Our ongoing needs assessment at ThrombosisClinic.com has identified that participants in our educational activities are interested in learning more about the design and interpretation of the results of clinical trials. With that in mind, we are fortunate to have Dr Marc Cohen with us for this expert commentary on clinical trials. Dr Cohen is Chief of the Division of Cardiology and Director of the Cardiology Fellowship at Newark Beth Israel Medical Center in Newark, New Jersey, as well as Professor of Medicine at the Mount Sinai School of Medicine in New York City. Speaking of expertise in this area, Dr Cohen has authored or coauthored more than 170 articles, including 95 peer-reviewed original papers. He has consulted on manuscripts for the New England Journal of Medicine, Circulation, and the Journal of the American College of Cardiology, and is a member of the editorial board of the Journal of the American College of Cardiology, The American Journal of Cardiology, and The Journal of Thrombosis and Thrombolysis. He has also contributed chapters on cardiology for several books. So I think Dr Cohen certainly knows his way around this topic. Dr Cohen, thank you for joining us on ThrombosisClinic.com. | | |  | | Good afternoon. | Back to top 
| | | | | | Why don't we start by defining a clinical trial? What are the various types, and can they be ranked in terms of the strength of their evidence? | | | | | Well, there are a lot of studies that are labeled "clinical trials." There are some trials that could be viewed as being retrospective; in other words, doctors studied data that had already been collected, either in medical records or charts, and then all of the activity with regard to the patients, and the medications, and the interventions that were done were all things that happened in the past and the doctors just collected that information and then drew some kinds of conclusions. So the doctors are looking backwards at events that occurred in the past.
Other types of clinical trials are prospective trials. So, in other words, investigators decide on a particular question that they want to ask and then they make a structure, which we call a trial, to answer that question. But this time, what they do is they are looking forward and inviting the patients to participate in this trial in a forward direction. So as the events happen, they are being immediately recorded.
Among the types of prospective clinical trials that we have, we can either have a prospective, randomized trial or a nonrandomized trial. In other words, we can be testing something against something else that would be a randomized trial if we allow the play of chance to evenly distribute the likelihood of the patients receiving one drug or one intervention versus another in a random, by chance manner. Alternatively, the clinical trial could just be a prospective, consecutive series. In other words, I studied 100 patients using this new drug and in every patient I used the new drug, and then I just sort of collected the information and drew some conclusions. But that would not be a randomized trial. I did not flip a coin and say, "You get the new drug," flip another coin and another patient gets the old drug.
So, in general, we have prospective versus retrospective trials, and we have prospective, randomized trials versus nonrandomized trials. We also have trials that are prospective and randomized, but they can be blinded or unblinded.
In other words if they are blinded, the patient or the doctor does not know what drug by chance they were assigned to. So in other words, if we are talking, for example, about blood thinners in a randomized, blinded trial, one patient may be getting heparin, for example, and another patient may be getting a low-molecular-weight heparin. And neither the patient nor the doctor would know. And then we would collect the data and the outcomes and not be biased in any way because we honestly wouldn’t know what the patients really got. We would just be faithfully recording their outcomes.
Alternatively, we could have a randomized clinical trial, but not be blinded. We could assign the patients by chance to receive one or another blood thinner, but we would know once the envelope was open what they were getting. To make a long story short, yes, these trials can be ranked in terms of the strength of evidence and the most convincing trial or the trial that would evoke the most confidence with regard to its results would be a prospective, randomized, blinded clinical trial. | | Back to top
| | | | | | Are there any disadvantages to that ideal type study? In other words, advantages of a randomized trial versus disadvantages, blinded versus nonblinded, etc? | | | | | Yes, it is complicated to set up prospective, randomized, blinded trials. There are a lot of logistical issues, a lot of levels of complexity, a lot of people that have to work together to execute a trial that would be of the highest strength of evidence, namely the prospective, randomized, blinded trial.
It takes a lot of energy. It takes a lot of effort. It takes a lot of discipline to stick to the exact protocol and not deviate. And, it is not so easy to do these trials, nor is it particularly inexpensive. We do have to spend a lot of money and a lot of resources in terms of doing these trials that way. And because these are resource-intense types of studies, not everybody can just at the flip of a hat say, “Yes, I would like to do this trial as a blinded trial” or “Yes, I'd like to do that trial as a randomized trial.” So it does take a lot of commitment to do the ideal type of trial, the prospective, randomized, blinded trial. | | Back to top
| | | | | | So it sounds like that certainly complicates the planning and design of the study? Could you touch on the basic elements of the design of, say, a prospective, randomized, blinded trial? | | | | | Well, the basic elements of the design are: number one, you have to try and first focus on a particular question. You have to decide in what arena you want to ask the question. You know, is the question something that not only is relevant today but presuming that it may take you a year or 2 to actually conduct the study, will it still be a relevant question tomorrow or a year from now?
That's something that always comes up. And once you focus on the question and the relevance of the question, then the next issue is, is the study something that is executable? Is the question being asked—can it be answered in a way that is reasonably straightforward or does the process of answering that question involve so many forks in the road and curves and things that it is actually overwhelming to execute the study?
Does the study, if it is going to be a blinded study where nobody knows what is actually being given to the patient, do you have the wherewithal to mask the drug so that somebody cannot figure out that this one got the old drug and another patient got the new drug? That is not easy to do sometimes.
The other issue that arises in terms of elements of the design is what dose of medication, for example, are you going to use? Have you done your homework? Have you figured out what the optimal dose is that you are going to test?
Then, very important as well, is what are the outcomes? What are you going to measure in terms of the end points? Are you going to ask the question simply, “How many people died?” or are you going to say, "Let's measure the number of people who died and the number of people who have a heart attack" Or "Let's measure the number who died and/or had a heart attack and/or had an intervention like angioplasty or surgery." So, you need to do your homework in terms of figuring out what you want to measure as the outcome.
And then, finally, you have to decide how long you want to follow your patients. Do you want to follow them until they get discharged from the hospital? Do you want to follow them until 1 month? Do you want to follow them until 1 year?
These are all pretty tough decisions because every decision that you make, you know, carries with it some baggage. Do you have enough money to follow all these patients? Do you have enough staff to follow all these patients? Etc. | | Back to top
| | | | | | You mentioned "blinding" several times, Dr Cohen. Could you just touch on the importance of double blinding or double-blind versus single-blind trial? And what are the particular problems faced in maintaining blinding? | | | | | Yes, well, we are all members of the human race and, therefore, almost by definition, we all have opinions and biases. We may be inclined to think that this drug is better than that drug. Or that this device is safer than that device.
And so whether you like it or not whenever you are presented with a drug or a device, you usually approach that with some bias—maybe not overwhelming, but some bias. So how do you avoid biasing your results against something or in favor of something without interfering, you know, with what the real truth is about the value of that drug or the value of that device? So, what you do then is you blind the people that are involved in the trial. A single-blind trial would mean that one of the two principles is blinded. Who are the principles? The patient is a principle. And the researcher is a principle.
So the ultimate, the strongest type of trial would be double blind where both the patient and the researcher are blinded. Neither one of them knows what new drug nor what old drug the patient is receiving. That way, neither the patient nor the researcher can introduce any kind of prejudice into the trial.
There are some trials that are single blinded. Maybe just the patient is not made aware of what they're getting, but the doctor is aware of what they're getting. And then there are studies like I told you before that are not blinded at all. Everybody knows what they got and the hope is that the biases are not going to influence the trial at all.
The main issue in terms of maintaining blinding is that it involves a lot of discipline. If someone all of a sudden has a bad outcome, is the doctor going to rush to unblind the patient and say, "Oh my God, you know, the patient's having a bad outcome. I better unblind and I better see what they got" and how do you make sure that the pharmacy hides what the true nature of the drug is that the patient received?
That involves a lot of cooperation and a lot of discipline. So it is not so easy to do in the short run, but the gains in terms of the strength of the quality of the evidence are very, very great when you have a double-blinded study. | | Back to top
| | | | | | You mentioned bias, Dr Cohen. Could you touch on selection bias in terms of inclusion and exclusion criteria for us? | | | | | Some physicians may be biased to allow some patients to enter a study and they may be biased to say, "Yeah, this patient is not such a great candidate. I won't enroll them in this study."
So, there is something called selection bias where the doctors do not encourage anybody and everybody to come in and see if they are candidates for the trial. But all trials have inclusion criteria and exclusion criteria. In other words, in order to enter this trial where I'm going to ask a specific question about hair coloring, I would like all the patients entering the trial to have blue hair or brown hair or something.
I can do that as a researcher. I can set up the boundaries of who I want in and who I want out of this study. And I try to do that in a way that allows me to answer the question that I'm interested in, as carefully and as specifically as possible.
Now, the other side of the coin is if I make my inclusion and exclusion criteria so specific in the hope of answering this specific question, then I lose a little bit on the other side on what I call "generalizability."
In other words, if I do a study and I discover that this drug is better than that drug, can I generalize that conclusion to the whole world or is that conclusion only good for the blue-haired patients and the brown-haired patients that I insisted could come in because of the inclusion and exclusion criteria?
So there is always a tug-of-war when a researcher sets up the inclusion and exclusion criteria. He wants to narrow the focus so that it is not a free-for-all and he can try to come up with some clear-cut answers to the question that he or she is asking. On the other hand, if they are too narrow in their scope of whom they let into the trial, then the conclusions of the trial may not be very “generalizable.”
So it is a little bit of a tug-of-war. There is no right way or wrong way here. It is a balance. | | Back to top
| | | | | | Thank you for that very useful clarification. What about a crossover study? What is the purpose of a crossover trial design? | | | | | Well, in a lot of drug studies where you're comparing one drug versus another or one strategy versus another, the most confident way of getting an answer is to start the person off on one drug, or one intervention, see the response and then literally flip him over, or her over, to the other intervention and see what the response is there.
And that would be like the ultimate way of really discriminating between the effects of one drug or another or one intervention and another, by literally having each and every patient experience the 2 different interventions. So that way you see the behavior of the same patient to 2 different kinds of drugs or 2 different kinds of strategies.
And that really makes things very, very sharp in terms of seeing how one intervention or strategy can be different than another. | | Back to top
| | | | | | Let us switch to discussing outcomes for a moment. You hear the terms "primary outcomes" and "secondary outcomes” or “end points" in terms of clinical trials. Could you define and differentiate those for us? | | | | | Yes, well, it is always very important for the investigator to be very focused on what he wants to measure as an outcome of the study in reference to the question that he wants to answer. Is this drug so good that it actually reduces mortality?
One of the more important things that researchers have to do is focus on the outcomes that they are going to measure in order to answer the specific question that they have in mind. And, it is sort of traditional for the researcher to be forced to focus on one primary outcome. It is almost like bad form or a sign of lack of clarity, or maybe even confusion on the part of the investigator if he or she says, "You know what? I'm going to measure a whole bunch of things."
The expectation of a good research trial person and of a good research trial is that they will have a primary focus that they will measure, but also we do not live and work in a vacuum. There will be other things that are of interest to measure, but they are not the primary focus.
So, for example, in a study of a new kind of drug that may be used in heart attack patients, it is traditional to focus on the primary end point of death. Now, of course, we will also be interested in how many recurrent heart attacks there are. We will be interested in how many patients go on to need bypass surgery. But that is not the primary focus. The primary focus is to see if this strategy really makes a difference with regard to the one most important end point, which is mortality.
And so we like to assign one outcome or a cluster of outcomes as being the primary end point, which helps everyone focus on the question being addressed and admit at the beginning that this is our primary focus, but also agree that we are not going to be wearing shades over our eyes, blinders over our eyes, we will measure lots of things, but those other things will not be the prime focus of our investigation.
And by doing that, the trial and the results of the trial are put in relief, so to speak. The main point is highlighted. Did the study reach its primary target or not? And it allows us to in a sense judge the value of a trial by whether or not what the researchers intended at the beginning to focus on they kept their focus on the same object and they did not go wandering around to different objects. | | Back to top
| | | | | | What issues do investigators need to address or keep in mind when determining the length of the follow-up period in a clinical trial? | | | | | Well, the most important thing is what are they going to measure? If you measure something that occurs quickly and early, then you do not need to follow patients for 5 years, if you are looking to measure something that, from your experience and from other studies, maybe happens often within the first month.
Alternatively, if you are asking a question about whether a drug is useful in preventing the signs and symptoms of an enlarging prostate gland in men; well, the prostate gland enlarges over months and months and maybe even years. So, it would be silly to just check the effect of that drug on prostate symptoms for 1 week or 1 month.
So, the length of follow-up period of a clinical trial should primarily be dictated by what it is you're focusing on, what problem you are attacking, and what are the elements relating to that area—what is the behavior of the elements in that area? In coronary disease, for example, if we are talking about acute MI [myocardial infarction] or acute coronary syndromes, then traditionally we know from previous experience that the most active period for outcomes to occur is in the first 30 days. After that, things quiet down. But there is still some activity up to 6 months or a year. So at the very least, you would like to follow the patients for 30 days. If you had the resources, it would be nice to follow them for 6 months or maybe a year.
Alternatively, if you are dealing with cancer drugs, for example, it is kind of silly to say, "My drug is great. You know, look, nothing happened in the first month." But maybe you know that the recurrences from this kind of cancer may take years to emerge. In which case, if you did not set up the study to follow patients for a few years, you may never appreciate the fact that one cancer drug caused less recurrence over the few years than the other cancer drug.
So, in order to determine the length of follow up, you really need to know about the disease and the area that you are studying. | | Back to top
| | | | | | You hear the term "power" used in the discussion about clinical trials. How does a researcher determine how many subjects should be recruited into a trial? And how can the reader of a research article determine if too few subjects were included? Or, in other words, whether or not the trial was adequately powered? | | | | | That is a very important question, so that people can be wise in drawing implications from trials because so many trials have the potential to be "negative" when in reality they did not have the power or they did not have enough subjects to actually prove that the new drug was not different than the old drug.
What we mean by "power" is that in order to statistically prove a point, you have to have enough events in order for you to make a calculation that would not occur by chance but would occur with a certain probability. And, so what we do is you sit down with the biostatistician. Again, you start by knowing the area that you are working in.
So, if I want to study the effect of a new blood thinner on heart attack patients, I know that heart attack patients usually have about 10% mortality at 1 year. So, you tell the statistician that you think your drug is 20% better, for argument's sake, or 10% better, or maybe you think your drug is so great that you have the expectation that this drug will be 40% better than traditional therapy. The biostatistician can then make an arithmetic calculation that would go as follows: for a disease process in whom the average patient population has a 10% event rate, in order to have the power to detect a 10% difference between the new strategy and the old strategy, please spit out from the computer how many patients I need to detect that difference with 90% power, or 80% power, or 85% power.
In other words, I actually can tell the statistician I would like to be very strict and I would like the results to be very confident and therefore I want to have 90% power. And the statistician will then come back to me and say, well, if you're going to be that strict, and these are the boundaries you are telling me—your drug is only 20% different than the old drug—then I am going to do the calculation and I tell you, you need 500 patients in each arm. If, after the study is done, you only have 300 then you will not have the statistical power to be confident about the decisions that you have made. | | Back to top
| | | | | | Results of clinical trials are reported according to their statistical significance or level of statistical significance. How do you determine what statistical methods you will use to analyze the data, first of all? And, if the data are found to be statistically significant can those results be assumed to be relevant to clinical practice? | | | | | Well, those are 2 very important questions lumped into 1 long question. So let us do one at a time. How do you determine what statistical methods you use in a particular trial? That is a function of the kind of data that you collect because different statistical tests are rightly or wrongly applied to different types of outcome measures.
So, for example, the statistical test known as the "Student t test" that is a test that can be only applied to numbers that are continuous, like the mortality, the average age. That is a number that can go from one to zero and the average age can be 50 in one group and 60 in another.
Age is what we call a continuous variable as opposed to, for example, the color of your eyes. Your eyes can only be blue or brown or black. You know there is no way to quantify light blue, darker blue, navy blue; so, you cannot assign a continuous number to the color of your eyes. You can only assign a category to them.
So, some outcomes are measured by a number that could be a continuous number from 1 to 10 or 1 to 100. Other outcomes, like dead or alive, can only be measured as yes or no, as a categorical variable. So, it would be correct to apply the chi-squared statistical test to categorical yes/no variables, because that test is specifically designed for that kind of outcome.
It would be wrong to apply the chi-square test to compare whether or not the average age of heart attack victims is older or younger than the average age of suicide victims, because that is a continuous number. It could be anywhere from 1 to 100. That is just a fact of life—in the mathematical domain known as probability theory that there are certain tests that need to have certain prerequisites before you can apply them.
Having applied the right statistical test, I can very easily come up with a trial that shows that one strategy is more statistically significant than the other, but whether or not it is relevant to clinical practice, that is a judgment that doctors have to make.
So, for example, I have discovered a new blood thinner that is statistically significantly better than heparin. Okay, but what if the rate of complications with one drug is 20% and the rate of complications with the new drug is 19%? Now, I did the power calculation and the statistician told me I needed to get 10,000 patients into this trial, and I did. So, I have the power to detect that drug A, with an event rate of 19%, is statistically different than drug B, with an event rate of 20%. So that is fine.
I've discovered a statistical difference. The only question is with regard to my clinical practice, is the difference in bleeding of 19% meaningfully or clinically significantly different than 20? Is 19 versus 20 that big a deal in terms of my day-to-day clinical practice? That is a judgment call. That is not a statistical call. | | Back to top
| | | | | | When you analyze the results of a clinical trial, Dr Cohen, do you look for a specific P value? Or is that related to the overall trial design? | | | | | In general, we do look for a P value—in other words, the likelihood that some thing would happen by chance—of being less than 0.05. In other words, this discovery that I made is not likely to happen by chance except for only 5 or less times out of 100. In other words, it is probably a true discovery that I made, 95 times out of 100. It could have popped up as a chance discovery that is not really true only 5 times out of 100, or less.
So when we talk about a P value of less than 0.05, what we are saying is that we are making a conclusion that is not likely to have happened by chance more often than 5 rolls out of 100 rolls of the dice. | | Back to top
| | | | | | So, in other words, you could potentially say that if you have a P value of 0.05, you are 95% sure that your hypothesis is correct? | | | | | Correct. | | Back to top
| | | | | | Should clinical practice be guided by the results of a single trial? Or do you in your own practice change your management of patients on the results of a single clinical trial? Or, do you or should clinicians wait for confirming trials or meta-analysis of several trials? | | | | | Well, you know, one large, well-executed trial can provide a strong foundation to change our thinking. But having 2 trials that reinforce each other and say the same thing really gives the average doctor a good level of confidence that if he or she were to change their practice, they would be doing the right thing.
So, I think in general a minimum of 2 well-executed, prospective, randomized trials pointing in the same direction gives doctors enough confidence to change their practice. I do not think that they have to wait necessarily for a meta-analysis of a whole bundle of trials. On the other hand, one trial by itself? You always think hard about whether or not you should start shifting the direction of the ship just based on one trial. | | Back to top
| | | | | | We have all heard, I am sure, a number of investigators over the years who speak negatively of meta-analyses. What is your position? | | | | | A meta-analysis is as valuable or as not valuable as the trials that you include in the meta-analysis.
If, in your inclusion and exclusion criteria, you include only good, well-done, prospective trials and you exclude the nonrandomized or nonblinded trials, or the trials that had a lot of problems, then having a meta-analysis is a very valuable way to see exactly where the general direction of things is going.
You also have to make sure that when you do a meta-analysis that you are combining trials that have similar characteristics. You know, they looked at the same population or they used drugs that were more or less in the same family. It does not make any sense to do a meta-analysis comparing trials that were very different in their nature because it is like jumbling apples and oranges together and hoping to come out with a grapefruit. | | Back to top
| | | | | | Any other comments or issues you would like to address, Dr Cohen? | | | | | I think in general what I said at the very beginning is really the biggest lesson that I would impart. It is not easy to do a good prospective, randomized, blinded clinical trial and because it is not easy, when they are done, they should be applauded. And, if people do participate in trials, they should remember that there is a lot of discipline involved in making sure that you stick to the protocol and obey the different inclusion and exclusion criteria so that the trial can be done in a manner that gives you the most confidence regarding the results when you are finished.
So, I think all of us should appreciate how much hard work is involved in doing these trials and when they are involved in these trials, they should respect the fact that there is a lot of discipline involved in executing them cleanly. | | Back to top
| | | | | | Excellent summary. We will wrap up by saying thank you to Dr Cohen for joining us on ThrombosisClinic.com and for his very informative expert commentary. | | | | | | Thank you very much for inviting me. | | Back to top
|
| | |
|
|
| |
|
|