Slide 1

Thank you very much. It’s a great honour to be here enjoying the weather in Spain compared to the weather in Michigan in the United States. You’re going to be hearing some more examples of analyses from DOPPS.  They talk about facility level analyses and comparisons to patient level analyses and I’m going to be giving some examples in this talk about why these are different and why you might want to consider both types of analyses.

Slide 2

But just as a background, I want to point out, I want to just review the fact that in all our scientific endeavours whether they are in laboratory, whether they are in randomised controlled clinical trials or whether they are in observational studies really have the same goal typically which is to identify and understand causal mechanisms. The tools that we use in all of those environments are very similar. In the laboratory of course, we try and control the environment, we manipulate the factors which might be affecting the outcome and typically we move one factor at a time. If we observe differences in the outcomes, when we have done that then we can typically infer a causal mechanism or relationship between that factor and the outcome.
Now when you’re treating patients of course, you can’t treat the same patient under exactly the same conditions with two different treatments, so what we do instead is we randomise patients to 2 different groups, so that on average if you have enough patients in the two groups, the two groups will be similar and the difference in outcomes between them must be due to the treatment effect. With observational studies it’s more complicated. We start with treatment groups that are not randomised. They are not the same. What we do is we observe differences in outcomes between those existing treatment groups but they are as treated and we must note that many different ways or the multiple ways in which those groups might differ with respect to potential causes of differences in outcomes and then we have the job of figuring out which part of the differences in outcome were caused by the treatment and which part was caused by other factors or differences that existed between the treatment groups. This is called the problem of confounding.

Slide 3

There are several statistical tools that we can use to help us address the problem of confounding which is to sort out the effective treatment when there are other differences between the treatment groups. The most important tool that we use is called statistical adjustment or regression models. Here we use equations to approximate the effects of al the factors simultaneously including the treatment factor on the outcome. Then we evaluate the equation twice, once with each treatment and see what the equation says about the outcomes when all else in the equation is held equal.
I’m going to be talking today about another approach called instrumental variables where we try and find groups of patients with different treatments or with different treatment levels and we try and find groups which differ with regards to treatment but where the patients are assigned approximately or pseudo-randomly to those different groups. In such analyses we also use statistical adjustment.
There is a third method which is --- a third statistical tool called marginal structural models which I will not be talking about today. I’m going to be focusing instead upon the instrumental variable approach where we use the facility as giving us a group of patients and to the extent that patients are assigned to the facilities by well, we know the reasons, sometimes by insurance, by patient preference or geography. To a certain extent somewhat independently of medical condition and to a certain extent is why I say pseudo-randomly. It’s not really a random assignment of patients to different treatments but to a certain extent it is somewhat random.

Slide 4

We note often that treatment patterns differ amongst those facilities not because only the patient conditions differ amongst the facilities but also because of differences in provider propensity to treat. I’m going to talk about this concept of propensity to treat. We can actually evaluate this empirically, we can measure the level of treatment at each facility. We can compare that to what would be expected based upon the patient mix and see if they’re treating at a higher level or a lower level than would be expected based upon patient condition. That’s what I refer to as propensity to treat. You’ll see this coming up in an example later on. Then finally, with instrumental variables we see if the outcomes differ according to the propensity to treat at different facilities.

Slide 5

I’m going to go over four examples, one of which I have shown before. I apologise to those of you who have seen it, it relates to EPO dose and hematocrit levels, another one related to duration of dialysis and phosphorous control. The third one is hospitalisation and readmission and by hospitalisation I mean the length of hospitalisation and the likelihood of readmission within 30 days. Finally, one which is a particular focus within the United States, the relationship between the race of the patient and the mortality rates amongst those patients.

Slide 6

So first of all, let me look at EPO dose and haemoglobin level.

Slide 7

From preliminary studies we know that higher EPO doses are associated or actually cause higher haemoglobin levels. So if we look at the data from an observational study and look at it patient by patient, what do you expect to see? I’m going to plot here for over 12,000 patients treated at 312 facilities from 12 different countries in DOPPS with adjustments. We’ll be looking at the relationship between EPO dose and haemoglobin levels.

Slide 8

This is the plot on the horizontal axis you have the EPO dose and each point represents a patient and on the vertical axis we have that patient’s haemoglobin level. You may be surprised because you expect to see a positive association here that is higher EPO doses would be expected to be associated with higher haemoglobin levels. What’s going on? You all know the answer of course. EPO doses are calibrated to achieve a targeted goal of haemoglobin. The goal is being achieved for nearly all levels of EPO dose. It’s between 11-12 there for the haemoglobin. So to the right of this plot we have somewhat resistant patients who require higher doses to achieve this same range of targeted 11-12 haemoglobins. Or here you have the patients who are very responsive who can achieve that with a low dose. So this is an example where we don’t have what is often called treatment by indication, we have treatment to achieve a target and when you have treatment to achieve a target typically you see the target achieved across the entire range of doses as we do here. In fact, the relationship is a little bit negative and that corresponds to the fact that resistant patients, even with a high dose, often don’t get to the same level of achieving the target as do patients who are more responsive.

Slide 9

The patient level analysis does not show what we know to be the true relationship between EPO dose and haemoglobin levels. If we characterise these patients to the facility level and instead of looking at the individual patient dose, if we look at the average dose given at the facility and compare that to the average haemoglobin level achieved here are the results that we see. Here each dot represents one dialysis facility from DOPPS and you can see a positive association, as expected facilities that have a higher average dose tend to have a higher haemoglobin level on average.
The slope of this relationship corresponds very well with the known results from previous clinical trials. So this is an example where the analysis by facility overcomes a bias of treatment to achieve a target which confounds or makes it difficult to interpret results from a patient level analysis.
I’m going to show another similar example that’s all I have on haemoglobin and EPO dose.

Slide 10

The next one is going to be treatment time and phosphorous control.

Slide 11

We again have the sample from DOPPS of over 12,000 haemodialysis patients, 300 plus facilities from 12 countries. We will be doing analyses with adjustment for the characteristics of those patients but more important than the adjustment really is the focus upon a facility level analysis.

Slide 12

Now, this is the relationship between treatment time which is shown on the horizontal axis and phosphorus on the vertical axis phosphorous, phosphate PO4 and you see virtually no relationship, that is patients who are given longer treatment times towards the right of the axis on average have about the same phosphorous phosphate level as do patients who have shorter treatment times. The relationship is not at all significant. If anything patients with higher phosphate levels are given somewhat higher or longer treatment times. Again perhaps I’m speculating here excuse me I am  a statistician not an – so forgive my ignorance here but perhaps because patients who are out of compliance with regards to phosphorous need a longer treatment time.

Slide 13

What happens if we look at this at the facility level? We grouped these patients and looked at the average treatment time compared to the average phosphorous level. What we see here, first of all let me explain the horizontal and the vertical axis. The horizontal axis is shown as a 0 here and plus or minus treatment time and this is treatment time compared to what would be expected given the patient mix and the patient mix with regards to treatment time is driven primarily by patient size here. Larger patients typically have a longer treatment time. But if we compare the treatment time to what would be expected based upon the characteristics such as the size of the patients, we can see that some facilities treat 20 minutes longer than that per dialysis session some 40 some even 60 and it’s very common to have 20 minutes less or 40 minutes less than what would be expected given the characteristics of the patients.

In addition, on the vertical axis we’ve also shown phosphorous relative to what is expected 0 means as expected given the overall characteristics of the patients at the facility and some facilities are a little bit different from the norm. They have higher phosphorous levels, others have lower phosphorous levels. What we see here although this looks like a relatively flat slope in fact, this is a substantial and statistically significant relationship. Longer treatment times are associated with lower phosphorous levels, better phosphorous control and it’s about 16% better in terms of the fraction of patients achieving a guideline per 30 minutes of extra dialysis beyond what is treated. So perhaps as expected, longer dialysis is associated with better phosphorous control.
When you look at it at the facility level it is significant and substantial. If you look at it at the patient level you see nothing.

Slide 14

Here’s another example actually where I’m not going to compare a facility level analysis with a patient level analysis but it’s per sentinel event. I’m looking at initial hospitalisations and we’re looking to see whether longer hospitalisation is associated with a higher rate of readmission within 30 days or a lower rate of readmission within 30 days.

Slide 15

What do you expect? Patients that are in the hospital for a sentinel event those who stay longer you expect them to be more likely to be readmitted or less likely. Well, if they’re in there longer they are probably sicker to begin with perhaps there will be a higher readmission rate later on and that’s a problem of by indication. The initial treatment is longer for patients who are sicker and the outcome is likely to be worse for patients who are sicker to begin with.
Let’s see how this actually works out again, we now have not 10.000 patients but we’re actually looking at 20,000 index hospitalisations and for each one we looked to see whether or not they were readmitted within 30 days. We will be focusing upon 300 facilities at which those 20.000 index hospitalisations occurred among 8000 patients. Again we will be adjusting for demographic characteristics and other factors related to the patients which strongly affect both the expected length of initial hospitalisation and the likelihood of readmission.

Slide 16

Here are the results and I’m actually showing both sets of results at the same time here. On the left we have the by patient or by hospitalisation analysis and as I indicated with my speculation, initial hospitalisations which are longer that are shown as more than 9 days on the horizontal axis compared to initial hospitalisation that are shorter less than 3 days on the horizontal axis correspond to higher rates of readmission which are shown here on the vertical scale with longer initial admissions compared to shorter initial admissions and it’s relatively monotonic. You can see the longer the initial hospitalisation, the more likely it is that there will be a readmission within 30 days. Very possibly this is because of the underlying health of the patient. Sicker patients have longer hospitalisations to begin with and a higher rate of readmission.
So how can we sort this out, if sickness or health of the candidate is driving this relationship? How can we learn whether or not it may be good to keep the patient in the hospital a bit longer? Perhaps give them a more complete therapy. What we can do is do this analysis at the facility level and what we have done is categorise all the patients into 3 groups of facilities. Here are facilities who typically or on the median have a length of stay less than 4 days. They tend to admit their patients for this index factor, for short hospitalisations less than 4 days. On the right we have facilities whose median length of hospitalisation is longer than 5 days and here we have facilities whose median length of index hospitalisation is between 4 and 5 days. What we can see is that the facilities whose patients initially are admitted for a longer period of time have now a lower rate of readmission within 30 days suggesting that perhaps it’s beneficial to the patients to assure that they get complete care even if it takes a longer hospitalisation to begin with, if your objective is to reduce the readmission rates. So this is an example again, where the by patient or by hospitalisation in this case analysis is probably confounded by the health of the patient but we can see a very different pattern when we look at this by groups of facility and their propensity to have either a longer or a shorter initial hospitalisation.

Slide 17

My final example will be perhaps unique from America but I suspect that many people in the audience understand some of the factors going on in America. We’ll be looking at the difference in mortality rates between black and non-black patients and we are limiting this analysis to US facilities in DOPPS I and DOPPS II.

Slide 18

We’re looking at mortality rates and one of the surprising perhaps effects that has been noted many, many times by many investigators is that in the United States black patients on dialysis have a lower rate of mortality and showing this as an RR or a relative risk of mortality than do non-black largely Caucasian patients. This corresponds in fact, to about a 22% lower mortality for black patients than for white patients even after you adjust for quite a few different characteristics. This is when you compare patients, all black patients to the non-black patients.

Slide 19

One question that has arisen in many investigators’ minds in the United States is why is this? Is this perhaps because black patients tend to be treated in urban environments and perhaps urban dialysis facilities are better than rural dialysis facilities. Maybe by the luck of the draw they’re getting better treatment because of the types of facilities that they’re being treated at. We can sort this question out by having a model that includes both simultaneously the patient indication of black, yes or no and the facility indication of their case mix. This is a percentage of their patients who are black.

Slide 20

So, we’re categorising facilities according to their case mix to see if facilities with more black patients that tend to have better or worse outcomes than facilities with fewer black patients. What this model suggests is it doesn’t matter at all what the case mix is at the facility on the overall mortality rate, no effect and it’s not at all statistically significant but it remains significant at the patient level suggesting that this is a person phenomenon rather than something about the dialysis facilities where blacks tend to be treated. This is a way that we can sort out fairly interesting questions to distinguish the individual factors from perhaps something that might be associated with where people are treated or the kind of medical care that they have access to.

Slide 21

So, I want to emphasise that there are 2 primary levels of analysis that we look at in DOPPS and in many studies. The patient level where we compare outcomes according to the treatment actually received that is as treated. We compare all the treated patients to all the untreated patients as 2 different groups. In contrast, we can look at groups of patients defined by facilities and we will notice that the facilities have different levels of treatment, so we still get to compare different treatment groups as when we compare facilities. When we look across those different levels of treatment, we can compare the outcomes and this leads to what we have been calling a facility level analysis. It’s an idea which also goes under the name of instrumental variables.

Slide 22

In conclusion I’ll say that patient-level analyses of treatment effects can be biased because treatment levels are often calibrated to achieve a particular target or because sicker patients are given higher doses. If instead we evaluate the impact on outcomes at different levels of treatment, at different facilities, we very often end up with a less biased analysis than if we work at the patient level.

Slide 23

Thank you very much.