Good morning, ladies and gentlemen, and thank you to Lou and to John for inviting me. Is this moving forward? Oh, there we are. I am from Northern Ireland, just outside Belfast. I didn't study at Queen's, I went to work there in 1992 after I finished my PhD. And about 1995 I started looking at the issues of keyhole surgery or minimally invasive surgery. And it was quite clear there were significant issues with learning laparoscopic surgery or minimally invasive surgery. And there certainly was a problem with higher morbidity and mortality at the start of the learning curve. But minimally invasive surgery, it was a disruptive technology. The surgical community assumed, because they attracted the brightest and the best and the most motivated probably, that it really wasn't going to be an issue for them to learn, and boy did they get that one wrong. But it forced the surgical community, which I think most of medicine would consider quite a conservative group, it forced them, and subsequently medicine, to think seriously about what it meant by training and education. And Rick Satava, who was a general surgeon, a military surgeon, suggested that maybe the operating room wasn't the best place to learn those skills. And over the last 20 years, one of the things that I've learned is you can't simply just get good technology or a good building, point the trainee towards it and say, OK, learn. Here's a learning curve that we've known what it looks like for over 100 years. We take a novice, we give them some education, they become an advanced beginner. We give them some training and they're competent. And what you can see is competency, which is what you're supposed to be doing, is really not that far up the learning curve. What I do is proficiency-based progression, which is significantly further up the learning curve and well on the way to being an expert. And what I do is I take a new skill or the use of a new device. We characterize that device use or that procedure in detail. We define rather than describe the metrics. Then we validate the metrics. Do your peers concur that those metrics appropriately characterize the procedure? Do the metrics distinguish between surgeons who are good at or experienced at performing the procedure and those that are not? Once we validate them, we then quantitatively define a performance benchmark based on the performance of experienced practitioners. The trainee is then required to engage in deliberate practice simulation training until they demonstrate that proficiency benchmark. And only when they demonstrate that quantitatively defined proficiency benchmark do they proceed to clinical practice. So metrics really are at the core of the entire process. So who do you want for the development of the metrics? Well, you want three to five procedure surgeons. I like majority. So with a three or a five, you're going to get a majority. They must be genuinely good at performing the procedure, not talk a good game. They should have no major ego or psychopathology issues. If they have to tell you how good they are frequently, if they have to have the last word, if they have to shout loudest, they're really not for that group. It's not a therapy session. They need good social skills. They need the ability to discuss, to debate and to compromise. They need good command of the English language because we define, we don't describe. They need a sense of humor because we don't always agree. They need to be pragmatic and they need time. They need to be committed. You're not going to do this in a couple of hours. The first time we used that approach was at the Department of Surgery at Yale when I was a Fulbrighter. We applied it to psychomotor training for laparoscopic surgery. The simulator that we used was the Minimally Invasive Surgical Trainer Virtual Reality. The surgical community had largely dismissed the simulator, said it wouldn't do because it didn't look like the real thing. I looked at it and said that'll do just fine. So we applied, we developed the metrics for laparoscopic cholecystectomy. We then took the surgical residents in years one to four. We randomized them to standard training which was two days intercorporeal suturing and knot tying training in Butch Rosser's skills laboratory or proficiency based progression virtual reality training on the simulator. The residents then went to the operating room and performed lap chole with an attending surgeon who didn't know how they were trained. Their performance was video recorded and that was assessed by two attending surgeons who didn't know how the person was trained or who they were. What we found was that those in the proficiency based progression trained group made significantly fewer objectively assessed interoperative errors. Up until the Copernicus study, this was the default study design for simulation based training studies. And there's multiple prospective randomized and blinded studies that have shown that this approach to training always trumps the traditional approach to training. And I'm not really going to go into those today. Karnas Pellegrini, we presented the results at the American Surgical Association in April 2002. Karnas was the first discussant. Karnas is a very good surgeon and he's a very good scientist and he thinks like a scientist. He said, Tony, maybe all that's happening in the simulation based training group is they're getting more training. Now I didn't think he was right, but that's a question that needs answered with data. And Anna came to me in the early 2010-2011 and said, we're building a new training center, but we also want to know is there a better way to train surgical skills. We picked a Bankart procedure as the vehicle to test this hypothesis. We used arthroscopic suturing and knot tying boards. We used a silicon shoulder training model. Trainees had the opportunity to practice their skills on a cadaver shoulder. And their index case was assessed on a quality assured cadaver shoulder. It meant that the trainees could complete the entire procedure so we knew precisely how they were doing. The most important lesson to take away from that slide is that we used what you had off the shelf. It's not what you have, it's what you do with it. There were 21 sites in the trial, everywhere from Amherst outside Chicago to Tufts to Yale. We randomized PGY-4s and 5s to the standard trained Anna. A simulation trained group and another simulation trained group. All of the trainees had an online orientation video that they had to view before coming on the training. All of them had their visual, spatial, perceptual and psychomotor aptitude assessed because we know that predicts learning curve and performance. And then group A got what they would normally get on a Bankart course, which is pretty good anyway. Group B got simulation, they had their lectures, they had their knot tying practice, then they could practice on a simulator. Group C had the exact same as group B with one difference. They had to test the proficiency the whole way. They had to pass a quantitatively defined performance benchmark on the online orientation material. On the knot tying they had to train to proficiency and on the simulator they had to train to proficiency. These were quantitatively defined performance benchmarks based on the performance of experienced arthroscopic surgeons. Then all of the trainees had to perform an assisted but donated Bankart repair on a fresh cadaver. This was assessed by five pairs of attending arthroscopic surgeons across the United States. The inter-rater reliability or the percentage of agreement between the raters was 93%. None of the assessments dropped below the gold standard of 80%. So the trainees come into the OLC on a Friday and they left on a Sunday afternoon. Here we have the mean and 95% confidence intervals of the number of steps completed by trainees. And what you can see is the proficiency-based progression train group are completing more steps of the procedure, but only 10% more. And what that tells us is that process metrics such as steps are really not that good a quality measure. You can complete all the steps and tick the box, but you can do it quite badly. Objectively assessed interoperative errors is a different kettle of fish. Here we have the mean and 95% confidence intervals of the three groups. And the proficiency-based progression train group were completing the procedure with 56% fewer objectively assessed interoperative errors and 41% fewer errors in the group that had the exact same training from the same level of faculty over the same weekend using the same resources. Why would you have A or B when you can have C? Rick Angelo then asked a difficult question the night before we started the study. We were in the pub and he says, how many do you think is going to reach proficiency? I had no idea. Because normally the trainees come down from the hospital to me in the training center and they come down for a couple of hours at a time and then they go back. They're not done until they demonstrate the proficiency. We had never crammed this into a weekend. What we found was that 29% of the trainees in the traditional train group demonstrated the proficiency benchmark. The simulation train group were only marginally better at 36%. 75% of the proficiency-based progression train group demonstrated the proficiency benchmark. When you model that data in a logistics regression model, you can see that the proficiency-based progression train group were more than seven times as likely to demonstrate the proficiency benchmark as the standard train group. So proficiency-based progression simulation produced an unambiguously superior surgical skill set. This is the dig. It has to be more than an interesting educational experience. Just adding simulation to it doesn't really give you much. So why are the results of this study so important? The first thing is that we now know that surgical skills are related to outcomes. Berkmeyer in the greater Michigan area assessed the performance of attending, not resident, laparoscopic gastric bypass surgeons. He had them send in their best video recording of them doing a laparoscopic gastric bypass. He had it objectively assessed. And on the basis of that objective assessment, they were banded into four quartiles. Those performing worst to middle quartiles. Those performing best. He then followed up all the patients that those surgeons operated over the next six years. The complication rates, readmission rates, re-operation rates, mortality rates. Not surprisingly, what they found was that those surgeons in the top quartile had significantly lower complication rates, lower readmission rates, lower mortality rates. That wasn't a surprise. That was a surprise. The lowest difference, the smallest difference between the two groups was that those in the top quartile had a 52% lower re-operation rate. And those in the bottom quartile had an 81% higher mortality rate. If you had asked me 10 years ago what the difference would have been, I'd have said maybe 5%, maybe 10%. That's not what it looks like. So performance and skills do impact on clinical outcomes. And what we're about is training skills. So why are the results of the Anna study important, the Copernicus study? Well, number one, the default gold standard experimental design in this field is now the Copernicus study. I mean, there's no doubt about that. The results from the Copernicus study offer the most compelling evidence on the impact of proficiency-based progression simulation training. I've just come from a consensus meeting in Europe where all the societies wanted to use robotic surgery. This was the evidence that convinced them, OK, we get it. This is a better way to train. The metrics are at the core of this process. The metrics are derived from experienced practicing surgeons that are clinically grounded. The society, the association has to protect those metrics. We've applied for a patent for the metrics because Anna has to own the intellectual property of those metrics. Now, I'm not too sure whether we're going to get the patent. I have no idea. But I'll tell you something, you're going to have to work out how you protect that intellectual property. The Copernicus study, for me, I think it's the best because of the collation of your evidence. It was led by a professional society, not some researcher in a university. It was led by a practical surgical society. It's the largest multi-center trial of simulation-based training. And it's a textbook example of the development and validation of metrics. It's currently the best available scientific evidence that simulation-based training is a better way to train surgical skills. And another proficiency-based progression course is due in September for two surgical procedures. Not only are you ahead of the game, but you're pushing the boundaries even further. You have no idea about all the other organizations in the United States and around. They're trying to do this. They're not even at first base yet. And here's you, you've completed a full prospective randomized and blinded clinical trial. And you're moving to teach two courses on the one weekend. And as an outsider, I don't think you're in this position by accident. This is a photograph taken from Chicago in 2013. And I think you're in this position because of a combined effort from the association. And take a look at that photograph on the right. How many past and future leaders do you see in that photo? And that's why you're in the lead. Halstead, I mean, we're still using the Halstead training paradigm, but Halstead published only one paper on his approach to training. And it was after a lecture that he gave at Yale with his alma mater. There's no point in being the best with the robust scientific evidence if nobody knows about it. And arthroscopy really did step up to the plate. I mean, there's something like seven valid quantitative validation papers on the Copernicus, including three editorials. I think Jim Leibovitz and Matt were looking at Rick and I when we first started talking to him about education and training, and they were wondering what we were smoking. When we put in the data, they published an editorial on what we had said. And the apprenticeship model for surgical training is inferior. It really says a lot. Proficiency-based progression training derived from a procedure characterization from surgeons who actually know what they're talking about and know what they're doing works. The principles and practices are the same across all of medicine and healthcare. This is not just about arthroscopic surgery. This is about a new approach to training in medicine. But the Copernicus has quantitatively shown that training must be more than an interesting educational experience. That is not a small fact. You just can't throw money at this here and hope that it's solved. You have to think your way through it. I think as an organization, you've got so much right about how skills metrics are developed and validated, because that's the core of it. About how surgical skills should be trained and how the training should be validated. So what I want to know is what are you going to do with it? Because whether you like it or not, you're currently lead medicine in the new science of how surgeons and doctors should be trained. Thank you.

surgical skills

training and education

minimally invasive surgery

proficiency-based progression

simulation-based training