Data were available for 70, attempts at Part 1 by 39, candidates at 24 diets, for 37, attempts at Part 2 by 23, candidates at 25 diets, and for 40, attempts at PACES by 21, candidates at 29 diets. For the present analyses, all candidates have been included, many of whom are non-UK graduates, and who on average perform somewhat less well than UK graduates, although that makes no difference to the present analyses. In interpreting these data it should be remembered that they are censored and truncated. The data are right-censored in that for recent diets some candidates may have taken the examination only once or twice and will continue in the future to make more attempts.
The data are also left-censored in that at the first attempt for which these data were available, some candidates were already on a second or higher attempt or may already have passed, for example, Part 1, and so results are only available for Part 2 or PACES. The data are also truncated in that some candidates voluntarily withdraw from the examination at, for instance, the nth attempt, without having passed. Figure 1 shows, for each part of the examination, the highest number of attempts recorded for each candidate and the attempt at which, if any, the examination was passed.
The highest number of attempts recorded for Part 1 was 26 two candidates , for Part 2 was 21 one candidate , and for PACES was 14 one candidate. The top row shows that the distributions are heavily skewed to the left, so that it is difficult to see the right-hand end of the distribution. The lower row shows the same results plotted on a logarithmic ordinate. To a first approximation, except for the first few attempts, the distributions are exponential, falling away by a similar proportion at each attempt. The lines for the attempt at which an examination is passed are generally steeper than the line of the highest attempt, implying that at each attempt a smaller proportion of candidates passes.
Top row: The figures show, for Part 1, Part 2 and PACES, the highest number of attempts at the examination grey bars , and if the examination was passed, the attempt at which it was passed pale green bars. Bottom row: The same data as in the top row but the ordinate is on a logarithmic scale.
The fitted lines are lowess curves, blue for highest attempt and green for attempt at which the examination was passed. As well as attempts at each individual part, the total number of attempts to pass all three parts of the examination was calculated although this is not straightforward, as not all candidates passing Part 1 go on to take Part 2 and so on. Since the concern is mainly with those passing MRCP UK overall, the analysis is restricted to the 10, individuals in the database who had taken and passed all three parts of the examination. The minimum number of attempts to gain MRCP UK is, of course, three one for each part , the mean number of attempts was 5.
The 90 th percentile to pass all three parts was 8, the 99 th percentile was 15, and the maximum number of attempts to pass was Although not shown here, the distribution was also exponential, being almost perfectly straight when plotted on a logarithmic ordinate. Considering only the 10, candidates who passed all three parts of the examination, 1.
Overall, 2. Although the main analyses will not, for reasons already discussed, be separating UK trained doctors from non-UK trained doctors, here we provide some brief descriptive statistics on three groups: UK graduates, UK trainees identified probabilistically as non-UK graduates with a UK correspondence address , and non-UK doctors neither UK graduates nor a UK correspondence address.
For the 6, UK graduates, the mean total number of attempts to pass all three parts was 4. The 90 th percentile was six, the 99 th percentile was 11, and the maximum number of attempts to pass was A total of 0. For the 2,UK trainees, the mean total number of attempts to pass all three parts was seven SD 3. The 90 th percentile was 12, the 99 th percentile was 18 and the maximum total number of attempts to pass was In all, 4. For the 1, non-UK doctors, the mean total number of attempts to pass all three parts was 5.
The 90 th percentile was nine, the 99 th percentile was 16, and the maximum number of attempts to pass was In all, 1. Because the data being analyzed are necessarily multilevel, simple descriptive statistics which do not take that structure into account are potentially very misleading. However, since that is the immediate way in which most users will encounter such data, we explore the data for the Part 1 examination only to give a sense of how the data look and the problems of interpreting them.
Figure 2 shows a histogram of the marks attained by all candidates on their first attempt at Part 1. The distribution is approximately normal, but skewed somewhat to the left, with a few candidates performing very badly. The marks in Figure 2 have been divided according to the outcome of candidates' second attempt at Part 1. Some of the candidates, shown in blue, do not take Part 1 again as they passed at their first attempt.
The candidates in green and pale yellow took the examination a second time, those in green passing on the second attempt, whereas those in pale yellow failed on the second attempt and they have rather lower marks at their first attempt than those who passed on the second attempt. There is also a large and rather problematic group, shown in purple, who never took Part 1 again and in some cases that was despite having a mark only just below the pass mark, so they would have had a high chance of passing at a second attempt.
Nothing further is known as to why the candidates in purple did not take the examination again, although it may be that some had taken the examination prior to a final decision about career choice and the examination had subsequently become irrelevant to their needs. Distribution of marks attained at the first attempt at MRCP UK Part 1, according to whether the examination was passed blue , the examination was passed at the second attempt green , the examination was failed at the second attempt pale yellow , or the examination was not taken again purple.
Figure 3 shows the average marks of candidates at each attempt at the examination for those who had a total of one, two, three, up to twelve, attempts at the examination. The lines 'fan out', those taking the examination only two or three times having steeper slopes than those taking it ten or more times.
Buy Maximum Occasions by John O'Loughlin (ISBN: ) from Amazon's Book Store. Free UK delivery on eligible orders. 'Occasional Maxims & Maximum Occasions' resembles other reverse-title books like 'Informal Maxims & Maximum Informality' () in its basic two-fold design.
All groups, even those taking the examination up to twelve times, appear to be improving across all attempts. There is also clear evidence of a 'jump' at the last attempt which is due to some candidates exceeding the pass mark and therefore not needing to take the examination again. The groups at each attempt who pass or fail the examination are separated out in Figure 4 , which shows the average mark of candidates on their nth attempt, according to whether they passed or failed at that attempt.
More interestingly, those who pass at later attempts have lower marks when they eventually passed than those who passed at earlier attempts; and conversely, those who fail at later attempts have higher marks than those who fail at earlier attempts. Also of particular interest is that the lines seem to flatten out after about the seventh or so attempt.
The average mark at each attempt at the Part 1 examination according to the number of attempts made at the examination, from 1 to N varies from 19 to 22, The dashed grey line shows the pass mark. Figures 2 , 3 and 4 do not show longitudinal results of individual candidates. In contrast, Figure 5 shows the marks of candidates at their second attempt, in relation to their performance at the first attempt and, of course, all of these candidates had marks of less than zero at the first attempt because they had failed previously.
On average, candidates do better on their second attempt than their first, with very poorly performing candidates improving the most. Although the latter is what might be expected from regression to the mean, it is worth noticing that the mean on the first attempt of all candidates is actually at about -4, and, therefore, it might be expected that those with marks greater than -4 would do worse on a second attempt, which they do not do.
Note that the fitted line is a Lowess curve, although it is almost indistinguishable from a straight line except for a slight change in direction between -5 and 0. Interpreting Figures 2 , 3 , 4 and 5 is possible, but is not straightforward, mainly because the data are inherently multi-level. A better approach is to model the data formally and for that MLMs are needed, with individual examination attempts being at level 1, and candidates being at level 2.
Since MLMs can be complex, to prevent the flow of the argument being disrupted or becoming too confusing in the main text, details are presented in Additional File 1. Readers with a technical understanding of MLMs are referred to that file, whereas other readers should hopefully be able to understand the key ideas of the main paper without needing to refer to the details.
It should be pointed out that MLMs can model two very separate aspects of the data, and these will be considered separately. Firstly MLMs can look at fixed effects, which consider the average performance of all candidates, and secondly it can ask questions about random effects, which consider how candidates differ in their performance around a fixed effect. Fixed effects are mainly of interest for considering the overall process, whereas random effects are of much greater interest for understanding the educational and psychological processes which underpin the changes in performance of candidates retaking examinations.
Fitted parameters are shown by MLwiN in green and give the estimate followed in brackets by its standard error.
In this paper we will firstly describe a simple linear random effects model for the Part 1 data, in order to give non-expert readers a flavour of what random effects models can do, and then we will go on to describe more complex random effects models using a dummy variables approach using MLwiN and the negative exponential curve using SAS. The same is also true of undergraduate examinations, where it is generally the case at present that only one or perhaps two attempts at finals or other examinations are allowed although historically it has not always been so. Minifourgonnette ou fourgonnette Because of the exponential function, the achievement only approaches the maximum level asymptotically, becoming ever closer in smaller and smaller steps, but never actually reaching it. You may receive your message by email, by mail or both. Some minor changes in the examinations have occurred since then, with the number of questions in Part 2 changing details are provided elsewhere [ 15 ].
At the measurement level level 1 there is variability resulting from individual attempts by candidates, and this has a variance of Individual attempts by candidates are nested within the second, candidate, level, the variance of which is The variances at the candidate and attempt levels are random factors. There are two fixed factors in model M1, both at the candidate level, and these are fitted as a conventional regression model according to the attempt number. For convenience, attempt at the examination is indicated by the variable Attempt0, which is the attempt number minus one, so that the first attempt is 0, the second attempt is 1, and so on.
The intercept is The slope is 2. On this model, candidates therefore show significant improvement at later attempts on the examination, improving on average by 2. Model M1 see text. The model is fitted in MLwiN and shows the 'Equations' screen from MLwin black and green fonts , annotated in red to indicate the meaning of the various components. Although model M1 is simple, it is clearly too simple as it implies that candidates improve by the same amount at each resit and if that continued for ever then as the number of attempts increases the performance of each candidate would eventually reach the pass mark and all candidates eventually would pass the examination.
A more intuitive approach is adopted in Model M2 in which candidates improve less and less at each attempt, perhaps eventually 'topping out' at some level. That possibility can be examined in MLwiN by fitting a purely empirical model in which there is a separate 'dummy variable' for each attempt, Dummy7, for instance, indicating by how much performance at the seventh and subsequent attempts is better than performance at the sixth attempt. The details of the fitting of model M2, are provided in Additional File 1. Here we restrict ourselves to showing, in Figure 7 , the estimates of the dummy variables at each attempt, along with their confidence intervals.
The left-hand graph shows the estimates for Part 1 and it can be seen that the extent of the improvement at each step falls with each attempt but that the improvement is still significant from the ninth to the tenth attempt.