substantial surgical skills are required to perform an arthroscopic procedure without risking iatrogenic injury of articular cartilage and within the routine time scheduled for the operation [ 16 , 17 , 20 , 23 ] .
learning arthroscopic skills takes considerable time and implicates an increased risk of surgical errors during the early stages of the learning curve when operating on patients [ 4 , 17 , 24 ] . the traditional learning model where
the trainee is supervised continuously by the surgeon attempts to minimize these surgical errors . however , as the training time for acquiring arthroscopic skills is being reduced [ 8 , 12 ] and demands from society for high - quality healthcare increase , initiatives have been taken to train basic skills away from the operating room [ 8 , 12 ] .
training arthroscopic surgical skills preferably is performed with actual instrument handling . the theory that states skilled motor behavior relies on accurate predictive models of our body and the environment we interact with ( eg , instruments ) supports this approach [ 5 , 14 , 28 , 33 , 34 ] .
these predictive models are stored in our central nervous system . to do a certain task ,
a key feature in this theory is that these predictive models are tuned , updated , and learned by providing feedback from our sensory organs ( vision and proprioception ) .
traditionally , cadaveric material has been used as a substitute for live patients [ 10 , 20 ] .
its importance is evident ; however , disadvantages are the availability of cadaveric material and preparation time
. two types of simulators have been introduced to overcome the disadvantages of cadaveric material : anatomic bench models [ 25 , 32 ] and virtual reality systems [ 4 , 15 , 17 , 26 , 35 ] .
however , it is unclear if these simulators qualify as suitable training means for arthroscopic skills .
we therefore addressed the following questions : ( 1 ) do commercial simulators have construct ( times to perform tasks ) and face validity ( realism ) , and ( 2 ) is the perception of usefulness ( educational value and user - friendliness ) related to level of experience ?
on february 18 , 2009 , we performed a systematic search with the google and yahoo !
search engines as these contain the largest set of web pages [ 1 , 7 ] .
combinations of search terms were used : arthroscopy , simulator , orthopaedic , models , simulation , and trainer .
companies were sent an invitation with the request to provide the simulator for 2  weeks at our institute .
two companies agreed to participate : toltech knee arthroscopy simulator ( touch of life technologies , aurora , co , usa ) and insightarthrovr arthroscopy simulator ( gmv , madrid , spain ) .
the other companies [ 6 , 9 , 19 , 21 , 25 , 29 , 30 ] refrained for various reasons unrelated to financial issues .
the toltech knee arthroscopy simulator ( simulator a ) is a virtual reality simulator for arthroscopic knee surgery with two handles that give haptic feedback ( fig .  1 ) ( appendix 1 ) . the insightarthrovr arthroscopy simulator ( simulator b ) is a virtual reality simulator for arthroscopic knee and shoulder surgery with a multitool that gives haptic feedback ( fig .  2 ) ( appendix 1).fig . 
2a photograph shows a participant performing tasks on simulator b. a photograph shows a participant performing tasks on simulator a. a photograph shows a participant performing tasks on simulator b. we recruited 37 participants , including ( 1 ) all staff members practicing arthroscopy routinely and present at the time of testing ( except the main researcher gmmjk ) , ( 2 ) all residents present at the time of testing , and ( 3 ) medical students and researchers of our orthopaedic department .
the participants were divided into three groups having different levels of arthroscopic experience : novices who had never performed an arthroscopic procedure , intermediates who had performed up to 59 arthroscopies , and experts who had performed more than 60 arthroscopies .
this boundary level of 60 arthroscopies was based on the average opinion of fellowship directors who were asked to estimate the number of operations that should be performed to allow a trainee to perform unsupervised meniscectomies .
simulator a was evaluated by 22 participants in april 2009 and simulator b by 22 participants in october 2009 ( fig . 
subgroups were made on arthroscopic experience at three levels based on the number of arthroscopies performed : novices ( 0 ) , intermediates ( 159 ) , and experts ( >  60 ) .
the age in years and the number of attended arthroscopies (  observation  ) are expressed as median with range in parentheses .
the number of participants who previously had used a simulator (  simulator  ) or had experience in playing computer games (  games  ) is shown .
subgroups were made on arthroscopic experience at three levels based on the number of arthroscopies performed : novices ( 0 ) , intermediates ( 159 ) , and experts ( >  60 ) .
the age in years and the number of attended arthroscopies (  observation  ) are expressed as median with range in parentheses .
the number of participants who previously had used a simulator (  simulator  ) or had experience in playing computer games (  games  ) is shown .
the researcher showed the selection of exercises and performance of the calibration protocol and tasks for the test .
the assessment of construct validity ( time to perform a task ) was based on one basic navigation task .
as the simulators were unlikely to offer a navigation task that was the same , one navigation task was prescribed that can and could be performed on all simulators for comparison . with the arthroscope placed in the anterolateral portal and the probe in the anteromedial portal
, nine anatomic landmarks had to be probed sequentially : medial femoral condyle , medial tibial plateau , posterior horn of the medial meniscus , midsection of the medial meniscus , acl , lateral femoral condyle , lateral tibial plateau , posterior horn of the lateral meniscus , and midsection of the lateral meniscus .
the participants were asked to repeat this navigation task up to five times in a limit of 10 minutes .
the navigation task time was defined as described previously   and determined with a separate video recording of the simulator monitor in which the virtual intraarticular joint is presented .
we recorded the median time per experience group for each of five repetitions of the navigation task .
face validity ( realism ) , educational value , and user - friendliness were determined by giving the participants a second task in which exercise(s ) had to be performed that were characteristic for that particular simulator and by asking them to fill out a questionnaire afterward .
the exercises were selected by the faculty surgeon ( gmmjk ) and the company to be sure that they best represented the capability of the simulator .
assistance in performing these exercise(s ) was given only if a participant failed to continue for a period of 2 minutes .
the characteristic exercise chosen for simulator a was  inspection of the suprapatellar pouch with only the 30 arthroscope .
 this exercise is set up in three stages : watching an instruction video of the exercise , performing the exercise once guided by example hint - images in a stepwise sequence , and performing the complete exercise once again without guidance .
the exercise chosen for simulator b was threefold : microfracture technique to treat a cartilage lesion in the femoral condyle , visual exploration and probing of a superior labrum anterior superior lesion , and placement of three suture anchors repairing a bankart lesion ( shoulder instability ) .
3 ) ; face validity of the outer appearance of the simulator , the intraarticular virtual joint , and the virtual instruments ( table  1 ) ; educational value ; and user - friendliness ( table  2 ) .
questions were answered using a 10-point numerical rating scale ( nrs ) ( eg , 0  =  completely unrealistic and 10  =  completely realistic ) or dichotomous requiring a yes / no answer .
a 10-point nrs was chosen as all participants were dutch and this grading system is used at all educational institutions .
thus , we expected the grading to be performed based on uniform interpretation of the nrs .
some questions featured a  not applicable ( n / a )  answer option , which could be used solely by novices , as these questions required prior knowledge of the real - life arthroscopic situation .
for the same reason , only the answers from the expert and intermediate groups were used on simulator realism and educational value .
only answers from the novice and intermediate groups were used on user-friendliness.table  1questions addressing face validityface validity aspectquestionouter appearancewhat is your opinion of the outer appearance of this simulator?is it clear in which joint you will be operating?is it clear which portals are being used?intraarticular jointhow realistic is the intraarticular anatomy?how realistic is the texture of the structures?how realistic is the color of the structures?how realistic is the size of the structures?how realistic is the size of the intraarticular joint space?how realistic is the arthroscopic image?instrumentshow realistic do the instruments look?how realistic is the motion of your instruments?how realistic does the tissue feel when you are probing?all questions were answered on a 10-point numerical rating scale.table  2questions addressing educational value and user - friendlinessparameterquestioneducational value ithe simulator allows training of joint inspection*the simulator allows training of therapeutic intervention*the simulator allows training of joint irrigation*the variation of exercises offered by the simulator is adequate*difference in required skill level between exercises is adequate*educational value iithe simulator is a good way to prepare for a real - life arthroscopic operation*user - friendliness ihow clear are the instructions to start an exercise on the simulator?how clear is the presentation of your performance by the simulator?is it clear how you can improve your performance?how motivating is the way the results are presented to improve your performance?user - friendliness iii felt the need to read a manual before operating the simulator * *  questions requiring a dichotomous yes / no answer ; all other questions were answered on a 10-point numerical rating scale .
questions addressing educational value and user - friendliness *  questions requiring a dichotomous yes / no answer ; all other questions were answered on a 10-point numerical rating scale .
the presence of normal distributions of task times was assessed by kolmogorov - smirnov tests .
owing to small sample sizes and skewed distributions , the task times were analyzed nonparametrically .
construct validity was determined for each simulator independently by using kruskal - wallis tests to calculate the overall presence of differences in task times between the three experience groups for each of the five task repetitions .
the significance level was adjusted for multiple comparisons with the bonferroni - holm procedure ( alpha  =  0.05 ) ; when we detect significant differences we performed pair - wise comparisons between the experience groups separately using mann - whitney u tests .
the scores of the three separate aspects of face validity of the simulators ( table  1 ) and user - friendliness i ( table  2 ) were expressed as mean summary scores of the corresponding questions .
educational value i ( table  2 ) was expressed as a sum score of five dichotomous questions and ranged from 0 to 5 . the mean summary scores ( face validity and user - friendliness i ) were verified for normality by kolmogorov - smirnov tests , expressed as mean and sd , and assessed for differences between both simulators with student s t tests .
the ordinal scale of educational value i was presented as medians with ranges and analyzed using a mann - whitney u test .
the dichotomous questions ( educational value ii and user - friendliness ii ) expressed as categorical yes / no answers were presented as frequencies and percentages ( % ) and analyzed by chi square tests or fisher s exact test ( in case one or more cells had expected counts less than five ) .
the significance level was adjusted for multiple comparisons with the bonferroni - holm procedure ( alpha  =  0.05 ) .
with the exception of two participants , the novice group completed only the first repetition of the navigation task or none at all on simulator a within the time limit ( fig . 
the novices were slower ( p  =  0.001 ) in completing the first repetition .
post hoc analysis showed the navigation task times of the experts ( median , 125 seconds ; range , 68245 seconds ) and the intermediates ( median , 129 seconds ; range , 60311 seconds ) were faster ( p  <  0.001 and p  =  0.01 , respectively ) than those of the novices ( median , 447 seconds ; range , 181600 seconds ) ( fig . 
the task times of the intermediates and the experts were similar ( p  =  0.93 ) .
no differences were observed between the experience groups for the other repetitions . for simulator b , we observed slower task completion of the novices for the second and third repetitions ( p  =  0.005 and p  =  0.008 , respectively )
. the navigation task times of the first repetition of the experts ( median , 90 seconds ; range , 65177 seconds ) were not faster than those of the novices ( median , 165 seconds ; range , 109605 seconds ) ( p  =  0.019 ) and those of the intermediates ( median , 105 seconds ; range , 75204 seconds ) ( p  =  0.503 ) ( fig .  4 ) .
post hoc comparisons of the second and third repetitions showed faster ( p  =  0.001 and p  =  0.002 , respectively ) task times of the experts compared with those of the novices ( fig .  4 ) .
the task times of the intermediates were not different compared with those of the experts or novices for these repetitions.fig . 
4a  bthe graphs show the results of the navigation repetitions for ( a ) simulator a and ( b ) simulator b. the results are presented as medians with ranges .
construct validity was observed for the first repetition of simulator a and the second and third repetitions of simulator b. the graphs show the results of the navigation repetitions for ( a ) simulator a and ( b ) simulator b. the results are presented as medians with ranges .
construct validity was observed for the first repetition of simulator a and the second and third repetitions of simulator b. the mean face validity scores of the outer appearance and simulated intraarticular joint were 7.3 ( sd , 1.4 ) and 6.4 ( sd , 1.4 ) for simulator a and 8.4 ( sd , 0.6 ) and 6.1 ( sd , 0.9 ) for simulator b , respectively .
the mean face validity score of the simulated instruments was 4.9 ( sd , 1.5 ) for simulator a and 5.7 ( sd , 1.2 ) for simulator b. thus , the face validity of the simulated instruments was judged barely sufficient for both simulators ( fig .  5 ) .
the median sum score for educational value i was 3 ( range , 15 ) for simulator a and 5 ( range , 25 ) for simulator b ( p  =  0.009 ) .
simulator a was judged suitable for real - life surgery ( educational value ii ) by 10 of 11 participants ( 91% ) , as was simulator b by all 13 participants ( 100% ) ( p  =  0.46 ) .
the mean score of 8.3 ( sd , 1.0 ) for user - friendliness i of simulator b was greater ( p  <  0.001 ) than that for simulator a ( 6.5 [ sd , 1.3 ] ) ( fig . 
( p  =  0.002 ) respondents felt the need to read the manual ( user - friendliness ii ) before operating simulator a ( 11 of 15 , 73.3% ) than before operating simulator b ( two of 13 , 15.4%).fig . 
5a graph shows the results of the normalized sum scores for face validity and user - friendliness i. the values are expressed as means with sds .
user - friendliness i is the combined opinion of the intermediates and novices ; the other columns are the combined opinions of the experts and the intermediates .
the mean score for user - friendliness i of simulator b was greater ( p  <  0.001 ) than that for simulator a. a graph shows the results of the normalized sum scores for face validity and user - friendliness i. the values are expressed as means with sds .
user - friendliness i is the combined opinion of the intermediates and novices ; the other columns are the combined opinions of the experts and the intermediates .
differences were not observed for any aspect of face validity between the simulators . the mean score for user - friendliness
i of simulator b was greater ( p  <  0.001 ) than that for simulator a.
as arthroscopic simulators gain maturity and become commercially available , it is unclear whether they are suitable for use in training .
we therefore addressed the following questions : ( 1 ) do commercial simulators have construct ( times to perform tasks ) and face validity ( realism ) , and ( 2 ) is the perception of usefulness ( educational value and user - friendliness ) related to level of experience ? we note limitations to our study .
first is the relatively small number of participants in each experience group , which could have led to nonsignificant results and the skewed distribution of task times .
the groups could not be enlarged owing to logistic limitations ; however , care was taken to include all experts and intermediates present at the time of testing to prevent selection bias .
other evaluation studies with simulators have experienced similar problems in recruiting participants [ 3 , 17 , 22 , 32 ] .
second is the absence of transfer or predictive validation , which was not feasible in the time frame .
studies performed with similar arthroscopy simulators [ 9 , 12 ] do show training on these systems decreases the operative learning curve .
these findings are in line with the opinion of all participants that training on either simulator will be good preparation before performing real - life arthroscopy .
third , our study is limited to two arthroscopic simulators , which were , in principle , not that distinctive as they are both virtual reality systems with haptic feedback devices .
if other types of simulators would have been included , such as anatomic bench models , a wider palette of alternatives could have been described and differences would be more pronounced .
the choice of this task is in line with tasks evaluated in other studies [ 12 , 17 , 22 , 32 ] and is indicated as an important arthroscopic skill to master before operating in the theater .
fifth , only a few tasks were used to determine face validity , educational value , and user - friendliness .
these tasks were chosen carefully and reflected the way exercises are built up and feedback on performance is given by each simulator .
therefore , we assumed the participants were given a good impression of the learning environment of each simulator .
sixth , the choice of expert level was somewhat arbitrary , especially for the novice versus intermediate groups .
this could have influenced the demonstration of construct validity as the experience level might have been insufficiently distinctive .
4 ) because the task times were substantially similar for all repetitions between the novices and experts , and similar between intermediates and experts .
these findings are comparable to those in the study by srivastava et al . , who used a similar division in experience levels and found no substantial differences between the groups .
they speculated the results may be influenced by the fact that experts knew what to expect and novices were very motivated .
a more detailed comparison with other studies can not be made as the criteria to qualify as expert , intermediate , or novice differ among studies [ 17 , 26 , 32 ] , or a different acceptable significance level was chosen .
it is recommended to set uniform experience levels when performing this type of study . by using the study of oneill et al . ,
task time was chosen as an outcome measure , as it is widely used and validated in assessing surgical skills learning , it can be measured using all commercially available arthroscopy simulators , and it makes overall objective comparison possible . face validity was observed for both simulators , although there is room for improvement .
the presence of tactile feedback in an arthroscopy simulator is considered essential to imitate clinical practice adequately and train safe manipulation [ 22 , 36 ] .
intermediates and experts indicated tissue probing was unrealistic on both simulators ( fig .  5 ) .
training skills without receiving natural feedback could lead to an offset in the internal models stored in our central nervous system .
this might increase errors in the operating room . performing realistic force feedback for cutting or shaving is another challenge to implement in these simulators .
the intraarticular joint space of simulator b was considered large . additionally , as both simulators present virtual reality images , they leave an artificial impression .
these face validity results are comparable to results of other studies , in which imitation of the real - life situation generally is sufficient , but none is given a perfect score [ 2 , 17 , 18 , 31 , 32 ] . an explanation could be that simulators that do not resemble a human joint are graded more mildly as it is so obvious that they do not resemble reality , whereas simulators that come close to the real - life representation are scrutinized more thoroughly for small deviations .
this subjective opinion is supported by issenberg et al . , who identified a top 10 list of most important educational criteria for medical simulators .
both simulators fulfill seven of 10 criteria , including the most important ones : give feedback on performance , allow repetitive practice , and allow integration into the curriculum .
unfortunately , they do not offer training of precise portal placement , which is another important skill to be mastered before starting to operate on patients .
overall , simulator b was considered more user - friendly than simulator a , although simulator a was graded satisfactory ( fig . 
the feedback given by simulator b resembles the way mainstream computer games do this . for both simulators
simulator b offers a larger variety of exercises , and is more user - friendly , whereas simulator a showed a more distinct difference in task time between experts and novices .
subsequently , the user interfaces guide the trainees through a calibration protocol , after which each of the simulators guide the trainee through the range of exercises in a self - learning curriculum .
simulator a has a dell pc [ dell , round rock , tx , usa ] with a quad core processor and high - end video card , two monitors , and two handles .
the left monitor displays a virtual mentor , and the right monitor displays the simulated arthroscopic view ( fig .  1 ) .
both handles incorporate haptic feedback ( phantom ; sensable technologies , woburn , ma , usa ) .
simulator a provides a complete self - learning curriculum for training of diagnostic knee arthroscopy with 10 assorted meniscal tears that can be inspected and probed . after an exercise , an overall score ( 0%100% ) is provided consisting of a combination of metrics : camera and scope angle , limb flexion , varus / valgus force , position of the tip of the arthroscope or probe , task time , and whether a particular structure has been probed . at the lowest difficulty level ,
its purchase price is $ 85,000 and comes with an optional annual service agreement of $ 15,000 , which covers hardware and software updates .
simulator b has a dell pc with a quad core processor and a high - end video card and software version v.5.0.2 , interchangeable models representing knee and shoulder anatomy , and one monitor with touch screen displays the simulated arthroscopic view or the curriculum ( fig .  2 ) .
a simulated 30 arthroscopic camera and multipurpose tool are provided with haptic feedback ( phantom omni ; sensable technologies ) .
the multipurpose tool represents graspers , a power shaver , a probe , or a  chondropick  depending on the exercise .
simulator b provides a complete self - learning curriculum that guides the trainee through the tasks to train basic diagnostic and therapeutic skills ( eg , microfracturing or meniscectomy ) in the knee and shoulder : seven meniscal tears and three subacromial and four glenohumeral disorders . after an exercise ,
simulator b offers a selection of metrics to provide feedback in a graphic presentation : score between 0 and 10 , score related to task completion dependent on the task , task time , covered distance of camera and instrument , roughness of camera and instrument motion , instrument collisions , and covered distance . during the exercise , textual hints are given on how to proceed .
its purchase price is $ 91,283 , including the shoulder and knee modules , and comes with an optional annual service agreement of $ 10,945 , which covers hardware and software .
 backgroundsome commercial simulators are available for training basic arthroscopic skills . 
 however , it is unclear if these simulators allow training for their intended purposes and whether the perception of usefulness relates to level of experience.questions/purposeswe addressed the following questions : ( 1 ) do commercial simulators have construct ( times to perform tasks ) and face validity ( realism ) , and ( 2 ) is the perception of usefulness ( educational value and user - friendliness ) related to level of experience?methodswe evaluated two commercially available virtual reality simulators ( simulators a and b ) and recruited 11 and nine novices ( no arthroscopies ) , four and four intermediates ( one to 59 arthroscopies ) , and seven and nine experts ( >  60 arthroscopies ) to test the devices . 
 to assess construct validity , we recorded the median time per experience group for each of five repetitions of one identical navigation task . to assess face validity , we used a questionnaire to judge up to three simulator characteristic tasks ; the questionnaire asked about the realism , perception of educational value , and perception of user-friendliness.resultswe observed partial construct validity for simulators a and b and considered face validity satisfactory for both simulators for simulating the outer appearance and human joint , but barely satisfactory for the instruments . 
 simulators a and b had equal educational value according to the participants . 
 user - friendliness was judged better for simulator b although both were graded satisfactory . 
 the perception of usefulness did not differ with level of experience.conclusionsour observations suggest training on either simulator is reasonable preparation for real - life arthroscopy , although there is room for improvement for both simulators.clinical relevancethese simulators provide training in surgical skills without compromising patient safety . 
Clinical Relevance
Introduction
Materials and Methods
Results
Discussion
Appendix 1. Detailed description of simulators