To access the supplementary material to this article please see [Supplementary Files](http://www.jiasociety.org/index.php/jias/article/view/18580) under Article Tools online.

Introduction {#S0001}
============

Over the past decade, combined international efforts have achieved a more than 20-fold increase in access to antiretroviral (ARV) treatment for individuals infected with the human immunodeficiency virus (HIV) in resource-limited settings (RLS) \[[@CIT0001]\]. Increased treatment access has been paralleled by an increased need for HIV drug-resistance (HIVDR) testing to monitor its emergence and transmission, a major threat to treatment success \[[@CIT0002]\].

Whether using commercial or in-house HIVDR testing, laboratory participation in external quality assessment (EQA) is recommended by expert committees \[[@CIT0003]--[@CIT0007]\]. Available HIVDR EQA programmes include a centralized laboratory certification approach \[[@CIT0004]\], a "collective" approach, with no personal communication between evaluator and evaluatees \[[@CIT0008], [@CIT0009]\] and "within network" and "within country" approaches, where providers liaise with participants to improve test outcome \[[@CIT0010]--[@CIT0014]\].

TREAT Asia (Therapeutics Research, Education, and AIDS Training in Asia \[[@CIT0015]\]), a programme of amfAR -- The Foundation for AIDS Research, is a network of more than 60 clinics, hospitals and research institutions in 13 countries working with civil society to ensure safe and effective delivery of ARVs in Asia and the Pacific \[[@CIT0016], [@CIT0017]\]. It also seeks to strengthen HIV/AIDS care, prevention, treatment and management skills among healthcare professionals through education and training programmes developed by experts in the region.

In 2006, TREAT Asia was funded by the Dutch Ministry of Foreign Affairs to build surveillance and monitoring capacity for HIVDR in Asia. The resulting TREAT Asia Studies to Evaluate Resistance (TASER) \[[@CIT0018]\] are part of a collaborative effort: Linking African and Asian Societies for an Enhanced Response (LAASER) to HIV/AIDS, in partnership with the PharmAccess Foundation, International Civil Society Support and the AIDS Fonds \[[@CIT0019]\]. While TASER is conducted in Asia, PharmAccess' parallel Studies to Evaluate Resistance (PASER) conduct HIVDR studies in Africa \[[@CIT0019]\]. With the rationale to support good quality and reliable HIVDR testing, TAQAS (TREAT Asia Quality Assessment Scheme), an EQA scheme for HIVDR genotyping, was established to support LAASER-related activities in the two regions \[[@CIT0020]\]. TAQAS offered EQA of HIVDR genotyping with quantitative assessment of laboratory performance, educational feedback, follow-up of suboptimal results, trouble-shooting support and establishment of a laboratory network.

We previously described findings from the initial implementation of TAQAS within 10 laboratories over a 19-month period in 2005--2007, and demonstrated improvement or maintenance of high standards of genotyping outcomes \[[@CIT0020]\]. The objectives of this study are to evaluate TAQAS performance following its expansion to include more laboratories, while testing additional EQA panels with diverse HIV-1 subtypes; develop a novel scoring system of test performance; and investigate predictors of HIVDR genotyping proficiency. A wide spectrum of laboratory expertise was included with the aim not only to evaluate, but also to improve the full spectrum of HIVDR genotyping performance, while creating a collaborative laboratory network and providing mentorship and support to emerging laboratories in RLS.

Materials and methods {#S0002}
=====================

Participating laboratories {#S0002-S20001}
--------------------------

TAQAS participants included laboratories that provided HIVDR genotyping for clinical TASER (*n*=19 in 11 Asian countries) and PASER (*n*=3 in 2 African countries) sites ([Figure 1](#F0001){ref-type="fig"}). A Virology Quality Assessment Program (VQA)-certified US laboratory with HIVDR genotyping expertise (Stanford University) participated in an "expert" capacity as a "positive control."

![The location of TAQAS participants: China: Beijing, Shanghai, Hong Kong; India: Chennai, Pune; Japan: Tokyo (*n=*2) Nagoya; Malaysia: Kota Bharu, Kuala Lumpur; Philippines: Manila; Singapore: Singapore; Taiwan: Taipei; South Korea: Seoul; Thailand: Bangkok (*n=*3), Chiang Mai; Vietnam: Ho Chi Minh; Africa (not included in the map): Entebbe (*n=*2), Johannesburg; USA (not included in map): reference laboratory.](JIAS-16-18580-g001){#F0001}

Sample panels and testing {#S0002-S20002}
-------------------------

Over the 4.5-year study period (12/5--6/10), nine 5-sample TAQAS panels (45 samples) were distributed to and tested by participating laboratories, using their standard protocols. Panels were prepared by, shipped from and results returned to the NRL (Melbourne, Australia), an independent quality assurance provider.

Samples in seven panels were either: (i) plasma sourced from HIV-1-patients (27/35), or (ii) culture-amplified virus in HIV-negative plasma (8/35) ([Table 1](#T0001){ref-type="table"}). Most samples contained multiple drug-resistance mutations (DRMs) and most were Subtype B sourced in Australia. Two panels (IV and VII) were *pol* electropherograms, derived from ARV-treated individuals, and used to account for inter-laboratory sequence production variation in HIVDR genotyping outcome. Panels varied in the number of DRMs; number of mixtures, defined as \>1 nucleotide base at one position; and number of DRMs present as mixtures ([Table 1](#T0001){ref-type="table"}). All plasma/virus samples (*n*=35) had a viral load \>1000 copies/mL; most (31/35) \>10,000 copies/mL.

###### 

TAQAS panels[a](#TF0001){ref-type="table-fn"} tested and detection of drug-resistance mutations by participants between December 2005 and June 2010

                                      DRMs[c](#TF0003){ref-type="table-fn"} in panel samples   DRMs positions in *pol* region in panel TG                                                                                                                                                                         
  --------- ------------------- ----- -------------------------------------------------------- -------------------------------------------- --------- ------------------------------------------ -------------------------------------------------------------------- ------------------------------------------- --------------
  I         B                   73    46                                                       35                                           11 (24)   V32M 46I I47V I54VM G73S V82A I84V L90M    M41L D67N L74V F116Y Q151M M184V L210W T215Y                         K103N V106A                                 91 (80--96)
  II        B and D             48    59                                                       48                                           11 (19)   M46I I47V I54LMV G73S V82SI84V L90M        M41L D67N T69ins L74V M184V L210W T215Y K219N                        K103H V106A V108I Y181C G190A               98 (85--100)
  III       B and C             109   42                                                       23                                           19 (45)   V32IM46I G48V I50V I54VT V82A I84V L90M    M41L D67N L74I V75T M184V L210W T215Y K219E                          K103N V108I Y181C G190A M230L               88 (55--97)
  IV (e)    A, B and CRF01_AE   46    36                                                       29                                           7 (19)    M46I G48V I50V I54T V82A L90M              M41L A62V D67N K70R V75I F77L F116Y Q151M M184V L210W T215FY K219Q   K101EH K103N V106M V108I Y181C G190A        97 (89--100)
  V         B and CRF07_BC      48    53                                                       50                                           3 (6)     M46I I54V V82A I84V L90M                   M41L D67N V75M M184V L210W T215Y                                     K103N                                       98 (96--100)
  VI        B                   68    59                                                       48                                           11 (18)   M46LI I50L V82A                            M41L D67N K70R L74V M184V L210W T215Y                                K103N Y181C P225H                           97 (88--98)
  VII (e)   B                   113   83                                                       63                                           20 (24)   V32I M46I I47V G48V I50V I54T V82A L90LM   M41L A62AV D67N T69ins L74V V75T M184V L210W T215Y K219E             K101Q K103H V106A V108I Y181C G190A H221Y   90 (81--99)
  VIII      B                   172   38                                                       27                                           11 (30)   V32I M46I I54V V82A L90M                   M41L D67N L74I V75T Y115F M184V L210W T215Y K219E                    L100I K103N V106I Y181C Y188L G190A         89 (76--100)
  IX        B and CRF01_AE      119   56                                                       47                                           9 (16)    M46I I47V I54M L76V I84V L90M              M41L K65R D67N L74V V75L M184V L210W T215FY K219E                    K101EP K103NH V108I Y181C G190A H221Y       98 (91--100)

Each panel consisted of five samples

Non-B subtype plasma samples included one Subtype A and one Subtype C from Uganda, one CRF07_BC from Taiwan and three CRF01_AE from Thailand. Non-B electropherogram samples included two Subtype A from Kenya and one CRF01_AE from Thailand

listed as mutations conferring HIV drug resistance in IAS-USA list and Stanford Database.

TG, target genotype; Mixtures: nucleotide mixtures; DRMs, drug-resistance mutations; PI, protease inhibitor; NRTI, nucleoside reverse transcriptase inhibitor; NNRTI, non-NRTI; e, electropherogram sample format used in panel.

Samples were shipped biannually frozen on dry ice (plasma panels) or sent electronically (electropherogram panels) to TAQAS participants after obtaining necessary country-specific permits. Within five weeks of panel receipt, participants were required to return three result outputs: (i) FASTA nucleotide *pol* sequences; (ii) lists of DRMs; and (iii) predicted susceptibility to a standardized list of ARVs. Detailed genotyping information on testing methods was collected via an electronic "Protocol Questionnaire" ([Supplementary File 1](http://www.jiasociety.org/index.php/jias/rt/suppFiles/18580/0)).

Laboratory performance evaluation {#S0002-S20003}
---------------------------------

Sequences returned underwent quality control and phylogenetic analyses to examine within-sample clustering using Sequence Quality Analysis Tool (SQUAT) \[[@CIT0021]\]. Outlier sequences (bootstraps \<99%) that did not cluster appropriately were manually inspected for mutation motifs and mis-alignment, and if consistent, they were reported and omitted from further analyses.

HIVDR genotyping performance was evaluated against a panel-specific consensus sequence, termed the "target genotype" (TG), deduced by aligning sample-specific FASTA nucleotide sequences returned by all participants and applying an algorithm to identify the most likely consensus quasi-species detectable by DNA sequencing \[[@CIT0022]\]. The algorithm considered that it was extremely unlikely for multiple participants to incorrectly sequence the same nucleotide at any single position; therefore, nucleotide mixtures reported by two or more participants were included in the TG. DRMs were defined as mutations listed in both the Stanford Resistance Database \[[@CIT0023]\] and the International AIDS Society-USA list \[[@CIT0024]\]. When participants were unable to return a complete set of panel results due to amplification or quality issues, analyses were executed only for completed samples that satisfied the quality control analyses.

Laboratory performance was scored using a system composed of eight criteria ([Table 2](#T0002){ref-type="table"}). Performance was not evaluated by the scoring system if participants reported results from fewer than four out of five samples per panel because of failure to amplify sequences, and/or production of poor quality sequences as deemed by the participant and/or the quality control analyses indicative of potential sample mix-up or contamination. "No Score" was by definition a poor performance outcome.

###### 

Scoring system developed and applied to evaluate participants' HIVDR genotyping outcome

  ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  Criteria name                                                            Criteria detail                                                                                                                     Penalty points                                                                                                                                                                                                  Evaluation context
  ------------------------------------------------------------------------ ----------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------- -----------------------------------------------------------------------------------------
  \(i\) Participation                                                      Submission of any results                                                                                                           10 points if fail to submit any result One point per week of late submission                                                                                                                                    Suitability of testing turnaround time for patient management and/ or research purposes

  \(ii\) Nucleotide sequence submission[H](#TF0004){ref-type="table-fn"}   Submission of both PR and RT sequences                                                                                              One point for each failure to submit PR or RT sequences per sample; *NO SCORE* if sequences from \<4 of 5 samples returned                                                                                      

  \(iii\) Sequence clustering[H](#TF0004){ref-type="table-fn"}             Adequate sequence clustering with same-samples from other participants                                                              Five points per PR or RT inadequate clustering; *NO SCORE* if sequences from \>4 of 5 samples do not cluster                 Disqualified from analysis if insufficient or outlier sequence data are returned   

  \(iv\) Nucleotide sequence agreement                                     Level of agreement (%) with the complete target genotype (TG) (consensus sequence)                                                  ≥99% agreement: 0 points\                                                                                                    Measures of technical aspects of HIVDR genotyping output                           
                                                                                                                                                                                                               98 to\<99% agreement: 2 points\                                                                                                                                                                                 
                                                                                                                                                                                                               95 to\<98% agreement: 5 points\                                                                                                                                                                                 
                                                                                                                                                                                                               90 to\< 95% agreement: 10 points\                                                                                                                                                                               
                                                                                                                                                                                                               \<90% agreement: 15 points                                                                                                                                                                                      

  \(v\) Detection of DRMs                                                  Detection (%) of DRMs in the TG (consensus sequence)                                                                                ≥95% detected: 0 points\                                                                                                                                                                                        
                                                                                                                                                                                                               90 to\<95% detected: 2 points\                                                                                                                                                                                  
                                                                                                                                                                                                               85 to\<90% detected: 5 points\                                                                                                                                                                                  
                                                                                                                                                                                                               80 to\<85% detected: 10 points\                                                                                                                                                                                 
                                                                                                                                                                                                               \<80%: 15 points                                                                                                                                                                                                

  \(vi\) Reporting of DRMs                                                 DRM(s) reported compared with majority                                                                                              Five points per DRM not reported                                                                                                                                                                                

  \(vii\) Detection of mixtures[a](#TF0005){ref-type="table-fn"}           Detection (%) of NMs in the TG (consensus sequence)                                                                                 Not scored                                                                                                                                                                                                      

  \(viii\) Reporting of drug-resistance profile                            Agreed interpretation of drug resistance compared with majority when using a single interpretation system, e.g. Stanford Database   Assessed as high, moderate or low level of agreement                                                                                                                                                            Peer group comparison of drug-resistance profile reported for patient management
  ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

"Hurdle requirement", participant must return sequence data from at least four of five samples that cluster with same-samples from other participants to be scored on their performance. "No Score" was by definition a poor performance outcome.

Included to highlight importance of detection of mixtures; not scored as variation in the detection of mixtures(%) was not proportional to detection of DRMs(%).

PR, protease; RT, reverse transcriptase; DRMs, drug-resistance mutations as in IAS-USA list and Stanford Database; TG, target genotype; mixtures, nucleotide mixtures.

At the conclusion of each panel nucleotide sequence alignments, analyses and scores of all participants' were made available on a password-protected website. Participants were encouraged to review and compare results, and, if necessary, amend and improve their methods. Sequence alignments were provided for participants whose performance was not scored and they were alerted to the reason. Resubmission of results was not permitted.

To provide support and follow-up, annual workshops were conducted for technical and scientific staff directly involved with HIVDR genotyping from participating laboratories, with detailed discussions and expert presentations of laboratory methods. This also facilitated development of a regional network of operators with a range of expertise. When suboptimal performance was identified, laboratories were asked to describe initiatives to address identified problems. In two cases, expert laboratory personnel from within the TAQAS network assisted in setting up HIVDR genotyping protocols or addressed recurring problems.

Statistical model to predict quality of HIVDR genotyping outcome {#S0002-S20004}
----------------------------------------------------------------

To evaluate participants' HIVDR genotyping outcome, a binary endpoint was defined as detection of ≥95% (high-quality) vs. \<95% (low-quality) of consensus DRMs in the TG. Factors potentially associated with *panel complexity* and *laboratory performance* were examined. *Panel complexity* was defined by: (i) number of DRMs in the TG; and (ii) number of DRMs present as mixtures in the TG; (iii) sample subtype (B or non-B); and (iv) sample format (plasma or electropherogram). *Laboratory performance* was defined by: (i) length of nucleotide sequence; (ii) nucleic acid agreement with TG; (iii) detection of DRMs in the TG; (iv) detection of mixtures in the TG; and (v) elements of HIVDR genotyping protocol (i.e. laboratory experience, throughput, time pressure, in-house or outsourced sequencing, nucleotide sequence editing practices and software used). Mixed-effect logistic regression models were used to derive the endpoint using panel complexity and laboratory performance as predictors. All results, including those from incomplete result sets, were included in the models. As laboratories participated in varying panels, and incomplete result sets were included, random-effect models were used to take into account variation both within and between laboratories. The final model included predictors that were significant at 0.10 level (two-sided). Data management and statistical analyses were performed using Stata (StataCorp, STATA 10.1 for Windows, College Station, TX, USA).

Results {#S0003}
=======

Participating laboratories {#S0003-S20001}
--------------------------

Nine TAQAS panels were delivered to 19 laboratories in the Asia-Pacific, three in Africa and one in the United States ([Figure 1](#F0001){ref-type="fig"}). Eight laboratories reported results for all nine panels. Laboratories were recruited over time and on average reported results for six panels ([Table 3](#T0003){ref-type="table"}). Inconsistent participation was due to cessation of HIVDR genotyping services (Lab 11), import restrictions (Lab 13), resource constraints such as reagents, staff or laboratory access shortfall (Labs 14 and 20), or an inability to process electropherograms due to software incompatibility (Labs 3, 7 and 13).

###### 

Panels tested by TAQAS participants over a 4.5-year period

                                        TAQAS panels                                      
  ------------------------------------- -------------- ---- ---- ---- ---- ---- ---- ---- ----
  1[a](#TF0006){ref-type="table-fn"}                                                      
  2                                                                                  DR   
  3                                                              NP                       
  4                                                              DR                       
  5                                                                                       
  6                                                                                       
  7                                     NP                  DR   DR                       DR
  8                                                                                       
  9                                                              DR                       
  10                                                                                      
  11                                                                       NP   NP   NP   NP
  12                                    NP                       NP   NP        NP        
  13                                    NP             NP        NP                       
  14                                    NP             NP   NP             NP        NP   
  15                                    NP             NP   NP                            
  16                                    NP             NP   NP                            
  17                                    NP             NP   NP   NP   NP                  
  18                                    NP             NP   NP                            
  19                                    NP             NP   NP   NP   NP                  
  20                                    NP             NP   NP   NP   NP   NP        NP   
  21                                    NP             NP   NP   NP   NP             DR   
  22                                    NP             NP   NP   NP   NP   NP             
  23[c](#TF0008){ref-type="table-fn"}   NP             NP   NP   NP   NP   NP   NP   NP   DR

Stanford University Laboratory

electropherogram panels

Lab 23 participated in only one panel; it was included in this presentation to show progressive laboratory enrolment.

NP, laboratory did not participated; DR, data removed from analysis because laboratory returned sequence from \<4 of 5 samples or reported outlier sequence for \>4 of 5 samples.

Protocol questionnaire {#S0004}
======================

The Questionnaire, which was completed between the fifth and sixth panels by all but one laboratory (Lab 11 ceased HIVDR testing after Panel VI), demonstrated a wide variability in HIVDR genotyping experience. Laboratories were conducting HIVDR testing for a median of six years (IQR: 5.5 years; range: \<1 to 14 years); the median testing throughput was 348 tests per annum (IQR: 625; range: 21 to \>4000); and the median per sample turnaround time was 14 days (IQR: 14.5 days; range: 2--30 days). Fourteen laboratories required staff qualifications of a bachelor degree or higher and three required training in molecular biological technique (five did not respond). Time pressure to complete HIVDR testing was perceived by 9 out of 22 laboratories.

The majority of laboratories (18/22) used locally assembled protocols, with wide variations in primers, sequencing probes and input sample volume (data not shown). These aspects of the protocols were not compared. Four laboratories used commercial kits (TruGene^®^ one laboratory; ViroSeq^®^ three laboratories). The impact on test outcome of participants' sequencing practices was assessed. Only a few laboratories (6/22) outsourced sequencing. Most laboratories (20/22) used an automatic base calling software and all reported manual checking and editing of automated base calls. In most laboratories (16/22), more than one person was involved in sequence editing. The peak height to call mixed bases was set at 20--30% by 19 laboratories. It was policy in most laboratories (15/22) to review sequence data at sites associated with ARV resistance. Twelve laboratories reported controlling for contamination using software such as Clustal ([www.clustal.org](http://www.clustal.org)) or Mega ([www.megasoftware.net](http://www.megasoftware.net)).

Of 18 laboratories that used in-house protocols, 15 used the Stanford Database for resistance interpretation and the remaining three used the Stanford Database in conjunction with IAS-USA \[[@CIT0024]\] or ANRS systems \[[@CIT0007]\]. The three laboratories that used Viroseq consulted the Stanford Database in addition to Viroseq guidelines. The laboratory that used TruGene relied solely on the manufacturer\'s guidelines.

Laboratory performance {#S0004-S20001}
----------------------

A total of 144 data sets were returned by the 23 participating laboratories; most within the specified turnaround time of five weeks; 10 participants returned results up to five weeks past the turnaround time for one (*n*=9) or two (*n*=1) panels. Late submission of results was recorded and these data sets were included in analyses. One hundred and thirty six data sets (107 plasma and 29 electropherograms) were suitable for assessment. Eight data sets were removed from analyses because they were derived from less than four out of five samples per panel due to sequence amplification failures (two datasets); production of sequence deemed of poor quality (three data sets); or did not satisfy quality control analyses (three data sets) ([Table 3](#T0003){ref-type="table"}). On follow-up, the two laboratories with outlier sequences achieved a non-outlier result by either re-sequencing and changing training protocols; or by preparing new batches of primers and reagents. Revised test outcomes were not re-scored.

The median detection of DRMs in the TG in the seven plasma panels ranged between 88% and 98% ([Table 1](#T0001){ref-type="table"}). In most reported data sets (102/107), the number of DRMs detected was above the median minus two standard deviations (median--2SD) in each panel. The lowest levels of detection of DRMs were in Panels III (88%), VIII (89%) and I (91%), which had the highest proportion of DRMs presented as mixtures (45%, 30% and 24%, respectively). Although overall detection of DRMs was high throughout the study period, some performance improvement was observed over time. Three laboratories detected \<80% of DRMs in early panels but subsequently demonstrated improved performance (Labs 7, 13 and 22: [Figure 2](#F0002){ref-type="fig"}). The observed considerable intra- and inter-laboratory variation in the detection of mixtures did not correlate with the percentage of DRMs detected. However, the two laboratories that consistently reported low levels of mixtures or none (Labs 2 and 5; [Figure 3](#F0002){ref-type="fig"}) both detected less than the median--2 SD deviations of TG DRMs in two panels, and both had a tendency to underreport the DRMs reported by the majority of participants (data not shown).

![Detection of drug-resistance mutations according to the target genotype (TG) in nine TAQAS panels (one graph per laboratory).](JIAS-16-18580-g002){#F0002}

![Detection of nucleotide mixtures according to the target genotype (TG) in nine TAQAS panels (one graph per laboratory).](JIAS-16-18580-g003){#F0003}

The median detection of DRMs in the electropherogram panels was 97% and 90%. Again, detection of mixtures varied (Panel IV: 0--89% with 11 participants; Panel VII: 1--79% with 20 participants). Labs 2 and 5 detected low levels of mixtures in the electropherogram panels ([Figure 3](#F0002){ref-type="fig"}). In addition Lab 5\'s detection of DRMs in Panel 4 was below the mean--2SD. When data from Labs 2 and 5 were removed, the range in the detection of mixtures decreased: Panel IV: 29--89% with 9 participants; Panel VII: 18--79% with 18 participants. TREAT Asia and TAQAS have worked with Labs 2 and 5 addressing software training in detection of mixtures. Lab 2\'s detection of mixtures increased (from ≤1 to 39% in Panel IX; [Figure 3](#F0002){ref-type="fig"})

Score as a measurement of performance {#S0004-S20002}
-------------------------------------

Data sets from all participants were quantitatively evaluated using the scoring system ([Table 2](#T0002){ref-type="table"}). High-quality test outcome, defined as detection of ≥95% of the DRMs in the TG, was associated with scoring criteria related to technical aspects of HIVDR genotyping, including adequate phylogenetic clustering, sequence alignment and level of agreement with the TG, and detection of TG DRMs ([Table 2](#T0002){ref-type="table"}, Criteria iii, iv and v; *p*\<0.05).

Predictors of HIVDR genotyping performance {#S0004-S20003}
------------------------------------------

Factors associated with *laboratory performance* that had an impact on HIVDR genotyping outcome included mixture detection (OR: 19.8; *p*\<0.001) and level of agreement with the TG (OR: 46.5 for 98-99% agreement, *p*=0.002; and OR: 129.4 for ≥99% agreement, *p*\<0.001) ([Table 4](#T0004){ref-type="table"}). Neither length of sequence nor between-laboratory differences in HIVDR genotyping sequencing protocols were associated with performance. Factors associated with *panel complexity* that had an impact on HIVDR genotyping outcome included number of TG DRMs (OR 31.7; *p*\<0.001), and number of DRMs present as mixtures (OR: 19.1 for 11-15 DRMs as mixtures, *p*=0.001; and OR: 240 for ≤10 DRMs as mixtures, *p*\<0.001). Subtype (B or non-B) and sample format (plasma or electropherogram) were not associated with genotyping performance.

###### 

Predictors of high-quality HIV drug-resistance genotyping outcome

  Covariate                                                 Odds ratio (95% CI)       *p*
  --------------------------------------------------------- ------------------------- ---------
  Laboratory performance[a](#TF0009){ref-type="table-fn"}                             
   Mixtures detected (%)                                                              
   ≤50[b](#TF0010){ref-type="table-fn"}                     1.00                      
   \>50                                                     19.78 (4.16, 94.09)       \<0.001
   Agreement with nucleotide sequence                                                 
   \<98%[b](#TF0010){ref-type="table-fn"}                   1.00                      
   98 to\<99%                                               46.54 (3.87, 559.51)      0.002
   ≥99%                                                     129.41 (10.32, 1622.32)   \<0.001
  Panel complexity                                                                    
   Number of DRMs as mutants                                                          
   ≤40[b](#TF0010){ref-type="table-fn"}                     1.00                      
   \>40                                                     31.66 (6.92, 144.81)      \<0.001
  Number of DRMs as mixtures                                                          
   \>15[b](#TF0010){ref-type="table-fn"}                    1.00                      
   11--15                                                   19.10 (3.43, 106.39)      0.001
   ≤10                                                      239.95 (27.89, 2064.56)   \<0.001

Compared with target genotype (TG)

reference category.

Discussion {#S0005}
==========

We report extended results from TAQAS, an EQA programme for HIVDR genotyping by a group of laboratories in Asia-Pacific and Africa \[[@CIT0020]\]. Twenty-two laboratories (not including the certified US laboratory) from 13 countries demonstrated proficient HIVDR genotyping of 45 HIV-1 multi-subtype samples in nine panels, as evidenced by a low level of amplification failure; minimal sample cross-contamination; high levels of DRMs detection and sequence homology to consensus sequences; and compliance with a test turnaround time indicating provision of results in a clinically relevant timeframe. Intra-and inter-laboratory variation in detection of mixtures was observed, and in some laboratories was associated with sub-optimal detection and reporting of DRMs. HIVDR genotyping execution was associated with panel complexity factors including numbers of DRMs and DRMs occurring as mixtures, and with laboratory performance factors including detection of mixtures and agreement with TG, but not with differences in laboratories' use of commercial vs. in-house tests or sequencing protocols. A new scoring system showed that quality of test outcome was related to technical proficiency in HIVDR genotyping. In contrast to other EQA programmes \[[@CIT0008], [@CIT0022], [@CIT0025]--[@CIT0028]\], TAQAS participants were supported to address testing deficiencies, and their performance improved in subsequent panels. The programme\'s feedback in response to suboptimal performance and its educationally oriented approach may have contributed to the high quality of the testing outcomes.

Similar to other EQA programmes, TAQAS participants varied in testing experience and access to technical resources \[[@CIT0025], [@CIT0029]\]. In contrast to other studies where laboratories predominantly used commercial kits and consensus protocols and no difference in test outcome was found, the majority of TAQAS participants used a variety of in-house technologies, as well as several sequence editing software programmes \[[@CIT0005], [@CIT0025], [@CIT0030], [@CIT0031]\]. However, most participants had similar sequence editing practices and used the Stanford Database for interpretation. Based on these data, low-cost in-house assembled assays can successfully be used for HIVDR testing with the support of EQA programmes.

EQA programmes should aim to assess the HIVDR laboratory testing process using clinically relevant sample types. The predominant use of plasma samples in TAQAS enabled assessment of detection of viral mixtures, important in DRM detection \[[@CIT0006], [@CIT0008], [@CIT0020], [@CIT0032]\], and useful in the assessment of inter-laboratory testing variation \[[@CIT0033]\]. The use of a sample type with inherent variability like plasma, in contrast to clones or plasmids, mandated the use of a TG rather than a sequence derived by a reference laboratory \[[@CIT0010], [@CIT0025], [@CIT0030]\]. While inter-laboratory comparison is complicated by inclusion of samples with mixtures in EQA panels, the laboratory\'s ability to detect mixtures is an important measure of the quality of the genotyping output and every effort should be made to monitor and support this in an EQA programme \[[@CIT0005]\].

Subtle differences in EQA programmes that reported higher levels of DRM detection than reported by TAQAS participants are noted \[[@CIT0025], [@CIT0027], [@CIT0029]--[@CIT0031]\]. Some programmes report on the distribution of a single panel or low-sample-number panels, and/or use virus derived from viral culture supernatant, cloned or extracted material as the sample format. Such programmes could expect higher detection levels of DRMs compared to those seen with clinical samples, due to differences in sample variability. Indeed, comparable detection of DRMs to that seen in TAQAS was reported in a four-plasma-sample distribution to 20 laboratories \[[@CIT0028]\]. Similarly, panels incorporating non-subtype B samples, as in TAQAS, can affect sequence variability and TG concordance, due to inter-subtype genetic differences. Such samples should be included in EQA panels, particularly for laboratories in settings where non-B subtypes predominate.

Panel complexity, detection of mixtures and level of sequence agreement were found to predict HIVDR genotyping quality, confirming previous reports \[[@CIT0005], [@CIT0020], [@CIT0025], [@CIT0028], [@CIT0030], [@CIT0034], [@CIT0035]\].

The electropherogram samples used in two TAQAS panels showed that variation in detection of mixtures can occur because of inter-laboratory differences post-sequence production, rather than genotyping extraction, amplification and sequencing protocols. Sequence editing has been suggested to contribute to disparity in HIVDR genotyping outcomes particularly with respect to the mixtures detection \[[@CIT0036]\]. However, while there was variation in sequence editing software used, most participants adhered to common, high standards to edit raw sequence data. This may account for the lack of association between the quality of the test outcome and sequence editing practices demonstrated by this group. Sequences with a high number of mixtures should be incorporated into EQA panels, and laboratories should be encouraged to develop proficiency in mixture calling, which is directly related to identification of DRM.

Objective assessment of EQA performance outcomes enables intra- and inter-laboratory and between-panel comparison, and potentially comparison between EQA programmes \[[@CIT0005], [@CIT0025], [@CIT0031], [@CIT0037]\]. The TAQAS scoring system described here extends systems previously reported, by adding a measure of clinical utility; a hurdle requirement of the production of good quality, sample-specific sequence thereby emphasizing the importance of quality control measures as performance indicators; detection and reporting of all consensus DRMs and DRMs presented as mixtures; and the interpretation of ARV resistance as per the peer group majority \[[@CIT0005], [@CIT0025], [@CIT0026], [@CIT0031], [@CIT0037]\]. Scores on technical aspects of testing were associated with the quality of test outcome. Scoring multiple test components improved the value of the EQA exercise for participants by flagging possible causes of suboptimal performance. Objective measurement of test outcomes enables fair, on-going assessment of participants' EQA outcomes that may be required for auditing, by funding bodies and for participation in multi-centre clinical trials.

Some previously reported limitations of TAQAS \[[@CIT0020]\] have been addressed here. Information about participants' testing procedures was documented, the number of participants more than doubled and criteria against which HIVDR genotyping outcomes were assessed were defined. Existing limitations include lack of ability to compare genotyping technologies among participants, as most used in-house assays \[[@CIT0025]\]. These are realistic circumstances in cost-constrained settings. The inclusion of non-B and recombinant viruses relevant to Asia has increased, though not sufficiently to assess the impact of different subtypes on test outcomes \[[@CIT0030]\]. The importance of including such samples is acknowledged \[[@CIT0005]\] although other EQA providers have provided predominantly subtype B samples \[[@CIT0030], [@CIT0031], [@CIT0035]\].

TAQAS was established to build capacity for and assure quality of HIVDR genotyping to support clinical care and research in Asia and Africa. Participation by 22 laboratories over an extended timeframe confirmed their HIVDR testing proficiency. Several reasons are proffered as to the high-quality test outcomes of this complicated test: diligent and conscientious attitude of participants, provision of complex clinical samples, comprehensive and comparative analysis of results, follow-up initiatives after suboptimal performance, and support and information dissemination by TREAT Asia and NRL. TAQAS facilitated successful application of a quantitative measure of laboratory performance of HIVDR genotyping and identification of predictors of test quality. Both tools can improve utility of future EQA programmes. The importance of continuous EQA participation to maintain and improve HIVDR genotyping outcome has been validated \[[@CIT0008], [@CIT0025], [@CIT0030]\]. Recent reports provide novel methods to standardize the interpretation of electropherograms for HIVDR testing \[[@CIT0038], [@CIT0039]\]. Though promising, it will take time until such methods are incorporated into laboratories' protocols. As methods are developed and modified, their incorporation into quality assessment programmes will be essential. As demonstrated by TAQAS, quality assessment programmes not only assess proficiency but can also be harnessed to establish, expand and improve testing, and be used as a vehicle for educational initiatives and the creation of collaborative and educational testing laboratory networks.

TAQAS is an initiative of TREAT Asia, a programme of amfAR, The Foundation for AIDS Research, and is supported in part by amfAR with additional support from the Dutch Ministry of Foreign Affairs through a partnership with Stichting Aids Fonds. The Kirby Institute for infection and immunity in society is funded by the Australian Government Department of Health and Ageing, and is affiliated with the Faculty of Medicine of The University of New South Wales. The content of this publication is, however, solely the responsibility of the authors and does not necessarily represent the official views of any of the institutions mentioned above.

The TAQAS Laboratory Network: L Kang, Shanghai Municipal Center for Disease Control and Prevention, Shanghai, China; WC Yam and J Chen, HIV Molecular Biology Laboratory, Department of Microbiology, Queen Mary Hospital, Hong Kong; S Oka and X Bi, AIDS Clinical Center, International Medical Center of Japan, Tokyo, Japan; W Sugiura, National Institute of Infectious Diseases, Tokyo, Japan; N Kumarasamy, V Madhavan and S Saravanan, Infectious Diseases Laboratory, YRG Centre for AIDS Research and Education, Chennai, India; KP Ng and A Kamarulzaman, HIV Research Laboratory, Centre of Excellence for Research in AIDS (CERiA), University of Malaya, Kuala Lumpur, Malaysia; W Stevens and C Wallis, National Health Laboratory Services (NHLS) -- University of Witwatersrand, Johannesburg, South Africa; YMA Chen and YJ Chen, AIDS Prevention and Research Center, National Yang-Ming University, Taipei, Taiwan; R Sutthent, Department of Microbiology, Faculty of Medicine, Siriraj Hospital, Bangkok, Thailand; K Ruxrungtham and S Sirivichayakul, Division of Allergy and Clinical Immunology, Department of Medicine, Faculty of Medicine, Chulalongkorn University, Bangkok, Thailand; W Chantratita, W Piroj and C Watitpun, Virology and Molecular Microbiology Unit, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand; D Katzenstein and M Balamane, Center for AIDS Research, Division of Infectious Diseases and Geographic Medicine, Stanford University, CA, USA; TTX Lien and TC Thanh, HIV/AIDS laboratory, Pasteur Institute in Hochiminh City, Ho Chi Minh City, Vietnam; F Kyeyune and L Bagenda, CFAR Molecular Biology Laboratory, Joint Clinical Research Centre (JCRC), Kampala, Uganda; J. Praparattanapan and T Sirisanthana, HIV-1 Genotyping Drug Resistance Laboratory, Faculty of Medicine and Research Institute for Health Sciences, Chiang Mai University, Chiang Mai, Thailand; PN Mukhopadhaya and A Acharya Medical Genetics Division, geneOmbio Technologies, Pune, India; J Hattori and W Sugiura, Department of Infection and Immunology, Clinical Research Center, National Nagoya Medical Center, Nagoya, Japan; E Segubre-Mercado and R Ditangco, Molecular Biology Laboratory, Research Institute for Tropical Medicine, Manila, Philippines; H Zhou and H Zeng, Institute of Infectious Diseases, Beijing Ditan Hospital, Capital Medical University, Beijing, China; KK Chew and OT Ng, Infectious Disease Research Laboratory, Communicable Disease Centre, Tan Tock Seng Hospital, Singapore; JY Choi, AIDS Research Institute, Yonsei University College of Medicine, Seoul, South Korea; N Ndembi and F Lyagoba, MRC/UVRI Uganda Research Unit on AIDS, Entebbe, Uganda, ZZ Deris and S Mohamad Medical Microbiology and Parasitology, Health Campus, School of Medical Sciences, Universiti Sains Malaysia, Kota Bharu, Malaysia.

Competing interests {#S0006}
===================

The authors declare that they have no competing interests.

Authors\' contributions {#S0007}
=======================

Sally Land, Julian Zhou, Philip Cunningham, Annette H Sohn, David Katzenstein and Rami Kantor conceived and designed the study protocol. Philip Cunningham, David Katzenstein, David Sayer and Rami Kantor provided expert training to participants. Thida Singtoroj coordinated panel and workshop logistics. David Sayer, Marita Mann and Rami Kantor undertook phylogenetic and sequence analyses. Julian Zhou realized the statistical analysis. Sally Land conducted the data analysis, wrote the first draft of the manuscript and worked with Rami Kantor to finalize the writing. Sally Land, Rami Kantor, Annette H Sohn, and David Katzenstein critically revised the manuscript. All authors read and approved the final manuscript.
