mydate []

# Quantification of Captopril using Ultra High Performance Liquid
Chromatography

Glenn Zammit<EMAIL_ADDRESS>Malta College of Arts,
Science and Technology, Poala, Malta  Ralph Cassar
<EMAIL_ADDRESS>Malta College of Arts, Science and Technology,
Poala, Malta  Mark Pace<EMAIL_ADDRESS>Malta College of Arts,
Science and Technology, Poala, Malta

()

###### Abstract

We develop an efficient, sensitive, accurate, cost effective, and greener
liquid chromatography method for the determination of a quantification method
concerning Captopril in pharmaceutical dosage forms by using ultra high
performance liquid chromatography (UHPLC) equipment. Satisfactory resolution
was able to be achieved using a Water Aquity UHPLC BEH C18 1.7 $\mu$m $\times$
2.1 mm $\times$ 50 mm column and an isocratic mobile phase composition of
Methanol, Milli-Q water and Trifluoroacetic acid in the ratio of 55:45:0.05
v/v/v respectively at a flow rate of 0.1mL/min and a wavelength detection of
220nm. The retention time for Captopril was found to be 1.744 minutes while
for Captopril disulfide it was found to be 2.657 minutes. The method was
validated based on the ICH and USP guidelines.

###### pacs:

06.60.-c, 06.60.Mr, 06.90.+v, 07.90.+c

##  I I. Introduction

Captopril (((2S)-1-[(2S)-2-Methyl-3-sulfanylpropanoyl] pyrrolidine-2-
carboxylic acid), molecular formula C9H15NO3S is a popular well-known drug
which subjects in the class of drugs called angiotensin-converting enzyme
(ACE) inhibitors vidt1982captopril . Captopril is the first orally active
inhibitor of ACE inhibitor in which the enzyme is responsible for the
conversion of inactive angiotensin I to the potent pressor peptide angiotensin
II vidt1982captopril . ACE inhibitors are mainly used for the treatment of
high blood pressure but they also contain some other medical properties such
as vasculo-protective and antithrombotic activities which plays a favourable
role in terms of cardiovascular morbidity bojarska2015captopril . Captopril,
has established a position in the medical treatment for hypertension and
congestive heart failure amar2021captopril ; sigit2020comparison . Amongst the
hypertension drugs, Captopril is a preferred drug amongst doctors, it is
prescribed to patients who are chronically ill and require a long-term
treatment, due to its therapeutic benefits and because of its effectiveness,
low price and low toxicity vidt1982captopril ; brogden1988captopril ;
captopril1988comparative ; amar2021captopril ; sigit2020comparison .

Captopril disulfide (1,1′-[Disulfanediylbis[(2S)-2-methyl-1-oxopropane3,1-
diyl]] bis[(2S)-pyrrolidine-2-carboxylic] acid), molecular formula
C18H28N2O6S2 is the major degradation impurity of Captopril
vasconcelos2021structural . Captopril disulfide is also referred to as
impurity A vasconcelos2021structural . The Captopril molecule starts to
degrade after eight hours when in solution and also when being exposed to air
by means of oxidation timmins1982factors ; hillaert1999determination . The
Captopril molecule was analytically tested for stability under different
stress conditions carje2019hplc . The stability testing was performed under
various pH and temperature conditions, in the presence of strong oxidizing
agents and under both natural light and ultra-violet (UV) light exposure, this
was done in order to distinguish it from other types of impurities that may be
present in the final product. Although approved in 1981 in the USA, Captopril
is still widely used today national2018livertox .

Figure 1: The degradation rate of Captopril (%) under different forced
degradation conditions carje2019hplc

The aim of this research is to develop a cost effective, efficient and a
greener liquid chromatography method for the determination of Captopril in
pharmaceutical dosage forms using shorter runs and using less mobile phase
than conventional liquid chromatography methods.

###  I.1 Greener instrumental analytical chemistry

The use of UHPLC methods in pharmaceutical analysis while reducing the time
for analysis of bathces for relese to patients and customers, decreases cost
of testing. Much less solvents are used. In fact this study shows that a
significant reduction of 75% of mobile phase is achieved compared to the USP
method for Captopril USP . This means that the UHPLC method proposed is
greener. UHPLC methods reduce energy consumption, reduce the use of solvents,
and make waste management more manageable in terms of costs and environmental
impacts cielecka2013uhplc .

The UHPLC method validated in this paper satisfies some of the principles of
green analytical chemistry as described in galuszka201312 . In particular
using UHPLC rather than a conventional HPLC method generates much less waste,
reduces analysis time and the use of energy. The number of samples to test for
assay in the pharmaceutical industry is determined according to guidelines
such as the ICH guidelines guideline2005validation  and good manufacturing
practices (GMP) nally2016good .

##  II II. Materials and Methods

###  II.1 Materials and reagents

USP grade CPT, USP grade CPT-DIS, captopril tablets, and necessary reagents.

###  II.2 Apparatus

Waters Acquity UHPLC and Aquity UHPLC BEH C18 1.7$\mu$m $\times$ 2.1mm
$\times$ 50mm column. ACQUITY H-CLASS UHPLC.

###  II.3 Chromatographic conditions

Both the sample cooler temperature and the column oven temperature were set at
25∘C. The compounds were separated isocratically with a mobile phase
composition of 550mL methanol (Carlo Erba. Gold Ultragradient grade), 450mL
milli-Q water and 0.50mL trifluoroacetic acid (Fisher, 99% HPLC Grade) having
a flow rate of 0.1mL/min with an injection volume of 0.8$\mu$L. The wavelength
detection for was set at 220nm.

###  II.4 Preparation of stock solutions for system suitability criteria

Captopril standard solution was prepared in duplicate, labelled as ‘standard
solution 1’ and ‘standard solution 2’, by dissolving 10mg of Captopril
reference standard in a 50mL volumetric flask. Dissolved and diluted up to
volume using mobile phase, filtered through a 0.45$\mu$m Millipore Millex-HV
Hydrophilic PVDF Filter discarding the first 6mL of filtrate. This solution
contains a concentration of 0.2mg/mL of Captopril API, prepared fresh on the
day of use.

Captopril-disulfide solution was prepared by dissolving 25mg of CPT-Dis into a
50mL volumetric flask. Dissolved and diluted up to volume using mobile phase.
This solution contains a concentration of 0.5mg/mL of Captopril-disulfide.
Prepared fresh on the day of use.

Resolution solution was prepared by dissolving 10mg of Captopril reference
standard into a 50mL volumetric flask. 5mL of Captopril disulfide solution
(0.5mg/mL) was added together with sufficient mobile phase. Dissolved and
diluted up to volume using mobile phase, filtered through a 0.45$\mu$m
Millipore Millex-HV Hydrophilic PVDF Filter discarding the first 6mL of
filtrate. This solution contains a concentration of 0.2mg/mL of captopril API
and 0.05mg/mL captopril disulfide, prepared fresh on the day of use.

###  II.5 Injection procedure

The UHPLC column was left to equilibrate with mobile phase, and blank runs
were initially injected. These were followed by six repeated injections of
Captopril ‘standard solution 1’ and a duplicate injection of ‘standard
solution 2’. The resolution solution was injected afterwards. The six
injections were used to determine the %RSD and to calculate the % recovery
with the average area of the duplicate injection of ‘standard solution 2’.
Resolution between Captopril and Captopril disulfide was determined and the
tailing factor was recorded. Test solutions were bracketed with a duplicate
injection of Captopril ‘standard solution 1’ and %RSD was calculated and
compared with the six repeated injection of Captopril ‘standard solution 1’
for system suitability.

###  II.6 Sample preparation

The contents of 20 Captopril 100mg tablets were crushed and homogenized using
a pestle and mortar. 96 mg of the Captopril crushed tablet sample were
transferred to a 100mL volumetric flask. About 40mL of diluent were added,
followed by sonication for 15 minutes. Dilution up to volume with diluent and
filtration through a 0.45$\mu$m Millipore Millex-HV Hydrophilic PVDF Filter
followed, discarding the first 6mL of filtrate. The final test solution had a
concentration of 0.2mg/mL of Captopril API.

Figure 2: Chromatogram of USP Captopril standard solution using UHPLC Figure
3: Chromatogram of Captopril in the test solution using UHPLC Figure 4:
Chromatogram of a well resolved Resolution solution showing Captopril and
Captopril disulfide using UHPLC

##  III III. Results and Discussion

During this study trials were conducted in order to find the optimum
chromatographic conditions. The objective of this study was to develop a
chromatographic method, which is more environmentally friendly compared to an
HPLC method, cost effective, yet following all regulations and achieving
results according to the standards set out in ICH and USP guidelines. The
chromatographic method was set to achieve a peak tailing factor between 0.8
and 1.2, a retention time between 1 to 2 minutes, along with a good resolution
of $>$ 1.5. A relative standard deviation of $\leq$ 2.0% and a similarity
factor between ‘standard 1’ and ‘standard 2’ injections between 98.0 and
102.0%. The peaks for Captopril molecule and its dimer, Captopril disulfide
were well defined, resolved and free from tailing. The elution order was
Captopril at 1.744 minutes and Captopril disulfide at 2.657 minutes for the
UHPLC method developed, whilst for the USP HPLC method the elution order is
Captopril at 5.461 minutes and Captopril disulfide at 10.809 minutes.

System suitability parameters
---
Parameter |  Limit |  Result
Resolution of CPT with CPT-DIS |  $>$1.5 |  4.3
%RSD on 6 injections of STD 1 |  $\leq$ 2.0% |  0.5%
Tailing factor |  0.8 - 1.2 |  0.9
Bracketing check |  98.0 - 102.0% |  99.5 - 99.8%
Similarity factor STD 1 and STD 2 |  98.0 - 102.0% |  100.9%
Table 1: System suitability parameters

Running a single batch for analyses using the HPLC and UHPLC methods consist
of 17 runs which includes; 2 blank injections at the beginning, 6 injections
of ‘standard solution 1’, 2 injections of ‘standard solution 2’ followed by a
single resolution injection. Then another 2 ‘standard solution 1’ injections,
batch sample in duplicate and another 2 ‘standard solution 1’ injections to
close off the bracketing of standards.

HPLC method: Running the above sample set, a mobile phase usage of 340mL for
the analysis of one batch.

Developed UHPLC method: Running the same sample set using the UHPLC method
parameters, a mobile phase usage of 8.5mL for the analysis of one batch.

On another note, the Captopril USP monograph declares that Captopril solutions
have a stability of 8 hours, meaning that when using the conventional USP HPLC
method only a total of 24 injections i.e., a maximum of 4 batch runs can be
conducted when excluding the 2 blank injections at the beginning of the run.
Whilst with the newly developed UHPLC method it is possible to run a total of
96 injections i.e., a maximum of 31 batches in a single 8-hour sample set,
excluding the 2 blank injections at the beginning of the run. This clearly
shows that the new greener UHPLC method is far more efficient and requires a
much less solvent to carry out the quantification analysis.

Comparing HPLC vs UHPLC method when conducting one batch of quantification
analysis
---
Parameter |  HPLC method |  UHPLC method |  Results in %
Run time (minutes) |  20 |  5 |  UHPLC is 75% more efficient when compared to HPLC method
Mobile phase used (mL) |  340 |  8.5 |  97.5% less mobile phase was used for the UHPLC analysis

Table 2: HPLC and UHPLC methods comparison

Batch runs that can be conducted within 8 hours for stability tests
---
HPLC method |  UHPLC method |  Percentage
4 |  31 |  775% more batches for the UHPLC method

Table 3: Batch run times for HPLC and UHPLC methods

##  IV IV. Method Validation Results

###  IV.1 Specificity

The specificity tests, for any interference found in the excipients (placebo),
mobile phase, blank, and degradation impurity with the main analyte Captopril.
As chromatograms were overlayed together the test is said to comply because
there was no interference from excipients and degradant impurity Captopril
disulfide with the main analyte of interest, Captopril, retention time 1.744
minutes.

Figure 5: An overlay chromatogram showing the specificity test

###  IV.2 Linearity and range

The linearity was determined by injecting a series of 5 standard solutions
whose concentrations were at 0.05mg/mL, 0.10mg/mL, 0.15mg/mL, 0.2mg/mL (Target
concentration, 100%), and 0.25mg/mL. As shown in Fig. (6) a plot of area of
response (UNITS) vs linearity concentration levels in mg/mL was plotted. The
correlation coefficient (R), range and the residual sum of squares graph were
determined. All values comply with the ICH and USP guidelines. The values of
the correlation coefficient were close to unity which indicates a good
linearity and shows that the developed UHPLC method has a high sensitivity.

Figure 6: Graph of Area [$\mu V*$ sec $\times$ 106] against Concentration
[mg/mL] Figure 7: Residuals Scattered Plot

###  IV.3 Summary of results

Parameter | ICH Guidelines Limit | Result |  |
---|---|---|---|---
Correlation coefficient (R) | >0.995 | 0.99995 |  |
The residual sum of squares should be randomly distributed and show no bias | Complies, distributed and show no bias |  |
Residual Sum of Squares (RSS) | 61563649.90 |  |
Range (Upper and lower intervals) | 25 to 125% |  |
Table 4: Results gained compared to ICH guidelines

###  IV.4 Accuracy

The accuracy of a method was determined by performing a recovery experiment.
Concentration levels and preparations were done according to the ICH and USP
guidelines. For 70% and 130% concentration levels, 3 preparations were done,
and for the 100% concentration levels 6 preparations were prepared and
injected. The percentage recovery i.e. the actual concentration divided by the
theoretical concentration multiplied by a 100 was calculated. The percent
recovery of each individual should be $>$ 98.0%. The %RSD was determined for
each concentration level where the limit is $<$ 2.0%.

None of the individual samples were less than 98.0% and the %RSD was found to
be less than 2.0% for all concentration levels. This indicates a good
repeatability and reliability of the proposed UHPLC method.

Accuracy Concentration Levels | %Recovery | %Mean | %RSD
---|---|---|---
Accuracy 70% S1 | 101.11 | 100.29 | 1.1
Accuracy 70% S2 | 99.08
Accuracy 70% S3 | 100.69
Accuracy 100% S1 | 100.35 | 99.30 | 0.8
Accuracy 100% S2 | 98.42
Accuracy 100% S3 | 99.18
Accuracy 100% S4 | 99.35
Accuracy 100% S5 | 98.36
Accuracy 100% S6 | 100.13
Accuracy 130% S1 | 101.36 | 99.50 | 1.7
Accuracy 130% S2 | 99.13
Accuracy 130% S3 | 98.02
Table 5: Accuracy concentration levels

###  IV.5 Precision and intermediate precision

The precision testing was performed by injecting six determinations of batch
sample at a concentration level of 100% (0.2mg/mL). It was imperative that the
limits for the precision for individual values fall between 90.0 to 110.0% of
the labelled amount and the %RSD on the six injections should be $<$ 2.0%.

The intermediate precision tests were conducted by a different analyst, on
different equipment and on a different day.

These results were gathered from 6 injections of precision and 6 of
intermediate combined together. The limits for intermediate precision is that
of $<$ 2.0%.

Precision | %Assay | %RSD on 6 samples | %RSD on 12 samples
---|---|---|---
Precision S1 | 96.91 | 0.7 | 0.8
Precision S2 | 96.96
Precision S3 | 97.98
Precision S4 | 98.56
Precision S5 | 97.36
Precision S6 | 97.02
Intermediate Precision S1 | 98.61 | 0.7
Intermediate Precision S2 | 97.86
Intermediate Precision S3 | 99.16
Intermediate Precision S4 | 98.24
Intermediate Precision S5 | 98.33
Intermediate Precision S6 | 97.27
Table 6: Precision of results

##  V V. Conclusion

In this study we have developed a new method for quantifying captopril using
UHPLC. The study shows that this is may be a viable way forward for the
pharmaceutical industry, given the increasing emphasis on environmental
responsibility and efficient and cost-effective use of resources. This
validation study of the new method was carried out according the ICH and USP
guidelines.

##  VI VI. Acknowledgements

We would like to thank Combino Pharm Malta Ltd. for their invaluable
contributions to this study.

## References

  * (1)
  * (2)

##  VII VII. References

* (3)
* (4) D. G. Vidt, E. L. Bravo, and F. M. Fouad, “Captopril,” _New England Journal of Medicine_ , vol. 306, no. 4, pp. 214–219, 1982.
* (5) J. Bojarska, W. Maniukiewicz, A. Fruziński, L. Sieroń, and M. Remko, “Captopril and its dimer captopril disulfide: comparative structural and conformational studies,” _Acta Crystallographica Section C: Structural Chemistry_ , vol. 71, no. 3, pp. 199–203, 2015.
* (6) M. Amar, S. W. Lam, K. Faulkenberg, A. Perez, W. W. Tang, and J. B. Williams, “Captopril versus hydralazine-isosorbide dinitrate vasodilator protocols in acute decompensated heart failure patients transitioning from sodium nitroprusside,” _Journal of cardiac failure_ , 2021.
* (7) M. L. Sigit, M. A. Rahman, and I. K. A. Utamayasa, “Comparison of the effects of giving captopril and valsartan in cases of heart failure for people with congenital heart disease,” _EurAsian Journal of BioSciences_ , vol. 14, no. 2, pp. 3463–3467, 2020.
* (8) R. N. Brogden, P. A. Todd, and E. M. Sorkin, “Captopril,” _Drugs_ , vol. 36, no. 5, pp. 540–600, 1988.
* (9) C.-D. M. R. Group _et al._ , “Comparative effects of therapy with captopril and digoxin in patients with mild to moderate heart failure,” _JAMA: The Journal of the American Medical Association_ , vol. 259, no. 4, pp. 539–544, 1988.
* (10) D. Vasconcelos, C. Silva, A. Teixeira, and P. Freire, “Structural and spectroscopic analysis of the cis-trans isomers of the captopril in the gaseous and aqueous phases,” _Journal of Molecular Structure_ , p. 130872, 2021.
* (11) P. Timmins, I. M. Jackson, and Y.-c. J. Wang, “Factors affecting captopril stability in aqueous solution,” _International Journal of Pharmaceutics_ , vol. 11, no. 4, pp. 329–336, 1982.
* (12) S. Hillaert and W. Van den Bossche, “Determination of captopril and its degradation products by capillary electrophoresis,” _Journal of pharmaceutical and biomedical analysis_ , vol. 21, no. 1, pp. 65–73, 1999.
* (13) A. G. Cârje, A. Balint, V. Ion, A. L. Pop, D.-L. Muntean, R. Sabău, and S. Imre, “Hplc-uv method approach for the analysis and impurity profiling of captopril,” _Stud. Univ. Babes-Bolyai Chem_ , vol. 64, pp. 231–242, 2019.
* (14) National Institutes of Health and others, “Livertox database,” 2018, accessed: 27/06/2021. [Online]. Available: https://pubmed.ncbi.nlm.nih.gov/31643176/
* (15) United States Pharmacopeia and National Formulary. (2021) United States Pharmacopeial Convention. Accessed: 27/06/2021. [Online]. Available: https://online.uspnf.com/uspnf/document/1_GUID-F8EE4E12-6364-46A6-A14F-B3EAC3466D40_5_en-US
* (16) J. Cielecka-Piontek, P. Zalewski, A. Jelińska, and P. Garbacki, “Uhplc: the greening face of liquid chromatography,” _Chromatographia_ , vol. 76, no. 21, pp. 1429–1437, 2013.
* (17) A. Gałuszka, Z. Migaszewski, and J. Namieśnik, “The 12 principles of green analytical chemistry and the significance mnemonic of green analytical practices,” _TrAC Trends in Analytical Chemistry_ , vol. 50, pp. 78–84, 2013\.
* (18) I. H. T. Guideline _et al._ , “Validation of analytical procedures: text and methodology,” _Q2 (R1)_ , vol. 1, no. 20, p. 05, 2005.
* (19) J. D. Nally, _Good manufacturing practices for pharmaceuticals_. CRC Press, 2016.


# Using Topology to Estimate Structural Similarities of Proteins

Jørgen Ellegaard Andersen  Danish Institute for Advanced Study, University of
Southern Denmark  Centre for Quantum Mathematics, Department of Mathematics
and Computer Science, University of Southern Denmark  Jens Ledet Jensen
Department of Mathematics, Aarhus University  Yuki Koyanagi  Centre for
Quantum Mathematics, Department of Mathematics and Computer Science,
University of Southern Denmark  Jakob Toudahl Nielsen  Interdisciplinary
Nanoscience Center (iNANO), Aarhus University  Department of Chemistry, Aarhus
University  Rasmus Villemoes  Prevas A/S

###### Abstract

An effective model for protein structures is important for the study of
protein geometry, which, to a large extent, determine the functions of
proteins. There are a number of approaches for modelling; one might focus on
the conformation of the backbone or H-bonds, and the model may be based on the
geometry or the topology of the structure in focus. We focus on the topology
of H-bonds in proteins, and explore the link between the topology and the
geometry of protein structures. More specifically, we take inspiration from
CASP Evaluation of Model Accuracy and investigate the extent to which
structural similarities, via GDT_TS, can be estimated from the topology of
H-bonds. We report on two experiments; one where we attempt to mimic the
computation of GDT_TS based solely on the topology of H-bonds, and the other
where we perform linear regression where the independent variables are various
scores computed from the topology of H-bonds. We achieved an average
$\Delta\text{GDT}$ of 6.45 with 54.5% of predictions inside 2
$\Delta\mathrm{GDT}$ for the first method, and an average $\Delta\mathrm{GDT}$
of 4.41 with 72.7% of predictions inside 2 $\Delta\mathrm{GDT}$ for the second
method.

##  1 Introduction

It is widely recognised that the diverse functions of proteins are highly
dependent on the three-dimensional structures of their native conformations.
An effective model for describing the geometric structures of proteins is
therefore important for the study of protein structures. One of the earliest
models for describing the backbone conformation of proteins is the
Ramachandran plots, which plots the dihedral angles $(\varphi,\psi)$ before
and after each $C^{\alpha}$ atoms in two-dimensional distributions [30]. The
method has since been updated and extended to be used in structural validation
lovell03, read11 and a number of other purposes (see, for example, [6] for a
review). The extensions to the Ramachandran plots include combining two
consecutive pairs of conformation angles [17], and characterising entire
proteins by the averages over $\varphi$ and $\psi$ [7]. Another approach is to
use the coordinates of backbone atoms instead of dihedral angles. Examples of
this approach include the notion of curvature and torsion taken from
differential geometry rackovsky78, rackovsky84, and projection of nearby atoms
to a small sphere centred at each $C^{\alpha}$ atoms [22]. While the above
methods are all based on the geometry of the backbone, an alternative approach
is possible by considering its topology. A number of studies have used ideas
from knot theory to study the link between topology and geometry of proteins
levitt83, chen96, rogen03. Yet another approach is to focus on H-bonds, which
is one of the main mechanisms determining and stabilising the native structure
of the proteins [5, 34, 21]. Studies suggest incorporating H-bond geometry
improves the quality of protein structure models grishaev04, morozov04. In
[23], spatial rotations were introduced as a systematic three-dimensional
descriptor of H-bond geometry, and were found to correspond well to the
concrete secondary structures and other local structural motifs. The dataset
from [23] has further been used by Penner in penner20, penner21 to estimate
free energy of coronarirus spike proteins, with a view to identifying specific
sites of interest for vaccine development. If we concentrate on the topology
of H-bonds, we obtain a graph, with backbone atoms as vertices and the
covalent and H-bonds as edges. Such H-bond graphs were used to study the
dynamics of membrane proteins [35], and for structural comparison [29]. In
[26], an extension to this structure was used to study protein structures.

In this paper, we investigate the link between H-bond topology of proteins and
their geometric structures. We take inspiration from CASP Evaluation of Model
Accuracy (EMA) [10], and investigate how well we can estimate the GDT_TS of
the submitted structures using H-bond graphs of the submitted and target
structures. GDT_TS has been criticised, among others, for being dependent on
the lengths of proteins and for having somewhat arbitrary distance cutoffs
xu10, garg16. Nonetheless it is a widely accepted measure used to compare
protein structures, and we use it here as an indication of structural
similarities. We designed two experiments. In the first experiment, we attempt
to follow the algorithm for computing GDT_TS, but with only the proteins’
topological information (from their H-bond graphs) as the input. The second
experiment is a linear regression where independent variables are certain
similarity scores computed from the protein H-bond graph, and the dependent
variable is GDT_TS. We note here, that our methods are not intended as an
attempt for the CASP EMA. Indeed, both methods require the target structure’s
H-bond graph as part of the input data, which is not available in CASP. They
are intended as an investigation into the usefulness of protein topology in
comparison of protein structures. However, one could of course imagine
combining our methods with an algorithm to predict H-bond graphs from primary
sequences to be used in CASP EMA or similar experiments. Indeed, the idea for
the investigation originated in a novel approach to the protein folding
problem, inspired in part by [23]. It is based on a two-stage process, where
in the first stage one or more H-bond graphs are predicted from a primary
sequence, then in the second stage the geometric structure is predicted from
the H-bond graph(s). We have a method to enumerate possible H-bond graphs, as
well as a method to predict local geometric structure of proteins from H-bond
graphs andersen21enum, andersen21pre, andersen21topo. The current study fits
in this programme as a “proof of concept” for the idea that H-bond topology of
proteins is strongly linked to their geometric structures. Our model is purely
based on the topology of protein structures, therefore is less affected by the
dynamic nature of the proteins, which is important in their diverse functions
[13]. Furthermore, it is independent of alignment, which simplifies its use in
potential high throughput applications.

##  2 Methods and Results

###  2.1 Dataset

The dataset consists of 33 target structures for CASP14 together with the
submitted candidate structures, downloaded from CASP data archive [8] (There
were 34 target structures available for download, but one, T1044, did not have
any corresponding candidates and was dropped.). The size of proteins, measured
in the number of residues, ranged from 74 to 922 (Figure 1). Majority of the
target structures had length less than 300 residues, with 6 targets having
more than 300 residues. The range of the number of candidate structures per
protein was from 204 to 599, with the majority of targets receiving more than
500 submissions (Figure 2; Participants are allowed to submit more than one
candidate structure). The larger target structures seems to have received as
many submissions as the smaller target structures.

Figure 1: Frequency of target structures in CASP14 by length. Figure 2: Number
of candidates per target structure in CASP14. Short targets are those with
fewer than 300 residues, and the long targets are with more than 300 residues.

We also utilised data from CASP13 to construct our regression model (Section
2.3). There were 20 target structures available for download, with length
ranging from 52 to 405 residues, and two structures having more than 300
residues (Figure 3)

Figure 3: Frequency of target structures in CASP13 by length.

The data was processed to obtain information about the H-bonds, following the
procedure described in [23]. The H-bonds were determined by the DSSP program
[14], with the additional conditions [4];

$\displaystyle\text{HO-distance}<2.7\text{Å}$
$\displaystyle\text{angle(NHO)},\text{angle(COH)}>90^{\circ}.$

For the majority of proteins in the resulting data, the number of H-bonds was
roughly half of the length measured as the number of residues (Figure 4).

Figure 4: Number of H-bonds versus length for the target structures in CASP13
& 14.

###  2.2 GDT-like algorithm based on H-bond graphs

We attempt to mimic the GDT algorithm [38], but based only on protein H-bond
graphs, i.e. based on information about the protein’s hydrogen bonds, but not
its geometric structure.

Let $T$ be the graph of the target protein, with vertices
$\\{v_{1},\dotsc,v_{l}\\}$ representing the residues, ordered along the
backbone, and edges $\\{e_{1},\dotsc,e_{m}\\}$ representing the backbone
peptide bonds and H-bonds. Similarly, let $C$ be the graph of the candidate
protein, with vertices $\\{w_{1},\dotsc,w_{l}\\}$ representing the residues,
ordered along the backbone, and edges $\\{f_{1},\dotsc,f_{n}\\}$ representing
primary and H-bonds. For a set $S$, let $\\#S$ denote the number of elements
in $S$, and for a graph $G$, let $\mathcal{V}(G)$ and $\mathcal{E}(G)$ denote
the set of vertices and edges in $G$, respectively. We have
$l=\\#\mathcal{V}(T)(=\\#\mathcal{V}(C))$. The idea is to start with small
subgraphs of $C$ and $T$ (corresponding to the same backbone segment), and to
“grow” them incrementally, until the difference between the subgraphs is over
a pre-determined threshold value. We repeat this for different initial
subgraphs, and determine the maximum subgraph of $C$,
$\hat{C}_{\mathrm{sub}}$, whose difference from the corresponding subgraph of
$T$ (the correspondence is defined below in the detailed description) is below
some threshold value $r$. The score for the candidate graph, which we call
$g_{r}$, is then given by

$g_{r}=100\times\frac{\\#\mathcal{V}(\hat{C}_{\mathrm{sub}})}{\\#\mathcal{V}(C)}.$
(1)

We give a detailed description of the algorithm below, together with the
pseudocode in Algorithm 1. Let $d(A,B)$ be a distance function between graphs
$A$ and $B$ and let $r\in(0,\infty)$. We compute the score of similarity
between the graphs $T$ and $C$ as follows;

  1. 1.

Set $i=1$.

  2. 2.

We select a subgraph $C_{\text{sub}}(i)$ of $C$, consisting of three vertices
$\\{w_{i},w_{i+1},w_{i+2}\\}$ starting from the $i$th position and the edges
connecting them. Select a subgraph $T_{\text{sub}}(i)$ of $T$ in the same way.

  3. 3.

Compute the distance measure $d_{i}=d(T_{\text{sub}}(i),C_{\text{sub}}(i))$.

  4. 4.

If $d_{i}\geq r$, where $r$ is the pre-determined limit, the initial segment
is already over the limit value. Increment $i$ by 1, go to 2.

  5. 5.

If $d_{i}<r$, “grow” the subgraph $C_{\text{sub}}(i)$ by adding all edges that
are connected to $C_{\text{sub}}(i)$, together with the vertices connected to
these edges. Call the selected edges and vertices, together with
$C_{\text{sub}}(i)$, $C_{\text{sub}}2(i)$. Select $T_{\text{sub}}2(i)$ from
$T$ in the same way.

  6. 6.

Compute the distance measure $d_{i}=d(T_{\text{sub}}2(i),C_{\text{sub}}2(i))$.

  7. 7.

Repeat the above two steps (“growing” subgraphs and computing the distance
measure), until $d_{i}\geq r$. If $d_{i}\geq r$, move to the next starting
segments by incrementing $i$ by 1, go to 2.

  8. 8.

If $C_{\text{sub}}(i)=C$, we have the entire graph under the limit value.

  9. 9.

After going through all starting segments, we have a set
$S=\left\\{C_{\text{sub}}(i)\middle|i\in\\{1,\dotsc,l-2\\}\right\\}$ of
maximal $C_{\text{sub}}(i)$’s. Select the longest $C_{\text{sub}}(i)$ in $S$,
which we call $\hat{C}_{\text{sub}}$.

  10. 10.

$g_{r}=100\times\frac{\\#\mathcal{V}(\hat{C}_{\text{sub}})}{\\#\mathcal{V}(C)}$.

for $i$ in $\\{1,\dotsc,l-2\\}$ do

Let $C_{\text{sub}}(i)$ be the subgraph of $C$ obtained by taking three
vertices $\\{w_{i},w_{i+1},w_{i+2}\\}$ and the edges connecting them in $C$

Let $T_{\text{sub}}(i)$ be the subgraph of $T$ obtained by taking three
vertices $\\{v_{i},v_{i+1},v_{i+2}\\}$ and the edges connecting them in $T$

Compute the distance measure $d_{i}=d(T_{\text{sub}}(i),C_{\text{sub}}(i))$

if $d_{i}\geq r$ then

Continue to next $i$

end if

while True do

Let $T_{\text{sub}}2(i)$ be the subgraph of $T$ obtained by taking
$T_{\text{sub}}(i)$ together with all edges connected to the vertices in
$T_{\text{sub}}(i)$, and the end-vertices of these edges (i.e. ”grow” the
subgraph by 1 edge+vertex pair)

Let $C_{\text{sub}}2(i)$ be the subgraph of $C$ obtained in the same manner

Compute the distance measure $d_{i}=d(T_{\text{sub}}2(i),C_{\text{sub}}2(i))$

if $d(i)\geq r$ then

Break out of while loop

end if

Set $T_{\text{sub}}(i)=T_{\text{sub}}2(i)$

Set $C_{\text{sub}}(i)=C_{\text{sub}}2(i)$

if $T_{\text{sub}}(i)==T$ then

Break out of while loop

end if

end while

end for

$\hat{C}_{\text{sub}}=\max\left\\{C_{\text{sub}}(i)\middle|i\in\\{1,\dotsc,l-2\\}\right\\}$

$\mathrm{Score}=100\cdot\\#\mathcal{V}(\hat{C}_{\text{sub}})/\\#\mathcal{V}(C)$

Algorithm 1 Pseudocode for GDT-like algorithm

For the current analysis we define the distance function $d$ by

$d(A,B)=\\#(\mathcal{E}(A)\ominus\mathcal{E}(B)),$ (2)

where $\ominus$ denotes the symmetric difference of two sets;

$A\ominus B=(A\setminus B)\cup(B\setminus A).$

The algorithm is also dependent on the limit value $r$ for the distance
between two subgraphs. We tested for the effect of different $r$ values by
computing the score $g_{r}$ for $r=5,10,20,40,80$ for all candidate structures
and looking at their distributions, together with their correlation with
GDT_TS (Figure 5). As a result $r=80$ was excluded as being too high
(resulting in more than 20% of all structures having score of 100). GDT_TS is
computed as an average of scores for four different cutoff values (1,2,4, and
8 Å) [16]. We imitate this by computing an average over different sets of
$r$-values. The distributions of (average) scores for different sets of
$r$-values are shown in Figure 6. Based on these, we chose the average over
$r$-values 10, 20 and 40 as our final score, since the combination had the
widest spread of values. We call the final composite score $\Gamma$-GDT;

$\mbox{$\Gamma$-GDT}=(g_{10}+g_{20}+g_{40})/3.$ (3)

Figure 5: Distribution of $g_{r}$ scores for different values of $r$, against
GDT_TS. $\rho$ is the Spearman’s correlation coefficient between GDT_TS and
$g_{r}$ scores. Figure 6: Distribution of $\Gamma$-GDT for different
combinations of $r$-values.

We then predicted the best candidate structure for each target by selecting
the structure with the highest $\Gamma$-GDT, and look at the difference
between GDT_TS of our prediction and GDT_TS of the best candidate for each
target structure, which we call $\Delta\mathrm{GDT}$. The distribution of
$\Delta\mathrm{GDT}$ is shown in Figure 7. The average $\Delta\mathrm{GDT}$
for all targets was 6.45, with the highest value of 36.38. We were able to
identify a candidate with $\Delta\mathrm{GDT}<2$ in 18 targets, and with
$\Delta\mathrm{GDT}<10$ in 24 targets (Figure 7). The distribution of GDT_TS
for the best candidate structure against GDT_TS for the selected structure is
shown in Figure 8. It turned out the particular set of $r$-values we chose for
the computation of $\Gamma$-GDT gives the best prediction result (Table 1).

Figure 7: Distribution of $\Delta\mathrm{GDT}$ against length (measured in number of residues) for 33 target structures, predicted using $\Gamma$-GDT. Figure 8: GDT_TS for the best candidate structure against GDT_TS for the selected structure. $r$-values | $\Delta\mathrm{GDT}<2$ | $\Delta\mathrm{GDT}<10$
---|---|---
10,20,40 | 18 | 24
5,10,20 | 14 | 22
5,10,20,40 | 17 | 23
Table 1: Number of predictions (out of 33) with specified $\Delta\mathrm{GDT}$
range for different combinations of $r$-values.

We also tested for two different distance function to (2). In the first, we
reduced the contribution made by an edge in the set
$\mathcal{E}(C)\ominus\mathcal{E}(T)$, if there is an edge that lies close to
it. For $x\in\mathbb{R}$, define a function $f_{1}$ by

$f_{1}(x)=\begin{cases}1&\text{if }\mathinner{\\!\left\lvert
x\right\rvert}>4\\\ \mathinner{\\!\left\lvert
x\right\rvert}/4&\text{otherwise}.\end{cases}$ (4)

We define a new distance function $d_{1}$ by

$\displaystyle d_{1}$ $\displaystyle(A,B)=$ $\displaystyle\sum_{(p,q)\in
A\setminus B}\min\\{\frac{f_{1}(p-p^{\prime})+f_{1}(q-q^{\prime})}{2}$
$\displaystyle\qquad\qquad|(p^{\prime},q^{\prime})\in B\setminus A\\}$
$\displaystyle+\sum_{(p,q)\in B\setminus
A}\min\\{\frac{f_{1}(p-p^{\prime})+f_{1}(q-q^{\prime})}{2}$
$\displaystyle\qquad\qquad|(p^{\prime},q^{\prime})\in A\setminus B\\}.$ (5)

In the second, we tried to reduce the contribution by a “close” edge further
by setting

$f_{2}(x)=\begin{cases}1&\text{if }\mathinner{\\!\left\lvert
x\right\rvert}>4\\\ \frac{\exp(\mathinner{\\!\left\lvert
x\right\rvert})-1}{\exp(4)-1}&\text{otherwise},\end{cases}$ (6)

and

$\displaystyle d_{2}$ $\displaystyle(A,B)=$ $\displaystyle\sum_{(p,q)\in
A\setminus B}\min\\{\frac{f_{2}(p-p^{\prime})+f_{2}(q-q^{\prime})}{2}$
$\displaystyle\qquad\qquad|(p^{\prime},q^{\prime})\in B\setminus A\\}$
$\displaystyle+\sum_{(p,q)\in B\setminus
A}\min\\{\frac{f_{2}(p-p^{\prime})+f_{2}(q-q^{\prime})}{2}$
$\displaystyle\qquad\qquad|(p^{\prime},q^{\prime})\in A\setminus B\\}.$ (7)

The prediction results for different distance functions are shown in Figure 9.
We see that the performance of the original distance function $d$ (2), which
is a simple count of the elements in the symmetric difference of the sets of
edges, is significantly better than the two modified distance functions.

Figure 9: Percentages of predictions with $\Delta\mathrm{GDT}<2$,
$2\leq\Delta\mathrm{GDT}<10$, and $\Delta\mathrm{GDT}\geq 10$ for different
distance functions.

###  2.3 Linear regression based on the protein fatgraph model

The second method is a linear regression on the similarity scores which we
compute based on the hydrogen bonds in the candidate and target structures.
Each hydrogen bond is identified by the position of its donor- and acceptor
atoms, so each bond can be expressed as a 2-tuple of integers $(p,q)$, where
the donor is the $p$’th atom along the backbone and the acceptor the $q$’th.

The first of our similarity scores is the proportion of the bonds, which are
correctly identified in the candidate structure. In other words, if
$H_{T},H_{C}$ are the sets of hydrogen bonds respectively in the target
structure and in the candidate structure, then the first score $P$ is defined
as;

$P=\frac{\\#(H_{T}\cap H_{C})}{\\#H_{T}},$

where we use the fact that for two bonds $(p,q)$ and
$(p^{\prime},q^{\prime})$, $(p,q)=(p^{\prime},q^{\prime})$ iff $p=p^{\prime}$
and $q=q^{\prime}$.

The second similarity score $S_{n}$ depends on a parameter $n\in\mathbb{N}$.
For a non-negative integer $x\in\mathbb{Z}$, define

$f_{n}(x)=\begin{cases}1-x/n&\text{ if }x\leq 2n\\\ -1&\text{
otherwise}\end{cases}.$ (8)

For a bond $(p,q)\in H_{C}$, set

$\displaystyle s_{C}$ $\displaystyle((p,q))$
$\displaystyle=\max\left\\{f_{n}(\mathinner{\\!\left\lvert
p-p^{\prime}\right\rvert})+f_{n}(\mathinner{\\!\left\lvert
q-q^{\prime}\right\rvert})\right.$
$\displaystyle\qquad\qquad\qquad\qquad\left.\mid(p^{\prime},q^{\prime})\in
H_{T}\setminus H_{C}\right\\}.$

Similarly for $(p,q)\in H_{T}$, set

$\displaystyle s_{T}$ $\displaystyle((p,q))$
$\displaystyle=\max\left\\{f_{n}(\mathinner{\\!\left\lvert
p-p^{\prime}\right\rvert})+f_{n}(\mathinner{\\!\left\lvert
q-q^{\prime}\right\rvert})\right.$
$\displaystyle\qquad\qquad\qquad\qquad\left.\mid(p^{\prime},q^{\prime})\in
H_{C}\setminus H_{T}\right\\}.$

$S_{n}$ is then given by

$\displaystyle S_{n}$ $\displaystyle=\frac{1}{\\#((H_{T}\setminus
H_{C})\cup(H_{C}\setminus H_{T}))}\times$ $\displaystyle\left(\sum_{(p,q)\in
H_{C}\setminus H_{T}}s_{C}((p,q))+\right.$
$\displaystyle\left.\quad\sum_{(p^{\prime},q^{\prime})\in H_{T}\setminus
H_{C}}s_{T}((p^{\prime},q^{\prime}))\right).$

So for a given candidate structure, we can compute $S_{n}$ for different
$n$’s.

Having calculated $P$ and $S_{n}$, $n\in I$, where $I$ is a subset of
$\\{1,2,\dotsc,10\\}$, for all candidate structures, we perform a linear
regression with $P$, $S_{n}$ as independent variables and GDT_TS as the
dependent variable. We estimated the regression model using data from CASP13,
and applied the model to data from CASP14. After testing for all subsets
$I\subset\\{1,2,\dotsc,10\\}$ by running multiple regression with $P$ and
$S_{n},n\in I$ as independent variables, we found that setting $I=\\{2\\}$
gave the best results with CASP13 data. The regression equation, based on all
candidate structures in CASP13, was determined to be

$\mathrm{GDT\\_TS}=10.70+0.63P+1.26S_{2}.$

Using this equation, we estimated the GDT_TS for CASP14 candidate structures,
and selected the structure with the highest estimated GDT_TS for each target.
We were able to identify a candidate structure with $\Delta\mathrm{GDT}<2$ for
23 out of 33 targets, with the average $\Delta\mathrm{GDT}$ of 4.57 (Table 2).
The frequency distribution of $\Delta\mathrm{GDT}$ is shown in Figure 10. The
large $\Delta\mathrm{GDT}$ values were observed in shorter proteins, although
it must be noted that most of targets have lengths less than 300 residues.

Figure 10: Distribution of $\Delta\mathrm{GDT}$ against length (measured in
number of residues) for 33 target structures, by linear regression method.

We also investigated the effect of the score function (8) by scaling it with
the exponential function;

$\tilde{f}_{n}(x)=\begin{cases}1-\frac{2(\exp(x)-1)}{\exp(2n)-1}&\text{ if
}x\leq 2n\\\ -1&\text{ otherwise}\end{cases}.$ (9)

Compared to (8), the new score function (9) gives smaller penalties to
difference in bond positions, especially when the difference is small. Using
the data from CASP13 and (9), we found that setting $I=\\{2,6,8,10\\}$ gave
the best result with regards to identifying the most candidates with
$\Delta\mathrm{GDT}<2$. However, the $S_{n}$ scores are strongly correlated,
and we decided to use $I=\\{2\\}$ again. This gave a result close to that
obtained with $I=\\{2,6,8,10\\}$ (11 targets with $\Delta\mathrm{GDT}<2$,
compared to 12 targets with $\Delta\mathrm{GDT}<2$). The regression equation
was

$\mathrm{GDT\\_TS}=9.68+0.63P+0.20S_{2}.$

The new score function (9) resulted in a small improvement for prediction with
CASP14 data, where we were able to identify a candidate structure with
$\Delta\mathrm{GDT}<2$ for 24 out of 33 targets, and the average
$\Delta\mathrm{GDT}$ of 4.41 (Table 2).

We then removed the $S_{n}$ scores from the independent variables, and ran the
regression with only $P$ scores as the independent variable. The regression
equation now read

$\mathrm{GDT\\_TS}=9.56+0.63P.$

This only resulted in a small drop in our ability to identify the best
candidate structure, with $\Delta\mathrm{GDT}<2$ for 22 out of 33 targets and
the average $\Delta\mathrm{GDT}$ of 5.53 (Table 2).

| % of candidates with
---|---
| $\Delta\mathrm{GDT}<2$ | $\Delta\mathrm{GDT}<10$
$f_{n}$ | 69.70 | 84.85
$\tilde{f}_{n}$ | 72.73 | 84.85
No $S_{n}$ | 66.67 | 84.85
Table 2: Prediction results for different score functions, showing the
percentages of targets (out of 33), where the selected candidate structure had
$\Delta\mathrm{GDT}$ less than 2 and 10, respectively.

##  3 Discussion

We have shown that the information on H-bonds alone can, to a large extent,
correctly assess similarities in geometric structures of proteins. Even though
a direct comparison between our results and CASP EMA is not possible, the
performance of our methods, measured as the percentages of predictions with
$\Delta\mathrm{GDT}<2$ and $\Delta\mathrm{GDT}<10$, are clearly numerically
superior to the best performance in CASP 14 EMA (Figure 11). It could be
argued that both our methods essentially rely on simply counting the matched
(or unmatched, in the case of $\Gamma$-GDT) H-bonds in two structures. The
modified distance functions in the $\Gamma$-GDT and the $S$ scores in the
linear regression, which measures the differences between unmatched H-bonds,
have negative or relatively small positive effect on the overall accuracy of
predictions. The fact that these relatively simple methods can nonetheless
assess similarities in protein structures correctly, demonstrates the strong
link between the topology and the geometry of proteins.

Figure 11: Percentages of the predictions with $\Delta\mathrm{GDT}<2$,
$2\leq\Delta\mathrm{GDT}<10$ and $\Delta\mathrm{GDT}\geq 10$. The figures for
CASP14 are averages of all models, top 10 models, and the best-performing
model, ordered by the number of predictions with $\Delta\mathrm{GDT}<2$. The
data for CASP14 was obtained from CASP Data Archive
(https://www.predictioncenter.org/download_area/CASP14/) and processed by the
authors.

In the linear regression analysis, we observe that the larger values of $n$ in
$S_{n}$ scores, which in effect enlarges the search window for “similar”
H-bonds, do not improve the prediction accuracy. An explanation could be that
it is simply a consequence of using GDT_TS as the measure of structural
similarity, as a small local difference (e.g. an extra turn where there should
be none) can result in a significantly lower GDT_TS. Further investigation is
needed to ascertain the cause of this behaviour.

There are broadly two types of methods used in CASP model accuracy estimation.
Consensus, or clustering methods take multiple candidate structures as input
and tries to identify a structure, that is the “best match” for the input
structures according to some criteria. Single-model methods, on the other
hand, takes a single candidate structure as an input and tries to estimate its
accuracy, independent of other candidate structures. The consensus methods
have generally outperformed the single-model methods, and this resulted in the
development effort being concentrated on the consensus methods in the past
[31]. More recently the single-model methods have received more attention and
development effort kryshtafovych18, as the potential issues with the consensus
methods are recognised. One issue, for example, is that the consensus methods
may not be very useful in the environment outside the CASP-setup, where a
large number of candidate structures may not be available for the input.
Another potential issue, related to the first, is that the consensus methods
may simply be taking advantage of the fact that many CASP models are now able
to produce high-quality candidate structures, which are, naturally, similar to
each other [36]. Our method is, by construction, unlikely to be improved to
outperform the best accuracy estimation methods, as it ignores the geometric
data in the candidate structures and only utilises the topological data.
However, the relative simplicity of our method means it should be relatively
easy to combine it with an existing method to improve its performance. We
chose not to attempt it in this paper, as our focus here has been to
investigate the link between the topology and the geometry of proteins, rather
than to participate in CASP EMA. Nonetheless, as we mentioned in Introduction,
one could easily imagine combining our method with an algorithm for predicting
hydrogen bonds from a primary sequence. When a high-accuracy prediction of
hydrogen bonds becomes possible, our method has the advantage that it could be
combined with both a consensus method and a single-model method.

### Acknowledgement

This paper is partly a result of the ERC-SyG project, Recursive and Exact New
Quantum Theory (ReNewQuantum) which received funding from the European
Research Council (ERC) under the European Union’s Horizon 2020 research and
innovation programme under grant agreement No. 810573.

## References

  * [1] Jørgen Ellegaard Andersen, Hiroyuki Fuji and Yuki Koyanagi  “Enumeration of protein structures by matrix model techniques” In preparation
  * [2] Jørgen Ellegaard Andersen, Hiroyuki Fuji and Yuki Koyanagi  “Topology of protein metastructure and $\beta$-sheet topology” Preprint
  * [3] Jørgen Ellegaard Andersen, Yuki Koyanagi and Jakob Toudahl Nielsen  “Prediction of H-bond rotations from protein H-bond topology” Preprint
  * [4] E.N. Baker and R.E. Hubbard  “Hydrogen bonding in globular proteins”  In _Progress in Biophysics and Molecular Biology_ 44.2, 1984, pp. 97–179
  * [5] Domenico Bordo and Patrick Argos  “The role of side-chain hydrogen bonds in the formation and stabilization of secondary structure in soluble proteins”  In _Journal of molecular biology_ 243.3 Elsevier, 1994, pp. 504–519
  * [6] Oliviero Carugo and K Djinović-Carugo  “Half a century of Ramachandran plots”  In _Acta Crystallographica Section D: Biological Crystallography_ 69.8 International Union of Crystallography, 2013, pp. 1333–1341
  * [7] Oliviero Carugo and Kristina Djinović-Carugo  “A proteomic Ramachandran plot (PRplot)”  In _Amino acids_ 44.2 Springer, 2013, pp. 781–790
  * [8] “CASP Data Archive”, https://predictioncenter.org/download_area/CASP14/
  * [9] Shi-Jie Chen and Ken A Dill  “Symmetries in proteins: A knot theory approach”  In _The Journal of chemical physics_ 104.15 American Institute of Physics, 1996, pp. 5964–5973
  * [10] Domenico Cozzetto, Andriy Kryshtafovych, Michele Ceriani and Anna Tramontano  “Assessment of predictions in the model quality assessment category”  In _Proteins: Structure, Function, and Bioinformatics_ 69.S8 Wiley Online Library, 2007, pp. 175–183
  * [11] Shikhin Garg, Smarth Kakkar and Ashish Runthala  “Improved protein model ranking through topological assessment”  In _Computational Biology and Bioinformatics_ CRC Press, 2016, pp. 410–428
  * [12] Alexander Grishaev and Ad Bax  “An empirical backbone- backbone hydrogen-bonding potential in proteins and its applications to NMR structure refinement and validation”  In _Journal of the American Chemical Society_ 126.23 ACS Publications, 2004, pp. 7281–7292
  * [13] Katherine Henzler-Wildman and Dorothee Kern  “Dynamic personalities of proteins”  In _Nature_ 450.7172 Nature Publishing Group, 2007, pp. 964–972
  * [14] Wolfgang Kabsch and Christian Sander  “Dictionary of protein secondary structure: Pattern recognition of hydrogen-bonded and geometrical features”  In _Biopolymers_ 22.12, 1983, pp. 2577–2637
  * [15] Andriy Kryshtafovych, Bohdan Monastyrskyy, Krzysztof Fidelis, Torsten Schwede and Anna Tramontano  “Assessment of model accuracy estimations in CASP12”  In _Proteins: Structure, Function, and Bioinformatics_ 86 Wiley Online Library, 2018, pp. 345–360
  * [16] Andriy Kryshtafovych, Andreas Prlic, Zinoviy Dmytriv, Pawel Daniluk, Maciej Milostan, Volker Eyrich, Tim Hubbard and Krzysztof Fidelis  “New tools and expanded data analysis capabilities at the Protein Structure Prediction Center”  In _Proteins: Structure, Function, and Bioinformatics_ 69.S8 Wiley Online Library, 2007, pp. 19–26
  * [17] Michael Levitt  “A simplified representation of protein conformations for rapid simulation of protein folding”  In _Journal of molecular biology_ 104.1 Elsevier, 1976, pp. 59–107
  * [18] Michael Levitt  “Protein folding by restrained energy minimization and molecular dynamics”  In _Journal of molecular biology_ 170.3 Elsevier, 1983, pp. 723–764
  * [19] Simon C Lovell, Ian W Davis, W Bryan Arendall III, Paul IW De Bakker, J Michael Word, Michael G Prisant, Jane S Richardson and David C Richardson  “Structure validation by C$\alpha$ geometry: $\phi$, $\psi$ and C$\beta$ deviation”  In _Proteins: Structure, Function, and Bioinformatics_ 50.3 Wiley Online Library, 2003, pp. 437–450
  * [20] Alexandre V Morozov, Tanja Kortemme, Kiril Tsemekhman and David Baker  “Close agreement between the orientation dependence of hydrogen bonds observed in protein structures and quantum mechanical calculations”  In _Proceedings of the National Academy of Sciences_ 101.18 National Acad Sciences, 2004, pp. 6946–6951
  * [21] C Nick Pace, J Martin Scholtz and Gerald R Grimsley  “Forces stabilizing proteins”  In _FEBS letters_ 588.14 Elsevier, 2014, pp. 2177–2184
  * [22] Xubiao Peng, Alireza Chenani, Shuangwei Hu, Yifan Zhou and Antti J Niemi  “A three dimensional visualisation approach to protein heavy-atom structure reconstruction”  In _BMC structural biology_ 14.1 BioMed Central, 2014, pp. 1–16
  * [23] Robert Penner, Ebbe Sloth Andersen, Jens Ledet Jensen, Adriana Krassimirova Kantcheva, Maike Bublitz, Poul Nissen, Anton Michael Havelund Rasmussen, Katrine Louise Svane, Bjørk Hammer, Reza Rezazadegan, Niels Christian Nielsen, Jakob Toudahl Nielsen and Jørgen Ellegaard Andersen  “Hydrogen bond rotations as a uniform structural tool for analyzing protein architecture”  In _Nature Communications_ 5 Nature Publishing Group, 2014
  * [24] Robert C Penner  “Conserved High Free Energy Sites in Human Coronavirus Spike Glycoprotein Backbones”  In _Journal of Computational Biology_ 27.11 Mary Ann Liebert, Inc., publishers 140 Huguenot Street, 3rd Floor New …, 2020, pp. 1622–1630
  * [25] Robert Clark Penner  “Antiviral Resistance against Viral Mutation: Praxis and Policy for SARS CoV-2”  In _BioRxiv_ Cold Spring Harbor Laboratory, 2021
  * [26] Robert, C. Penner, Micheal Knudsen, Carsten Wiuf and Jørgen Ellegaard Andersen  “Fatgraph models of proteins”  In _Communications on Pure and Applied Mathematics_ 63.10, 2010, pp. 1249–1297
  * [27] S Rackovsky and HA Scheraga  “Differential geometry and polymer conformation. 1. Comparison of protein conformations1a, b”  In _Macromolecules_ 11.6 ACS Publications, 1978, pp. 1168–1174
  * [28] S Rackovsky and HA Scheraga  “Differential geometry and protein folding”  In _Accounts of Chemical Research_ 17.6 ACS Publications, 1984, pp. 209–214
  * [29] Ofer Rahat, Uri Alon, Yaakov Levy and Gideon Schreiber  “Understanding hydrogen-bond patterns in proteins using network motifs”  In _Bioinformatics_ 25.22 Oxford University Press, 2009, pp. 2921–2928
  * [30] G.. Ramachandran, C. Ramakrishnan and V. Sasisekharan  “Stereochemistry of polypeptide chain configurations”  In _J. Mol. Biol._ 7, 1963, pp. 95–99
  * [31] Arjun Ray, Erik Lindahl and Björn Wallner  “Improved model quality assessment using ProQ2”  In _BMC bioinformatics_ 13.1 Springer, 2012, pp. 224
  * [32] Randy J Read, Paul D Adams, W Bryan Arendall III, Axel T Brunger, Paul Emsley, Robbie P Joosten, Gerard J Kleywegt, Eugene B Krissinel, Thomas Lütteke and Zbyszek Otwinowski  “A new generation of crystallographic validation tools for the protein data bank”  In _Structure_ 19.10 Elsevier, 2011, pp. 1395–1412
  * [33] Peter Røgen and Boris Fain  “Automatic classification of protein structure by using Gauss integrals”  In _Proceedings of the National Academy of Sciences_ 100.1 National Acad Sciences, 2003, pp. 119–124
  * [34] George D Rose and Richard Wolfenden  “Hydrogen bonding, hydrophobicity, packing, and protein folding”  In _Annual review of biophysics and biomolecular structure_ 22.1 Annual Reviews 4139 El Camino Way, PO Box 10139, Palo Alto, CA 94303-0139, USA, 1993, pp. 381–415
  * [35] Malte Siemers, Michalis Lazaratos, Konstantina Karathanou, Federico Guerra, Leonid S Brown and Ana-Nicoleta Bondar  “Bridge: A graph-based algorithm to analyze dynamic H-bond networks in membrane proteins”  In _Journal of chemical theory and computation_ 15.12 ACS Publications, 2019, pp. 6781–6798
  * [36] Jonghun Won, Minkyung Baek, Bohdan Monastyrskyy, Andriy Kryshtafovych and Chaok Seok  “Assessment of protein model structure accuracy estimation in CASP13: Challenges in the era of deep learning”  In _Proteins: Structure, Function, and Bioinformatics_ 87.12 Wiley Online Library, 2019, pp. 1351–1360
  * [37] Jinrui Xu and Yang Zhang  “How significant is a protein structure similarity with TM-score= 0.5?”  In _Bioinformatics_ 26.7 Oxford University Press, 2010, pp. 889–895
  * [38] Adam Zemla  “LGA: a method for finding 3D similarities in protein structures”  In _Nucleic acids research_ 31.13 Oxford University Press, 2003, pp. 3370–3374