drug induced safety issues , to include renal impairment , continue to be a significant reason that drugs fail the development process 1 .
however , the ability to identify drug induced renal function changes during the development process continues to be a challenge .
the traditional renal biomarkers of urea and creatinine are frequently obtained in all phases of clinical trials as these markers are accessible , analytically stable 3 , and cost effective , but viewed as insensitive and non - specific monitors of renal function 4
. the non - specific nature of urea and creatinine as renal markers may be advantageous in clinical trials if the ability to identify drug induce renal impairment can be improved .
therefore , a sensitive statistical tool to identify drug induced changes in renal function , using these traditional biomarkers , will be valuable in the drug development process .
the key element to identify a drug induced change in renal function from serum urea and creatinine is understanding that small changes are significant for these biomarkers .
multiple definitions of a toxicity signal exist such as published toxicity tables , identifying values outside the reference interval , using multiples of the reference interval , evaluating multiples of the baseline value ( 2 fold increase , 3 fold increase ) , and % change from baseline .
these definitions of a drug induced signal often fail because important drug induced changes for serum urea or creatinine will be small relative to the reference interval . alternatively stated , the index of individuality ( ioi = cvi / cvg where cvi is the intra - individual biological variability and cvg is the inter - individual biological variability ) is 0.67 for urea and 0.56 for creatinine 5 .
a z - score can be defined as the number of standard deviations between a visit value and a baseline mean : where cva is the coefficient of analytical variation and n is the number of baseline readings 5 . assuming a cva of 5% , the 99% confidence interval ( z=2.58 ) for a change from two baseline values , and given cvi 's of 5% , 10% , and 20% , the important change for an analyte value is approximately 22% , 35% , and 65% increases from the baseline mean respectively .
these % changes could easily be masked by the definition of the renal impairment signal .
the cvi for creatinine is typically about 5% while urea 's cvi is typically in the range of 10% 5 . consequently looking at z=2.58 shift in the patient 's values , a 22% change in a serum creatinine value or a 35% change in a urea value ,
should be identified as a potential drug effect in a subject in a clinical trial . in comparison ,
if the mean value of the reference interval is used , 75 mol / l for creatinine and 5.0 mmol / l for urea , the absolute changes would be 16.5 mol / l ( final value 91.5 mol / l ) for creatinine and 1.8 mmol / l for urea ( final value of 6.8
these changes are well within the reference intervals ( see methods for reference intervals ) and all traditional definitions of renal impairment for urea and creatinine would be negative for more than half of the trial population if all subjects had a significant increase in biomarker value based on the estimated % cvi .
the solution for improving the sensitivity of urea and creatinine as renal toxicity markers is to move from a traditional fixed limit definition of a toxic signal to a statistical evaluation of the signal .
a z - score approach of defining an important change from the biological and analytical variation is partial solution to this issue 5 .
recently , it has been shown that both traditional reference intervals and z - score formulas are two particular cases of a more general adaptive bayesian model 6 .
population cvi and cvg can be used as prior information in a bayesian network ( bn ) to move from population- to individual - based intervals as the number of individual readings increases .
heterogeneous factors leading to stratified reference intervals ( such as age and gender ) can be integrated in the bn to further remove between - subject variation .
an assumption in the z - score formula is that the population % cvi is constant across the population . by moving to an adaptive bayesian approach ,
the detection algorithm is customized for biomarker and subject thereby minimizing assumptions and maximizing the sensitivity of signal identification .
the adaptive bayesian approach has been successfully applied for the evaluation of biomarkers as formalized in the so - called biological passport 7 - 8 .
biomarker signal defined statistically and detected through a very sensitive statistical technique may find an unimportant signal .
however , in phase i and phase ii trials where populations are small , identifying a signal permits one to watch the signal in the development process to see if the signal becomes medically important 9 .
the lack of a toxicity signal viewed through a very sensitive statistical tool early in the development process minimizes the risk of a toxicity being found later in development .
two phase iii clinical trial datasets , with known drug induced renal impairment , are analyzed for a renal impairment signal by traditional methods and an adaptive bayesian approach in this report .
interestingly , the knowledge of renal impairment makes it possible to estimate the specificity ( control group ) and sensitivity ( treated group ) of the applied methods for the first visits post - administration when the signal is most probably weak .
two phase iii clinical trial datasets , one for urea , one for creatinine , were analyzed retrospectively .
the clinical trial database for urea was composed of 590 patients ( aged 26 - 92 , median 65 ) distributed among the placebo ( 200 patients ) and two drug concentrations ( group 1 : 194 patients , group 2 : 196 patients ) .
the creatinine clinical trial database had 532 patients ( aged 26 - 97 , median 65 ) distributed among the placebo ( 178 patients ) and two drug concentrations ( group 1 : 174 patients , group 2 : 180 patients ) .
the patients had between 1 and 3 pre - treatment visits and data was analyzed for the first 3 visits after drug administration .
the blood for the urea and creatinine measurements was collected in bd vacutainers ( becton dickinson , franklin lakes , nj ) , serum separated by the site , and transported to a central laboratory ( covance central laboratory ) .
samples were analyzed on roche chemistry analyzers ( roche , basel , switzerland ) using roche urea and rate blanked compensated alkaline picrate creatinine reagents .
all statistical simulations were performed on matlab version 7.7.0 with statistics toolbox version 7.0 ( mathworks , natick , ma ) .
the following stratified reference intervals were used for creatinine : males 18 - 50 years , 40 - 110 mol / l ; females 18 - 70 years , 31 - 101 mol / l ; males 50 - 70 years , 40 - 119 mol / l ; males 70 - 80 years , 40 - 137
mol / l ; females 70 - 80 years , 31 - 110 mol / l ; males > 80 years , 40 - 145 mol / l ; females > 80 years , 31 - 128
mol / l ( covance central laboratory ) and for urea : males and females 18 - 70 years , 1.4 - 8.6 mmol / l ; males and females 70 - 80 years , 1.4 - 10.4
mmol / l ; males and females > 80 years , 1.4 - 12.1 mmol / l ( covance central laboratory ) .
toxicity of grade 1 were computed for all data following the 2009 toxicity table of the national institute of allergy and infectious disease ( niaid ) .
this table lists a grade 1 creatinine toxic value as 1.1 to 1.5 times the upper limit of normal ( uln ) and a urea toxic value as 1.25 to 2.5 uln .
the adaptive bayesian approach was implemented in a hierarchical bn ( see figure 1 ) with biomarker signal decomposed into within - subject and between - subject components .
it is assumed that a series of urea or creatinine values is normally distributed with mean and variance specific to each subject .
prior distributions of within - subject variation ( denoted in figure 1 by the variable cvi ) and of the mean have been modeled as follows .
the cvi is assumed to be log - normally distributed with geometric mean ( gm ) equal to -1.97 and -2.6 and geometric standard deviation ( gsd ) equal to 0.14 and 0.095 , for urea and creatinine respectively . with these parameters ,
the 99%-interval of prior distribution of cvi is [ 8 - 22]% for urea and [ 5.6 - 9.2]% for creatinine .
the mean is assumed to be log - normally distributed with gm=0.89 and gsd of 0.15 for urea , gm=3.92 and gsd=0.13 for creatinine .
the latter gsd gives the amplitude of the between - subject variations post - stratification .
the stratification according to gender and age was modeled so that to obtain the stratified reference intervals given above ( assuming 99%-intervals ) .
a bn was associated to each subject , with age and gender integrated as evidence in the bn before the start of the trial .
then , bayesian inference techniques were used to iteratively and adaptively integrate pre - treatment visit values .
posterior distributions of expected values were generated by the bn to analyze the values obtained in treatment .
finally , 99%-intervals were computed as the 0.5 and 99.5 percentiles of the posterior distributions .
these intervals can be viewed as biomarker and subject and pre - treatment data specific reference intervals .
in addition to testing single visit values , the bn was also used to test sequences of visit values obtained after treatment 6 .
for example , if the first two visit values fall both at the 95 percentile of the distribution of expected values , the sequence composed of these two values falls approximately at the 99.75 percentile of the distribution of expected sequences of length 2 .
it has been shown that the latter approach has a significantly better sensitivity in the analysis of longitudinal biomarker data in the presence of a weak , but consistent over time , signal 9 - 11 .
on pre - treatment data , an analysis of variance returned a cvi of 14% for urea and 6% for creatinine .
these cvi were used in the z - score formula of equation ( 1 ) .
the magnitude of the z - score is a measure in standard deviations of how far the value has moved from the baseline values .
it has been shown that the z - score formula of equation ( 1 ) is a particular case of the bn described above 10 .
the bn gives the same results as the z - score formula when ( 1 ) between - subject variations are assumed to be infinite , ( 2 ) the cvi is assumed to be universal ( i.e. no subject - based cvi ) and ( 3 ) no heterogeneous factors are taken into account 10 .
similarly , by design , the reference intervals returned by the bn before the integration of any individual biomarker data correspond to the traditional reference intervals stratified according to age and gender 11 .
the greatest false - positive signal for both urea and creatinine was % values above the reference intervals .
the adaptive bayesian approach applied to single visit values or to sequences returned false - positive signals between 0.5 and 3.2% .
these results are in agreement with a theoretical rate of 1% induced by the use of 99%-intervals .
the mean and sd baseline values for urea and creatinine values across the cohorts are very similar ( control , group 1 , group 2 respectively , urea mean , 5.7 , 5.8 , 5.6 mmol / l , sd , 1.8 , 1.9 , 1.6 mmol / l ; creatinine , 75.6 , 76.6 , 76.4 mol / l , sd 20.5 , 17.9 , 22.7 mol / l ) prior to the administration of the drug indicating a well - executed randomization of the patients . the number of subjects per visit ranged from 187 to 173 for the urea data and ranged from 178 to 77 for the creatinine data
. the number of patients for the creatinine measurement decreased by approximately 50% , by design , in moving across the visits .
the creatinine mean values increased with time and drug dose while the urea mean values increased most directly with drug dose .
a small signal was found with the traditional signal definitions of % high flags , niaid toxicity grade 1 , and 2 fold increases from baseline values .
the adaptive bayesian analysis of sequences was still more sensitive than the bayesian approach based on a single visit value .
no values were observed with a toxicity grade greater than 1 , no values exceeded twice the upper limit of the reference interval , and no values were observed with a 3 fold increase or greater . at the population level , the mean values by group and visit show that the mean values increased after drug administration .
although the increases in mean values are small relative to the reference intervals , all increases were significant since the first visit in - treatment ( p<0.01 ) . in figure 2
, the graphs of % change from baseline to the adaptive bayesian z - value demonstrate the same % change can have different z values and that the placebo distributions are different from the drug treated patient distributions .
placebo patients have only one z greater than 3 for urea and 2 z greater than three for creatinine .
the adaptive bayesian methodology generated a prominent drug induced signal for renal impairment on both analytes at the first visit after drug administration .
all traditional definition of renal toxicity produced none or significantly weaker signals than the adaptive bayesian approach .
a dose dependent increase in the adaptive bayesian sequential signal indicates a drug effect is present and is increasing over time .
the data shows that a drug effect can be identified at the first time point after drug administration .
this urea change , although significant from a statistical point of view , is difficult to interpret from a biological perspective as this change remains in the reference interval and the renal handling of urea is complex and dependent on patient hydration .
mol / l is more easily interpreted relative to the modification of diet in renal disease ( mdrd ) equation 12 .
subsequently , an increase in the group 2 creatinine results of 21% by visit 3 results in a decrease in the population gfr by 16% .
the signal definition for renal toxicity needs to reliably identify this magnitude of population serum creatinine increase .
the out - of - reference range high values and toxicity grade 1 values do provide an indication of a drug effect and as the results are different between the placebo and treated groups .
the weak signal could be due to noise in the system , due to expected disease related changes , or a function of the patient 's baseline value being close to the upper limit of the reference interval .
the comparison of the out of range high results to the toxicity grade 1 results would imply that the high flagged results were false - positives as the % high results are 2 to 3 times greater that the tox grade 1 results . the use of multiples of the reference interval ( 2 or 3 uln ) to define renal toxicity is less sensitive that the toxicity grade 1 limits .
for instance calculating a z - value using literature cvi values , at the boundary of 2 uln from the low and high ends of the reference interval , the patient z - values are very different . the z - value change in moving from the lower limit of normal for urea and a male creatinine results to 2 times
the z - value change in moving from the upper limit of normal to 2 times the upper limit of normal is z=152 for a urea result and z=17 for a male creatinine result .
this calculation demonstrates that the probability of generating a toxicity signal varies greatly with the baseline value and reference interval using traditional definitions of the biomarker signal .
also multiples of the reference interval require a substantial drug effect on the kidney to define a toxic signal as the z - values are very high .
the drug effect is concentration and time dependent using the adaptive bayesian methodology to calculate the signal .
the false - positive reference interval high results were significantly greater for both biomarkers than the % mean false - positive adaptive bayesian results .
the false - positive adaptive bayesian signal is in the range of one tenth of the true signal . on the contrary ,
the false - positive rate for the % above the reference interval signal was approximately 50% of the treated group 's signal .
the toxicity grading had a low false - positive rate equal to the adaptive bayesian approach .
however , the toxicity grading had a very low rate of true positive signal identification also .
the adaptive sequential bayesian analysis rapidly produced a signal that exceeded the signal from the traditional definitions .
the adaptive bayesian approach identified a drug effect on both analytes at the first visit after drug was administered on a significant number of patients .
the rapid identification of a drug induced effect on renal function permits the rapid medical evaluation of this observed effect and , if appropriate , generate requests for additional renal biomarker data to define the mechanism and magnitude of the effect on key kidney function parameters .
the same approach has been successfully applied for the evaluation of creatinine data in early phase clinical trials 9 .
the data in figure 2 shows a marginal correlation of the % change from baseline to the adaptive sequential bayesian z - value results .
the uncertain part of looking at % change from baseline is that the significance of the same % change from baseline will be different for different biomarkers and subjects .
the % change from baseline methodology creates difficulties in scoring individual patients as having moved significantly from the baseline value .
the challenge of a development protocol is to move from population data to individual patient data and be able to identify the individual patient who needs to be medically evaluated .
this dataset provides insight as to the magnitude of an adaptive bayesian signal with a change in the population mean value .
an approximate 20% change in the urea mean biomarker value generated an adaptive bayesian signal for 31% of the treated subjects while the toxicity grade 1 signal was 3.4% .
an approximate 20% change in the creatinine mean value generated an adaptive bayesian signal for 29% of the treated subjects while the tox grade 1 signal was 6.5% .
change from baseline analysis makes defining the point of significant change more difficult than the adaptive bayesian z value approach as the same % change has a different statistical meaning across biomarkers and across subjects . the adaptive bayesian approach will identify the specific patients that have changed from the baseline value and minimize the dilution of true positives with false - positive observations .
both the creatinine and urea concentrations were rapidly effected by the drug even though the mechanism of clearance of creatinine and urea are significantly different . if renal impairment using non - specific markers such as creatinine and urea can reliably identify important changes from baseline with the adaptive bayesian approach , the non - specific nature of these biomarkers may be very appropriate for first line screening of renal effects in clinical trial populations .
although much has been done by the predictive safety testing consortium ( ptsc ) to find new clinically relevant biomarkers of renal function 13 , our study suggests that conventional biomarkers like creatinine and urea are able to detect early kidney dysfunction when using individual reference ranges .
also , the personalization of a biomarker signal makes possible the evaluation of drug efficacy and safety on an individual basis and in turn to tailor drug therapy at a dosage that is most appropriate for an individual patient .
correlations with genotyping data on metabolic pathways in drug metabolism may be searched for still improved personalization 14 .
a further advantage of the adaptive bayesian approach is that as one looks at historical biomarker data , the approach is independent of the analytical method and the reference interval and therefore is very appropriate for data - mining historical data sets .
data mining using the adaptive bayesian approach should permit one to take clinical issues identified in large datasets and follow the biomarker data back through the phases of development .
if a signal is found in early small populations , the long - term implications of small signals in early phases of development can be more thoroughly understood .
two phase iii clinical trial datasets were used to evaluate traditional methods of renal impairment and an adaptive bayesian approach .
these two datasets were appropriate for this analysis because they present a weak increase in mean urea and creatinine values after drug administration as well as a known drug induced renal impairment .
traditional definition of toxicity limits such as greater than the upper limit of normal , toxicity table ranges , multiples of the reference interval , and greater than 2 or 3 fold changes from the baseline value all suffer from low sensitivity of signal generation and varying probabilities that the threshold will be exceed dependent on the baseline value .
the % change from baseline is better than the fixed limit methods however there is not a single % change from baseline to define a significant signal for all biomarkers and for the same biomarker across subjects .
the significance of a fixed value for the % change from baseline varies depending on the subjects cvi of the biomarker .
the adaptive bayesian approach using all known factors provided superior sensitivity and specificity in these two large clinical trials for drug dependent renal impairment signal generation .
 a major concern with the identification of renal toxicity using the traditional biomarkers , urea and creatinine , is that toxicity signal definitions are not sensitive to medically important changes in these biomarkers . 
 traditional renal signal definitions for urea and creatinine have not adequately identified drugs that have generated important medical issues later in development . here , 
 two clinical trial databases with a posteriori known drug induced renal impairment were analyzed for the presence of a renal impairment biomarker signal from urea ( 590 patients ; age 26 - 92 , median 65 ) and creatinine ( 532 patients ; age 26 - 97 , median 65 ) . 
 data was analyzed retrospectively using multiple definitions for the biomarker signal to include values outside stratified reference intervals , values exceeding twofold increases from baseline , values classified by the 2009 niaid renal toxicity table , change from baseline represented as a z - score based on intra - individual biological variations , and an adaptive bayesian methodology that generalizes population- with individual - based methods for evaluating a biomarker signal . 
 the data demonstrated that the adaptive bayesian methodology generated a prominent drug induced signal for renal impairment at the first visit after drug administration . 
 the signal was directly related to dose and time of drug administration . 
 all other data analysis methods produced none or significantly weaker signals than the adaptive bayesian approach . 
 interestingly , serum creatinine and urea are able to detect early kidney dysfunction when the biomarker signal is personalized . 