for well over a decade , 
 the cost of in vitro and in vivo screening of absorption , distribution , metabolism , 
 excretion , and toxicity ( adme / tox ) properties of molecules has motivated 
 efforts to develop various in silico methods to efficiently 
 pre - filter candidates for actual physical testing . by relying on very large , internally consistent datasets ,
large 
 pharmaceutical companies have succeeded in developing highly predictive 
 but ultimately proprietary models . at one pharmaceutical company , for example , many 
 of these models ( e.g. , volume of distribution , aqueous kinetic solubility , 
 acid dissociation constant , distribution coefficient , microsomal clearance , 
 cyp3a4 time - dependent inhibition ) as well as other endpoints have achieved such 
 high accuracy that they have essentially put the experimental assays 
 out of business .
it is likely that most large pharmaceutical companies 
 can now perform experimental assays for a small fraction of compounds 
 pre - filtered through the proprietary adme / tox and physicochemical 
 property computational models , thus improving cost efficiency while 
 minimizing in vitro and animal experimentation .
extra - pharma 
 computational efforts have not been so successful , largely because 
 they have , by necessity , drawn upon considerably smaller datasets , 
 in many cases trying to combine information from the literature .
this situation , however , has improved with larger datasets publicly 
 available in pubchem , chembl , cdd , and others , and some drug companies depositing 
 their data ( e.g. , the recently deposited astrazeneca data in chembl ) , 
 which can be useful for model building .
adme / tox properties have 
 been modeled by us and many other groups using an array of machine 
 learning algorithms such as support vector machines , bayesian modeling , gaussian 
 processes , and many others .
a more exhaustive review of the different machine 
 learning approaches is outside the scope of this work .
these combined 
 efforts at adme / tox model building have likely resulted in hundreds 
 of published models which are , unfortunately , inaccessible to anyone 
 but their authors in most cases .
this limited access problem for published 
 models is also likely the case with computational models for bioactivity 
 or other physicochemical properties of interest .
the ability 
 to share such models freely still remains a major challenge when dealing 
 with issues of proprietary samples or data , as repercussions for such 
 for - profit pharmaceutical companies could be severe .
the current development 
 of technologies for open models and descriptors builds on established 
 methodologies .
datasets for quantitative structure  activity relationships 
 ( qsar ) have previously been represented in a reproducible way via 
 qsar - ml .
these methods also come with 
 a reference implementation for the bioclipse workbench , which provides a graphical interface .
there have been several early 
 efforts at cheminformatics web services ; e.g. , indiana university 
 provides access to cheminformatics methods ( fingerprints , 2d depiction , 
 and various molecular descriptors ) and statistical techniques .
in addition , there are web tools for the prediction of bioactivities 
 and physicochemical properties , like the chemistry activity 
 predictor ( gusar ) . also , the open notebook 
 science ( ons ) project has developed models 
 for solubility and melting point using web services based on open 
 descriptors and algorithms .
these tools all enable parties to collaborate 
 publicly but do not facilitate private or selective collaboration .
we have previously demonstrated a proof of concept using open descriptors 
 and modeling tools to model very large adme datasets at pfizer .
models were constructed with open descriptors 
 and keys ( cdk + smarts ) using open software ( c5.0 ) and performed essentially 
 identically to expensive proprietary descriptors and models ( moe2d 
 + smarts + rulequest s cubist ) across all metrics of performance 
 when evaluated on human liver microsomal stability ( hlm ) , rrck passive 
 permeability , p - gp efflux , and aqueous solubility datasets .
pfizer s hlm dataset , used in this study , 
 contained more than 230,000 compounds and covered a diverse range 
 of chemistry space as well as addressing many therapeutic areas .
the 
 hlm dataset was split into a training set ( 80% ) and a test set ( 20% ) 
 using the venetian blind splitting method .
in addition , a newly screened 
 set of 2310 compounds was evaluated as a blind dataset .
all the key 
 metrics of model performance , e.g. , r , rmse , kappa , sensitivity , specificity , and positive predictive 
 value ( ppv ) , were nearly identical for the open source approach vs 
 proprietary software ( e.g. , ppv = 0.80 vs 0.82 ) . our goal is 
 to enable extra - pharma drug discovery projects to exploit in silico machine learning methods that have , until now , 
 been confined in practice to pharma and to a few knowledgeable academics . 

these methods better exploit limited screening resources and will 
 enable such projects to cover more unexplored chemical space and to 
 address adme / tox earlier in the discovery process .
extra - pharma projects 
 represent a growing trend for commercial drug discovery to be the principal efforts to find cures for many neglected diseases 
 ( e.g. , tuberculosis , malaria , chagas disease , visceral leishmaniasis , 
 etc . ) , and thousands of orphan indications will require more collaborations and data , and therefore model sharing . 

this approach has the potential to accelerate the discovery of promising 
 drug - like lead compounds with acceptable properties in vivo and ultimately yield a significant impact on global health .
we now describe the creation of a reference implementation of a 
 bayesian model - building software module , which we have released as 
 an open source component that is now included in the chemistry development 
 kit ( cdk ) project and incorporated using the fcfp6 
 descriptors in the cdd vault , which was also recently made open source .
we make use of the cdd public database , which has over 100 public datasets that can 
 be used to generate community - based models , including extensive neglected 
 infectious disease sar datasets ( malaria , tuberculosis , chagas disease , 
 etc . ) , and admedata.com datasets that are broadly applicable to many 
 projects .
an accompanying paper uses this software to develop models 
 on a much larger scale .
all molecules for malaria , 
 tuberculosis , and cholera datasets from table 1 are available in cdd public ( https://app.collaborativedrug.com/register ) , and the models from table 2 are available 
 from http://molsync.com/bayesian1 .
bayesian models have been 
 a useful part of computer - aided drug 
 discovery for many years and were popularized in pipeline 
 pilot .
the statistical 
 method is particularly useful for correlated structure - derived fingerprint 
 bit strings with an activity measurement that has been classified 
 as active or inactive on the basis 
 of 
 a selected threshold .
variants on the original bayes theorem can be 
 used to produce an estimate of the likelihood of activity for proposed 
 compounds . for reproducing binary classifications , bayesian methods 

the model creation process is typically 
 very fast and can be implemented in o(n ) time , which means that an 
 ordinary desktop computer can build and evaluate models with hundreds 
 of thousands of compounds with minimal delay .
when general purpose 
 structure - derived fingerprints are used , the methods tend to be quite 
 robust , which is in contrast to methods such as qsar , which require 
 some expertise to select appropriate descriptors , avoid overtraining , 
 and ensure domain applicability . unlike many other applications 
 of bayesian methods , the use of chemical structure fingerprints as 
 inputs means priors often numbers in the thousands , which produces 
 scale distortions .
even if the prior probabilities are approximately 
 0.5 in each case , multiplying thousands of such values together tends 
 to warp the distribution of the posterior probabilities to being asymptotically 
 close to 0 or 1 , which introduces numerical precision issues . by summing 
 the logarithms of the ratios rather than multiplying the fractions , 
 and incrementing the numerator and denominator ,
the precision issues 
 are eliminated , and the resulting predictions tend to follow a linear 
 distribution .
the main drawback with this particular bayesian 
 variant is that 
 the resulting prediction is not a probability , but rather an arbitrary 
 number that has no particular upper or lower bound .
the results can 
 be converted into a two - state classification by selecting a threshold , 
 or into a probability - like value by picking a linear transformation 
 function , but these are post - bayesian calibrations that must be made 
 by applying judgment criteria that are not an intrinsic part of the 
 method . we have used our work on 
 the reference 
 extended connectivity ecfp and fcfp fingerprints to create a software 
 class that allows bayesian models to be created from a collection 
 of molecules and activity data , used to predict probabilities for 
 new molecules , and to serialize / deserialize the model as a structure 
 text string that can be saved to a file or shared with any other package 
 that implements the same functionality .
the laplacian - modified 
 nave bayesian ( which we call bayesian 
 models for simplicity ) formula uses a simple definition , which pre - supposes 
 that each molecule has been described by enumerating a list of fingerprints 
 that applies to it , and has a determination of whether it is active 
 or inactive . for each fingerprint code in the entire dataset : where ci is the contribution 
 associated with the presence of a fingerprint 
 hash code i , which is in turn derived from ai and ti , which are respectively the number of active molecules with the fingerprint and the total number 
 of molecules with the fingerprint , while r is the overall 
 fraction of actives .
building the bayesian model is a simple 
 matter of determining the 
 total set of fingerprints in the dataset and , for each of them , calculating 
 the value of ci .
any fingerprint 
 that is theoretically possible but not encountered in the training 
 set has an implied value of 0 .
any fingerprint hash code that is observed 
 equally often in active and inactive molecules ( or not at all in either ) 
 has a ratio of 1 , for which the log value is zero . when making 
 a prediction for an incoming molecule , the value is 
 determined by adding up the contributions for each fingerprint hash 
 code for the molecule : the resulting prediction , pm
, is an uncalibrated value : unlike for the 
 conventional bayes theorem , the result is not a probability and is 
 generally not directly interpretable , meaning that there is no significance 
 to either the scale or offset .
creating a bayesian model using 
 this method is very fast and has 
 favorable scalability properties , because it requires just two passes 
 through the input collection : the total number of actives and inactives 
 needs to be summed , and after that , each compound needs to be considered 
 only individually .
the total memory required to build the bayesian 
 model is bounded by the theoretical number of fingerprints .
for each 
 possible unique fingerprint hash code , it is necessary to store two 
 integers ( ai , ti ) and derive one floating point value 
 per fingerprint ( ci ) . for 
 small , relatively dense fingerprint schemes ,
these can be stored in 
 a flat array ( e.g. , when folding fingerprints into 1024 possible values ) , 
 but for larger schemes with sparse occupancy it is better to use a 
 dictionary object ( e.g. , when the full 32-bit range of ecfp6 or fcfp6 
 fingerprints is allowed ) .
the pseudocode for the model building 
 is as follows : let t = empty dictionary ( key : hash code i , value : 
 total t ) let a = empty dictionary ( key : hash code i , value : 
 actives a)for m in all molecules in training 
 set : determine 
 list of fingerprints f for molecule mfor fingerprint hash code i in f : increment tiif molecule m is active : increment ailet r = total actives / total moleculeslet c = empty dictionary ( key : hash code i , value : 
 contribution v)let l = unique list of keys for tfor i in l : put ci = log([ai + 1]/[tir + 1])for making 
 a prediction for an incoming molecule : let m = molecular 
 structuredetermine list of fingerprints f for molecule mlet pm = 0for i in l : if i is one of the fingerprints in f : let pm = pm + ci let t = empty dictionary ( key : hash code i , value : 
 total t )   let a = empty dictionary ( key : hash code i , value : 
 actives a ) for m in all molecules in training 
 set : determine 
 list of fingerprints f for molecule mfor fingerprint hash code i in f : increment tiif molecule m is active : increment ai determine 
 list of fingerprints f for molecule m for fingerprint hash code i in f : increment tiif molecule m is active : increment ai if molecule m is active : increment ai let r = total actives / total molecules let c = empty dictionary ( key : hash code i , value : 
 contribution v ) let l = unique list of keys for t for i in l : put ci = log([ai + 1]/[tir + 1 ] ) put ci = log([ai + 1]/[tir + 1 ] ) let m = molecular 
 structure determine list of fingerprints f for molecule m if i is one of the fingerprints in f : implementing these algorithms using a flat 
 array rather than a 
 dictionary object is analogous and differs only in the way indices 
 are looked up .
the method 
 described in this 
 article is implemented in the chemistry development kit ( cdk ) project 
 and made available under the terms of the lesser gnu public license 
 ( lgpl ) .
the latest version of the project can be obtained from its 
 sourceforge host and underlying git repository ( http://sourceforge.net/p/cdk/code/ci/master/tree ) .
the bayesian modeling capabilities are available within the tools section , the main class for which is org.openscience.cdk.fingerprint.model.bayesian . using the cdk library to create a new bayesian model from 
 a collection of molecule objects and boolean activity values
for example , given the filename for an mdl sdfile with a field called 
  pic50  , for which any molecule with a value of 6 or 
 greater is considered active , the following java code snippet can 
 be used to create a serialized model : the resulting serialized form can be stored for future use . 

if 
 it is stored in a file , it can be easily retrieved and used to apply 
 to a different sdfile , e.g. : the first step is to 
 read the model from the pre - existing file 
 (  activity.bayesian  ) .
the second step iterates over 
 the input sdfile , while writing to another sdfile with two extra fields 
 appended :  rawprediction  , which contains the uncalibrated 
 outcome from the modified bayesian method , and 
scaledprediction  , 
 which contains the prediction that has been scaled using metrics originally 
 derived from the internal cross - validation .
these two examples 
 demonstrate the  create and consume 
 use cases and can be easily adapted to scenarios besides reading and 
 writing from files .
serialized bayesian models can be embedded in 
 any kind of text - friendly data structure , e.g. , xml documents , json 
 messages , sql tables , etc .
use of models to provide predictions can 
 be applied to a variety of invocations , such as command line tools , 
 incorporation into modeling packages with a graphical interface , web 
 services accessible via api , etc . for 
 saving models for subsequent reuse , 

the information necessary to apply the model to make predictions for 
 new molecules can be stored in a text - based file format .
the molecules 
 that were used to build the model are not included 
 in the serialized form , nor are the fingerprints that were generated 
 from them .
this means that sharing a serialized model allows the recipient 
 to make inferences on the basis of the original data without explicitly 
 having access to it . for confidentiality purposes , sharing models 
 without the underlying data
is useful in a number of situations , but 
 it should be noted that this can not be considered as entirely foolproof : 
 a determined hacker with some context would likely be able to make 
 a well educated guess as to the actives contained in the training 
 set .
the default file extension is .bayesian , and 
 the mime type is chemical / x - bayesian .
the text should 
 be encoded as utf-8 unicode , for which all of the content is limited 
 to the ascii subset , except for the freeform text notes .
end of line 
 should be encoded unix - style , and floating point numbers can be encoded 
 with a decimal point ( e.g. , 1.23 , with a period symbol for the separator , 
 invariant of localization ) or scientific notation ( e.g. , 1.23  
 10 ) . the format is case- and whitespace - sensitive . 

the body of the format consists of individual lines , each of which 
 encodes a discrete property , and is of arbitrary length .
the default file extension is .bayesian , and the 
 mime type is chemical / x - bayesian .
the first line contains the header , which consists 
 of the recognition 
 sequence and essential information about the model .
the first nine 
 characters are always set to the ascii characters for the string  bayesian !  
 ( hex : 42 61 79 65 73 69 61 6e 21 ) , which can be used as a recognition 
 sequence .
this is useful for situations such as embedding in streams , 
 within whitespace - padded subfields such as xml elements , or when the 
 file extension or mime type is unavailable or unreliable .
the 
 recognition sequence is followed by four comma - separated fields : fingerprint type , folding length , calibration minimum , and maximum .
only 
 the first two fields are mandatory , and parsers should ignore additional 
 fields , in case the format is subsequently extended
. the fingerprint type must be one of ecfpn or 
 fcfpn , where n is 
 0 , 2 , 4 , or 6 .
these correspond to the eight different permutations 
 of circular fingerprints that are implemented in the cdk library . 

when a parser encounters 
 a fingerprint type that it does not recognize , it should invoke an 
 error pathway if there is any intention of applying the model to new 
 molecules , since the ability to produce the exact same fingerprints 
 is a pre - requisite .
the folding length should either 
 be 0 ( no folding , i.e. , full range of 32-bit integers ) or a power 
 of 2 ( e.g. , 512 , 1024 , 2048 , etc . ) .
the parser should fail for invalid 
 folding lengths . the calibration minimum and maximum values
this is calculated by analyzing the 
 cross - validation metrics , which is an optional step , the information 
 may not be available . if not included , then the model can only be 
 used to generate raw prediction values .
note that sometimes the minimum 
 and maximum values are equal , which can occur for datasets that are 
 small or trivial . in this case , the degenerate value should be treated 
 like a simple threshold , giving results of 0 or 1 , rather than a probability - like 
 transform .
the model specification ends with a line beginning 
 with the string 
  ! end  .
this should be considered as the terminator 
 sequence regardless of trailing characters or whitespace .
all lines 
 in between the header and footer can be examined out of order .
lines that match the template { bit index}={contribution } , where bit index is 
 an integer and contribution is a floating point number , 
 make up the payload of the model . the contributions - per - bit are typically 
 stored in a dictionary object , since the bit coverage is sparse , i.e. 
 usually not all of the possible bits are represented .
if the fingerprints 
 are folded , then the bit indices range from 0 to folding-1 .
if not 
 folded , the indices are represented as signed integers : 
 approximately half of the values will be negative .
for generating 
 raw bayesian predictions , all that is needed is 
 fingerprint type , folding , and contribution list .
all remaining lines 
 are optional and must follow the general format of { category}:{key}={value } , whereby category and key must 
 be plain ascii without whitespace , category must 
 begin with an alphanumeric character , and value may 
 contain any unicode characters , except for end - of - line .
when software needs to parse , modify then write a model 
 file , any optional lines that are not understood should be preserved 
 as - is .
the optional data that are currently used by the cdk 
 implementation 
 include the following:training : size and training : actives : the 
 total number of compounds 
 in the training set , and the number of actives , respectivelyroc : auc : integral 
 of the receiver operator characteristic , which is a number between 
 0 and 1roc : type : the 
 method used to partition the data for internal cross - validation , which 
 is one of leave - one - out , three - fold , or five-fold.roc:x and roc : y : two comma - separated lists of numbers from 0 to 1 
 which can be used to recreate the roc curve visually .
note that for 
 large datasets , the total number of points may be reduced in order 
 to limit the impact on file size .
this means that while the resolution 
 is indistinguishable for graph plotting purposes , recalculating the 
 integral from these points is less precise than using the stored roc : auc value.note:title : 
 ideally a short free - text description of the model that communicates 
 to a scientist what data were being modeled .
it should be expected 
 to be displayed in a single line.note:origin : 
 a short free - text description that provides information about the 
 provider of the model , be it the software algorithm or the source 
 of the data , or both.note:comment : a completely freeform field , which 
 may be of any length .
since 
 newlines are disallowed , multiple paragraphs of comments should be 
 encoded by having multiple comment lines .
training : size and training : actives : the 
 total number of compounds 
 in the training set , and the number of actives , respectively roc : auc : integral 
 of the receiver operator characteristic , which is a number between 
 0 and 1 roc : type : the 
 method used to partition the data for internal cross - validation , which 
 is one of leave - one - out , three - fold , or five - fold .
roc : x and roc : y : two comma - separated lists of numbers from 0 to 1 
 which can be used to recreate the roc curve visually .
note that for 
 large datasets , the total number of points may be reduced in order 
 to limit the impact on file size .
this means that while the resolution 
 is indistinguishable for graph plotting purposes , recalculating the 
 integral from these points is less precise than using the stored roc : auc value .
note : title : 
 ideally a short free - text description of the model that communicates 
 to a scientist what data were being modeled .
note : origin : 
 a short free - text description that provides information about the 
 provider of the model , be it the software algorithm or the source 
 of the data , or both . note : comment : a completely freeform field , which 
 may be of any length .
since 
 newlines are disallowed , multiple paragraphs of comments should be 
 encoded by having multiple comment lines .
the file size for a serialized model depends on the size and diversity 
 of the molecules .
one of the main reasons for opting to fold the fingerprints is that it places a reasonable maximum limit on 
 the file size .
for example , a collection of 7000 molecules with experimentally 
 determined activity against mycobacterium tuberculosis using ecfp6 with no folding produced a file size of 1.4 mb .
folding 
 the fingerprints into 32,768 bits reduced the file size to 646 kb , 
 and into 2048 down to 67 kb .
the cdd vault product makes use of 
 the fingerprinting functionality in the cdk 
 to provide bayesian model - building capabilities , which we have termed 
 cdd models . while the bayesian implementation is proprietary , the 
 underlying algorithm for model generation is equivalent to the method 
 described in this article
. models can be created and used within the 
 cdd vault environment , and at any time they can be exported using 
 the format described above , which means that they can be utilized 
 by any software that either implements the requisite algorithms described 
 in this article or makes use of the cdk library .
a model is created by 
 separating a set of molecules into two collections : those that could 
 be considered 
inactives. these classes are then used to train the 
 model , after a series of steps are taken to ensure logical consistency . 

these include ensuring that duplicates do not appear in either collection , 
 that there is no overlap between the collections , and that each collection 
 contains at least two molecules .
the standard inchikey of each molecule is used as the criteria for 
 detecting and removing duplicates .
these precautions are addressed 
 in a series of pre - processing steps in the cdd ruby on rails application , 
 where the modeling process and molecule management system is hosted , 
 wherein the training sets are algorithmically curated via optimized 
 raw sql code .
once the training set has passed these checks 
 and pre - processing , 
 the model is generated .
this 
 machine learning model is stored as a special type of protocol ( category 
 = machine - learning model ) , which provides an roc plot generated by 
 stratified three - fold cross - validation .
this roc plot is interactive , 
 allowing the user to explore the sensitivity , specificity , and corresponding 
 score cutoff at each point along the curve ( figure 2a ) .
after the model has been created , each molecule in the 
 user s selected 
project receives a relative 
 score , applicability number ( fraction of structural features shared 
 with the training set ) , and maximum similarity number ( maximum tanimoto / jaccard 
 similarity to any of the  good  molecules ) .
the model 
 can be subsequently shared with both other users and the user s 
 other  projects to score any molecule of interest .
the model can also be exported from cdd vault by making use of 
 the aforementioned .bayesian file format ( figure 2b ) . to render a serialized version of a model , cdd vault feeds 
 the training set structures into the serialization implementation 
 described in the previous section .
the connection between the ruby 
 code in cdd vault and the java - based serialization code is accomplished 
 using rjb ( ruby - java bridge ) .
( a ) model derived from whole - cell 
 datasets from antimalarial 
 screening across four cdd public datasets ( mmv , st .
jude , novartis , 
 and tcams ) , 20,000 ec50 values , cutoff < 10 
 nm .
once the bayesian model building was formalized 
 as part of the cdk project , with a rigorously defined file format , 
 it became a straightforward matter to implement the algorithm on other 
 platforms .
we have previously described the implementation of ecfp 
 and fcfp fingerprints in a way that is agnostic to the specifics of 
 any particular cheminformatics toolkit and so can be easily ported 
 to other platforms , such that it is literally compatible 
 with the original reference .
we have taken the same approach with 
 the bayesian modeling and ported enough of the functionality to the 
 ios mobile platform such that models created with the cdk or cdd vault using ecfp6 fingerprints can be parsed from within mobile 
 apps and used to make predictions .
currently bayesian model prediction 
 capabilities have been incorporated into the mobile molecular datasheet 
 ( mmds ) app ( figure 3 ) , approved drugs , and 
 molprime+ ( all apps produced by molecular materials informatics ) . 

several useful bayesian models have been packaged with the apps as 
 default functionality , and it is also possible to import user - created 
 models in order to make structure - based predictions within the mobile 
 app .
( c ) results scored with this 
 model ( herg measured ic50 = 24 nm ) showing a visually intuitive 
 atom coloring for this and other bayesian models .
this compound would 
 appear to be an inhibitor of herg and possibly kcnq1 potassium channels . to illustrate the utility of 
 cdd models we evaluated several datasets available in cdd public as 
 well as in our own cdd vaults ( table 1 ) .
these include screening datasets for malaria , 
 tuberculosis , and cholera from whole - cell screens , in vivo data from mice treated with potential antituberculars , as well as 
 several adme / tox properties such as ames mutagenicity , mouse and human 
 intrinsic clearance , caco-2 , 5-ht2b , solubility , pxr activation , 
 maximum recommended therapeutic dose , and blood brain barrier permeability 
 data . in all cases ,
several 
 datasets were also selected for integration into mobile apps ( including 
 mmds and approved drugs ) .
these datasets included solubility , probe - like , herg , kcnq1 , screening data for whole - cell 
 phenotypic screens against bubonic plague as well as chagas disease . 
 in all cases ,
five - fold roc data were collated , summarized , and compared 
 to published results .
as mentioned previously , 
 the reference implementation for the bayesian algorithm and the underlying 
 ecfp / fcfp fingerprints is available in the cdk library , which can 
 be freely downloaded from github .
the open source implementation is 
 also accompanied by a testing library , which runs a battery of tests 
 to ensure that the basic functionality is operating as - described .
cdd models using fcfp6 fingerprints 
 have been demonstrated using diverse datasets , such as from public 
 phenotypic screening , and published adme / tox datasets ( table 1 ) . in most cases ,
we have also illustrated with the tuberculosis 
 ( m. tuberculosis ) and malaria ( p. falciparum ) screening datasets that very large dose 
response or single - point 
 datasets can be constructed by combining datasets in cdd public .
other 
 datasets were collected for this study by manual mining of chembl 
 ( mouse intrinsic clearance , human intrinsic clearance , caco-2 ) .
although 
 these datasets are generated from many different published datasets , 
 the roc values are a good starting point ( > 0.80 ) and comparable 
 to 
 those obtained from proprietary datasets .
several of these datasets 
 ( blood brain barrier permeability and pxr ) were used recently for 
 a comparison across svm and bayesian methods , and the three - fold cross - validation roc values were similar 
 to those obtained with five - fold cross - validation in this study . the 
 use of the roc value in this way is a reasonable method to evaluate 
 the utility of the computational models .
however , ideally the use 
 of an additional external test set would provide further confidence . 

the roc values for the m. tuberculosis models are 
 comparable to those published recently using a commercial tool . for 
 example , in this study mlsmr single - point model three - fold roc = 0.87 
 ( figure s1 , five - fold roc 0.87 , leave out 50%  100
response 
 model three - fold cross - validation roc = 0.75 ( leave out 50%  
 100 cross - validation roc = 0.73 ) , m. tuberculosis efficacy in mouse three - fold roc = 0.73 
 ( five - fold roc = 0.73 ) , and ames mutagenicity 
 three - fold roc = 0.83 ( five - fold roc = 0.84 ) . the models developed using 
 the same underlying code in mobile apps used the ecfp6 descriptors , 
 and all eight models described had five - fold roc values > 0.75 ( table 2 ) .
several of these datasets 
 have previously been used to generate svm and bayesian methods with 
 fcfp6 descriptors using other software . 

for example , the three - fold roc for the probe - like dataset in this 
 study was 0.76 ( five - fold roc = 0.73 ) , the three - fold roc for the herg dataset was 0.85 ( five - fold roc 
 = 0.84 ) , and the three - fold roc for 
 the kcnq1 dataset was 0.84 ( five - fold roc = 0.86 ) .
the models derived with fcfp6 ( table 1 ) and ecfp6 descriptors ( table 2 ) can 
 be compared ; e.g. , the three - fold roc for the malaria dataset using 
 fcfp6 was 0.97 ( table 1 ) ( using ecfp6 for the 
 same dataset five - fold roc = 0.98 , table 2 ) .
these examples of models generated previously and now with open 
 source descriptors and algorithms suggest they are likely comparable 
 ( based on roc values ) and will be evaluated prospectively in future 
 studies .
we have also made the models in the mobile app freely accessible 
 via the link http://molsync.com/bayesian1 , which is summarized 
 in figure 4 .
screenshots summarizing the roc plots 
 and active and inactive compounds 
 for eight models implemented in mmds .
we have recently suggested how providing 
 computational models tightly 
 integrated in software used for storing and sharing chemistry and 
 biology data will be useful for decision making .
some resources exist such as qsardb.org and ochem.eu for 
 public model sharing and development , while another , 
 chembench , provides a resource for creating and using models and other 
 cheminformatics tools privately .
our 
 work , proposing that open source descriptors and algorithms are comparable 
 to commercial software in performance , will ideally lead to more sharing of computational models . at approximately 
 the same time , qsar - ml was developed 
 to enable standards for interoperability of qsar models .
we now build on this prior work as the current study sets the stage 
 for being able to generate a model in proprietary software such as 
 cdd vault and export a model in a format that could be run in open 
 source software using cdk components .
this is a significant advance , 
 because it means that a shared bayesian model can in principle be 
 used by anyone , regardless of which commercial software packages they 
 have licenses to , since the model capabilities are implemented by 
 an open source toolkit that runs on essentially every desktop platform 
 ( cdk is written in cross - platform java ) .
the creation of additional 
 products that implement the same identical reference algorithm , e.g. , 
 mobile apps , makes use of shared models increasingly convenient .
none of the 
 existing web sites for creating or storing qsar models appear to offer 
 this capability . in terms of willingness to share models , sharing 
 with collaborators 

is one thing , while sharing models openly with the community at large 
 is another , but we have at least removed the main technology hurdle 
 for fingerprint - based bayesian models in this study .
as previously 
 noted , the shared models do not contain chemical structures or the 
 fingerprints corresponding directly to them .
however , the direct correlation 
 between structural features and fingerprint does provide clues as 
 to what active molecules an organization may have been using to build 
 their models , and so this caveat must be taken into account when trustworthiness 
 can not be assumed . while additional security measures are appropriate 
 for the world of proprietary high - value disease targets , this is much 
 less of an issue for rare or neglected diseases , which is where we 
 believe that open model sharing will have the greatest impact .
there has been considerable research and discussion 
 on efforts to securely share chemistry data , and some of these approaches could be implemented to encrypt models 
 in future .
we have now described how implementation of bayesian 
 models with 
 fcfp6 descriptors generated in the cdd vault enables the rapid creation 
 of machine learning models from public datasets or the user s 
 own proprietary data .
we also enable the resultant models to be selectively 
 shared ( or not ) within cdd without having to disclose the underlying 
 data  this represents a practical middle ground , where a trusted 
 broker ( cdd ) allows a research group to share some of the benefits 
 of their results , but not necessarily full access to the raw data , 
 nor sufficient detail to reverse engineer it . since sharing is not 
 mandatory , and the option exists to export a model in an open format 
 that can be used by anyone , this means that the full spectrum of model 
 sharing options is available .
providing researchers with greater flexibility 
 to designate and share models with specified collaborators , over particular 
 time intervals , and with clear rights encourages data exchange by 
 allowing researchers to share on terms they control .
more fine - grained 
 access control will expand the boundary of what models can be shared 
 to fit the comfort levels of scientists ( and their management and 
 lawyers ) .
from having been involved in a number of collaborations 
 large and small , we have observed considerable variation in the need 
 for security and desired degree of openness .
the possibility 
 of using such models to further drug discovery 
 for neglected diseases is of considerable interest , since the available 
 software has traditionally catered to the proprietary market that 
 provides most of the funding .
there is now a significant amount of 
 sar data for rare and neglected diseases that is publicly available , 
 and following up on open data with open source modeling algorithms 
 is an important step .
for example , pharmaceutical companies and other 
 research groups have performed high - throughput screens on likely millions 
 of compounds in the search for antimalarials , but they have generally 
 only offered up the active compounds , some of which are available 
 in cdd public . by selecting a cut - off for activity for the antimalarial 
 data that is very stringent ( e.g. , < 10 nm ) in cdd models one can 
 construct a bayesian model with a three - fold roc = 0.97 upon combining 
 four public datasets ( table 1 ) .
this model 
 may be useful for virtual screening of future compound libraries and 
 complements our other efforts at machine learning models for antimalarial 
 research .
ideally having access to the 
 millions of other inactive compounds would also be useful , although 
 one could imagine a company could just make a model available by selecting 
 a cut - off for inactives as we have demonstrated herein .
any 
 efficiencies that can be gained in drug discovery would be 
 highly desirable as it is widely known it is both time - consuming and 
 very costly .
therefore , the use of tools like 
 computational models that can point out drug candidate liabilities 
 earlier will have considerable value . with a considerable percentage of drug failures attributed 
 to adme / tox issues
, it is still important to assess 
 these qualities early in the drug development process .
running experimental 
 adme / tox assays on each compound for initial screening of chemical 
 libraries is cost- and labor - intensive , while computational 
 approaches that rapidly and reliably predict these qualities are gaining 
 more acceptance in the drug discovery community .
it is therefore possible 
 to exclude compounds that are most likely to exhibit undesirable adme 
 or toxicity problems sooner .
we present an approach to drug discovery 
 using computational methods for predicting whole - cell activity as 
 well as adme / tox and physicochemical properties that can be 
 broadly applied and do not have to be restricted to large companies 
 with sophisticated software and big budgets .
for example , modeling 
 of microsomal metabolism has been used with large datasets , and such models are now more 
 accessible through availability of public data .
the results summarized 
 in tables 1 and 2 suggest 
 that reliable bayesian models for various bioactivity and adme / tox 
 models can be generated with simple fingerprint descriptors ( fcfp6 
 and ecfp6 ) and the same bayesian algorithm .
this is enabled in such 
 a way that experience in building computational models , while valuable , 
 may not be essential to facilitate model generation , compound scoring , 
 and interpretation . the models described in table 2 which are 
 available in mmds are now also freely accessible ( http://molsync.com/bayesian1 ) .
our main motivation for creating and disseminating this work is 
 to enable the sharing of bayesian models between a diverse set of 
 toolkits and computing platforms .
we have previously described our 
 open source implementation of ecfp6 and fcfp6 fingerprints , inspired by the original commercial implementation 
 that was partially reported in the literature without the disclosure 
 of key details , which remain a trade secret . while there are several other examples of the general approach , 
 our intent was to create a reference implementation and document it 
 so that identical results could be reproduced .
the 
 algorithm herein is explicitly documented in a stepwise fashion , and 
 the reference method is available publicly in source code , and hence 
 can be used to compare against when re - implementing in another environment . 

we believe that taking such care to ensure that the algorithms can 
 be implemented in a way that is 100% compatible with the formal reference 
 removes a major barrier to scientific progress , since building and 
 using models is no longer an isolated activity .
we have deliberately 
 taken a two - prong approach : by releasing a fully functional implementation 
 as part of a popular open source toolkit , and also taking the effort 
 to document the algorithm in fine grained detail , to encourage creators 
 of commercial software to consider the advantages of interoperability 
 within their own proprietary products . because the source code 
 is a part of the cdk
, the modeling functionality 
 that we describe can be used in a variety of scenarios as - is . any 
 software environment that is capable of linking to a java virtual 
 machine ( either directly or through a pipe ) can make use of this functionality . 
 since the cdk is made available under the lgpl license
, it can be 
 incorporated into proprietary products as long as it is linked as 
 a separate library , but for internal projects , back - end services for 
 which the software is not distributed , or open source projects with 
 a compatible license , it can be used essentially without restrictions . 
 for
wholly closed - source products , and platforms that are not compatible 
 with the java virtual machine , the methodology can be re - implemented 
 without difficulty .
the exact implementation of bayesian model building 
 and subsequent calibration is straightforward , and we have represented 
 it in pseudocode form ( see the accompanying paper for details of algorithms 
 for additional analysis ) .
the cdk version 
 is readily available to verify literal compatibility and can be used 
 as a limitless source of validation data for direct comparison .
thus 
 far , the method has been ported to objective - c , in order to enable 
 the use of bayesian models within several different mobile apps ( figure 3 ) and cdd vault as cdd models .
the use of cdd models 
 online in the cdd vault data sharing platform to create bayesian models , 
 the use of mobile apps to apply them to small collections of proposed 
 compounds , and integration into other products and scripts via the 
 cdk library present a number of opportunities for making computational 
 modeling potentially more useful and widespread . currently structure 
activity 
 models are generally only able to be created and used by one specific 
 platform , or if they have some portability , they often suffer from 
 serious compatibility issues due to differences in the underlying 
 technology ( e.g. , aromaticity models , ylide representations , smarts 
 implementations , partial charge models , etc . ) by releasing a well - documented 
 reference implementation as open source and building powerful and 
 useful functionality on top of it , we hope to encourage computational 
 chemists and software creators to make use of this increased inter - operability .
future work related to this project will include the implementation 
 of further measures to assess model quality and the applicability of a model to a test compound . in the accompanying paper ,
we describe several additional algorithms , 
 including calibration of raw bayesian results to a probability - like 
 scale , the effects of folding fingerprints into a smaller range , methods 
 for extracting suitable validation test sets from large public datasets , 
 automated determination of thresholds for active / inactive , and the 
 impact of training set selection on internal cross - validation metrics . 

as others begin to use the new cdk functionality , cdd models , and 
 bayesian functionality implemented in various mobile apps , we will 
 expect to see further prospective and retrospective testing of the 
 underlying technology and descriptions of the utility and limitations .
 on the order of hundreds of absorption , 
 distribution , metabolism , 
 excretion , and toxicity ( adme / tox ) models have been described in the 
 literature in the past decade which are more often than not inaccessible 
 to anyone but their authors . 
 public accessibility is also an issue 
 with computational models for bioactivity , and the ability to share 
 such models still remains a major challenge limiting drug discovery . 
 
 we describe the creation of a reference implementation of a bayesian 
 model - building software module , which we have released as an open 
 source component that is now included in the chemistry development 
 kit ( cdk ) project , as well as implemented in the cdd vault and 
 in several mobile apps . 
 we use this implementation to build an array 
 of bayesian models for adme / tox , in vitro and in vivo bioactivity , and other physicochemical properties . 
 
 we show that these models possess cross - validation receiver operator 
 curve values comparable to those generated previously in prior publications 
 using alternative tools . 
 we have now described how the implementation 
 of bayesian models with fcfp6 descriptors generated in the cdd vault 
 enables the rapid production of robust machine learning models from 
 public data or the user s own datasets . 
 the current study sets 
 the stage for generating models in proprietary software ( such as cdd ) 
 and exporting these models in a format that could be run in open source 
 software using cdk components . 
 this work also demonstrates that we 
 can enable biocomputation across distributed private or public datasets 
 to enhance drug discovery . 
Introduction
Experimental
Section
Results
Discussion