disease research in pathology informatics requires archiving , retrieving , organizing , sharing , and analyzing diverse pathology - related data sources .
the most important data source for pathologists is the data collected by anatomic and clinical pathologists ( e.g. , blood tests , surgical pathology reports , autopsy reports , annotated images , and specialized studies ) .
it is a sad irony that the data collected by pathologists is seldom available for serious scientific inquiry .
many pathologists can not freely access the full set of anatomic and clinical data collected within their own departments , though de - identified pathology data is exempted from regulation by hipaa and the common rule .
any pathology department could , without violating federal laws , de - identify and distribute their archived anatomic and clinical pathology datasets to the scientific community . for reasons
economic , legalistic , and psychologic , no pathology departments have , as yet , distributed de - identified collections of their collected datasets to the public . as a result , there is virtually no scientific research currently being conducted on large , multi - institutional collections of pathology data .
moreover , the primary data for research done on collections of single - institution data is seldom released to the public . when research is conducted on pathology data , and
the data is withheld from the public , there is no way to validate the conclusions .
hence , the u.s . national academy of sciences , along with the editors of many scientific journals , have established a policy requiring authors to release the primary data supporting their conclusions . because pathology departments have not released their de - identified research records to the public , record - based pathology informatics research
can not be published in journals that conform to the national academy of sciences recommendations . in the rare instance where an institution has published scientific results based on global analyses of their own datasets , the raw data , upon which those results are based ,
scientific results have no value unless the results are backed by data that can be openly examined . where pathology departments have failed , the u.s .
enormous sources of individualized but de - identified death records and cancer records are available at no cost to medical researchers .
eighty - five million de - identified death records are available from the cdc ( center for disease control and prevention ) .
each record contains basic demographics on the decedent ( age , race , gender , place of death ) , the cause of death , and ( if provided in the death record ) the underlying causes of death and significant additional medical conditions .
each annual data file provides about 2 million byte - indexed sequential - line records .
this means that there is one death record per line , and each line contains coded data indexed for ranges of bytes that are designated by a data dictionary .
for example , the icd-10 code of the underlying cause of death , for deaths occurring in 1999 , are found in bytes 142 - 145 of each record .
this may seem like an awkward way of organizing data , when you consider the ease with which modern specifications ( such as rdf ) can encapsulate data with metadata .
nonetheless , the byte - indexed sequential files can be parsed very efficiently with just a few lines of code .
though pathologists prefer autopsy data over death certificate data , autopsies are performed on a very small fraction of decedents . despite early efforts to standardize and collect autopsy data , pathologists have not succeeded in sharing their autopsy data in a national database .
consequently , death certificates are the most important source of mortality data available to medical researchers .
seer offers a public use dataset , containing de - identified records on about 4 million cancers that have occurred since 1973 .
each seer record is a single cancer case . with about 4 million carefully curated cases
, scientists can draw certain types of inferences that could not possibly be made with the data accumulated at a single medical institution . to get the seer public use data files
, you must first complete a data access request available at : seer.cancer.gov/data/request.html seer sends you a username and password that you will need to access the data files .
each seer record is a line on a data file , and consists of 264 alphanumeric characters .
byte data includes the patient 's race , gender , age at diagnosis , primary tumor site , diagnosis , and information related to tumor size and occurrence of metastases .
a data dictionary provides the byte location of the various field values contained in each record .
in the past several years , i have written hundreds of blog entries explaining how open source materials ( i.e. , data , algorithms , nomenclature , metadata specifications , and software ) can be used to collect , organize , integrate , and analyze pathology - related information .
the blogs are available at : julesberman.blogspot.com a blog tutorial for seer data files appeared on november 14 , 2008 . a blog tutorial on the cdc mortality files appeared on december 2 , 2008 .
all of the 300 + blogs entries for the blog site can be accessed through a linked archive web page at : http://www.julesberman.info/blog_in.htm blog entries related to the cdc and seer data include topics such as data mashups ( e.g. , mapping the geographic locations of disease occurrences ) , age distributions ( e.g. , following the average age of occurrence of diseases that progress through diagnostic categories over time ) , and trends in disease incidence ( e.g. , measuring the annual incidence of genetically screened diseases ) .
there are blog entries of general interest to those working with pathology datasets , including entries on de - identification methods , image manipulation and annotation , data specification methods , and various computational algorithms .
 the day has not arrived when pathology departments freely distribute their collected anatomic and clinical data for research purposes . 
 nonetheless , several valuable public domain data sets are currently available , from the u.s . 
 government . 
 two public data sets of special interest to pathologists are the seer ( the u.s . 
 national cancer institute 's surveillance , epidemiology and end results program ) public use data files , and the cdc ( center for disease control and prevention ) mortality files . 
 the seer files contain about 4 million de - identified cancer records , dating from 1973 . 
 the cdc mortality files contain approximately 85 million de - identified death records , dating from 1968 . 
 this editorial briefly describes both data sources , how they can be obtained , and how they may be used for pathology research . 
PATHOLOGY DATA SOURCES
CDC MORTALITY FILES
SEER DATA FILES
BLOG TUTORIAL