eye diseases such as diabetic retinopathy ( dr ) and diabetic macular edema ( dme ) are the most common causes of irreversible vision loss in individuals with diabetes .
just in united states alone , health care and associated costs related to eye diseases are estimated at almost $ 500  m . moreover ,
the prevalent cases of dr are expected to grow exponentially affecting over 300  m people worldwide by 2025 . given
this scenario , early detection and treatment of dr and dme play a major role in preventing adverse effects such as blindness .
dme is characterized as an increase in retinal thickness within 1-disk diameter of the fovea center with or without hard exudates and sometimes associated with cysts .
fundus images which have proven to be very useful in revealing most of the eye pathologies [ 4 , 5 ] are not as good as oct images which provide information about cross - sectional retinal morphology .
many of the previous works on oct image analysis have focused on the problem of retinal layers segmentation , which is a necessary step for retinal thickness measurements [ 7 , 8 ] .
however , few have addressed the specific problem of dme and its associated features detection from oct images .  
proposed a classification method to distinguish dme , age - related macular degeneration ( amd ) , and normal sd - oct volumes .
the oct images are preprocessed by reducing the speckle noise by enhancing the sparsity in a transform - domain and flattening the retinal curvature to reduce the interpatient variations .
then , histograms of oriented gradients ( hog ) are extracted for each slice of a volume and linear support vector machine ( svm ) is used for classification . on a dataset of 45 patients equally subdivided into the three aforementioned classes , this method leads to a correct classification rate of 100% , 100% , and 86.67% for normal , dme , and amd patients , respectively .
the images that have been used in their paper are publicly available but are already preprocessed ( i.e. , denoised ) , have different sizes for the oct volumes , and do not offer a huge variability in terms of dme lesions , and some of them , without specifying which , have been excluded for the training phase ; all these reasons prevent us from using this dataset to benchmark our work .
proposed a method for oct images classification using the bag - of - words ( bow ) model .
the method starts with the detection and selection of key points in each individual b - scan , by keeping the most salient points corresponding to the top 3% of the vertical gradient values .
then , a texton of size 9  9 pixels is extracted around each key point , and principal component analysis ( pca ) is applied to reduce the dimension of every texton to get a feature vector of size 9 .
all extracted feature vectors are used to create a codebook using k - means clustering .
then , each oct volume is represented in terms of this codebook and is characterized as a histogram that captures the codebook occurrences .
these histograms are used as feature vector to train a random forest ( rf ) with a maximum of 100 trees .
the method was used to classify oct volumes between amd and normal cases and achieved an area under the curve ( auc ) of 0.984 with a dataset of 384 oct volumes .
liu et al . proposed a methodology for detecting macular pathology in oct images using lbp and gradient information as attributes .
the method starts by aligning and flattening the images and creating a 3-level multiscale spatial pyramid .
the edge and lbp histograms are then extracted from each block of every level of the pyramid .
all the obtained histograms are concatenated into a global descriptor whose dimensions are reduced using pca .
finally a svm with a radial basis function ( rbf ) kernel is used as classifier .
the method achieved good results in detection oct scan containing different pathologies such as dme or amd , with an auc of 0.93 using a dataset of 326 oct scans .
proposed using 2d and 3d lbp features extracted from denoised volumes and dictionary learning using the bow models . in the proposed method
the work described in this paper is an extension of our previous work . in this research , beside the comparison of 2d and 3d features , we explore different possible representations of the features and different preprocessing steps for oct data ( i.e. , aligning , flattening , and denoising ) .
this paper is organized as follows : the proposed framework is explained in section 2 , while the experiments and results are discussed through sections 3 and 4 .
the proposed method , as well as its experimental setup , for oct volume classification is outlined in figure 2 .
first , the oct volumes are preprocessed as presented in detail in section 2.1 . then
, lbp and lbp - top features are detected , mapped , and represented as discussed in depth in sections 2.2 , 2.3 , and 2.4 , respectively .
this section describes the set of preprocessing techniques which aim at enhancing the oct volume .
the influences of these preprocessing methods and their possible combinations are extensively studied in section 3 .
oct images suffer from speckle noise , like other image modalities such as ultrasound ( us ) .
the oct volumes are enhanced by denoising each b - scan ( i.e. , each ( x - z ) slice ) using the nlm , as shown in figure 3 .
nlm has been successfully applied to us images to reduce speckle noise and outperforms other common denoising methods .
nlm filtering preserves fine structures as well as flat zones , by using all the possible self - predictions that the image can provide rather than local or frequency filters such as gaussian , anisotropic , or wiener filters .
however , the oct scans suffer from large type of variations : inclination angles , positioning , and natural curvature of the retina .
therefore , these variations have to be taken into account to ensure a consistent characterization of the tissue disposition , regardless of the location in the retina .
this invariance can be achieved in different manners : ( i ) using a rotation invariant descriptor ( cf .  
this latter correction is known as image flattening which theoretically consists of two distinct steps : ( i ) estimate and fit the curvature of the retinal pigment epithelium ( rpe ) and ( ii ) warp the oct volume such that the rpe becomes flat .
each b - scan is thresholded using otsu 's method followed by a median filtering to detect the different retina layers ( see figures 4(c ) and 4(d ) ) . then
, a morphological closing and opening is applied to fill the holes and the resulting area is fitted using a second - order polynomial ( see figure 4(d ) ) . finally , the scan is warped such that the curve becomes a line as presented in figures 4(e ) and 4(f ) .
thus , in addition to the flattening correction , the warped curves of each b - scan are positioned at the same altitude in the z - axis . in this research , we choose to detect simple and efficient lbp texture features with regard to each oct slice and volume .
lbp is a texture descriptor based on the signs of the differences of a central pixel with respect to its neighboring pixels .
these differences are encoded in terms of binary patterns as follows:(1)lbpp , r=p=0p1sgpgc2p , sx=1if  x00otherwise , where gc , gp are the intensities of the central pixel and a given neighbor pixel , respectively , and p is the number of sampling points in the circle of radius r. ojala et al .
further extended the original lbp formulation to achieve rotation invariance at the expense of limiting the texture description to the notion of circular  uniformity  . referring to the coordinate system defined in figure 3(a ) , the lbp codes are computed on each ( x - z ) slice , leading to a set of lbp maps , a map for each ( x - z ) slice .
volume encoding is later proposed by zhao et al . by computing lbp descriptors in three orthogonal planes , so - called lbp - top .
more precisely , the lbp codes are computed considering the ( x - z ) plane , ( x - y ) plane , and ( y - z ) plane , independently .
thus , three sets of lbp maps are obtained , one for each orthogonal plane . in this work ,
we consider rotation invariant and uniform lbp and lbp - top features with various sampling points ( i.e. , { 8,16,24 } ) with respect to different radius ( i.e. , { 1,2 , 3 } ) .
the number of patterns ( lbp#pat ) in regard to each configuration is reported in table 2 .
the mapping stage is used to partition the previously computed lbp maps ; for this work , two mapping strategies are defined : ( i ) global and ( ii ) local mapping .
global mapping extracts the final descriptors from the 2d feature image for lbp and 3d volume for lbp - top .
therefore , for a volume with d slices , the global - lbp mapping will lead to the extraction of d elements , while the global - lbp - top represents the whole volume as a single element .
the global mapping for 2d images and 3d volume is shown in figures 5(a ) and 5(b ) .
local mapping extracts the final descriptors from a set of ( m  m ) 2d patches for lbp and a set of ( m  m  m ) subvolumes for lbp - top .
given n and n as the total number of 2d patches and 3d subvolumes , respectively , the local - lbp approach provides n  d elements , while local - lbp - top provides n elements .
the texture descriptor of an oct volume is defined as the concatenation of the lbp histograms with the global mapping .
the lbp histograms are extracted from the previously computed lbp maps ( see section 2.2 ) .
therefore , the lbp - top final descriptor is computed through the concatenation of the lbp histograms of the three orthogonal planes with the final size of 3  lbp#pat .
more precisely , an lbp histogram is computed for each set of lbp maps ( x - z ) plane , ( x - y ) plane , and ( y - z ) plane , respectively .
similarly , the lbp descriptor is defined through concatenation of the lbp histograms per each ( x - z ) slice with the final size of d  lbp#pat .
the concatenation of histograms employed in the low - level representation in conjunction with either global or local mapping can lead to a high - dimensional feature space .
for instance , local mapping results in a size of n  d  lbp#pat for the final lbp descriptor and n  lbp#pat for the final lbp - top descriptor , where n and n are the total number of 2d patches and 3d subvolumes , respectively .
high - level representation simplifies this high - dimensional feature space into a more discriminant lower space .
this model represents the features by creating a codebook or visual dictionary , from the set of low - level features .
the set of low - level features are clustered using k - means to create the codebook with k clusters or visual words . after creating the codebook from the training set ,
the final descriptor is a histogram of size k which represents the codebook occurrences for a given mapping .
the last step of our framework consists in the classification of sd - oct volumes as normal or dme . for that matter , five different classifiers
are used : ( i ) k - nearest neighbor ( nn ) , ( ii ) logistic regression ( lr ) , ( iii ) random forest ( rf ) , ( iv ) gradient boosting ( gb ) [ 21 , 22 ] , and ( v ) support vector machines ( svm ) [ 23 , 24 ] .
a set of three experiments is designed to test the influence of the different blocks of the proposed framework in comparison to our previous work .
these experiments are designed as follows : experiment 1 evaluates the effects of number of words used in bow ( high - level representation).experiment 2 evaluates the effects of different preprocessing steps and classifiers on high - level representation.experiment 3 evaluates the effects of different preprocessing steps and classifiers on low - level representation.table 4 reports the experiments which have been carried out in   as a baseline and outlines the complementary experimentation here proposed .
the reminder of this section details the common configuration parameters across the experiments , while the detailed explanations are presented in the following subsections .
experiment 1 evaluates the effects of number of words used in bow ( high - level representation ) .
experiment 2 evaluates the effects of different preprocessing steps and classifiers on high - level representation .
experiment 3 evaluates the effects of different preprocessing steps and classifiers on low - level representation .
all the experiments are performed using a private dataset ( see section 3.1 ) and are reported as presented in section 3.2 . in all the experiments , lbp and lbp - top features
are extracted using both local and global mapping for different sampling points of 8 , 16 , and 24 for radius of 1 , 2 , and 3 pixels , respectively .
the partitioning for local - mapping is set to ( 7  7)-pixel patch for 2d lbp and ( 7  7  7)-pixel subvolume for lbp - top .
this dataset was acquired by the singapore eye research institute ( seri ) , using cirrus ( carl zeiss meditec , inc . ,
the dataset consists of 32 oct volumes ( 16 dme and 16 normal cases ) .
all sd - oct images are read and assessed by trained graders and identified as normal or dme cases based on evaluation of retinal thickening , hard exudates , intraretinal cystoid space formation , and subretinal fluid .
all the experiments are evaluated in terms of sensitivity ( se ) and specificity ( sp ) using the lopo - cv strategy , in line with .
se and sp are statistics driven from the confusion matrix as depicted in figure 6 .
the se evaluates the performance of the classifier with respect to the positive class , while the sp evaluates its performance with respect to negative class .
the use of lopo - cv implies that , at each round , a pair of dme - normal volumes is selected for testing while the remaining volumes are used for training .
however , lopo - cv strategy has been adopted despite this limitation due to the reduced size of the dataset .
this experiment intends to find the optimal number of words and its effect on the different configurations ( i.e. , preprocessing and feature representation ) , on the contrary to , where the codebook size was arbitrarily set to k = 32 .
several preprocessing strategies are used : ( i ) nlm , ( ii ) a combination of nlm and flattening ( nlm+f ) , and ( iii ) a combination of nlm , flattening , and aligning ( nlm+f+a ) .
volumes are represented using bow , where the codebook size ranges within k  { 10,20,30 ,  , 100,200 ,  , 500 , 1000}. finally , the volumes are classified using lr .
the choice of this linear classifier avoids the case that the results get boosted by the classifier . in this manner
, any improvement would be linked to the preprocessing and the size of the codebook .
the usual build of the codebook consists of clustering the samples in the feature space using k - means ( see section 2.4 ) . however , this operation is rather computationally expensive and the convergence of the k - means algorithm for all codebook sizes is not granted .
nonetheless , nowak et al .   pointed out that randomly generated codebooks can be used at the expense of accuracy .
thus , the codebook is randomly generated since the final aim is to assess the influence of the codebook size and not the performance of the framework . for this experiment ,
the codebook building is carried out using random initialization using k - means++ algorithm , which is usually used as a k - means initialization algorithm .
for this experiment , se and sp are complemented with acc and f1 score ( see ( 2 ) ) .
acc offers an overall sense of the classifier performance , and f1 illustrates the trade - off between se and precision . precision or positive predictive value is a measure of algorithm exactness and is defined as a ratio of true positive over the total predicted positive samples:(2)acc = tp+tntp+tn+fp+fn , f1=2tp2tp+fp+fn.table 6 in appendix shows the results obtained for the optimal dictionary size while the complete set of all acc and f1 graphics can be found at .
according to the obtained results , it is observed that the optimum number of words is smaller for local - lbp features in comparison to local - lbp - top and global - lbp , respectively .
using lr classifier , the best performances were achieved using local - lbp with 70 words ( se and sp of 75.0% ) and local - lbp - top with 500 words ( se and sp of 75.0% as well ) .
this experiment explores the improvement associated with ( i ) different preprocessing methods and ( ii ) using larger range of classifiers ( i.e. , linear and nonlinear ) on the high - level representation .
all the preprocessing stages are evaluated ( nlm , nlm+f , and nlm+f+a ) . in this experiment ,
the codebooks for the bow representation of lbp and lbp - top features are computed using regular k - means algorithm which is initialized using k - means++ , where k is chosen according to the findings of experiment 1 .
finally , the volumes are classified using k - nn , rf , gb , and svm .
the k - nn classifier is used in conjunction with the 3 nearest neighbors rule to classify the test set .
the rf and gb classifiers are trained using 100 unpruned trees , while svm classifier is trained using an rbf kernel and its parameters c and  are optimized through grid - search .
complete list of the obtained results from this experiment is shown in table 7 in appendix . despite the fact that highest performances are achieved
when nlm+f or nlm+f+a is used , most configurations decline when applied with extra preprocessing stages .
this experiment replicates experiment 2 for the case of low - level representation of lbp and lbp - top features extracted using global mapping .
the obtained results from this experiment are listed in table 8 in appendix . in this experiment , flattening the b - scan boosts the results of the best performing configuration . however , its effects is not consistent across all the configurations .
rf has a better performance by achieving better se ( 81.2% , 75.0% , and 68.7% ) , while svm achieves the highest sp ( 93.7% ) , see table 8 in the appendix . in terms of classifier
, rf has a better performance than the others despite the fact that the highest sp is achieved using svm .

 table 5 combines the obtained results from section 3 with those reported by lemaitre et al . , while detailing the frameworks configurations .
the obtained results indicate that expansion and tuning of our previous framework improve the results . tuning the codebook size , based on the finding of experiment 1 , leads to an improvement of 6% in terms of se ( see table 5 at lines 7 and 13 ) .
furthermore , the fine - tuning of our framework ( see section 2 ) also leads to an improvement of 6% in both se and sp ( see table 5 at lines 1 and 13 ) .
our framework also outperforms the proposed method of   with an improvement of 20% and 36% in terms of se and sp , respectively .
note that although the effects of preprocessing are not consistent through all the performances , the best results are achieved with nlm+f and nlm+f+a configurations as preprocessing stages . in general
, the configurations presented in experiment 2 outperform the others , in particular the high - level representation of locally mapped features with an svm classifier . focusing on the most desirable radius and sampling point configuration ,
smaller radius and sampling points are more effective in conjunction with local mapping , while global mapping benefits from larger radius and sampling points .
the work presented here addresses automatic classification of sd - oct volumes as normal or dme . in this regard ,
an extensive study is carried out covering the ( i ) effects of different preprocessing steps , ( ii ) influence of different mapping and feature extraction strategies , ( iii ) impact of the codebook size in bow , and ( iv ) comparison of different classification strategies .
while outperforming the previous studies [ 10 , 12 ] , the obtained results in this research showed the impact and importance of optimal codebook size , the potential of 3d features , and high - level representation of 2d features while extracting from local patches .
the strengths of svm while being used along with bow approach and rf classifier while being used with global mapping were shown . in terms of preprocessing steps , although the highest performances are achieved while alignment and flattening were used in the preprocessing , it was shown that the effects of these extra steps are not consistent for all the cases and do not guarantee a better performance .
the flattening method proposed by liu et al . flattens roughly the rpe due to the fact that the rpe is not segmented .
thus , in order to have a more accurate flattening preprocessing , the rpe layer should be presegmented as proposed by garvin et al . .
in this work , the lbp invariant to rotation was used and the number of patterns encoded is reduced .
once the data are flattened , the nonrotation invariant lbp could be studied since this descriptor encodes more patterns .
 this paper addresses the problem of automatic classification of spectral domain oct ( sd - oct ) data for automatic identification of patients with dme versus normal subjects . 
 optical coherence tomography ( oct ) has been a valuable diagnostic tool for dme , which is among the most common causes of irreversible vision loss in individuals with diabetes . here , a classification framework with five distinctive steps is proposed and we present an extensive study of each step . 
 our method considers combination of various preprocessing steps in conjunction with local binary patterns ( lbp ) features and different mapping strategies . using linear and nonlinear classifiers , we tested the developed framework on a balanced cohort of 32 patients . 
 experimental results show that the proposed method outperforms the previous studies by achieving a sensitivity ( se ) and a specificity ( sp ) of 81.2% and 93.7% , respectively . 
 our study concludes that the 3d features and high - level representation of 2d features using patches achieve the best results . 
 however , the effects of preprocessing are inconsistent with different classifiers and feature configurations . 
1. Introduction
2. Materials and Methods
3. Experiments
4. Results and Discussion
5. Conclusions