title: How to validate clara algorithm while clustering in R?
body: <p>I'm using <strong>clara algorithm</strong> for clustering geo-spatial data in R. My data set size is of more than 3 Million observations, with variables Longitude and Latitude. I'm good with R programming.   </p>
  
  <ol>
  <li>I have used <code>earth.dist</code> function from <code>fossil</code> package for the metric attribute of the <code>clara</code> function from <code>cluster</code> package. <code>metric</code> says that it supports only <code>"euclidean"</code> or <code>"manahattan"</code>. But, when I'm using <code>earth.dist</code> it not throwing an error or warning. When I checked with sample of random 1000 observations to see the difference <code>clara</code> is not giving different results with "euclidean" and <code>"earth.dist"</code> distance. Can I use <code>"earth.dist"</code> as metric? also please explain me why it is not throwing an error.  </li>
  <li>I have also used <strong>Silhouette Method</strong> to find the number of clusters. I found the number of clusters to be 12 when I set the number of samples to 5 but when I increased the samples to 15, I'm getting the number of clusters to be 6, with input range from 2 to 50</li>
  </ol>
  
  <p>I have followed the <a href="https://stackoverflow.com/questions/15376075/cluster-analysis-in-r-determine-the-optimal-number-of-clusters/15376462#15376462">stackoverflow page</a> to find the optimal number of clusters  </p>
  
  <p>My questions:  </p>
  
  <ol>
  <li>How to find the optimal number of samples?  </li>
  <li>How to find the optimal number of clusters?  </li>
  <li>How to find the optimal sample size for clustering using <code>clara</code>? I think <code>samplesize = min(n, 40 + 2k)</code> i.e. maximum <code>samplesize = 140</code> when <code>k = 50</code> is very small sample for data set size of more than 3 Million observation.   </li>
  </ol>
  
  <p>what are the other methods that can be considered while validating clustering? To keep it simple, how to validate clustering of <strong>clara algorithm</strong>?  </p>
  
  <p>Update 1: When I increased <code>sample = 10</code> and <code>sampsize = 400</code> from default values in R, the number of clusters is converging to 2 when Silhouette Method is used. Why did the number of clusters converge? How does the algorithm work?</p>