title: R : Calculate a P-value of a random distribution
body: <p>I want to get the P-value of two randomly distributed observations x and y, for example :</p>
  
  <pre><code>&gt; set.seed(0)
  &gt; x &lt;- rnorm(1000, 3, 2)
  &gt; y &lt;- rnorm(2000, 4, 3)
  </code></pre>
  
  <p>or:</p>
  
  <pre><code>&gt; set.seed(0)
  &gt; x &lt;- rexp(50, 10)
  &gt; y &lt;- rexp(100, 11)
  </code></pre>
  
  <p>let's say that T is my test-statistic defined as mean(x) - mean(y) = 0 (this is H0), the P-value is then defined as : p-value = P[T>T_observed | H0 holds].<br>
  I tried doing this :</p>
  
  <pre><code>&gt; z &lt;- c(x,y) # if H0 holds then x and y are distributed with the same distribution
  &gt; f &lt;- function(x) ecdf(z) # this will get the distribution of z (x and y)
  </code></pre>
  
  <p>then to calculate the p-value i tried this:</p>
  
  <pre><code>&gt; T &lt;- replicate(10000, mean(sample(z,1000,TRUE))-mean(sample(z,2000,TRUE))) # this is 
  supposed to get the null distribution of mean(x) - mean(y)
  &gt; f(quantile(T,0.05)) # calculating the p-value for a significance of 5%
  </code></pre>
  
  <p>obviously this doesn't seem to work, what am i missing ?</p>