I had a non-normal distribution of my variable of interest which required log10-transformation to reduce outliers. This was then standardized, to a mean of 0 and sd of 1 (z-score?). I now want to obtain percentile values from the standardized values. For example, how do I obtain the 75th percentile of my variable given I know it was not normal to begin with and even after log10-transformation it is not normally distributed. Thanks for insight.

  • 1
    $\begingroup$ So you have large number of observations of the said distribution? Sample percentile won't work for you? $\endgroup$ – Art Sep 5 '19 at 4:16
  • $\begingroup$ ~450 observations. Can you expand on sample percentile? In it's standardized and transformed version, getting the Xth percentile is straightforward? $\endgroup$ – mindhabits Sep 5 '19 at 5:05
  • $\begingroup$ 75th percentile = upper quartile. $\endgroup$ – Nick Cox Sep 5 '19 at 5:10
  • 2
    $\begingroup$ I think so since all the transformations you mentioned preserve rankings $\endgroup$ – Art Sep 5 '19 at 5:13

You may have started with data somewhat similar to the 450 observations in the vector w, which are summarized below using R:

summary(y);  sd(y)
Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
 914    27299    83975   380413   220675 16507340 
[1] 1381348  # sd

enter image description here

The transformation $W = \log_{10}(Y)$ gets rid of some of the skewness, and considerably reduces the number of boxplot outliers.

 w = log10(y);  summary(w);  sd(w)
  Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 2.961   4.436   4.924   4.930   5.344   7.218 
[1] 0.7004566  # sd

enter image description here

Finally, we transform to get a random variable $Z$ with $E(Z) = 0$ and $SD(Z) = 1.$ That is. $Z = \frac{W - \bar W}{S_W}.$ [The histogram shows that $Z$ is not normal.]

z = (w - mean(w))/sd(w)
summary(z);  sd(z)
      Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
 -2.810906 -0.705027 -0.008336  0.000000  0.590705  3.265995 
 [1] 1  # sd

enter image description here

Notice that the the summaries have given sample upper quartiles (75th percentiles throughout: 220675 for $Y,$ 5.344 for $W,$ and 0.590705 for $Z.$ Because the transformations have not changed the order of the observations, the sample quartiles are related according to the transformations: $\log_{10}(220675) = 5.344$ and $(5.344 - 4.930)/0.700 = 0.591.$

 [1] 5.343753
 (5.343753 - mean(w))/sd(w)
 [1] 0.5907055

With a sample as large as $n = 450,$ sample quartiles should not be very far from population quartiles. Because of the way I simulated the data, I know that the exact 75th quantile of the distribution of the $W$'s is 5.457, which is not far from the upper quartile 5.344 of the observed values of $W.$ Of course, a larger sample size would tend to give a better match between sample and population upper quartiles.

qgamma(.75, 50, 10)
[1]  5.457062

One style of 95% nonparametric bootstrap confidence interval for the population 75th percentile is $(5.27, 5.41).$ Unfortunately, our sample of 450 observations was an 'unlucky' one, with an unusually low 75th percentile, so this CI does not quite include 5.457. [I could have started over with a different seed to get a 'more cooperative' sample of 450, but it is wrong to give the impression that 95% CIs are 100% CIs.]

d.re = replicate(2000, quantile(sample(w, repl=T),.75))
quantile(d.re, c(.025, .975))
    2.5%    97.5% 
5.267784 5.413533 

Let's say you have an RV $X$. Your transformations are all included in this function which result in another RV $Y = f(X)$. My understanding is that you'd like to know what the 75th percentile of $Y$ is. If that's the case, you could just use the samples (450 observations of $\hat Y$) to get an estimate.


Your Answer

