Non-parametric Archimedean generator estimation with implications for multiple testing
André Neumann Thorsten Dickhaus





Non-parametric Archimedean generator estimation with implications
for multiple testing
    Andr NeumannUniversity of Bremen, Institute for Statistics, Bibliothekstrae 1,
D-28359 Bremen, Germany, Email: neumann@uni-bremen.de  and Thorsten DickhausUniversity of Bremen, Institute for Statistics, Bibliothekstrae 1,
D-28359 Bremen, Germany, Email: dickhaus@uni-bremen.de
    December 30, 2023
========================================================================================================================================================================================================================================================================================


In multiple testing, the family-wise error rate can be bounded under
some conditions by the copula of the test statistics. Assuming that
this copula is Archimedean, we consider two non-parametric Archimedean
generator estimators. More specifically, we use the non-parametric
estimator from <cit.> and a slight
modification thereof. In simulations, we compare the resulting multiple
tests with the Bonferroni test and the multiple test derived from
the true generator as baselines.




§ INTRODUCTION


De copulis non est disputandum! <cit.>.
Indeed, the utilization of copulas for the construction and the mathematical
analysis of multiple tests has meanwhile become an established technique
in simultaneous statistical inference; see <cit.>, <cit.>,
<cit.>, <cit.>, <cit.>,
<cit.>, <cit.>,
and Sections 2.2.4 and 4.4 of <cit.>. In particular,
<cit.> have explicitly shown that the copula approach
leads to the most general construction method for single-step multiple
tests under known univariate marginal null distributions of the test
statistics. This is due to Sklar's Theorem (see <cit.>),
which implies that any dependency structure among test statistics
can be expressed by a copula. Thus, by standardizing the marginal
null distributions (i.e., by transforming test statistics into p-values),
the copula carries the complete distributional information which is
necessary for the calibration of a (multivariate) multiplicity-adjusted
rejection threshold or local significance level, respectively.

One important class of copula functions is constituted by so-called
Archimedean copulas (cf., e.g., Section 6 of <cit.>,
Section 5.4 of <cit.>, Section 3 of <cit.>,
and <cit.> for the relevance of Archimedean copulas in
various applications). Such copulas are defined by a generator function
ϕ. In the case that ϕ is a completely monotone function,
the corresponding Archimedean copula possesses the property of being
multivariate totally positive of order 2 (MTP_2), see <cit.>.
This positive dependency property is very helpful for proving type
I error control of a variety of multiple tests, including the extremely
popular linear step-up test by <cit.> for
control of the false discovery rate. <cit.>
have derived an adjustment factor for optimizing the power of the
linear step-up test in the case that the copula of the p-values
is of Archimedean type.

In practice, the copula of test statistics or p-values, respectively,
is often an unknown nuisance parameter (of potentially infinite dimension).
Therefore, it is near at hand to pre-estimate this copula and to utilize
the estimate in the construction of the rejection rule of the multiple
test. We call the resulting multiple test an empirically calibrated
multiple test. In the context of Archimedean copulas, <cit.>
and <cit.> have investigated method-of-moments
and maximum likelihood estimators, respectively. Their examples, however,
are restricted to simple parametric copula families. <cit.>
studied non-parametric Bernstein copula estimators with respect to
an empirical calibration of multiple tests. Their methodology, however,
is not specifically tailored towards Archimedean copula models.

In the present work, we therefore consider the problem of estimating
the generator function ϕ of an Archimedean copula non-parametrically,
and we analyze the impact of the estimation and its uncertainty on
the performance of empirically calibrated multiple tests. The material
is organized as follows. In sec: Section 2, estimation
methods based on Kendall's processes, as well as a novel modification
thereof, are introduced. sec: Section 3 deals with multiple
testing methodology based on such estimators. Simulation results are
presented in sec: Simulations, and we provide some conclusions
in sec: Conclusions.




§ ESTIMATION OF THE ARCHIMEDEAN GENERATOR FUNCTION USING KENDALL'S
PROCESS


The application of Kendall's distribution functions for statistical
inference has been treated in quite some detail in <cit.>.
In this section, we shortly summarize their findings  and consider
additionally a slightly modified generator estimator. <cit.>
propose an algorithm for estimating an Archimedean copula based on
a slightly modified version of the estimated Kendall's distribution
function as defined by <cit.>. In <cit.> the
asymptotic properties of Kendall's process are analyzed. The estimator
of the Kendall's distribution function is modified such that these
properties still hold and the estimator itself is a Kendall's distribution
function. This avoids numerical issues and additionally, the continuous
mapping theorem (CMT) can be applied. The resulting copula estimator
is (weakly) consistent. We will see in the next section that this
translates under some conditions directly to the realized family-wise
error rate (FWER) in the case that the modified estimator is utilized
in the empirical calibration of a multiple single-step test.

Further, <cit.> discussed the identifiability
of statistical models based on Kendall's distribution function. In
particular, the two- and three-dimensional cases are proven. In the
two-dimensional case this property can be verified directly, i.e.,
the Archimedean generator can be directly calculated from Kendall's
distribution function. In the three-dimensional case they show that
the model is identified by reducing the dimensionality in a sense.
More precisely, they show that from Kendall's distribution function
K_ϕ_1,m=3=K_ϕ_2,m=3 follows K_ϕ_1,m=2=K_ϕ_2,m=2,
where ϕ_1, ϕ_2 are Archimedean generator functions
and m is the dimension. Inspired by this argumentation, we consider
additionally an estimator based on averaging over two-dimensional
marginal data samples in our simulations, i.e.,

    ϕ̂_n,m=m  2^-1∑_j_1=1^m∑_j_2=j_1+1^mϕ̂_n,2^GNZ(X_i,j_1,X_i,j_2:1≤ i≤ n),

where ϕ̂_n,m^GNZ is the generator estimator
from <cit.> and n is the sample
size.

This estimator can be theoretically motivated as follows. Let C_ϕ
defined by C_ϕ(u)=ψ(∑_j=1^mϕ(u_j)),
u∈[0,1]^m, be an m-dimensional Archimedean
copula, where the generator ϕ:[0,1]→ℝ_+
is a continuous, convex and strictly decreasing function with ϕ(1)=0
and ψ:v↦inf{ u∈[0,1]|ϕ(u)≤ v}
is the generalized inverse of ϕ. For convenience, we will call
both - ϕ and ψ - the Archimedean generator and C_ψ=C_ϕ,
K_ψ=K_ϕ . Let ϕ_n be a sequence of generators
corresponding to given Kendall's distribution functions K_n,m=2
which converge weakly[A distribution function converges weakly (w→)
if and only if it converges pointwise for all continuity points of
the limit function.] to K. <cit.> used their Proposition
6 in order to translate the convergence of the generators to the convergence
of the Archimedean copulas. Let us restate this proposition:
[Proposition 6 in <cit.>]
Let ψ, ψ_1, ψ_2, ... be m-monotone[A generator ψ is m-monotone if and only if it has m-2
derivatives such that (-1)^iψ^(i)≥0
for all 0≤ i≤ m-2, and (-1)^m-2ψ^(m-2)
is decreasing and convex.] Archimedean generators. Then C_ψ_n,mw→C_ψ,m,
as n→∞, if and only if there exists a sequence (a_n)
of constants a_n>0 such that, for any x∈[0,∞), ψ_n(a_nx)→ψ(x),
as n→∞.

Assume that ψ_n is only 2-monotone. Then C_ψ_n,m
is not an Archimedean copula anymore. However, the argumentation for
the converse only uses the uniform convergence of ψ_n and
ϕ_n for any dimension m≥2. This means that the following
proposition holds:

Let ψ, ψ_1, ψ_2, ... be 2-monotone Archimedean
generators. If there exists a sequence (a_n) of constants
a_n>0 such that, for any x∈[0,∞), ψ_n(a_nx)→ψ(x),
as n→∞, then C_ψ_n,mw→C_ψ,m,
as n→∞.


Applying Algorithm 1 from <cit.> to
two-dimensional marginal data results in a 2-monotone estimator
ψ̂_n (see Section 4.3 in <cit.>).
For a specific sequence (a_n)_n∈ℕ this
estimator converges uniformly to the true generator ψ on compact
sets in probability (see Section 7 in <cit.>).
Thus, C_ψ̂_n,mw→C_ψ,m holds
in probability (and therefore, uniformly in probability since C_ψ,m
is continuous and [0,1] is a compact interval).

On the other hand, for a 2-monotone generator ψ_n the Archimedean
copula restricted to the diagonal line u↦ C_ψ_n,m(u,…,u)=ψ_n(m·ϕ_n(u))
is still a one-dimensional distribution function. We will use this
function in the next section for calibrating the multiple test under
equal local significance levels. This means that we will receive valid
(i.e., in [0,1]) local significance levels.. Therefore,
the negative impact of using two-dimensional marginal data on multiple
testing is potentially small. We will explore this further in the
simulations of sec: Simulations.    



§ MULTIPLE TESTING


In this section, we apply the Archimedean copula estimator of <cit.>
in a multiple testing context. The consistency of the realized FWER
is a direct consequence of the connection between the FWER and the
copula of the test statistics (see Theorem 2 in <cit.>)
and the consistency of the estimator (see Theorem 2 in <cit.>).



  §.§.§ Model


We perform a multiple test φ=(φ_1,…,φ_1)^⊤
for m≥2 null hypotheses H_j⊂Θ versus
K_j⊆Θ\ H_j, 1≤ j≤ m.
The parameter space Θ=Θ_1×…×Θ_m
is a subset of ℝ^m. The marginal test φ_j
rejects the null hypothesis H_j if and only if the corresponding
p-value p_j is smaller than a local significance level α_loc^(j)∈(0,1).
Our goal is to find α_loc^(1),…,α_loc^(m)
such that the FWER is controlled at the global significance level
α∈(0,1). A FWE occurs whenever we reject at least
one true null hypothesis.



  §.§.§ Assumptions


Let ϑ=(ϑ_1,…,ϑ_m)^⊤∈Θ
be the parameter vector of interest and 1≤ j≤ m. We assume
that the following statements hold true. 


  * The null hypothesis H_j has the form H_j:{ϑ|ϑ_j∈Θ_j^*},
where ∅≠Θ_j^*⊂Θ_j⊆ℝ.

This means that we can only test marginal parameters. In particular,
the number of parameters is equal to the number of hypotheses m.
With a slight abuse of notation, H_j denotes also Θ_j^*.

We define the p-value p_j by 1-inf_ϑ∈ H_jF_T_j|ϑ(T_j),
where T_j is known.


  * The test statistic T_j tends to larger values under the alternative
K_j.

This means that the p-values tend to smaller values under alternatives.


  * The cdf F_T_j|ϑ=F_T_j|ϑ_j
ofT_j depends only on ϑ_j.

We test the marginal parameter ϑ_j∈ H_j by using
only the p-value p_j. It makes sense that p_j does not
depend on the other parameters.


  * There exists a ϑ_j^*∈ H_j such that F_T_j|ϑ_j^*≡inf_ϑ_j∈ H_jF_T_j|ϑ_j.

The important part here is that this equality holds for every point.
This means that the parameters ϑ_1^*,…,ϑ_m^*
are independent of the observed test statistics.


  * The cdf F_T_j|ϑ_j^* is continuous.

Under this assumption, we have the following properties: The corresponding
p-value p_j=1-F_T_j|ϑ_j^*(T_j)
is valid, i.e., Prob((p_j| H_j)≤ u)≤ u,
and uniformly distributed under ϑ_j^*. We write Z| H_j
or Z| K_j for a random variable Z=Z|ϑ_j under
true null hypothesis ϑ_j∈ H_j or under true alternative
ϑ_j∈ K_j, respectively. Further, we have almost surely
that p_j<α_j, α_j∈[0,1], if and
only if T_j>c_j(α_j), where c_j(α_j)=F_T_j|ϑ_j^*^-1(1-α_j).
In addition, the copula of the test statistics under ϑ^*
is unique by Sklar's theorem.


  * The parameter vector ϑ^* is a least favorable
configuration (LFC).

An LFC is a parameter vector which maximizes the FWER. In our case
the LFC ϑ^* is located in the global null
hypothesis H_0=⋂_j=1^mH_j. This means that weak FWER
control (i.e., control on H_0) entails strong FWER control (i.e.,
control on Θ). In our setting the subset pivotality
condition (see <cit.>) is sufficient for this assumption
(see Lemma 3.1 in <cit.> and Lemma 3.3 in <cit.>).


  * The copula of the test statistics C_T|ϑ^*
under the LFC ϑ^* is Archimedean.

Note that C_T|ϑ^* can
depend on the whole parameter vector ϑ^*.


  * A pseudo sample of the test statistics under ϑ^*
is at hand.

This sample is needed to calibrate the multiple test and can be (approximately)
achieved by bootstrapping under ϑ^* (see
<cit.>) for suitable test statistics.



  §.§.§ Calibration and consistency


Under some assumptions, <cit.> have shown that the
FWER is uniformly bounded on the parameter space Θ
by the copula of the random vector (1-p_1,…,1-p_m)
under the LFC ϑ^*. In our setup this is
the same copula as the copula of the test statistics. More precisely,
under our assumptions we have

    FWER_ϑ(φ_1,…,φ_m)    ≤FWER_ϑ^*(φ_1,…,φ_m)
        =Prob(⋃_j=1^m{(p_j|ϑ_j^*)<α_loc^(j)})
        =1-Prob(⋂_j=1^m{ 1-(p_j|ϑ_j^*)≤1-α_loc^(j)})
        =1-C_T|ϑ^*(1-α_loc^(1),…,1-α_loc^(m)).

The bound depends on the copula of the test statistics under the LFC
ϑ^*. This has the advantage that we only
need to estimate the Archimedean copula under one parameter vector
in order to calibrate the multiple test. Setting α_loc=α_loc^(1)=…=α_loc^(m),
the estimated local significance level α̂_loc,n
is given by

    α̂_loc,n=1-Ĉ_T,n^-1(1-α),

where Ĉ_T,n^-1 is the quantile of the estimated
Archimedean copula on the diagonal line u↦Ĉ_T,n(u,…,u)=ψ̂_n(m·ϕ̂_n(u)).
For convenience, we will write occasionally C(u) instead
of C(u,…,u). Note that Ĉ_T,n^-1(v)=ψ̂_n(m^-1·ϕ̂_n(v))
for v∈(0,1] and zero otherwise. This means that α̂_loc,n
can be drawn in the graph of ϕ̂_n (see fig: alpha_loc).


The quantile function
Ĉ_T,n^-1 is uniformly consistent, i.e.,
it converges uniformly to C_T|ϑ^*^-1
in probability as n→∞. In particular, α̂_loc,n
is consistent.



The argumentation is the same as for the consistency of ψ̂_n
in <cit.>. Let C_n, n∈ℕ,
be non-decreasing functions with C_nw→C.
Then C_n^-1w→C^-1 (see Proposition
0.1 in <cit.>). Again, C^-1 denotes the quantile
of u↦ C(u,…,u). Thus, the function C↦ C^-1
is continuous on the space of cdfs with a suitable norm for the weak
convergence. By the CMT, Ĉ_T,n^-1 is pointwise
consistent on the compact set of continuity points (0,1] and therefore,
uniformly on [0,1].


The realized FWER defined by rFWER_n=1-C_T|ϑ^*(1-α̂_loc,n)
converges in probability to α as n→∞.



This result follows directly from lem: hat_C_T is uniformly consistent
and the CMT applied to u↦ C_T|ϑ^*(u,…,u).




§ SIMULATIONS


 



  §.§.§ Overview


In this section we consider three simulations. The first simulation
compares pointwise the estimated copulas on the diagonal line. The
second simulation is more of theoretic interest. There, we sample
the test statistics under the LFC directly and continue as in the
first simulation. We compare the estimated generators in terms of
their calibrated local significance level and empirical power. In
the last simulation we consider a misspecified setting by sampling
normally distributed data. We use the sample means as test statistics
and bootstrap to derive a pseudo sample for the test statistics under
the LFC.



  §.§.§ Simulation 1 - comparison of the generators


In this simulation a sample of the test statistics is directly taken
from a Gumbel copula under various parameters derived from Kendall's
τ. For consistency with Simulation 3, we denote the sample size
of the test statistics by B instead of n. We consider the estimators
ϕ̂_B,m^GNZ from <cit.>
and ϕ̂_B,m^MC, which is defined by 

    ϕ̂_B,m^MC=M^-1∑_ℓ∈ Mϕ̂_B,2^GNZ(X_i,j_1(ℓ),X_i,j_2(ℓ):1≤ i≤ B),

where 1≤ j_1≠ j_2≤ m, m is the dimension and M=100
is the number of Monte Carlo repetitions. In low dimensions it is
possible to average over all two-dimensional subsamples as in sec: Section 2.
We compare the estimated copulas regarding the pointwise absolute
distance to the true copula C_ϕ, i.e., |C_ϕ̂_B(u,…,u)-C_ϕ(u,…,u)|,
u∈{ 0,1/(B+1),…,1}.

The default setting is τ=0.5, B=100 and m=6. In fig: Simulation 1-1
and fig: Simulation 1-2, we plot the resulting Archimedean
copulas C(u,…,u), where u lies on the grid { 0,1/(B+1),…,1}.
At each point the copula is averaged over L repetitions. It can
be generally observed that the copula of both estimators are comparable
with some exceptions. Under independence, GNZ performs considerably
worse. In dimensions m∈{ 30,60}, the distance between
MC and the true copula is slightly smaller and more consistent compared
to GNZ. Further, it can be observed that the sample size should not
be too small (B≥100). There is a slight but visible improvement
from B=50 to B=100. We will consider additional settings in
the next two simulations. 





  §.§.§ Simulation 2 - calibration in an artificial setting


The setting in this simulation is very similar to Simulation 1. Again,
no raw data is simulated. Instead, a sample of the test statistics
T_1^*,…,T_B^* from a Gumbel
copula C_θ^*, θ^*≥1, with standard normal
marginals is given, where θ^* corresponds to the copula
parameter under the LFC. We use this sample to calibrate the multiple
test. This means that the estimation procedure from the first simulation
is applied to T_1^*,…,T_B^*.
Then, we derive the estimated local significance level α̂_loc,B
as in sec: Section 3.

We test the mean vector ϑ=μ
by the hypotheses H_j:{μ_j=0} versus K_j:{μ_j≠0}.
The observed test statistics T=(T_1,…,T_m)^⊤
are taken from a Gumbel copula C_θ, θ≥1, with
normal marginals. More specifically, T_j| H_j∼𝒩(0,n^-1)
and T_j| K_j∼𝒩(μ,n^-1) , where
μ>0 denotes the unknown global effect size (for simplicity) and
n∈ℕ emulates a effect of the data sample size. The copula
parameter θ^*=θ^*(τ) is directly derived
from Kendall's τ. We set θ=θ^*+(1-π_0)μ,
where π_0 is the proportion of true null hypotheses. The two-sided
p-value p_j is given by 2(1-F_T_j| H_j(|T_j|)).

In this setup, the copula of the test statistics depends indirectly
on μ. In particular, we have θ=θ^*
under H_0. Note that 0≡μ^*∈ H_0
is an LFC since the Gumbel copula on the diagonal line u↦ C_θ(u,…,u)=u^m^1/θ
is non-increasing for θ↓θ^*, where m>1
and 0≤ u≤1. Therefore, it is straightforward to check that
FWER_μ(φ)≤1-C_θ(1-α_loc)≤1-C_θ^*(1-α_loc)=FWER_μ^*(φ). 

The default setting is τ=0.5, B=100, n=100, m=6 and
μ=0.2. In fig: Simulation 2-1 to fig: Simulation 2-3,
the estimated local significance level and empirical power are compared,
respectively. The empirical FWER is omitted but always below 5%.
 With increasing sample size B, both α̂_loc,B^GNZ
and α̂_loc,B^MC converge to a local
significance level close to the true significance level α_loc^ϕ_θ.
With larger dimension m, α̂_loc,B^GNZ
and α̂_loc,B^MC fall drastically and
are close to the Bonferroni correction.  Under stronger dependence
τ, α̂_loc,B^GNZ and α̂_loc,B^MC
deviate again from the Bonferroni correction and get closer to α_loc^ϕ_θ.
In most settings, the GNZ estimator is slightly better in terms of
the mean local significance level but slightly worse in terms of standard
deviation (see the supplement for the standard deviation plots and
numerical results). Overall , this simulation suggests that both estimators
are comparable for calibrating multiple tests.






  §.§.§ Simulation 3 - calibration in a misspecified setting


In this simulation the data X_1,…,X_n
is sampled from a multivariate normal distribution with equi-correlation
coefficient ρ and known variances equal to one (i.e., the covariance
matrix Σ=ρ1_m× m+(1-ρ)I_m× m,
where I_m× m is the identity matrix). We test
the mean vector ϑ=μ by the
hypotheses H_j:{μ_j=0} versus K_j:{μ_j≠0}.
The test statistic T_j is given by |X_j|=|n^-1∑_i=1^nX_i,j|
and the two-sided p-value p_j is equal to 1-F_T_j| H_j(T_j)=2(1-F_X_j| H_j(T_j)).
We have X_j| H_j∼𝒩(0,n^-1)
and X_j| K_j∼𝒩(μ,n^-1)
, where μ denotes the global effect size. Then μ^*≡0
is the LFC since the Gaussian copula does not depend on μ
and the LFC is marginally attained at zero. A sample of the test
statistics T_1^*,…,T_B^*
is calculated by bootstrapping the data. Under the LFC, T_b,j^*
is equal to |X_j^*-μ̂_j|=|X_j^*-X_j|.
We use the bootstrap sample of the test statistics to estimate the
generators. The estimation procedure is then the same as in the first
two simulations.

The default setting is τ=0.5, B=100, n=100, m=6 and
μ=0.2. In fig: Simulation 3-1 and fig: Simulation 3-2,
the mean empirical power over L tests resulting from the estimated
local significance level are compared. The empirical FWER is omitted
and (almost) always below 5%. The only exceptions are due to numerical
issues. With increasing m, the  function 
tends to fail in this setup. This function is used for the estimator
α̂_loc,B^GNZ. We skipped these cases
such that the results regarding GNZ in the right plot of fig: Simulation 3-1
are potentially averaged over less than L tests and can contain
numerical errors. In most settings the empirical power is close to
the Bonferroni method. A notable improvement is achieved under large
equi-correlation ρ. Overall, this simulation unfortunately suggests
that the Bonferroni correction is preferable when the assumption that
the test statistics copula is Archimedean is violated.





§ CONCLUSIONS


We considered two non-parametric Archimedean generator estimator in
the context of multiple testing. The consistency of the modified estimator
is established and can be observed in fig: Simulation 1-1
and fig: Simulation 2-1 as well. 

The simulations suggest that both estimators perform comparable but
the modified estimator is easier to implement and numerically more
stable. Some improvements of this estimator which are visible when
comparing both estimators directly do not carry over to a better performance
in multiple testing. For example, the modified estimator performs
considerably better under independence but the performance of both
resulting multiple tests is almost identical to the Bonferroni test
in this situation.

The modified estimator is still affected by the curse of dimensionality.
However, we did not perform separate sample size calculations given
the dimension and power. By Simulation 2, it is likely that the sample
size needs to be much larger than the dimension for a performance
comparable with the true generator. For example, in fig: Simulation 2-1
the dimension is 6 and the sample size needs to be larger than
250.

We observed that the Archimedean assumption is essential unless strong
dependencies are present. However, this might be only because the
dependency is not utilized by the Bonferroni test. We did not perform
comparisons with other copula estimation methods. Hence, we can not
generally conclude that multiple tests should be performed using non-parametric
Archimedean generator estimators under strong dependencies.



  §.§.§ Supplementary Materials


The source code, plots and numerical results of the simulations are
available online.



[Barbe et al., 1996]MR1405589
Barbe, P., Genest, C., Ghoudi, K., and Rémillard, B. (1996).
On Kendall's process.
Journal of Multivariate Analysis, 58(2):197–229.

[Benjamini and Hochberg, 1995]benjamini1995controlling
Benjamini, Y. and Hochberg, Y. (1995).
Controlling the False Discovery Rate: A Practical and Powerful
  Approach to Multiple Testing.
Journal of the Royal Statistical Society: Series B
  (Methodological), 57(1):289–300.

[Bodnar and Dickhaus, 2014]Bodnar-Dickhaus-EJS
Bodnar, T. and Dickhaus, T. (2014).
False discovery rate control under Archimedean copula.
Electronic Journal of Statistics, 8(2):2207–2241.

[Cerqueti and Lupi, 2018]Cerqueti-Lupi2018
Cerqueti, R. and Lupi, C. (2018).
Copulas, uncertainty, and false discovery rate control.
International Journal of Approximate Reasoning, 100:105–114.

[Dickhaus, 2014]Dickhaus-Buch2014
Dickhaus, T. (2014).
Simultaneous Statistical Inference with Applications in the
  Life Sciences.
Springer-Verlag Berlin Heidelberg.

[Dickhaus and Gierl, 2013]stp-copulae
Dickhaus, T. and Gierl, J. (2013).
Simultaneous test procedures in terms of p-value copulae.
In Proceedings on the 2nd Annual International Conference on
  Computational Mathematics, Computational Geometry & Statistics (CMCGS
  2013), pages 75–80. Global Science and Technology Forum (GSTF).

[Dickhaus and Stange, 2013]Dickhaus_Stange_2013
Dickhaus, T. and Stange, J. (2013).
Multiple point hypothesis test problems and effective numbers of
  tests for control of the family-wise error rate.
Calcutta Statistical Association Bulletin,
  65(257-260):123–144.

[Efron, 1979]MR515681
Efron, B. (1979).
Bootstrap methods: another look at the jackknife.
The Annals of Statistics, 7(1):1–26.

[Embrechts et al., 2003]embrechts-copulae-survey
Embrechts, P., Lindskog, F., and McNeil, A. (2003).
Modelling Dependence with Copulas and Applications to Risk
  Management.
In Rachev, S. T., editor, Handbook of Heavy Tailed Distributions
  in Finance, pages 329–384. Elsevier Science B.V.

[Genest et al., 2011]Genest_Neslehova_Ziegel_2011
Genest, C., Nešlehová, J., and Ziegel, J. (2011).
Inference in multivariate Archimedean copula models.
TEST, 20(2):223–256.

[Härdle and Okhrin, 2010]okhrin2010
Härdle, W. K. and Okhrin, O. (2010).
De copulis non est disputandum - Copulae: an overview.
AStA Advances in Statistical Analysis, 94(1):1–31.

[McNeil et al., 2005]Q-Risk-Management
McNeil, A. J., Frey, R., and Embrechts, P. (2005).
Quantitative Risk Management: Concepts, Techniques and Tools.
Princeton, NJ: Princeton University Press.

[McNeil and Nešlehová, 2009]McNeil2009
McNeil, A. J. and Nešlehová, J. (2009).
Multivariate Archimedean copulas, d-monotone functions and
  ℓ_1-norm symmetric distributions.
The Annals of Statistics, 37(5B):3059–3097.

[Müller and Scarsini, 2005]Scarsini2005
Müller, A. and Scarsini, M. (2005).
Archimedean copulae and positive dependence.
Journal of Multivariate Analysis, 93(2):434–445.

[Neumann et al., 2019]Neumann_Bodnar_Pfeifer_Dickhaus_2019
Neumann, A., Bodnar, T., Pfeifer, D., and Dickhaus, T. (2019).
Multivariate multiple test procedures based on nonparametric copula
  estimation.
Biometrical Journal, 61(1):40–61.

[Resnick, 1987]Resnick_1987
Resnick, S. I. (1987).
Extreme Values, Regular Variation and Point Processes.
Springer New York.

[Schmidt et al., 2015]Schmidt2014b
Schmidt, R., Faldum, A., and Gerß, J. (2015).
Adaptive designs with arbitrary dependence structure based on
  Fisher's combination test.
Statistical Methods & Applications, 24(3):427–447.

[Schmidt et al., 2014]Schmidt2014a
Schmidt, R., Faldum, A., Witt, O., and Gerß, J. (2014).
Adaptive designs with arbitrary dependence structure.
Biometrical Journal, 56(1):86–106.

[Sklar, 1959]Sklar1959
Sklar, M. (1959).
Fonctions de répartition à n dimensions et leurs
  marges.
Publications de l'Institut de statistique de l'Université
  de Paris, 8:229–231.

[Stange et al., 2015]stange2015uncertainty
Stange, J., Bodnar, T., and Dickhaus, T. (2015).
Uncertainty quantification for the family-wise error rate in
  multivariate copula models.
AStA Advances in Statistical Analysis, 99(3):281–310.

[Westfall and Young, 1993]westfallyoung
Westfall, P. H. and Young, S. S. (1993).
Resampling-based multiple testing: examples and methods for
  p-value adjustment.
Wiley Series in Probability and Mathematical Statistics. Applied
  Probability and Statistics. Wiley, New York.






