Paper Title: Online Linear Programming with Batching.












  
  
   












same

*assumption*

assumption[1]
 
  
  
  @edefcurrentlabel#1
 
 
  
 


breakablealgorithm
  
   

     algorithm
     height.8pt depth0pt 2pt
     
  
     2pt
  




propProposition
thmTheorem
lemLemma
corCorollary
obvObservation
remarkRemark
DefDefinition







=-0.45in
=0in
=0in
=6.5in
=9.0in
=0.25in

1.1




1]Haoran Xu
1]Peter W. Glynn
1]Yinyu Ye
[1]Department of Management Science and Engineering, Stanford University
[1]{haoran14, glynn, yyye}@stanford.edu





    

    
    [
    
===========

§ ABSTRACT 
    We study Online Linear Programming (OLP) with batching. The planning horizon is cut into K batches, and the decisions on customers arriving within a batch can be delayed to the end of their associated batch. Compared with OLP without batching, the ability to delay decisions brings better operational performance, as measured by regret. Two research questions of interest are: (1) What is a lower bound of the regret as a function of K? (2) What algorithms can achieve the regret lower bound? These questions have been analyzed in the literature when the distribution of the reward and the resource consumption of the customers have finite support. By contrast, this paper analyzes these questions when the conditional distribution of the reward given the resource consumption is continuous, and we show the answers are different under this setting. When there is only a single type of resource and the decision maker knows the total number of customers, we propose an algorithm with a O(log K) regret upper bound and provide a Ω(log K) regret lower bound. We also propose algorithms with O(log K) regret upper bound for the setting in which there are multiple types of resource and the setting in which customers arrive following a Poisson process. All these regret upper and lower bounds are independent of the length of the planning horizon, and all the proposed algorithms delay decisions on customers arriving in only the first and the last batch. We also take customer impatience into consideration and establish a way of selecting an appropriate batch size.



§ INTRODUCTION

Many resource allocation problems can be formulated as Linear Programming (LP) problems. In a static environment, the decision maker can first collect the information of all the customers and then solve the LP problem to obtain the optimal allocation decision. However, assuming a static environment may not be realistic in many real-world applications. Thus, a variant of the problem in a dynamic environment called online resource allocation has attracted attentions of the community of operations research and management science. In this paper, we use Online Linear Programming (OLP) (See <cit.>) as the framework to study the online resource allocation problem.

A key feature of online resource allocation is that customers arrive sequentially and the decision maker is required to make immediate and irrevocable decisions without the information of future customers. This requirement brings online resource allocation problem closer to real-world applications in which data and information reveals sequentially; however, since customers are usually willing to wait for a while though not forever, this requirement maybe also too restrictive because it implicitly assumes that all customers are completely impatient. Under this strong assumption, the decision maker may lose the opportunity to improving performance by delaying decisions on some customers. Thus, it is worthwhile to study the online resource allocation problem when batching is allowed.

The impact of postponing real-time decisions has been recently studied in many applications of online decision making, including order fulfillment (<cit.>), kidney exchange (<cit.>), and ride hailing (<cit.>). In this paper, we study the impact of batching operation on OLP. We are interested in the following two research questions: (1) What is a lower bound of the regret as a function of the number of batches? (2) What algorithms can achieve the lower bound? Main contributions of this paper are summarized in the following subsection.


 §.§ Main Contributions

We study OLP with batching under the random input setting introduced in <cit.>. Reward and resource consumption of customers are modeled as i.i.d. random variables, and we assume the conditional distribution of the reward given the resource consumption is continuous. The total amount of resource available to the decision maker is limited, and the goal of the decision maker is to maximize the expected total reward by accepting and rejecting the requests of the customers. Let T be the length of the planning horizon, and we consider two natural models of T. The first model specifies T in terms of the number of customers, and the second model specifies T in terms of time. In this paper, we use the first model when the total number of customers is known in advance, and we use the second model when the total number of customers is random. In the batching operation, the planning horizon is cut into K batches with equal length, and the decisions on the customers arriving within a batch can be delayed to the end of their associated batch. Throughout this paper, we call B=T/K the batch size.   

We propose algorithms and provide regret analyses for different settings of OLP with batching. We first study the setting in which the total number of customers is known. When there is only a single type of resource, we propose an algorithm that only delays the decisions on customers arriving in the first and the last batch. In addition, the proposed algorithm assumes the distribution of the reward and the resource consumption is unknown. We prove the regret of the proposed algorithm is O(log K). In addition, we formulate the problem as a Markov Decision Process and define the optimal online policy using the Bellman equation. Compared with the proposed algorithm, the optimal online policy needs to delay decisions on customers arriving in all the batches, and it also assumes the distribution of the reward and the resource consumption is known. We show the regret of the optimal online policy is Ω(log K). Thus, the regret of this setting is Θ(log K), and only delaying decisions on customers arriving in the first and the last batch is sufficient to achieve the optimal order of regret. Knowing the distribution of the reward and the resource consumption is also not necessary for achieving the optimal order of the regret. When there are multiple types of resource, we additionally assume that some historical data is available at the beginning of the planning horizon. With this extra assumption, we propose an algorithm having O(log K) regret, and the proposed algorithm only delays the decisions on the customers arriving in the last batch.  

We then study the setting in which customers arrive following a Poisson process, and we only focus on the setting in which there is a single type of resource. The proposed algorithm still only delays the decisions on the customers arriving in the first and the last batch, and it assumes that the arrival rate of the Poisson process and the distribution of the reward and the resource consumption are unknown. By assuming the inter-arrival time is independent of the reward and resource consumption, we show that the proposed algorithm also has a O(log K) regret upper bound. Thus, under appropriate assumptions, the total number of customers being random does not lead to a higher order of regret when the batching operation is allowed. In addition, this O(log K) regret upper bound implies that, if we fix the the length of the planning horizon and all the customers arriving in the first and the last batch are willing to wait for at least B units of time, the regret is uniformly bounded in the expected total number of customers. If batching is not allowed, the uniformly bounded regret is impossible when the conditional distribution of the reward given the resource consumption is continuous (<cit.>).

All algorithms proposed in this paper are based on the Action-History-Dependent Learning Algorithm (AhdLA) proposed in <cit.>. AhdLA needs to solve a LP for each customer arriving in the planning horizon, but our algorithms only solve one LP for each batch. Thus, in addition to reducing the regret, our algorithms show that the batching operation also improves the computation complexity.  

In this paper, we also consider customer impatience. When the decisions on the customers are delayed, some impatient customers may leave the resource allocation procedure. The decision maker cannot allocate resource to these impatient customers and lose the opportunity to collect rewards from them. We propose an algorithm to deal with this additional setting and provide a regret upper bound for it. We observe that the regret upper bound does not decay monotonically in the batch size B. This regret upper bound suggests that there should be an optimal batch size when customer impatience is considered. We then provide a way to select the batch size by minimizing the regret upper bound.


 §.§ Related works

Online resource allocation has been extensively studied by the community of operations research and management science. We refer readers to <cit.> for a survey of the online resource allocation problem. There has been some works studying the impact of delay and batching on online resource allocation problem. <cit.> study an online resource allocation problem with both impatient customers and partially patient customers. The decision maker needs to make immediate and irrevocable decisions on impatient customers, but the decisions on the partially patient customers can be delayed for several time periods.  <cit.> study two different settings of postponing real-time decisions when there are finite types of customers. In one setting, the decision on each customer is delayed for B time periods. In the other setting, customers are batched together with the batch size equal B+1, i.e., the decisions on the jth customer of a given batch is delayed B+1-j time periods. <cit.> show that the regret of online resource allocation decays exponentially in B in the delay setting and decays in the order of O(1/B+1) in the batching setting. One difference between this paper and <cit.> is that we assume that the conditional distribution of the reward given the resource consumption is continuous. This difference has significant influence on the regret when delaying real-time decision is not allowed. If the distribution of the reward and the resource consumption have finite support, the regret of the online resource allocation has a constant regret upper bound (<cit.>, <cit.>); however, if the conditional distribution of the reward given the resource consumption is continuous, <cit.> provides a logarithmic lower bound for the regret. Compared with <cit.>, our regret analysis show that the impacts of batching are also different in these two settings. Another difference is that our algorithms need to delay the decisions at most on customers arriving in the first and the last batch. For customers arriving in all the other batches, our algorithms can make immediate irrevocable decisions on them. We show that we can still reduce the regret significantly even if we violate the online assumption in this weaker way. 

There are also works studying how batching and delay can improve algorithm performance in online matching problem. Many of them focus on the analysis of competitive ratio. <cit.> study the online matching problems in which the online arrivals of edges of a graph are batched into s stages. For the s-stage integral general matching problem, they propose a polynomial time algorithm with 1/2+1/2^O(s) competitive ratio. For the two-stage fractional general matching problem, they propose a polynomial time algorithm with 0.6 competitive ratio. <cit.> study a vertex weighted matching problem, where online arrivals are batched into K stages. They propose a matching algorithm with 1-(1-1/K)^K competitive ratio for the problem. Randomness of online arrivals and the impact of customer impatience are also studied in the literature related to online matching with batching and delay. For instance, <cit.> consider online matching problems in a two-side market, where buyers and sellers arrive following Poisson processes and leave the market after an exponentially distributed amount of time due to impatience. The decision maker needs to decide when to match and how to match based on the number of buyers and sellers and the utility of possible matches. In our paper, we study similar issues in the context of online resource allocation, and we propose a method to decide appropriate batch size by taking the randomness of online arrivals and customer impatience into consideration. For more works about the impact of delay on online matching, we refer readers to <cit.> and references therein.

In this paper, we study the online resource allocation problem using the framework of OLP. There are two major classes of online linear programming algorithms. One class of OLP algorithms make decisions by repeatedly solving LPs, and the other class of OLP algorithms use first-order methods. <cit.> study the OLP under random permutation model and propose the Dynamic Learning Algorithm (DLA). DLA periodically solves dual LPs to update the dual price and uses the dual price to make resource allocation decision. These dual LPs are formulated based on revealed information of the customers, but the average resource used in the formulation is always the initial average resource. <cit.> study the OLP under random input model and propose the Action-History-Dependent Learning Algorithm (AhdLA). AhdLA can be viewed as a modification of DLA, which formulates dual LPs using the remaining average resource instead of the initial average resource. <cit.> introduce the concept of dual convergence and use the dual convergence result to prove that AhdLA has O(log nloglog n) regret. <cit.> improves the dual convergence result and shows that the regret of AhdLA is O(log n). In addition, <cit.> provides a Ω(log n) lower bound for the problem and shows that AhdLA achieves the optimal order of regret. AhdLA needs to solve a LP when each customer arrives. Thus, AhdLA has a high computation complexity. Compared to algorithms that repeatedly solving LPs, algorithms using first-order methods have lower computation complexity. <cit.> propose a dual mirror decent method, and <cit.> propose a projected stochastic subgradient descent method. Both of the methods have O(√(n)) regret. <cit.> propose a first-order method that has o(√(n)) regret. Although first-order method has lower computation complexity, to our best knowledge, there is no first-order method that can achieve logarithmic regret. Our work is mostly related to <cit.>. We propose algorithms based on the idea of Action-History-Dependent. Since batching is allowed, our algorithms only need to solve one LP for each batch. Thus, our algorithms have lower computation complexity than AhdLA..

The paper is organized as follows. Section <ref> introduce preliminaries and useful properties of dual price used in the regret analysis. The algorithms and the regret analysis are provided in Section <ref>. Numerical experiments and discussions of the algorithms are given in Section <ref>. Finally, we summarize the paper in Section <ref>.


§ PRELIMINARIES

The linear resource allocation problem can be formulated as the following linear programming model.

    [                  max        ∑_j=1^nr_jx_j;                  S.T ∑_j=1^na_j· x_j≤ b_0;                      0≤ x_j≤ 1  ∀ j=1,⋯,n ]

where n is the total number of customers, b_0∈ℝ^m_+ is the total resource, m is the number of the types of resources, and  (r_j,a_j)∈ℝ^m+1 is the reward and resource consumption of the jth customer. The dual formulation of (<ref>) is

    [                          min         b_0^Tp + ∑_j=1^n y_j;                          S.T  a_j^Tp+y_j≥ r_j   ∀ j=1,⋯,n;                              p≥ 0, y_j≥ 0  ∀ j=1,⋯,n    . ]

Let x_j^* and p_n^* be the optimal solution of (<ref>) and (<ref>). By complementary slackness,

    x_j^*={[                      1      if r_j>a_j^Tp_n^*;                      0 if r_j<a_j^Tp_n^*    . ].

(<ref>) implies that {1{r_j>a_j^Tp_n^*}}_j=1^n is a near-optimal solution. In addition, if p_n^* is provided to the decision maker in advance, the decision 1{r_j>a_j^Tp_n^*} for each customer j can be implemented without knowing the reward and the resource consumption of all the other customers. In the setting of OLP, customers arrive sequentially, and the decision maker needs to make the allocation decision on the current customer without knowing the information of future customers. Therefore, (<ref>) motivates the extensively studied dual-based algorithms of OLP. Let ℋ_j be all the information collected by the decision maker up to the arrival of customer j. Then, the idea of dual-based algorithms is to compute a dual price p(ℋ_j) and make the allocation decision on customer j through the dual decision rule (<ref>).

    x_j=1{r_j>a_j^Tp(ℋ_j)}

Since the information of a customer is unknown before his arrival in the setting of Online Linear Programming, we model the reward and resource consumption (r_j,a_j) as a random vector. We also model the total number of customers n as a random variable. When n is deterministic, we view it as a trivial random variable. Throughout this paper, we make the following assumptions.
1
     

    (a) {(r_j,a_j)}_j=1^+∞ is a sequence of i.i.d. random vectors in ℝ^m+1

    (b) Let (r_j,a_j)d=(r,a), and the distribution of (r,a) satisfies
    
    ℙ{0≤ r≤r̅}=1
       ℙ{a_2≤a̅}=1
       ℙ{a≥ae}=1

    with r̅>0, a̅>0, a>0, and e∈ℝ^m is the vector with all components being 1.

    (c) The initial average resource d_0=b_0/𝔼[n]∈⊗_i=1^m(d,d̅) where d̅>d>0.

For a positive integer N, define the offline dual price p_N^*(·):R^m_+⟶R^m_+ to be the random function

    p_N^*(d) = min_p≥ 0 d^Tp + 1/N∑_j=1^N(r_j-a_j^Tp)^+

We call N the sample size of the offline dual price in this paper. In addition, for a non-positive integer N≤ 0, define

    p_N^*(d) = 0  ∀ d≥ 0

Define the population dual price p^*(·):R^m_+⟶R^m_+ to be a deterministic function

    p^*(d) = min_p≥ 0 d^Tp + 𝔼[(r-a^Tp)^+]

To conduct the regret analysis, we make the following additional assumptions throughout the paper. These assumptions are inherited from <cit.> and <cit.>.
2
     

    (a) e_i^Tp^*(d_0)>0 ∀ i=1,⋯, m, where e_1,⋯,e_m is the standard basis of ℝ^m.

    (b) Given a, r has a bounded conditional density f_r|a, i.e, there exists a constant β>0 such that, for all v in the support of a,
    
    f_r|a(u|v)≤β  ∀ u

    (c) There exists δ_p>0 such that the Hessian of g(p)=d^Tp+𝔼[(r-a^Tp)^+] exists, is positive definite, and is continuous in p in ⊗_i=1^m [e_i^Tp^*(d_0)-δ_p,e_i^Tp^*(d_0)+δ_p].

Assumption <ref>(a) can be viewed as a non-degeneracy assumption, which requires all resources are asymptotically consumed when the number of customers goes to infinity. Assumption <ref>(b) and (c) indicates that the conditional distribution of the reward given the resource consumption is continuous. Assumption <ref> and Assumption <ref> are used to guarantee that we can apply the following useful properties of the offline and population dual prices from the literature.

     

    (a) (Proposition 1 of <cit.>) For all d∈⊗_i=1^m(d,d̅) and positive integers N>m,

    
    e^Tp^*(d)∈[0,r̅/d]
       ℙ{e^Tp_N^*(d)∈[0,r̅/d]}=1

    where e∈ℝ^m is the vector with all components being 1.

    (b) (Lemma 1 of <cit.>, Lemma 2 of <cit.>, Lemma 12 of <cit.>) There exists a neighborhood Ω_d of d_0 such that Ω_d⊆⊗_i=1^m(d,d̅), and for all d∈Ω_d,
    
    e_i^Td = 𝔼[e_i^Ta1{r>a^Tp^*(d)}]

    and p^*(·) is Lipschitz continuous on Ω_d, i.e., there exists a constant L such that,
    
    p^*(d)-p^*(d)_2^2≤ Ld-d_2^2  ∀ d,d∈Ω_d


The above assumptions are also used to derive the dual convergence results in the literature, which allows us to give an upper bound of 𝔼[p_N^*(d)-p^*(d)_2^2]. <cit.> first show the following pointwise convergence result.

    𝔼[p_N^*(d)-p^*(d)_2^2]=O(loglog N/N)  ∀ d∈ [d,d̅]

<cit.> then improves the convergence rate and proves a uniform convergence result. We summarize it in Lemma <ref>.

    There exists a neighborhood Ω_d of d_0 such that Ω_d⊆⊗_i=1^m(d,d̅), and there exist constants C_Dual and N_Dual such that, when N>N_Dual,
    
    𝔼[sup_d∈Ω_dp_N^*(d)-p^*(d)_2^2]≤C_Dual/N


Note that we can assume the neighborhood Ω_d in Lemma <ref> and Lemma <ref> to be the same. 


§ ONLINE LINEAR PROGRAMMING WITH BATCHING
 
In this section, we provide the problem formulation of Online Linear Programming with batching, define performance measure, propose algorithms, and provide regret analysis. We first state the problem formulation by specifying the length of the planning horizon in terms of time, and we then show that specifying the length of the planning horizon in terms of the number of customers is a special case.

Let T be the length of the planning horizon, and let t_0,t_1,⋯,t_K be a sequence of time points in the planning horizon with t_0=0 and t_K=T. These time points cut the planning horizon into K batches, and t_0,t_1,⋯,t_K are determined before the decision making process. We assume that

    t_k-t_k-1 = B  ∀ k=1,⋯,K

where B is called the batch size. In OLP with batching, the decision maker is not required to make irrevocable decisions instantly when a customer arrives. However, it is also not realistic to delay the decisions on the customers for too long. Thus, for k=1,⋯,K, we assume that the decisions on the customers arriving in (t_k-1,t_k] need to be made no later than t_k. Denote (N(t):t≥ 0) to be the customer arrival process, i.e., N(t) is the total number of customer arrives in [0,t]. Denote x_j^π∈{0,1} to be the decision on the jth customer made by an algorithm π. Let A^π(t)=x^π_N(t). Then, the above assumption requires that

    A^π(t) is measurable w.r.t σ({(r_j,a_j)}_j=1^N(t_k),(N(t):t≤ t_k)) if t∈(t_k-1,t_k]  ∀ k=1,⋯,K

Define the offline optimal reward to be

    [                  R_T^*=                     max        ∑_j=1^N(T)r_jx_j;                                             S.T   ∑_j=1^N(T)a_jx_j≤ b_0;                         0≤ x_j≤ 1  ∀ 1≤ j≤ N(T) ]

and define the online reward given by an algorithm π to be

    R_π(T)=∑_j=1^N(T)r_jx^π_j

The performance of the algorithm is measured by the regret defined as

    Δ_T(π) = 𝔼[R_T^* - R_π(T)]

If N(t) is deterministic, and there exists an integer λ_C such that

    N(t) = ⌊λ_Ct⌋  ∀ t≥ 0

the above formulation is equivalent to specifying the length of the planning horizon in terms of the number of customers. In this special case, we can also specify the batch size to be λ_CB customers. In the following subsections, we first study the setting in which the total number of customers is known. We start the discussion of this setting from the case in which there is only a single type of resource, and we then extend the results to the case in which there are multiple types of resource. We then switch to the setting in which customers arrive following a Poisson process, and we also discuss the impact of customer impatience in this setting.


 §.§ Known Total Number of Customers



  §.§.§ Single Type of Resource

In this section, we assume the total number of customers is known and there is only a single type of resource. In addition, we assume 

    N(t) = ⌊ t⌋  ∀ t≥ 0

Then, the total number of customers n=T, and the batch size B is just the number of customers arriving in each batch. We propose Algorithm<ref> for this setting.

    
    

Because of the ability to delay decisions on customers, the regret of Algorithm <ref> is much smaller than the regret of AhdLA in <cit.>. The high-level explanation of how delaying decisions reduce the regret is summarized into the following two points. First, Algorithm <ref> has more information on the customers than AhdLA in the first batch. This extra information allows Algorithm <ref> to learn the distribution of the reward and the resource consumption better, and it also makes the stochastic process {b_j}_j=0^n have smaller variation and stay closer to {(n-j)d_0}_j=0^n. These two benefits from delaying decisions in the first batch both help to reduce the regret significantly. Secondly, because Algorithm <ref> makes decisions in the last batch by solving the offline problem, there is almost no remaining resource left at the end of the planning horizon with high probability. 

To provide a more detailed explanation than the two points above, we give a sketch of the regret analysis of Algorithm <ref>. A complete and rigorous proof can be found in the appendix. For k=0,⋯,K-1, define 

    d_k = b_t_k/n-t_k
 
Recall that Ω_d is the neighborhood of the initial average resource d_0 in Lemma <ref> and Lemma <ref>. Select δ_d>0 such that

    [d_0-δ_d,d_0+δ_d]⊆Ω_d

Define the stopping time κ̅ as

    κ̅=min{K}∪{k:d_k∉[d-δ_d,d+δ_d]}

Denote Algorithm <ref> as π_1, and we have the following regret upper bound. In fact, this regret upper bound holds for all online policies. 

    Δ_n(π_1)
    ≤   𝔼[∑_k=1^K∑_j=t_k-1+1^t_k(r_j-a_jp^*(d_0))(1{r_j>a_jp^*(d_0)}-1{r_j>a_jp_k})(1{k<κ̅}+1{k=K})]
       +O(𝔼[t_K-1-t_κ̅-1])
       +𝔼[b_np^*(d_0)]

The derivation of the above regret upper bound is similar to the derivation of the generic regret upper bound in Theorem 2 of <cit.>. The three components of the upper bound provide some key properties of the algorithm that affect its regret. (<ref>) imply that using dual prices that are too different from p^*(d_0) to make decisions may cause high regret, (<ref>) shows that the stochastic process {d_k}_k=0^K-1 has large variation may lead to high regret, and (<ref>) indicates that having too much remaining resource left at the end of the planning horizon may result in high regret.

We complete the regret analysis by upper bounding the three components in the above regret upper bound. An important step is to analyze, for 1≤ k≤ K and t_k-1<j≤ t_k, 

    𝔼[(r_j-a_jp^*(d_0))(1{r_j>a_jp^*(d_0)}-1{r_j>a_jp_k})]
    ≤   𝔼[a_j(p_k-p^*(d_0))1{a_jp^*(d_0)<r_j≤ a_jp_k}+a_j(p^*(d_0)-p_k)1{a_jp_k<r_j≤ a_jp^*(d_0)}]

When (r_j,a_j) and p_k are independent, by conditioning on (a_j,p_k) and applying Assumption <ref>(b), we can upper bound the right-hand-side of (<ref>) by

    C𝔼[p_k-p^*(d_0)^2_2]

where C is a constant. We can then use the dual convergence result to further upper bound it. However, the independence between (r_j,a_j) and p_k does not hold for 1≤ j≤ t_1 and t_K-1<j≤ t_K, i.e., the customers arriving in the first and the last batch. To be more detailed, since (r_j,a_j) with 1≤ j≤ t_1 is used to compute p_1, there is a weak dependence between p_1 and (r_j,a_j). Similarly, there is a weak dependence between p_K and (r_j,a_j) with t_K-1<j≤ t_K. In this paper, we use the idea of Leave-One-Out to analyze the weak dependence, which needs the following extra assumption. 
3
    There exists ϵ_d>0 such that
    
    𝔼[a]>d̅+ϵ_d


Assumption <ref> implies that p^*(d_0)>0 and can be viewed as a stronger version of Assumption <ref>(a). Given d∈Ω_d, N ∈ℕ^+, denote {1,⋯,N} as [N]. Let J be a subset of [N], and define p_N,J(d) to be

    p_N,J(d) = min_p≥ 0 dp + 1/N-J∑_j∈ [N] \ J (r_j-a_jp)^+

Thus, except for {(r_j,a_j):j∈ J}, the rewards and the resource consumptions used to compute p_N,J(d) and p_N^*(d) are the same. With Assumption <ref>, we can provide an upper bound on the probability of making different decisions using p_N^*(d) and p_N,J(d). 

    Assume Assumption <ref>, <ref> and <ref>. For k∈{1,2}, ∃ N_LOO,k and C_LOO,k such that, if N>N_LOO,k, for all d∈Ω_d and J⊆ [N] with J=k, 
    
    ℙ{1{r_j>a_jp_N^*(d)}≠1{r_j>a_jp_N,J(d)}}≤C_LOO,k/N  ∀ 1≤ j≤ N


Now, for 1≤ j≤ t_1, define

    p_j,1 = min_p≥ 0(t_1-1)b_0/np+∑_l=1^j-1(r_l-a_lp)^++∑_l=j+1^t_1(r_l-a_lp)^+

Then, by Lemma <ref> and the boundedness of (r_j,a_j) and p^*(d_0), for 1≤ j≤ t_1

    𝔼[(r_j-a_jp^*(d_0))1{1{r_j>a_jp_j,1}≠1{r_j>a_jp_1}}]
    ≤   (r̅+a̅r̅/d)C_LOO,1/B

In addition, since p_j,1 and (r_j,a_j) are independent,

    𝔼[(r_j-a_jp^*(d_0))(1{r_j>a_jp^*(d_0)}-1{r_j>a_jp_j,1})]
    ≤   𝔼[(r_j-a_jp^*(d_0))(1{r_j>a_jp^*(d_0)}-1{r_j>a_jp_j,1})]
       +𝔼[(r̅+a̅r̅/d)1{1{r_j>a_jp_j,1}≠1{r_j>a_jp_1}}]
    ≤   O(𝔼[p_j,1-p^*(d_0)^2_2]) + O(1/B)

We still can use the dual convergence result to upper bound the first term on the right-hand-side, and the O(1/B) term will only bring a constant to the regret upper bound in the end. Similar things can be done for t_K-1<j≤ t_K.

Another important step in the regret analysis is to study the remaining average resource process {d_k}_k=0^K-1. Since the property of d_k is hard to analyze when it is too far from d_0, we define the following auxiliary stochastic process by freezing d_k once it leaves [d_0-δ_d,d_0+δ_d].

    d'_0 = d_0
       d'_k+1 = (n-t_k)d'_k-∑_t=t_k+1^t_k+1a_t1{r_t>a_tp_k+1}/n-t_k+11{k<κ̅}+d'_k1{k≥κ̅}

The following Lemma <ref> states a useful property of {d'_k}_k=0^K-1, which is used to analyze the stopping time κ̅ and the remaining resource at the end of the planning horizon.

    Under Assumption <ref>, <ref> and <ref>, there exists constants Λ_Res, C_Res such that, when B>Λ_Res,
    
    ∑_k=1^K-1𝔼[d'_k-d_0^2]≤C_Reslog K/B


Together with Lemma <ref> and Lemma <ref>, we can show (<ref>), (<ref>) and (<ref>) are all O(log K). Thus, we get the regret upper bound of Algorithm <ref> summarized in Theorem <ref>.

Under Assumption <ref>, <ref> and <ref>, there exists a constant Λ_DA such that, when B>Λ_DA,
    
    Δ_n(π_1)≤ O(log K)


Theorem <ref> implies that the regret of Algorithm <ref> only depends on the number of batches K. Thus, if the decision maker chooses K independent of the length of the planning horizon, Algorithm <ref> has a constant regret upper bound with respect to the total number of customers n. Another interesting property of Algorithm <ref> is that it achieves a significant regret reduction by only delaying decisions on customers arriving in the first and the last batch. However, since we can delay decisions on customers arriving in all the batches, a natural question arises: if we delay decisions on more customers, how much more regret reduction can we achieve? We answer this question by introducing a lower bound result. Define value functions V_1,⋯,V_K through the following Bellman Equation.

    {   V_k(b) = 𝔼[max_x_t_k+j∈[0,1] ∀ 1≤ j≤ B∑_j=t_k+1^t_k+1a_jx_j≤ b∑_j=t_k+1^t_k+1r_jx_j+V_k+1(b-∑_j=t_k+1^t_k+1a_jx_j)] ∀ k=0,⋯,K-1
       V_K(b) = 0
        .

Let π be the policy given by solving the above Bellman equation. Theorem <ref> provides a lower bound to the regret of π.

    Under Assumption <ref>, <ref> and <ref>, there exists a constant Λ_DA such that, when B>Λ_DA,
    
    Δ_n(π)≥Ω(log K)


Since the regret lower bound of π will also be a regret lower bound of any online policy, when the total number of customers is known and there is only a single type of resource, Theorem <ref> and Theorem <ref> imply that the regret of OLP with batching is Θ(log (K)). Thus, although delaying decisions on more customers may achieve a smaller regret, it is impossible to design an algorithm whose regret has a smaller order than the regret of Algorithm <ref>. In addition, π assumes the distribution of the reward and the resource consumption is known; however, Algorithm <ref> assumes the distribution of the reward and resource consumption is unknown. Thus, knowing the distribution of the reward and the resource consumption also does not lead to an algorithm having smaller order of regret than Algorithm <ref>.


  §.§.§ Multiple Resources

In this section, we study the setting in which there are multiple types of resource. We still assume that the total number of customers is known, and we make the same assumption on N(t) as in the previous section. We need the following assumption to design an algorithm with provable regret upper bound.
4
     

    (a) {(r_j,a_j)}_j=-∞^0 is a sequence of i.i.d. random vector such that (r_0,a_0)d=(r_1,a_1). In addition, {(r_j,a_j)}_j=-∞^0 and {(r_j,a_j)}_j=1^+∞ are independent.

    (b) Realizations of {(r_j,a_j)}_j=-B+1^0 is provided to the decision maker at t_0.

We can view realizations of {(r_j,a_j)}_j=-B+1^0 as the historical data collected at the beginning of the planning horizon. With Assumption <ref>, we propose Algorithm <ref> for the case that there are m≥ 1 types of resources and the total number of customers is known.

    
    

The main difficulty of analyzing Algorithm <ref> is that we cannot generalize Lemma <ref> to the case with multiple types of resource. However, we can use a different regret analysis technique together with Assumption <ref> to deal with the difficulty. Similar to the case with a single type of resource, we define

    d_k = b_t_k/n-t_k  ∀ k=0,⋯,K-1

Select δ_d>0 such that

    ⊗_i=1^m[e_i^Td_0-δ_d,e_i^Td_0+δ_d]⊆Ω_d

Define the stopping time κ̅ to be

    κ̅=min{K}∪{k:d_k∉⊗_i=1^m[e_i^Td_0-δ_d,e_i^Td_0+δ_d]}
 
In addition, define

    p_kB:n(d_k) = min_p≥ 0d_k^Tp + 1/n-kB∑_j=kB+1^n(r_j-a_j^Tp)^+  ∀ 1≤ k≤ K-1

and

    [          R_n-(K-1)B^*(d_K-1)=                           max          ∑_j=(K-1)B+1^nr_jx_j;                                                         S.T ∑_j=(K-1)B+1^na_jx_j≤ b_t_K-1;                                 0≤ x_j≤ 1  ∀ (K-1)B+1≤ j≤ n ]

Denote Algorithm <ref> as π_2, and we have the following regret upper bound.

    Δ_n(π_2)
    ≤   𝔼[∑_k=1^κ̅-1∑_t=t_k-1+1^t_k(r_j-a_j^Tp_kB:n^*(d_k))(1{a_j^Tp_kB:n^*(d_k)<r_j≤a_j^Tp_k}-1{a_j^Tp_k<r_t≤a_t^Tp_kB:n^*(d_k)})]
       +O(𝔼[t_K-1-t_κ̅-1])
       +𝔼[R_n-(K-1)B^*(d_K-1)-∑_j=t_K-1+1^nr_jx_j^π_2]

The derivation of this regret upper bound is similar to the regret decomposition in <cit.>. There are three major differences between this regret upper bound and the regret upper in Section <ref>. First, compared with (<ref>), (<ref>) does not contain any term related to the last batch. Thus, we do not need the Leave-One-Out analysis for p_K in the analysis of (<ref>). In addition, by Assumption <ref>, p_1 and {(r_j,a_j)}_j=1^t_1 are independent because p_1 is computed using the historical data. Thus, in the analysis of (<ref>), we do not need the Leave-One-Out analysis for p_1 either. Secondly, before batch κ̅, the online decisions made in each batch is compared with the decisions made based on p_kB:T^*(d_k) instead of p^*(d_0). Thus, for all 1≤ k≤ K-1 and t_k-1<j≤ t_k, we need to deal with the weak dependence between (r_j,a_j) and p^*_kB:T(d_k) ; however, this week dependence comes from the weak dependence between r_j and d_k, which can be dealt with the Lipschtiz property of population dual price in Lemma <ref>(b) and the uniform dual convergence in Lemma <ref>. For more details, we refer the reader to the complete and rigorous proof in the appendix. Finally, the only term related the last batch in the above regret upper bound is (<ref>), which allows us to make a more direct use of the fact that Algorithm <ref> makes decisions in the last batch by solving an offline problem. For t_K-1<j≤ n, x_j^π_2=1{r_j>a_j^Tp_K} and p_K is the optimal dual solution of (<ref>). Under our assumption of the distribution of (r_j,a_j)'s, there will be at most m fractional decisions in the optimal solution of (<ref>). By strong duality and complementary slackness, we can conclude that (<ref>) is O(1). Since only the theory of Linear Programming is involved in the analysis of (<ref>), we also do not need the Leave-One-Out analysis for p_K in the analysis of (<ref>). Theorem <ref> summarizes the regret upper bound of Algorithm <ref>.

    Under Assumption <ref>, <ref> and <ref>, there exists a constant Λ_MultiDA such that, when B>Λ_MultiDA,
    
    Δ_n(π_2)≤ O(log K)


If we modify Algorithm <ref> by computing the dual price for the first batch only using the information of the customers arriving in the first batch, we get an algorithm that does not rely on historical data. A reasonable conjecture is that the regret of this modified algorithm is still O(log K). To prove this conjecture, we need to deal with the weak dependence issue in the first batch carefully, and we leave the proof for the future works.


 §.§ Random Number of Customers



  §.§.§ Poisson Arrival Process

In this section, we assume again that there is only a single type of resource, but we assume the customers arrival process (N(t):t≥ 0) is a Poisson process with an unknown rate λ. We propose Algorithm <ref> for this setting.

    
    

We can use almost the same regret analysis techniques introduced in Section <ref> to analyze Algorithm <ref>, but we need additional steps to deal with the randomness of the number of customers arriving in each batch. The following Assumption <ref> simplifies our analysis.
5
    (N(t):t≥ 0) is independent of {(r_j,a_j)}_j=1^+∞.


Denote Algorithm <ref> as π_3. Theorem <ref> states an upper bound of the regret of Algorithm <ref>. Details of the proof is provided in the appendix.

    Under Assumption <ref>, <ref>, <ref> and <ref>, there exists a constant Λ_PA such that, when λ B>Λ_PA,
    
    Δ_T(π_3)≤ O(log K)


Theorem <ref> implies that, under appropriate assumptions, switching the setting from known total number of customers to random total number of customers does not worsen the order of the regret. The regret upper bound still only depends on the total number of batches K. Thus, if the decision maker chooses K independent of the arrival rate of the Poisson process and the length of the planning horizon, Algorithm <ref> has a constant regret upper bound with respect to the expected total number of customers. 

If we assume the historical data of the customers arriving in [-B,0] is available to the decision maker, with appropriate independence assumption, we can extend the results of this section to the case in which there are multiple types of resource. Similar to the idea in Section <ref>, the only modification made to Algorithm <ref> is that we use the historical data to compute dual price p_1. We can then combine the regret analysis techniques in the proof of Theorem <ref> and Theorem <ref> to get a regret upper bound. We do not provide a detailed and rigorous analysis of the case with multiple types of resource and Poisson arrival process in this paper, and it may be left as a future work. 


  §.§.§ Customer Impatience

Regret bounds provided in Section <ref> assumes that all the customers arriving in the first and last batch are willing to wait until the end of the batch. Thus, the regret bounds in Theorem <ref> suggests that the regret decreases in the batch size B. However, it is more realistic to assume that some of the customers arriving in the first and last batch are impatient and will leave before receiving the decisions from the decision maker. In this section, we still assume that there is a single type of resource and the customers arrive following a Poisson process, but we take customer impatience into consideration. We model the customer impatience by the amount of time that a customer is willing to wait for. Let V_j be the arrival time of the jth customer and W_j be the time for which the jth customer is willing to wait. We propose Algorithm <ref> to deal with the customer impatience. 

    Action-History-Dependent Learning Algorithm with Customer Impatience and Poisson Arrival Process
    

Algorithm <ref> is almost the same as Algorithm <ref> except for two differences. First, if j≤ N(t_1) and V_j+W_j ≤ t_1, the jth customer will be rejected because the customer has left the resource allocation procedure; however, (r_j,a_j) is stilled used to compute p_1. Secondly, if N(t_K-1)<j≤ N(T) and V_j+W_j ≤ T, customer j will be rejected, and (r_j,a_j) is also not used to compute p_K. To analyze the regret of Algorithm <ref>, we make the following extra assumption.
6
     

    (a) {W_j}_j=1^+∞ are i.i.d. nonnegative random variables.

    (b) {V_j}_j=1^+∞, {(r_j,a_j)}_j=1^+∞, and {W_j}_j=1^+∞ are mutually independent.

Let F be the c.d.f of W_j, and denote Algorithm <ref> as π_4, then Proposition <ref> gives a regret upper bound.

    Under Assumption <ref>, <ref>, <ref>, <ref>, and <ref>, there exists a constant Λ_CI such that, if λ B>Λ_CI, 
    
    Δ_T(π_4)≤ O(log K ) + O(λ∫_0^BF(u)du)


Since K=T/B, Proposition <ref> implies that the regret of Algorithm <ref> does not decrease monotonically in B. Thus, when customer impatience is considered, there should exist an optimal batch size B^*. In section <ref>, we discuss how to select appropriate batch size using Proposition <ref>.


§ EXPERIMENTS AND DISCUSSIONS



 §.§ Numerical Results: Known Total Number of Customers

In this section, we test the proposed algorithms for the case in which the total number of customers is known. We test Algorithm <ref> when there is only one type of resource (m=1). The total initial resource is set to b_0=5n. The reward r_ji.i.d.∼Uniform(1,19), and the reward a_ji.i.d.∼Uniform(1,19). In addition, r_j a_j for all j. We test Algorithm <ref> when there are four types of resource (m=4). For each type of resource, the total initial resource is set to 5 times of the total number of customers, i.e., b_0(i) = 5n for all i. The reward r_ji.i.d.∼Uniform(1,19), and the reward a_iji.i.d.∼Uniform(1,19). In addition, r_j a_1j a_2j a_3j a_4j for all j. For both of the settings, we run numerical experiments for different number of batches K(=2,8,32,64,128) and different total number of customers n(=1280,6400,32000,64000,128000). The values of n are selected to make them divisible by the largest value of K. For each numerical test instance, we run 1000 simulation trials to estimate the regret. Table <ref> and Table <ref> summarize the estimated regrets of Algorithm <ref> and Algorithm <ref> respectively.


We use Figure <ref> and Figure <ref> to discuss the insights of the numerical results in Table <ref> and Table <ref>. Figure <ref> demonstrates the impact of the total number of customers on the regret of Algorithm <ref> and Algorithm <ref>. For each fixed K, Figure <ref> shows that the regret is constant in n when there is only a single type of resource. This is consistent with Theorem <ref>, which suggests that there is a regret upper bound that is independent of n. When there are four types of resource, Figure <ref> shows that the regret becomes constant after n is large enough for each K. Although we observe that the regret increases in n when n is small and larger n is required to see the regret becomes constant for larger K, the numerical results still justifies the fact that there is a regret upper bound that is independent of the total number of customers for OLP with batching when there are multiple types of resource. Figure <ref> demonstrates the impact of the number of batches on the regret. For both the case with a single type of resource and the case with four types of resource, Figure <ref> shows that regret is logarithmic in K, which justifies the O(log K) regret upper bounds in Theorem <ref> and <ref>. In addition, Figure <ref> shows that the regret increases in m when n and K are fixed. In the regret analysis of this paper, we always fix m and view it as a constant. An interesting future research will be studying the impact of the number of types of resource on the regret.




 §.§ Numerical Results: Poisson Arrival Process

We test Algorithm <ref> in this section. The initial total resource is set to b_0 = 5λ T. The reward r_ji.i.d.∼Uniform(1,19), and the reward a_ji.i.d.∼Uniform(1,19). In addition, r_j a_j for all j. We run numerical experiments for different values of arrival rate λ(=10,50,100,500) and length of planning horizon T(=128,512,1024), and number of batches K(=2,8,32,64,128,256). For each numerical test instance, we run 1000 simulation trials to estimate regrets. Table <ref> summarizes the estimated the regrets of Algorithm <ref>.

The arrival rate of the Poisson process is assumed to be unknown in Algorithm <ref>, and only decisions on customers arriving in the first and the last batch are delayed. It is interesting to investigate how much improvement can be achieved if we assume the arrival rate is known at the beginning of the planning horizon and decisions on customers arriving in every batch are delayed. We make the following two modifications to Algorithm <ref>. First, we replace the estimated arrival rate λ̂_k with the known arrival rate λ, i.e.,

    λ̂_k = λ  ∀ k=0,⋯,K-1

Secondly, we include the information from customers arriving in the current batch into the computation of dual prices used in batch k=2,⋯,K-1, i.e.,

    p_k = min_0≤ p≤r̅/d N(t_k)d_k-1p+∑_j=1^N(t_k)(r_j-a_jp)^+  ∀ k=2,⋯,K-1

After making these two modifications, O(log K) is still a regret upper bound, but these two modifications should improve the performance of Algorithm <ref>. Table <ref> summarizes the estimated regrets of the modified version of Algorithm <ref>.

To illustrate the insights provided by the numerical results in Table <ref> and <ref>, we select several numerical test instances to make Figure <ref> and Figure <ref>. Figure <ref> justifies Theorem <ref> by indicating that the regret of Algorithm <ref> is independent of the arrival rate λ and the length of the planning horizon T. We may interpret T as the time between two inventory replenishment and λ as the scale of the business. It is reasonable to assume T is fixed since it may be subject to the operation of other components of the supply chain. Then, by setting the initial total resource b_0=5λ T, we mean that the amount of the inventory ordered in each replenishment is proportional to the scale of the business. Let B be fixed as well. Under the assumption on the initial total resource, if we always can persuade customers arriving during the first B units of time and the last B units of time to wait regardless of business scale, then Theorem <ref> together with the above numerical results show that Algorithm <ref> can achieve uniformly bounded regret. Furthermore, Figure <ref> shows that, although knowing arrival rate and delaying decisions of customers arriving in every batch improve the performance, the order of the regret seems not change.




 §.§ Batch Size Selection

Although it is impossible to compute the optimal batch size B^* without knowing the distributions of (r_j,a_j) and λ, Proposition <ref> may still provide a recommended way of scaling the batch size B in terms of λ. In the following, we fix T=10, set F to be the c.d.f of Exp(1). Then,

    λ∫_0^BF(u)du = λ B + λexp(-B) - λ

Suppose we restrict B=λ^-γ with γ∈ [0,1]. Plugging this B into the upper bound in Proposition <ref>, we observe that

    {   log(T/B) = γlogλ+log T
       lim_λ→+∞λ^1-γ+λexp(-λ^-γ)-λ/logλ=0  if γ≥ 0.5
       lim_λ→+∞λ^1-γ+λexp(-λ^-γ)-λ/logλ=∞  if γ< 0.5
        .

(<ref>) then suggests that choosing γ=0.5 minimize the upper bound in Proposition <ref> when λ is large enough. We run numerical experiments to verify if choosing γ=0.5 is also good for minimizing the regret of Algorithm <ref>. The plot on the left of Figure <ref> indicates that γ=0.1 and γ=0.3 are significantly worse than the other three options. In the plot on the right of Figure <ref>, we show the difference among γ=0.5, γ=0.7, γ=0.9 by also doing the test with larger arrival rate. It shows that γ = 0.5 is the best among at least these five options.

In fact, when customer impatience is taken into consideration, the optimal batch size B^* should be viewed as a function of the arrival rate λ. We can minimize the upper bound in Proposition <ref> by solving the following equation. 

    BF(B) = C/λ

where C is a constant. Thus, a recommended batch size should has the same order with respect to the arrival rate as the solution of (<ref>).


§ CONCLUSION

In this paper, we study OLP with batching, and we focus on the case in which the conditional distribution of the reward given the resource consumption is continuous. We propose algorithms following the idea of Action-History-Dependent, which solve one LP for each batch. When the total number of customers is known and there is only a single type of resource, we show the regret of OLP with batching is Θ(log K). We also extend the O(log K) regret upper bound to the case in which there are multiple types of resource and the case in which customers arrive following a Poisson process. Our regret analysis justifies the intuition that OLP with batching has smaller regret than OLP without batching. In addition, one managerial insight given by the design of the algorithms and the regret analysis is that only delaying decisions on customers arriving in the first and the last batch is sufficient to achieve a significant regret reduction. Since all the regret upper bounds only depend on the total number of batches K, another managerial insight given by the regret analysis is that OLP with batching has a constant regret upper bound with respect to the length of the planning horizon and the arrival rate of the customers if the total number of batches is fixed. Furthermore, we consider the setting in which customers are only willing to wait for a random amount of time. The regret upper bound of this setting does not monotonically decrease in the batch size, which implies that there exists an optimal batch size when we take customer impatience into consideration. We then provide a way to select the batch size by minimizing the regret upper bound. 

There are some future research questions about OLP with batching. First, how should we do the regret analysis for the cases with multiple types of resource when there is no historical data available? Developing a Leave-One-Out analysis for the dual price when there are multiple types of resource is one possible way to solve the problem, though there may be other regret analysis techniques that can be used to address this question. Secondly, if we fix the total number of batches, our numerical experiments show that the regret is larger when there are more types of resource. Thus, another future research question is about the impact of the number of types of resource on the regret of OLP with batching. Finally, we only develop a Ω(log K) regret lower bound for the case in which there is only a single type of resource and the total number of customers is known. Although it is reasonable to assume that Ω(log K) is still a lower bound for more complicated settings, it is still interesting to provide rigorous proof for the regret lower bound of other more complicated settings.

apalike



§ APPENDIX



 §.§ Proof of Lemma <ref>

 
Fix k∈{1,2}, and fix N>N_LOO,k be a positive integer, where N_LOO,k is given in Lemma <ref>. Let {(r_(j),a_(j))}_j=1^N be a permutation of {(r_t,a_t)}_t=1^N such that 

    r_(1)/a_(1)≥⋯r_(j)/a_(j)≥r_(j+1)/a_(j+1)≥⋯≥r_(N)/a_(N)

Let 𝒥_k be all the subsets J of {1,⋯,N} with J=k. For each d∈Ω_d, define the following random set

    I_N(d,k) = ⋃_J∈𝒥_k{j:1{r_(j)>a_(j)p_N^*(d)}≠1{r_(j)>a_(j)p_N,J(d)}}

By Lemma <ref>, for all d∈Ω_d, J⊆ [N] with J=k and 1≤ t≤ N,

    ℙ{1{r_t>a_tp_N^*(d)}≠1{r_t>a_tp_n,J(d)}}
    ≤   ℙ{∃ j∈ I_n(d,k)  s.t (a_t,r_t)=(a_(j),r_(j))}
    
        =   𝔼[ℙ{∃ j∈ I_n(d,k)  s.t (a_t,r_t)=(a_(j),r_(j))|(a_(1),r_(1)),⋯,(a_(n),r_(n)) }]
    
        =   𝔼[I_n(d,k)/n]
    ≤   C_LOO,k/n



     Given k∈{1,2}, ∃ N_LOO,,k and C_LOO,k such that, if N>N_LOO,k, for any d∈Ω_d,
    
    𝔼{I_N(d,k)}≤ C_LOO,k



     Fix k∈{1,2}, and fix N>2. Given d∈Ω_d, define event E_1 and E_2 to be
    
    E_1 = {r_(1)/a_(1)≠⋯r_(j)/a_(j)≠r_(j+1)/a_(j+1)≠⋯≠r_(N)/a_(N)}
       E_2 = {∑_t=1^Na_t>Nd}

    Given ω∈ E_1∩ E_2, then set q(ω)∈{1,⋯,N} such that
    
    {   r_(j)>a_(j)p_N^*(d) if j < q(ω)
       r_(j)≤a_(j)p_N^*(d) if j ≥ q(ω)
       ∑_j=1^q(ω)-1a_(j)(ω)≤ Nd
       ∑_j=1^q(ω)a_(j)(ω)≥ Nd
        .

    Given J⊆[N] such that J=k. Define
    
    I(J,ω) = {i:∃ j∈ J s.t. (r_j(ω),a_j(ω))=(r_(i)(ω),a_(i)(ω)))}

    If ∑_j∈[N]\ Ja_j(ω)>(N-k)d, then set q_J(ω)∈ [N] such that
    
    {   r_(j)>a_(j)p_N,J if j < q_J(ω)
       r_(j)≤a_(j)p_N,J if j ≥q_J(ω)
       ∑_j∈ [q_J(ω)-1]\ I(J,ω)a_(j)(ω)≤ (N-k)d
       ∑_j∈ [q_J(ω)]\ I(J,ω)a_(j)(ω)≥ (N-k)d
        .

    Suppose that q(ω)-q_J(ω)≥kd/a+2, then
    
    ∑_j∈ [q_J(ω)]\ I(J,ω)a_(t)(ω)≤   ∑_j=1^q_J(ω)a_(j)(ω)
    
            =    ∑_j=1^q(ω)-1a_(j)(ω) - ∑_j=q_J(ω)+1^q(ω)-1a_(j)(ω)
    ≤    Nd - a(q(ω)-1-q_J(ω))
    ≤    (N-k)d - a

    which gives a contradiction, and implies that q(ω)-q_J(ω)<kd/a+2. On the other hand, suppose that q_J(ω)-q(ω)≥ka̅/a+2, then
    
    ∑_j∈ [q_J(ω)-1]\ I(J,ω)a_(j)(ω)≥   ∑_j=1^q_J(ω)-1a_(j)(ω)-ka̅
    
            =    ∑_j=1^q(ω)a_(j)(ω) + ∑_j=q(ω)+1^q_J'(ω)-1a_(j)(ω)-ka̅
    ≥    Nd + a(q_J(ω)-q(ω)-1) - ka̅
    ≥    Nd + a

    which gives a contradiction, and implies that q(ω)-q_J(ω)<ka̅/a+2. Thus,
    
    q(ω)-q_J(ω)<kmax{d,a̅}/a+2

    Then,
    
    {j:1{r_(j)>a_(j)p_N^*}(ω)≠1{r_(j)>a_(j)p_N,J}(ω)}
    ⊆   {q(ω)-⌈kmax{d,a̅}/a+2⌉,⋯,q(ω)+⌈2kmax{d,a̅}/a+2⌉}

    If ∑_j∈[N]\ Ja_j(ω)≤ (N-k)d, 
    
    ∑_j=q(ω)+1^Na_(j)(ω)≤   ∑_j=1^Na_(j)(ω) - ∑_j=1^q(ω)a_(j)(ω)
    ≤   ∑_j∈[N]\ Ja_j(ω) + ka̅ - Nd
    ≤   k(a̅-d)

    Thus, 
    
    q(ω)≥ N - k(a̅-d)/a

    Then, 
    
    {j:1{r_(j)>a_(j)p_N^*}(ω)≠1{r_(j)>a_(j)p_N,J}(ω)}⊆{⌈ N - k(a̅-d)/a⌉,⋯,N}

    Thus, for any J⊆[N] such that J=k,
    
    {j:1{r_(j)>a_(j)^Tp_n^*}(ω)≠1{r_(j)>a_(j)^Tp_n,J'}(ω)}
    ⊆   {q(ω)-⌈2kmax{d,a̅}/a+2⌉,⋯,q(ω)+⌈2kmax{d,a̅}/a+2⌉}∪{⌈ N - k(a̅-d)/a⌉,⋯,N}

    Set C(d,k)=k(a̅-d)/a+2kmax{d,a̅}/a+5, then
    
    I_n(d,k)(ω)≤ C(d,k)

    Under Assumption <ref> and Assumption <ref>, ℙ{E_1}=1, and
    
    ℙ{E_2}   ≥ℙ{1/n∑_t=1^na_t≥𝔼[a]-ϵ_d}
       ≥ 1-exp(-Nϵ_d^2/a̅^2)

    Then, 
    
    ℙ{I_N(d,k)≤ C(d,k)}≥ 1-exp(-Nϵ_d^2/a̅^2)

    Thus, there exists N such that, if n>N,
    
    𝔼[I_N(d,k)]
    ≤   C(d,k) + Nℙ{I_N(d,k)> C(d,k)}
    ≤   C(d,k) +1

    Take N_LOO,k=max{N,2} and C_LOO,k=sup_d∈Ω_dC(d,k)+1. Then, if N>N_LOO,k, for all d∈Ω_d,
    
    𝔼[I_N(d,k)]≤ C_LOO,k




 §.§ Proof of Theorem <ref>


Fix an integer B such that

    B>Λ_DA=max{a̅/d,N_Dual,N_LOO,1,N_LOO,2}

Fix δ_d such that

    [d_0-δ_d,d_0+δ_d]⊆Ω_d

Let d_k=b_t_k/n-t_k, and define 

    κ̅=min{K}∪{k:d_k∉[d-δ_d,d+δ_d]}

To simplify the notation, we write x^π_1_j as x_j, and p^* in this section refers to the population dual price evaluated at the initial average resource d_0, i.e., p^*=p^*(d_0). The total reward given by π_1 can be written as

    𝔼[R_n(π_1)]
        =   𝔼[∑_k=1^K∑_t=t_k-1+1^t_kr_tx_t]
    
        =   𝔼[∑_k=1^K∑_t=t_k-1+1^t_kr_tx_t+(b_0-∑_k=1^K∑_t=t_k-1+1^t_ka_tx_t)p^*]-𝔼[b_np^*]
    
        =   𝔼[b_0p^*+∑_k=1^K∑_t=t_k-1+1^t_k(r_t-a_tp^*)x_t]-𝔼[b_np^*]

The second line is by the definition of b_n, which is the remaining resource at the end of the planning horizon. Since p^* is a feasible solution to the offline problem, we have the following upper bound of the offline optimal total reward.

    𝔼[R_n^*]   ≤𝔼[b_0p^*+∑_t=1^n(r_t-a_tp^*)^+]
       =𝔼[b_0p^*+∑_k=1^K∑_t=t_k-1+1^t_k(r_t-a_tp^*)1{r_t>a_tp^*}]

Then, we have the following generic regret upper bound

    𝔼[R_n^*-R_n(π_1)]
    ≤   𝔼[∑_k=1^K∑_t=t_k-1+1^t_k(r_t-a_tp^*)(1{r_t>a_tp^*}-x_t)]+𝔼[b_np^*]
    ≤   𝔼[∑_k=1^κ̅-1∑_t=t_k-1+1^t_k(r_t-a_tp^*)(1{r_t>a_tp^*}-1{r_t>a_tp_k})]
       +(r̅+a̅r̅/d)(t_K-1-t_κ̅-1)
       +𝔼[∑_t=t_K-1+1^n(r_t-a_tp^*)(1{r_t>a_tp^*}-1{r_t>a_tp_K})]
       +𝔼[b_np^*]

(<ref>) comes from the fact that the dual decision rule can be applied to customers 1,⋯,t_κ̅-1 with probability 1, which is because the remaining resource after making decisions on customer t_κ̅-1 is at least a̅ with probability 1. (<ref>) is by Assumption <ref>(b) and Lemma <ref>(a). In the following, we analyze each of the four components on the right-hand-side of the above inequality. For (<ref>), define

    d_k = d_k1{k<κ̅} + d_01{k≥κ̅}  ∀ k

and

    p_t,1=min_p≥ 0d_0p+1/t_1-1∑_j=1^t-1(r_j-a_jp)^++1/t_1-1∑_j=t+1^t_1(r_j-a_jp)^+  ∀ t=1,⋯,t_1

and

    p'_k = min_p≥ 0d_k-1p +1/t_k-1∑_t=1^t_k-1(r_t-a_tp)^+  ∀ k=2,⋯, K-1

Then, we can upper bound (<ref>) by

    𝔼[∑_k=1^κ̅-1∑_t=t_k-1+1^t_k(r_t-a_tp^*)(1{r_t>a_tp^*}-1{r_t>a_tp_k})]
    (i)≤   𝔼[∑_t=1^t_1(r_t-a_tp^*)(1{r_t>a_tp^*}-1{r_t>a_tp_t,1})+∑_k=2^K-1∑_t=t_k-1+1^t_k(r_t-a_tp^*)(1{r_t>a_tp^*}-1{r_t>a_tp'_k})]
       +𝔼[∑_t=1^t_1(r_t-a_tp^*)(1{r_t>a_tp_t,1}-1{r_t>a_tp_1})]
    (ii)≤   βa̅^2(∑_t=1^t_1𝔼[p_t,1-p^*^2]+B∑_k=2^K-1𝔼[p'_k-p^*^2])+(r̅+a̅r̅/d)ℙ{1{r_t>a_tp_t,1}≠1{r_t>a_tp_1}}
    (iii)≤   βa̅^2(C_Dual(2+∑_k=2^K-11/k-1)+LB∑_k=1^K-1𝔼[d_k-d_0^2]+2√(C_Dual(2+∑_k=2^K-11/k-1))√(LB∑_k=1^K-1𝔼[d_k-d_0^2]))
       +C_LOO,1(r̅+a̅r̅/d)

(i) is by the fact that

    (r_t-a_tp^*)(1{r_t>a_tp^*}-1{r_t>a_tp})≥ 0 a.s.  ∀ p
 
(ii) is by Lemma <ref>(a), and (iii) is by Lemma <ref>, Lemma <ref>(b) and Cauchy Inequality. 

In addition, define d'_0=d_0, and for 1≤ k≤ K-2,

    d'_k+1 = (n-t_k)d'_k-∑_t=t_k+1^t_k+1a_t1{r_t>a_tp_k+1}/n-t_k+11{k<κ̅}+d'_k1{k≥κ̅}

This definition implies that d'_k=d_k if k<κ̅. Then, d_k-d_0≤d'_k-d_0 almost sure. Thus, by Lemma <ref>,

    B∑_k=1^K-1𝔼[d_k-d_0^2]≤ B∑_k=1^K-1𝔼[d'_k-d_0^2]≤ C_Reslog K

Combine (<ref>) and (<ref>), we can conclude that 

    𝔼[∑_k=1^κ̅-1∑_t=t_k-1+1^t_k(r_t-a_tp^*)(1{r_t>a_tp^*}-1{r_t>a_tp_k})]≤ O(log K)

For (<ref>),

    ℙ{κ̅>k}   ≥ℙ{d'_k-d≤δ_d}
       ≥ 1-𝔼[d'_k-d^2]/δ_d^2

Then, by Lemma <ref>

    𝔼[K-κ̅]=   (K-1)-∑_k=1^K-1ℙ{κ̅>k}
    ≤   ∑_k=1^K-1𝔼[d'_k-d^2]/δ_d^2
    ≤   C_Reslog(K)/δ_d^2B

Thus,

    (r̅+a̅r̅/d)(t_K-1-t_κ̅-1)≤(r̅+a̅r̅/d)𝔼[K-κ̅]B≤ O(log K)

For (<ref>), define

    p_t,K(d) = min_p≥ 0 (n-t_K-1-1)dp + ∑_j=t_K-1+1^t-1(r_j-a_jp)^+ + ∑_j=t+1^n(r_j-a_jp)^+

Then,

    𝔼[∑_t=t_K-1+1^n(r_t-a_tp^*)(1{r_t>a_tp^*}-1{r_t>a_tp_K})]
    
            =   𝔼[∑_t=t_K-1+1^n(r_t-a_tp^*)(1{r_t>a_tp^*}-1{r_t>a_tp_t,K(d_K-1)})]
       +𝔼[∑_t=t_K-1+1^n(r_t-a_tp^*)(1{r_t>a_tp_K}-1{r_t>a_tp_t,K(d_K-1)})]
    ≤   βa̅^2∑_t=t_K-1+1^n𝔼[p_t,K(d_K-1)-p^*^2]+(r̅+a̅r̅/d)∑_t=t_K-1+1^nℙ{1{r_t>a_tp_K}≠1{r_t>a_tp_t,K(d_K-1)}}

The last line is by Lemma <ref>(a). In addition, by Lemma <ref>(c) and Lemma <ref>,

    𝔼[p_t,K(d_K-1)-p^*^2]
    ≤   4r̅^2/(min{d,a})^2ℙ{κ̅<K}+C_Dual/B-1+L𝔼[d'_K-1-d_0^2}]+2√(C_Dual/B-1)√(L𝔼[d'_K-1-d_0^2])
    ≤   4r̅^2/(min{d,a})^2C_Reslog K/Bδ_d^2+C_Dual/B-1+LC_Reslog K/B+2√(C_Dual/B-1)√(LC_Reslog K/B)

By Lemma <ref>,

    ℙ{1{r_t>a_tp_K}≠1{r_t>a_tp_t,K(d_K-1)}}
    ≤   ℙ{κ̅<K}+𝔼[𝔼[1{1{r_t>a_tp_K}≠1{r_t>a_tp_t,K(d_K-1)}}| d_K-1]1{κ̅=K}]
    ≤   C_Reslog K/Bδ_d^2+C_LOO,1/B

Thus,

    𝔼[∑_t=t_K-1+1^n(r_t-a_tp^*)(1{r_t>a_tp^*}-1{r_t>a_tp_K})]
    ≤   βa̅^2B(4r̅^2/d^2C_Reslog K/Bδ_d^2+C_Dual/B-1+LC_Reslog K/B+2√(C_Dual/B-1)√(LC_Reslog K/B))
       +(r̅+a̅r̅/d)B(C_Reslog K/Bδ_d^2+C_LOO,1/B)
    ≤   O(log K)

For (<ref>),

    𝔼[b_n]=𝔼[b_n1{κ̅=K}]+𝔼[b_n1{κ≤ K-1}]

Define

    A̅=∑_t=t_K-1+1^na_t/n-t_K-1

Then,

    𝔼[b_n1{κ̅=K}]=𝔼[b_n1{κ̅=K}1{A̅>d'_K-1}]+𝔼[b_n1{κ̅=K}1{A̅≤ d'_K-1}]

On the intersection of {κ̅=K} and {A̅>d'_K-1}, the remaining resource at the beginning of the last batch is not sufficient for accepting of the orders in the last batch. Since Algorithm <ref> makes decisions on customers arriving in the last batch by solving an offline problem, the remaining resource in the end should be smaller that a̅, thus,

    𝔼[b_n1{κ̅=K}1{A̅>d'_K-1}]≤a̅

By Hoeffding's Inequality,

    𝔼[b_n1{κ̅=K}1{A̅≤ d'_K-1}]
    ≤   (d_0+δ_d)Bℙ{A̅≤ d_0+δ_d}
    ≤   (d_0+δ_d)Bexp(-2Bϵ_d^2/a̅^2)

Thus,

    𝔼[b_n1{κ̅= K}]≤ O(1)

In addition,

    𝔼[b_n1{κ̅≤ K-1}]
    ≤   𝔼[(K-κ̅)Bd_κ̅1{κ̅≤ K-1}]

and

    d_κ̅1{κ̅≤ K-1}   =b_t_κ̅/(K-κ̅)B1{κ̅≤ K-1}
       ≤b_t_κ̅-1/(K-κ̅)B1{κ̅≤ K-1}
       ≤(d+δ_d)(K-κ̅+1)B/(K-κ̅)B1{κ̅≤ K-1}
       ≤ 2(d+δ_d)

Then,

    𝔼[b_n1{κ≤ K-1}]
    ≤   2(d+δ_d)B𝔼[K-κ̅]
    ≤   O(log K)

Thus, 

    𝔼[b_np^*]≤r̅/d𝔼[b_n]≤ O(log K)

Put (<ref>), (<ref>), (<ref>), (<ref>) together, we can conclude that if B>Λ_DA,

    𝔼[R_n^*-R_n(π_1)]≤ O(log K)

which completes the proof of Theorem <ref>.



  §.§.§ Proof of Lemma <ref>


Fix an integer B such that

    B>Λ_Res=max{a̅/d,N_Dual,N_LOO,1,N_LOO,2}

By definition,

    d'_k+1-d_0   =d'_k-d_0+∑_t=t_k+1^t_k+1d'_k-a_t1{r_t>a_tp_k+1}/n-t_k+11{k<κ̅}
       =d'_k-d_0+∑_t=t_k+1^t_k+1d'_k-a_t1{r_t>a_tp^*(d'_k)}/n-t_k+11{k<κ̅}
       +∑_t=t_k+1^t_k+1a_t(1{r_t>a_tp^*(d'_k)}-1{r_t>a_tp_k+1})/n-t_k+11{k<κ̅}

Thus,

    𝔼[(d'_k+1-d_0)^2]
    
        =   𝔼[(d'_k-d_0)^2]
       +𝔼[(∑_t=t_k+1^t_k+1d'_k-a_t1{r_t>a_tp^*(d'_k)})^2/(n-t_k+1)^21{k<κ̅}]
       +𝔼[(∑_t=t_k+1^t_k+1a_t(1{r_t>a_tp^*(d'_k)}-1{r_t>a_tp_k+1}))^2/(n-t_k+1)^21{k<κ̅}]
       +2𝔼[(d'_k-d_0)(∑_t=t_k+1^t_k+1d'_k-a_t1{r_t>a_tp^*(d'_k)}/n-t_k+1)1{k<κ̅}]
       +2𝔼[(d'_k-d_0)(∑_t=t_k+1^t_k+1a_t(1{r_t>a_tp^*(d'_k)}-1{r_t>a_tp_k+1})/n-t_k+1)1{k<κ̅}]
       +2𝔼[(∑_t=t_k+1^t_k+1d'_k-a_t1{r_t>a_tp^*(d'_k)}/n-t_k+1)(∑_t=t_k+1^t_k+1a_t(1{r_t>a_tp^*(d'_k)}-1{r_t>a_tp_k+1})/n-t_k+1)1{k<κ̅}]

In the next, we analyze the right-hand-side of the above equation term by term. First, for (<ref>), 

    𝔼[(∑_t=t_k+1^t_k+1d'_k-a_t1{r_t>a_tp^*(d'_k)})^2/(n-t_k+1)^21{k<κ̅}]
    
        =   1/(n-t_k+1)^2𝔼[(∑_i=t_k+1^t_k+1∑_j=t_k+1^t_k+1𝔼[(d'_k-a_i1{r_i>a_ip^*(d'_k)})(d'_k-a_j1{r_j>a_jp^*(d'_k)})| d'_k])1{k<κ̅}]

For i≠ j,

    𝔼[(d'_k-a_i1{r_i>a_ip^*(d'_k)})(d'_k-a_j1{r_j>a_jp^*(d'_k)})| d'_k]1{k<κ̅}
    
        =   𝔼[(d'_k-a_i1{r_i>a_ip^*(d'_k)})1{k<κ̅}| d'_k]𝔼[(d'_k-a_j1{r_j>a_jp^*(d'_k)})1{k<κ̅}| d'_k]
    
        =   0

Thus,

    𝔼[(∑_t=t_k+1^t_k+1d'_k-a_t1{r_t>a_tp^*(d'_k)})^2/(n-t_k+1)^21{k<κ̅}]
    
        =   1/(n-t_k+1)^2𝔼[(∑_t=t_k+1^t_k+1𝔼[(d'_k-a_t1{r_t>a_tp^*(d'_k)})^2| d'_k])1{k<κ̅}]
    ≤   (t_k+1-t_k)(d̅+a̅)^2/(n-t_k+1)^2
    
        =   (d̅+a̅)^2/(K-k-1)^2B

For (<ref>),

    𝔼[(∑_t=t_k+1^t_k+1a_t(1{r_t>a_tp^*(d'_k)}-1{r_t>a_tp_k+1}))^2/(n-t_k+1)^21{k<κ̅}]
    
        =   1/(n-t_k+1)^2𝔼[(∑_i=t_k+1^t_k+1∑_j=t_k+1^t_k+1𝔼[(a_i(1{r_i>a_ip^*(d'_k)}-1{r_i>a_ip_k+1}))·...
       ...(a_j(1{r_j>a_jp^*(d'_k)}-1{r_j>a_jp_k+1}))| d'_k])1{k<κ̅}]

If i=j,

    𝔼[(a_i(1{r_i>a_ip^*(d'_k)}-1{r_i>a_ip_k+1}))(a_j(1{r_j>a_jp^*(d'_k)}-1{r_j>a_jp_k+1}))| d'_k]≤a̅^2

If i≠ j, for the case k=0, define

    p_i,j = min_p≥ 0d'_0p+1/t_1-2∑_j∈{1,⋯,t_1}\{i,j}(r_j-a_jp)^+

Then,

    𝔼[(a_i(1{r_i>a_ip^*(d'_0)}-1{r_i>a_ip_1}))(a_j(1{r_j>a_jp^*(d'_0)}-1{r_j>a_jp_1}))| d'_0]
    ≤   𝔼[(a_i(1{r_i>a_ip^*(d'_0)}-1{r_i>a_ip_i,j}))(a_j(1{r_j>a_jp^*(d'_0)}-1{r_j>a_jp_i,j}))]
       +a̅^2𝔼[1{1{r_i>a_ip_1}≠1{r_i>a_ip_i,j}}+1{1{r_j>a_jp_1}≠1{r_j>a_jp_i,j}}]
    ≤   β^2a̅^4𝔼[p_i,j-p^*(d'_0)^2]+a̅^2𝔼[1{1{r_i>a_ip_1}≠1{r_i>a_ip_i,j}}+1{1{r_j>a_jp_1}≠1{r_j>a_jp_i,j}}]
    ≤   β^2a̅^4C_Dual/B-2+2a̅^2C_LOO,2/B

The second last line is by Lemma <ref>(d), and the last line is by Lemma <ref> and Lemma <ref>. For the case k≥ 1, by Lemma <ref>(d)

    𝔼[(a_i(1{r_i>a_ip^*(d'_k)}-1{r_i>a_ip_k+1))(a_j(1{r_j>a_jp^*(d'_k)}-1{r_j>a_jp_k+1))| d'_k]
    ≤   𝔼[β^2a̅^4p_k+1-p^*(d'_k)}^2| d'_k]

To summarize, when k=0,

    𝔼[(∑_t=t_k+1^t_k+1a_t(1{r_t>a_tp^*(d'_k)}-1{r_t>a_tp_k+1}))^21{k<κ̅}]
    ≤   Ba̅^2+B^2(4β^2a̅^4C_Dual/B-2+2a̅^2C_LOO,2/B)
    ≤   B(a̅^2+12β^2a̅^4C_Dual+2a̅^2C_LOO,2)

When k≥ 1,

    𝔼[(∑_t=t_k+1^t_k+1a_t(1{r_t>a_tp^*(d'_k)}-1{r_t>a_tp_k+1}))^21{k<κ̅}]
    ≤   𝔼[(Ba̅^2+B^2𝔼[β^2a̅^4p_k+1-p^*(d'_k)}^2| d'_k])1{k<κ̅}]
    ≤   Ba̅^2+B^2𝔼[β^2a̅^4p_k+1-p^*(d'_k)}^21{k<κ̅}]
    ≤    Ba̅^2+B^2𝔼[β^2a̅^4sup_d∈Ω_dp_kB^*(d)-p^*(d)}^2]
    ≤    Ba̅^2+β^2a̅^4B^2C_Dual/kB
    ≤   B(a̅^2+β^2a̅^4C_Dual)

Thus, there exists a constant C_res,1 such that

    𝔼[(∑_t=t_k+1^t_k+1a_t(1{r_t>a_tp^*(d'_k)}-1{r_t>a_tp_k+1}))^2/(n-t_k+1)^21{k<κ̅}]
    ≤   C_Res,1/(K-k-1)^2B

For (<ref>),

    2𝔼[(d'_k-d)(∑_t=t_k+1^t_k+1d'_k-a_t1{r_t>a_tp^*(d'_k)}/n-t_k+1)1{k<κ̅}]
    ≤   2𝔼[(d'_k-d)1{k<κ̅}1/n-t_k+1∑_t=t_k+1^t_k+1𝔼[d'_k-a_t1{r_t>a_tp^*(d'_k)}| d'_k]]
    
        =    0

For (<ref>), when k=0, d'_0 = d_0, then (<ref>) equals 0. When k≥ 1,

    2𝔼[(d'_k-d_0)(∑_t=t_k+1^t_k+1a_t(1{r_t>a_tp^*(d'_k)}-1{r_t>a_tp_k+1})/n-t_k+1)1{k<κ̅}]
    
        =    2/n-t_k+1∑_t=t_k+1^t_k+1𝔼[(d'_k-d_0)1{k<κ̅}a_t(1{r_t>a_tp^*(d'_k)}-1{r_t>a_tp_k+1})]

and

    𝔼[(d'_k-d_0)1{k<κ̅}a_t𝔼[(1{r_t>a_tp^*(d'_k)}-1{r_t>a_tp_k+1})| a_t,p_k+1,d'_k]]
    ≤   βa̅^2𝔼[(d'_k-d)1{k<κ̅}p_t,k-p^*(d'_k)]
    ≤   βa̅^2√(𝔼[(d'_k-d)^2])√(𝔼[p_k+1-p^*(d'_k)^21{k<κ̅}])
    ≤   βa̅^2√(C_Dual)/√(kB)√(𝔼[(d'_k-d)^2])
    ≤   √(2)βa̅^2√(C_Dual)/√((k+1)B)√(𝔼[(d'_k-d)^2])

Thus, there exists a constant C_Res,2 such that

    2𝔼[(d'_k-d)(∑_t=t_k+1^t_k+1a_t(1{r_t>a_tp^*(d'_k)}-1{r_t>a_tp_k+1})/n-t_k+1)1{k<κ̅}]
    ≤   √(C_Res,2)/(K-k-1)√(k+1)√(B)√(𝔼[(d'_k-d)^2])

For (<ref>), by Cauchy-Schwarz Inequality and the upper bounds for (<ref>) and (<ref>), there exists a constant C_Res,3 such that

    2𝔼[(∑_t=t_k+1^t_k+1d'_k-a_t1{r_t>a_tp^*(d'_k)}/n-t_k+1)(∑_t=t_k+1^t_k+1a_t(1{r_t>a_tp^*(d'_k)}-1{r_t>a_tp_k+1})/n-t_k+1)1{k<κ̅}]
    ≤   2√((d̅+a̅)^2/(K-k-1)^2B)√(C_Res,1/(K-k-1)^2B)
    ≤   C_Res,3/(K-k-1)^2B

Thus, there exists a constant C_Res,0 such that

    𝔼[d'_k+1-d_0^2]≤𝔼[d'_k-d_0^2] + C_Res,0/(K-k-1)^2B+√(C_Res,0)/(K-k-1)√(k+1)√(B)√(𝔼[d'_k-d_0^2])  ∀ k=0,⋯,K-2

By Lemma <ref>, there exists a constant C_Res=8C_Res,0 such that

    ∑_k=1^K-1𝔼[d'_k-d_0^2]≤C_Reslog K/B



    If z_0=0, and
    
    z_k+1≤ z_k + C/(K-k-1)^2+√(C)/(K-k-1)√(k+1)√(z_k)  ∀ k=0,⋯,K-2

    Then,
    
    ∑_k=1^K-1z_k≤ 8Clog K



    It suffices to show the result under the following assumption.
    
    z_k+1=z_k+C(1/(K-k-1)^2)+√(C)√(z_k)/(K-k-1)√(k+1)

    Then,
    
    ∑_k=0^K-2(K-k-1)(z_k+1-z_k)=   C∑_k=0^K-2(1/K-k-1)+√(C)∑_k=0^K-2√(z_k)/√(k+1)
    ≤   4Clog(K)+√(C)∑_k=0^K-2√(z_k)/√(k+1)

    and 
    
    ∑_k=0^K-2(K-k-1)(z_k+1-z_k)=∑_k=1^K-1z_k

    In addition,
    
    (√(C)∑_k=0^K-2√(z_k)/√(k+1))^2≤    C(∑_k=0^K-2z_k)(∑_k=0^K-21/k+1)
    ≤    2Clog(K)(∑_k=1^K-1z_k)
    ≤   2Clog(K)(4Clog(K)+√(C)∑_k=0^K-2√(z_k)/√(k+1))

    This implies that
    
    √(C)∑_k=0^K-2√(z_k)/√(k+1)≤ 4Clog(K)

    Thus,
    
    ∑_k=1^K-1z_k≤ 8Clog(K)



     

    (a) If p(r,a), then
    
    𝔼[(r-ap^*)(1{r>ap^*}-1{r>ap})]≤βa̅^2𝔼[p-p^*^2]

    (b) Let d be a random variable such that ℙ{d∈Ω_d}=1, then
    
    𝔼[p_n^*(d)-p^*(d_0)^2]≤C_Dual/n+L𝔼[d-d_0^2]+2√(C_Dual/n)√(L𝔼[d-d_0^2])

    (c) 
    
    𝔼[p_t,K(d_K-1)-p^*(d_0)^2]
    ≤   4r̅^2/(min{d,a})^2ℙ{κ̅<K}+C_Dual/B-1+L𝔼[d'_K-1-d_0^2]+2√(C_Dual/B-1)√(L𝔼[d'_K-1-d_0^2])

    (d) If (r_1,a_1) (r_2,a_2) (p,d), then
    
    𝔼[(a_1(1{r_1>a_1p^*(d)}-1{r_1>a_1p}))(a_2(1{r_2>a_2p^*(d)}-1{r_2>a_2p}))|d]
    ≤   β^2a̅^4𝔼[p-p^*(d)^2|d]



    For part (a),
    
    𝔼[(r-ap^*)(1{r>ap^*}-1{r>ap})]
    ≤   𝔼[a(p-p^*)1{ap^*<r≤ ap}1{ap^*<ap}]
       +𝔼[a(p^*-p)1{ap<r≤ ap^*}1{ap<ap^*}]

    By Assumption <ref> and the independence between (r,a) and p,
    
    𝔼[a(p-p^*)1{ap^*<r≤ ap}1{ap^*<ap}]
    
                =   𝔼[a(p-p^*)𝔼[1{ap^*<r≤ ap}| a,p]1{ap^*<ap}]
    ≤   βa̅^2𝔼[p-p^*^21{ap^*<ap}]

    Similarly,
    
    𝔼[a(p^*-p)1{ap<r≤ ap^*}1{ap<ap^*}]≤βa̅^2𝔼[p-p^*^21{ap<ap^*}]

    Thus,
    
    𝔼[(r-ap^*)(1{r>ap^*}-1{r>ap})]≤βa̅^2𝔼[p-p^*^2]

    For part (b),
    
    𝔼[p_n^*(d)-p^*(d_0)^2]
    ≤   𝔼[p_n^*(d)-p^*(d)^2] + 𝔼[p^*(d)-p^*(d_0)^2]+2√(𝔼[p_n^*(d)-p^*(d)^2])√(𝔼[p^*(d)-p^*(d_0)^2])
    ≤   𝔼[sup_d∈Ω_dp_n^*(d)-p^*(d)^2] + 𝔼[p^*(d)-p^*(d_0)^2]+2√(𝔼[sup_d∈Ω_dp_n^*(d)-p^*(d)^2])√(𝔼[p^*(d)-p^*(d_0)^2])
    ≤   C_Dual/n+L𝔼[d-d_0^2]+2√(C_Dual/n)√(L𝔼[d-d_0^2])

    where the last line is by Lemma <ref> and Lemma <ref>(b), and the proof is complete.
     

    For part (c),
    
    𝔼[p_t,K(d_K-1)-p^*(d_0)^2]
    
                =   𝔼[p_t,K(d_K-1)-p^*(d_0)^21{κ̅<K}]+𝔼[p_t,K(d_K-1)-p^*(d_0)^21{κ̅=K}]

    By Assumption <ref>(b) and Assumption <ref>(a), we can assume p_t,K(d_K-1)≤r̅/a with probability 1. Then,
    
    𝔼[p_t,K(d_K-1)-p^*(d_0)^21{κ̅<K}]≤4r̅^2/(min{d,a})^2ℙ{κ̅<K}

    In addition, following similar proof of part (b),
    
    𝔼[p_t,K(d_K-1)-p^*(d_0)^21{κ̅=K}]
    ≤   𝔼[p_t,K(d_K-1)-p^*(d_K-1)^21{κ̅=K}] + 𝔼[p^*(d_K-1)-p^*(d_0)^21{κ̅=K}]
       +2√(𝔼[p_t,K(d_K-1)-p^*(d_K-1)^21{κ̅=K}])√(𝔼[p^*(d_K-1)-p^*(d_0)^21{κ̅=K}])
    ≤   𝔼[sup_d∈Ω_dp_t,K(d)-p^*(d)^2] + L𝔼[d_K-1-d_0^21{κ̅=K}]
       +2√(𝔼[sup_d∈Ω_dp_t,K(d)-p^*(d)^2])√(L𝔼[d_K-1-d_0^21{κ̅=K}])
    ≤   C_Dual/B-1+L𝔼[d'_K-1-d_0^2}]+2√(C_Dual/B-1)√(L𝔼[d'_K-1-d_0^2])

    where the last line is by Lemma <ref>, Lemma <ref>(b), and the fact that 
    
    d_K-1-d_01{κ̅=K}≤d'_K-1-d_0   a.s.

    The proof is complete.
     

    For part(d),
    
    𝔼[(a_1(1{r_1>a_1p^*(d)}-1{r_1>a_1p}))(a_2(1{r_2>a_2p^*(d)}-1{r_2>a_2p}))|d]
    
                =   𝔼[𝔼[(a_1(1{r_1>a_1p^*(d)}-1{r_1>a_1p}))(a_2(1{r_2>a_2p^*(d)}-1{r_2>a_2p}))| a_1,a_2,p,d]|d]
    
                =   𝔼[a_1𝔼[(1{r_1>a_1p^*(d)}-1{r_1>a_1p})| a_1,a_2,p,d]a_2𝔼[(1{r_2>a_2p^*(d)}-1{r_2>a_2p})| a_1,a_2,p,d]|d]
    ≤   β^2a̅^4𝔼[p-p^*(d)^2|d]




 §.§ Proof of Theorem <ref>

Following similar notation in the upper bound proof, we denote d_k to be the average remaining resource after making allocations to first k batches using policy π. Let x_j^π be the decision made by π when the initial total resource is b_0, and

    R_n-kB(π) = ∑_j=t_k+1^nr_jx_j^π

Let R_n-kB^*(d), x_k+1,j^*(d), p_kB:n(d) be the optimal objective value, the optimal primal solution and the optimal dual solution corresponding to the resource constraint of the following problem.

    [                         max           ∑_j=t_k+1^nr_jx_j;                         S.T ∑_j=t_k+1^na_jx_j≤ (n-t_k)d;                                   0≤ x_j≤ 1  ∀ t_k<j≤ n ]

Define

    d_k+1^*(d) = 1/n-(k+1)B((n-kB)d-∑_j=t_k+1^t_k+1a_jx_k+1,j^*(d))

Then,

    R_n^*-R_n(π)
    
            =   ∑_j=1^t_1r_jx_1,j^*(d_0)+R_n-B^*(d_1^*(d_0))-∑_j=1^t_1r_jx_j^π-R_n-B(π)
    
            =   ∑_j=1^t_1r_j(x_1,j^*(d_0)-x_j^π)+R_n-B^*(d_1^*(d_0))-R_n-B^*(d_1)+R_n-B^*(d_1)-R_n-B(π)

By LP Duality Theory,

    R_n-B^*(d_1^*(d_0))-R_n-B^*(d_1)   ≥ (n-B)(d_1^*(d_0)-d_1)p_B:n^*(d_1^*(d_0))
       =∑_j=1^t_1(x_j^π-x_1,j^*(d_0))a_jp_B:n^*(d_1^*(d_0))

Then,

    R_n^*-R_n(π)
    ≥   ∑_j=1^t_1(r_j-a_jp_B:n^*(d_1^*(d_0)))(x_1,j^*(d_0)-x_j^π)+R_n-B^*(d_1)-R_n-B(π)

Inductively,

    R_n^*-R_n(π)
    ≥   ∑_k=1^K-1∑_j=t_k-1+1^t_k(r_j-a_jp^*_kB:n(d_k^*(d_k-1)))(x_k,j^*(d_k-1)-x_j^π)+R^*_n-(K-1)B(d_K-1)-R_n-(K-1)B(π)
    ≥   ∑_k=1^K-1∑_j=t_k-1+1^t_k(r_j-a_jp^*_kB:n(d_k^*(d_k-1)))(x_k,j^*(d_k-1)-x_j^π)
    
            =   ∑_k=1^K-1∑_j=t_k-1+1^t_k(r_j-a_jp^*_kB:n(d_k^*(d_k-1)))(1{r_j>a_jp^*_kB:n(d_k^*(d_k-1))}-x_j^π)

where the last line comes from the fact that

    1{r_j>a_jp^*_kB:n(d_k^*(d_k-1))}=1{r_j>a_jp^*_(k-1)B:n(d_k-1)}

Given k<K-1, define

    {x_j^π(d)}_j=t_k+1^t_k+1=max_x_t_k+j∈[0,1] ∀ 1≤ j≤ B∑_j=t_k+1^t_k+1a_jx_j≤ (n-t_k)d∑_j=t_k+1^t_k+1r_jx_j+V_k+1((n-t_k)d-∑_j=t_k+1^t_k+1a_jx_j)

In other words, x_j^π=x_j^π(d_k) for any t_k<j≤ t_k+1. Fix δ such that

    {   [d-2δ,d+2δ]⊆Ω_d
       δ<δ_p/2√(L)
       δ<δ̅
       δ<min{δ_4,1,δ_-4,1}/2.

where δ_p,δ̅,δ_4,1,δ_-4,1 are given in Lemma <ref>(c), Lemma <ref> and Lemma <ref>. Then, fix k<K-1 and t_k<j≤ t_k+1,

    𝔼[(r_j-a_jp^*_(k+1)B:n(d_k+1^*(d_k)))(1{r_j>a_jp^*_(k+1)B:n(d_k+1^*(d_k))}-x_j^π)]
    ≥   𝔼[(r_j-a_jp^*_(k+1)B:n(d_k+1^*(d_k)))(1{r_j>a_jp^*_(k+1)B:n(d_k+1^*(d_k))}-x_j^π)1{d_k∈ [d_0-δ,d_0+δ]}]
    
            =   𝔼[𝔼[(r_j-a_jp^*_(k+1)B:n(d_k+1^*(d_k)))(1{r_j>a_jp^*_(k+1)B:n(d_k+1^*(d_k))}-x_j^π)| d_k]1{d_k∈ [d_0-δ,d_0+δ]}]
    ≥   ℙ{d_k∈ [d_0-δ,d_0+δ]}}inf_d∈[d_0-δ,d_0+δ]𝔼[(r_j-a_jp^*_(k+1)B:n(d_k+1^*(d)))(1{r_j>a_jp^*_(k+1)B:n(d_k+1^*(d))}-x_j^π(d))]

Given d∈ [d_0-δ,d_0+δ],

    𝔼[(r_j-a_jp^*_(k+1)B:n(d_k+1^*(d)))(1{r_j>a_jp^*_(k+1)B:n(d_k+1^*(d))}-x_j^π(d))]
    
            =   𝔼[(1-x_j^π(d))(r_j-a_jp^*_(k+1)B:n(d_k+1^*(d)))1{r_j>a_jp^*_(k+1)B:n(d_k+1^*(d))}]
       +𝔼[x_j^π(d)(a_jp^*_(k+1)B:n(d_k+1^*(d))-d_j)1{r_j≤ a_jp^*_(k+1)B:n(d_k+1^*(d))}]

Define

    p_j,kB:n(d) = min_p≥ 0 (n-KB)dp + ∑_t=t_k+1^j-1(r_t-a_tp)^+ + ∑_t=j+1^n(r_t-a_tp)^+

and

    d_j,k+1(d) = 1/(K-k-1)B((K-k)Bd-∑_t=t_k+1^j-1a_t1{r_t≥ a_tp_j,kB:n(d)}-∑_t=j+1^t_k+1a_t1{r_t≥ a_tp_j,kB:n(d)}-a̅)

Then, d_j,k+1(d)≤ d^*_k+1(d) almost sure. Thus, p^*_(k+1)B:n(d_k+1^*(d))≤ p^*_(k+1)B:n(d_j,k+1(d)) almost sure. Then, for the first term on the right-hand-side of (<ref>),

    𝔼[(1-x_j^π(d))(r_j-a_jp^*_(k+1)B:n(d_k+1^*(d)))1{r_j>a_jp^*_(k+1)B:n(d_k+1^*(d))}] 
    ≥   𝔼[(1-x_j^π(d))(r_j-a_jp^*_(k+1)B:n(d_j,k+1(d))1{r_j>a_jp^*_(k+1)B:n(d_j,k+1(d))}]

Define

    E_1,1(d,k,j) = {d_j,k+1(d)-d≤1/√(L(K-k-1)B)}
       E_1,2(d,k,j) = {√((K-k-1)B)(p^*_(k+1)B:n(d_j,k+1(d))-p^*(d_j,k+1(d))+4≤ 1}

Then, on the event E_1,1(d,k,j)∩ E_1,2(d,k,j)

    p^*_(k+1)B:n(d_j,k+1(d))-p^*(d) =    (p^*_(k+1)B:n(d_j,k+1(d))-p^*(d_j,k+1(d)))+(p^*(d_j,k+1(d))-p^*(d))
    ≤   -2/√((K-k-1)B)

Then,

    𝔼[(1-x_j^π(d))(r_j-a_jp^*_(k+1)B:n(d_j,k+1(d))1{r_j>a_jp^*_(k+1)B:n(d_j,k+1(d))}] 
    ≥   𝔼[1_E_1,1(d,k,j)1_E_1,2(d,k,j)(1-x_j^π(d))(r_j-a_j(p^*(d)-2/√((K-k-1)B)))1{r_j>a_j(p^*(d)-2/√((K-k-1)B))}]

In addition,

    1{r_j≥ a_j(p^*(d)-1/√((K-k-1)B))}(r_j-a_j(p^*(d)-2/√((K-k-1)B)))1{r_j>a_j(p^*(d)-2/√((K-k-1)B))}
       ≥1{r_j≥ a_j(p^*(d)-1/√((K-k-1)B))}a_j/√((K-k-1)B)
       ≥(1{r_j> a_j(p^*(d)-1/√((K-k-1)B))}-1{r_j> a_j(p^*(d)+1/√((K-k-1)B))})a_j/√((K-k-1)B)

Thus,

    𝔼[(1-x_j^π(d))(r_j-a_jp^*_(k+1)B:n(d_k+1^*(d)))1{r_j>a_jp^*_(k+1)B:n(d_k+1^*(d))}] 
    ≥   𝔼[1_E_1,1(d,k,j)1_E_1,2(d,k,j)(1-x_j^π(d))(1{r_j> a_j(p^*(d)-1/√((K-k-1)B))}..
                           -..1{r_j> a_j(p^*(d)+1/√((K-k-1)B))})a_j/√((K-k-1)B)]

Define

    d̂_j,k+1(d) = 1/(K-k-1)B((K-k)Bd-∑_t=t_k+1^j-1a_t1{r_t≥ a_tp_j,kB:n(d)}-∑_t=j+1^t_k+1a_t1{r_t≥ a_tp_j,kB:n(d)}+a̅^2/a)

Then, d_j,k+1(d)≥ d^*_k+1(d) almost sure. Thus, p^*_(k+1)B:n(d_k+1^*(d))≥ p^*_(k+1)B:n(d_j,k+1(d)) almost sure. Then, for the second term on the right-hand-side of (<ref>),

    𝔼[x_j^π(d)(a_jp^*_(k+1)B:n(d_k+1^*(d))-r_j)1{r_j<a_jp^*_(k+1)B:n(d_k+1^*(d))}] 
    ≥   𝔼[x_j^π(d)(a_jp^*_(k+1)B:n(d̂_j,k+1(d)-r_j)1{r_j<a_jp^*_(k+1)B:n(d̂_j,k+1(d))}]

Define

    E_2,1(d,k,j) = {d̂_j,k+1(d)-d≤1/√(L(K-k-1)B)}
       E_2,2(d,k,j) = {√((K-k-1)B)(p^*_(k+1)B:n(d̂_j,k+1(d))-p^*(d̂_j,k+1(d))-4≤ 1}

Then, on event E_2,1(d,k,j)∩ E_2,2(d,k,j),

    p^*_(k+1)B:n(d̂_j,k+1(d))-p^*(d)≥2/√((K-k-1)B)

Then, following similar argument above,

    𝔼[x_j^π(d)(a_jp^*_(k+1)B:n(d̂_j,k+1(d)-r_j)1{r_j<a_jp^*_(k+1)B:n(d̂_j,k+1(d))}]
    ≥   𝔼[1_E_2,1(d,k,j)1_E_2,2(d,k,j)x_j^π(d)(a_j(p^*(d)+2/√((K-k-1)B))-r_j)1{r_j<a_j(p^*(d)+2/√((K-k-1)B))}]

In addition,

    1{r_j≤ a_j(p^*(d)+1/√((K-k-1)B))}(a_j(p^*(d)+2/√((K-k-1)B))-r_j)1{r_j<a_j(p^*(d)+2/√((K-k-1)B))}
       ≥1{r_j≤ a_j(p^*(d)+1/√((K-k-1)B))}a_j/√((K-k-1)B)
       ≥(1{r_j> a_j(p^*(d)-1/√((K-k-1)B))}-1{r_j> a_j(p^*(d)+1/√((K-k-1)B))})a_j/√((K-k-1)B)

To summarize,

    𝔼[x_j^π(d)(a_jp^*_(k+1)B:n(d_k+1^*(d))-d_j)1{r_j≤ a_jp^*_(k+1)B:n(d_k+1^*(d))}]
    ≥   𝔼[1_E_2,1(d,k,j)1_E_2,2(d,k,j)x_j^π(d)(1{r_j> a_j(p^*(d)-1/√((K-k-1)B))}..
                           -..1{r_j> a_j(p^*(d)+1/√((K-k-1)B))})a_j/√((K-k-1)B)]

If 

    K-k>max{2(a̅+δ-d)/δ,2(d̅+δ)/δ} and B>2a̅^2/aδ

We have

    1/(K-k-1)B((K-k)Bd-Ba̅-a̅^2/a)≥ d_0-2δ
       1/(K-k-1)B((K-k)Bd+a̅^2/a)≤ d_0+2δ

which implies that

    d_j,k+1(d),d̂_j,k+1(d)∈ [d_0-2δ,d_0+2δ]  ∀ t_k<j≤ t_k+1

Then, define

    E_1,3(k) = {sup_d∈[d_0-2δ_d,d_0+2δ_d]√((K-k-1)B)(p^*_(k+1)B:n(d)-p^*(d)+4≤ 1}
       E_2,3(k) = {sup_d∈[d_0-2δ_d,d_0+2δ_d]√((K-k-1)B)(p^*_(k+1)B:n(d)-p^*(d)-4≤ 1}

and

    E_1,3(k)⊆ E_1,2(d,k,j)     E_2,3(k)⊆ E_2,2(d,k,j)

In addition, E_1,3(k) and {(r_j,a_j)}_j=t_k+1^t_k+1 are independent. Then,

    𝔼[(1-x_j^π(d))(r_j-a_jp^*_(k+1)B:n(d_k+1^*(d)))1{r_j>a_jp^*_(k+1)B:n(d_k+1^*(d))}] 
    ≥   𝔼[1_E_1,1(d,k,j)1_E_2,1(d,k,j)1_E_1,3(k)(1-x_j^π(d))(1{r_j> a_j(p^*(d)-1/√((K-k-1)B))}..
                           -..1{r_j> a_j(p^*(d)+1/√((K-k-1)B))})a_j/√((K-k-1)B)]
    
            =   ℙ{E_1,3(k)}𝔼[1_E_1,1(d,k,j)1_E_2,1(d,k,j)(1-x_j^π(d))(1{r_j> a_j(p^*(d)-1/√((K-k-1)B))}..
                           -..1{r_j> a_j(p^*(d)+1/√((K-k-1)B))})a_j/√((K-k-1)B)]

Similarly,

    𝔼[x_j^π(d)(a_jp^*_(k+1)B:n(d_k+1^*(d))-d_j)1{r_j≤ a_jp^*_(k+1)B:n(d_k+1^*(d))}]
    ≥   ℙ{E_2,3(k)}𝔼[1_E_1,1(d,k,j)1_E_2,1(d,k,j)x_j^π(d)(1{r_j> a_j(p^*(d)-1/√((K-k-1)B))}..
                           -..1{r_j> a_j(p^*(d)+1/√((K-k-1)B))})a_j/√((K-k-1)B)]

Then, summing the left-hand-side and right-hand-side of the above two inequalities, we have

    𝔼[(r_j-a_jp^*_(k+1)B:n(d_k+1^*(d)))(1{r_j>a_jp^*_(k+1)B:n(d_k+1^*(d))}-x_j^π(d))]
    ≥   min{ℙ{E_1,3(k)},ℙ{E_2,3(k)}}𝔼[1_E_1,1(d,k,j)1_E_2,1(d,k,j)(1{r_j> a_j(p^*(d)-1/√((K-k-1)B))}..
                           -..1{r_j> a_j(p^*(d)+1/√((K-k-1)B))})a_j/√((K-k-1)B)]
    
            =   min{ℙ{E_1,3(k)},ℙ{E_2,3(k)}}ℙ{E_1,1(d,k,j)∩ E_1,2(d,k,j)}𝔼[(1{r_j> a_j(p^*(d)-1/√((K-k-1)B))}..
                           -..1{r_j> a_j(p^*(d)+1/√((K-k-1)B))})a_j/√((K-k-1)B)]

where the last line comes from the fact that 1_E_1,1(d,k,j)1_E_2,1(d,k,j) (r_j,a_j). Define,

    C_P(d,k,j) = min{ℙ{E_1,3(k)},ℙ{E_2,3(k)}}ℙ{E_1,1(d,k,j)∩ E_1,2(d,k,j)}

Then, if k<K-max{2(a̅+δ-d)/δ,2(d̅+δ)/δ} and B>2a̅^2/aδ, then for any  t_k<j≤ t_k+1 and d∈ [d_0-δ,d_0+δ],

    𝔼[(r_j-a_jp^*_(k+1)B:n(d_k+1^*(d)))(1{r_j>a_jp^*_(k+1)B:n(d_k+1^*(d))}-x_j^π(d))]
    
            =   C_P(d,k,j)/√((K-k-1)B)∫_-1^1d/ds𝔼[a_j1{r_j>a_j(p^*(d)-s/√((K-k-1)B))}]ds

Then, by Lemma <ref>(c), there exists a constant σ>0 such that

    𝔼[(r_j-a_jp^*_(k+1)B:n(d_k+1^*(d)))(1{r_j>a_jp^*_(k+1)B:n(d_k+1^*(d))}-x_j^π(d))]
    ≥   C_p(d,k,j)/√((K-k-1)B)∫_-1^1σ/√((K-k-1)B)ds
    
            =   2σ C_p(d,k,j)/(K-k-1)B

In the next, we show that, if K-k and B are large enough, there exists a constant C>0 such that C_P(d,k,j)>C for any d∈ [d_0-δ,d_0+δ] and t_k<j≤ t_k+1. First, if 

    (K-k-1)B > max{N_4,1,N_-4,1}

by Lemma <ref> and our selection of δ, 
then

    ℙ{E_1,3(k)}≥  C_4,1>0  ∀ t_k<j≤ t_k+1
       ℙ{E_2,3(k)}≥  C_-4,1>0  ∀ t_k<j≤ t_k+1

Secondly, by Lemma <ref>, if B>max{N_Dual,N_LOO,2,d̅/δ+2}, for any d∈[d_0-δ,d_0+δ] and k<K-1,

    ℙ{d_j,k+1(d)-d≤1/√(L(K-k-1)B)}
    ≥    1 - 𝔼[(d_j,k+1(d)-d)^2]L(K-k-1)B
    ≥    1-LC_off/(K-k-1)  ∀ t_k<j≤ t_k+1

and

    ℙ{d̂_j,k+1(d)-d≤1/√(L(K-k-1)B)}
    ≥    1 - 𝔼[(d̂_j,k+1(d)-d)^2]L(K-k-1)B
    ≥    1-LC_off/(K-k-1)  ∀ t_k<j≤ t_k+1

Then,

    ℙ(E_1,1(d,k,j)∩ E_1,2(d,k,j))≥ 1 - LC_off/(K-k-1)-LC_off/(K-k-1)  ∀ t_k<j≤ t_k+1

If K-k>1 + 4LC_off, then

    ℙ(E_1,1(d,k,j)∩ E_1,2(d,k,j))≥1/2  ∀ t_k<j≤ t_k+1

To summarize, there exists positive constants k_0, Λ and C such that, if K-k>k_0 and B>Λ, then for any d∈ [d_0-δ,d_0+δ] and t_k<j≤ t_k+1,

    C_P≥ C > 0

Then, (<ref>), (<ref>) and (<ref>) imply that, if K-k>k_0 and B>Λ,

    𝔼[(r_j-a_jp^*_(k+1)B:n(d_k+1^*(d_k)))(1{r_j>a_jp^*_(k+1)B:n(d_k+1^*(d_k))}-x_j^π)]
    ≥   ℙ{d_k∈ [d_0-δ,d_0+δ]}2σ C/(K-k-1)B

Let 

    Λ_DA = max{Λ,Λ_DA,4}

and

    K_0 = max{k_0^4/3,N̅/Λ_DA,16}

where N̅ is given in Lemma <ref>. Then, by Lemma <ref>, if B>Λ_DA and K>K_0,

    𝔼[(r_j-a_jp^*_(k+1)B:n(d_k+1^*(d_k)))(1{r_j>a_jp^*_(k+1)B:n(d_k+1^*(d_k))}-x_j^π)]
    ≥   σ C/(K-k-1)B    ∀K/2≤ k≤ K-K^3/4 t_k<j≤ t_k+1

Then, by (<ref>), there exists K_LB,C_LB such that, if B>Λ_DA and K>K_LB, 

    R_n^*-R_n(π)≥∑_k=⌈K/2⌉^⌊ K-K^3/4⌋1/K-k-1≥ C_LBlog (K)

which completes the proof.

     

    (Lemma 10 of <cit.>) For any γ∈ℝ and ϵ>0, there exists δ_γ,ϵ, C_γ,ϵ, N_γ,ϵ>0 such that, when (K-k)B>N_γ,ϵ, 
    
    ℙ{sup_d∈ [d_0-δ_γ,ϵ,d_0+δ_γ,ϵ]√((K-k)B)(p_kB:n^*(d)-p^*(d))≤ϵ}≥ C_γ,ϵ



    If B>max{N_Dual,N_LOO,2,d̅/δ+2}, then there exists a constant C_off such that, for any d∈ [d_0-δ,d_0+δ] and k<K-1,
    
    {   𝔼[(d_j,k+1(d)-d)^2]≤C_off/(K-k-1)^2B  ∀ t_k<j≤ t_k+1
       𝔼[(d̂_j,k+1(d)-d)^2]≤C_off/(K-k-1)^2B  ∀ t_k<j≤ t_k+1.



    Given d∈ [d_0-δ,d_0+δ] and k<K-1, for any t_k<j≤ t_k+1,
    
    (d_j,k+1(d)-d)^2 
    
                =    (d-a̅/(K-k-1)B)^2+1/((K-k-1)B)^2(∑_t∈{t_k+1,⋯,t_k+1}\ j d-a_t1{r_t>a_tp_j,kB:n(d)})^2
       +2/((K-k-1)B)^2((d-a̅)(∑_t∈{t_k+1,⋯,t_k+1}\ j d-a_t1{r_t>a_tp_j,kB:n(d)}))

    In the following, we show that there exists a constant C_off,1,1 such that, for any d∈ [d_0-δ,d_0+δ] and k<K-1,
    
    𝔼[(∑_t∈{t_k+1,⋯,t_k+1}\ j d-a_t1{r_t>a_tp_j,kB:n(d)})^2]≤ C_off,1,1B  ∀ t_k<j≤ t_k+1

    Define
    
    d_k(d) = (K-k)B/(K-k)B-1d

    Then,
    
    𝔼[(∑_t∈{t_k+1,⋯,t_k+1}\ j d-a_t1{r_t>a_tp_j,kB:n(d)})^2]
    
                =    𝔼[(∑_t∈{t_k+1,⋯,t_k+1}\ j d- d_k(d)+d_k(d)-a_t1{r_t>a_tp^*(d_k(d))}+a_t1{r_t>a_tp^*(d_k(d))}-a_t1{r_t>a_tp_j,kB:n(d)})^2]

    First,
    
    𝔼[(∑_t∈{t_k+1,⋯,t_k+1}\ j d- d_k(d))^2]≤d̅^2

    Secondly, since B>d̅/δ+2, d_k(d)∈Ω_d, then
    
    𝔼[(∑_t∈{t_k+1,⋯,t_k+1}\ j d_k(d)-a_t1{r_t>a_tp^*(d_k(d))})^2]
    
                =   ∑_t∈{t_k+1,⋯,t_k+1}\ j𝔼[(d_k(d)-a_t1{r_t>a_tp^*(d_k(d))})^2]
    ≤   (d̅+a̅)^2B

    Thirdly, define
    
    p_k,j,t_1,t_2(d) = min_p≥ 0d_k(d)p + 1/(K-k)B-3∑_t∈{t_k+1,⋯,t_k+1}\{j,t_1,t_2}(r_t-a_tp)^+

    Then,
    
    𝔼[(∑_t∈{t_k+1,⋯,t_k+1}\ j a_t1{r_t>a_tp^*(d_k(d))}-a_t1{r_t>a_tp_j,kB:n(d)})^2]
    ≤   Ba̅^2+∑_t_1≠ t_2t_1,t_2∈{t_k+1,⋯,t_k+1}\ j𝔼[(a_t_11{r_t_1>a_t_1p^*(d_k(d))}-1{r_t_1>a_t_1p_k,j,t_1,t_2(d)}).·
                           .(a_t_21{r_t_2>a_t_2p^*(d_k(d))}-1{r_t_2>a_t_2p_k,j,t_1,t_2(d)})]
       +a̅^2∑_t_1≠ t_2t_1,t_2∈{t_k+1,⋯,t_k+1}\ j𝔼[1{1{r_t_1>a_t_1p_k,j,t_1,t_2(d)}≠1{r_t_1>a_t_1p_j,kB:n(d)}}]
       +a̅^2∑_t_1≠ t_2t_1,t_2∈{t_k+1,⋯,t_k+1}\ j𝔼[+1{1{r_t_2>a_t_2p_k,j,t_1,t_2(d)}≠1{r_t_2>a_t_2p_j,kB:n(d)}}]
    ≤    Ba̅^2+B^2a̅^4β^2C_Dual/(K-k)B+2B^2a̅^2C_LOO,2/(K-k)B
    ≤   B(a̅^2+a̅^4β^2C_Dual+2a̅^2C_LOO,2)

    where the second last line is by Lemma <ref> and Lemma <ref>. By (<ref>), (<ref>), (<ref>) and Cauchy inequality, we can conclude that there exists a constant C_off,1,1 such that, for any d∈ [d_0-δ,d_0+δ] and k<K-1,
    
    𝔼[(∑_t∈{t_k+1,⋯,t_k+1}\ j d-a_t1{r_t>a_tp_j,kB:n(d)})^2]≤ C_off,1,1B  ∀ t_k<j≤ t_k+1

    Then, together with (<ref>) and Cauchy inequality, we can show that there exists a constant C_off,1 such that for any d∈ [d_0-δ,d_0+δ] and k<K-1,
    
    𝔼[(d_j,k+1(d)-d)^2]≤C_off,1/(K-k-1)^2B  ∀ t_k<j≤ t_k+1

    Following similar argument, we can show that there exists a constant C_off,2 such that for any d∈ [d_0-δ,d_0+δ] and k<K-1,
    
    𝔼[(d̂_j,k+1(d)-d)^2]≤C_off,2/(K-k-1)^2B  ∀ t_k<j≤ t_k+1

    Let C_off = max{C_off,1, C_off,2}, we finish the proof.


    If B>Λ_DA, there exists N̅ and δ̅ such that, when n=KB>N̅, and δ<δ̅, and (KB)^3/4≤ (K-k)B≤KB/2,
    
    ℙ{d_k∈ [d_0-δ,d_0+δ]}≥ 1 - (KB)^-1/2



The proof of Lemma 7 in <cit.> can be directly applied here. For completeness, we state the proof again here using our notations. Define

    [                             V_1,k =                                 max                    ∑_j=1^t_k r_jx_j;                                                                     S.T ∑_j=1^t_k a_jx_j ≤ b_0 - (n-t_k)d_k;                                                                                      0≤ x_j≤ 1  ∀ 1≤ j≤ t_k;                                     ]

and

    [                    V_2,k =                        max         ∑_j=t_k+1^n r_jx_j;                                                   S.T ∑_j=1^t_k a_jx_j ≤ t_k d_k;                                                       0≤ x_j≤ 1  ∀ t_k + 1≤ j≤ n;                            ]

Then,

    R_n(π)≤ V_1,k + V_2,k

An useful conclusion from the proof of Lemma 7 in <cit.> is that there exists N̅ and δ̅ such that, when n=KB>N̅, and δ<δ̅, and (KB)^3/4≤ (K-k)B≤KB/2,

    ℙ{R_n^*-V_1,k - V_2,k≥ (KB)^2/3| d_k∉ [d_0-δ,d_0+δ]}≥1/2

Then,

    𝔼[R_n^*-R_n(π)]
    ≥   (KB)^2/3ℙ{ d_k∉ [d_0-δ,d_0+δ]}ℙ{R_n^*-R_n(π)≥ (KB)^2/3| d_k∉ [d_0-δ,d_0+δ]}
    ≥   (KB)^2/3ℙ{ d_k∉ [d_0-δ,d_0+δ]}ℙ{R_n^*-V_1,k - V_2,k≥ (KB)^2/3| d_k∉ [d_0-δ,d_0+δ]}
    ≥   (KB)^2/3/2ℙ{ d_k∉ [d_0-δ,d_0+δ]}

Together with the fact that

    𝔼[R_n^*-R_n(π)]≤ O(log K)

We can conclude that, when n=KB>N̅, and δ<δ̅, and (KB)^3/4≤ (K-k)B≤KB/2,

    ℙ{d_k∈ [d_0-δ,d_0+δ]}≥ 1 - (KB)^-1/2




 §.§ Proof of Theorem <ref>


Fix an integer B such that

    B>Λ_MultiDA=max{a̅/d,N_Dual}

Let e_1,⋯,e_m be the standard basis of ℝ^m, i.e., the ith component of e_i equals 1 and all the other components equal 0. Fix δ_d such that

    ⊗_i=1^m [e_i^Td_0-δ_d,e_i^Td_0+δ_d]⊆Ω_d

Let d_k=b_t_k/n-t_k, and define 

    κ̅=min{K}∪{k:∃ i s.t. e_i^Td_k∉[e_i^Td_0-δ_d,e_i^Td_0+δ_d]}

To simplify the notation, p^* in this section refers to the population dual price evaluated at the initial average resource d_0, i.e., p^*=p^*(d_0). Define d'_0=d_0, and for 1≤ k≤ K-2,

    d'_k+1 = (n-t_k)d'_k-∑_t=t_k+1^t_k+1a_t1{r_t>a_tp_k+1}/n-t_k+11{k<κ̅}+d'_k1{k≥κ̅}

Then, d'_k=d_k if k<κ̅. Define R_n-kB^*(d), p_kB:n^*(d) and R_n-kB(π_2) in the same way as in Theorem <ref>.

    R_n^*-R_n(π_2)
    
            =   max_(x_1,⋯,x_t_1)∈ [0,1]^B∑_t=1^t_1a_tx_t≤ nd_0∑_t=1^t_1r_tx_t + R_n-B^*(1/n-B(nd_0-∑_t=1^t_1a_tx_t))-∑_t=1^t_1r_tx_t^π_2-R_n-B(π_2)
    
            =   max_(x_1,⋯,x_t_1)∈ [0,1]^B∑_t=1^t_1a_tx_t≤ nd_0∑_t=1^t_1r_t(x_t-x_t^π_2) + R_n-B^*(1/n-B(nd_0-∑_t=1^t_1a_tx_t))-R_n-B^*(d_1)+R_n-B^*(d_1)-R_n-B(π_2)

By LP Duality Theory,

    R_n-B^*(1/n-B(nd_0-∑_t=1^t_1a_tx_t))-R_n-B^*(d_1)≤∑_t=1^t_1(x_t^π_2-x_t)a_t^Tp_B:n^*(d_1)

Then,

    R_n^*-R_n(π_2)
    ≤   max_(x_1,⋯,x_t_1)∈ [0,1]^B∑_t=1^t_1a_tx_t≤ nd_0∑_t=1^t_1(r_t-a_t^Tp_B:n^*(d_1))(x_t-x_t^π_2)+(R_n-B^*(d_1)-R_n-B(π_2))
    ≤   max_(x_1,⋯,x_t_1)∈ [0,1]^B∑_t=1^t_1(r_t-a_t^Tp_B:n^*(d_1))(x_t-x_t^π_2)+(R_n-B^*(d_1)-R_n-B(π_2))
    
            =   ∑_t=1^t_1(r_t-a_t^Tp_B:n^*(d_1))(1{r_t>a_t^Tp_B:n^*(d_1)}1{x_t^π_2=0}-1{r_t≤a_t^Tp_B:n^*(d_1)}1{x_t^π_2=1})+(R_n-B^*(d_1)-R_n-B(π_2))

In a similar way, for all k=0,⋯,K-2

    R_n-kB^*(d_k)-R_n-kB(π_2)
    ≤   ∑_t=t_k+1^t_k+1(r_t-a_t^Tp_(k+1)B:n^*(d_k+1))(1{r_t>a_t^Tp_(k+1)B:n^*(d_k+1)}1{x_t^π_2=0}-1{r_t≤a_t^Tp_(k+1)B:n^*(d_k+1)}1{x_t^π_2=1})
       +(R_n-(k+1)B^*(d_k+1)-R_n-(k+1)B(π_2))

Since the algorithm allocate the remaining resource in the last batch in an offline fashion,

    R_n-(K-1)B^*(d_K-1)-R_n-(K-1)B(π_2)≤ O(1)

This assumption is not necessary but will simplify the proof. Then, by the definition of κ̅,

    𝔼[R_n^*-R_n(π_2)]
    ≤   𝔼[∑_k=0^κ̅-2∑_t=t_k+1^t_k+1(r_t-a_t^Tp_(k+1)B:n^*(d_k+1))(1{a_t^Tp_(k+1)B:n^*(d_k+1)<r_t≤a_t^Tp_k+1}-1{a_t^Tp_k+1<r_t≤a_t^Tp_(k+1)B:n^*(d_k+1)})]
       +(r̅+a̅r̅/a)(t_K-1-t_κ̅-1)+O(1)

Define d_k such that, for each 1≤ i≤ m and k=1,⋯, K-1,

    e_i^Td_k = min{max{e_i^Td'_k,e_i^Td_0-δ_d},e_i^Td_0+δ_d}

and for t=1,⋯, n, define

    q_t = p_11{1≤ t≤ t_1} + ∑_k=1^K-21{t_k<t≤ t_k+1}(min_p≥ 0d_k^Tp+1/t_k∑_j=1^t_k(r_j-a_j^Tp)^+)

Then, by the fact that

    (r_t-a_t^Tp_(k+1)B:n^*(d_k+1))(1{a_t^Tp_(k+1)B:n^*(d_k+1)<r_t≤a_t^Tp}-1{a_t^Tp<r_t≤a_t^Tp_(k+1)B:n^*(d_k+1)})≥ 0 a.s.  ∀ p

and

    d_k = d'_k = d_k  ∀ k<κ̅

We have the following regret decomposition

    𝔼[R_n^*-R_n(π_2)]
    ≤   ∑_k=0^K-2∑_t=t_k+1^t_k+1𝔼[(r_t-a_t^Tp_(k+1)B:n^*(d_k+1))(1{a_t^Tp_(k+1)B:n^*(d_k+1)<r_t≤a_t^Tq_t}-1{a_t^Tq_t<r_t≤a_t^Tp_(k+1)B:n^*(d_k+1)})]
    
            +   (r̅+a̅r̅/a)(t_K-1-t_κ̅-1)+O(1)

From now on, fix 0≤ k≤ K-2 and t_k<t≤ t_k+1. Note that there is a weak dependence (r_t,a_t) and d_k+1. To deal with this weak dependence, we define the Leave-One-Out remaining average resource d_t such that for all 1≤ i≤ m,

    e_i^Td_t =1{k<κ̅}min{max{e_i^Td'_k+1+e_i^Ta_t1{r_t>a_t^Tq_t}/(K-k-1)B,e_i^Td_0-δ_d},e_i^Td_0+δ_d}+e_i^Td_k1{k≥κ̅}

Then, 

    e_i^Td_t-e_i^Td_k+1≤a̅/(K-k-1)B

Thus,

    a_t^Tp_(k+1)B:n^*(d_k+1)
    
            =   a_t^Tp_(k+1)B:n^*(d_t)-a_t^T(p_(k+1)B:n^*(d_t)-p^*(d_t))-a_t^T(p^*(d_t)-p^*(d_k+1))-a_t^T(p^*(d_k+1)-p_(k+1)B:n^*(d_k+1))
    ≥   a_t^Tp_(k+1)B:n^*(d_t)-2a̅(sup_d∈Ω_dp_(k+1)B:n^*(d)-p^*(d)_2)-a̅√(L)d_k+1-d_t_2
    ≥   a_t^Tp_(k+1)B:n^*(d_t)-2a̅(sup_d∈Ω_dp_(k+1)B:n^*(d)-p^*(d)_2)-a̅^2√(mL)/(K-k-1)B

Thus,

    𝔼[(r_t-a_t^Tp_(k+1)B:T^*(d_k+1))1{a_t^Tp_k+1B:T^*(d_k+1)<r_t≤a_t^Tq_t}]
    ≤   𝔼[(a_t^T(q_t-p_k+1B:T^*(d_t))+2a̅(sup_d∈Ω_dp_(K-k-1)B^*(d)-p^*(d)_2)+a̅^2√(mL)/(K-k-1)B)·.
       .1{a_t^Tp_k+1B:T^*(d_t)-2a̅(sup_d∈Ω_dp_(K-k-1)B^*(d)-p^*(d)_2)-a̅^2√(mL)/(K-k-1)B≤ r_t≤a_t^Tq_t}]

By our construction of q_t and d_t, we have (r_t,a_t) (q_t, p_k+1B:T^*(d_t)). Then, by Assumption <ref>(b),

    𝔼[(r_t-a_t^Tp_(k+1)B:T^*(d_k+1))1{a_t^Tp_k+1B:T^*(d_k+1)<r_t≤a_t^Tq_t}]
    ≤   𝔼[β(a̅q_t-p^*_(k+1)B:T(d_t)_2+2a̅(sup_d∈Ω_dp_(K-k-1)B^*(d)-p^*(d)_2)+a̅^2√(mL)/(K-k-1)B)^2]

By Lemma <ref>,

    𝔼[sup_d∈Ω_dp_(K-k-1)B^*(d)-p^*(d)_2^2]≤C_Dual/(K-k-1)B

In addition,

    𝔼[q_t-p^*_(k+1)B:T(d_t)_2^2]
    ≤   𝔼[(q_t-p^*(d_k)_2+p^*(d_k)-p^*(d_k+1)_2+p^*(d_k+1)-p^*(d_t)_2+p^*(d_t)-p^*_(k+1)B:T(d_t)_2)^2]

If k=0,

    𝔼[q_t-p^*(d_k)_2^2]≤𝔼[sup_d∈Ω_dp_B^*(d)-p^*(d)_2^2]≤C_Dual/B

If k≥ 1,

    𝔼[q_t-p^*(d_k)_2^2]≤𝔼[sup_d∈Ω_dp_kB^*(d)-p^*(d)_2^2]≤C_Dual/kB

Also,

    𝔼[p^*(d_t)-p^*_(k+1)B:T(d_t)_2^2]≤𝔼[sup_d∈Ω_dp_(K-k-1)B^*(d)-p^*(d)_2^2]≤C_Dual/(K-k-1)B

and

    𝔼[p^*(d_k+1)-p^*(d_t)_2^2]≤Lma̅^2/(K-k-1)^2B^2

Furthermore,

    𝔼[p^*(d_k)-p^*(d_k+1)_2^2]≤ L𝔼[d_k+1-d_k_2^2]≤ L𝔼[d'_k+1-d'_k_2^2]

By definition,

    d'_k+1-d'_k = 1/(K-k-1)B(∑_t=t_k+1^t_k+1d'_k-a_t1{r_t>a_t^Tp^*(d'_k)}+∑_t=t_k+1^t_k+1a_t(1{r_t>a_t^Tp^*(d'_k)}-1{r_t>a_t^Tp_k+1}))1{k<κ̅}

Following the analysis of the first and the second component in Lemma <ref>, there exists a constant C_res such that

    𝔼[d'_k+1-d'_k_2^2]≤C_res/(K-k-1)^2B

Thus, there exists a constant C_1 such that

    𝔼[q_t-p^*_(k+1)B:T(d_t)_2^2]≤C_1/B1{k=0}+C_1/kB1{k≥ 1} + C_1/(K-k-1)B

Then, by (<ref>), (<ref>), (<ref>) and Cauhy-Schwarz Inequality, we can conclude that there exists a constant C such that,

    𝔼[(r_t-a_t^Tp_(k+1)B:T^*(d_k+1))1{a_t^Tp_k+1B:T^*(d_k+1)<r_t≤a_t^Tq_t}]≤C/B1{k=0}+C/kB1{k≥ 1} + C/(K-k-1)B

Similarly,

    𝔼[(a_t^Tp_(k+1)B:T^*(d_k+1)-r_t)1{a_t^Tq_t<r_t≤a_t^Tp_k+1B:T^*(d_k+1)}]≤C/B1{k=0}+C/kB1{k≥ 1} + C/(K-k-1)B

Thus,

    ∑_k=0^K-2∑_t=t_k+1^t_k+1𝔼[(r_t-a_t^Tp_(k+1)B:T^*(d_k+1))(1{a_t^Tp_k+1B:T^*(d_k+1)<r_t≤a_t^Tq_t}-1{a_t^Tq_t<r_t≤a_t^Tp_(k+1)B:T^*(d_k+1)})]
    ≤   ∑_k=0^K-2 2B(C/B1{k=0}+C/kB1{k≥ 1} + C/(K-k-1)B)
    ≤   O(log K)

For the second term in the regret bound (<ref>),

    ℙ{κ̅>k}   ≥ℙ{e_i^Td'_k-e_i^Td_0≤δ_d ∀ i}
       ≥ 1-∑_i=1^m𝔼[e_i^Td'_k-e_i^Td_0^2]/δ_d^2

By almost the same argument in the proof of Lemma <ref>, we can show that there exists a constant C_MultiRes such that, for all 1≤ i≤ m,

    ∑_k=1^K-1𝔼[e_i^Td'_k-e_i^Td_0^2]≤C_MultiReslog K/B

Thus,

    𝔼[K-κ̅]=   (K-1)-∑_k=1^K-1ℙ{κ̅>k}
    ≤   ∑_k=1^K-1∑_i=1^m𝔼[e_i^Td'_k-e_i^Td_0^2]/δ_d^2
    ≤   mC_MultiReslog(K)/δ_d^2B

Then,

    (r̅+a̅r̅/a)(t_K-1-t_κ̅-1)
    ≤   (r̅+a̅r̅/a)𝔼[K-κ̅]B
    ≤   O(log K)

(<ref>), (<ref>) and (<ref>) together shows that

    𝔼[R_n^*-R_n(π_2)]≤ O(log K)

which completes the proof.



 §.§ Proof of Theorem <ref>

Choose δ_d>0 such that

    [d_0 - 2δ_d, d_0 + 2δ_d]⊆Ω_d and δ_d<d̅/2

Fix T,λ,B such that λ B>Λ_PA=max{a̅/d,d̅-2δ_d/2δ_d}. To simplify the notation, we write x^π_3_j as x_j, and we define p^* to be the population dual price evaluated at b_0/λ T, i.e., p^*=p^*(b_0/λ T). Then, similar to the proof of Theorem <ref>,

    𝔼[R_T^*] =    𝔼[min_p≥ 0 b_0p + ∑_j=1^N(T)(r_j-a_jp)^+]
    ≤   𝔼[b_0p^*+∑_j=1^N(T)(r_j-a_jp^*)^+]

and

    𝔼[R_T(π_2)] =    𝔼[∑_j=1^N(T)r_jx_j]
    
        =    𝔼[∑_j=1^N(T)r_jx_j + b_N(T)p^* - b_N(T)p^*]
    
        =    𝔼[∑_j=1^N(T)r_jx_j + (b_0-∑_j=1^N(T)a_jx_j)p^*]-𝔼[b_N(T)p^*]

Thus,

    𝔼[R_T^*] - 𝔼[R_T(π_3)]
    ≤   𝔼[∑_j=1^N(T)(r_j-a_jp^*)(1{r_j>a_jp^*}-x_j)]+𝔼[b_N(T)p^*]

Define, for k=0,⋯,K-1, 

    d_k = b_N(t_k)/λ(T-t_k)

and

    {   τ̅ = inf{t≥ 0:b_N(t)<a̅}
       κ̅ = minK∪{k≥ 1: d_k∉ [d_0-δ_d,d_0+δ_d]}
       τ = min{τ̅,t_κ̅,t_K-1}.

In addition, define d'_0=d_0 and

    d'_k+1 = λ(T-t_k)d'_k-∑_j=N(t_k)+1^N(t_k+1)a_j1{r_j>a_jp_k+1}/λ(T-t_k+1)1{k<κ̅} + d'_k1{k≥κ̅}

Then,

    d'_k1{k<κ̅} = d_k1{k<κ̅}

and  let d_k be defined in Algorithm <ref>, then

    d_k1{k<κ̅} = λ/λ̂_kd'_k1{k<κ̅}

Then,

    𝔼[R_T^*] - 𝔼[R_T(π_3)]
    ≤   𝔼[∑_j=1^N(τ)(r_j-a_jp^*)(1{r_j>a_jp^*}-x_j)]
       +𝔼[∑_j=N(τ)+1^N(t_K-1)(r_j-a_jp^*)(1{r_j>a_jp^*}-x_j)]
       +𝔼[∑_j=N(t_K-1)+1^N(T)(r_j-a_jp^*)(1{r_j>a_jp^*}-1{r_j>a_jp_K})]
       +𝔼[b_N(T)p^*]

In the following, we analyze the four terms on the right-hand-side of the above inequality. For (<ref>), define, for any j∈ℕ^+,

    n(j) = N(t_1)1{j≤ N(t_1)} + ∑_k=2^KN(t_k-1)1{N(t_k-1)<j≤ N(t_k)}

and

    d_j = ∑_k=1^K(d_k-11{N(t_k-1)<j≤ N(t_k)}1{k≤κ̅}+d_01{N(t_k-1)<j≤ N(t_k)}1{k>κ̅})

and

    p_j = (min_0≤ p≤r̅/dn(j)d_jp+∑_j=1^n(j)(r_j-a_jp)^+)1{j≤ N(t_K-1)}+p_K1{N(t_K-1)<j≤ N(T)}

In addition, define

    J_k(j) = min{max{j,N(t_k-1)+1},N(t_k)}

Then,

    𝔼[∑_j=1^N(τ)(r_j-a_jp^*)(1{r_j>a_jp^*}-x_j)]
    ≤   𝔼[∑_j=1^N(t_K-1)(r_j-a_jp^*)(1{r_j>a_jp^*}-1{r_j>a_jp_j})]
    
        =    ∑_k=1^K-1𝔼[∑_j=1^+∞(r_J_k(j)-a_J_k(j)p^*)(1{r_J_k(j)>a_J_k(j)p^*}-1{r_J_k(j)>a_J_k(j)p_J_k(j)})1{N(t_k-1)<j≤ N(t_k)}]

If k≥ 2, by Assumption <ref>, (r_J_k(j),a_J_k(j)) is independent of (p_J_k(j),N(t_k-1),N(t_k)) for all j. Then, by Lemma <ref>(a),

    𝔼[∑_j=1^+∞(r_J_k(j)-a_J_k(j)p^*)(1{r_J_k(j)>a_J_k(j)p^*}-1{r_J_k(j)>a_J_k(j)p_J_k(j)})1{N(t_k-1)<j≤ N(t_k)}]
    
            =   𝔼[∑_j=1^+∞𝔼[(r_J_k(j)-a_J_k(j)p^*)(1{r_J_k(j)>a_J_k(j)p^*}-...
                                 ...1{r_J_k(j)>a_J_k(j)p_J_k(j)})| N(t_k-1),N(t_k)]1{N(t_k-1)<j≤ N(t_k)}]
    ≤   βa̅^2𝔼[∑_j=1^+∞(p_J_k(j)-p^*)^21{N(t_k-1)<j≤ N(t_k)}]

If k = 1, define

    p_j = min_0≤ p≤r̅/ddp+∑_j∈{1,⋯,n(j)}\{j}(r_j-a_jp)^+

Then,

    𝔼[∑_j=1^+∞(r_J_1(j)-a_J_1(j)p^*)(1{r_J_1(j)>a_J_1(j)p^*}-1{r_J_1(j)>a_J_1(j)p_J_1(j)})1{N(t_0)<j≤ N(t_1)}]
    ≤   𝔼[∑_j=1^+∞(r_J_1(j)-a_J_1(j)p^*)(1{r_J_1(j)>a_J_1(j)p^*}-1{r_J_1(j)>a_J_1(j)p_J_1(j)})1{j≤ N(t_1)}]
       +(r̅+a̅r̅/d)𝔼[∑_j=1^+∞1{1{r_J_1(j)>a_J_1(j)p_J_1(j)}≠1{r_J_1(j)>a_J_1(j)p_J_1(j)}}1{j≤ N(t_1)}]

By Assumption <ref>(b), Lemma <ref>(a) and similar arguments for the case k≥ 2,

    𝔼[∑_j=1^+∞(r_J_1(j)-a_J_1(j)p^*)(1{r_J_1(j)>a_J_1(j)p^*}-1{r_J_1(j)>a_J_1(j)p_J_1(j)})1{N(t_0)<j≤ N(t_1)}]
    ≤   βa̅^2𝔼[∑_j=1^+∞(p_J_1(j)-p^*)^21{j≤ N(t_1)}]
       +(r̅+a̅r̅/d)𝔼[∑_j=1^+∞1{1{r_J_1(j)>a_J_1(j)p_J_1(j)}≠1{r_J_1(j)>a_J_1(j)p_J_1(j)}}1{j≤ N(t_1)}]

To summarize,

    𝔼[∑_j=1^N(τ)(r_j-a_jp^*)(1{r_j>a_jp^*}-x_j)]
    ≤   βa̅^2(𝔼[∑_j=1^+∞(p_J_1(j)-p^*)^21{j≤ N(t_1)}]+∑_k=2^K-1𝔼[∑_j=1^+∞(p_J_k(j)-p^*)^21{N(t_k-1)<j≤ N(t_k)}])
       +(r̅+a̅r̅/d)𝔼[∑_j=1^+∞1{1{r_J_1(j)>a_J_1(j)p_J_1(j)}≠1{r_J_1(j)>a_J_1(j)p_J_1(j)}}1{j≤ N(t_1)}]

Then, by Lemma <ref>(b) and (c), 

    𝔼[∑_j=1^N(τ)(r_j-a_jp^*)(1{r_j>a_jp^*}-x_j)]
    ≤   O(log K)+O(𝔼[∑_j=1^N(t_K-1)(d_j-d_0)^2])+√(O(log K))√(O(𝔼[∑_j=1^N(t_K-1)(d_j-d_0)^2]))

Furthermore,

    𝔼[∑_j=1^N(t_K-1)(d_j-d_0)^2]=   𝔼[∑_k=1^K-1∑_j=N(t_k-1)+1^N(t_k)(d_k-1-d_0)^21{k≤κ̅}]
    ≤   𝔼[∑_k=1^K-1(N(t_k)-N(t_k-1))(d_k-1-d'_k-1+d'_k-1-d_0)^21{k≤κ̅}]
    ≤   𝔼[∑_k=1^K-1(N(t_k)-N(t_k-1))d_k-1-d'_k-1^21{k≤κ̅}]+𝔼[∑_k=1^K-1(N(t_k)-N(t_k-1))d'_k-1-d_0^2]
       +2𝔼[∑_k=1^K-1(N(t_k)-N(t_k-1))d_k-1-d'_k-1d'_k-1-d_01{k≤κ̅}]

By Lemma <ref>(b) and (d),

    𝔼[∑_k=1^K-1(N(t_k)-N(t_k-1))d_k-1-d'_k-1^21{k≤κ̅}]
    
        =   𝔼[N(t_1)d_0-d_0^2]+λ B∑_k=2^K-1𝔼[d_k-1-d'_k-1^21{k≤κ̅}]
    ≤   𝔼[N(t_1)d_0-d_0^2]+λ Bd̅^2∑_k=2^K-1𝔼[λ/λ̂_k-1-1^2]
    ≤   d_0^2+4d̅^2∑_k=2^K-11/k-1
    ≤   O(log K)

By Lemma <ref>

    𝔼[∑_k=1^K-1(N(t_k)-N(t_k-1))d'_k-1-d_0^2]=λ B∑_k=2^K-1𝔼[d'_k-1-d_0^2]≤  O(log K)

and

    2𝔼[∑_k=1^K-1(N(t_k)-N(t_k-1))d_k-1-d'_k-1d'_k-1-d_01{k≤κ̅}]
    
        =   2λ B∑_k=2^K-1𝔼[d_k-1-d'_k-1d'_k-1-d_01{k≤κ̅}]
    ≤   2λ B∑_k=2^K-1√(𝔼[d_k-1-d'_k-1^21{k≤κ̅}])√(𝔼[d'_k-1-d_0^2])
    ≤   2√(λ B∑_k=2^K-1𝔼[d_k-1-d'_k-1^21{k≤κ̅}])√(λ B∑_k=2^K-1𝔼[d'_k-1-d_0^2])
    ≤   2√(4d̅^2∑_k=2^K-11/k-1)√(λ B∑_k=2^K-1𝔼[d'_k-1-d_0^2])
    ≤    O(log K)

Thus,

    𝔼[∑_j=1^N(t_K-1)(d_j-d_0)^2]≤ O(log K)

Then,

    𝔼[∑_j=1^N(τ)(r_j-a_jp^*)(1{r_j>a_jp^*}-x_j)]≤ O(log K)

For (<ref>), since τ is a stopping time adapted to the filtration generated by ((N(t),r_N(t),a_N(t)),t≥ 0), by the memoryless property of Poisson process, 

    𝔼[N(t_K-1)-N(τ)]≤λ𝔼[t_K-1-τ]≤λ𝔼[t_K-1-t_κ̅-1]= λ B𝔼[K-κ]

where the second inequality comes from the fact that t_κ̅-1≤τ almost sure by definition.
Thus,

    𝔼[∑_j=N(τ+1)^N(t_K-1)(r_j-a_jp^*)(1{r_j>a_jp^*}-x_j)]≤(r̅+a̅r̅/d)λ B𝔼[K-κ]

With similar argument in proof of Theorem <ref>, by Lemma <ref>,

    𝔼[K-κ̅]≤∑_k=1^K-1𝔼[d'_k-d_0^2]/δ_d^2≤ O(log K)

Thus,

    𝔼[∑_j=N(τ)+1^N(t_K-1)(r_j-a_jp^*)(1{r_j>a_jp^*}-x_j)]≤  O(log K)

For (<ref>), define

    p̂_j = min_p≥ 0 b_N(t_K-1)p+∑_j∈{N(t_K-1)+1,⋯,N(T)}\j(r_j-a_jp)^+

Then,

    𝔼[∑_j=N(t_K-1)+1^N(T)(r_j-a_jp^*)(1{r_j>a_jp^*}-1{r_j>a_jp_K})]
    ≤   𝔼[∑_j=1^+∞(r_J_K(j)-a_J_K(j)p^*)(1{r_J_K(j)>a_J_K(j)p^*}-1{r_J_K(j)>a_J_K(j)p̂_J_K(j)})1{N(t_K-1)<j≤ N(T)}]
       +(r̅+a̅r̅/d)𝔼[∑_j=1^+∞1{1{r_J_K(j)>a_J_K(j)p̂_J_K(j)}≠1{r_J_K(j)>a_J_K(j)p_J_K(j)}}1{N(t_K-1)<j≤ N(T)}]

By Lemma <ref>(a),

    𝔼[∑_j=1^+∞(r_J_K(j)-a_J_K(j)p^*)(1{r_J_K(j)>a_J_K(j)p^*}-1{r_J_K(j)>a_J_K(j)p̂_J_K(j)})1{N(t_K-1)<j≤ N(T)}]
    ≤   βa̅^2𝔼[∑_j=1^+∞(p̂_J_K(j)-p^*)^21{N(t_K-1)<j≤ N(T)}]

Then, by Lemma <ref>(d) and Lemma <ref>(e) 

    𝔼[∑_j=N(t_K-1)+1^N(T)(r_j-a_jp^*)(1{r_j>a_jp^*}-x_j)]≤ O(log K)

For (<ref>),

    𝔼[b_N(T)]≤𝔼[d_κ̅λ (T-t_κ̅)1{κ̅≤ K-1}+b_N(T)1{κ̅= K}]

and

    𝔼[d_κ̅λ (T-t_κ̅)1{κ̅≤ K-1}]
    ≤   𝔼[d_κ̅-1λ (T-t_κ̅-1)/λ (T-t_κ̅)λ (T-t_κ̅)1{κ̅≤ K-1}]
    ≤   2d̅𝔼[K-κ̅]
    ≤   O(log K)

In addition,

    𝔼[b_N(T)1{κ̅=K}]
    
            =   𝔼[b_N(T)1{κ̅=K}1{d̅λ B≤∑_j=N(t_K-1)^N(t_K)a_j}]+𝔼[b_N(T)1{κ̅=K}1{d̅λ B>∑_j=N(t_K-1)^N(t_K)a_j}]
    ≤   a̅+λ Bd̅ℙ{λ Bd̅>∑_j=N(t_K-1)^N(t_K)a_j}

and 

    ℙ{d̅λ B>∑_j=N(t_K-1)^N(t_K)a_j}
    ≤   𝔼[1{d̅λ B>∑_j=N(t_K-1)^N(t_K)a_j}1{λ Bd̅<((N(T)-N(t_K-1))𝔼[a]-ϵ_d/2(N(T)-N(t_K-1)))}1{N(T)-N(t_K-1)≥ 1}]
       +𝔼[1{λ Bd̅≥((N(T)-N(t_K-1))𝔼[a]-ϵ_d/2(N(T)-N(t_K-1)))}+1{N(T)-N(t_K-1)=0}]
    ≤   𝔼[ℙ{∑_j=N(t_K-1)^N(t_K)a_j<(N(T)-N(t_K-1))𝔼[a]-ϵ_d/2(N(T)-N(t_K-1))| N(T),N(t_K-1)}1{N(T)-N(t_K-1)≥ 1}]
       +𝔼[1{λ Bd̅≥((N(T)-N(t_K-1))𝔼[a]-ϵ_d/2(N(T)-N(t_K-1)))}+1{N(T)-N(t_K-1)=0}]
    ≤   𝔼[exp(-ϵ_d^2(N(T)-N(t_K-1))/2a̅^2)]+exp(-λ B) + exp(-(1-d̅/𝔼[a]-ϵ_d/2)^2(λ B)^2/2(2-d̅/𝔼[a]-ϵ_d/2)λ B)

Together with Lemma <ref>(i), there exists R_1 and R_2 such that

    𝔼[b_N(T)1{κ̅=K}]   ≤a̅+λ Bd̅R_1exp(-R_2λ B)≤ O(1)

Thus,

    𝔼[b_N(T)p^*]≤r̅/d𝔼[b_N(T)]≤  O(log K)

To summarize, (<ref>), (<ref>), (<ref>) and (<ref>) imply that, if B>Λ_PA,

    𝔼[R_T^*-R_T(π_2)]≤ O(log K)

which completes the proof of Theorem <ref>.

    If λ B>Λ_PA, then there exists a constant M_Res such that
    
    ∑_k=1^K-1𝔼[d'_k-d_0^2]≤M_Reslog K/λ B



By the definition of d'_k, we have for k≥ 0,

    𝔼[(d'_k+1-d_0)^2]
    
        =   𝔼[((d'_k-d_0)+(λ (t_k+1-t_k)-(N(t_k+1)-N(t_k))/λ (T-t_k+1)d'_k1{k<κ̅})+(∑_j=N(t_k)+1^N(t_k+1)d'_k-a_j1{r_j>a_jp^*(d'_k)}/λ (T-t_k+1)1{k<κ̅})..
       ..+(∑_j=N(t_k)+1^N(t_k+1)a_j(1{r_j>a_jp^*(d'_k)}-1{r_j>a_jp_k+1})/λ (T-t_k+1)1{k<κ̅}))^2]

First,

    𝔼[(λ (t_k+1-t_k)-(N(t_k+1)-N(t_k))/λ (T-t_k+1)d'_k1{k<κ̅})^2]
    
        =   𝔼[(d'_k1{k<κ̅})^2]𝔼[(λ (t_k+1-t_k)-(N(t_k+1)-N(t_k))/λ (T-t_k+1))^2]
    ≤   d̅^2/(λ B)^2(K-k-1)^2Var(N(t_k+1)-N(t_k))
    
        =   d̅^2/(λ B)(K-k-1)^2

Secondly,

    𝔼[(∑_j=N(t_k)+1^N(t_k+1)d'_k-a_j1{r_j>a_jp^*(d'_k)}/λ (T-t_k+1)1{k<κ̅})^2]
    
        =   1/(λ (T-t_k+1))^2𝔼[∑_i=N(t_k)+1^N(t_k+1)∑_j=N(t_k)+1^N(t_k+1)(d'_k-a_i1{r_i>a_ip^*(d'_k)})(d'_k-a_j1{r_j>a_jp^*(d'_k)}1{k<κ̅})]
    
        =   1/(λ (T-t_k+1))^2𝔼[∑_i=1^+∞∑_j=1^+∞((d'_k-a_J_k+1(i)1{r_J_k+1(i)>a_J_k+1(i)p^*(d'_k)})·..
           ..(d'_k-a_J_k+1(j)1{r_J_k+1(j)>a_J_k+1(j)p^*(d'_k)})1{k<κ̅}1{N(t_k)<i≤ N(t_k+1)}1{N(t_k)<j≤ N(t_k+1)})]
    
        =   1/(λ (T-t_k+1))^2𝔼[∑_i=1^+∞∑_j=1^+∞𝔼[(d'_k-a_J_k+1(i)1{r_J_k+1(i)>a_J_k+1(i)p^*(d'_k)})·(d'_k-..
       ..a_J_k+1(j)1{r_J_k+1(j)>a_J_k+1(j)p^*(d'_k)})1{k<κ̅}| d'_k,N(t_k),N(t_k+1)]1{N(t_k)<i≤ N(t_k+1)}1{N(t_k)<j≤ N(t_k+1)}]

If i=j,

    𝔼[(d'_k-a_J_k+1(i)1{r_J_k+1(i)>a_J_k+1(i)p^*(d'_k)})^21{k<κ̅}| d'_k,N(t_k),N(t_k+1)]≤(d̅+a̅)^2

If i≠ j,

    𝔼[(d'_k-a_J_k+1(i)1{r_J_k+1(i)>a_J_k+1(i)p^*(d'_k)})(d'_k-a_J_k+1(j)1{r_J_k+1(j)>a_J_k+1(j)p^*(d'_k)})1{k<κ̅}| d'_k,N(t_k),N(t_k+1)]
    
            =   𝔼[(d'_k-a_J_k+1(i)1{r_J_k+1(i)>a_J_k+1(i)p^*(d'_k)})1{k<κ̅}| d'_k,N(t_k),N(t_k+1)]·
       𝔼[(d'_k-a_J_k+1(j)1{r_J_k+1(j)>a_J_k+1(j)p^*(d'_k)})1{k<κ̅}| d'_k,N(t_k),N(t_k+1)]
    
            =   0

Then,

    𝔼[(∑_j=N(t_k)+1^N(t_k+1)d'_k-a_j1{r_j>a_jp^*(d'_k)}/λ (T-t_k+1)1{k<κ̅})^2]   ≤(d̅+a̅)^2/(λ (T-t_k+1))^2𝔼[N(t_k+1)-N(t_k)]
       =(d̅+a̅)^2/(λ B)(K-k-1)^2

Thirdly, 

    𝔼[(∑_j=N(t_k)+1^N(t_k+1)a_j(1{r_j>a_jp^*(d'_k)}-1{r_j>a_jp_k+1})/λ (T-t_k+1)1{k<κ̅})^2]
    ≤   1/(λ(T-t_k+1))^2𝔼[∑_i=N(t_k)+1^N(t_k+1)∑_j=N(t_k)+1^N(t_k+1)a_i(1{r_i>a_ip^*(d'_k)}-1{r_i>a_ip_k+1})·.
                           .a_j(1{r_j>a_jp^*(d'_k)}-1{r_j>a_jp_k+1})1{k<κ̅}]

and

    𝔼[∑_i=N(t_k)+1^N(t_k+1)∑_j=N(t_k)+1^N(t_k+1)a_i(1{r_i>a_ip^*(d'_k)}-1{r_i>a_ip_k+1})a_j(1{r_j>a_jp^*(d'_k)}-1{r_j>a_jp_k+1})1{k<κ̅}]
    
            =    𝔼[∑_i=1^+∞∑_j=1^+∞𝔼[a_J_k+1(i)(1{r_J_k+1(i)>a_J_k+1(i)p^*(d'_k)}-1{r_J_k+1(i)>a_J_k+1(i)p_k+1})·..
       ..             a_J_k+1(j)(1{r_J_k+1(j)>a_J_k+1(j)p^*(d'_k)}-1{r_J_k+1(j)>a_J_k+1(j)p_k+1})·..
       ..            1{k<κ̅}| N(t_k),N(t_k+1)]1{N(t_k)<i≤ N(t_k+1)}1{N(t_k)<j≤ N(t_k+1)}]

For all k, if i=j,

    𝔼[(a_J_k+1(i)(1{r_J_k+1(i)>a_J_k+1(i)p^*(d'_k)}-1{r_J_k+1(i)>a_J_k+1(i)p_k+1}))^21{k<κ̅}| N(t_k),N(t_k+1)]≤a̅^2

For the case k=0, define

    p̂_i,j=(min_p≥ 0N(t_1)d_0p+∑_j={1,⋯,N(t_1)}\{i,j}(r_j-a_jp)^+)1{N(t_1)>2}

Then, if i≠ j,

    𝔼[a_J_1(i)(1{r_J_1(i)>a_J_1(i)p^*(d'_0)}-1{r_J_1(i)>a_J_1(i)p_1})·.
           .a_J_1(j)(1{r_J_1(j)>a_J_1(j)p^*(d'_0)}-1{r_J_1(j)>a_J_1(j)p_1})| N(t_1)]
    ≤   𝔼[a_J_1(i)(1{r_J_1(i)>a_J_1(i)p^*(d'_0)}-1{r_J_1(i)>a_J_1(i)p̂_J_1(i),J_1(j)})·.
           .a_J_1(j)(1{r_J_1(j)>a_J_1(j)p^*(d'_0)}-1{r_J_1(j)>a_J_1(j)p̂_J_1(i),J_1(j)})| N(t_1)]
       +a̅^2𝔼[1{1{r_J_1(j)>a_J_1(j)p_1}≠1{r_J_1(j)>a_J_1(j)p̂_J_1(i),J_1(j)}}+.
       .            1{1{r_J_1(i)>a_J_1(i)p_1}≠1{r_J_1(i)>a_J_1(i)p̂_J_1(i),J_1(j)}}| N(t_1)]

Together with Lemma <ref>(f),

    𝔼[∑_i=1^N(t_1)∑_j=1^N(t_1)a_i(1{r_i>a_ip^*(d'_0)}-1{r_i>a_ip_1})a_j(1{r_j>a_jp^*(d'_0)}-1{r_j>a_jp_1})]
    ≤   β^2a̅^4𝔼[∑_i=1^+∞∑_j=1^+∞(p̂_J_1(i),J_1(j)-p^*(d'_0))^21{i≤ N(t_1)}1{j≤ N(t_1)}]+a̅^2𝔼[N(t_1)]
       +a̅^2𝔼[∑_i=1^+∞∑_j=1^+∞(1{1{r_J_1(j)>a_J_1(j)p_1}≠1{r_J_1(j)>a_J_1(j)p̂_J_1(i),J_1(j)}}..
       ..    +1{1{r_J_1(i)>a_J_1(i)p_1}≠1{r_J_1(i)>a_J_1(i)p̂_J_1(i),J_1(j)}})1{i≤ N(t_1)}1{j≤ N(t_1)}]

For the case k≥ 1,

    𝔼[a_J_k+1(i)(1{r_J_k+1(i)>a_J_k+1(i)p^*(d'_k)}-1{r_J_k+1(i)>a_J_k+1(i)p_k+1})·.
       .             a_J_k+1(j)(1{r_J_k+1(j)>a_J_k+1(j)p^*(d'_k)}-1{r_J_k+1(j)>a_J_k+1(j)p_k+1})1{k<κ̅}| N(t_k),N(t_k+1)]
    
            =   𝔼[𝔼[a_J_k+1(i)(1{r_J_k+1(i)>a_J_k+1(i)p^*(d'_k)}-1{r_J_k+1(i)>a_J_k+1(i)p_k+1})·..
       ..a_J_k+1(j)(1{r_J_k+1(j)>a_J_k+1(j)p^*(d'_k)}-1{r_J_k+1(j)>a_J_k+1(j)p_k+1})| d'_k,N(t_k),N(t_k+1)]1{k<κ̅}| N(t_k),N(t_k+1)]

Together with Lemma <ref>(f),

    𝔼[∑_i=N(t_k)+1^N(t_k+1)∑_j=N(t_k)+1^N(t_k+1)a_i(1{r_i>a_ip^*(d'_k)}-1{r_i>a_ip_k+1})a_j(1{r_j>a_jp^*(d'_k)}-1{r_j>a_jp_k+1})1{k<κ̅}]
    ≤   β^2a̅^4𝔼[(N(t_k+1)-N(t_k))^2]𝔼[(p_k+1-p^*(d'_k))^21{k<κ̅}]+a̅^2𝔼[N(t_k+1)-N(t_k)]

(<ref>) and (<ref>) together with Lemma <ref>(g), (h), (i) imply that  
Thus, there exists a constant C_res,1 such that

    𝔼[(∑_j=N(t_k)^N(t_k+1)a_j(1{r_j>a_jp^*(d'_k)}-1{r_j>a_jp_k+1})/λ (T-t_k+1)1{k<κ̅})^2]≤C_res,1/(λ B)(K-k-1)^2

Fourthly,

    2𝔼[(d'_k-d_0)(λ (t_k+1-t_k)-(N(t_k+1)-N(t_k))/λ (T-t_k+1)d'_k1{k<κ̅})]
    
        =   2𝔼[(d'_k-d_0)d'_k1{k<κ̅}]𝔼[λ (t_k+1-t_k)-(N(t_k+1)-N(t_k))/λ (T-t_k+1)]
    
        =   0

Fifthly,

    2𝔼[(d'_k-d_0)(∑_j=N(t_k)^N(t_k+1)d'_k-a_j1{r_j>a_jp^*(d'_k)}/λ (T-t_k+1)1{k<κ̅})]
    
        =   2/λ (T-t_k+1)𝔼[(d'_k-d)𝔼[∑_j=N(t_k)+1^N(t_k+1)d'_k-a_j1{r_j>a_jp^*(d'_k)}| N(t_k),N(t_k+1),d'_k]1{k<κ̅}]

and

    𝔼[(d'_k-d)𝔼[∑_j=N(t_k)+1^N(t_k+1)d'_k-a_j1{r_j>a_jp^*(d'_k)}| N(t_k),N(t_k+1),d'_k]1{k<κ̅}]
    
        =   𝔼[(d'_k-d)∑_j=1^+∞𝔼[d'_k-a_J_k+1(j)1{r_J_k+1(j)>a_J_k+1(j)p^*(d'_k)}| N(t_k),N(t_k+1),d'_k]1{k<κ̅}1{N(t_k)<j≤ N(t_k+1)}]

and

    𝔼[d'_k-a_J_k+1(j)1{r_J_k+1(j)>a_J_k+1(j)p^*(d'_k)}| N(t_k),N(t_k+1),d'_k]1{k<κ̅}=0   a.s.

Thus,

    2𝔼[(d'_k-d_0)(∑_j=N(t_k)^N(t_k+1)d'_k-a_j1{r_j>a_jp^*(d'_k)}/λ (T-t_k+1)1{k<κ̅})]
    
        =   0

Sixthly, if k=0,

    2𝔼[(d'_k-d_0)(∑_j=N(t_k)^N(t_k+1)a_j(1{r_j>a_jp^*(d'_k)}-1{r_j>a_jp_k+1})/λ (T-t_k+1)1{k<κ̅})]=0

If k≥ 1, by Lemma <ref>(f),

    𝔼[∑_j=N(t_k)+1^N(t_k+1)a_j(1{r_j>a_jp^*(d'_k)}-1{r_j>a_jp_k+1})(d'_k-d_0)1{k<κ̅}]
    
            =   𝔼[∑_j=1^+∞𝔼[a_J_k+1(j)(1{r_J_k+1(j)>a_J_k+1(j)p^*(d'_k)}...
       ...        -1{r_J_k+1(j)>a_J_k+1(j)p_k+1})| d'_k,N(t_k),N(t_k+1)]1{N(t_k)<j≤ N(t_k+1)}(d'_k-d_0)1{k<κ̅}]
    ≤   βa̅^2𝔼[(N(t_k+1)-N(t_k))(d'_k-d)p_k+1-p^*(d'_k)1{k<κ̅}]
    ≤   βa̅^2𝔼[(N(t_k+1)-N(t_k))]√(𝔼[(d'_k-d_0)^2])√(𝔼[p_k+1-p^*(d'_k)^21{k<κ̅}])

Then, by Lemma <ref>(i), there exists a constant C_res,2,1 such that

    √(𝔼[p_k+1-p^*(d'_k)^21{k<κ̅}])
    ≤   √(C_res,2,1/λ B k)
    ≤   √(2C_res,2,1/λ B (k+1))

Thus, there exists a constant C_res,2 such that

    2𝔼[(d'_k-d_0)(∑_j=N(t_k)^N(t_k+1)a_j(1{r_j>a_jp^*(d'_k)}-1{r_j>a_jp_k+1})/λ (T-t_k+1)1{k<κ̅})]
    ≤   √(C_res,2)/√(λ B)(K-k-1)√(k+1)√(𝔼[(d'_k-d_0)^2])

(<ref>), (<ref>), (<ref>), (<ref>), (<ref>), (<ref>), (<ref>) together with Cauchy-Schwarz Inequality imply that there exists a constant C_res such that

    𝔼[(d'_k+1-d_0)^2]≤C_res/(λ B)(K-k-1)^2+√(C_res)/√(λ B)(K-k-1)√(k+1)√(𝔼[(d'_k-d_0)^2])  ∀ 0≤ k≤ K-2

Thus, by Lemma <ref>, there exists a constant M_Res=8C_res such that

    ∑_k=1^K-1𝔼[(d'_k-d_0)^2]≤M_Res/λ Blog(K)



     

    (a) If (r,a)(p,N_1,N_2), then
    
    𝔼[(r-ap^*)(1{r>ap^*}-1{r>ap})| N_1,N_2]≤βa̅^2𝔼[(p-p^*)^2| N_1,N_2]

    (b) There exists C such that
    
    𝔼[∑_j=1^+∞(p_J_1(j)-p^*)^21{j≤ N(t_1)}]
    ≤    C+L𝔼[∑_j=1^N(t_1)(d_j-d_0)^2]+2√(C)√(L𝔼[∑_j=1^N(t_1)(d_j-d_0)^2])

    and, for all k≥ 2,
    
    𝔼[∑_j=1^+∞(p_J_k(j)-p^*)^21{N(t_k-1)<j≤ N(t_k)}]
    ≤   C/k+L𝔼[∑_j=N(t_k-1)+1^N(t_k)(d_j-d_0)^2]+2√(C/k)√(L𝔼[∑_j=N(t_k-1)+1^N(t_k)(d_j-d_0)^2])

    (c) There exists a constant C such that
    
    𝔼[∑_j=1^+∞1{1{r_J_1(j)>a_J_1(j)p_J_1(j)}≠1{r_J_1(j)>a_J_1(j)p_J_1(j)}}1{j≤ N(t_1)}]≤C

    (d) There exists a constant Ĉ such that
    
    𝔼[∑_j=1^+∞(p̂_J_K(j)-p^*)^21{N(t_K-1)<j≤ N(T)}]≤Ĉlog K + Ĉ

    (e) There exists a constant C such that
    
    𝔼[∑_j=1^+∞1{1{r_J_K(j)>a_J_K(j)p̂_J_K(j)}≠1{r_J_K(j)>a_J_K(j)p_J_K(j)}}1{N(t_K-1)<j≤ N(T)}]≤Clog K +C

    (f) If (r_1,a_1)(r_2,a_2)(p,d,N_1,N_2), then
    
    𝔼[a_1(1{r_1>a_1p^*(d)}-1{r_1>a_1p})|d,N_1,N_2]≤βa̅^2𝔼[p-p^*(d)|d,N_1,N_2]

    and
    
    𝔼[a_1(1{r_1>a_1p^*(d)}-1{r_1>a_1p})a_2(1{r_2>a_2p^*(d)}-1{r_2>a_2p})|d,N_1,N_2]
    ≤   β^2a̅^4𝔼[(p-p^*(d))^2|d,N_1,N_2]

    (g) There exists a constant C_7 such that
    
    𝔼[∑_i=1^+∞∑_j=1^+∞(p̂_J_1(i),J_1(j)-p^*(d'_0))^21{i≤ N(t_1)}1{j≤ N(t_1)}]≤ C_7λ B

    (h) There exists a constant C_8 such that
    
    𝔼[∑_i=1^+∞∑_j=1^+∞(1{1{r_J_1(j)>a_J_1(j)p_1}≠1{r_J_1(j)>a_J_1(j)p̂_J_1(i),J_1(j)}}..
       ..    +1{1{r_J_1(i)>a_J_1(i)p_1}≠1{r_J_1(i)>a_J_1(i)p̂_J_1(i),J_1(j)}})1{i≤ N(t_1)}1{j≤ N(t_1)}]≤ C_8λ B

    (i) There exists a constant C_9, for all k≥ 1,
    
    𝔼[(p_k+1-p^*(d'_k))^21{k<κ̅}]≤C_9/λ Bk



For part (a),
    
    𝔼[(r-ap^*)(1{r>ap^*}-1{r>ap})| N_1,N_2]
    ≤   𝔼[(ap-ap^*)1{ap^*<r≤ ap}| N_1,N_2]+𝔼[(ap^*-ap)1{ap<r≤ ap^*}| N_1,N_2]

    By Assumption <ref> and (r,a)(p,N_1,N_2),
    
    𝔼[(ap-ap^*)1{ap^*<r≤ ap}| N_1,N_2]
    
                =   𝔼[(ap-ap^*)1{ap^*≤ ap}𝔼[1{ap^*<r≤ ap}| a,p,N_1,N_2]| N_1,N_2]
    ≤   βa̅^2𝔼[(p-p^*)^21{ap^*≤ ap}| N_1,N_2]

    Similarly,
    
    𝔼[(ap^*-ap)1{ap<r≤ ap^*}| N_1,N_2]≤βa̅^2𝔼[(p-p^*)^21{ap<ap^*}| N_1,N_2]

    Thus,
    
    𝔼[(r-ap^*)(1{r>ap^*}-1{r>ap})| N_1,N_2]≤βa̅^2𝔼[(p-p^*)^2| N_1,N_2]

Throughout the following parts, define

    N_max=max{N_Dual,N_LOO,1,N_LOO,2,2}

For part(b),

    𝔼[∑_j=1^+∞(p_J_1(j)-p^*)^21{j≤ N(t_1)}]
    ≤   𝔼[∑_j=1^+∞(p_J_1(j)-p^*)^2(1{d_J_1(j)-d_0>2δ_d}+1{N(t_1)≤ N_max})1{j≤ N(t_1)}]
       +𝔼[∑_j=1^+∞((p_J_1(j)-p^*(d_J_1(j)))+(p^*(d_J_1(j))-p^*))^21{d_J_1(j)-d_0≤ 2δ_d}1{N(t_1)> N_max}1{j≤ N(t_1)}]

By definition of p_j, d_j, and J_1(j) and Lemma <ref>(a), there exists a constant M_1 such that

    𝔼[∑_j=1^+∞(p_J_1(j)-p^*)^2(1{d_J_1(j)-d_0>2δ_d}+1{N(t_1)≤ N_max})1{j≤ N(t_1)}]
    ≤   4r̅^2/d^2𝔼[∑_j=1^+∞(1{d_J_1(j)-d_0>2δ_d}+1{N(t_1)≤ N_max})1{j≤ N(t_1)}]
    
            =   4r̅^2/d^2𝔼[N(t_1)1{d_0-d_0>2δ_d}] +4r̅^2/d^2𝔼[N(t_1)1{N(t_1)≤ N_max}]≤4r̅^2/d^2(M_1+N_max)

By Lemma <ref>,

    𝔼[∑_j=1^+∞(p_J_1(j)-p^*(d_J_1(j)))^21{d_J_1(j)-d_0≤ 2δ_d}1{N(t_1)> N_max}1{j≤ N(t_1)}]
    
        =   𝔼[∑_j=1^+∞𝔼[(p_J_1(j)-p^*(d_J_1(j)))^21{d_J_1(j)-d_0≤ 2δ_d}| N(t_1)]1{N(t_1)> N_max}1{j≤ N(t_1)}]
    ≤   𝔼[∑_j=1^+∞𝔼[sup_d∈Ω_d(p^*_N(t_1)-1(d)-p^*(d))^2| N(t_1)]1{N(t_1)> N_max}1{j≤ N(t_1)}]
    ≤   𝔼[∑_j=1^+∞C_Dual/max{N(t_1)-1,1}1{j≤ N(t_1)}]≤ 2C_Dual

By Lemma <ref>(b),

    𝔼[∑_j=1^+∞(p^*(d_J_1(j))-p^*)^21{d_J_1(j)-d_0≤ 2δ_d}1{N(t_1)> N_max}1{j≤ N(t_1)}]≤ L𝔼[∑_j=1^N(t_1)(d_j-d_0)^2]

By (<ref>), (<ref>), (<ref>), and (<ref>) and Cauchy-Schwarz Inequality, there exists a constant C_1 such that

    𝔼[∑_j=1^+∞(p_J_1(j)-p^*)^21{j≤ N(t_1)}]≤ C_1+L𝔼[∑_j=1^N(t_1)(d_j-d_0)^2]+2√(C_1)√(L𝔼[∑_j=1^N(t_1)(d_j-d_0)^2])

For k≥ 2,

    𝔼[∑_j=1^+∞(p_J_k(j)-p^*)^21{N(t_k-1)<j≤ N(t_k)}]
    
            =   𝔼[∑_j=1^+∞(p_J_k(j)-p^*)^2(1{d_J_k(j)-d_0>2δ_d}+1{N(t_k-1)≤ N_max})1{N(t_k-1)<j≤ N(t_k)}]
       +𝔼[∑_j=1^+∞((p_J_k(j)-p^*(d_J_k(j)))+(p^*(d_J_k(j))-p^*))^2·.
       .                        1{d_J_k(j)-d_0≤ 2δ_d}1{N(t_k-1)> N_max}1{N(t_k-1)<j≤ N(t_k)}]

By definition of p_j, d_j, and J_k(j),

    𝔼[∑_j=1^+∞(p_J_k(j)-p^*)^2(1{d_J_k(j)-d_0>2δ_d}+1{N(t_k-1)≤ N_max})1{N(t_k-1)<j≤ N(t_k)}]
    ≤   4r̅^2/d^2(𝔼[(N(t_k)-N(t_k-1))1{d_k-1-d_0>2δ_d}1{k≤κ̅}]+𝔼[(N(t_k)-N(t_k-1))1{N(t_k-1)≤ N_max}])
    
            =   4λ Br̅^2/d^2(𝔼[1{d_k-1-d_0>2δ_d}1{k≤κ̅}]+𝔼[1{N(t_k-1)≤ N_max}])

By Lemma <ref>(b),

    𝔼[1{d_k-1-d_0>2δ_d}1{k≤κ̅}]≤   𝔼[1{d_k-1-d_k-1'>δ_d}1{k≤κ̅}]
    ≤   d̅^2/δ_d^2𝔼[λ/λ̂_k-1-1^2]
    ≤   d̅^2/δ_d^2·4/λ B (k-1)

By Lemma <ref>(c), there exists a constant M_3 such that

    𝔼[1{N(t_k-1)≤ N_max}]≤M_3/λ B(k-1)

Also, similar to the case in which k=1,

    𝔼[∑_j=1^+∞(p_J_k(j)-p^*(d_J_k(j)))^21{d_J_k(j)-d_0≤ 2δ_d}1{N(t_k-1)> N_max}1{N(t_k-1)<j≤ N(t_k)}]
    ≤   𝔼[C_Dual(N(t_k)-N(t_k-1))/max{N(t_k-1),1}]≤𝔼[C_Dualλ B/max{N(t_k-1),1}]≤2C_Dual/k-1

and

    𝔼[∑_j=1^+∞(p^*(d_J_k(j))-p^*)^21{d_J_k(j)-d_0≤ 2δ_d}1{N(t_k-1)> N_max}1{N(t_k-1)<j≤ N(t_k)}]
    ≤   L𝔼[∑_j=N(t_k-1)+1^N(t_k)(d_j-d_0)^2]

Then, by (<ref>), (<ref>), (<ref>), (<ref>), (<ref>) and Cauchy-Schwarz Inequality, there exists a constant C_2 such that, for k≥ 2,

    𝔼[∑_j=1^+∞(p_J_k(j)-p^*)^21{N(t_k-1)<j≤ N(t_k)}]
    ≤   C/k+L𝔼[∑_j=N(t_k-1)+1^N(t_k)(d_j-d_0)^2]+2√(C/k)√(L𝔼[∑_j=N(t_k-1)+1^N(t_k)(d_j-d_0)^2])

Take C=max{C_1,C_2}, and the proof is complete.

 

For part (c),

    𝔼[∑_j=1^+∞1{1{r_J_1(j)>a_J_1(j)p_J_1(j)}≠1{r_J_1(j)>a_J_1(j)p_J_1(j)}}1{j≤ N(t_1)}]
    ≤   𝔼[∑_j=1^+∞(1{d_J_1(j)-d_0>2δ_d}+1{N(t_1)≤ N_max})1{j≤ N(t_1)}]
       +𝔼[∑_j=1^+∞(1{1{r_J_1(j)>a_J_1(j)p_J_1(j)}≠1{r_J_1(j)>a_J_1(j)p_J_1(j)}}·..
           ..1{d_J_1(j)-d_0≤ 2δ_d}1{N(t_1)> N_max}1{j≤ N(t_1)})]

By Lemma <ref>(a),

    𝔼[∑_j=1^+∞(1{d_J_1(j)-d_0>2δ_d}+1{N(t_1)≤ N_max})1{j≤ N(t_1)}]≤ M_1+N_max

By Lemma <ref>,

    𝔼[∑_j=1^+∞(1{1{r_J_1(j)>a_J_1(j)p_J_1(j)}≠1{r_J_1(j)>a_J_1(j)p_J_1(j)}}·..
           ..1{d_J_1(j)-d_0≤ 2δ_d}1{N(t_1)> N_max}1{j≤ N(t_1)})]
    
            =   𝔼[∑_j=1^+∞𝔼[(1{1{r_J_1(j)>a_J_1(j)p_J_1(j)}≠1{r_J_1(j)>a_J_1(j)p_J_1(j)}}·...
           ...1{d_J_1(j)-d_0≤ 2δ_d}1{N(t_1)> N_max}| N(t_1)]1{j≤ N(t_1)})]
    ≤   C_LOO,1𝔼[N(t_1)/max{N(t_1),1}]≤ C_LOO,1

(<ref>) and (<ref>) completes the proof.

 

For part(d), 

    𝔼[∑_j=1^+∞(p̂_J_K(j)-p^*)^21{N(t_K-1)<j≤ N(T)}]
    
            =   𝔼[∑_j=1^+∞𝔼[(p̂_J_K(j)-p^*)^2| N(t_K-1),N(T)]1{N(t_K-1)<j≤ N(T)}]

Define

    d̂=b_N(t_K-1)/max{N(T)-N(t_K-1),1}1{N(T)-N(t_K-1)≥ 1}+2d̅1{N(T)-N(t_K-1)= 0}

then

    𝔼[(p̂_J_K(j)-p^*)^2| N(t_K-1),N(T)]
    ≤   4r̅^2/d^2𝔼[1{κ̅<K}+1{κ̅=K}1{d̂-d_0>2δ_d}+1{N(T)-N(t_K-1)≤ N_max}| N(t_K-1),N(T)]
       +𝔼[(p̂_J_K(j)-p^*)^21{κ̅=K}1{d̂-d_0≤ 2δ_d}1{N(T)-N(t_K-1)≥ N_max}| N(t_K-1),N(T)]

Thus,

    𝔼[∑_j=1^+∞(p̂_J_K(j)-p^*)^21{N(t_K-1)<j≤ N(T)}]
    ≤   4r̅^2/d^2𝔼[(N(T)-N(t_K-1))(1{κ̅<K}+1{κ̅=K}1{d̂-d_0>2δ_d}+1{N(T)-N(t_K-1)≤ N_max})]
       +𝔼[∑_j=1^+∞𝔼[(p̂_J_K(j)-p^*)^21{κ̅=K}1{d̂-d_0≤ 2δ_d}·..
                               ..1{N(T)-N(t_K-1)≥ N_max}| N(t_K-1),N(T)]1{N(t_K-1)<j≤ N(T)}]

By Lemma <ref>,

    𝔼[(N(T)-N(t_K-1))1{κ̅<K}]
    ≤   𝔼[(N(T)-N(t_K-1))]𝔼[1{d'_K-1-d_0>δ_d}]
    ≤   M_Reslog K/δ_d^2

By Lemma <ref>(f),

    𝔼[(N(T)-N(t_K-1))1{κ̅=K}1{d̂-d_0>2δ_d}]
    ≤   𝔼[(N(T)-N(t_K-1))1{κ̅=K}1{d̂-d_K-1'>δ_d}]
    ≤   𝔼[(N(T)-N(t_K-1))(1{λ B/max{N(T)-N(t_K-1),1}-1≥δ_d/d̅}1{N(T)-N(t_K-1)≥ 1})]≤  M_6

Also,

    𝔼[(N(T)-N(t_K-1))1{N(T)-N(t_K-1)≤ N_max}]≤ N_max

Thus,

    4r̅^2/d^2𝔼[(N(T)-N(t_K-1))(1{κ̅<K}+1{κ̅=K}1{d̂-d_0>2δ_d}+1{N(T)-N(t_K-1)≤ N_max})]
    ≤   4r̅^2/d^2(M_Reslog K/δ_d^2+M_6+N_max)

By Lemma <ref>,

    𝔼[(p̂_J_K(j)-p^*(d̂))^21{κ̅=K}1{d̂-d_0≤ 2δ_d}1{N(T)-N(t_K-1)> N_max}| N(t_K-1),N(T)]
    ≤   𝔼[sup_d∈Ω_dp_max{N(T)-N(t_K-1)-1,1}^*(d)-p^*(d)| N(t_K-1),N(T)]1{N(T)-N(t_K-1)> N_max}
    ≤   C_Dual/max{N(T)-N(t_K-1)-1,1}

By Lemma <ref>(b),

    𝔼[(p^*(d̂)-p^*)^21{κ̅=K}1{d̂-d_0≤ 2δ_d}1{N(T)-N(t_K-1)> N_max}| N(t_K-1),N(T)]
    ≤   𝔼[L(d̂-d_0)^21{κ̅=K}1{d̂-d_0≤ 2δ_d}| N(t_K-1),N(T)]

Together with Cauchy-Schwarz Inequality,

    𝔼[∑_j=1^+∞𝔼[(p̂_J_K(j)-p^*)^21{κ̅=K}1{d̂-d_0≤ 2δ_d}·..
                               ..1{N(T)-N(t_K-1)> N_max}| N(t_K-1),N(T)]1{N(t_K-1)<j≤ N(T)}]
    ≤   𝔼[C_Dual(N(T)-N(t_K-1))/max{N(T)-N(t_K-1)-1,1}]+𝔼[(N(T)-N(t_K-1))L(d̂-d_0)^21{κ̅=K}1{d̂-d_0≤ 2δ_d}]
       +2√(𝔼[C_Dual(N(T)-N(t_K-1))/max{N(T)-N(t_K-1)-1,1}])√(𝔼[(N(T)-N(t_K-1))L(d̂-d_0)^21{κ̅=K}1{d̂-d_0≤ 2δ_d}])
    ≤    2C_Daul+𝔼[(N(T)-N(t_K-1))L((d̂-d'_K-1)+(d'_K-1-d_0))^21{κ̅=K}1{d̂-d_0≤ 2δ_d}]
       +2√(2C_Dual)√(𝔼[(N(T)-N(t_K-1))L(d̂-d_0)^21{κ̅=K}1{d̂-d_0≤ 2δ_d}])

By Lemma <ref>,

    𝔼[(N(T)-N(t_K-1))L(d'_K-1-d_0)^21{κ̅=K}1{d̂-d_0≤ 2δ_d}]
    ≤   L𝔼[N(T)-N(t_K-1)]𝔼[(d'_K-1-d_0)^2]
    ≤   L M_Reslog K

By Lemma <ref>(e),

    𝔼[(N(T)-N(t_K-1))L(d̂-d'_K-1)^21{κ̅=K}1{d̂-d_0≤ 2δ_d}]
    ≤   Ld̅^2𝔼[(N(T)-N(t_K-1))(λ B/max{N(T)-N(t_K-1),1}-1)^21{N(T)-N(t_K-1)≥ 1}]
    ≤   M_5

Together with Cauchy-Schwarz Inequality,

    𝔼[(N(T)-N(t_K-1))L((d̂-d'_K-1)+(d'_K-1-d_0))^21{κ̅=K}1{d̂-d_0≤ 2δ_d}]
    ≤    M_Reslog K + M_5 + 2√(M_Reslog K)√(M_5)

(<ref>), (<ref>), and (<ref>) imply that there exists a constant Ĉ such that

    𝔼[∑_j=1^+∞(p̂_J_K(j)-p^*)^21{N(t_K-1)<j≤ N(T)}]≤Ĉlog K

For part (e),

    𝔼[∑_j=1^+∞1{1{r_J_K(j)>a_J_K(j)p̂_J_K(j)}≠1{r_J_K(j)>a_J_K(j)p_J_K(j)}}1{N(t_K-1)<j≤ N(T)}]
    
            =   𝔼[∑_j=1^+∞𝔼[1{1{r_J_K(j)>a_J_K(j)p̂_J_K(j)}≠1{r_J_K(j)>a_J_K(j)p_J_K(j)}}| N(t_K-1),N(T)]1{N(t_K-1)<j≤ N(T)}]
    ≤   𝔼[(N(T)-N(t_K-1))(1{κ̅<K}+1{κ̅=K}1{d̂-d_0>2δ_d}+1{N(T)-N(t_K-1)≤ N_max})]
       +𝔼[∑_j=1^+∞𝔼[1{1{r_J_K(j)>a_J_K(j)p̂_J_K(j)}≠1{r_J_K(j)>a_J_K(j)p_J_K(j)}}·..
                       ..1{d̂-d_0≤ 2δ_d}1{N(T)-N(t_K-1)> N_max}| N(t_K-1),N(T)]1{N(t_K-1)<j≤ N(T)}]

By the same proof for (<ref>),

    𝔼[1{κ̅<K}+1{κ̅=K}1{d̂-d_0>2δ_d}+1{N(T)-N(t_K-1)≤ N_max}| N(t_K-1),N(T)]
    ≤   M_Reslog K/δ_d^2+M_6+N_max

By Lemma <ref>,

    𝔼[1{1{r_J_K(j)>a_J_K(j)p̂_J_K(j)}≠1{r_J_K(j)>a_J_K(j)p_J_K(j)}}·.
                               .1{d̂-d_0≤ 2δ_d}1{N(T)-N(t_K-1)> N_max}| N(t_K-1),N(T)]
    ≤   C_LOO,1/max{N(T)-N(t_K-1),1}

Then, there exists a constant C,

    𝔼[∑_j=1^+∞1{1{r_J_K(j)>a_J_K(j)p̂_J_K(j)}≠1{r_J_K(j)>a_J_K(j)p_J_K(j)}}1{N(t_K-1)<j≤ N(T)}]
    ≤   𝔼[C_LOO,1(N(T)-N(t_K-1))/max{N(T)-N(t_K-1),1}]+M_Reslog K/δ_d^2+M_6+N_max
    ≤   Clog K +C

For part (f), we only show the second inequality, and the proof for the first inequality is similar.
    
    𝔼[a_1(1{r_1>a_1p^*(d)}-1{r_1>a_1p})a_2(1{r_2>a_2p^*(d)}-1{r_2>a_2p})|d,N_1,N_2]
    
                =    𝔼[a_1𝔼[1{r_1>a_1p^*(d)}-1{r_1>a_1p}| a_1,a_2,p,d,N_1,N_2].
               .a_2𝔼[1{r_2>a_2p^*(d)}-1{r_2>a_2p}| a_1,a_2,p,d,N_1,N_2]|d,N_1,N_2]
    ≤   β^2a̅^4𝔼[(p-p^*(d))^2|d,N_1, N_2]

    For part (g),
    
    𝔼[∑_i=1^+∞∑_j=1^+∞(p̂_J_1(i),J_1(j)-p^*(d'_0))^21{i≤ N(t_1)}1{j≤ N(t_1)}]
    ≤   𝔼[4r̅^2N(t_1)^2/d^2(1{N(t_1)≤ N_max}+1{d_0-d'_0>2δ_d})]
       +𝔼[∑_i=1^+∞∑_j=1^+∞(p̂_J_1(i),J_1(j)-p^*(d'_0))^21{N(t_1)>N_max}1{d_0-d'_0≤ 2δ_d}1{i≤ N(t_1)}1{j≤ N(t_1)}]

    By Lemma <ref>(g), 
    
    𝔼[4r̅^2N(t_1)^2/d^2(1{N(t_1)≤ N_max}+1{d_0-d'_0>2δ_d})]≤4r̅^2/d^2(N_max^2+M_7)

    Also, with similar arguments in part (b), by Lemma <ref>,
    
    𝔼[∑_i=1^+∞∑_j=1^+∞((p̂_J_1(i),J_1(j)-p^*(d_0)))^21{N(t_1)>N_max}1{d_0-d'_0≤ 2δ_d}1{i≤ N(t_1)}1{j≤ N(t_1)}]
    ≤   𝔼[C_DualN(t_1)^2/max{N(t_1)-2,1}]≤ 3C_Dualλ B

    and by Lemma <ref>(b) and Lemma <ref>(h)
    
    𝔼[∑_i=1^+∞∑_j=1^+∞((p^*(d_0)-p^*(d'_0)))^21{N(t_1)>N_max}1{d_0-d'_0≤ 2δ_d}1{i≤ N(t_1)}1{j≤ N(t_1)}]
    ≤   L𝔼[N(t_1)^2(d_0-d'_0)^2]≤ 2d̅^2Lλ B

    Together with Cauchy-Schwarz Inequality, we can conclude that there exists a constant C_7 such that
    
    𝔼[∑_i=1^+∞∑_j=1^+∞(p̂_J_1(i),J_1(j)-p^*(d'_0))^21{i≤ N(t_1)}1{j≤ N(t_1)}]≤ C_7λ B

    For part (h), with similar arguments in part (c), by Lemma <ref>,
    
    𝔼[1{1{r_J_1(i)>a_J_1(i)p̂_J_1(i),J_1(j)}≠1{r_J_1(i)>a_J_1(i)p_1}}.
           +.1{r_J_1(j)>a_J_1(j)p̂_J_1(i),J_1(j)}≠1{r_J_1(j)>a_J_1(j)p_1}| N(t_1)]
    ≤   1{N(t_1)≤ N_max}+1{d_0-d'_0>2δ_d}+C_LOO,2/max{N(t_1),1}

    Then, by Lemma <ref>(g),
    
    𝔼[∑_i=1^+∞∑_j=1^+∞(1{1{r_J_1(j)>a_J_1(j)p_1}≠1{r_J_1(j)>a_J_1(j)p̂_J_1(i),J_1(j)}}..
       ..    +1{1{r_J_1(i)>a_J_1(i)p_1}≠1{r_J_1(i)>a_J_1(i)p̂_J_1(i),J_1(j)}})1{i≤ N(t_1)}1{j≤ N(t_1)}]
    ≤   𝔼[N(t_1)^21{N(t_1)≤ N_max}]+𝔼[N(t_1)^21{d_0-d'_0>2δ_d}]+𝔼[N(t_1)^2C_LOO,2/max{N(t_1),1}]
    ≤    N_max^2 + M_7 + C_LOO,2λ B

    which completes the proof.

     

    For part (i), given k≥ 1,
    
    𝔼[(p_k+1-p^*(d'_k))^21{k<κ̅}]
    ≤   4r̅^2/d^2𝔼[1{N(t_k)≤ N_max}+1{d_k-d'_k>δ_d}1{k<κ̅}]
       +𝔼[((p_k+1-p^*(d_k))+(p^*(d_k)-p^*(d'_k)))^21{N(t_k)> N_max}1{d_k-d'_k≤δ_d}1{k<κ̅}]

    With similar arguments in part (b), by Lemma <ref>(b) and (c),
    
    4r̅^2/d^2𝔼[1{N(t_k)≤ N_max}+1{d_k-d'_k>δ_d}1{k<κ̅}]≤4r̅^2/d^2(M_3/λ Bk+4d̅^2/δ_d^2λ Bk)

    and by Lemma <ref>,
    
    𝔼[(p_k+1-p^*(d_k))^21{N(t_k)> N_max}1{d_k-d'_k≤δ_d}1{k<κ̅}]≤2C_Dual/λ Bk

    and by Lemma <ref>(b) and Lemma <ref>(b),
    
    𝔼[(p^*(d_k)-p^*(d'_k))^21{N(t_k)> N_max}1{d_k-d'_k≤δ_d}1{k<κ̅}]≤ Ld̅^2𝔼[λ/λ̂_k-1^2]≤4Ld̅^2/λ Bk

    Then, together with Cauchy-Schwarz Inequality, there exists a constant C_9, for all k≥ 1,
    
    𝔼[(p_k+1-p^*(d'_k))^21{k<κ̅}]≤C_9/λ Bk



     

    (a) If δ_d <d̅/2 and λ B> = d̅-2δ_d/δ_d, then there exists a constant M_1 such that
    
    𝔼[N(t_1)1{d_0-d_0>2δ_d}]≤ M_1

    (b) For any k≥ 2, 
    
    λ B𝔼[λ/λ̂_k-1-1^2]≤4/k-1

    (c) If λ B>1, there exists a constant M_3 such that, for any k≥ 2,
    
    λ Bℙ{N(t_k-1)≤ N_max}≤M_3/k-1

    (d)
    
    𝔼[N(t_1)d_0-d_0^2]≤d̅^2

    (e) There exists a constant M_5 such that
    
    𝔼[(N(T)-N(t_K-1))(λ B/max{N(T)-N(t_K-1),1}-1)^21{N(T)-N(t_K-1)≥ 1}]≤ M_5

    (f) If λ B> = 2d̅-2δ_d/δ_d, there exists a constant M_6 such that
    
    𝔼[(N(T)-N(t_K-1))(1{λ B/max{N(T)-N(t_K-1),1}-1≥δ_d/d̅}1{N(T)-N(t_K-1)≥ 1})]≤ M_6

    (g) If δ_d <d̅/2 and λ B> = d̅-2δ_d/2δ_d, there exists a constant M_7 such that
    
    𝔼[N(t_1)^21{d_0-d'_0>2δ_d}]≤ M_7

    (h) If λ B>1,
    
    𝔼[N(t_1)^2d_0-d'_0^2]≤ 2d̅^2λ B

    (i) There exists constants M_8 such that
    
    𝔼[exp(-ϵ_d^2(N(T)-N(t_K-1))/2a̅^2)]≤exp(-M_8λ B)



 

For part (a),

    𝔼[N(t_1)1{d_0-d_0>2δ_d}]   ≤√(𝔼[N(t_1)^2])√(ℙ{d_0-d_0>2δ_d})
       ≤√(λ B+(λ B)^2)√(ℙ{λ/λ̂_0-1>2δ_d/d̅})

and


    ℙ{λ/λ̂_0-1>2δ_d/d̅}≤   ℙ{N(t_1)-λ B≤ -2δ_d/d̅+2δ_dλ B} + ℙ{N(t_1)-λ B≥δ_d/d̅-2δ_dλ B}
    ≤   exp(-2δ_d^2/(d̅+2δ_d)(d̅+4δ_d)λ B) + exp(-δ_d^2/2(d̅-δ_d)(d̅-2δ_d)λ B)

Thus, there exists a constant M_1 such that

    𝔼[N(t_1)1{d_0-d_0>2δ_d}]
    ≤   √(λ B+(λ B)^2)√(exp(-2δ_d^2/(d̅+2δ_d)(d̅+4δ_d)λ B) + exp(-δ_d^2/2(d̅-δ_d)(d̅-2δ_d)λ B))
    ≤    M_1

For part (b), Let N be a Poisson random variable with mean λ, then

    λ𝔼[(λ/N+1-1)^2]=   ∑_x=0^+∞λ^2/(x+1)^2λ^x+1exp(-λ)/x!-2λ/x+1λ^x+1exp(-λ)/x!+λ^x+1exp(-λ)/x!
    
        =   ∑_x=3^+∞(x-1)x/x-2λ^xexp(-λ)/x!-2∑_x=2^+∞xλ^xexp(-λ)/x!+λ
    
        =   𝔼[(N-1)N/N-21{N≥ 3}-2N1{N≥ 2}]+λ
    
        =    𝔼[N1{N≥ 3}+N/N-21{N≥ 3}-2N1{N≥ 2}] +λ
    ≤   λ-ℙ{N=1}-2ℙ{N=2}+3-2(λ-ℙ{N=1})+λ
    ≤   4

Thus, for any k≥ 2,

    λ B𝔼[λ/λ̂_k-1-1^2]≤λ B·4/λ(k-1)B≤4/k-1

For part (c), if λ B>1, for any k≥ 2,

    λ Bℙ{N(t_k-1)≤ N_max}
    
        =   λ B∑_x=0^N_max((k-1)λ B)^xexp(-(k-1)λ B)/x!
    ≤   N_maxλ B (λ(k-1)B)^N_maxexp(-λ(k-1)B)
    
        =    1/k-1· N_max (λ(k-1)B)^N_max+1exp(-λ(k-1)B)

Thus, there exists a constant M_3 such that

    λ Bℙ{N(t_k-1)≤ N_max}≤M_3/k-1

For part (d),

    𝔼[N(t_1)d_0-d_0^2]≤   d̅^2𝔼[N(t_1)λ/λ̂_0-1^2]
    ≤   d̅^2𝔼[(N(t_1)-λ B+1)^2/N(t_1)+1]
    
        =   d̅^2∑_x=0^+∞(x+1-λ B)^2/x+1(λ B)^xexp(-λ)/x!
    
        =   d̅^2(∑_x=0^+∞(x+1)(λ B)^xexp(-λ)/x!-2λ B∑_x=0^+∞(λ B)^xexp(-λ)/x!+λ B∑_x=0^+∞(λ B)^x+1exp(-λ)/(x+1)!)
    ≤   d̅^2(λ B+1-2λ B+λ B)=d̅^2

For part (e), 

    𝔼[(N(T)-N(t_K-1))(λ B/max{N(T)-N(t_K-1),1}-1)^21{N(T)-N(t_K-1)≥ 1}]
    
            =   ∑_x=1^+∞(λ B-x)^2/x(λ B)^xexp(-λ B)/x!
    ≤   2(∑_x=1^+∞(λ B)^x+2exp(-λ B)/(x+1)!-2∑_x=1^+∞x(λ B)^x+1exp(-λ B)/(x+1)!+∑_x=1^+∞x^2(λ B)^xexp(-λ B)/(x+1)!)
    ≤   2(λ B∑_x=1^+∞(λ B)^x+1exp(-λ B)/(x+1)!-2∑_x=1^+∞(x+1)(λ B)^x+1exp(-λ B)/(x+1)!.
       .+2∑_x=1^+∞(λ B)^x+1exp(-λ B)/(x+1)!+∑_x=0^+∞(x+1)(λ B)^xexp(-λ B)/x!)
    ≤   2(λ B-2λ B + λ Bexp(-λ B)+2+λ B + 1)

Thus, there exists a constant M_5 such that

    𝔼[(N(T)-N(t_K-1))(λ B/max{N(T)-N(t_K-1),1}-1)^21{N(T)-N(t_K-1)≥ 1}]≤ M_5

For part (f),

    𝔼[(N(T)-N(t_K-1))(1{λ B/max{N(T)-N(t_K-1),1}-1≥δ_d/d̅}1{N(T)-N(t_K-1)≥ 1})]
    ≤   √(𝔼[(N(T)-N(t_K-1))^2])√(ℙ{λ B/max{N(T)-N(t_K-1),1}-1≥δ_d/d̅})
    ≤   √(λ B + (λ B)^2)√(ℙ{λ B/max{N(T)-N(t_K-1),1}-1≥δ_d/d̅})

Since N(T)-N(t_K-1) and N(t_1) have the same distribution, by similar arguments in part (a),

    ℙ{λ B/max{N(T)-N(t_K-1),1}-1≥δ_d/d̅}   ≤ℙ{N(t_1)-λ B≤ -δ_d/d̅+δ_dλ B} + ℙ{N(t_1)-λ B≥δ_d/2d̅-2δ_dλ B}
       ≤exp(-δ_d^2/(d̅+δ_d)(2d̅+4δ_d)λ B) + exp(-δ_d^2/4(2d̅-δ_d)(d̅-δ_d)λ B)

Thus, there exists a constant M_6 such that

    𝔼[(N(T)-N(t_K-1))(1{λ B/max{N(T)-N(t_K-1),1}-1≥δ_d/d̅}1{N(T)-N(t_K-1)≥ 1})]≤ M_6

For part (g),

    𝔼[N(t_1)^21{d_0-d'_0>2δ_d}]≤√(𝔼[N(t_1)^4])√(ℙ{d_0-d_0>2δ_d})

Then, by the upper bound of ℙ{d_0-d_0>2δ_d} in the proof of part (a),

    𝔼[N(t_1)^21{d_0-d'_0>2δ_d}]
    ≤   √((λ B)^4+6(λ B)^3+7(λ B)^2+λ B)√(exp(-2δ_d^2/(d̅+2δ_d)(d̅+4δ_d)λ B) + exp(-δ_d^2/2(d̅-δ_d)(d̅-2δ_d)λ B))

Thus, there exists a constant M_7 such that

    𝔼[N(t_1)^21{d_0-d'_0>2δ_d}]≤ M_7

For part (h),

    𝔼[N(t_1)^2d_0-d'_0^2]=   d_0'^2𝔼[N(t_1)^2(N(t_1)+1-λ B)^2/(N(t_1+1))^2]
    ≤   d_0'^2𝔼[(N(t_1)+1-λ B)^2]
    ≤   2d̅^2λ B

For part (i), let N be a Poisson random variable with mean λ and c be a constant,

    𝔼[exp(-cN)]=   ∑_x=0^+∞(e^-cλ)^xe^-λ/x!
    
            =   e^-λ/e^-λ e^-c∑_x=0^+∞(e^-cλ)^xe^-λ e^-c/x!
    
            =   exp(-(1-exp(-c))λ)

Thus, there exists constants M_8 such that
    
    𝔼[exp(-ϵ_d^2(N(T)-N(t_K-1))/2a̅^2)]≤exp(-M_8λ B)




 §.§ Proof of Proposition <ref>


    Define
    
    [                                                                                                    R_T=                                                                                                     max    ∑_j=1^N(t_1)r_j1{V_j+W_j> t_1}x_j+∑_j=N(t_1)+1^N(t_K-1)r_jx_j+∑_j=N(t_K-1)+1^N(T)r_j1{V_j+W_j> T}x_j;                                                                                                                                                                                                             S.T ∑_j=1^N(t_1)a_j1{V_j+W_j> t_1}x_j+∑_j=N(t_1)+1^N(t_K-1)a_jx_j+∑_j=N(t_K-1)+1^N(T)a_j1{V_j+W_j> T}x_j≤ b;                                                                                                                                                                                         0≤ x_j≤ 1  ∀ j=1,⋯,N(T) ]

    Let
    
    p = min_p≥ 0 bp + ∑_j=1^N(t_1)(r_j-a_jp)^+1{V_j+W_j> t_1}+∑_j=N(t_1)+1^N(t_K-1)(r_j-a_jp)^++∑_j=N(t_K-1)+1^N(T)(r_j-a_jp)^+1{V_j+W_j> T}

    By the Duality Theory of Linear Programming, the following holds almost sure.
    
    R_T^*-R_T
    ≤   (bp+∑_j=1^N(T)(r_j-a_jp)^+)
       -(bp+∑_j=1^N(t_1)(r_j-a_jp)^+1{V_j+W_j> t_1}+∑_j=N(t_1)+1^N(t_K-1)(r_j-a_jp)^++∑_j=N(t_K-1)+1^N(T)(r_j-a_jp)^+1{V_j+W_j> T})
    ≤   ∑_j=1^N(t_1)(r_j-a_jp)^+1{V_j+W_j≤ t_1}+∑_j=N(t_K-1)+1^N(T)(r_j-a_jp)^+1{V_j+W_j≤ T}
    ≤   (r̅+a̅r̅/a)(∑_j=1^N(t_1)1{V_j+W_j≤ t_1}+∑_j=N(t_K-1)+1^N(T)1{V_j+W_j≤ T})

    Thus,
    
    𝔼[R_T^*-R_T]≤(r̅+a̅r̅/a)(𝔼[∑_j=1^N(t_1)1{V_j+W_j≤ t_1}]+𝔼[∑_j=N(t_K-1)+1^N(T)1{V_j+W_j≤ T}])

    By the property of Poisson process, condition on {N(t_1)=n}, {V_j}_j=1^n are i.i.d. Uniform(0,B) random variables. Together with the Assumption <ref>,
    
    𝔼[∑_j=1^N(t_1)1{V_j+W_j≤ t_1}| N(t_1)=n]
    
                =   nℙ{V_j+W_j≤ B| N(t_1)=n}
    
                =   n/B∫_0^BF(u)du

    Thus,
    
    𝔼[∑_j=1^N(t_1)1{V_j+W_j≤ t_1}]=λ∫_0^BF(u)du

    Similarly,
    
    𝔼[∑_j=N(t_K-1)+1^N(T)1{V_j+W_j≤ T}]=λ∫_0^BF(u)du

    In addition, we can show
    
    𝔼[R_T-R_T(π_4)]≤ O(log K)

    by almost the same arguments in the proof of Theorem <ref>, and we omit the details. Thus, there exists constant C_1 and C_2 such that
    
    𝔼[R_T^*-R_T(π_3)]≤ C_1log(T/B)+C_2λ∫_0^BF(u)du

    which completes the proof.





 Paper Title: Nonlinear Derivative-free Constrained Optimization with a Mixed Penalty-Logarithmic Barrier Approach and Direct Search.





Nonlinear Derivative-free Constrained Optimization with a Mixed Penalty-Logarithmic Barrier Approach and Direct Search
    Andrea Brilli “Sapienza” University of Rome, Department of Computer Control and Management Engineering “A. Ruberti”, Rome, Italy  Ana L. Custódio NOVA School of Science and Technology, Center for Mathematics and Applications (NOVA MATH), Campus de Caparica, 2829-516 Caparica, Portugal . This work was funded by national funds through FCT - Fundação para a Ciência e a Tecnologia I.P., under the scope of projects UIDP/00297/2020 and UIDB/00297/2020 (Center for Mathematics and Applications). Giampaolo Liuzzi“Sapienza” University of Rome, Department of Computer Control and Management Engineering “A. Ruberti”, Rome, Italy  Everton J. SilvaNOVA School of Science and Technology, Center for Mathematics and Applications (NOVA MATH), Campus de Caparica, 2829-516 Caparica, .  This work was funded by national funds through FCT - Fundação para a Ciência e a Tecnologia I.P., under the scope of projects UI/BD/151246/2021, UIDP/00297/2020, and UIDB/00297/2020 (Center for Mathematics and Applications).
    
===========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================

§ ABSTRACT 
In this work, we propose the joint use of a mixed penalty-logarithmic barrier approach and generating set search, for addressing nonlinearly constrained derivative-free optimization problems. A merit function is considered, wherein the set of inequality constraints is divided into two groups: one treated with a logarithmic barrier approach, and another, along with the equality constraints, addressed using a penalization term. This strategy, initially proposed in the framework of LOG-DFL <cit.>, is adapted and incorporated into SID-PSM <cit.> algorithm, a generalized pattern search method, allowing to effectively handle general nonlinear constraints. Under reasonable assumptions regarding the smoothness of the functions, convergence is established, without any convexity assumptions. Using CUTEst test problems, numerical experiments demonstrate the robustness, efficiency, and overall effectiveness of the proposed method, when compared with state-of-art solvers and with the original SID-PSM and LOG-DFL implementations.



Keywords: Derivative-free optimization; Constrained optimization; Interior point methods; Direct search. 


AMS Classification: 65K05, 90C30, 90C56.





§ INTRODUCTION

In this work, we consider the derivative-free optimization problem with general constraints (linear and nonlinear), defined by:

    [      min f(); s.t.  g()≤ 0;      h() = 0;          ∈ X;              ]

where f:X⊆^n→∪{+∞}, g:X⊆^n→∪+∞^m, h:X⊆^n→∪+∞^p, and X=∈^n|A≤b, with A∈^q× n and b∈^q. Furthermore, we assume that the functions involved (f, g, and h) are continuously differentiable, but derivatives  cannot be either calculated or explicitly approximated. This is a common setting in a context of simulation-based optimization, where function evaluations are computed through complex and expensive computational simulations <cit.>. 

Considering an initial point _0∈ X we define two sets: 𝒢^ log=ℓ| g_ℓ(_0)<0, and 𝒢^ ext=ℓ| g_ℓ(_0)≥ 0. We denote the sets defined by the inequality constraints as Ω_𝒢^ log and Ω_𝒢^ ext, respectively. The first corresponds to the inequality constraints, to be addressed with a logarithmic barrier approach, while the constraints in the second will be handled using a penalization term. Further explanation will be given in Section <ref>. Additionally, Ω_h corresponds to the set defined by the equality constraints, and X is a polyhedron, defining a set that we assume to be compact. Therefore, the feasible region , which is assumed to be nonempty, is given by 
    = X∩Ω_𝒢^ log∩Ω_𝒢^ ext∩Ω_h≠∅,
 and it is also a compact set.

Constrained derivative-free optimization is not new. In the context of pattern search methods, the early works are co-authored by Lewis and Torczon, first considering bound <cit.> or linearly constrained problems <cit.>, and after for general nonlinear constraints <cit.>. When the constraints are only linear, inspired by the work of May <cit.>, procedures were developed that allow to conform the directions to be used by the algorithms to the geometry of the feasible region imposed by the nearby constraints <cit.>, including specific strategies to address the degenerated case <cit.>. In the presence of nonlinear constraints, augmented Lagrangian approaches have been proposed <cit.>, reducing the problem solution to a sequence of bound constrained minimizations of an augmented Lagrangian function.

In the original presentation of Mesh Adaptive Direct Search (MADS) <cit.>, a generalization of pattern search methods, constraints were addressed with an extreme barrier approach, only evaluating feasible points. If this saves in function evaluations, a very relevant feature for the target problem class, it does not take advantage on the local information obtained about the feasible region. Following the filter approaches proposed for derivative-based optimization <cit.> and already explored in pattern search methods <cit.>, linear and nonlinear inequalities started to be treated in MADS with the progressive barrier technique <cit.>. Later, the approach was extended to linear equality constraints <cit.>, by reformulating the optimization problem, possibly reducing the number of original variables.

For directional direct search methods using linesearch, other approaches have been taken to address general nonlinear constraints. These include nonsmooth exact penalty functions <cit.>, where the original nonlinearly constrained problem is converted into the unconstrained or linearly constrained minimization of a nonsmooth exact penalty function. To overcome the limitations associated to this approach, in <cit.> a sequential penalty approach has been studied, based on the smoothing of the nondifferentiable exact penalty function, including a well-defined strategy for updating the penalty parameter.  Recently, in the same algorithmic framework, Brilli et. al. <cit.> proposed the use of a merit function that handles inequality constraints by means of a logarithmic barrier approach and equality constraints by considering a penalization term. This approach allows an easy management of relaxable and unrelaxable constraints and avoids the abrupt discontinuities introduced by the extreme barrier approach.

In this work, the strategy of <cit.> will be adapted and incorporated into generalized pattern search. Starting from the SID-PSM algorithm <cit.>, an implementation of a generalized pattern search method, where polynomial models are used for both the search and the poll steps to improve the numerical performance of the code, LOG-DS has been developed, a direct search method able to explicitly address nonlinear constraints by a mixed penalty-logarithmic barrier approach.

The manuscript is organized as follows. Section <ref> details the proposed algorithmic structure and the related convergence properties are analyzed in Section <ref>. Details of the numerical implementation are provided in Section <ref> and numerical results are reported in Section <ref>. We summarize our conclusions in Section <ref>, also discussing some future avenues of research. An appendix completes the paper, including some auxiliary technical results.



 §.§ Notation

Throughout this paper, vectors will be written in lowercase boldface (e.g., v∈^n, n ≥ 2) while matrices will be written in uppercase boldface (e.g., S∈^n× p). The set of column vectors of a matrix D will be denoted by , and more generally sets such as , , will be denoted by blackboard letters. The set of nonnegative real numbers will be denoted by _+. Sequences indexed by  will be denoted by {a_k}_k∈ or {a_k} in absence of ambiguity. Given a point x∈^n and a set Ω⊂^n, we use the notation T_Ω(x) to denote  the tangent cone to Ω at x.



§ A DIRECT SEARCH ALGORITHM FOR NONLINEAR CONSTRAINED OPTIMIZATION



This section is devoted to the introduction of a new DS algorithm, based on the sequential minimization of an adequate merit function, to solve nonlinearly constrained derivative-free optimization problems. Sequential minimization consecutively solves linearly constrained subproblems, differing from each other in a penalty parameter ρ.

Following the original idea of <cit.>, to solve Problem (<ref>), a merit function is considered:

    Z(;ρ) = f() - ρ∑_ℓ∈𝒢^ loglog(-g_ℓ()) +
    1/ρ^ν-1(∑_ℓ∈𝒢^ ext(max{g_ℓ(),0})^ν+∑_j=1^p |h_j()|^ν),

where ρ>0 and ν∈(1,2]. Therefore, Z(;ρ)=+∞, for all ∈ X such that g_ℓ()≮ 0, for ℓ∈𝒢^ log.

For the linear constraints, defining the set X, strategies that conform the search directions to the geometry of the nearby feasible region are implemented, using the construction procedure detailed in <cit.>.

The following problem

    [         min Z(;ρ); s.t. ∈ X∩Ω_𝒢^ log;                   ]


will be considered at each iteration, with an adequate strategy for updating the parameter ρ>0. We assume that a point _0∈ X∩Ω_𝒢^ log is known, so that the set X ∩Ω_𝒢^ log is nonempty. 
 

Let ρ > 0, ν∈(1,2], and α∈ be given parameters. If X∩Ω_𝒢^ log is compact, then, 

    L(α) = {∈ X∩Ω_𝒢^ log: Z(;ρ)≤α}

is compact.


The set L(α) is bounded since, by definition, it is a subset of X∩Ω_𝒢^ log which is compact. It remains to prove that L(α) is closed. To this end, we will show that for any sequence {_k}⊂ L(α) such that lim_k→+∞_k =, it results ∈ L(α). 

Since _k∈ L(α), for all k, we have

    Z(_k;ρ) ≤α.

Taking the limit for k→+∞ in the above relation we get

    lim_k→+∞Z(_k;ρ) ≤α.

Then, ∉∂Ω_𝒢^ log, otherwise we would get lim_k→+∞Z(_k;ρ) =+∞ >α. Thus, ∈ X∩Ω_𝒢^ log. We know that Z(;ρ) is continuous on X∩Ω_𝒢^ log. Thus, by (<ref>), we have 

    lim_k→+∞Z(_k;ρ) = Z(;ρ)≤α

which means that ∈ L(α). That concludes the proof.


One might notice that nonlinear constraints still appear, due to the presence of the inequalities in 𝒢^ log which still need to be satisfied. Nevertheless, considering the properties of the merit function and the structure of the proposed scheme, it will be clear that any point ∉Ω_𝒢^ log will be rejected. That is, Problem <ref> can be reformulated using only linear constraints, i.e.,

    [ min Z(;ρ); s.t. ∈ X. ]


In the proposed approach, the acceptance of new points will rely on the notion of sufficient decrease, justifying the classification of the algorithm as a Generating Set Search (GSS) method. The next definition adjusts the concept of forcing function (see <cit.>) and is used to define the sufficient decrease condition required to accept new points in LOG-DS.



Let ξ: [0,+∞)→ [0,+∞) be a continuous and nondecreasing function. We say that ξ is a forcing function if:


  - ξ(t)/t→ 0 when t↓ 0;

  - if ξ(t)→ 0 then t→ 0.



The proposed algorithmic structure is detailed below.


[]0.95
 LOG-DS



Data. _0∈ X such that g_ℓ(_0)<0 for all ℓ∈𝒢^ log, D a family of sets of directions, α_0 > 0, ρ_0>0, ν∈(1,2],
θ_α,θ_ρ∈ (0,1), ϕ≥ 1, and β>1.



For k = 0,1,2,… do


   Step 1. (Search Step, optional)



 If _k∈ X can be computed such that Z(_k;ρ_k)≤ Z(_k;ρ_k)-ξ(α_k),

 Then set _k+1=_k, α_k+1 =ϕα_k, and go to Step 3.


   Step 2. (Poll Step)

   Select a set _k∈D 

 If ∃ _k^i∈_k: _k+α_k _k^i∈ X and Z(_k+α_k _k^i;ρ_k)≤ Z(_k;ρ_k)-ξ(α_k), 

 Then set _k+1 = _k + α_k _k^i
and α_k+1=ϕα_k.

   Else set _k+1 = _k and α_k+1 = θ_αα_k. 



   Step 3. Set (g_min)_k = min_ℓ∈𝒢^ log{|g_ℓ(_k+1)|}

   If α_k+1≤min{ρ_k^β,(g_min)_k^2}  and α_k+1<α_k,

 Then set ρ_k+1 =θ_ρρ_k.

 Else set ρ_k+1 = ρ_k.


Endfor




Following the general structure proposed by Audet and Dennis <cit.> for generalized pattern search, each iteration of LOG-DS is organized into two main steps, plus an additional one related to the novel approach: 


  * Step 1 and Step 2, namely the Search Step and the Poll Step, which are part of the basic structure of a generalized pattern search method, here applied to the solution of (<ref>); 

  * Step 3, the penalty parameter updating step, which is the original feature of the proposed approach.


As usual, the search step is very flexible, not even requiring the projection of the generated points in some type of implicit mesh, since a sufficient decrease condition is used for the acceptance of new iterates. As it will be detailed in Section <ref>,  the original SID-PSM algorithm uses quadratic interpolation models, which are minimized to generate new trial points. The latter approach is adapted into LOG-DS as described in Section 4.1, but since it is not relevant for establishing convergence properties, for now it will be omitted.





For the theoretical analysis, we will consider general assumptions on the directions used by the algorithm, formalized below.



For every ∈ X, i.e., such that A≤b:

    I_X()     = {i |a_i^⊤ = b_i}  
    
    T_X()     = {∈^n |a_i^⊤≤ 0, i∈ I_X()}  
    
    T^∘_X()     = {v∈^n |v = ∑_i∈
    I_X()λ_i a_i, λ_i≥
    0}



Given an iterate _k∈ X (possibly not belonging to the boundary of X), it is important to be able to capture the geometry of the set X near _k. Hence, the previous sets are approximated by the following ones, depending on a parameter ε > 0:




    I_X(_k,ε)     = {i |a_i^⊤_k ≥ b_i -ε}  
    
    T_X(_k,ε)      = {∈^n |a_i^⊤≤ 0, i∈ I_X(_k,ε)}  
    
    T^∘_X(_k,ε)     = {v∈^n |v = ∑_i∈
    I_X(_k,ε)λ_i a_i, λ_i≥
    0}



A relation between the two groups of sets of Definition <ref> and Definition <ref> has been established in <cit.>. We recall the result in the following proposition.

 Let {_k}_k∈ℕ be a sequence of points in X, converging to ^*∈ X. Then, there exists an ε^*>0 (depending only on ^*) such that for any ε∈ (0,ε^*] there exists k_ε∈ℕ such that

    I_X(^*)     = I_X(_k,ε) 
    
    T_X(^*)     = T_X(_k,ε)

for all k≥ k_ε.


We are now in position of specifying the requests on the sets of directions, _k, used by the algorithm (see <cit.>).
 
 
Let {_k}_k∈ℕ be a sequence of points in X. The sequence {_k} of poll directions satisfies:

    _k = {_k^i |_k^i = 1, i=1,…,|_k|}

and for some ε̅> 0, 
    (_k∩ T_X(_k,ε)) =
    T_X(_k,ε),  ∀ ε∈(0,ε̅].

Furthermore, D=⋃_k=0^+∞_k is a finite set, and |_k| is uniformly bounded.


One problem still to be addressed is related to the fact that the merit function is defined and differentiable only in the interior of the feasible region, i.e. Ω_𝒢^ log.

An analysis based on differentiability assumptions seems not to be applicable. It turns out that the structure of the merit function can be exploited to extract information that allows the infeasible points to be treated as feasible failures, as it will be stated in Proposition <ref>.

Finally, some comments on the penalty parameter updating rule. This approach was originally proposed in <cit.>, in a clear difference with respect to interior point methods when applied to nonlinear optimization problems (even when in presence of first or second-order information). In <cit.>, the convergence of sequential penalization methods with a mixed interior-exterior strategy for nonlinear optimization problems has been established considering the sequence of exact minimizers of the subproblems. In general, finding the exact solution of any subproblem is practically unviable. In <cit.>, inspired by  <cit.>, the authors proposed a method using Newton-type directions and the same mixed penalty approach, but allowing a monotonically decreasing sequence of errors on the unconstrained minimizers. More precisely, the algorithm decreases the penalty parameter when the Euclidean norm of the gradient of the merit function is below the penalty parameter itself. In the present work, the updating rule plays a crucial role in the convergence properties of the algorithm, allowing to establish the boundedness of the sequence of associated Lagrange multipliers.




§ CONVERGENCE ANALYSIS
 

In order to prove the global convergence of the method, we will establish that the sequences of stepsizes and penalty parameters converge to zero. Initially, we derive an auxiliary result analyzing the behavior of the algorithm for fixed values of the penalty parameter.



Let {ρ_k}_k∈ℕ and {α_k}_k∈ℕ be the sequences of penalty parameters and stepsizes, respectively, generated by algorithm LOG-DS. Assume that 



    lim_k→+∞ρ_k = ρ̅> 0

Then, 


    lim_k→+∞α_k = 0.



From the updating rule of the penalty parameter, i.e. ρ_k+1 = θ_ρρ_k, we have that {ρ_k} is a monotone non-increasing sequence. Furthermore, if ρ_k is updated infinitely many times we would have ρ̅= 0. Hence, we have that ρ_k+1 = ρ_k = ρ̅ for all k sufficiently large. Let us split the iteration sequence into the following two sets

    K_s     = {k: _k+1≠_k},
    
    K_u     = {k: _k+1 = _k}.

At every iteration k of the algorithm, for k sufficiently large, we have either  Z(_k+1;ρ̅)=Z(_k;ρ̅) (when k∈ K_u) or Z(_k+1;ρ̅)≤ Z(_k;ρ̅) -ξ(α_k) (when k∈ K_s). Hence, the sequence of function values {Z(_k;ρ̅)} is monotonically nonincreasing. By Lemma <ref>, Z(;ρ̅) has compact level sets, thus it is bounded from below. Hence, 

    lim_k→ +∞Z(_k;ρ̅) = Z̅.


If K_s is infinite, 
from (<ref>) we get

    lim_k→ +∞ k∈ K_sξ(α_k) = 0.

Recalling Definition <ref>, we get 

    lim_k→ +∞ k∈ K_sα_k = 0.

If K_u is infinite, for every k∈ K_u, let us define m_k to be the largest index such that m_k∈ K_s and m_k < k (the result is immediate if K_s is empty
). Then, we can write

    α_k = α_m_kϕθ_α^k-m_k-1.

When k→ +∞, k∈ K_u, we have that either m_k→ +∞ as well (when K_s is infinite) or k-m_k-1→ +∞ (when K_s is finite). Thus, by (<ref>) and the fact that θ_α∈(0,1), we have

    lim_k→ +∞ k∈ K_uα_k = lim_k→ +∞ k∈ K_uα_m_kϕθ_α^k-m_k-1 = 0.

The proof is concluded considering (<ref>) and (<ref>).




























Lemma <ref> is used to show that the sequence of penalty parameters converges to zero, which is required to ensure that in the limit the algorithm will solve the original problem. As a consequence, we are able to establish that the sequence of stepsizes will also converge to zero in the general case. Since it is well-known that the stepsize is related to some measures of stationarity of the problem <cit.>, that property is also relevant.


 Let ρ_k_k∈ℕ and α_k_k∈ℕ be the sequences of penalty parameters and stepsizes generated by LOG-DS. Then,

    lim_k→ +∞ρ_k=0,


    lim_k→ +∞α_k=0.



We first prove (<ref>). 
The algorithmic structure implies that ρ_k_k∈ℕ is a monotone nonincreasing sequence of positive numbers. Hence, we have that

    lim_k→+∞ρ_k = ρ̅≥ 0.

By contradiction, let us assume that ρ̅> 0.
If this is the case, there must exist an integer  k̅ such that

    ρ_k+1 = ρ_k = ρ̅>0,   ∀ k≥k̅.

Again, the instructions of the algorithm imply that, for all k≥k̅,

    α_k+1 > min{ρ̅^β,(g_min)_k^2}
    or α_k+1≥α_k

and 

    Z(_k+1;ρ̅) ≤ Z(_k;ρ̅) ≤ Z(_k̅;ρ̅) = Z̅ < +∞.


Since ρ_k = ρ̅, for all k≥k̅, by 
Lemma <ref> we have lim_k→ +∞α_k = 0. 
Then, there must exist an infinite index set K such that α_k+1<α_k, for all k∈ K. Hence, for every k∈ K, k≥k̅, we also have that

    α_k+1 > min{ρ̅^β,(g_min)^2_k}

(otherwise, the algorithm would have updated ρ). Taking the limit in relation (<ref>), we obtain

    lim_k→ +∞ (g_min)_k=0.

Thus, we have

    lim_k→ +∞ Z(_k;ρ̅)= +∞.

However, this limit is in contradiction with (<ref>), thus proving (<ref>).



















Now, we prove (<ref>). We proceed by contradiction and assume that a subset of iterations K̅ exists such that, for all k∈K̅,

    α_k ≥α̅> 0.

Since ρ_k→ 0, we also have lim_k→+∞, k∈K̅ρ_k = 0. Then, since {ρ_k} is a monotone nonincreasing sequence, a further subset K̃⊆K̅ exists such that

    lim_k→+∞, k∈K̃ρ_k = 0,
       ρ_k+1<ρ_k,  for all k∈K̃.

If k∈K̃ is a successful iteration, then α_k+1 = ϕα_k ≥ϕα̅; otherwise, α_k+1 = θ_αα_k ≥θ_αα̅. Hence, for k∈K̃, we can write

    α_k+1≥θ_αα̅.

Thus, for all k∈K̃, we have

    θ_αα̅≤α_k+1≤min{ρ_k^β,(g_min)_k^2}≤ρ_k^β

which contradicts ρ_k→ 0, thus concluding the proof.


















Let us define the index set 

    K = {k∈: ρ_k+1 < ρ_k},

i.e., the set of iteration indices where the penalty parameter is updated. Note that, by the instructions of the algorithm, α_k+1<α_k for all k∈ K, i.e., every iteration k∈ K is an unsuccessful iteration. Recall that, by Theorem <ref>, K is an infinite index set.



















We use the following extended Mangasarian-Fromovitz constraint qualification (MFCQ).

 The point ∈ X is said to satisfy the MFCQ for Problem (<ref>) if the two following conditions are satisfied: 


  (a) There does not exist a nonzero vector α=(α_1,…,α_q) such that:

    (∑_i=1^qα_i∇ h_i())^⊤≥ 0,    ∀∈ T_X();


  (b) There exists a feasible direction ∈ T_X(), such that:

    ∇ g_ℓ()^⊤ < 0,   ∀ℓ∈ I_+(),  ∇ h_j()^⊤=0,   ∀ j=1,…,p

where I_+()={ℓ| g_ℓ()≥ 0}.



Consider the Lagrangian function L(,λ,μ), associated with the nonlinear constraints of Problem (<ref>), defined by: 
    L(,λ,μ)= f()+λ^⊤ g()+μ^⊤ h().
 The following proposition is a well-known result (see <cit.>), which states necessary optimality conditions for Problem (<ref>).


 
Let ^*∈ F be a local minimum of Problem (<ref>) that satisfies the MFCQ. Then, there exist vectors λ^*∈^m, μ^*∈^p such that

    ∇_x L(^*,λ^*,μ^*)^⊤ (-^*)≥ 0,    ∀∈ X


    (λ^*)^⊤ g(^*)=0,  λ^*≥ 0.



Therefore, we consider the following definition of stationarity. 

A point ^*∈ is said to be a stationary point for Problem (<ref>) if vectors λ^*∈^m and μ^*∈^p exist such that (<ref>) and (<ref>) are satisfied.


Before proving the main convergence result, since the merit function is not defined in  ^n∖Ω_𝒢^ log and the trial points corresponding to failures might not be feasible, we present the following proposition that allows us to use the mean value theorem in the proof of Theorem <ref>. 


 Given any ρ̅>0, ∈  X∩Ω_𝒢^ log, ∈^n, and α̅∈_+ such that +α̅∈ X and +α̅∉Ω_𝒢^ log, there exists α̂<α̅ such that:

    +α∈ X∩Ω_𝒢^ log  for all  α∈(0,α̂],
        Z(+α̂;ρ̅) > Z(;ρ̅)-ξ(α̂).



By definition X is convex, then +α d ∈ X for all α∈ [0,α̅].

Since +α̅∉Ω_𝒢^ log there exists an index ℓ∈𝒢^ log such that 

    min_i∈𝒢^ log, g_i(+α̅)≥ 0{g_i(+α̅)} = g_ℓ(+α̅)≥ 0.
 
The continuity of g and the fact that x∈Ω_𝒢^ log allow us to conclude that there exists at least one scalar α̃∈ (0,α̅] such that g_ℓ(+α̃)=0. If such a scalar is not unique, let α̃ be the smallest one. This implies that x+α∈Ω_𝒢^ log for all α∈ [0,α̃). 

Recall that, by definition of Z(;ρ), for all ∈∂Ω_𝒢^ log, it results

    lim_→,∈Ω_𝒢^ log Z(;ρ̅) = +∞.

Thus, there exists α̂∈ (0,α̃), sufficiently close to α̃, such that  x+α̂∈Ω_𝒢^ log and Z(;ρ̅)< Z(+α̂;ρ̅)+ξ(α̂), which conludes the proof.
We are now in conditions of stating the main convergence result.

 Let _k_k∈ℕ be the
sequence of iterates generated by LOG-DS and recall definition (<ref>) of K.

Assume that the
sets of directions {_k}_k∈ℕ, used by the
algorithm, satisfy Assumption <ref> and define J_k = {i∈{1,2,…,|_k|}: _k^i∈_k∩ T_X(_k;ε)}, with ε∈(0,min{ε̅,ε^*}] where ε^* and ε̅ are the constants appearing in Proposition <ref> and Assumption <ref>, respectively. Then, any limit point of {_k}_k∈ K
that satisfies the MFCQ is a stationary point of Problem (<ref>).



First note that, by Theorem <ref>, 
we have

    lim_k→+∞ k∈ Kρ_k = 0,
       lim_k→+∞ k∈ Kα_k = 0.


Now, let ^* be any limit point of {_k}_k∈ K. Then, there exists a set K̂⊆ K such that

    lim_k→+∞ k∈K̂ρ_k = 0,
       lim_k→+∞ k∈K̂α_k = 0,
       lim_k→+∞ k∈K̂_k = ^*,

with α_k+1<α_k, for all k∈K̂.
Recall that _k = {_k^1,_k^2,…,_k^r_k}. 
Then, for all k∈K̂ sufficiently large, we know that _k+α_k_k^i∈ X for all i∈ J_k. For every i∈ J_k, if _k+α_k_k^i∈Ω_𝒢^ log, by the instructions of the algorithm we have

    Z(_k+α_k _k^i;ρ_k)>
    Z(_k;ρ_k)-ξ(α_k).

Otherwise, i.e. when _k+α_k_k^i∉Ω_𝒢^ log, 
Proposition <ref> 

allows us to ensure the existence of a scalar
α̂_k^i≤α_k such that

    Z(_k+α̂_k^i _k^i;ρ_k)>
    Z(_k;ρ_k)-ξ(α̂_k^i).

Applying the mean value theorem to (<ref>), we can write

    -ξ(α̂_k^i)≤ Z(_k+α̂_k^i _k^i;ρ_k)
    - Z(_k;ρ_k) = α̂_k^i ∇ Z(_k^i;ρ_k)^⊤_k^i,

for all k∈K̂ sufficiently large and all i∈ J_k, where
_k^i=_k+t_k^iα̂_k^i _k^i, with t_k^i∈
(0,1). Thus, we have

    ∇ Z(_k^i;ρ_k)^⊤_k^i ≥
    -ξ(α̂_k^i) /α̂_k^i,   ∀ i∈ J_k.

By considering the expression of Z(;ρ_k), we can write

    ∇ Z(_k^i;ρ_k)^⊤_k^i    = [∇ f(_k^i) +
    ∑_ℓ∈𝒢^ logρ_k-g_ℓ(_k^i)∇
    g_ℓ(_k^i) +
    ν(∑_ℓ∈𝒢^ ext(max{g_ℓ(_k^i),0}/ρ_k)^ν-1∇ g_ℓ(_k^i) +.. 
       . . ∑_j=1^p(|h_j (_k^i)|/ρ_k)^ν-1∇ h_j (_k^i))]^⊤_k^i ≥
    -ξ(α̂_k^i)/α̂_k^i,   ∀
    i∈ J_k  k∈K̂  sufficiently large.

By Assumption <ref>, we can extract a further subset of indices K̃⊆K̂ such that, α_k+1<α_k, for all k∈K̃ and

    lim_k→+∞ k∈K̃ρ_k = 0
       lim_k→+∞ k∈K̃α_k = 0,
       lim_k→+∞ k∈K̃_k = ^*,
        J_k = J,  ∀ k∈K̃,
       _k^i = ^i,  ∀ i∈ J, k∈K̃,

and D^*={^i}_i∈ J. When k∈K̃ is sufficiently large, for all i∈ J, with _k^i=_k+t_k^iα̂_k^i^i, and t_k^i∈ (0,1), since α̂_k^i≤α_k, by Theorem <ref>, we have that,  lim_k→+∞ k∈K̃_k^i = ^*.

Let us define the following approximations to the Lagrange multipliers of each constraint:


  - for ℓ=1,…,m set
λ_ℓ(;ρ) = ρ-g_ℓ(),     if ℓ∈𝒢^ log
ν( max{g_ℓ(),0}ρ)^ν-1,     if ℓ∈𝒢^ ext

  - for j=1,…,p set μ_j(;ρ) = ν( |h_j()|ρ)^ν-1.










The sequences
λ_ℓ(_k;ρ_k)_k∈ K and μ_j(_k;ρ_k)_k∈ K, are bounded (see the Appendix <ref>). Thus, it is possible to consider
K^'⊆K̃, such that


    lim_k→+∞ k∈ K^'λ_ℓ(_k;ρ_k) = λ_ℓ^*,     ℓ=1,…,m
          lim_k→+∞ k∈ K^'μ_j(_k;ρ_k) = μ_j^*,      j=1,…,p

and define λ_ℓ^*=0 for ℓ∉I_+(^*).

Multiplying (<ref>) by ρ_k^ν-1 and taking the limit for k→+∞, k∈ K^', recalling ν∈(1,2], we have that ρ_k^ν-1→ 0, so we obtain the following 

    (∑_ℓ∈𝒢^ extνmax{g_ℓ(^*),0}^ν-1∇ g_ℓ(^*)+∑_j=1^pν |h_j (^*)|^ν-1∇ h_j
    (^*))^⊤^i≥ 0,   ∀^i∈ D^*.


From Proposition <ref> and Assumption <ref>, we know that there is ε>0 such that for all k∈ K^' sufficiently large

    T_X(^*) = T_X(_k;ε) = (_k∩ T_X(_k;ε))= (D^*).

Then, for every ∈ T_X(^*), there exist nonnegative numbers β_i such that 

    = ∑_i∈ Jβ_i ^i, with ^i∈ D^*.

Let us recall that, by assumption, ^* satisfies MFCQ conditions, and let  be the direction satisfying (<ref>) in point (b). Then we have, 

    0≤      ∑_i∈ Jβ_i (∑_ℓ∈𝒢^ extνmax{g_ℓ(^*),0}^ν-1∇ g_ℓ(^*) +∑_j=1^pν |h_j (^*)|^ν-1∇ h_j (^*))^⊤^i =
    
          (∑_ℓ∈𝒢^ extνmax{g_ℓ(^*),0}^ν-1∇ g_ℓ(^*)+∑_j=1^pν |h_j (^*)|^ν-1∇ h_j (^*))^⊤ =
    
    
          (∑_ℓ∈ I_+(^*)∩𝒢^ extνmax{g_ℓ(^*),0}^ν-1∇ g_ℓ(^*)+∑_j=1^pν |h_j (^*)|^ν-1∇ h_j (^*))^⊤.


Again by (<ref>), ∇ g_ℓ(^*)^T  < 0, for all ℓ∈ I_+(^*), and ∇ h_j(^*)^⊤ = 0, for all j. Then, we get max{g_ℓ(^*),0}=0 for all ℓ∈ℓ∈ I_+(^*)∩𝒢^ ext, so that g_ℓ(^*) ≤ 0 for all ℓ∈𝒢^ ext.
Furthermore, that implies

    (∑_ℓ∈𝒢^ extνmax{g_ℓ(^*),0}^ν-1∇ g_ℓ(^*)+∑_j=1^pν |h_j (^*)|^ν-1∇ h_j (^*))^⊤=
          (∑_j=1^pν |h_j (^*)|^ν-1∇ h_j (^*))^⊤≥ 0, for all ∈ T_X(^*).


Using (<ref>), we get h_j(^*) = 0 for all j=1,…,p. Therefore, the point ^* is feasible.

By simple manipulations, inequality (<ref>) can be
rewritten as

    (∇ f(_k^i)+∑_ℓ=1^m ∇ g_ℓ(_k^i) λ_ℓ(_k;ρ_k). 
        +∑_ℓ=1^m ∇ g_ℓ(_k^i) (λ_ℓ(_k^i;ρ_k)-λ_ℓ(_k;ρ_k))+∑_j=1^p ∇ h_j (_k^i) μ_j(_k;ρ_k) 
       .+∑_j=1^p ∇
    h_j(_k^i)(μ_j(_k^i;ρ_k)-μ_j(_k;ρ_k)))^⊤^i ≥-ξ(α̂_k^i)/α̂_k^i,   ∀ i ∈ J

Taking limits for k→ +∞, k∈ K^' and
considering (<ref>) and (<ref>), we
get:

    (∇ f(^*) + ∑_ℓ=1^m∇ g_ℓ(^*)λ_ℓ^* +
    ∑_j=1^p∇ h_j (^*)μ_j^*)^⊤^i≥ 0,
      ∀ i∈ J.

Again, by (<ref>) and (<ref>), we have, for all ∈ T_X(^*),

    (∇ f(^*) + ∑_ℓ=1^m∇ g_ℓ(^*)λ_ℓ^* + ∑_j=1^p∇ h_j (^*) μ_j^*)^⊤ = 
              ∑_i∈ Jβ_i(∇ f(^*) +
    ∑_ℓ=1^m∇ g_ℓ(^*)λ_ℓ^* + ∑_j=1^p∇
    h_j (^*)μ_j^*)^⊤^i≥ 0.

Since ^* is feasible and considering the definition of λ_ℓ^* for ℓ∈ I_+(^*), λ_ℓ^*  g_ℓ (^*)=0, for all ℓ=1,…,m, and the proof is concluded.




§ IMPLEMENTATION DETAILS

In this section, we describe a practical implementation for LOG-DS, based on the original implementation of SID-PSM <cit.>.




 §.§ LOG-DS vs SID-PSM

LOG-DS enhances SID-PSM with the capability of handling general constraints through a mixed penalty log-barrier approach. Thus, the original structure and algorithmic options of SID-PSM implementation are kept. In this subsection, we will provide a general overview of the main features of SID-PMS, highlighting the differences with LOG-DS. For more details on SID-PSM, the original references <cit.> could be used.

The main difference between LOG-DS and SID-PSM is the use of a merit function to address constraints, instead of an extreme barrier approach. In SID-PSM, only feasible points are evaluated, being the function value set equal to +∞ for infeasible ones. LOG-DS allows infeasibility regarding the nonlinear constraints. The merit function also replaces the original objective function through the different algorithmic steps. So, at the Search step, quadratic polynomial models are built for the objective and constraints functions and, after being aggregated into the merit function, are minimized inside a ball with radius directly related to the stepsize parameter.

The sets of points used in model computation do not require feasibility regarding the nonlinear constraints, always resulting from previous evaluations of the merit function. No function evaluations are spent solely for the purpose of model building. Depending on the number of points available, minimum Frobenius norm models, quadratic interpolation, or regression approaches can be used <cit.> to compute the model coefficients.

After the model minimization, LOG-DS needs to make a decision on accepting or rejecting the new trial point. Differently from SID-PSM, where only simple decrease is required for accepting new points, in LOG-DS points are accepted if they satisfy the sufficient decrease condition 
    Z(_k+1;ρ_k) ≤ Z(_k;ρ_k)-γα_k^2,
 where γ=10^-9. The use of a sufficient decrease condition for the acceptance of new points changes the type of globalization strategy used by the algorithm, which is no longer classified as a Generalized Pattern Search method, being now a Generating Set Search (GSS) method.

The algorithm proceeds with an opportunistic Poll Step, accepting the first poll point that satisfies the sufficient decrease condition. Before initiating the polling procedure, previously evaluated points are again used to build a simplex gradient <cit.>, which will be used as an ascent indicator. Poll directions are reordered according to the largest angle made with this ascent indicator, before initiating polling (see <cit.>). In LOG-DS, the simplex gradient is built for the merit function, while in SID-PSM the original objective function is considered.



 §.§ Penalty parameter details

In the initialization of LOG-DS, we define the two sets of indices 𝒢^ log and 𝒢^ ext, considering the values of the inequality constraints at the initial point _0∈ X: 
    𝒢^ log=i| g_ℓ(_0)<0 and 𝒢^ ext=i| g_ℓ(_0)≥ 0.
 


Moreover, we define two penalty parameters, one corresponding to the logarithmic barrier term (ρ^ log) and another one associated with the penalty exterior component (ρ^ ext), and we initialize them as ρ^ log_0=10^-1 and ρ^ ext_0=1/max{|f(x_0)|,10}. Note that we treat ρ^ext as ρ^ν-1 in equation (<ref>). Therefore, we can write the merit function as 

    Z(;ρ_k) = f() - ρ^ log_k∑_ℓ∈𝒢^ loglog(-g_ℓ()) + 1ρ^ ext_k(∑_ℓ∈𝒢^ ext(max{0,g_ℓ()})^ν + ∑_j=1^p|h_j()|^ν).

The penalty parameters are updated only at unsuccessful iterations, considering two different criteria:

    α_k+1≤min(ρ^ log_k)^β,(g_min)^2_k, for updating ρ^ log_k;
          α_k+1≤min{(ρ^ log_k)^β,(ρ^ ext_k)^β, (g_min)^2_k}, when updating ρ^ ext_k.


Recall that (g_min)_k is the minimum absolute value for the constraints in 𝒢^ log at iterate x_k. In the implementation, we have considered β=1+10^-9 and ν=2. Thus, ρ^ν-1=ρ in equation (<ref>). 

If inequality (<ref>) holds, LOG-DS uses the following rule

    ρ^ log_k+1=ζρ^ log_k,


whereas if inequality (<ref>) holds,
LOG-DS performs the following update

    ρ^ ext_k+1=ζρ^ ext_k,

in both cases with ζ=10^-2.

The use of two different penalty parameters for the two different terms of penalization is a practical need to be able to properly scale the constraints and the different ways they are handled. In particular, in our numerical experience, the logarithmic term seemed not to suffer with the different scales of the objective function, while the exterior penalty seemed to be very sensitive to it. In practice, if the exterior penalty parameter is set too high, the algorithm might be slow at reaching feasible solutions. If it is set too low, the algorithm might not be good at reaching solutions with the best objective function value,  even though it might be very capable of attaining feasibility. While scaling the initial exterior penalty parameter with respect to the initial value of the objective function improved the numerical results, it is possible that it might not work for specific problems. Indeed, we are implicitly assuming that the gradient of the objective function is closely related to the objective function value, which might be true for many real problems, but it is certainly not true in general. Scaling the objective function and the constraints for general nonlinear optimization problems is currently an active field of research.

Furthermore, to keep our method aligned with the theoretical framework proposed, in Appendix <ref> we provide some results related to our choices.




§ NUMERICAL EXPERIMENTS

 
This section is dedicated to the numerical experiments and performance evaluation of the proposed mixed penalty-logarithmic barrier derivative-free optimization algorithm, LOG-DS, on a collection of test problems available in the literature.

We considered the test set used in <cit.>, part of the CUTEst collection <cit.>. Specifically, we included problems where the number of variables does not exceed 50 and at least one inequality constraint is strictly satisfied by the initialization provided, ensuring _0∈Ω_𝒢^ 𝓁ℴℊ.

Table <ref> details the test set, providing the name, the number of variables, n_p, the number of inequality constraints, m_p=|𝒢^ log∪𝒢^ ext|, the number of inequality constraints treated by the logarithmic barrier m̅_p=|𝒢^ log|, and the number of nonlinear equalities, meq_p, for each problem. Additional information can be found in <cit.>.






 §.§ Performance and data profiles


Numerical experiments will be analyzed using performance <cit.> and data <cit.> profiles. To provide a brief overview of these tools, consider a set of solvers 𝒮 and a set of problems 𝒫. Let t_p,s represent the number of function evaluations required by solver s to satisfy the convergence test adopted for problem p.

For an accuracy τ=10^-k, where k∈{1,3,5}, we adopted the convergence test:

    f_M - f() ≥ (1-τ)(f_M-f_L),

where f_M represents the objective function value of the worst feasible point determined by all solvers for problem p, and f_L is the best objective function value obtained by all solvers, corresponding to a feasible point of problem p.

The convergence test given by (<ref>) requires a significant reduction in the objective function value by comparison with the worst feasible point f_M. We assign an infinite value to the objective function at points that violate the feasibility conditions, defined by c(x)>10^-4, where

    c(x) = ∑_i=1^m max{0, g_i(x)} + ∑_j=1^p |h_j(x)|.


The performance of solver s∈𝒮 is measured by the fraction of problems in which the performance ratio is at most α, given by:

    ρ_s(α) =1|P||{p ∈ P|t_p, smin{t_p, s^': s^'∈
    S}≤α}|.

A performance profile provides an overview of how well a solver performs across a set of optimization problems. Particularly relevant is the value ρ_s(1), that reflects the efficiency of the solver, i.e., the percentage of problems for which the algorithm performs the best. Robustness, as the percentage of problems that the algorithm is able to solve, can be perceived for high values of α.

Data profiles focus on the behavior of the algorithm during the optimization process. A data profile measures the percentage of problems that can be solved (given the tolerance τ) with κ estimates of simplex gradients and is defined by: 

    d_s(κ) =1|P||{p ∈ P| t_p, s≤κ(n_p+1)}|,

where n_p represents the dimension of problem p.



 §.§ Results and comparison

This subsection aims to demonstrate the good numerical performance of the LOG-DS algorithm. 



  §.§.§ Comparison between strategies for linear constraints

In this subsection, our focus lies on evaluating the performance of LOG-DS using two distinct approaches for managing linear inequality constraints, other than bounds. The first approach addresses each linear inequality constraint as a general nonlinear inequalitiy, i.e. through the penalty approach discussed previously. The second approach entails addressing the linear inequality constraints via an extreme barrier method, adjusting the directions in accordance with the geometry of the feasible region.

The works by Lucidi et al. <cit.> and Lewis and Torczon <cit.> propose methods for computing directions conforming to linear inequality constraints but do not consider degeneracy. Abramson et al. <cit.> provide a detailed algorithm for generating the set of desired directions, regardless of whether the constraints are degenerate or not. 

In order to use Definition <ref>, of ε-active constraints, we assume a preliminary scaling of the constraints. For this purpose, we multiply each ith constraint by a_i^-1, since the vectors a_i, i∈ I_X, are not null (we have that a_i≠ 0 for all i∈ I_X). Therefore, we consider

    a_i=a_ia_i,  b_i=b_ia_i, i∈ I_X.

The ε-active index set is computed using the matrix A, a scaled version of the matrix A, and the vector b. Consequently, we have that a_i=1 for all i∈ I_X and X={∈ℝ^n| A≤b}≃{∈ℝ^n|a_i≤b_i,  i∈ I_X}={∈ℝ^n|A≤b}.

To compute the set of directions _k that conform to the geometry of the nearby constraints, we use the algorithm proposed in <cit.>. The latter is divided into two parts: the first constructs the index set corresponding to ε-active non-redundant constraints, and the second the set of directions _k, which include the generators of the cone T_X(_k,ε).

It is important to understand why we are comparing the two strategies. First, linear constraints might not be explicitly given for a black-box type problem, making it impossible to conform the directions to the linear constraints. In such cases, we would be forced to treat the linear inequalities with a mixed penalty. Furthermore, the logarithmic barrier approach is well-known for handling linear constraints very efficiently, especially in the presence of a large number of constraints. Finally, conforming the directions to the nearby linear constraints might affect the geometry of the generated points, impacting the quality of the surrogate models built to improve the performance of the algorithm. Using the penalty approach allows us to keep using the coordinate directions, which are known to have good geometry for building linear models. Any other orthonormal basis could be used, though the coordinate directions additionally conform to bound constraints on the variables, so that we can treat them separately from the penalty approach.

For identifying a constraint as being ε-active, we considered ε=10^-5 and we used _k = [1  -1   𝕀_n  -𝕀_n] as the default set of directions, for every k. Figure <ref> depicts the performance of LOG-DS using the two strategies described before, considering a maximum number of 2000 function evaluations and a minimum stepsize tolerance equal to 10^-8.






As we can see, the performance of the LOG-DS algorithm when addressing the linear inequality constraints within the penalty approach outperforms the competing strategy, addressing the linear constraints directly. 
Therefore, in the rest of the work, the experiments will be carried out using the winning strategy. Note that the significant difference in the performance might be due to the specific choice of the tested problems and/or the specific strategy used to conform the directions to the linear constraints, rather than to a flaw in the approach by itself.



  §.§.§ Comparison with the original Extreme Barrier

We are proposing an alternative strategy to address constraints within the SID-PSM algorithm. Thus, we start by illustrating that the use of a mixed penalty-logarithmic barrier is competitive against the extreme barrier approach. The latter can only be adopted for problems without equality constraints and for which a strictly feasible point is given as initialization, so we selected a subset of problems satisfying these conditions. The subset consists of a total of 28 problems, highlighted in Table <ref>. 

Figure <ref> presents the comparison between LOG-DS, which exploits the mixed penalty-logarithmic barrier, and the original SID-PSM, which employs an extreme barrier approach. The default values of SID-PSM were considered for both algorithms, allowing a maximum number of 2000 function evaluations and a minimum stepsize tolerance equal to 10^-8.



As Figure <ref> shows, LOG-DS presents a better performance than SID-PSM, especially when a higher precision is considered. Furthermore, the possibility of initializing LOG-DS with infeasible points allows to handle a wider class of practical problems.



  §.§.§ Comparison with state-of-the-art solvers

This subsection focuses on comparing LOG-DS against state-of-the-art derivative-free optimization solvers that are able to address general nonlinear constraints. Comparisons were made with MADS <cit.>, implemented in the well-known NOMAD package (version 4), which can be freely obtained at <https://www.gerad.ca/en/software/nomad> <cit.>. Additionally, the

X-LOG-DFL algorithm <cit.>, available through the DFL library as the LOGDFL package at <https://github.com/DerivativeFreeLibrary/LOGDFL>, was also tested. Comparison with LOG-DFL <cit.> is particularly relevant since it uses the same merit function of LOG-DS. Default settings were considered for all codes and results, reported in Figure <ref>, were obtained for a budget of 2000 function evaluations.




It can be observed that LOG-DS presents the best performance, for any of the three precision levels considered, both in terms of efficiency and robustness, across the different computational budgets. 




Figure <ref> compares the different solvers considering the subset of problems with only inequality constraints (61 out of 96 problems), again allowing a maximum of 2000 function evaluations. Once more, LOG-DS is clearly the solver with the best performance.

In summary, considering the outcomes of the different numerical experiments, we can conclude that LOG-DS is the most efficient and robust solver across different scenarios, making it the top-performing choice.























































§ CONCLUSIONS

 
The primary objective of this work was to extend the approach introduced in <cit.> to generalized pattern search, allowing to efficiently address nonlinear constraints.

To accomplish it, we adapted the SID-PSM algorithm, a generalized pattern search method, where polynomial models are used both at the search and at the poll steps to improve the numerical performance. We proposed a new algorithm, LOG-DS, that keeps the basic algorithmic features of SID-PSM but uses a mixed penalty-logarithmic barrier merit function to address general nonlinear and linear constraints.

Under standard assumptions, not requiring convexity of the functions defining the problem, convergence was established towards stationary points. Furthermore, an extensive numerical experimentation allowed to compare the performance of LOG-DS with several state-of-art solvers on a large set of test problems from the CUTEst collection. The numerical results indicate the robustness, efficiency, and overall effectiveness of the proposed algorithm.




§ APPENDIX


 Let a∈_+, b∈_+, a+b>0 and p ∈ [0,1], it results

    (a+b)^p ≤ a^p + b^p.



We have aa+b≤ 1 and ba+b≤ 1, so 

    (a+b)^p     = (a+b)/(a+b)^1-p = a/(a+b)^1-p + b/(a+b)^1-p
        = a^p (a/a+b)^1-p + b^p (b/a+b)^1-p
       ≤ a^p + b^p.




 Let a∈, b∈ and p ∈ [0,1]. Then the following inequalities hold 

    ||a|^p-|b|^p|≤ |a-b|^p,

and 

    |max{a,0}^p-max{b,0}^p| ≤ |a - b|^p.



To prove (<ref>) consider

    |a|^p    =    |a-b+b|^p
    
    (Triangular inequality )    ≤    (|a-b|+|b|)^p
    
    (Lemma <ref>)    ≤    |a-b|^p+|b|^p

On the other hand,

    |b|^p    =    |b-a+a|^p
    
    (Triangular inequality )    ≤    (|b-a|+|a|)^p
    
    (Lemma <ref>)    ≤    |a-b|^p+|a|^p

Then, from inequalities (<ref>) and (<ref>) we derive inequality (<ref>).

Now, to prove (<ref>), we will analyze four different cases:


  i) If a≤ 0 and b≤ 0, the result holds trivially. 

  ii) If a≤ 0 and b>0, then |-b^p|=b^p≤ |b-a|^p=|a-b|^p.

  iii) If a>0 and b≤ 0, then |a|^p=a^p≤ |a-b|^p.

  iv) If a>0 and b>0, using (<ref>) we conclude that:

    |max{a,0}^p-max{b,0}^p| =|a^p-b^p|=||a|^p-|b|^p|≤ |a - b|^p.








§ APPENDIX














 Let {a^i_k}_k∈ℕ,
i=1,…,ℓ, be sequences of scalars. Two cases can occur:


  (i) It results lim_k→+∞ a_k^i = 0, i=1,…,ℓ. In particular, all the sequences are bounded;

  (ii) An index j∈{1,…,ℓ}, an infinite index set K_j⊆{0,1,…} and a positive scalar a̅_j  exist such that

    |a_k^j| > a̅_j > 0,  ∀ k∈ K_j,

i.e., at least one sequence is not convergent to zero.
Then, there exists an
index s∈{1,…,ℓ} and an infinite subset K⊆ℕ such that:

    lim_k→+∞, k∈ Ka_k^i|a_k^s|=z_i,    |z_i|<+∞,   
    i=1,…,ℓ,

i.e., all the sequences {a_k^i|a_k^s|}_k∈ K are bounded.




Point (i) directly follows from the properties of convergent sequences.

Then, let us now assume that at least a sequence is not convergent to zero. If this is the case, we can reorder the sequences in such a way that:


  * lim_k→+∞a_k^1 = lim_k→+∞a_k^2=…=lim_k→+∞a_k^r-1 = 0;

  * {a_k^i}, i=r,…,ℓ, are not convergent to zero.

We now prove that an index s∈{r,…,ℓ} and an infinite index set K̂ exist such that

    lim_k→+∞, k∈K̂a_k^i/|a_k^s| = z_i,   |z_i| < +∞,   i=r,…,ℓ.

This is (obviously) true when ℓ=r. Indeed, when ℓ=r, there is K_r such that

    lim_k→+∞, k∈ K_ra_k^r/|a_k^r| = ± 1.

Furthermore, we have

    lim_k→+∞, k∈ K_ra_k^i/|a_k^r| = 0,   i=1,…,r-1.

The thesis follows by choosing s=r and K̂=K_r.

Now, we prove the thesis by induction on ℓ. Then,  we have sequences {a_k^i}, i=1,…,ℓ-1 such that:


  * lim_k→+∞a_k^1 = lim_k→+∞a_k^2=…=lim_k→+∞a_k^r-1 = 0;

  * {a_k^i}, i=r,…,ℓ-1, are not convergent to zero,

and an index ∈{r,…,ℓ-1} and an infinite index set K̂ exist such that

    lim_k→+∞, k∈K̂a_k^i/|a_k^| = z_i,   |z_i| < +∞,   i=r,…,ℓ-1.




  * If lim_k→+∞a_k^ℓ = 0, then 
we have

    lim_k→+∞, k∈K̂a_k^j/|a_k^| = z_j,   j=1,…,ℓ-1
       lim_k→+∞, k∈K̂a_k^ℓ/|a_k^| = 0;

and the thesis follows by choosing s= and K=K̂.

  * Suppose now that {a_k^ℓ} is not convergent to zero. 
In this situation, two subcases can occur:


  * the sequence {a_k^ℓ/a_k^}_k∈K̂ is bounded;


  * an infinite index set K_1⊆K̂ exists such that {a_k^ℓ/a_k^}_k∈ K_1 is unbounded.

In the first case, an infinite index set K_2⊆K̂ exists such that {a_k^ℓ/a_k^}_k∈ K_2 is convergent. Then, the thesis follows by taking s= and K = K_2.

In the second case, we have

    lim_k→+∞, k∈ K_1a_k^/a_k^ℓ = 0.

Furthermore, for every i=1,…,ℓ-1, we have

    lim_k→+∞, k∈ K_1a_k^i/a_k^ℓ = lim_k→+∞, k∈ K_1a_k^i/a_k^a_k^/a_k^ℓ = 0,

and again the thesis is proved with s = ℓ and K=K_1.

Thus, the proof is concluded.



In the conditions of Theorem <ref>,

    λ_ℓ(;ρ) = ρ-g_ℓ(),     if ℓ∈𝒢^ log
    ν(max{g_ℓ(),0}ρ)^ν-1,     if ℓ∈𝒢^ ext
          μ_j(;ρ) = ν(|h_j()|ρ)^ν-1,   
    j=1,…,p,

the subsequences λ_ℓ(_k;ρ_k)_k∈K̂, ℓ=1,…,m and μ_j(_k;ρ_k)_k∈K̂, j=1,…,p are bounded.


Recalling the expressions of λ_ℓ(;ρ), ℓ=1,…,m, and of μ_j(;ρ), j=1,…,p, we can rewrite inequality (<ref>) as 

    (∇ f(_k^i) + ∑_ℓ=1^mλ_ℓ(_k^i;ρ_k) ∇ g_ℓ(_k^i) + 
           ∑_j=1^pμ_j(_k^i;ρ) ∇ h_j (_k^i) )^⊤_k ≥ -ξ(α̂_k^i)/α̂_k^i,   ∀ i∈ J_k  k∈K̂,

where _k^i=_k+t_k^iα̂__k_k^i, with
t_k^i∈ (0,1) and α̂_k^i≤α_k.

We will start by establishing that

    lim_k→+∞ k∈K̂ |λ_ℓ(_k;ρ_k) - λ_ℓ(_k^i;ρ_k) | = 0,   ℓ∈𝒢^ log,   ∀ i∈ J_k

In fact,

    |ρ_k/-g_ℓ(_k)-ρ_k/-g_ℓ(_k^i)|
       =   ρ_k|g_ℓ(_k)-g_ℓ(_k^i)/(-g_ℓ(_k^i))(-g_ℓ(_k))|
       =   ρ_k |∇ g_ℓ(_k^i)^⊤(_k-_k^i)|/|g_ℓ(_k^i)|
    |g_ℓ(_k)|
       ≤   ρ_k
    ∇
    g_ℓ(_k^i)_k^i-_k/|g_ℓ(_k^i)
     g_ℓ(_k)|,

where _k^i= _k+t̃_k^i(_k^i-_k) with t̃_k^i∈ (0,1).

Then, there is c_1>0 such that

    ρ_k ∇
    g_ℓ(_k^i)_k^i-_k/|g_ℓ(_k^i) 
    g_ℓ(_k)|   ≤   ρ_k c_1 _k^i-_k
    |g_ℓ(_k^i)||g_ℓ(_k)|
       =   ρ_k c_1
    _k+t_k^iα̂_k^i _k^i-_k
    |g_ℓ(_k^i)||g_ℓ(_k)|
       =   ρ_k c_1
    t_k^iα̂_k^i _k^i
    |g_ℓ(_k^i)||g_ℓ(_k)|
       =   ρ_k c_1
    t_k^i α̂_k^i _k^i
    |g_ℓ(_k^i)||g_ℓ(_k)|.

Now, we will prove that there is another constant c_2>0 such
that

    1|g_ℓ(_k^i)|≤
    c_21|g_ℓ(_k)|.

Suppose, in order to arrive to a contradiction, that c_2 does not exist. This would imply that there  exists K^'⊆K̂⊆ K such that 

    lim_k→+∞ k∈ K^'1/|g_ℓ(_k^i)|1/|g_ℓ(_k)| =
    lim_k→+∞ k∈ K^'|g_ℓ(_k)||g_ℓ(_k^i)|=+∞.

Let us consider the case where

    lim_k→+∞ k∈ K^' |g_ℓ(_k)|=0.

Since g_ℓ(_k)<0 and g_ℓ(_k^i)<0 for all k∈ K^', by (<ref>) there exists k∈ℕ such that, for all k≥k, k∈ K^', we have 
    -g_ℓ(_k)>-g_ℓ(_k^i)=-g_ℓ(_k+t_k^iα̂_k^i _k^i).
 Using the Lipschitz continuity of g_ℓ, ℓ=1,…,m and the fact that _k^i=1 for all i∈ J_k, we get 
    -g_ℓ(_k+t_k^iα̂_k^i _k^i)≥-g_ℓ(_k)-L_g_ℓt_k^iα̂_k^i _k^i=-g_ℓ(_k)-L_g_ℓt_k^iα̂_k^i.

The definition of K guarantees that 
    α_k+1≤minρ_k^β,(g_ min)_k^2,     α_k+1=θ_αα_k,
 so that 

    α_k≤minρ_k^β,(g_ min)_k^2θ_α

Hence, since α̂_k^i≤α_k, we have 
    -g_ℓ(_k)-L_g_ℓ t_k^i α̂_k^i≥ -g_ℓ(_k)-L_g_ℓ t_k^i 1/θ_α(g_ℓ(_k))^2,  ∀ k≥k, k∈ K^'
 Thus, 

    lim_k→+∞ k∈ K^'-g_ℓ(_k)-g_ℓ(_k^i)   =   lim_k→+∞ k∈ K^'-g_ℓ(_k)-g_ℓ(_k+t_k^iα̂_k^i _k^i)
       ≤   lim_k→+∞ k∈ K^'-g_ℓ(_k)-g_ℓ(_k)-L_g_ℓ t_k^i 1/θ_α(g_ℓ(_k))^2= 1,

which leads to a contradiction, proving (<ref>).

Now, by considering the other case

    lim_k→+∞ k∈ K^'
    |g_ℓ(_k)|=c<+∞,

we have

    lim_k→+∞ k∈ K^'-g_ℓ(_k)-g_ℓ(_k^i)=lim_k→+∞ k∈ K^'-g_ℓ(_k)-g_ℓ(_k+t_k^iα̂_k^i
    _k^i)<+∞.

Again, this leads to a contradiction, proving (<ref>).

Hence, the existence of the constant c_2>0, (<ref>), and recalling that α̂_k^i≤α_k, allow us to write 

    ρ_k c_1 t_k^i
    α̂_k^i|g_ℓ(_k^i)||g_ℓ(_k)|   ≤   ρ_k
    c_1 c_2 t_k^i α_k|g_ℓ(_k)|^2.

The instructions of Step 3 imply that _k+1 = _k, so that (g_min)_k = min_ℓ∈𝒢^ log{|g_ℓ(_k+1)|} = min_ℓ∈𝒢^ log{|g_ℓ(_k)|}. Recalling (<ref>), we get 
    ρ_k α_k(g_ min)_k^2≤ρ_kθ_α.
 Then, recalling Theorem <ref>, (<ref>) is proved.
Furthermore, we will establish   


    lim_k→+∞ k∈K̂ |λ_ℓ(_k;ρ_k) - λ_ℓ(_k^i;ρ_k) | = 0,   ℓ∈𝒢^ ext,   ∀ i∈ J_k



    |νρ_k^ν-1(max{g_ℓ(_k),0})^ν-1.   -   .νρ_k^ν-1(max{g_ℓ(_k^i),0})^ν-1|=
        =    νρ_k^ν-1|max{g_ℓ(_k), 0}^ν-1-max{g_ℓ(_k)+∇ g_ℓ(_k^i)^⊤(_k-_k^i), 0}^ν-1|
    
    (Lemma <ref>–(<ref>))   ≤   νρ_k^ν-1|g_ℓ(_k) -g_ℓ(_k)-∇ g_ℓ(_k^i)^⊤(_k-_k^i)|^ν-1
        =    νρ_k^ν-1 |∇ g_ℓ(_k^i)^⊤ (_k-_k^i)|^ν-1≤νρ_k^ν-1∇ g_ℓ(_k^i)^ν-1_k-_k^i^ν-1
       ≤   c_3νρ_k^ν-1(_k-(_k+t_k^iα̂_k^i _k^i) )^ν-1= c_3νρ_k^ν-1 (t_k^iα̂_k^i)^ν-1_k^i^ν-1
       ≤    c_3νρ_k^ν-1 (α̂_k^i)^ν-1_k^i^ν-1≤ c_3ν(α_kρ_k)^ν-1_k^i^ν-1
       ≤    c_3ν(ρ_k^β-1/θ_α)^ν-1_k^i^ν-1 = c_3νθ_α^1-νρ_k^(β-1)(ν-1)_k^i^ν-1,











where _k^i= _k+t̃_k^i(_k^i-_k) with t̃_k^i∈ (0,1), and c_3>0. Thus, using β>1, ν∈(1,2],  _k^i=1 for all i∈ J_k, and recalling Theorem <ref>,  (<ref>) is proved. Therefore, we have that 

    lim_k→+∞ k∈K̂ |λ_ℓ(_k;ρ_k) - λ_ℓ(_k^i;ρ_k) | = 0,   ℓ=1,…,m,   ∀ i∈ J_k

We will now establish that

    lim_k→+∞ k∈K̂
    |μ_j(_k;ρ_k) - μ_j(_k^i;ρ_k) | = 0,   
    j=1,…,p,   ∀ i∈ J_k

In fact, recalling that ν∈(1,2] so that ν-1∈(0,1], we have

    |ν|h_j(_k)/ρ_k|^ν-1-ν|h_j (_k^i)/ρ_k|^ν-1|    =   ν/ρ_k^ν-1||h_j(_k)|^ν-1-| h_j(_k)+∇ h_j(_k^j)^⊤ (_k^i-_k)|^ν-1| 
     
    (Lemma <ref>–(<ref>))
       ≤   ν/ρ_k^ν-1|h_j(_k)-h_j(_k)-∇ h_j(_k^j)^⊤ (_k^i-_k)|^ν-1
    
       =   ν/ρ_k^ν-1|∇ h_j(_k^j)^⊤ (_k^i-_k)
    |^ν-1
       ≤   ν/ρ_k^ν-1∇ h_j(_k^j)
    ^ν-1(_k^i-_k)^ν-1,

where _k^j=_k+t̃_k^i (_k^i-_k), with t̃_k^i∈ (0,1) 
Now, recalling that h_j, j=1,…,p are continuously
differentiable functions and _k^i=_k+t_k^i α̂_k^i _k^i, with t_k^i∈ (0,1)  and _k^i=1,
from (<ref>) and the fact that α̂_k^i≤α_k we can write

    |ν|h_j(_k)/ρ_k|^ν-1-
    ν|h_j (_k^i)/ρ_k|^ν-1|
       ≤   ν/ρ_k^ν-1 c_3 (t_k^i α̂_k^i )^ν-1
       ≤    c_3ν/ρ_k^ν-1(t_k^i α_k )^ν-1≤ c_3ν(α_k/ρ_k)^ν-1
       ≤    c_3ν(ρ_k^β-1/θ_α)^ν-1 = c_3νθ_α^1-νρ_k^(β-1)(ν-1).

Given that β > 1, ν∈ (1,2], and recalling Theorem <ref>, we can conclude that (<ref>) holds.


Now, we are able to prove the boundness of the sequences λ_ℓ(_k;ρ_k)_k∈K̂, ℓ=1,…,m and μ_j(_k;ρ_k)_k∈K̂, j=1,…,p.

In fact, we can rewrite (<ref>) as

    (∇ f(_k^i)+∑_ℓ=1^m ∇ g_ℓ(_k^i) λ_ℓ(_k;ρ_k)+. 
        +∑_ℓ=1^m ∇ g_ℓ(_k^i) (λ_ℓ(_k^i;ρ_k,)-λ_ℓ(_k;ρ_k))+∑_j=1^p ∇ h_j (_k^i) μ_j(_k;ρ_k)+ 
       .+∑_j=1^p ∇
    h_j(_k^i)(μ_j(_k^i;ρ_k)-μ_j(_k;ρ_k)))^⊤_k ≥-ξ(α̂_k^i)/α̂_k^i,   ∀ i ∈ J_k  and  k∈K̂.

Let

    a_k^1,…,a_k^m=λ_1(_k;ρ_k),…,λ_m(_k;ρ_k),
          a_k^m+1,…,a_k^m+p=μ_1(_k;ρ_k),…,μ_p(_k;ρ_k).

Assume, by contradiction, that there exists at least one index
l∈1,…,m+p such that

    lim_k→+∞ k∈ K
    |a_k^l|=+∞.

Hence, the sequence {a_k^i}, i=1,…,m+p, cannot be all convergent to zero. 
Then, from Lemma <ref>, there exists an infinite subset
K^'⊆K̂ and an index s∈1,…,m+p such
that,

    lim_k→+∞ k∈
    K^'a_k^i|a_k^s|=z_i,   |z_i|<+∞,  
    i=1,…,m+p

If there is an unique index l that satisfies (<ref>),
then s=l. If we have more than one index satisfying the
equation, then s is selected as one of the indexes such that
a_k^s_k∈ K^' tends to +∞ faster than the
others. Note also that

    z_s=1,    and     |a_k^s|→+∞.


Dividing the relation (<ref>) by
|a_k^s|, we have

    (∇ f(_k^i)|a_k^s|+∑_ℓ=1^m ∇ g_ℓ(_k^i) a_k^ℓ|a_k^s|. 
        +∑_ℓ=1^m ∇ g_ℓ(_k^i)λ_ℓ(_k^i;ρ_k)-λ_ℓ(_k;ρ_k)|a_k^s|+∑_j=1^p ∇ h_j (_k^i) a_k^m+j|a_k^s|
       .+∑_j=1^p ∇
    h_j(_k^i)μ_j(_k^i;ρ_k)-μ_j(_k;ρ_k)|a_k^s|)^⊤_k ≥-ξ(α̂_k^i)/α̂_k^i
    |a_k^s|,   ∀ i∈ J_k  and  k∈K̂.


Since lim_k→+∞ k∈K̂_k = ^*, Assumption <ref> and Proposition <ref> ensure the existence of ε>0 such that for  k∈K̃⊆K̂ sufficiently large, T_X(^*)=T_X(_k,ε) =(_k∩ T_X(_k,ε))=(D^*).



Taking the limit for k→+∞ and k∈K̃, and using (<ref>), (<ref>), and (<ref>), we obtain 

    (∑_ℓ=1^m
    z_ℓ∇ g_ℓ(^*)+∑_j=1^p z_m+j∇
    h_j(^*))^⊤^*≥ 0,  ∀^*∈ D^*.


We recall that ^* satisfies the MFCQ conditions. Let  be the
direction satisfying condition (b) of Definition <ref>. For every ∈ T_X(^*), there
exist nonnegative numbers β_i such that

    = ∑_^*i∈ D^*β_i ^*i.

Thus, from (<ref>)
and (<ref>), we obtain

    (∑_ℓ=1^m z_ℓ∇ g_ℓ(^*)+∑_j=1^p z_m+j∇
    h_j(^*))^⊤= 
          = ∑_d^*i∈ D^*β_i(∑_ℓ=1^m z_ℓ∇ g_ℓ(^*)+∑_j=1^p z_m+j∇ h_j(^*))^⊤^*i
          = ∑_d^*i∈ D^*β_i ∑_ℓ=1^m z_ℓ∇
    g_ℓ(^*)^⊤^*i+∑_d^*i∈ D^*β_i ∑_j=1^p
    z_m+j∇ h_j(^*)^⊤^*i≥ 0.


Considering Definition <ref>, the relation (<ref>) becomes

    ∑_ℓ=1^m z_ℓ∇ g_ℓ(^*)^⊤≥ 0.

Theorem <ref> and the definition of z_ℓ for ℓ∈1,…,m, guarantee 
    z_ℓ=0,   for all   ℓ∉ I_+(^*).


Since ^* satisfies the MFCQ conditions, (<ref>) implies

    z_ℓ=0,   for all ℓ∈ I_+(^*).


Therefore equation (<ref>) becomes 

    (∑_j=1^p z_m+j∇ h_j(^*))^⊤^*≥ 0,   for all   ^*∈ D^*,

using again Definition <ref> and (<ref>), we obtain

    z_m+j=0,   for all    j∈1,…,p.

In conclusion, we get (<ref>), (<ref>), and (<ref>), contradicting (<ref>) and this concludes the proof.




§ APPENDIX


In the following we provide three lemmas to support our implementation choices. In particular the first two results are the equivalent of Lemma <ref> and Theorem <ref>, which are the fundamental results the theoretical analysis is based on. Furthermore, since the analysis in Section 3 is carried out by exploiting the subsequence of iterates where the penalty parameter is updated, we state one additional result to prove that the subsequence where both penalty parameters are simultaneously updated is infinite.


Let {ρ^ log_k}_k∈ℕ, {ρ^ ext_k}_k∈ℕ, and {α_k}_k∈ℕ be the sequences of penalty parameters and stepsizes respectively, generated by algorithm  LOG-DS. Assume that 

    lim_k→+∞ρ^ log_k = ρ̅_1 > 0


    lim_k→+∞ρ^ ext_k = ρ̅_2 > 0

Then,

    lim_k→+∞α_k = 0.



Using similar arguments to Lemma <ref>.



 Let ρ^ log_k_k∈ℕ, ρ^ ext_k_k∈ℕ, and α_k_k∈ℕ be the sequences of penalty parameters and stepsizes generated by LOG-DS. Then,

    lim_k→ +∞ρ^ log_k=0,


    lim_k→ +∞ρ^ ext_k=0,


    lim_k→ +∞α_k=0.



By (<ref>) we have that ρ^ ext_k → 0 only if ρ^ log_k → 0, so we can use the same arguments of Theorem <ref> to prove (<ref>). Now, by (<ref>) and (<ref>) it is clear that (<ref>) implies (<ref>)

Finally, we can use similar arguments of Theorem <ref> to prove (<ref>), using Lemma <ref> instead of Lemma <ref>.



 Let ρ^ log_k_k∈ℕ, ρ^ ext_k_k∈ℕ, and α_k_k∈ℕ be the sequences of penalty parameters and stepsizes generated by LOG-DS. Let K_ log = {k∈ℕ: ρ^ log_k+1<ρ^ log_k } and K_ ext = {k∈ℕ: ρ^ ext_k+1<ρ^ ext_k }. Then |K_ log∩ K_ ext| = +∞.


Using (<ref>) and (<ref>), we get |K_ ext| = +∞. Let k∈ K_ ext. From (<ref>), we have

    α_k+1≤min{(ρ^ log_k)^β,(ρ^ ext_k)^β, (g_min)^2_k}≤min(ρ^ log_k)^β,(g_min)^2_k,

so that for all k∈ K_ ext we also have k∈ K_ log, and |K_ log∩ K_ ext| = |K_ ext| = +∞, concluding the proof.
















 Paper Title: Optimal design problem with thermal radiation.







[mytitlenote]
K.K. is partially supported by JSPS KAKENHI Grant Numbers JP24K16955 and JP24KJ0010.
K.M. is partially supported by Mizuho Foundation for the Promotion of Sciences and JSPS KAKENHI Grant Numbers JP24K17191 and JP23H03413.
T.O. is partially supported by JSPS KAKENHI Grant Numbers
JP22K20331 and JP23K12997.
This work is also supported by the Research Institute for Mathematical Sciences, an International Joint Usage/Research Center located in Kyoto University.


mainaddress]Kosuke Kita
kosuke.kita.c3@tohoku.ac.jp
mymainaddress]Kei Matsushima
matsushima@mech.t.u-tokyo.ac.jp
mymainaddress2]Tomoyuki Okamycorrespondingauthor
[mycorrespondingauthor]Corresponding author
t-oka@fit.ac.jp
[mainaddress]Graduate School of Science, Tohoku University, Aoba-ku, Sendai 980-8578, Japan
[mymainaddress]Graduate School of Engineering, The University of Tokyo, Bunkyo-ku, Tokyo 113-8656,Japan
[mymainaddress2]Faculty of Engineering, Fukuoka Institute of Technology, Higashi-ku, Fukuoka 811-0295, Japan





 
Optimal design problem, 
Topology optimization, 
Nonlinear boundary condition, 
Thermal radiation, 
Homogenization, 
Level set method
[2020] Primary: 80M50; Secondary: 35J65, 80M40






§ ABSTRACT 
This paper is concerned with configurations of two-material thermal conductors that minimize the Dirichlet energy for steady-state diffusion equations with nonlinear boundary conditions described mainly by maximal monotone operators. 
To find such configurations, a homogenization theorem will be proved and applied to an existence theorem for minimizers of a relaxation problem whose minimum value is equivalent to an original design problem. 
As a typical example of nonlinear boundary conditions, thermal radiation boundary conditions will be the focus, and then the Fréchet derivative of the Dirichlet energy will be derived, which is used to estimate the minimum value.
Since optimal configurations of the relaxation problem involve the so-called grayscale domains that do not make sense in general, a perimeter constraint problem via the positive part of the level set function will be introduced as an approximation problem to avoid such domains, and moreover, the existence theorem for minimizers of the perimeter constraint problem will be proved.
In particular, it will also be proved that 
the limit of minimizers for the approximation problem becomes that of the relaxation problem in a specific case, and then candidates for minimizers of the approximation problem will be constructed by employing time-discrete versions of nonlinear diffusion equations. 
In this paper, it will be shown that optimized configurations deeply depend on force terms 
as a characteristic of nonlinear problems and will also be applied to real physical problems.








§ INTRODUCTION
 
Transfer of thermal energy can be classified into conduction, convection, and radiation. Generally, the first two are usually treated as linear Partial Differential Equations (PDEs)  as long as thermal conductivity, specific heat and heat source are independent of temperature. In the case of the steady-state, many problems determining distributions of composite materials such that energies (or objective functionals) are minimized have been studied in various fields such as mathematics, physics, engineering and computer science as optimal design problem (or shape and topology optimization problem).
Material distributions of two-material composites are represented by employing characteristic functions, and therefore, the optimal design problem is described by a minimization problem of functionals with respect to the characteristic function. 
To guarantee the existence of minimizers, homogenization theory plays a crucial role since the minimizing sequence of the characteristic functions usually oscillates (see, e.g., <cit.>).
Although there are no minimizers in general, it has been proved that there exists a pair of volume fractions (or densities) and homogenized coefficients that achieve the minimum value (see, e.g., <cit.>). 
Hence it is a crucial issue to construct volume fractions that are similar to characteristic functions and give values close to the minimum, and therefore, various numerical techniques that do not involve intermediate sets have been devised (see e.g., <cit.>).
As for the unsteady-state, the existence of minimizers for a true relaxation problem has been proved in <cit.> along with the result in <cit.> and clarified the long-time behavior of optimal configurations concerning volume fraction in <cit.>. 

On the other hand, thermal radiation depends on temperature due to Stefan–Boltzmann law (cf. <cit.>) and is stated as nonlinear PDEs (in particular, it is formulated by
PDEs with nonlinear boundary conditions). From a physical point of view, the Stefan–Boltzmann law implies that the intensity of electromagnetic radiation from a black body is nonlinearly dependent on its surface temperature. Thermal radiation is especially important in high-temperature environments or thermally isolated systems in terms of conduction and convection, e.g., satellites and spacecrafts in outer space <cit.>. 

Such potential applications motivate us to develop an optimal design method for nonlinear PDEs. As shape and topology optimization is widely used for enhancing static and dynamic properties of solid deformation, much effort has been devoted to shape and topology optimization for nonlinear elasticity, e.g., large deformation <cit.>, hyper-elasticity <cit.>, and elasto-plasticity <cit.>. Another important subject is the fluid dynamics with the aim of designing, e.g., aircraft wings <cit.>, microfluidic systems <cit.>, and heat-dissipating structures with natural convection <cit.>. See also <cit.> for optimal design or shape optimization problems of other nonlinear PDEs.


    Some earlier studies have presented shape-optimized designs of heat radiators. For example, Dems and Korycki calculated a shape derivative associated with steady radiative heat transfer and optimized some parameters representing the boundary of a heat radiator <cit.>. Transient heat transfer with radiation was also studied in <cit.>. Recent work by Liu and Hasegawa developed a level set-based method for shape optimization in radiative heat transfer <cit.>. However, rigorous mathematical theory has not yet been established for optimal design problems with thermal radiation, which is crucial to constructing a unified theory that involves radiation as a natural extension of linear problems such as conduction and convection. 




 §.§ Setting of the problem

In this paper, for a given composite material consisting of two materials with different diffusion coefficients, we consider the material distribution that minimizes the Dirichlet energy described by the solution to the steady-state diffusion equation with the maximal monotone operator to handle various thermal transfers in a unified way.
Let Ω be a bounded domain of ^d with 
Lipschitz 
boundary ∂Ω, d≥ 1 and
Ω_i⊂Ω (i=0,1) be such that Ω=Ω_0∩Ω_1 and Ω_0∩Ω_1=∅.
For α,β>0 such that β>α>0 and i,j∈{1,…,d}, the class of diffusion coefficients is defined by 

    ℳ(α,β):={A∈ [L^∞(Ω)]^d× d A_ij=A_ji and α|ξ|^2≤ A(x)ξ·ξ≤β|ξ|^2  for all ξ∈^d and a.e. x∈Ω}.

Let u_χ_Ω_1∈ K be a weak solution to 

    -(A_χ_Ω_1∇ u_χ_Ω_1)=f      in Ω,
    
    -A_χ_Ω_1∇ u_χ_Ω_1·ν∈β(u_χ_Ω_1)
          on ∂Ω,

where ν is the unit outward normal vector on ∂Ω, f∈(H^1(Ω))^∗, A_χ_Ω_1∈ℳ(α,β) is the matrix field depending on the characteristic function χ_Ω_1∈ L^∞(Ω;{0,1}) given as

    χ_Ω_1(x)=
        
            1,     x∈Ω_1, 
    
            0,     x∈Ω_0,

K is some closed convex subset of a Hilbert space (explained later), and β is a (possibly multi-valued) maximal monotone graph in × such that β(0)∋ 0. Thus β satisfies 


  (i) (Monotonicity)
Let G(β) be the graph of β, i.e., 
G(β)={[z,ξ]∈×ξ∈β(z)}. Then it holds that 

    ⟨ z-w,ξ-ζ⟩≥0   for all  [z,ξ], [w,ζ]∈ G(β).
    

  (ii) (Maximality)
Any monotone graph
F:→ 2^
whose graph G(F) involves G(β) coincides with β.

Since β is a maximal monotone graph on ℝ×ℝ, 
there exists a proper, convex and lower semicontinuous function j:ℝ→ (-∞,+∞] such that β=∂ j, where ∂ j denotes a subdifferential of j, i.e., for w∈ℝ,

    ∂ j(w):={ξ∈ℝ j(z)-j(w)≥ξ(z-w)  for all  z∈ℝ}.


Now, our target minimization problem is formulated as follows:

    inf_χ_Ω_1∈𝒞𝒟{ℰ(χ_Ω_1):=∫_Ω A_χ_Ω_1(x) ∇ u_χ_Ω_1(x)·∇ u_χ_Ω_1(x) x̣},

where 𝒞𝒟 denotes the classical design domain given as 

    𝒞𝒟={χ_Ω_1∈ L^∞(Ω;{0,1})χ_Ω_1_L^1(Ω)= γ |Ω|},

γ∈ (0,1) denotes the volume ratio, and |Ω| describes the Lebesgue measure of Ω⊂^d.

[Two-material composite with different isotropic diffusion coefficients]
Typically, 

    κ[χ_Ω_1]:=α(1-χ_Ω_1)+βχ_Ω_1

stands for the diffusion coefficient of a composite material consisting of Ω_0 with diffusion coefficient α>0 and Ω_1 with diffusion coefficient β>0, and κ[χ_Ω_1]𝕀 belongs to ℳ(α,β). Here 𝕀 is the identity matrix of ^d× d.



[Maximal monotone operator] 
By noting that every subdifferential operator is maximal monotone (see, e.g., <cit.>), the following thermal radiation boundary condition can be taken:
    
    β(u_χ_Ω_1)=σ|u_χ_Ω_1|^du_χ_Ω_1,

    where σ>0 stands for the Stefan–Boltzmann coefficient. 
    In particular, one can represent standard linear boundary conditions with maximal monotone graph β.
    Indeed, if β(w)≡ 0 (resp. β(w)=aw for some a>0), then the boundary condition of (<ref>) is nothing but the homogeneous Neumann boundary condition (resp. Robin boundary conditions).
    Moreover, setting
    
    β_ D (w)=
            {   ℝ,            w=0,
       ∅,            w≠ 0,
            .

    we can understand the boundary condition of (<ref>) as the homogeneous Dirichlet boundary condition with β=β_ D (for details see <cit.>).
    
    Hence describing the boundary conditions using the maximal monotone graph enables us to treat conduction, convection and radiation in a unified manner.






 §.§ Aims and plan of the paper

This paper aims to prove that there exist minimizers of a true relaxation problem for (<ref>) and
construct an approximated characteristic function concerning χ_Ω_1∈ L^∞(Ω;{0,1}) that achieves (<ref>) with (<ref>) and (<ref>) numerically as typical examples of optimal design problems with nonlinear boundary conditions.
To this end, we shall first discuss the existence of minimizers of (<ref>) by using the direct method. 
Thus we shall take a minimizing sequence (χ_Ω_1^n) in 𝒞𝒟, i.e.,

    lim_n→ +∞ℰ(χ_Ω_1^n)= inf_χ_Ω_1∈𝒞𝒟ℰ(χ_Ω_1).

Since 𝒞𝒟 involves functions that oscillate rapidly, we shall see that the minimization problem (<ref>) is ill-posed in general due to χ_Ω_1^n_k→θ weakly-∗ in L^∞(Ω) but θ∉𝒞𝒟 for some subsequence (n_k) and volume fraction θ∈ L^∞(Ω;[0,1]).
Hence showing that optimal volume fractions exist, we shall next consider how to numerically construct the optimal volume fraction θ∈ L^∞(Ω;[0,1]) with few intermediate set [0<θ<1]:={x∈Ω 0<θ(x)<1}.  
In this paper, we shall employ the method devised in <cit.> as one way to resolve the so-called grayscale problem, and
in particular, it will reveal numerically how the linearity and nonlinearity of the boundary conditions affect the material distribution. 
Furthermore, we shall present some applications of the proposed optimization algorithm and show that optimal design is superior to some physically reasonable but intuitive structures. 


This paper is composed of seven sections.
Before discussing the limit of (<ref>), the next section is devoted to proving the well-posedness of (<ref>) and a corresponding homogenization theorem. 
Section <ref> deals with the existence theorem for minimizers of a relaxation problem with respect to the volume fraction.
In particular, we shall prove that the minimum value of the relaxation problem coincides with that of the original design problem (<ref>) and discuss how to construct the volume fraction that achieves the minimum value numerically. In section <ref>, we shall consider a perimeter constraint problem via the positive part of the level set function as an approximate problem for the relaxation problem. In particular, the existence theorem for minimizers of the approximation problem will be proved, and then the energy convergence of minimizers with respect to the perturbation parameter (>0) will also be shown. Furthermore, we shall provide a numerical algorithm that does not raise the grayscale problem. 
Section 5 details the validity of the numerical calculations and the effect of the nonlinearity of boundary conditions for optimized configurations, and then numerical examples for thermal radiation problems as more physical settings will be presented in Section 6.
The final section concludes this paper. 



§ HOMOGENIZATION PROBLEM FOR (<REF>)

As a preliminary to discuss the existence of minimizers in (<ref>), we consider the homogenization problem for (<ref>) with A_χ_Ω_1 being replaced by A^n.
Here and henceforth, we only consider the single-valued case of β for simplicity. To define a weak solution to (<ref>),
we first prepare some notations.
For simplicity, we set V=H^1(Ω), A_χ_Ω_1=A and u_χ_Ω_1=u.
Let a closed convex subset K⊂ V be given by 
K={v∈ V  j(v)∈ L^1(∂Ω)}, and ⟨·,·⟩ denotes the V^*-V dual coupling.
We define some functionals

    a(u,v):= ∫_Ω A(x)∇ u(x)·∇ v(x) x̣       for all  u,v∈ V,
        J(u):= ∫_∂Ω j(u)(x) σ̣         for all  u∈ K.

In addition, we assume that there exist δ>0 and C_0∈ℝ such that

    J(v) ≥δv_L^2(∂Ω)^2 - C_0      for all  v∈ K.

The typical example of j:→ (-∞,∞] satisfying (<ref>) is j(w)=1r|w|^r (i.e., β(w)=|w|^r-2w) for r>1.
We first touch on an inequality, which plays an important role in dealing with nonlinear boundary conditions.


    Let Ω⊂ℝ^d be a bounded Lipschitz domain. Then
    there exists a constant C>0 such that 
    
    v_L^2(Ω)^2 ≤ C ( ∇ v _L^2(Ω)^2 + v_L^2(∂Ω)^2 )

    for all v∈ H^1(Ω). As a consequence, (∇ v _L^2(Ω)^2 + v_L^2(∂Ω)^2)^1/2 is a equivalent norm in H^1(Ω).


    Suppose that for any n∈ℕ there exist (v_n)⊂ H^1(Ω) such that 
    
    v_n_L^2(Ω)^2 > n ( ∇ v_n _L^2(Ω)^2 + v_n_L^2(∂Ω)^2 ).

    Setting w_n=v_nv_n_L^2(Ω)^-1, we have w_n_L^2(Ω)=1 and 
    
    ∇ w_n_L^2(Ω)^2 + w_n_L^2(∂Ω)^2 < 1/n.

    Hence, since Ω is bounded and (w_n) is bounded in H^1(Ω), we can deduce that there exists a subsequence of (w_n) (denoted by (w_n) again) and w∈ H^1(Ω) such that 
    
    w_n → w        H^1(Ω),
       w_n → w        L^2(Ω)  L^2(∂Ω).

    From (<ref>), we can see that w=0 a.e. on ∂Ω.
    Moreover, the lower semicontinuity leads ∇ w_L^2(Ω)≤lim inf_n→ +∞∇ w_n_L^2(Ω) = 0, which implies w is constant, especially w=0.
    However, this contradicts that w_L^2(Ω)=1.
    The equivalence of the norm follows from (<ref>) and the trace theorem. This completes the proof.




In the framework of the variational inequality,
the weak solution of (<ref>) is defined as follows:

[Weak solution of (<ref>)]
For given f∈ V^∗, a function u∈ K is said to be a weak solution of (<ref>) if and only if the following inequality holds:

    a(u,v-u) + J(v) - J(u) ≥⟨ f,v-u ⟩   for all  v∈ K.




[Weak form of (<ref>)]
    We note that the definition of weak solutions for (<ref>) is defined by employing the variational inequality (<ref>).
    From physical motivation, we deal exclusively with the case of β(u)=σ |u|^r-2u (r>1, σ>0). 
In this case, setting v=u±λφ∈ K for all φ∈ K and λ>0 and letting λ→ 0_+, we see by ⟨ J'(u), ϕ⟩_L^r(∂Ω)= ⟨σ|u|^r-2u, ϕ⟩_L^r(∂Ω) that 
 
    a(u,φ) + σ∫_∂Ω |u|^r-2u(x) φ(x) σ̣= 
        ⟨ f, φ⟩    φ∈ K.

Thus the weak solution u∈ K to (<ref>) satisfies (<ref>). In particular, by the monotonicity of β, uniqueness of solutions to (<ref>) also follows, and then 
 every weak solution of (<ref>) coincides with the solution of this weak form (<ref>).
The reason why weak solutions are defined by (<ref>) instead of (<ref>) will be explained in Remark <ref>. 





 §.§ Solvability of (<ref>)
 
Let us briefly touch on the existence and uniqueness of a weak solution to (<ref>).


There exists a unique weak solution to (<ref>).



We first seek a function satisfying the following minimizing problem: 

    For given f∈ V^∗, find u∈ K such that 
        E(u) = inf_v∈ K E(v),

where E V⊃ K→ (-∞,∞] is the functional defined by

    E(v) = 1/2 a(v,v) - ⟨ f,v ⟩ + J(v).

It is easy to see that E is convex, lower semicontinuous and E≢∞.
Moreover, it follows from (<ref>) and Lemma <ref> that

    lim_v∈ K, v_V→ +∞ E(v) = +∞,
 
which along with <cit.> ensures the existence of
u∈ K satisfying (<ref>).
Thus setting E_1 and E_2 as the first two terms and the third term of the right-hand side in (<ref>), respectively, we have E_1(u)+E_2(u)=inf_v∈ K(E_1+E_2), and hence, <cit.> yields

    ⟨ E_1'(u),v-u⟩+E_2(v)-E_2(u)≥ 0    for all  v∈ K,
 
which implies (<ref>). 
The uniqueness of weak solutions follows immediately from the strict convexity of E.
Indeed, let u_1 and u_2∈ K be two minimizers for (<ref>).
If u_1≠ u_2, then

    E(u_i) ≤ E(u_1+u_2/2) < 1/2{ E(u_1) + E(u_2) }     (i=1,2),

that is, E(u_1)<E(u_2) and E(u_2)<E(u_1).
This completes the proof.







 §.§ Homogenization problem for (<ref>)



As for the proof of the homogenization theorem for (<ref>) with A_χ_Ω_1 being replaced by A^n, the following notion developed by Murat-Tartar is very useful.  

[H-convergence, cf. <cit.>]
Let A∈ℳ(α,β).  
For n>0, a sequence A^n∈ℳ(α,β) H-converges to an element A_ hom∈ [L^∞(Ω)]^d× d (denoted by A^nH→ A_ hom) if and only if, for any ω⋐Ω and any f ∈ H^-1(ω), the weak solution u^n∈ H^1_0(ω) of

    -(A^n ∇ u^n)=f   in  H^-1(ω)

is such that 

    u^n   → u_ hom       weakly in   H^1_0(ω),
    
       A^n∇ u^n   → A_ hom∇ u_ hom       weakly in   [L^2(ω)]^d,

where u_ hom∈ H^1_0(ω) is the weak solution to

    -(A_ hom∇ u_ hom)=f   in  H^-1(ω).



Then we have the following 

Let u^n∈ K be the unique weak solution to

    -(A^n ∇ u^n)=f      in Ω,
    
    -A^n ∇ u^n·ν
    = β(u^n)
          on ∂Ω,

where A^n∈ℳ(α,β), f∈ V^* and β is a maximal monotone operator in .
Then there exist a (not relabeled) subsequence of (n), u_ hom∈ K, the homogenized matrix A_ hom∈ℳ(α',β') and β'>α'>0  such that  

    u^n   → u_ hom       weakly in   V,
    
       A^n∇ u^n   → A_ hom∇ u_ hom       weakly in   [L^2(Ω)]^d.

Moreover, u_ hom∈ K is a weak solution to the homogenized equation,

    -(A_ hom∇ u_ hom)=f      in Ω,
    
    -A_ hom∇ u_ hom·ν
    = β(u_ hom)
          on ∂Ω.




By (<ref>) and (<ref>), it holds that

    a^n(u^n,u^n) + δu^n_L^2(∂Ω)^2 ≤ C_0 + ⟨ f,u^n ⟩,

which along with Lemma <ref> yields u^n_V≤ C. 
By virtue of the boundedness of (u^n)_n>0 in K, there exist a (not relabeled) subsequence of (n) and u_ hom∈ V such that

    u^n → u_ hom     V,
       u^n → u_ hom     L^2(Ω)∩ L^2(∂Ω).

Furthermore, by the H-compactness <cit.>, A^nH→ A_ hom also holds.
In particular, since it is obvious that u^n±φ∈ K for any φ∈ H^1_0(Ω), by choosing v=u^n±φ in (<ref>), it follows that

    a^n(u^n,φ) = ⟨ f,φ⟩     for all φ∈ H^1_0(Ω),

that is, 

    -div( A^n ∇ u^n ) = f      H^-1(Ω).

Hence applying <cit.> to (<ref>), we obtain

    A^n∇ u^n → A_ hom∇ u_ hom     [L^2(Ω)]^d,
 

    ∫_Ω A^n(x)∇ u^n(x)·∇ u^n(x) φ(x) x̣→∫_Ω A_ hom(x)∇ u_ hom(x)·∇ u_ hom(x) φ(x)  x̣

for all φ∈ C_ c^∞(Ω). 
Choosing φ∈ C_ c^∞(Ω) such that 0 ≤φ≤ 1 in (<ref>), 
we can deduce that

    ∫_Ω A_ hom(x)∇ u_ hom(x)·∇ u_ hom(x) φ(x) x̣≤lim inf_n→ +∞ a^n(u^n,u^n)   for all  φ∈ C_ c^∞(Ω),

whence follows 
the lower bound inequality,

    a_ hom(u_ hom,u_ hom) 
     =
     sup_φ∈ C_ c^∞(Ω),
    φ_L^∞(Ω)≤ 1∫_ΩA_ hom(x)∇ u_ hom(x)·∇ u_ hom(x)φ(x) x̣≤lim inf_n→ +∞ a^n(u^n,u^n)

(see <cit.>). Here we 
note that the upper bound inequality is more delicate (see Remark <ref> below).
Since K is closed convex subset in V, u_ hom∈ K holds.
Therefore the weakly lower semicontinuous on L^2(∂Ω) of J ensures that

    J(u_ hom) ≤lim inf_n→ +∞ J(u^n),

which together with the definition of weak solutions, (<ref>), (<ref>) and (<ref>) yields

    a_ hom(u_ hom,v-u_ hom) + J(v) - J(u_ hom) ≥⟨ f,v-u_ hom⟩   for all  v∈ K.

Thus u_ hom∈ K turns out to be a weak solution to the homogenized equation.
This completes the proof.


[Energy convergence]
Under the usual definition via the weak form, the following weak convergence will be required in the proof of Theorem <ref>: 

    β(u^n)→β(u_ hom)
       weakly in  L^ℓ(∂Ω)

for some 1<ℓ<+∞ (e.g., ℓ=r/(r-1) with β(w)=|w|^r-2w).
However, this proof is more complicated in general.
Thus Definition <ref> is more reasonable than the usual definition using the weak form (<ref>).
Moreover, we note that even if the homogeneous Dirichlet boundary condition, the upper bound inequality,

    lim sup_n→ +∞ a^n(u^n,u^n)
    ≤
    a_ hom(u_ hom,u_ hom)

is derived with the aid of the weak form for the homogenized equation. 
On the other hand, as soon as u_ hom turns out to be a weak solution to the homogenized equation satisfying the weak form
(for instance, 
it suffices to assume that 

    lim_λ→ 0_+J(w+λ v)-J(w)/λ=∫_∂Ω∂ j(w)(x)v(x) σ̣   for all  v∈ K),

we readily obtain (<ref>) by noting that 

    lim sup_n→ +∞   a^n(u^n,u^n)
       =
    lim_n→ +∞⟨ f,u^n⟩- lim inf_n→ +∞ J(u^n)
       ≤⟨ f,u_ hom⟩ - J(u_ hom)
    =  a_ hom(u_ hom,u_ hom).

Therefore convergence of the energy ℰ(χ_Ω_1^n) in (<ref>) can be obtained, which is applied to the proof of Theorem <ref> below.  


[Qualitative properties of homogenized matrices]
It is noteworthy that the homogenized matrix A_ hom∈ [L^∞(Ω)]^d× d is completely unaffected by (nonlinear) boundary conditions, and then it is also characterized exactly as in the case of the homogeneous Dirichlet boundary condition. 
Hence, in case d=1, it can be written as the so-called harmonic mean. Conversely (i.e., d≥ 2), it cannot be written explicitly in general. 
However, since A is symmetric, the following upper and lower bounds can be obtained:

    Aξ·ξ≤ A_ homξ·ξ≤Aξ·ξ   for all ξ∈^d,

where A is the inverse of the weak limit of A^-1 and A is the weak limit of A (see, e.g., <cit.>).





§ RELAXATION PROBLEM FOR (<REF>)


This section is devoted to proving the existence theorem for minimizers of a relaxation problem (see Theorem <ref> below). Thanks to Theorem <ref>, most of the proof relies on the results in <cit.>; however, we shall show it for completeness and the reader's convenience. 
Furthermore, we shall describe how to construct a candidate for the minimizers of the relaxation problem numerically. 
In what follows, we set A_χ_Ω_1=κ[χ_Ω_1] as in (<ref>) and write κ[χ_Ω_1]=κ and χ_Ω_1=χ for simplicity.




 §.§ Existence theorem for minimizers

Since the limit of the minimizing sequence does not belong to 𝒞𝒟 in general, we consider the following relaxation problem: 

    inf_(θ, κ_ hom)∈ℛ𝒟ℰ_ hom(θ, κ_ hom),

where ℛ𝒟 is the relaxed design domain defined by 

    ℛ𝒟
    := {
    (θ, κ_ hom)∈ L^∞(Ω;[0,1]×^d× d)   there exists 
    (χ^n,κ^n)∈  L^∞(Ω;{0,1}×{α,β})
       such that κ^n=α(1-χ^n)+βχ^n,
       χ^n→θ weakly-∗ in  L^∞(Ω;[0,1]),
       κ^n 𝕀H→κ_ hom and θ_L^1(Ω)= γ |Ω|
    },

ℰ_ hom:L^∞(Ω;[0,1]×^d× d)→ is the relaxed Dirichlet energy given as 

    ℰ_ hom(θ,κ_ hom)
    =∫_Ωκ_ hom(x)∇ u_ hom(x)·∇ u_ hom(x)  dx

and u_ hom∈ K is a weak solution to the homogenized equation (<ref>) with A_ hom=κ_ hom.

Then we see that the relaxation problem (<ref>) is a true relaxation of the original design problem (<ref>) in the following sense:

Let u^n∈ K be a weak solution to (<ref>) satisfying the weak from.
Let u_ hom∈ K be a weak solution to the homogenized equation
(<ref>). 
Then there exists at least one minimizer of (<ref>).
Furthermore, it holds that

    inf_χ∈𝒞𝒟ℰ(χ)
    =
    min_(θ,κ_ hom)∈ℛ𝒟ℰ_ hom(θ, κ_ hom),

and every minimizer of (<ref>) is characterized as a limit of the minimizing sequence in (<ref>), that is, for any minimizing sequence (χ^n) in 𝒞𝒟, there exist a (not relabeled ) subsequence of (n) and (θ^∗,κ_ hom^∗)∈ L^∞(Ω;[0,1]×^d× d) such that

    χ^n→θ^∗ weakly-∗ in  L^∞(Ω;[0,1]),  κ^n𝕀H→κ_ hom^∗

and

    inf_χ∈𝒞𝒟ℰ(χ)
    =
    ℰ_ hom(θ^∗,κ_ hom^∗).

Conversely, every minimizer in (<ref>) is attained by a limit of the minimizing sequence (χ^n) in  (<ref>).



Let (χ^n) be a minimizing sequence in 𝒞𝒟 for (<ref>).
Due to χ^n_L^∞(Ω)≤ 1 and Theorem <ref>, there exist
a (not relabeled) subsequence of (n) and (θ^∗, κ_ hom^∗)∈ L^∞(Ω;[0,1]×^d× d) such that (<ref>)
and 

    f
    ∫_Ωθ^∗(x) x̣=lim_n→ +∞∫_Ωχ^n(x) x̣=γ|Ω|.

Moreover, one can derive by Remark <ref> that

    inf_χ∈𝒞𝒟ℰ(χ)
        =lim_n→ +∞ℰ(χ^n)   =ℰ_ hom (θ^∗,κ_ hom^∗).

In particular, the above continuity (<ref>) is valid for non-minimizing sequences.

Now, we show that (θ^∗,κ_ hom^∗)∈ℛ𝒟 is a minimizer of (<ref>). For any (θ,κ_ hom)∈ℛ𝒟, there exists (χ^n,κ^n)∈ L^∞(Ω;{0,1}×{α,β}) such that

    χ^n→θ   weakly-∗ in  L^∞(Ω;[0,1])   and   κ^n 𝕀H→κ_ hom.

In particular, we can construct the sequence (χ^n) in L^∞(Ω;{0,1}) such that χ^n_L^1(Ω)=γ|Ω|, i.e., χ^n∈𝒞𝒟. 
Indeed, 
let χ̂^n∈ L^∞(Ω;{0,1}) be such that χ̂^n_L^1(Ω)=γ|Ω|.
Combining χ̂^n_L^∞(Ω)≤ 1 with

    lim_n→∞∫_Ωχ̂^n(x) x̣
    =
    γ|Ω|=
    ∫_Ωθ(x) x̣,

we have

    χ̂^n→θ   weakly-∗ in  L^∞(Ω).

Then defining Ω^n⊂Ω by 

    Ω^n={ x∈Ωχ^n(x)≠χ̂^n(x)  for a.e.  x∈Ω},

one obtain |Ω^n|→ 0 as n→ +∞, which along with   
the locality of the H-convergence (see  <cit.>) yields κ[χ̂^n]𝕀H→κ_ hom, and therefore, (χ̃^n) turns out to be the desired sequence.
Hence the continuity of ℰ and (<ref>) ensure that

    ℰ_ hom(θ,κ_ hom)=lim_n→ +∞ℰ(χ^n)≥inf_χ∈𝒞𝒟ℰ(χ),

which along with (<ref>) yields (<ref>) and (<ref>).  

On the other hand, let (θ,κ_ hom)∈ℛ𝒟 be a minimizer of (<ref>).
As already mentioned the above, (<ref>) follows for some χ^n∈𝒞𝒟 and some κ^n∈ L^∞(Ω;{α,β}) such that κ^n=α(1-χ^n)+βχ^n, and moreover,
ℰ_ hom(θ,κ_ hom)=lim_n→ +∞ℰ(χ^n) also holds, which implies that (χ^n) is a minimizing sequence in (<ref>). 
This completes the proof. 


[Interpretation of Theorem <ref>]
We note that the following facts:


  (i)
Theorem <ref> asserts that at least one minimizer exists in the relaxation problem (<ref>), which gives the same minimum value as the original design problem (<ref>). However, there is no guarantee for the uniqueness of minimizers.
In particular, if we further add geometric constraints (see, e.g., <cit.> for perimeter constraints) such that χ^n converges some characteristic function a.e. in Ω, the above proof ensures the existence of minimizers of the original design problem (<ref>) (with geometric constraints). 


  (ii)
As for the relaxation problem (<ref>), the volume fraction θ∈ L^∞(Ω;[0,1]) takes a value other than {0,1} a.e. in Ω.
Thus there are intermediate sets that are neither the material with diffusion coefficient α>0 (i.e., Ω_0⊂Ω) nor the material with diffusion coefficient β>0 (i.e., Ω_1⊂Ω), and optimal volume fractions are characterized by using intermediate sets.
In terms of the original design problem (<ref>), it is necessary to construct (θ,κ_ hom)∈ℛ𝒟 that attains the value close to the minimum in (<ref>) such that the so-called gray-scale problem is rarely raised. 









 §.§ Numerical algorithm for optimization of volume fractions

In this section, we describe a method to construct candidates for optimal volume fractions in (<ref>) numerically such that the minimum value of (<ref>) is achieved.
In the rest of this paper, 
we adapt ·_V=(∇·_L^2(Ω)^2 + ·_L^2(∂Ω)^2)^1/2 which is a norm in V=H^1(Ω) equivalent to the usual one, and then
we set β(w)=σ|w|^r-2w (i.e., j(w)=σ/r|w|^r), r≥ 2 and write
κ_ hom=κ, u_ hom=u and  
ℰ_ hom(θ,κ_ hom)=ℰ_ hom(κ) for simplicity.
In addition, we assume that f∈ L^2(Ω;_+) to get the following 

Let u∈ K be a unique weak solution to the (homogenized) state equation,

    -(κ∇ u)=f      in Ω,
    
    -κ∇ u·ν=σ|u|^r-2u       on ∂Ω.

Suppose that f≥ 0, ≢0. Then it holds that u≥ 0 a.e. in Ω.



Multiplying u^-:=max(-u,0) by (<ref>) and using integration by parts, we can derive that 

    0 ≤∫_Ω f(x)u^-(x) x̣   = ∫_Ωκ(x) ∇ u(x) ·∇ u^-(x) x̣ + σ∫_∂Ω |u|^r-2 u(x) u^-(x) σ̣[0]
       =∫_{u≤ 0}κ(x) ∇ u(x) ·∇ (-u)(x) x̣ + σ∫_u≤ 0 |u|^r-2 u(x) (-u)(x) σ̣[0]
       = - ∫_{u≤ 0}κ(x) ∇ (-u)(x) ·∇ (-u)(x)  x̣ - σ∫_u≤ 0 |u(x)|^r-2 (-u(x))^2 σ̣[0]
       ≤ - α' ∇ u^-_L^2(Ω)^2 - σ∫_∂Ω |u(x)|^r-2|u^-(x)|^2 σ̣,

which implies
∇ u^- =  u^-|_∂Ω = 0,
and therefore, u≥ 0 a.e. in Ω. This completes the proof.


Thanks to Lemma <ref>, |u|^r-2u can be described as u^r-1 below. 
We first derive the Fréchet derivative of ℰ_ hom:ℳ(α',β')→.

 Let u∈ K be a nonnegative weak solution to (<ref>) and let v∈ K be a weak solution to the (homogenized) adjoint equation,

    -(κ∇ v)=f      in Ω,
    
    -κ∇ v·ν=σ((r-1)u^r-2v+ru^r-1)       on ∂Ω.

Then ℰ_ hom is differentiable at κ, and it holds that

    ⟨ℰ_ hom'(κ),h⟩_[L^∞(Ω)]^d× d
    =
    -∫_Ωh(x)∇ u(x)·∇ v(x) x̣

for any h∈ [L^∞(Ω)]^d× d. 



Define ℒ:ℳ(α',β')× K× K → by

    ℒ(κ,u,w_κ)
       =
    ∫_Ωκ(x)∇ u(x)·∇ w_κ(x) x̣
    -
    ∫_∂Ωσu^r-1(x)(u(x)-w_κ(x)) σ̣+
    ∫_Ω f(x)(u(x)-w_κ(x)) x̣,

where ℳ(α',β')∋κ↦ w_κ∈ K is a differentiable at κ. 
Note that, for any h∈ [L^∞(Ω)]^d× d,

    ⟨ℒ'(κ,u, w_κ),h⟩_[L^∞(Ω)]^d× d
    =
    ⟨∂_κℒ(κ,u,w_κ), h⟩_[L^∞(Ω)]^d× d
    +
    ⟨∂_uℒ(κ,u,w_κ),u'h ⟩_V
    +
    ⟨∂_w_κℒ(κ,u,w_κ),w'_κ h ⟩_V.

Here we used the fact that κ↦ u=u_κ is differentiable (see Lemma <ref> below). 
We derive by the symmetry of κ that, for any φ∈ K,

    ⟨∂_uℒ(κ,u,w_κ),φ⟩_V   =
    ∫_Ωκ(x)∇ w_κ(x)·∇φ(x) x̣
          -∫_∂Ωσ[ru^r-1(x)
    -(r-1)u^r-2(x)w_κ(x)]φ(x) σ̣+
    ∫_Ω f(x)φ(x) x̣,

whence follows 
⟨ℰ_ hom'(κ),h⟩_[L^∞(Ω)]^d× d
=
⟨∂_κℒ(κ,u,-v), h⟩_[L^∞(Ω)]^d× d due to the differentiability of κ↦ v=v_κ (see Lemma <ref> below),
ℰ_ hom(κ)=ℒ(κ,u,-v) and ∂_uℒ(κ,u,-v)=∂_w_κℒ(κ,u,-v)=0. Thus we obtain (<ref>).


[Existence and regularity of solutions to the adjoint equation]
The existence of a unique weak solution to (<ref>) is assured by some natural assumption.
    Indeed, if the weak solution u∈ K to (<ref>) satisfies 
    
    u∈ L^∞(Ω),      u(x)≥ C>0   (x∈Γ_0)

    for some C>0 and Γ_0⊂∂Ω with |Γ_0|>0, then we can deduce that (<ref>) possesses a unique weak solution v∈ H^1(Ω).
    This result comes from the following inequality;
    there exists C>0 such that
    
    v_L^2(Ω)^2 ≤ C (  ∇ v_L^2(Ω)^2 + ∫_Γ_0κ̃v^2 σ̣)

    for any v∈ H^1(Ω), where κ̃∈ L^∞(∂Ω) which satisfies κ̃(x)≥ C>0 (x∈Γ_0) for some C>0 and Γ_0∈∂Ω with |Γ_0|>0.
    The above inequality can be proved in a similar way to the proof of Lemma <ref> with slight modification.
    By virtue of the assumptions on u and this inequality, the usual method by Lax–Milgram theorem can be applied to (<ref>) in order to show the existence of a weak solution.
    Unfortunately, it is difficult to prove the above assumption (<ref>) on u rigorously.
    However, if Ω and κ are sufficiently smooth, the solution u belongs to H^2(Ω) (for detail, see <cit.>),
    and we can deduce that u satisfies the above conditions with Γ_0=∂Ω.
    Therefore, our assumptions (<ref>) are quite natural, and after this, we always impose (<ref>) on the solution of (<ref>) implicitly whenever we consider a solution v to (<ref>).
    Moreover, in this setting, we can derive v∈ H^1(Ω)∩ L^∞(Ω) and in particular v∈ K.



As for u'=u_κ', we have the following
 
Suppose that (<ref>).
Then the nonnegative weak solution κ↦ u=u_κ of (<ref>) is differentiable at κ and  
    u_κ'h= ũ
    for the direction h∈ [L^∞(Ω)]^d× d. Here ũ∈ V satisfies

    ∫_Ωκ(x)∇ũ(x)·∇φ(x) x̣
        +
        ∫_∂Ωσ(r-1)u^r-2(x)ũ(x)φ(x) σ̣=
        -
       ∫_Ωh(x)∇ u(x)·∇φ(x) x̣

for all φ∈ V.



Let κ[s]=κ+sh for s>0 and let u[s] be a solution to (<ref>) with κ=κ[s].  
In this proof, we set σ=1 for simplicity.
Differentiating with respect to s>0 in the weak form of (<ref>) with κ=κ[s], we have   

    ∫_Ωκ[s](x)∇ u'[s](x)·∇φ(x) x̣
        +
        ∫_∂Ω(r-1)u^r-2[s](x)u'[s](x)φ(x) σ̣=
        -
       ∫_Ωh∇ u[s](x)·∇φ(x) x̣,

which coincides with (<ref>) as ũ=u'[s] and s=0.
Noting that κ=κ[0]=κ[1]-h, u_κ+h=u[1] and u_κ=u[0], we observe that, for any φ∈ V,

    ∫_Ωκ(x)∇(u_κ+h(x)-u_κ(x)-ũ(x))·∇φ(x) x̣[0]
    
         +
        ∫_∂Ω
        (u_κ+h^r-1(x)-u_κ^r-1(x)-(r-1)u_κ^r-2(x)ũ(x))φ(x) σ̣[0]
       =
    ∫_Ω[(κ[1](x)-h(x))∇ u[1](x)-κ[0](x)∇ u[0](x)-κ(x)∇ũ(x)]·∇φ(x) x̣[0]
         +
        ∫_∂Ω
        (u[1]^r-1(x)-u[0]^r-1(x)-(r-1)u[0]^r-2(x)ũ(x))φ(x) σ̣[0]
        =
        -
       ∫_Ωh(x)∇ (u[1](x)-u[0](x))·∇φ(x) x̣≤h_L^∞(Ω)∇(u_κ+h-u_κ)_L^2(Ω)∇φ_L^2(Ω).

As for the upper bound of ∇(u_κ+h-u_κ)_L^2(Ω), we deduce from the same argument that 

    ∫_Ωκ(x)∇(u_κ+h(x)-u_κ(x))·∇ (u_κ+h(x)-u_κ(x)) x̣
    +
    ∫_∂Ω
      (u_κ+h^r-1(x)-u_κ^r-1(x))(u_κ+h(x)-u_κ(x)) σ̣
       =
      -
       ∫_Ωh(x)∇ u_κ+h(x)·∇ (u_κ+h(x)-u_κ(x)) x̣≤h_L^∞(Ω)∇ u_κ+h_L^2(Ω)∇ (u_κ+h-u_κ)_L^2(Ω),

which along with the boundedness of (u_k+h) in V and the uniform ellipticity of κ yields 

    ∇(u_κ+h-u_κ)_L^2(Ω)≤ Ch_L^∞(Ω).

Hence, setting φ=u_κ+h-u_κ-ũ in  (<ref>), we have

    ∇(u_κ+h-u_κ-ũ)_L^2(Ω)^2
        +
     ∫_∂Ω
        (u_κ+h^r-1(x)-u_κ^r-1(x)-(r-1)u_κ^r-2(x)ũ(x))(u_κ+h(x)-u_κ(x)-ũ(x)) σ̣
         ≤ Ch_L^∞(Ω)^2 ∇(u_κ+h-u_κ-ũ)_L^2(Ω).

By noting that the integrand of the second term of the left-hand side in (<ref>) is written as

    (u_κ+h^r-1-u_κ^r-1-(r-1)u_κ^r-2ũ)(u_κ+h-u_κ-ũ)
       =
    (u_κ+h^r-1-u_κ^r-1-(r-1)u_κ^r-2ũ-(u_κ+h-u_κ-ũ))(u_κ+h-u_κ-ũ)
    +
    (u_κ+h-u_κ-ũ)^2.

The integral of the first term in the last line can be estimated as follows:

    ∫_∂Ω
    (u_κ+h^r-1(x)-u_κ^r-1(x)-(r-1)u_κ^r-2(x)ũ(x)-(u_κ+h(x)-u_κ(x)-ũ(x)))(u_κ+h(x)-u_κ(x)-ũ(x)) σ̣
         ≤ 
    Ch_L^∞(Ω)^2 ∇(u_κ+h-u_κ-ũ)_L^2(Ω).

Hence one can derive that

    ∇(u_κ+h-u_κ-ũ)_L^2(Ω)≤ Ch_L^∞(Ω)^2,

which together with (<ref>), (<ref>) and (<ref>) yields 

    u_κ+h-u_κ-ũ_L^2(∂Ω)≤ Ch_L^∞(Ω)^2.

Combining (<ref>) with (<ref>), we conclude that

    lim_h_L^∞(Ω)→ 0_+u_k+h-u_k-ũ_V/h_L^∞(Ω)=0,

which completes the proof.


By the same argument as in the proof of Lemma <ref>, we have the following 
 
Suppose that (<ref>). 
Let u∈ K and ũ∈ V be weak solutions to (<ref>) and (<ref>), respectively.
Then the weak solution κ↦ v=v_κ∈ K of (<ref>) is differentiable at κ and  
    v_κ'h= ṽ
    for the direction h∈ [L^∞(Ω)]^d× d. Here ṽ∈ V satisfies
    
    ∫_Ωκ(x)∇ṽ(x)·∇φ(x) x̣
          +σ∫_∂Ω
        [
        r(r-1)u^r-2(x)ũ(x)
        +(r-1)(r-2)u^r-3(x)ũ(x)v(x)
        +(r-1)u^r-2ṽ(x)
        ]φ(x) σ̣
       =
        -
       ∫_Ωh∇ v(x)·∇φ(x) x̣

for all φ∈ V.



Let κ[s]=κ+sh for s>0 and σ=1 for simplicity. Let u[s] and v[s] be weak solutions to (<ref>) with κ=κ[s] and (<ref>) with κ=κ[s], respectively. Then, by differentiating with respect to s>0 in the weak form of (<ref>), we obtain (<ref>) as u'[s]=ũ, v'[s]=ṽ and s=0. 
Furthermore, we get 
∇(v_k+h-v_k)_L^2(Ω)≤ C h_L^∞(Ω)
as in (<ref>).
Here we used the fact that ∇ v_k+h_L^2(Ω)≤ C by virtue of (<ref>) (see Remark <ref>).
Then we observe that, for any φ∈ V,

    ∫_Ωκ(x)∇(v_κ+h(x)-v_κ(x)-ṽ(x))·∇(v_κ+h(x)-v_κ(x)-ṽ(x)) x̣[0]
         +
    (r-1)    ∫_∂Ω[u_κ+h^r-2(x)
        v_κ+h(x)-u_κ^r-2(x)v_κ(x)-u_κ^r-2(x)ṽ(x)](v_κ+h(x)-v_κ(x)-ṽ(x)) σ̣[0]
         -
    ∫_∂Ω[
     r(r-1)u_κ^r-2(x)ũ(x)
        +(r-1)(r-2)u_κ^r-3(x)ũ(x)v_κ(x)
    ](v_κ+h(x)-v_κ(x)-ṽ(x)) σ̣[0]
       =
        -
       ∫_Ωh(x)∇ (v_k+h(x)-v_k(x))·∇ (v_k+h(x)-v_k(x)-ṽ(x)) x̣
       ≤h_L^∞(Ω)∇(u_κ+h-u_κ)_L^2(Ω)∇ (v_k+h-v_k-ṽ)_L^2(Ω)≤ 
        Ch_L^∞(Ω)^2∇ (v_k+h-v_k-ṽ)_L^2(Ω).

Here we note that the integrand of the second line is written as

    [u_κ+h^r-2
        v_κ+h-u_κ^r-2v_κ-u_κ^r-2ṽ](v_κ+h-v_κ-ṽ)
         =
    [(u_κ+h^r-2
        v_κ+h-u_κ^r-2v_κ-u_κ^r-2ṽ)
    -(v_κ+h-v_κ-ṽ)](v_κ+h-v_κ-ṽ)
    +
        (v_κ+h-v_κ-ṽ)^2.

Thus we see by 
the same argument as in the proof of Lemma <ref> and 
the uniform ellipticity of κ that

    ∇(v_κ+h-v_κ-ṽ)_L^2(Ω)^2+
    v_κ+h-v_κ-ṽ_L^2(∂Ω)^2
    ≤ 
    Ch_L^∞(Ω)^2
    ∇ (v_k+h-v_k-ṽ)_L^2(Ω),

which implies that 
v_κ+h-v_κ-ṽ_V≤ 
Ch_L^∞(Ω)^2, and hence,

    lim_h_L^∞(Ω)→ 0_+v_k+h-v_k-ṽ_V/h_L^∞(Ω)=0.

This completes the proof.



We next observe the relation between u and v.


Assume that (<ref>).
Let u∈ K and v∈ V be weak solutions to (<ref>) and (<ref>), respectively.
Then it holds that

    ∇ (u-v)_L^2(Ω)^2≤r-1/α'√(2/α_0)f_L^2(Ω){1/2α_0f_L^2(Ω)^2 + r-2/2σ |∂Ω| }^1/2,

where α_0 := min(α',r/2σ). 




By (<ref>) and  (<ref>), we have

    ∫_Ωκ(x) ∇(u(x)-v(x)) ·∇(u(x)-v(x)) x̣ - ∫_∂Ωκ(x) ∇ (u(x)-v(x)) ·ν(x) (u(x)-v(x))  σ̣=0.

From the boundary conditions, the second term of the left-hand side implies that

    -∫_∂Ωκ(x) ∇ (u(x)-v(x)) ·ν(x) (u(x)-v(x)) σ̣
       =  
    ∫_∂Ωσ{ u^r-1(x) -(r-1) u^r-2(x)v(x) -ru^r-1(x) } (u(x)-v(x)) σ̣[0]
       = 
    - σ (r-1) ∫_∂Ω u^r-2(x)(u^2(x)-v^2(x)) σ̣≥ 
    - σ (r-1) ∫_∂Ω u^r(x) σ̣.

Here we used the nonnegativity of u≥0 in the last inequality.
Hence we have

    α' ∇(u-v)_L^2(Ω)^2 ≤σ (r-1) ∫_∂Ω u^r(x) σ̣(<ref>)≤ (r-1)∫_Ω fu x̣≤ (r-1)f_L^2(Ω)u_V.

In the rest of the proof, it suffices to show that

    u_V ≤√(2/α_0){1/2α_0f_L^2(Ω)^2 + r-2/2σ |∂Ω| }^1/2.

By (<ref>), it holds that

    α' ∇ u_L^2(Ω)^2 + σ∫_∂Ω u^r(x)  σ̣≤∫_Ω fu x̣.

Moreover, it follows from Hölder's inequality and Young's inequality that

    ∫_∂Ω u^2(x) σ̣≤( ∫_∂Ω u^r(x) σ̣)^2/r |∂Ω|^r-2/r≤2/r∫_∂Ω u^r(x)  σ̣+ r-2/r |∂Ω|.

Thus we obtain

    α'∇ u_L^2(Ω)^2 + r/2σu_L^2(∂Ω)^2 - r-2/2σ |∂Ω| ≤f_L^2(Ω)u_V.

Using Young's inequality, we can derive

    α_0 u_V^2 ≤α_0/2u_V^2 + 1/2α_0f_L^2(Ω)^2 + r-2/2σ |∂Ω|,

which implies (<ref>).
Therefore, by (<ref>) and (<ref>), we obtain 

    ∇ (u-v)_L^2(Ω)^2≤r-1/α'√(2/α_0)f_L^2(Ω){1/2α_0f_L^2(Ω)^2 + r-2/2σ |∂Ω| }^1/2,

which is the desired result.

Combining Propositions <ref> with <ref>, we see by f≢0 
that, for any κ, h∈ℳ(α',β'),

    ⟨ℰ_ hom'(κ),h⟩_[L^∞(Ω)]^d× d     =
      -  ∫_Ωh(x)∇ u(x)·∇ v(x) x̣
       = 
    1/2∫_Ωh(x)∇ (u(x)-v(x))·∇(u(x)- v(x)) x̣
         
    -
    1/2∫_Ωh(x)∇ u(x)·∇ u(x) x̣
    -
    1/2∫_Ωh(x)∇ v(x)·∇ v(x) x̣
       <
    β'/2∇ (u-v)_L^2(Ω)^2
       ≤β'/2r-1/α'√(2/α_0)f_L^2(Ω){1/2α_0f_L^2(Ω)^2 + r-2/2σ |∂Ω| }^1/2.

Thus one may expect  
ℰ_ hom'(κ)=-∇ u·∇ v≤ 0 
a.e. in Ω at least in the case where f≥ 0 is small (see Remark <ref> below). 
In this particular case, due to ℰ(θ)≤ℰ_ hom(κ) by (<ref>), one can estimate the minimum value by replacing (<ref>) with the following minimization problem: 

    inf_θ∈Θℰ(θ),

where Θ:={θ∈ L^∞(Ω;[0,1])θ_L^1(Ω)=γ|Ω|}[In the self adjoint problem (i.e., v=± u), there is a case where an optimal homogenized matrix can be characterized as κ^∗=κ[θ^∗]=α(1-θ^∗)+βθ^∗. Here θ^∗ is an optimal volume fraction (i.e., ℰ_ hom(κ^∗)=ℰ_ hom(θ^∗,κ_ hom^∗)=ℰ(θ^∗)). Hence it suffices to consider (<ref>) instead of (<ref>); however, nonlinear problems cause non-self adjoint problems in general (see <cit.> for an optimal homogenized flux). 
Thus the problem with χ_Ω_1 being replaced by θ as in (<ref>) is just a problem to estimate the infimum value in general. 
On the other hand, as in the homogeneous Dirichlet boundary condition, one can construct the self adjoint problem for the homogeneous Robin boundary condition (i.e., r=2) by setting ℰ(χ)=⟨ f,u_χ⟩. 
]

Now, we are in a position to describe a numerical algorithm for the volume fraction θ∈Θ.
Based on the (steepest gradient) descent method (or time-discrete version of the gradient flow) and Proposition <ref>, we set 

    θ_i+1=θ_i-τℰ'(θ_i)=θ_i-τ (β-α)∇ u_θ_i·∇ v_θ_i   for  i∈∪{0}.

Here θ_0∈ L^∞(Ω) is an initial volume fraction,
τ>0 stands for the step width (or time step, i.e., θ_i implies θ_i=θ(x,τ i)) and
u_θ∈ K and v_θ∈ K are unique weak solutions to (<ref>) with κ=κ[θ] and (<ref>) with κ=κ[θ], respectively. 
Repeating (<ref>) until θ_i+1_L^1(Ω)=γ|Ω| and θ_i+1-θ_i_L^1(Ω)≤η for η>0 small enough, one can estimate the minimum value of ℰ(χ_Ω_1) numerically by Theorem <ref>. The following is the numerical algorithm:





[Linearization of the thermal radiation boundary condition] 
      To solve (<ref>) with κ=κ[θ] numerically,  
we first approximate β(u_θ) as β(u_ old)+β'(u_ old)(u_θ-u_ old) as in the Newton–Raphson method. 
Here u_ old is an arbitrarily given function, and we choose r=d+2 in (<ref>) based on <cit.>, that is, 
β(u_θ)≈σu_ old^d+1+
σ(d+1)u_ old^d(u_θ-u_ old)
=
σ(d+1)u_ old^du_θ-σ du_ old^d+1.
We next solve the following linearized equation: 

    0   =-∫_Ω f(x) φ(x) x̣
    +
    ∫_Ωκ[θ](x) ∇ u_θ(x)·∇φ(x) x̣
         + 
    ∫_∂Ω[σ (d+1)u_ old^d (x)u_θ(x)
        -
        σ du_ old^d+1(x)]_linearized thermal radiation boundary conditionφ(x) σ̣   for all φ∈ V.

We finally check the following convergence condition:

    |
        ∫_Ωκ[θ](x) |∇ u_θ(x)|^2 x̣+ 
        σ∫_∂Ω |u_θ (x)|^d+2 σ̣-∫_Ω f(x) u_θ(x) x̣|≤η_3
       for some η_3>0.

If (<ref>) is not satisfied, we set u_ old← u_θ, and then we solve (<ref>) again. 
This procedure is repeated until (<ref>)
is satisfied. 




[Convergence condition]
By (<ref>), it holds that

    |θ_i+1-θ_i|=τ (β-α)|∇ u_θ_i·∇ v_θ_i|.

If θ_i+1 attains the critical point of ℰ(θ), then the right-hand side vanishes.  
Since it belongs to L^1(Ω) at least, the convergence condition (<ref>) is reasonable.
In this paper, we do not mention the regularization of sensitivity ℰ'(θ_i) to become θ_i+1∈ L^∞(Ω) since Algorithm <ref> is only used to estimate the minimum value of the original optimal design problem (<ref>) and inf_θ∈Θℰ(θ)= inf_θ∈Θ̃ℰ(θ),    
where Θ̃:={θ∈ L^1(Ω)θ(x)∈ [0,1]  and θ_L^1(Ω)=γ|Ω|}.


[Self adjointness and convexity of a linearized problem] 
Let u_ L be a solution to (<ref>) 
with |u|^r-2u being replaced by u_ old^r-1, and then consider the minimization problem (<ref>) 
with u_ hom being replaced by u_ L. Here u_ old is a function that appears in Remark <ref>.
Then it can be regarded as a self-adjoint problem by the same argument as in Proposition <ref>. Thus (<ref>) with u_ hom being replaced by u_ L turns out to be a true relaxation problem by Theorem <ref>, Proposition <ref> and <ref>, and moreover, it has only global minimizers in terms of double minimization; indeed, define ℰ̃ V→ by

    ℰ̃(w)=
     1/2∫_Ωκ[θ](x)|∇ w(x)|^2 x̣
        +
        ∫_∂Ωσu_ old^r-1(x) w(x) σ̣-∫_Ω f(x) w(x) x̣.

Then we see that u_ L= argmin_w∈ Kℰ̃(w) and  

    ∫_Ωκ[θ](x)|∇ u_ L(x)|^2 x̣   =
    2(∫_Ω f(x) u_ L(x) x̣    
    -∫_∂Ωσu_ old^r-1(x) u_ L(x) σ̣)
    -
    ∫_Ωκ[θ](x)|∇ u_ L(x)|^2 x̣
       = -2 inf_w∈ Kℰ̃(w).

Let ℰ̂:V× [L^2(Ω)]^d→ be such that ℰ̂(w,∇ w)=ℰ̃(w). 
Since ℰ̂ is convex, the dual energy yields 

    inf_w∈ Kℰ̃(w)=inf_P^∗∈ [L^2(Ω)]^d,
     - P^∗=f  in  Ω, 
    
    -P^∗·ν=σu_ old^r-1 on  ∂Ω∫_Ωκ[θ]^-1(x)|P^∗(x)|^2 x̣

(see, e.g., <cit.>).
Thus the minimization problem (<ref>) with u_ hom being replaced by u_ L is equivalent to the following double minimization problem:

    min_(θ,P^∗)∈𝒲∫_Ωκ[θ]^-1(x)|P^∗(x)|^2 x̣,

where 

    𝒲:={
    (θ,P)∈Θ× [L^2(Ω)]^d
    - P^∗=f  in  Ω and  
    -P^∗·ν=σu_ old^r-1 on  ∂Ω}.

Since 𝒲 is convex, and (θ, P^∗)↦κ[θ]^-1|P^∗|^2 is also convex, the assertion is obtained. 
Hence, if u_ old^r-1 sufficiently approximates u^r-1 on ∂Ω, the convergence value of energies via Algorithm <ref> also approximates the minimum value for (<ref>) with 
A_χ_Ω_1=κ[χ_Ω_1] and
β(u_χ_Ω_1)=σu_χ_Ω_1^r-1.  


In this paper, to estimate the minimum value of (<ref>) with A_χ_Ω_1=κ[χ_Ω_1] and β(u_Ω_1)=σu_Ω_1^d+1 numerically, we consider the state equation as an approximated equation with inhomogeneous Neumann boundary conditions in optimization of the volume fraction; in other words, ℰ'(θ_i) in (<ref>) is regarded as ℰ'(θ_i)=-(β-α)|∇ u_θ_i|^2.




§ APPROXIMATION PROBLEM FOR (<REF>) VIA POSITIVE PARTS OF LEVEL SET FUNCTIONS

In this section, we shall prepare a numerical analysis to find 
two-material distributions that give a value close to the minimum for (<ref>) with 
A_χ_Ω_1=κ[χ_Ω_1] and 
β(w)=σw^d+1. 
As already mentioned in (ii) of Remark <ref>, 
we need to construct the optimal volume fraction θ numerically such that
the intermediate set [0<θ<1] rarely appears due to non-existence of minimizers for (<ref>) in general. 
As one of the methods to avoid the so-called grayscale problem, 
level set methods (see, e.g., <cit.>) are known and employed to construct an approximated minimizer below.
In level set methods, the following level set function is introduced to represent two-material domains:  

    ϕ(x)
    
    >0,      x∈Ω_1,
    
    =0,      x∈∂Ω_1∩∂Ω_0,
    
    <0,      x∈Ω_0.

Based on <cit.>, we consider the following perimeter constraint problem via the positive part of the level set function as an approximation problem of (<ref>):

    inf_ϕ∈ U_ ad{J_(ϕ):=ℰ(ϕ_+)+/p∫_Ω |∇ϕ(x)|^p x̣},

where ϕ_+=max{0,ϕ}, U_ ad:={ϕ∈ W^1,p(Ω) |ϕ|≤ 1  and ϕ_+_L^1(Ω)=γ |Ω|}, 1<p<+∞ and >0.
In particular, the second term of J_ (i.e., the p-Dirichlet energy) plays a role of perimeter constraint (cf. <cit.>). 




 §.§ Characterization of minimizers for level set functions

In order to form the basis of numerical analysis for (<ref>), we first show the following

    There exists at least one minimizer of (<ref>).



Let (ϕ^n) be a minimizing sequence in U_ ad. Thus ϕ^n satisfies

    lim_n→ +∞ J_(ϕ^n)=inf_ϕ∈ U_ adJ_(ϕ).

Since (ϕ^n) is bounded in W^1,p(Ω) due to J_(ϕ^n)→ +∞ as ϕ^n_W^1,p(Ω)→ +∞, there exist a (not relabeled) subsequence of (n) and ϕ^∗∈ U_ ad
such that

    ϕ^n→ϕ^∗ weakly in  W^1,p(Ω)

and 

    ϕ^n_+→ϕ^∗_+  a.e. in Ω.

Hence Theorem <ref> ensures that

    lim_n→ +∞ℰ(ϕ^n_+)= ℰ(ϕ^∗_+).

Combining (<ref>) with the weak lower semicontinuity of norm, we obtain

    inf_ϕ∈ U_ ad  J_(ϕ)
    ≤
    J_(ϕ^∗)
    ≤lim inf_n→ +∞ J_(ϕ^n)
    =
    inf_ϕ∈ U_ ad  J_(ϕ),

which completes the proof.





Furthermore, we have the following

Let ϕ^ be a minimizer of (<ref>). 
Then there exist a (not relabeled) subsequence of () and ϕ^∗∈ U_ ad such that
ϕ^→ϕ^∗ weakly in W^1,p(Ω) and

    ℰ(ϕ^∗_+)=
    lim_→ 0_+J_(ϕ^)=inf_ϕ∈ U_ adℰ(ϕ_+).




We first note that a_:=J_(ϕ^) has a limit; indeed, if s≥ t for s,t>0, we have

    a_s≥ J_t(ϕ^s)≥ a_t≥inf_θ∈Θℰ(θ)≥ 0,

which yields the assertion. 
Furthermore, due to a_→ +∞ as ϕ^_W^1,p(Ω)→ +∞, we see that (ϕ^) is bounded in W^1,p(Ω). Thus, as in the proof of Theorem <ref>, there exist a (not relabeled) subsequence of () and ϕ^∗∈ U_ ad such that (<ref>) and (<ref>) with n=. 
Therefore, it follows that

    lim_→ 0_+ a_=ℰ(ϕ^∗_+).

Noting that, for any ϕ∈ U_ ad, 

    a_   ≤∫_Ωκ[ϕ_+](x)|∇ u_ϕ(x)|^2 x̣
        +/p∫_Ω|∇ϕ(x)|^p x̣→ℰ(ϕ_+)   as   → 0_+,

one obtains
ℰ(ϕ^∗_+)≤ℰ(ϕ_+) for all ϕ∈ U_ ad. This completes the proof.



[Approximate solutions for (<ref>)]
By Theorem <ref>, it holds that

    ℰ(ϕ^∗_+)≤ℰ(θ)   for all   θ∈Θ∩ W^1,p(Ω).

Thus ϕ_+^∗ turns out to be a minimizer of (<ref>) under
 inf_θ∈Θℰ(θ)
=
inf_θ∈Θ∩ W^1,p(Ω)ℰ(θ). 
In this case, Theorem <ref> ensures that ϕ_+^ for >0 small enough can be regarded as an approximate solution for (<ref>) under the optimal homogenized matrix can be written as the upper bound.
Since χ_Ω_1∈ BV(Ω) is required at least in the perimeter constraint problem for (<ref>), it is reasonable to assume additional regularity as a setting that avoids the grayscale problem. 
In particular,
the optimal volume fraction θ^∗ of (<ref>) is weakly differentiable in the direction of ∇ u_θ^∗ under u_θ^∗∈ H^2(Ω); indeed, we observe that, for any φ∈ C^∞_ c(Ω),

    ( F,φ)_L^2(Ω)   :=-∫_Ωθ^∗(x) [(∇ u_θ^∗(x))φ(x)] x̣
        =
    -∫_Ωθ^∗(x)Δ  u_θ^∗(x) φ(x) x̣
    -∫_Ωθ^∗(x) ∇ u_θ^∗(x)·∇φ(x) x̣.

Then the second term in the last line is written as

    -∫_Ωθ^∗(x) ∇ u_θ^∗(x)·∇φ(x) x̣
    =
    1/β-α∫_Ω [α∇ u_θ^∗(x)·∇φ(x)-f(x)φ(x)] x̣,

and therefore, (∇θ^∗·∇ u_θ^∗,φ)_L^2(Ω) makes sense
by noting that

    |( F,φ)_L^2(Ω)|
    =
    |-∫_Ω[(θ^∗(x)+α/β-α)Δ  u_θ^∗(x)+ 
    1/β-αf(x)]φ(x) x̣|≤ C_θ^∗, fφ_L^2(Ω)

(see <cit.> for the homogeneous Dirichlet boundary condition).



[Extension from ϕ_+ to ϕ_+^m]
In Theorems <ref> and <ref>, one can replace ϕ_+ in (<ref>) with ϕ_+^m for all m≥ 1. Indeed, let A:L^m+1(Ω)→ L^(m+1)/m(Ω) be an operator defined by A(w)= w_+^m.
Then A is maximal monotone in L^m+1(Ω)× L^(m+1)/m(Ω). Noting that

    ϕ^n   →ϕ^∗       weakly in  L^m+1(Ω),
               strongly in  L^p(Ω),
    
     A(ϕ^n)    →ξ         weakly in  L^p/(p-1)(Ω)∩ L^(m+1)/m(Ω)

for some ξ∈ L^p/(p-1)(Ω)∩ L^(m+1)/m(Ω) and n=n_k, we have

    ⟨ A(ϕ^n),ϕ^n⟩_L^m+1(Ω)=
    ⟨ A(ϕ^n),ϕ^n⟩_L^p(Ω)→⟨ξ,ϕ^∗⟩_L^p(Ω)
    =
    ⟨ξ,ϕ^∗⟩_L^m+1(Ω)   as  n→ +∞ .

Hence Minty’s trick (see e.g., <cit.>) ensures that ξ=A(ϕ^∗)=(ϕ_+^∗)^m. 
Furthermore, 
since (ϕ_+^n) is also bounded in W^1,p(Ω), 
one can extract a subsequence of (n_k) (still denoted by n_k) such that ϕ_+^n_k→ϕ_+^∗ a.e. in Ω, which along with the boundedness of (ϕ_+^n) in L^∞(Ω) yields A(ϕ^n_k)→ A(ϕ^∗) strongly in L^ℓ(Ω) for all ℓ≥ 1.
The rest of the proofs runs as before. 

In this paper, we select m=1 to compare the results in <cit.> (cf. <cit.> for m>1). 





 §.§ Numerical algorithm for optimization of level set functions

Before describing the numerical algorithm, we derive the equation to update the level set function. 
As in (<ref>), we introduce the following (gradient) descent method:

    ϕ_i+1=ϕ_i-τ J_'(ϕ_i)   for  i∈∪{0}.

Since the Fréchet derivative of p-Dirichet energy for the level set function is -Δ_pϕ=-(|∇ϕ|^p-2∇ϕ) under homogeneous Dirichlet/Neumann boundary condition, we have

    ϕ_i+1=ϕ_i-τ (∂_ϕ_iℰ((ϕ_i)_+)-Δ_p ϕ_i+1)   for  i∈∪{0}.

Here we note that Δ_p ϕ_i is replaced with Δ_p ϕ_i+1 in order to satisfy ϕ_i+1∈ W^1,p(Ω)
(see, e.g. <cit.>) for forward-backward splitting schemes).
Furthermore, by Remark <ref> and Proposition <ref>, the above update equation
(<ref>) with (ϕ_i)_+ being replaced by (ϕ_i)_+^m can be written as 

    ϕ_i+1=ϕ_i-ζ[ϕ_i](-m(β-α)(ϕ_i)_+^m-1χ_ϕ_i∇ u_ϕ_i·∇ v_ϕ_i -Δ_p ϕ_i+1)
       for  i∈∪{0},

where u_ϕ_i and v_ϕ_i are the unique weak solutions to (<ref>) with κ=κ[(ϕ_i)_+] and (<ref>) with κ=κ[(ϕ_i)_+], respectively.
Here we note that τ>0 in (<ref>) is extended to 
ζ[ϕ_i]≥ 0, which is the generalized step width such that ζ[0]=0. Indeed,
in our setting (<ref>) (i.e., m=1),
although ϕ_+ is not differentiable at [ϕ=0], thanks to  
ζ:U_ ad→ L^∞(Ω), the sensitivity with the weight can be denoted by ζ[ϕ_i]∂_ϕℰ((ϕ_i)_+)=-ζ[ϕ_i](β-α)χ_ϕ_i∇ u_ϕ_i·∇ v_ϕ_i formally.

Now, as in <cit.>, we characterized the level set function by a solution to the time discrete version of the following doubly nonlinear diffusion equation <cit.>:  

    ∂_tϕ^q-Δ_p ϕ=
        (β-α)χ_ϕ∇ u_ϕ·∇ v_ϕ   in Ω× (0,+∞).

If we set q∈ (0,1) such that q≈ 1 for simplicity of linearization, one has |ϕ_i+1|^q-1≈ |ϕ_i|^q-1, and then the time discrete equation of (<ref>) is described as follows:  

    |ϕ_i|^q-1ϕ_i+1-ϕ_i/τ-Δ_p ϕ_i+1=
        (β-α)χ_ϕ_i∇ u_ϕ_i·∇ v_ϕ_i   in Ω.

In particular, multiplying it by τ|ϕ_i|^1-q, we obtain (<ref>) with ζ[ϕ_i]=τ|ϕ_i|^1-q. 
In this paper, we choose p=2 since one expects that the positive parts of optimal level set functions belong to V from Remark <ref>.
Thus ϕ_i+1 satisfies

    ∫_Ω   |ϕ_i|^q-1(x)ϕ_i+1(x)-ϕ_i(x)/τφ(x) x̣
    +
    ∫_Ω∇ϕ_i+1(x)·∇φ(x) x̣
          =∫_Ω (β-α)χ_ϕ_i(x)∇ u_ϕ_i(x)·∇ v_ϕ_i(x)φ(x) x̣   for all  φ∈ V∩ L^∞(Ω).

As in Algorithm <ref>, the following algorithm is proposed:






§ NUMERICAL RESULTS FOR (<REF>)


Based on the previous sections, we shall numerically construct the material distribution of two materials with diffusion coefficients of α>0 and β>0 such that the Dirichlet energy is minimized by using FreeFEM++ <cit.> with piecewise linear Lagrange elements on a triangular mesh. 
Throughout this section, we set σ=1, d=2, Ω=(0,1)^2, α=1, β=10 and θ_0≡ϕ_0≡γ for γ∈ (0,1).



 §.§ Numerical validity

We first check the numerical validity. 
Based on Algorithms <ref> and <ref>, we set γ=0.6 and f=0.001.
As for (<ref>), the characteristic function χ_ϕ_i is treated approximately as 0.5tanh (ϕ_i/0.1)+0.5. 
Then we obtain Figures <ref> and <ref>. From Figures <ref>–<ref>, it is confirmed that Algorithm <ref> makes Ω⊂^2 that almost consists of materials with diffusion coefficients of α>0 (the blue domain) and β (the red domain). 
In particular, it is noteworthy that [ϕ_+=1]:={x∈Ωϕ_+(x)=1} and [ϕ_+=0] involve [θ=1] and [θ=0], respectively. 
Furthermore, Figure <ref> shows that the convergence value of the Dirichlet energy ℰ is monotonically decreasing with respect to >0, which means that 
the necessary condition (<ref>) is satisfied, and 
ℰ(ϕ_+) asymptotically tends to ℰ(θ) constructed by the optimized volume fraction.
As a qualitative property of (locally) optimal configurations, it is suggested that the family of optimal configurations contain two-phase configurations, and then we see that the Dirichlet energy decreases by increasing the perimeter of the interface.
This completes the confirmation of the validity of the proposed method (see also Remark <ref> below).





       


[Worst conductor] 
    The minimization problem (<ref>) corresponds to the problem for determining the so-called best two-material thermal conductor. Conversely, as for the worst case, we obtain Figure <ref>. 
    Here we set the objective functional and the diffusion coefficient as -ℰ (χ_Ω_1) and κ[χ_Ω_1]=αχ_Ω_1+(1-χ_Ω_1)β, respectively. Comparing Figure <ref> with Figure <ref>, we see that similar configurations are obtained, and moreover, it can be confirmed that the convergence values are almost equivalent in Figure <ref>. These results suggest the effectiveness of the proposed method.  






 §.§ Characteristics of nonlinear problems




This subsection focuses on how the optimized configurations vary for different heat sources to see the characteristics of nonlinear problems.
Here we set =10^-6 and γ=0.5.
In particular, we compare the cases with the thermal radiation and the Robin boundary conditions. 
Since (<ref>) with the Robin boundary condition is linear, the solution is a constant multiple of the original if the heat source is multiplied by a constant, and therefore, 
optimized configurations do not vary by multiplied by a constant of the heat source. 
This is confirmed by Figure <ref>. 
In contrast, with the radiation boundary condition, the solution is different from the original solution multiplied by a constant even if the heat source is multiplied by a constant, and therefore, we see by Figure <ref> that optimized configurations deeply depend on variations in heat sources, which implies that one of the characteristics of nonlinear problems can be obtained.
Furthermore, based on Figure <ref>, the following physical interpretation can be made:



  (i)
 Since both solutions with convection and with radiation asymptotically reach the trivial solution as |f|→ 0_+, similar optimized configurations are obtained
 in the case where the value of the heat source is small,
 and therefore, the convergence values of the objective functionals are almost the same; in other words, the contributions of convection and radiation to minimize the energy are almost the same (see (a)).

  (ii)
On the other hand, it can be confirmed that, in the process of increasing the value of the heat source, the optimized configurations with radiation asymptotically tend to be the same as those with the homogeneous Dirichlet boundary condition (see, e.g., <cit.>). 
In some cases, convection seems to contribute more to energy minimization than radiation (see (b)). 

  (iii)
Figures (c)–(d) suggest that the contribution of radiation to energy minimization increases with increasing temperature.













§ APPLICATION TO THERMAL RADIATION PROBLEMS

This section is devoted to the application of the optimal design theory and level set-based optimization algorithm to some practical engineering design problems. As the nonlinear boundary condition under consideration describes the thermal radiation, one of the most straightforward but yet important applications is the design of heat radiators. 

Here we assume that a two-phase heat conductor occupying the domain Ω is situated in a vacuum. The domain Ω is convex so that its view factor is zero, i.e., no radiating waves can hit the surface ∂Ω. Our aim is to find a piecewise-constant distribution of the coefficient κ in Ω such that it efficiently emits the thermal energy into the ambient vacuum.


As shown in Figure <ref> (a), let Ω = (-L/2,L/2)^3 be the cube with side length L>0. The cube contains a heat source f.
The boundary of the cube Ω comprises a radiative surface Γ_R and thermally insulated one ∂Ω∖Γ_R.
Then the temperature u in Ω solves

    -(κ∇ u)=f      in Ω,
            
    
            -κ∇ u·ν=σu^4       on Γ_R,
            
    
            -κ∇ u·ν=0       on ∂Ω∖Γ_R.


As in Section <ref>, we seek the distribution of diffusion coefficients (thermal conductivities) α and β such that the Dirichlet (internal) energy is minimized under the volume constraint. Throughout this section, the thermal conductivities are set as α = 15 W m^-1 K^-1 (nichrome) and β=400 W m^-1 K^-1 (copper), respectively.

Let us start with the case of Γ_R = ∂Ω, i.e., all the surfaces are radiative. In this numerical experiment, the volume constraint is set to γ = 0.15, and the heat source f is uniformly distributed in the ball of radius L/10 located at the center of Ω, i.e., f= f_0 χ_B_L/10(0) with positive constant f_0>0.

Unlike usual conductivity problems with linear boundary conditions, the constants L and f_0 may affect the optimizer of the best-conductor problem. This can be confirmed from the results in Figure <ref>, where the optimal designs are shown for two parameter pairs (L,f_0). Figure <ref> (b) and (c) show the optimized configuration of the conductivity κ and convergence history of the objective functional. From the results, we observe some clear differences between the two shapes, e.g., the number of spikes. 
    
Another numerical example is shown in Figure <ref>. As in the previous example, the fixed design domain is a cube Ω=(-L/2,L/2)^3 with the length L>0. We give a heat source f in the bottom part of Ω as f=f_0χ_Ω_source with constant f_0>0 and Ω_source = (-L/4,L/4)× (-L/4,L/4)× (-L/2,-9L/20). Unlike the previous example, the radiative surface is set to Γ_R = { x_3 = L/2 : x∈∂Ω}, i.e., only the upper surface is radiative. In terms of physics, our aim is to enhance the radiation of heat energy generated in the bulk.

As in the previous example, we consider two cases: (L,f_0) = (0.1 m, 8× 10^5 W/m^3) and (L,f_0) = (10 m, 3.3× 10^3 W/m^3). The optimized configurations and temperature fields are shown in <ref> (b) and (c) along with convergence history of the objective functional. While both results attain convergence, the obtained design and corresponding temperature fields are quite distinct. The significant difference originates from the nonlinearity in terms of u. To see this, let us consider the energies

    E_in =   ℰ(ϕ_+),
        
    
        E_rad =   σ∫_∂Ω u^5 (x) dσ,
        
    
        E_source =   ∫_Ω f(x)u(x) x̣.

From the weak form, it immediately follows the energy conservation E_in + E_rad = E_source. From a physical point of view, this ratio E_in / E_source represents the amount of internal energy stored inside the structure. We calculated the ratio E_in / E_source for the two optimized designs (Figure <ref> (b) and (c)) and obtained the values 4.11× 10^-3 and 0.179, respectively. This indicates that the usual scaling law does not hold due to the thermal radiation effect.


We finally discuss the performance of the designed heat radiators. As shown in Figure <ref>, let us consider a fin-like structure with N× N tilted pillars (cylinders) inside the cube Ω=(-L/2,L/2)^3. The conductivity takes κ=β inside the pillars and κ=α elsewhere. 
The fin-like structure is a reasonable design of a heat radiator as it conducts heat from the bottom to the top radiative surface via the highly conductive pillars. We wish to check that the optimal design is superior to this non-optimized radiator in terms of the objective functional. 


To this end, we calculate the value of the energy ∫_Ωκ(x)|∇ u(x)|^2dx for various N using the same finite element analysis with quadratic Lagrange elements on a body-fitted tetrahedral mesh. Note that the radius R of each pillar is determined such that the fin-like design satisfies the same volume constraint with γ=0.15 for fair comparison. The calculated values are shown in Table <ref>. The results indicate that the energy decreases as the number of pillars increases in both the cases of L=0.1 m and L=10 m. These values are, however, greater than the optimal values 15.1 W/m^3 and 2.63× 10^6 W/m^3 for L=0.1 m and L=10 m, respectively. These results suggest that the optimal designs yield much more efficient heat radiation than a physically reasonable but non-optimized radiator.



§ CONCLUSION

In this paper, we considered the optimal design problem for the steady-state diffusion equation with nonlinear boundary conditions described by the maximal monotone operator. 
The main target was to analyze the distribution (or shape and topology) of the two-material composite that minimizes the Dirichlet energy with thermal radiation. 
The results obtained in this paper are as follows:


  * We proved that there exists at least a pair of the optimal volume fraction and the optimal homogenized matrix for a true relaxation problem such that the value of the relaxed Dirichlet energy coincides with the minimum value of the original design problem. To this end, we also proved the existence and uniqueness of the weak solution to the state equation with nonlinear boundary conditions described by the maximal monotone operator and the corresponding homogenization theorem.

  * In order to estimate the minimum value of the original design problem, the sensitivity of the relaxed Dirichlet energy was derived rigorously, at least under the smoothness assumptions for the domain and the two-material diffusion coefficient.

  * We considered the perimeter constraint problem via the positive part of the level set function as an approximation problem for the relaxation problem and proved the existence of minimizers. In particular, it was shown that the limit of the sequence of minimizers with respect to the perturbation parameter becomes a minimizer of the restricted relaxation problem in the Sobolev space.

  * By deriving the so-called weighted sensitivity, the level set function was updated by employing the time-discrete version of the nonlinear diffusion equation, and optimized configurations with almost no intermediate sets were obtained. Furthermore, it was numerically verified that the convergence value of the Dirichlet energy is asymptotic to a minimum value if the perturbation parameter is sufficiently small.

  * As one of the characteristics of the nonlinear problem, it was confirmed that the optimized configuration deeply depends on the value of the heat source. In particular, the contribution of radiation to energy minimization seems to increase with increasing temperature.

  * Three-dimensional numerical examples were also provided. We designed the distribution of thermal conductivity such that it minimizes internal energy due to an external heat source. The performance of the designed radiators was tested via comparison with a simple fin-like structure.





§ DECLARATION OF COMPETING INTEREST

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.






§ DATA AVAILABILITY

Data will be made available on request.


99

        AAC86 N. Aguilera, H.W. Alt, L.A. Caffarelli,
        An optimization problem with volume constraint,
        SIAM J. Control Optim. 24 (1986), 191–198.

    
        A02 G. Allaire, 
        Shape optimization by the homogenization method,
        Vol. 146 of Applied Mathematical Sciences, 
        Springer-Verlag, New York (2002).

        ACMOY19 G. Allaire, L. Cavallina, N. Miyake, T. Oka, T. Yachimura, 
        The homogenization method for topology optimization of structures: old and new, 
        Interdiscip. Inform. Sci. 25 (2019), 75–146.

        AJT02 G. Allaire, F. Jouve, A.M. Toader,  
        A level-set method for shape optimization,
        C. R. Acad. Sci. Paris 334 (2002), 125–1130.

        AJT04 G. Allaire, F. Jouve, A.M. Toader,  
        Structural optimization using sensitivity analysis and a level-set method,
        J. Comp. Phys. 194 (2004), 363–393.

        AMP10 G. Allaire, A. Münch, F. Periago, 
        Long time behavior of a two-phase optimal design for the heat equation,
        SIAM J. Control Optim. 48 (2010), 5333–5356.

        AB93 L. Ambrosio, G. Buttazzo,
        An optimal design problem with perimeter penalization,
        Calc. Var. 1 (1993), 55–69.

        AA06 S. Amstutz, H. Andrä, 
        A new algorithm for topology optimization using a level-set method, 
        J. Comput. Phys. 216 (2006), 573–588.

        B10 V. Barbu,
        Nonlinear differential equations of monotone types in Banach spaces,
        Springer Monographs in Mathematics, vol. 190, Springer, New York, 2010.
        
        BC11 H.H. Bauschke, P.L. Combettes,
        Convex Analysis and Monotone Operator Theory in Hilbert Spaces, Springer, New York (2011). 
   
        BMW06 J.F. Bonder, S. Martínez, N. Wolanski, 
        An optimization problem with volume constraint for a degenerate quasilinear operator,    
        J. Differential Equations 227 (2006) 80–101. 

        BFF92 S. Brahim-Otsmane, G.A. Francfort, F. Murat, 
        Correctors for the homogenization of the wave and heat equations, 
        J. Math. Pures Appl. 71 (1992), 197–231.
  
        B71 H. Brézis, 
        Monotonicity methods in Hilbert spaces and some applications to nonlinear partial differential equations, in Contributions to Nonlinear Funct. Analysis, 
        Madison, 1971, (E. Zarantonello ed.), Acad. Press (1971), 101–156.

        B11 H. Brezis, 
        Functional Analysis. Sobolev Spaces and Partial Differential Equations, Universitext. Springer, New York, 2011.

        bruns2001topology T.E. Bruns, D.A. Tortorelli, Topology optimization of non-linear elastic structures and compliant mechanisms, Comput. Methods Appl. Mech. Engrg. 190 (2001), 3443–3459.

        chen2016topology X. Chen, Topology optimization of microfluidics — A review, Microchemical Journal 127 (2016), 52–61.
        
        D15 J. Casado-Díaz,
        Some smoothness results for the optimal design of a two-composite material which minimizes the energy,
        Calc. Var. 53 (2015), 649–673.

        DLM23 J. Casado-Díaz, M. Luna-Laynez, F. Maestre, 
        Optimal design problems through the homogenization method,
        Handbook of Numerical Analysis 24 (2023), 1–28.

        JCD22 J. Casado-Díaz, C. Conca, D. Vasquez-Varas
        
        Minimization of the p-Laplacian first eigenvalue for a two-phase material,
	J. Comput. Appl. Math. 399 (2022), 113722.

        dems2005sensitivity K. Dems, R. Korycki Sensitivity analysis and optimal design for steady conduction problem with radiative heat transfer, J. Therm. Stresses 28 (2005), 213–232.

        gilmore2002spacecraft D.G. Gilmore, Spacecraft Thermal Control Handbook: Fundamental Technologies, 1, 2nd ed., The Aerospace Press, 2002.
      
        Td2 L. He, C.Y. Kao, S. Osher, 
        Incorporating topological derivatives into shape derivatives based level set methods, 
        J. Comput. Phys. 225 (2007), 891–909.
    
        H12F. Hecht,
	    New development in FreeFem++,
	    J. Numer. Math. 20 (2012), 251–266.

        HMP21 A. Henrot, I. Mazari, Y. Privat,
        Shape optimization of a Dirichlet type energy for semilinear elliptic partial differential equations
        ESAIM: COCV 27 (2021), S6.

        jog1996distributed C. Jog, Distributed-parameter optimization and topology design for non-linear thermoelasticity, Comput. Methods Appl. Mech. Engrg. 132 (1996), 117–134.

         KS86 R.V. Kohn, G. Strang, 
        Optimal design and relaxation of variational problems, I, II, III, 
        Comm. Pure Appl. Math. 39 (1986), 113–137, 139–182, 353–377.

        korycki2006sensitivity R. Korycki, Sensitivity analysis and shape optimization for transient heat conduction with radiation, Int. J. Heat Mass Transf.
 49 (2006), 2033–2043.

     
        LV89 P.T. Landsberg, A. De Vos,
        The Stefan-Boltzmann constant in n-dimensional space,
        J. Phys. A: Math. Gen. 22  (1989), 1073–1084.

        LZ11 B.S. Lazarov, O. Sigmund,
        Filters in topology optimization based on Helmholtz-type differential equations,
        Int. J. Numer. Meth. Engng 86 (2011), 765–781.

        Lions J.-L. Lions, 
        Optimal control of systems governed by partial differential equations, 
        Translated from the French by S. K. Mitter, Die Grundlehren der mathematischen Wissenschaften, Band 170 Springer-Verlag, New York-Berlin 1971.

        liu2023adjoint M. Liu, Y. Hasegawa, Adjoint-based shape optimization for radiative transfer using level-set function and volume penalization method, Int. J. Heat Mass Transf. 210 (2023), 124158.

        maute2004conceptual K. Maute, M. Allen, Conceptual design of aeroelastic structures by topology optimization, Struct. Multidiscip. Optim. 27 (2004), 27–42.
        
	    Murat-Tartar F. Murat, L. Tartar, H-convergence, Topics in the mathematical modelling of composite materials,  Progr. Nonlinear Differential Equations Appl. 31 (1997), 21-43.
	    
	    MPP08 A. Münch, P. Pedregal, F. Periago,
	    Relaxation of an optimal design problem for the heat equation,
	    J. Math. Pures Appl. 89 (2008), 225-247.

        OS88S. Osher, J.A. Sethian,
        Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations,
        J. Comput. Phys. 79 (1988), 12–49.

        O23 T. Oka,
        Level set method via positive parts for optimal design problems of two-material thermal conductors, 
        Math. Comput. Simul.
        223 (2024), 299–306.

        OY23 T. Oka, T. Yamada,
        Topology optimization method with nonlinear diffusion, 
        Comput. Methods Appl. Mech. Engrg. 408 (2023), 115940.

        yoon2010topological G.H. Yoon, Topological design of heat dissipating structure with forced convective heat transfer, J. Mech. Sci. Tech. 24 (2010), 1225–1233.

        yuge1995optimization K. Yuge, N. Kikuchi , Optimization of a frame structure subjected to a plastic deformation, Structural optimization 10 (1995), 197–208.

        ZHW18 S. Zhu, X. Hu, Q. Wu,  A level set method for shape optimization in semilinear elliptic problems,
        J. Comp. Phys. 355 (2018), 104–120.







