our work and the existing methods see Table \[fig:fiveplots\]. In none of the existing methods,]{}]{} the finite sum form for the lower level problem is [[considered]{}]{}. The sum structure is very rampant in practice when we have separate objective functions related to different agents in a distributed setting. This is the case for example in machine learning for very large datasets [@Bottou05], where each $f_i$ represents an agent that is cooperating with others. When the complete information of all the agents, i.e. summation of all (sub)gradients is not available, these agents can be treated distinctly. Due to its wide range of applications in distributed optimization, finite sum problem has been extensively studied. Among popular methods are *incremental (sub)gradient* (IG) [@Bertsekas99; @Nedich01] and *incremental aggregated (sub)gradients* (IAG) [@Blatt07; @Tseng14] for deterministic and *stochastic average gradient* (SAG) [@Roux12], SAGA [@Defazio14] and MISO methods [@Mairal13] for stochastic regimes. These algorithms have faster convergence and are computationally efficient in large-scale optimization since a very less amount of memory is required at each step in order to store only one agent’s information and subsequently update the iterate based on that [@Ozdaglar17]. Despite the widespread use of these first-order methods, they do not address [[the]{}]{} bilevel 