---
author:
- 'Zhanfeng Wang Yuan-chin Ivan Chang[^1]'
title: Distributed sequential method for analyzing massive data 
---

**Abstract**: To analyse a very large data set containing lengthy variables, we adopt a sequential estimation idea and propose a parallel divide-and-conquer method. We conduct several conventional sequential estimation procedures separately, and properly integrate their results while maintaining the desired statistical properties. Additionally, using a criterion from the statistical experiment design, we adopt an adaptive sample selection, together with an adaptive shrinkage estimation method, to simultaneously accelerate the estimation procedure and identify the effective variables. We confirm the cogency of our methods through theoretical justifications and numerical results derived from synthesized data sets. We then apply the proposed method to three real data sets, including those pertaining to appliance energy use and particulate matter concentration.

[**Keywords**]{}: Sequential sampling; Stopping rule; Confidence set; Distributed/Parallel computation

[[**[AMS Subject Classification (2000)]{}**]{}: Primary 62F12; Secondary 62E20]{}

Introduction
============

While the development of modern measurement and communication technologies has frequently made data collection procedures more efficient, we as researchers have been hard-pressed to analyse and extract information from large data sets and to keep up with our data collection capacity. Although we can leverage concepts from the divide-and-conquer 