title: Android reading and graphing data from text file using GraphView
body: <p>In my android application, I am attempting to read from a text file containing data points which I wish to graph. The number of data points in each file is expected to be anywhere between 24,000 to 150,000.</p>
  
  <p>Currently, I have implemented an asynchronous task that reads the data from a compressed text file and stores it in a local vector. On the completion of the task, the data points are added to a data set in order to be graphed. </p>
  
  <p>The code implementation for the asynchronous task is as follows:</p>
  
  <pre><code>class ReadFileService extends AsyncTask&lt;Void, String, Boolean&gt; {
  
      @Override
      protected Boolean doInBackground(Void... args) {        
          Scanner strings = null;
          InputStream stream = null;
          ZipInputStream zipInput = null;
          try {
              System.out.println(externalStorageDirectory + Constants.APP_DIRECTORY + recordingName + Constants.ZIP_FILE_EXTENTION);
              File file = new File(externalStorageDirectory + Constants.APP_DIRECTORY, recordingName + Constants.ZIP_FILE_EXTENTION);
              ZipFile zipFile = new ZipFile(file);
              Enumeration&lt;? extends ZipEntry&gt; entries = zipFile.entries();
  
              while (entries.hasMoreElements()) {
                  ZipEntry zipEntry = entries.nextElement();
                  stream = zipFile.getInputStream(zipEntry);
                  strings = new Scanner(stream);
  
                  // Extract the value of sampling frequency from header
                  System.out.println("Extracting value of sampling frequency.");
                  String regexPattern = "\"ColumnLabels\"";
                  strings.useDelimiter(regexPattern);
                  String extracted = strings.next();
                  Pattern pattern = Pattern.compile("\"SamplingFrequency\": \"(\\d+)\"");
                  Matcher matcher = pattern.matcher(extracted);
                  if (matcher.find()) {
                      samplingFrequency = Integer.parseInt(matcher.group(1));
                      System.out.println(samplingFrequency);
                  }
  
                  // Locate the end of the header and use tabs as the delimiter
                  strings.findWithinHorizon(endOfHeader,0);           
                  strings.useDelimiter("\t *");
                  strings.next();
              }
          }
          catch (FileNotFoundException error) {
              System.out.println("@IOERROR: " + error);
              return false;
          }
          catch (IOException error) {
              System.out.println("@IOERROR: " + error);
              return false;
          }
          while (strings.hasNext())
          {
              // Get raw value of data point
              double dataPoint = Double.parseDouble(strings.next());
              // Convert data point into the proper scale
              dataSet.add(SensorDataConverter.scaleEMG(dataPoint));
              // Skip the index value if it exists and break from loop if it does not
              if (strings.hasNext())
                  strings.next();
              else
                  break;
          }
          System.out.println("Closing strings.");
          try {
              stream.close();
          } catch (IOException e) {
              // TODO Auto-generated catch block
              e.printStackTrace();
          }
          strings.close();            
          return true;
      }
  
      protected void onProgressUpdate(String...progress) {
          //called when the background task makes any progress
      }
  
      protected void onPreExecute() {
          //called before doInBackground() is started
          super.onPreExecute();
          // Show Progress Bar Dialog before calling doInBackground method
          prgDialog.setTitle("Opening File");
          prgDialog.setMessage("Opening " + recordingName + "\nPlease wait...");
          prgDialog.show();
      }
  
      //called after doInBackground() has finished
      protected void onPostExecute(Boolean readFileSuccess) { 
          // If unable to read from file, print error and generate a random set of sample data
          if(!readFileSuccess) {
              Random randomGenerator = new Random();          
              System.out.println("@IOERROR: Unable to read from file. Creating random dataset");
              for(int i=0; i&lt;100; i++)
              {
                  dataSet.add(randomGenerator.nextDouble());
              }
          }
          // Create a set of graph data to use with GraphView
          exampleSeries1 = new GraphViewSeries(new GraphViewData[] {
          });
          for (int i=0; i&lt;dataSet.size(); i++) {
              double pointX = i;
              double pointY = dataSet.get(i);
              exampleSeries1.appendData(new GraphViewData(pointX, pointY), true, dataSet.size());
          }
          // Plot the data points
          graphData();
          prgDialog.dismiss();
          prgDialog = null;
      }
  }
  </code></pre>
  
  <p>This appears to work when saving and loading approximately 10,000 data points and takes approximately 15 seconds to load. However, when the number of data points jumps up to 20,000, it takes upwards to 45 seconds to load. </p>
  
  <p>My main question is: <strong>Is there another way of more efficiently handling all this data in order to minimize the length of time required to load data?</strong> For example, would it be possible to dynamically graph data points depending on the view port range?</p>
  
  <p>Any help and suggestions would be much appreciated.</p>