title: Is there a faster alternative to ioutil.ReadFile?
body: <p>I am trying to make a program for checking file duplicates based on md5 checksum.
  Not really sure whether I am missing something or not, but this function reading the XCode installer app (it has like 8GB) uses 16GB of Ram</p>
  
  <pre><code>func search() {
      unique := make(map[string]string)
      files, err := ioutil.ReadDir(".")
      if err != nil {
          log.Println(err)
      }
  
      for _, file := range files {
          fileName := file.Name()
          fmt.Println("CHECKING:", fileName)
          fi, err := os.Stat(fileName)
          if err != nil {
              fmt.Println(err)
              continue
          }
          if fi.Mode().IsRegular() {
              data, err := ioutil.ReadFile(fileName)
              if err != nil {
                  fmt.Println(err)
                  continue
              }
              sum := md5.Sum(data)
              hexDigest := hex.EncodeToString(sum[:])
              if _, ok := unique[hexDigest]; ok == false {
                  unique[hexDigest] = fileName
              } else {
                  fmt.Println("DUPLICATE:", fileName)
              }
          }
      }
  }
  </code></pre>
  
  <p>As per my debugging the issue is with the file reading
  Is there a better approach to do that?
  thanks</p>