hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieInputFormatUtils.java
/**
   * Checks the file status for a race condition which can set the file size to 0. 1. HiveInputFormat does
   * super.listStatus() and gets back a FileStatus[] 2. Then it creates the HoodieTableMetaClient for the paths listed.
   * 3. Generation of splits looks at FileStatus size to create splits, which skips this file
   * @param conf
   * @param dataFile
   * @return
   */
Checks the file status for a race condition which can set the file size to 0. 1. HiveInputFormat does
super.listStatus() and gets back a FileStatus[] 2. Then it creates the HoodieTableMetaClient for the paths listed.
3. Generation of splits looks at FileStatus size to create splits, which skips this file
@param conf
@param dataFile
@return
Checks the file status for a race condition which can set the file size to 0 . 1 . HiveInputFormat does super . listStatus () and gets back a FileStatus [] 2 . Then it creates the HoodieTableMetaClient for the paths listed . 3 . Generation of splits looks at FileStatus size to create splits which skips this file @param conf @param dataFile @return
private static HoodieBaseFile refreshFileStatus(Configuration conf, HoodieBaseFile dataFile) {
    Path dataPath = dataFile.getFileStatus().getPath();
    try {
      if (dataFile.getFileSize() == 0) {
        FileSystem fs = dataPath.getFileSystem(conf);
        LOG.info("Refreshing file status " + dataFile.getPath());
        return new HoodieBaseFile(fs.getFileStatus(dataPath), dataFile.getBootstrapBaseFile().orElse(null));
      }
      return dataFile;
    } catch (IOException e) {
      throw new HoodieIOException("Could not get FileStatus on path " + dataPath);
    }
  }
private static HoodieBaseFile refreshFileStatus ( Configuration conf , HoodieBaseFile dataFile ) { Path dataPath = dataFile . getFileStatus ( ) . getPath ( ) ; try { if ( dataFile . getFileSize ( ) == 0 ) { FileSystem fs = dataPath . getFileSystem ( conf ) ; LOG . info ( "Refreshing file status " + dataFile . getPath ( ) ) ; return new HoodieBaseFile ( fs . getFileStatus ( dataPath ) , dataFile . getBootstrapBaseFile ( ) . orElse ( null ) ) ; } return dataFile ; } catch ( IOException e ) { throw new HoodieIOException ( "Could not get FileStatus on path " + dataPath ) ; } }
Checks the file status for a race condition which can set the file size to 0.
Checks the file status for a race condition which can set the file size to 0 .