public static synchronized void resetOrUpdateFromCache(KryoSerialization instance, KryoInstantiator kryoInst){
      if(kryoInst != cachedKryoInst) {
        cachedPool = KryoPool.withByteArrayOutputStream(MAX_CACHED_KRYO, kryoInst);
        cachedKryoInst = kryoInst;
      }
      instance.kryoPool = cachedPool;
    }
public static synchronized void resetOrUpdateFromCache ( KryoSerialization instance , KryoInstantiator kryoInst ) { if ( kryoInst != cachedKryoInst ) { cachedPool = KryoPool . withByteArrayOutputStream ( MAX_CACHED_KRYO , kryoInst ) ; cachedKryoInst = kryoInst ; } instance . kryoPool = cachedPool ; }
Hadoop will re-initialize the KryoSerialization on every spill
This gets very expensive if you output a lot from a mapper to initialize the chill/kryo stack
The KryoInstantiator's already do some caching, and figuring out if its safe to cache,
so here we piggy back on that to avoid generating new kryo's or kryo pools