title: Emr 4.2 Spot Prices on Core Nodes
body: <p>Now that EMR supports downsizing of Core nodes on EMR, if I create an EMR cluster with 1 of the core nodes as a spot instance. What happens when the spot price exceeds the bid price for my core node? Will it gracefully decomission that core node?</p>
  
  <p>Here is Amazon's description of the process of shrinking the number of core nodes:</p>
  
  <blockquote>
    <p>On core nodes, both YARN NodeManager and HDFS DataNode daemons must be
    decommissioned in order for the instance group to shrink. For YARN,
    graceful shrink ensures that a node marked for decommissioning is only
    transitioned to the DECOMMISIONED state if there are no pending or
    incomplete containers or applications. The decommissioning finishes
    immediately if there are no running containers on the node at the
    beginning of decommissioning.</p>
    
    <p>For HDFS, graceful shrink ensures that the target capacity of HDFS is
    large enough to fit all existing blocks. If the target capacity is not
    large enough, only a partial amount of core instances are
    decommissioned such that the remaining nodes can handle the current
    data residing in HDFS. You should ensure additional HDFS capacity to
    allow further decommissioning. You should also try to minimize write
    I/O before attempting to shrink instance groups as that may delay the
    completion of the resize operation.</p>
    
    <p>Another limit is the default replication factor, dfs.replication
    inside /etc/hadoop/conf/hdfs-site. Amazon EMR configures the value
    based on the number of instances in the cluster: 1 with 1-3 instances,
    2 for clusters with 4-9 instances, and 3 for clusters with 10+
    instances. Graceful shrink does not allow you to shrink core nodes
    below the HDFS replication factor; this is to prevent HDFS from being
    unable to close files due insufficient replicas. To circumvent this
    limit, you must lower the replication factor and restart the NameNode
    daemon.</p>
  </blockquote>