title: Indexoutofboundsexception after scrolling up while swipe refresh is refreshing
body: <p><img src="https://i.stack.imgur.com/PtpcX.png" alt="swipe to refresh refreshing bug">
  Hello! I've got a recyclerview and a swipe to refresh layout . When i swipe down and it starts refreshing when i scroll up while it's still refreshing the app crashes and the log says the thing in the picture. After refrwshing is completed everything works fine until you scroll up while refreshing again . It says indexoutof bounds and position 2. Thanks in advance! :) </p>
  
  <p>Edit 1: </p>
  
  <pre><code>&lt;?xml version="1.0" encoding="utf-8"?&gt;
  &lt;android.support.v4.widget.SwipeRefreshLayout
      android:id="@+id/swipe_refresh"
      android:layout_width="match_parent"
      app:layout_behavior="@string/appbar_scrolling_view_behavior"
      android:layout_height="wrap_content"
      android:background="@color/colorPrimaryNewsBG"
      xmlns:android="http://schemas.android.com/apk/res/android"
      xmlns:app="http://schemas.android.com/apk/res-auto"&gt;
  
  &lt;android.support.v7.widget.RecyclerView
      android:id="@+id/recyclerview"
      android:clipChildren="false"
      android:clipToPadding="false"
      android:layout_width="match_parent"
      android:layout_height="wrap_content"/&gt;
  
  &lt;/android.support.v4.widget.SwipeRefreshLayout&gt;
  </code></pre>