title: Cassandra has multiple rows with the same parition and clustering key
body: <p>There is a Cassandra cluster of 5 nodes. Recently there was done an update  from 2.2.7 to 3.9 version, node-by-node. The update was done according to Datastax described process: <a href="http://docs.datastax.com/en/latest-upgrade/upgrade/cassandra/upgrdCassandraDetails.html" rel="nofollow noreferrer">upgrade instruction</a>. Everything went smoothly. The whole process took ~1hour.
  However, after couple of hours I found the following problem:
  for the update period some data is inconsistent, i.e. for specific partition key and clustering key one and only one row should be returned. But sometimes one, sometimes two are returned - for the same query.</p>
  
  <p>The table has following <code>PRIMARY KEY: ((id, year), date time)</code>. So, query</p>
  
  <pre><code>SELECT * FROM table_name 
    WHERE id=1 and year=2017 and datetime='2017-01-01T01:01:01:000Z';
  </code></pre>
  
  <p>Sometimes returns one, sometimes two rows.</p>
  
  <p>Moreover, those rows are different, only primary key's fields are the same.</p>
  
  <p>Result example:</p>
  
  <pre><code>id | year | datetime                | field1 | field2 | field3 | field4
  ---+------+-------------------------+--------+--------+--------+--------
  1  | 2017 | 2017-01-01 01:01:01.000 |   null |   null |      5 |      6
  1  | 2017 | 2017-01-01 01:01:01.000 |      3 |      4 |   null |   null
  </code></pre>
  
  <p>So, as you see 1st row has fields: field3 and field4 not null while the 2nd has field1 and field2 not null. The same pattern is for the rest corrupted data.</p>
  
  <p>Additionally, once I try to delete these rows, only the first disappears and the second keeps. I'm sure that field 'datetime' is the same, this is not milliseconds question, because <strong>blobAsBigint(timestampAsBlob(datetime))</strong> returns the same value for both rows.</p>
  
  <p>What was done:</p>
  
  <p><strong>nodetool upgradesstables my_keyspace</strong> (on every node one by one)</p>
  
  <p><strong>nodetool repair my_keyspace</strong> (on every node one by one)</p>
  
  <p>upgradesstable was required because initially repair returned with 'Validation failed' error:
  <a href="https://support.datastax.com/hc/en-us/articles/205256895--Validation-failed-when-running-a-nodetool-repair" rel="nofollow noreferrer">https://support.datastax.com/hc/en-us/articles/205256895--Validation-failed-when-running-a-nodetool-repair</a></p>
  
  <pre><code>CREATE TABLE my_keyspace.my_table (
      id bigint,
      year int,
      datetime timestamp,
      field1 int,
      field2 int,
      field3 set&lt;bigint&gt;,
      field4 boolean,
      field5 map&lt;int, text&gt;,
      field6 timestamp,
      field7 decimal,
      field8 decimal,
      PRIMARY KEY ((id, year), datetime)
  ) WITH CLUSTERING ORDER BY (datetime ASC)
      AND bloom_filter_fp_chance = 0.01
      AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
      AND comment = ''
      AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
      AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
      AND crc_check_chance = 1.0
      AND dclocal_read_repair_chance = 0.1
      AND default_time_to_live = 0
      AND gc_grace_seconds = 864000
      AND max_index_interval = 2048
      AND memtable_flush_period_in_ms = 0
      AND min_index_interval = 128
      AND read_repair_chance = 0.0
      AND speculative_retry = '99PERCENTILE';
  </code></pre>