title: Filter values not equal in pyspark
body: <p>I have a table like the following:</p>
  <pre><code>+---+----+----+
  | id|Sell| Buy|
  +---+----+----+
  |  A|null|null|
  |  B|   Y|   Y|
  |  C|null|   Y|
  |  D|   Y|null|
  |  E|null|null|
  +---+----+----+
  </code></pre>
  <p>I can easilty filter single or both columns when value equals &quot;Y&quot;. For example, the following filters when both columns are &quot;Y&quot;:</p>
  <pre><code>df.filter((df[&quot;Buy&quot;] == &quot;Y&quot;) &amp; (df[&quot;Sell&quot;] == &quot;Y&quot;))
  </code></pre>
  <p>However, how can I filter for when a single column or both columns do not equal &quot;Y&quot;?? What is the code for each case? I have tried these codes, and they return no rows:</p>
  <pre><code>df.filter((df[&quot;Buy&quot;] != &quot;Y&quot;) &amp; (df[&quot;Sell&quot;] != &quot;Y&quot;))
  df.filter(~((df[&quot;Buy&quot;] == &quot;Y&quot;) | (df[&quot;Sell&quot;] == &quot;Y&quot;)))
  </code></pre>
  <p>It seems it does not capture null values</p>