<p>This is a well-known problem known as "finding extremal sets"; unfortunately, there is nothing fundamentally faster known than the obvious approach of testing a newly inserted set against all existing sets, but good heuristic improvements exist. Here is a recent paper discussing this problem: <a href="https://arxiv.org/abs/1508.01753" rel="nofollow noreferrer">https://arxiv.org/abs/1508.01753</a></p>

<p>An open-source implementation of a related algorithm: 
<a href="https://code.google.com/archive/p/google-extremal-sets/" rel="nofollow noreferrer">https://code.google.com/archive/p/google-extremal-sets/</a></p>
Data structure for subset-reduced growing list
<p>I'm working on a problem which involves going through a lot of data. To reduce the work (because current calculations take about two weeks of compute time, and I'd like to reduce that dramatically) I came up with an algorithm which would be much faster if it was able to avoid a certain type of duplication. (The current algorithm avoids storing this information because it is too large, unreduced, to fit in memory.)</p>

<p>I have a collection of sets, and I don't want to insert a set <code>A</code> if there is already a set <code>B</code> which is a subset of <code>A</code>. At the moment the sets are represented by integers where individual binary digits represent a particular element being present or absent. In that interpretation the set/integer <code>A</code> should not be inserted if there is already a set/integer <code>B</code> such that <code>(~A) &amp; B</code> is 0, where <code>~</code> is bitwise negation and <code>&amp;</code> is bitwise AND.</p>

<p>For example, if my collection has the following sets</p>

<pre><code>[ {a,b}, {b,c}, {b,d,e} ]
</code></pre>

<p>and I asked to add {b,c,e} it should not be added (since {b,c} is already there) and similarly with {a,b} (since {a,b} is there) but {a,e} should be added.</p>

<p>The numeric equivalent would be starting with</p>

<pre><code>[ `0b11`, `0b110`, `0b11010` ]
</code></pre>

<p>where <code>0b10110</code> is not added since <code>(~0b10110) &amp; 0b110 == 0</code>, <code>0b11</code> is not added since <code>(~0b11) ^ 0b11 == 0</code>, but <code>0b10001</code> can be added.</p>

<p><em>Ideally</em> the structure would prune itself as new sets are added, so if <code>{c}</code> were added all existing sets containing <code>c</code> would be removed. But it's acceptable if it doesn't update in that way as long as I can normalize it to that form in some not-too-expensive way every so often.</p>