title: A distinct list based on split values
body: <p>My end goal is to have a unique list of ID's I can iterate through. Here goes:</p>
  
  <p>I have an XML of products (Items). In the complete XML there will be +200,000 items. In this example there is two:</p>
  
  <pre><code>&lt;?xml version="1.0" encoding="utf-8"?&gt;
  &lt;Export Shop="Demo Webshop" Type="Full" Clean="true" CleanIsolationShopID="SHOP1"&gt;
  &lt;Items&gt;
      &lt;Item ItemNo="1001" ShopID="SHOP1" VariantCode="1616_42.1615_01.ct_HD"&gt;
      &lt;/Item&gt;
      &lt;Item ItemNo="1001" ShopID="SHOP1" VariantCode="1616_42.1615_02.ct_HD" &gt;
      &lt;/Item&gt;
  &lt;/Items&gt;
  </code></pre>
  
  <p></p>
  
  <p>The content of attribute VariantCode I need to split. For the first Item that should give me 1616_42 and 1615_01 and ct_HD. The end result is to import it to a table with the composite primary key ItemNo+VariantOption (VariantOption being the split value).</p>
  
  <p>The XSLT furthermore has:</p>
  
  <pre><code>&lt;table tableName="EcomVariantOptionsProductRelation"&gt;
  &lt;xsl:for-each select="Export/Items/Item"&gt;
      &lt;xsl:call-template name="split"&gt;
      &lt;xsl:with-param name="pText" select="@VariantCode"/&gt;
      &lt;xsl:with-param name="ProductID" select="concat(@ItemNo,'@@',@ShopID)"/&gt;
      /xsl:call-template&gt;
  &lt;/xsl:for-each&gt;
  </code></pre>
  
  <p></p>
  
  <p>The template being called that performs the actual split:</p>
  
  <pre><code>    &lt;xsl:template match="text()" name="split"&gt;
      &lt;xsl:param name="pText" select="."/&gt;
      &lt;xsl:param name= "ProductID" select="." /&gt;
      &lt;xsl:choose&gt;
          &lt;xsl:when test="string-length($pText) &gt; 0"&gt;
              &lt;xsl:choose&gt;
                  &lt;xsl:when test="contains($pText, '.')"&gt;
                      &lt;!-- has dot (more than one variantOption) --&gt;
                      &lt;item tableName="EcomVariantOptionsProductRelation"&gt;
                          &lt;column columnName="VariantOptionsProductRelationVariantID"&gt;
                              &lt;xsl:value-of select="substring-before($pText,'.')"/&gt;
                          &lt;/column&gt;
                          &lt;column columnName="VariantOptionsProductRelationProductID"&gt;
                              &lt;xsl:value-of select="$ProductID"/&gt;
                          &lt;/column&gt;
                      &lt;/item&gt;
                  &lt;/xsl:when&gt;
                  &lt;xsl:otherwise&gt;
                      &lt;item tableName="EcomVariantOptionsProductRelation"&gt;
                          &lt;column columnName="VariantOptionsProductRelationVariantID"&gt;
                              &lt;xsl:value-of select="$pText"/&gt;
                          &lt;/column&gt;
                          &lt;column columnName="VariantOptionsProductRelationProductID"&gt;
                              &lt;xsl:value-of select="$ProductID"/&gt;
                          &lt;/column&gt;
                      &lt;/item&gt;
                  &lt;/xsl:otherwise&gt;
              &lt;/xsl:choose&gt;
              &lt;xsl:call-template name="split"&gt;
                  &lt;xsl:with-param name="pText" select="substring-after($pText, '.')"/&gt;
                      &lt;xsl:with-param name="ProductID" select="$ProductID"/&gt;
              &lt;/xsl:call-template&gt;
          &lt;/xsl:when&gt;
          &lt;xsl:otherwise&gt;
              &lt;!-- empty string (no variants) --&gt;
              &lt;xsl:value-of select="$pText"/&gt;
          &lt;/xsl:otherwise&gt;
      &lt;/xsl:choose&gt;
  &lt;/xsl:template&gt;
  </code></pre>
  
  <p>The problem is that the transformed output, ie</p>
  
  <pre><code>        &lt;item tableName="EcomVariantOptionsProductRelation"&gt;
          &lt;column columnName="VariantOptionsProductRelationVariantID"&gt;&lt;![CDATA[1616_42]]&gt;&lt;/column&gt;
          &lt;column columnName="VariantOptionsProductRelationProductID"&gt;&lt;![CDATA[1001@@SHOP1]]&gt;&lt;/column&gt;
      &lt;/item&gt;
  </code></pre>
  
  <p>is repeated, because the "1616_42" (and "ct_HD" also) part exists twice in two different items. And I need for the output to be unique, since it finally goes to a table where this composite key (VariantID+ProductID) is unique.</p>
  
  <p>The desired result for the two  should be:</p>
  
  <pre><code>    &lt;table tableName="EcomVariantOptionsProductRelation"&gt;
      &lt;item tableName="EcomVariantOptionsProductRelation"&gt;
          &lt;column columnName="VariantOptionsProductRelationVariantID"&gt;&lt;![CDATA[1616_42]]&gt;&lt;/column&gt;
          &lt;column columnName="VariantOptionsProductRelationProductID"&gt;&lt;![CDATA[1001@@SHOP1]]&gt;&lt;/column&gt;
      &lt;/item&gt;
      &lt;item tableName="EcomVariantOptionsProductRelation"&gt;
          &lt;column columnName="VariantOptionsProductRelationVariantID"&gt;&lt;![CDATA[1615_01]]&gt;&lt;/column&gt;
          &lt;column columnName="VariantOptionsProductRelationProductID"&gt;&lt;![CDATA[1001@@SHOP1]]&gt;&lt;/column&gt;
      &lt;/item&gt;
      &lt;item tableName="EcomVariantOptionsProductRelation"&gt;
          &lt;column columnName="VariantOptionsProductRelationVariantID"&gt;&lt;![CDATA[ct_HD]]&gt;&lt;/column&gt;
          &lt;column columnName="VariantOptionsProductRelationProductID"&gt;&lt;![CDATA[1001@@SHOP1]]&gt;&lt;/column&gt;
      &lt;/item&gt;
      &lt;item tableName="EcomVariantOptionsProductRelation"&gt;
          &lt;column columnName="VariantOptionsProductRelationVariantID"&gt;&lt;![CDATA[1615_02]]&gt;&lt;/column&gt;
          &lt;column columnName="VariantOptionsProductRelationProductID"&gt;&lt;![CDATA[1001@@SHOP1]]&gt;&lt;/column&gt;
      &lt;/item&gt;
      &lt;item tableName="EcomVariantOptionsProductRelation"&gt;
          &lt;column columnName="VariantOptionsProductRelationVariantID"&gt;&lt;![CDATA[1616_50]]&gt;&lt;/column&gt;
          &lt;column columnName="VariantOptionsProductRelationProductID"&gt;&lt;![CDATA[1001@@SHOP1]]&gt;&lt;/column&gt;
      &lt;/item&gt;
      &lt;item tableName="EcomVariantOptionsProductRelation"&gt;
          &lt;column columnName="VariantOptionsProductRelationVariantID"&gt;&lt;![CDATA[ct_NHD]]&gt;&lt;/column&gt;
          &lt;column columnName="VariantOptionsProductRelationProductID"&gt;&lt;![CDATA[1001@@SHOP1]]&gt;&lt;/column&gt;
      &lt;/item&gt;
  &lt;/table&gt;
  </code></pre>
  
  <p>Point being: no duplicates.</p>
  
  <p>Searching the web I can see the possibility of creating lists with some kind of unique identifier. But I have no clue if it is possible in my scenario, and even if it is, no clue as how to implement.</p>
  
  <p>Ideas? XSLT 1.0 is used.</p>