title: Equivalent to StringTokenizer with multiple characters delimiters
body: <p>I try to split a String into tokens.  </p>
  
  <p>The token delimiters are not single characters, some delimiters are included into others (example, &amp; and &amp;&amp;), and I need to have the delimiters returned as token.<br>
  StringTokenizer is not able to deal with multiple characters delimiters. I presume it's possible with String.split, but fail to guess the magical regular expression that will suits my needs.  </p>
  
  <p>Any idea ?</p>
  
  <p>Example:  </p>
  
  <pre><code>Token delimiters: "&amp;", "&amp;&amp;", "=", "=&gt;", " "  
  String to tokenize: a &amp; b&amp;&amp;c=&gt;d  
  Expected result: an string array containing "a", " ", "&amp;", " ", "b", "&amp;&amp;", "c", "=&gt;", "d"
  </code></pre>
  
  <p>--- Edit ---<br>
  Thanks to all for your help, Dasblinkenlight gives me the solution. Here is the "ready to use" code I wrote with his help:  </p>
  
  <pre><code>private static String[] wonderfulTokenizer(String string, String[] delimiters) {
    // First, create a regular expression that matches the union of the delimiters
    // Be aware that, in case of delimiters containing others (example &amp;&amp; and &amp;),
    // the longer may be before the shorter (&amp;&amp; should be before &amp;) or the regexpr
    // parser will recognize &amp;&amp; as two &amp;.
    Arrays.sort(delimiters, new Comparator&lt;String&gt;() {
      @Override
      public int compare(String o1, String o2) {
        return -o1.compareTo(o2);
       }
    });
    // Build a string that will contain the regular expression
    StringBuilder regexpr = new StringBuilder();
    regexpr.append('(');
    for (String delim : delimiters) { // For each delimiter
      if (regexpr.length() != 1) regexpr.append('|'); // Add union separator if needed
      for (int i = 0; i &lt; delim.length(); i++) {
        // Add an escape character if the character is a regexp reserved char
        regexpr.append('\\');
        regexpr.append(delim.charAt(i));
      }
    }
    regexpr.append(')'); // Close the union
    Pattern p = Pattern.compile(regexpr.toString());
  
    // Now, search for the tokens
    List&lt;String&gt; res = new ArrayList&lt;String&gt;();
    Matcher m = p.matcher(string);
    int pos = 0;
    while (m.find()) { // While there's a delimiter in the string
      if (pos != m.start()) {
        // If there's something between the current and the previous delimiter
        // Add it to the tokens list
        res.add(string.substring(pos, m.start()));
      }
      res.add(m.group()); // add the delimiter
      pos = m.end(); // Remember end of delimiter
    }
    if (pos != string.length()) {
      // If it remains some characters in the string after last delimiter
      // Add this to the token list
      res.add(string.substring(pos));
    }
    // Return the result
    return res.toArray(new String[res.size()]);
  }
  </code></pre>
  
  <p>It could be optimize if you have many strings to tokenize by creating the Pattern only one time.</p>