title: LDAP Continuation Reference error in search results from Active Directory when using GSSAPI authentication in Java
body: <p><strong>UPDATE:</strong> Based on the comment from @Michael-O below, it seems like the correct way to handle this issue if for the LDAP JNDI provider or the SASL implementation to canonicalize the host name by doing a forward then a revers DNS lookup before issue in KRN service ticket request.  I will try to reach out to the Open JDK security list and see if any answers come back from there.</p>
  
  <p>I am trying to do a recursive LDAP search on the root DN against an Active Directory server using a session that is authenticated via GSSAPI using a <em>Subject</em> from a Kerberos LoginContext. </p>
  
  <p>I am able to successfully bind to the server with URL <code>ldap://dc1.example.com</code>.  The <em>InitidalDirContext</em> has <strong>java.naming.referral</strong> set to <code>follow</code>.</p>
  
  <p>When I execute the search <code>(&amp;(objectClass=user)(userPrincipalName=sample_user@EXAMPLE.COM))</code> against the root DN of <code>dc=example,dc=com</code>, I get one <em>SearchResult</em> back:</p>
  
  <pre class="lang-none prettyprint-override"><code>CN=Sample User,OU=ExampleUsers,DC=example,DC=com
  </code></pre>
  
  <p>And several Continuation References:</p>
  
  <pre class="lang-none prettyprint-override"><code>ldap://example.com/CN=Configuration,DC=example,DC=com
  ldap://ForestDnsZones.example.com/DC=ForestDnsZones,DC=example,DC=com
  ldap://DomainDnsZones.example.com/DC=DomainDnsZones,DC=example,DC=com
  </code></pre>
  
  <p>I can iterate over the <em>SearchResult</em> just fine, but as soon as I encounter a continuation, I get a <em>PartialResultsException</em>.  I checked DNS and all of the above host names resolve correctly. The exception I get looks like this: </p>
  
  <pre class="lang-none prettyprint-override"><code>javax.naming.PartialResultException 
    [Root exception is javax.naming.AuthenticationException: GSSAPI 
      [Root exception is javax.security.sasl.SaslException: GSS initiate failed 
        [Caused by GSSException: No valid credentials provided 
          (Mechanism level: Server not found in Kerberos database (7))]]].
  </code></pre>
  
  <p>Looking at the Kerberos trace, this error makes sense.  When trying to follow the continuation, the LDAP library attempts to bind to <code>ldap://example.com</code>.  Since we're using GSSAPI for authentication, this triggers a service ticket request for <code>ldap/example.com</code>.  The response I see in the log is:</p>
  
  <pre class="lang-none prettyprint-override"><code>&gt;&gt;&gt;KRBError:
       sTime is Thu Aug 21 14:27:20 EDT 2014 0000000000000
       suSec is 414575
       error code is 7
       error Message is Server not found in Kerberos database
       realm is EXAMPLE.COM
       sname is ldap/example.com
       msgType is 30
  </code></pre>
  
  <p>I checked Active Directory, and sure enough there isn't any <strong>servicePrincipalName</strong> attribute with value <code>ldap/example.com</code> anywhere on any of the domain controllers.  I've tried manually adding a SPN for <code>ldap/example.com</code> to the SAVANT-DC1 domain controller's machine account.  This works temporarily, but Active Directory seems to automatically purge the SPN entry after a couple of minutes.</p>
  
  <p>It seems like the solution would be to do one of</p>
  
  <ol>
  <li>Get Active Directory to return continuations that contain the name of the domain controller instead of the domain.  We know that we are able to obtain service tickets for SPNs in the form of <code>ldap/dc1.example.com</code>.</li>
  <li>Somehow map continuations on the java end of things to redirect <code>ldap://example.com</code> to <code>ldap://dc1.example.com</code></li>
  </ol>
  
  <p>I haven't bee able to figure out how to do (1).</p>
  
  <p>I tried doing (2) using the <a href="http://docs.oracle.com/javase/jndi/tutorial/ldap/referral/src/Throw.java" rel="nofollow" title="JNDI Manually Following Referrals Tutorial">JNDI Manual Referral Handling Example</a> as a guide. I switched the <strong>java.naming.referral</strong> property to <code>throw</code> and wrote a custom referral handler that manually overrides the <strong>java.naming.provider.url</strong> property in the referral context. However <code>LdapReferralException.getReferralContext()</code> seems to ignore the <strong>java.naming.provider.url</strong> environment property.  Looking at the OpenJDK code to <a href="http://www.docjar.com/html/api/com/sun/jndi/ldap/LdapReferralContext.java.html" rel="nofollow" title="LdapReferralContext.java">LdapReferralContext.java</a> seems to confirm this (line 105).</p>
  
  <p>So that's where I am: I can't intercept and manipulate the referrals on the Java side because they are treated as a black box by the JNDI API.  I can't manually create an LDAP SPN on the AD side of things because it won't stay persistent in the directory.  Is there anything else I am missing?</p>
  
  <hr>
  
  <p>Here is the code I am running</p>
  
  <pre><code>import java.io.File;
  import java.security.PrivilegedExceptionAction;
  import java.util.HashMap;
  import java.util.Map;
  import java.util.Properties;
  import javax.naming.Context;
  import javax.naming.NamingEnumeration;
  import javax.naming.ReferralException;
  import javax.naming.directory.Attributes;
  import javax.naming.directory.DirContext;
  import javax.naming.directory.InitialDirContext;
  import javax.naming.directory.SearchControls;
  import javax.naming.directory.SearchResult;
  import javax.security.auth.Subject;
  import javax.security.auth.callback.Callback;
  import javax.security.auth.callback.CallbackHandler;
  import javax.security.auth.callback.NameCallback;
  import javax.security.auth.callback.PasswordCallback;
  import javax.security.auth.login.AppConfigurationEntry;
  import javax.security.auth.login.AppConfigurationEntry.LoginModuleControlFlag;
  import javax.security.auth.login.Configuration;
  import javax.security.auth.login.LoginContext;
  
  public class LdapContinuationDemoAction implements PrivilegedExceptionAction&lt;Object&gt; {
    private final String ldapUrl;
    private final String ldapDn;
    private final String username;
  
    public static void main(String[] argv) {
      try {
        String username = "example_user@EXAMPLE.COM";
        String password = "Password1";
        String ldapUrl  = "ldap://dc1.example.com";
        String searchDn = "dc=example,dc=com";
        String pwd      = System.getProperty("user.dir");
        String krb5Conf = new File(pwd, "krb5.conf").getAbsolutePath();
  
        System.setProperty("java.security.krb5.conf", krb5Conf);
        System.setProperty("sun.security.krb5.debug", "true");
  
        // Login to the domain via Kerberos
        LoginContext loginCtx = new LoginContext("doesn't matter", null,
          getUsernamePasswordHandler(username, password),
          getKrb5Configuration());
  
        System.out.println("********************************");
        System.out.println("      KRB5 Login");
        System.out.println("********************************");
        loginCtx.login();
  
        // Execute the LDAP search as the user logged in above
        LdapContinuationDemoAction action = new LdapContinuationDemoAction(ldapUrl,
          searchDn, username);
  
        Subject.doAs(loginCtx.getSubject(), action);
      } catch( Exception e) {
        System.out.println();
        System.out.println("*** ERROR: " + e);
      }
    }
  
    private LdapContinuationDemoAction(String ldapUrl, String ldapDn,
      String username) {
      this.ldapUrl  = ldapUrl;
      this.ldapDn   = ldapDn;
      this.username = username;
    }
  
    // Perform a recursive LDAP search for a user principal and print the results
    @Override
    public Object run() throws Exception {
      System.out.println("********************************");
      System.out.println("      LDAP Login");
      System.out.println("********************************");
  
      //Setup the directory context environment
      Properties dirCtxProps = new Properties();
      dirCtxProps.put(Context.INITIAL_CONTEXT_FACTORY,      "com.sun.jndi.ldap.LdapCtxFactory");
      dirCtxProps.put(Context.PROVIDER_URL,                 this.ldapUrl);
      dirCtxProps.put(Context.SECURITY_AUTHENTICATION,      "GSSAPI");
      dirCtxProps.put("java.naming.ldap.attributes.binary", "objectSID");
      dirCtxProps.put(Context.REFERRAL,                     "follow");
  
      DirContext dirCtx = new InitialDirContext(dirCtxProps);
  
      // enable recursive searching
      SearchControls ctrls = new SearchControls();
      ctrls.setSearchScope(SearchControls.SUBTREE_SCOPE);
  
      // do the search
      NamingEnumeration&lt;SearchResult&gt; results = dirCtx.search(this.ldapDn,
        "(&amp;(objectClass=user)(userPrincipalName={0}))",
        new Object[] { this.username }, ctrls);
  
      System.out.println("********************************");
      System.out.println("      LDAP User Info");
      System.out.println("********************************");
      int resultNum = 0;
  
      while (results.hasMore()) {
        resultNum++;
  
        Attributes userAttr = results.next().getAttributes();
  
        System.out.println("ldap result " + resultNum + ": User DN: "
          + userAttr.get("distinguishedName").get());
        System.out.println();
      }
  
  
      return null;
    }
  
    // JAAS callback handler for username and password Kerberos authn
    private static CallbackHandler getUsernamePasswordHandler(
      final String username, final String password) {
  
      final CallbackHandler handler = new CallbackHandler() {
        @Override
        public void handle(final Callback[] callback) {
          for (int i = 0; i &lt; callback.length; i++) {
            if (callback[i] instanceof NameCallback) {
              final NameCallback nameCallback = (NameCallback) callback[i];
              nameCallback.setName(username);
            } else if (callback[i] instanceof PasswordCallback) {
              final PasswordCallback passCallback = (PasswordCallback) callback[i];
              passCallback.setPassword(password.toCharArray());
            } else {
              System.err.println("Unsupported Callback: "
                + callback[i].getClass().getName());
            }
          }
        }
      };
  
      return handler;
    }
  
    // dynamically build a Kerberos JAAS configuration so we don't need a login.conf
    private static Configuration getKrb5Configuration() {
      return new Configuration() {
  
        @Override
        public AppConfigurationEntry[] getAppConfigurationEntry(String name) {
          Map&lt;String, String&gt; options = new HashMap&lt;String, String&gt;();
          options.put("client", "true");
          return new AppConfigurationEntry[] {
            new AppConfigurationEntry(
              "com.sun.security.auth.module.Krb5LoginModule",
              LoginModuleControlFlag.REQUIRED, options)
          };
        }
      };
    }
  
  }
  </code></pre>
  
  <hr>
  
  <p>Here is my krb5.conf:</p>
  
  <pre><code>[libdefaults]
    default_realm = EXAMPLE.COM
  
  [realms]
    EXAMPLE.COM = {
      kdc = dc1.example.com
      default_domain = example.com
    }
  
  [domain_realm]
    .example.com = EXAMPLE.COM
    example.com = EXAMPLE.COM
  </code></pre>
  
  <hr>
  
  <p>Here is the output from the above code</p>
  
  <pre class="lang-none prettyprint-override"><code>********************************
        KRB5 Login
  ********************************
  Config name: C:\src\scratch\krb5\krb5.conf
  &gt;&gt;&gt; KdcAccessibility: reset
  Using builtin default etypes for default_tkt_enctypes
  default etypes for default_tkt_enctypes: 18 17 16 23 1 3.
  &gt;&gt;&gt; KrbAsReq creating message
  &gt;&gt;&gt; KrbKdcReq send: kdc=dc1.example.com UDP:88, timeout=30000, number of retries =3, #bytes=158
  &gt;&gt;&gt; KDCCommunication: kdc=dc1.example.com UDP:88, timeout=30000,Attempt =1, #bytes=158
  &gt;&gt;&gt; KrbKdcReq send: #bytes read=227
  &gt;&gt;&gt;Pre-Authentication Data:
       PA-DATA type = 19
       PA-ETYPE-INFO2 etype = 18, salt = EXAMPLE.COMexample_user, s2kparams = null
       PA-ETYPE-INFO2 etype = 23, salt = null, s2kparams = null
       PA-ETYPE-INFO2 etype = 3, salt = EXAMPLE.COMexample_user, s2kparams = null
  
  &gt;&gt;&gt;Pre-Authentication Data:
       PA-DATA type = 2
       PA-ENC-TIMESTAMP
  &gt;&gt;&gt;Pre-Authentication Data:
       PA-DATA type = 16
  
  &gt;&gt;&gt;Pre-Authentication Data:
       PA-DATA type = 15
  
  &gt;&gt;&gt; KdcAccessibility: remove dc1.example.com
  &gt;&gt;&gt; KDCRep: init() encoding tag is 126 req type is 11
  &gt;&gt;&gt;KRBError:
       sTime is Thu Aug 21 16:35:42 EDT 2014 0000000000000
       suSec is 659371
       error code is 25
       error Message is Additional pre-authentication required
       realm is EXAMPLE.COM
       sname is krbtgt/EXAMPLE.COM
       eData provided.
       msgType is 30
  &gt;&gt;&gt;Pre-Authentication Data:
       PA-DATA type = 19
       PA-ETYPE-INFO2 etype = 18, salt = EXAMPLE.COMexample_user, s2kparams = null
       PA-ETYPE-INFO2 etype = 23, salt = null, s2kparams = null
       PA-ETYPE-INFO2 etype = 3, salt = EXAMPLE.COMexample_user, s2kparams = null
  
  &gt;&gt;&gt;Pre-Authentication Data:
       PA-DATA type = 2
       PA-ENC-TIMESTAMP
  &gt;&gt;&gt;Pre-Authentication Data:
       PA-DATA type = 16
  
  &gt;&gt;&gt;Pre-Authentication Data:
       PA-DATA type = 15
  
  KrbAsReqBuilder: PREAUTH FAILED/REQ, re-send AS-REQ
  Using builtin default etypes for default_tkt_enctypes
  default etypes for default_tkt_enctypes: 18 17 16 23 1 3.
  Using builtin default etypes for default_tkt_enctypes
  default etypes for default_tkt_enctypes: 18 17 16 23 1 3.
  &gt;&gt;&gt; EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
  &gt;&gt;&gt; KrbAsReq creating message
  &gt;&gt;&gt; KrbKdcReq send: kdc=dc1.example.com UDP:88, timeout=30000, number of retries =3, #bytes=240
  &gt;&gt;&gt; KDCCommunication: kdc=dc1.example.com UDP:88, timeout=30000,Attempt =1, #bytes=240
  &gt;&gt;&gt; KrbKdcReq send: #bytes read=1425
  &gt;&gt;&gt; KdcAccessibility: remove dc1.example.com
  &gt;&gt;&gt; EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
  &gt;&gt;&gt; KrbAsRep cons in KrbAsReq.getReply example_user
  ********************************
        LDAP Login
  ********************************
  Found ticket for example_user@EXAMPLE.COM to go to krbtgt/EXAMPLE.COM@EXAMPLE.COM expiring on Fri Aug 22 02:35:42 EDT 2014
  Entered Krb5Context.initSecContext with state=STATE_NEW
  Found ticket for example_user@EXAMPLE.COM to go to krbtgt/EXAMPLE.COM@EXAMPLE.COM expiring on Fri Aug 22 02:35:42 EDT 2014
  Service ticket not found in the subject
  &gt;&gt;&gt; Credentials acquireServiceCreds: same realm
  Using builtin default etypes for default_tgs_enctypes
  default etypes for default_tgs_enctypes: 18 17 16 23 1 3.
  &gt;&gt;&gt; CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
  &gt;&gt;&gt; EType: sun.security.krb5.internal.crypto.ArcFourHmacEType
  &gt;&gt;&gt; KrbKdcReq send: kdc=dc1.example.com UDP:88, timeout=30000, number of retries =3, #bytes=1392
  &gt;&gt;&gt; KDCCommunication: kdc=dc1.example.com UDP:88, timeout=30000,Attempt =1, #bytes=1392
  &gt;&gt;&gt; KrbKdcReq send: #bytes read=1398
  &gt;&gt;&gt; KdcAccessibility: remove dc1.example.com
  &gt;&gt;&gt; EType: sun.security.krb5.internal.crypto.ArcFourHmacEType
  &gt;&gt;&gt; KrbApReq: APOptions are 00000000 00000000 00000000 00000000
  &gt;&gt;&gt; EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
  Krb5Context setting mySeqNumber to: 774790609
  Krb5Context setting peerSeqNumber to: 0
  Created InitSecContextToken:
  0000: 01 00 6E 00 00 00 00 00   00 00 A0 03 02 01 05 A1  ..n..)0..%......
  0010: 03 02 01 0E A2 00 00 00   00 00 00 00 00 A3 82 04  ................
  0020: 00 00 00 00 00 00 00 00   2D A0 03 02 01 05 A1 0E  5a..10..-.......
  0030: 1B 0C 55 54 42 53 41 56   2E 4C 4F 43 41 4C A2 2A  ..EXAMPLE.COM.*
  0040: 30 28 A0 03 02 01 00 A1   21 30 1F 1B 04 6C 64 61  0(......!0...lda
  0050: 70 1B 17 73 61 76 61 6E   74 2D 64 63 31 2E 75 74  p..dc1.ut
  0060: 62 73 61 76 2E 6C 6F 63   61 6C A3 82 03 E8 30 82  bsav.local....0.
  0070: 03 E4 A0 03 02 01 12 A1   03 02 01 08 A2 82 03 D6  ................
  ---8&lt;--- Snipping a bunch of binary
  
  Krb5Context.unwrap: token=[05 04 01 ff 00 0c 00 0c 00 00 00 00 2e 2e 5d d1 f5 d2 e8 21 c1 23 92 20 61 f4 77 a8 07 a0 00 00 ]
  Krb5Context.unwrap: data=[07 a0 00 00 ]
  Krb5Context.wrap: data=[01 01 00 00 ]
  Krb5Context.wrap: token=[05 04 00 ff 00 0c 00 00 00 00 00 00 2e 2e 5d d1 00 00 00 00 00 00 00 00 fa b6 79 67 ce db 58 d2 ]
  ********************************
        LDAP User Info
  ********************************
  ldap result 1: User DN: CN=Sample User,OU=ExampleUsers,DC=example,DC=com
  
  Found ticket for example_user@EXAMPLE.COM to go to krbtgt/EXAMPLE.COM@EXAMPLE.COM expiring on Fri Aug 22 02:35:42 EDT 2014
  Entered Krb5Context.initSecContext with state=STATE_NEW
  Found ticket for example_user@EXAMPLE.COM to go to krbtgt/EXAMPLE.COM@EXAMPLE.COM expiring on Fri Aug 22 02:35:42 EDT 2014
  Found ticket for example_user@EXAMPLE.COM to go to ldap/dc1.example.com@EXAMPLE.COM expiring on Fri Aug 22 02:35:42 EDT 2014
  Service ticket not found in the subject
  &gt;&gt;&gt; Credentials acquireServiceCreds: same realm
  Using builtin default etypes for default_tgs_enctypes
  default etypes for default_tgs_enctypes: 18 17 16 23 1 3.
  &gt;&gt;&gt; CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
  &gt;&gt;&gt; EType: sun.security.krb5.internal.crypto.ArcFourHmacEType
  &gt;&gt;&gt; KrbKdcReq send: kdc=dc1.example.com UDP:88, timeout=30000, number of retries =3, #bytes=1381
  &gt;&gt;&gt; KDCCommunication: kdc=dc1.example.com UDP:88, timeout=30000,Attempt =1, #bytes=1381
  &gt;&gt;&gt; KrbKdcReq send: #bytes read=94
  &gt;&gt;&gt; KdcAccessibility: remove dc1.example.com
  &gt;&gt;&gt; KDCRep: init() encoding tag is 126 req type is 13
  &gt;&gt;&gt;KRBError:
       sTime is Thu Aug 21 16:35:46 EDT 2014 0000000000000
       suSec is 918178
       error code is 7
       error Message is Server not found in Kerberos database
       realm is EXAMPLE.COM
       sname is ldap/example.com
       msgType is 30
  KrbException: Server not found in Kerberos database (7)
      at sun.security.krb5.KrbTgsRep.&lt;init&gt;(KrbTgsRep.java:70)
      at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192)
      at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203)
      at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:311)
      at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115)
      at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:442)
      at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641)
      at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
      at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
      at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
      at com.sun.jndi.ldap.sasl.LdapSasl.saslBind(LdapSasl.java:123)
      at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:232)
      at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740)
      at com.sun.jndi.ldap.LdapCtx.&lt;init&gt;(LdapCtx.java:316)
      at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193)
      at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:152)
      at com.sun.jndi.url.ldap.ldapURLContextFactory.getObjectInstance(ldapURLContextFactory.java:52)
      at javax.naming.spi.NamingManager.getURLObject(NamingManager.java:601)
      at javax.naming.spi.NamingManager.processURL(NamingManager.java:381)
      at javax.naming.spi.NamingManager.processURLAddrs(NamingManager.java:361)
      at javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:333)
      at com.sun.jndi.ldap.LdapReferralContext.&lt;init&gt;(LdapReferralContext.java:111)
      at com.sun.jndi.ldap.LdapReferralException.getReferralContext(LdapReferralException.java:150)
      at com.sun.jndi.ldap.LdapNamingEnumeration.hasMoreReferrals(LdapNamingEnumeration.java:357)
      at com.sun.jndi.ldap.LdapNamingEnumeration.hasMoreImpl(LdapNamingEnumeration.java:226)
      at com.sun.jndi.ldap.LdapNamingEnumeration.hasMore(LdapNamingEnumeration.java:189)
      at LdapContinuationDemoAction.run(LdapContinuationDemoAction.java:123)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at LdapContinuationDemoAction.main(LdapContinuationDemoAction.java:52)
  Caused by: KrbException: Identifier doesn't match expected value (906)
      at sun.security.krb5.internal.KDCRep.init(KDCRep.java:143)
      at sun.security.krb5.internal.TGSRep.init(TGSRep.java:66)
      at sun.security.krb5.internal.TGSRep.&lt;init&gt;(TGSRep.java:61)
      at sun.security.krb5.KrbTgsRep.&lt;init&gt;(KrbTgsRep.java:55)
      ... 29 more
  
  *** ERROR: java.security.PrivilegedActionException: javax.naming.PartialResultException [Root exception is javax.naming.AuthenticationException: GSSAPI [Root exception is javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7))]]]
  </code></pre>