{"action":"created","issue":{"url":"https://api.github.com/repos/pingidentity/ldapsdk/issues/47","repository_url":"https://api.github.com/repos/pingidentity/ldapsdk","labels_url":"https://api.github.com/repos/pingidentity/ldapsdk/issues/47/labels{/name}","comments_url":"https://api.github.com/repos/pingidentity/ldapsdk/issues/47/comments","events_url":"https://api.github.com/repos/pingidentity/ldapsdk/issues/47/events","html_url":"https://github.com/pingidentity/ldapsdk/issues/47","id":343112524,"node_id":"MDU6SXNzdWUzNDMxMTI1MjQ=","number":47,"title":"SocketException(Socket closed) when adding a very big entry","user":{"login":"fanf","id":44649,"node_id":"MDQ6VXNlcjQ0NjQ5","avatar_url":"https://avatars1.githubusercontent.com/u/44649?v=4","gravatar_id":"","url":"https://api.github.com/users/fanf","html_url":"https://github.com/fanf","followers_url":"https://api.github.com/users/fanf/followers","following_url":"https://api.github.com/users/fanf/following{/other_user}","gists_url":"https://api.github.com/users/fanf/gists{/gist_id}","starred_url":"https://api.github.com/users/fanf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fanf/subscriptions","organizations_url":"https://api.github.com/users/fanf/orgs","repos_url":"https://api.github.com/users/fanf/repos","events_url":"https://api.github.com/users/fanf/events{/privacy}","received_events_url":"https://api.github.com/users/fanf/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-07-20T13:42:45Z","updated_at":"2018-07-20T16:07:14Z","closed_at":null,"author_association":"NONE","body":"Hello, \r\n\r\nWhen I try to add a very big entry (big == (nunber of values for an attribute x size of values) goes above some threshold), I get the following exception: \r\n\r\n```Exception in thread \"main\" LDAPException(resultCode=81 (server down), errorMessage='An error occurred while attempting to send the LDAP message to server localhost:1389:  SocketException(Socket closed), ldapSDKVersion=4.0.6, revision=27850')\r\n\tat com.unboundid.ldap.sdk.LDAPConnectionInternals.sendMessage(LDAPConnectionInternals.java:611)\r\n\tat com.unboundid.ldap.sdk.LDAPConnection.sendMessage(LDAPConnection.java:4406)\r\n\tat com.unboundid.ldap.sdk.AddRequest.processAsync(AddRequest.java:1121)\r\n\tat com.unboundid.ldap.sdk.AddRequest.process(AddRequest.java:1025)\r\n\tat com.unboundid.ldap.sdk.LDAPConnection.add(LDAPConnection.java:2078)\r\n       ......\r\nCaused by: java.net.SocketException: Socket closed\r\n\tat java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118)\r\n\tat java.net.SocketOutputStream.write(SocketOutputStream.java:155)\r\n\tat java.io.BufferedOutputStream.write(BufferedOutputStream.java:122)\r\n\tat com.unboundid.util.ByteStringBuffer.write(ByteStringBuffer.java:1662)\r\n\tat com.unboundid.asn1.ASN1Buffer.writeTo(ASN1Buffer.java:1012)\r\n\tat com.unboundid.ldap.sdk.LDAPConnectionInternals.sendMessage(LDAPConnectionInternals.java:562)\r\n\t... 12 more\r\n```\r\n\r\nSo it seems that somewhere inside unboundid, there is something that is limited by Integer.maxvalue. \r\n\r\nYou can reproduce it with the following code (it's scala, but should really be understandable anyhonw: add a sufficient number of sufficiently big values => you reach the point where the error happens). \r\n\r\n```\r\n  val provider = new RWPooledSimpleAuthConnectionProvider(\r\n      host = \"localhost\",\r\n      port = 1389,\r\n      authDn = \"cn=Manager,cn=rudder-configuration\",\r\n      authPw = \"secret\",\r\n      poolSize = 1\r\n  )\r\n  def str(size: Int) = new String( Array.fill[Char](size)('x'))\r\n\r\n  // run\r\n  def main(args: Array[String]): Unit = {\r\n    val dn = new DN(\"cn=Nodes Configuration,ou=Rudder,cn=rudder-configuration\")\r\n    // here, if I use 454 bytes for the last value, I get the exception. Below that, OK\r\n    val values = for(i <- 0 until 167) yield i + \"-\"+str(100000) ++ str(454)\r\n\r\n\r\n  val addRequest = new AddRequest(\r\n        dn\r\n      , new Attribute(\"objectClass\", \"top\", \"nodeConfigurations\")\r\n      , new Attribute(\"cn\", \"Nodes Configuration\")\r\n      , new Attribute(\"nodeConfig\", values:_*) // here, we add 168 big value and one small\r\n    )\r\n\r\n    val x = for {\r\n      ldap <- provider\r\n      _ =  println(\"**** got connection\")\r\n      _ <- ldap.delete(dn) // that's just to clean up when testing multiple time\r\n      _ =  println(\"**** deleted\")\r\n      _ =  ldap.backed.add(addRequest) // KABOUM\r\n      _ =  println(\"**** added\")\r\n    } yield {\r\n      \"done\"\r\n    }\r\n    println(x)\r\n  }\r\n```\r\n\r\nI tried to understand both the code and the debug output, but failed for each one. \r\n\r\nI understand that the use case is a little bit extreme (even if encoutered in Rudder https://www.rudder-project.org/redmine/issues/10646#note-24), but I'm wondering if there is a workaround ?\r\n\r\nThanks\r\n"},"comment":{"url":"https://api.github.com/repos/pingidentity/ldapsdk/issues/comments/406647090","html_url":"https://github.com/pingidentity/ldapsdk/issues/47#issuecomment-406647090","issue_url":"https://api.github.com/repos/pingidentity/ldapsdk/issues/47","id":406647090,"node_id":"MDEyOklzc3VlQ29tbWVudDQwNjY0NzA5MA==","user":{"login":"dirmgr","id":12702251,"node_id":"MDQ6VXNlcjEyNzAyMjUx","avatar_url":"https://avatars0.githubusercontent.com/u/12702251?v=4","gravatar_id":"","url":"https://api.github.com/users/dirmgr","html_url":"https://github.com/dirmgr","followers_url":"https://api.github.com/users/dirmgr/followers","following_url":"https://api.github.com/users/dirmgr/following{/other_user}","gists_url":"https://api.github.com/users/dirmgr/gists{/gist_id}","starred_url":"https://api.github.com/users/dirmgr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dirmgr/subscriptions","organizations_url":"https://api.github.com/users/dirmgr/orgs","repos_url":"https://api.github.com/users/dirmgr/repos","events_url":"https://api.github.com/users/dirmgr/events{/privacy}","received_events_url":"https://api.github.com/users/dirmgr/received_events","type":"User","site_admin":false},"created_at":"2018-07-20T16:07:14Z","updated_at":"2018-07-20T16:07:14Z","author_association":"COLLABORATOR","body":"This is almost certainly a case of the directory server closing the connection rather than a problem with the LDAP SDK itself. Most directory servers impose a limit on the size of a request that the client can send, although that limit is usually configurable.\r\n\r\nImposing a limit on the maximum size of an LDAP request is an important safety feature because of the way that LDAP works at the protocol level. LDAP requests are encoded using the ASN.1 Basic Encoding Rules (BER), as described at [https://ldap.com/ldapv3-wire-protocol-reference-asn1-ber/](https://ldap.com/ldapv3-wire-protocol-reference-asn1-ber/). A BER element contains three components:\r\n\r\n* A byte (or multiple bytes in rare cases that you’re not likely to encounter in LDAP) that provides information about the type of data held in that element.\r\n* One or more bytes that specify the number of bytes in the element’s value.\r\n* An encoded representation of the element value.\r\n\r\nEach LDAP message is encoded as a BER sequence, which is a kind of container that basically holds an array of other elements (the message ID, the body of the request or response, and an optional set of controls). But ultimately, each LDAP request and response is encapsulated in a BER element that has all of the other elements inside it.\r\n\r\nWhen the server receives a request from the client, it first reads the the type and length of the BER element that holds the LDAP message, and then it needs to allocate enough memory to hold the value of that element. In this case, the client is sending a big request, and the server is refusing to allocate that much memory. It’s probably a relatively small amount of memory (maybe just a couple of megabytes), but it’s a safety mechanism nonetheless because it prevents a malicious client from establishing a connection and sending the start of a BER message that says the element value is something big, like a gigabyte, and causing the server to allocate that much memory. If the client establishes a lot of connections that all do that, it can cause the server to run out of memory and crash or start swapping or exhibit some other kind of erratic behavior.\r\n\r\nThe directory server’s only real safeguard against this type of attack is to terminate any connection on which a client indicates that it wants to send a big request. It could send a notice of disconnection unsolicited notification before it does that to explain why it’s closing the connection, but not all servers do that, and you would have also needed to register an unsolicited notification handler in the LDAP SDK to be able to get that notification anyway.\r\n\r\nBut the good news is that the maximum request size limit is probably configurable. Most servers have a relatively low limit (I think it’s usually in the 1–5 megabyte range) since LDAP requests are typically pretty small, and about the only time you really run into this issue is when you’re trying to add a really big entry. But you can probably update the configuration to raise the limit. In the Ping Identity Directory Server, there’s a max-request-size property in the LDAP connection handler configuration. In directory servers derived from the Netscape Directory Server codebase (including the Fedora 389 Directory Server, Red Hat Directory Server, and Oracle DSEE), I think that the configuration attribute is called nsslapd-maxbersize. I’m not as familiar with the configuration for other types of servers like OpenLDAP or Active Directory, but there’s probably some way to raise the limit. I’d recommend checking the documentation or using the support channels for the server that you’re using to see what the options are for the server that you’re using.\r\n"}}
{"actor":{"display_login":"dirmgr"}}