title: Java ClassLoading from client's memory
body: <p>I am developing a client-server application using Java <code>Socket</code>s where at some point the client has to download some classes from the server (through the already bound <code>Socket</code>) and use them. I am able to read the class file bytes and send them to the client. So then, the client has to use a <code>ClassLoader</code> to load the classes from memory. So, in my opinion, the problem is not really related to <code>Socket</code>s, but it is about custom class loading (you will see why I say that, later in this post).</p>
  <p>My setup is as follows: the client project with a single package named <code>client</code> with a single class in it, named <code>LiveClientTest</code> and then the server project with a single package named <code>server</code> with 3 classes in it: <code>ClientMain</code>, <code>LiveServerTest</code> (the entry point) and <code>MyStrings</code>. In short, all server-side code is under the package <code>server</code> and all client-side under the <code>client</code>. Each of the two packages is in a different project also.</p>
  <p>The problem occurs when the custom <code>ClassLoader</code> of the client (named <code>LiveClientTest.MemoryClassLoader</code>) tries to load a non-static nested class (named <code>MyStrings.NestedNonStaticSubclass</code>) which refers to its enclosing class (named <code>MyStrings</code>) before constructing the object (of type <code>MyStrings.NestedNonStaticSubclass</code>). Although the classes compile fine, the error appears at runtime, while loading the class <code>MyStrings.NestedNonStaticSubclass</code> with the custom <code>ClassLoader</code>. I know it sounds weird, but you can see what I mean if you take a look at the code.</p>
  <h2>Server side code:</h2>
  <p>LiveServerTest class (entry point):</p>
  <pre><code>package server;
  
  import java.io.DataOutputStream;
  import java.io.File;
  import java.io.IOException;
  import java.io.OutputStream;
  import java.net.ServerSocket;
  import java.net.Socket;
  import java.net.URISyntaxException;
  import java.nio.file.Files;
  import java.nio.file.Path;
  import java.util.Objects;
  import javax.swing.JOptionPane;
  
  public class LiveServerTest {
      
      //Convert class binary name representation to file path:
      public static String classNameToResourcePath(final String className) {
          return className.replace('.', File.separatorChar) + &quot;.class&quot;;
      }
      
      //Get directory location of the given class (packages not included):
      public static Path getDirectoyPath(final Class clazz) throws URISyntaxException {
          return new File(clazz.getProtectionDomain().getCodeSource().getLocation().toURI()).toPath();
      }
      
      //Get absolute file location of the given class:
      public static Path getClassFilePath(final Class c) throws URISyntaxException {
          return getDirectoyPath(c).resolve(classNameToResourcePath(c.getName()));
      }
      
      public static void main(final String[] args) {
          ServerSocket srv = null;
          try {
              srv = new ServerSocket(Integer.parseInt(Objects.requireNonNull(JOptionPane.showInputDialog(null, &quot;Enter host port number:&quot;))));
              System.out.println(&quot;Waiting for client connection...&quot;);
              try (final Socket sck = srv.accept();
                   final OutputStream os = sck.getOutputStream()) {
                  srv.close();
                  srv = null;
                  
                  //These are the classes we need the client to load:
                  final Class[] clientClasses = new Class[] {
                      ClientMain.class,
                      MyStrings.class,
                      MyStrings.NestedStatic.class,
                      MyStrings.NestedNonStaticSubclass.class
                  };
                  
                  System.out.println(&quot;Sending all client classes' bytes to client...&quot;);
                  final DataOutputStream dos = new DataOutputStream(os);
                  dos.writeInt(clientClasses.length);
                  for (final Class clazz: clientClasses) {
                      final byte[] contents = Files.readAllBytes(getClassFilePath(clazz));
                      dos.writeUTF(clazz.getName());
                      dos.writeInt(contents.length);
                      dos.write(contents);
                  }
                  
                  System.out.println(&quot;Main application now starts...&quot;);
                  //Here would go the server side code for the client-server application.
              }
          }
          catch (final IOException | URISyntaxException | RuntimeException x) {
              JOptionPane.showMessageDialog(null, x.toString());
          }
          finally {
              System.out.println(&quot;Done.&quot;);
              try { if (srv != null) srv.close(); } catch (final IOException iox) {}
          }
      }
  }
  </code></pre>
  <p>ClientMain class:</p>
  <pre><code>package server;
  
  import java.io.DataInputStream;
  import java.net.Socket;
  
  public class ClientMain {
      //This method is called by the client to start the application:
      public static void main(final Socket sck,
                              final DataInputStream dis,
                              final ClassLoader loader,
                              final String[] args) {
          System.out.println(&quot;Running...&quot;);
          //Here would go the client side code for the client-server application.
          
          //Just test that all the classes are loaded successfully:
          System.out.println(new MyStrings(&quot;A&quot;, &quot;B&quot;, &quot;C&quot;).new NestedNonStaticSubclass().getFunction().apply(2)); //Should print &quot;C&quot;.
      }
  }
  </code></pre>
  <p>MyStrings class:</p>
  <pre><code>package server;
  
  import java.util.function.IntFunction;
  
  public class MyStrings {
      
      public static class NestedStatic {
          private final IntFunction&lt;String&gt; f;
          
          public NestedStatic(final IntFunction&lt;String&gt; f) {
              this.f = f;
          }
          
          public IntFunction&lt;String&gt; getFunction() {
              return f;
          }
      }
      
      //This class produces the error when loaded:
      public class NestedNonStaticSubclass extends NestedStatic {
          public NestedNonStaticSubclass() {
              super(i -&gt; getString(i)); //Here we refer to MyStrings object before constructing the NestedNonStaticSubclass object.
          }
      }
      
      private final String[] array;
      
      public MyStrings(final String... array) {
          this.array = array.clone();
      }
      
      public String getString(final int i) {
          return array[i];
      }
  }
  </code></pre>
  <h2>Client side code:</h2>
  <pre><code>package client;
  
  import java.io.DataInputStream;
  import java.io.IOException;
  import java.io.InputStream;
  import java.lang.reflect.InvocationTargetException;
  import java.net.Socket;
  import java.util.ArrayList;
  import java.util.HashMap;
  import java.util.Iterator;
  import java.util.Objects;
  import javax.swing.JOptionPane;
  
  public class LiveClientTest {
      
      public static class MemoryClassLoader extends ClassLoader {
          
          //The key is the name of the class, and value is the compiled class file bytes:
          private final HashMap&lt;String, byte[]&gt; classByteCode = new HashMap&lt;&gt;();
  
          @Override
          public /*synchronized*/ Class findClass(final String name) throws ClassNotFoundException {
              try {
                  return super.findClass(name);
              }
              catch (final ClassNotFoundException cnfx) {
                  if (!classByteCode.containsKey(name))
                      throw new ClassNotFoundException(name);
                  final byte[] byteCode = classByteCode.get(name);
                  return defineClass(name, byteCode, 0, byteCode.length);
              }
          }
  
          //Try to load all classes that are downloaded (with readClass):
          ArrayList&lt;String&gt; loadClasses() {
              final ArrayList&lt;String&gt; classNames = new ArrayList&lt;&gt;(classByteCode.keySet());
              int oldSize;
              do {
                  oldSize = classNames.size();
                  final Iterator&lt;String&gt; classNamesIter = classNames.iterator();
                  while (classNamesIter.hasNext()) {
                      final String className = classNamesIter.next();
                      try {
                          loadClass(className);
                          classNamesIter.remove();
                      }
                      catch (final ClassNotFoundException x) {
                      }
                  }
              }
              while (classNames.size() &lt; oldSize); //If we reached a point where there are some classes that can not be loaded, then quit...
              return classNames; //Return the classes that failed to be loaded (if any) (should be empty).
          }
          
          //Read class bytes from supplied DataInputStream:
          void readClass(final DataInputStream dis) throws IOException {
              final String name = dis.readUTF();
              final byte[] contents = new byte[dis.readInt()];
              int i = 0, n = dis.read(contents);
  
              //Make sure all 'contents.length' multitude of bytes are read:
              while (n &gt;= 0 &amp;&amp; (i + n) &lt; contents.length) {
                  i += n;
                  n = dis.read(contents, i, contents.length - i);
              }
              if (/*n &lt; 0 ||*/ (i + n) != contents.length)
                  throw new IOException(&quot;Unfinished class input (&quot; + name + &quot; - &quot; + contents.length + &quot;).&quot;);
  
              classByteCode.put(name, contents);
          }
      }
      
      public static void main(final String[] args) {
          try {
              final String host = Objects.requireNonNull(JOptionPane.showInputDialog(null, &quot;Enter host name or address:&quot;));
              final int port = Integer.parseInt(Objects.requireNonNull(JOptionPane.showInputDialog(null, &quot;Enter host port number:&quot;)));
              try (final Socket sck = new Socket(host, port);
                   final InputStream is = sck.getInputStream()) {
                  final DataInputStream dis = new DataInputStream(is);
                  final MemoryClassLoader loader = new MemoryClassLoader();
                  
                  //Download all classes and put them into the class loader:
                  System.out.println(&quot;Downloading...&quot;);
                  for (int i = dis.readInt(); i &gt; 0; --i)
                      loader.readClass(dis);
                  
                  //Load all downloaded classes from the class loader:
                  System.out.println(&quot;Loading...&quot;);
                  System.out.println(&quot;Failed to load: &quot; + loader.loadClasses() + '.');
  
                  //Call main method in main downloaded class:
                  loader
                  .loadClass(&quot;server.ClientMain&quot;) //package is from server side.
                  .getDeclaredMethod(&quot;main&quot;, Socket.class, DataInputStream.class, ClassLoader.class, String[].class)
                  .invoke(null, sck, dis, loader, args);
              }
          }
          catch (final IOException | ClassNotFoundException | NoSuchMethodException | IllegalAccessException | InvocationTargetException | RuntimeException x) {
              x.printStackTrace();
              JOptionPane.showMessageDialog(null, x);
          }
      }
  }
  </code></pre>
  <h2>Client side output:</h2>
  <pre><code>Downloading...
  Loading...
  Exception in thread &quot;main&quot; java.lang.ClassFormatError: Illegal field name &quot;server.MyStrings$this&quot; in class server/MyStrings$NestedNonStaticSubclass
      at java.lang.ClassLoader.defineClass1(Native Method)
      at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
      at java.lang.ClassLoader.defineClass(ClassLoader.java:642)
      at client.LiveClientTest$MemoryClassLoader.findClass(LiveClientTest.java:30)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      at client.LiveClientTest$MemoryClassLoader.loadClasses(LiveClientTest.java:44)
      at client.LiveClientTest.main(LiveClientTest.java:89)
  </code></pre>
  <h2>So my question is:</h2>
  <p>Why does the code fail with a <code>ClassFormatError</code>, what does that mean, and how to avoid it in this particular scenario?</p>
  <h2>My question is not:</h2>
  <p>What alternatives exist? (such as using a <code>URLClassLoader</code>, or alternative ways of class loading from memory other than a custom <code>ClassLoader</code>, etc...)</p>
  <h2>How to reproduce:</h2>
  <ol>
  <li>I am using JRE 1.8.0_251 (and I would like this to work for 1.8), so <strong>I think you must</strong> put the source files in different projects (one for the client and one for the server) in order to make sure that the client does not already have direct visibility of the server's classes while <em>class-loading</em> them.</li>
  <li>Run the server's main class <code>server.LiveServerTest</code> and input a port number for the server in the dialog that pops up. Then, run the client's main class <code>client.LiveClientTest</code> and enter <em>localhost</em> for the host (first dialog that pops up) and then the port number of the server (second dialog that pops up).</li>
  <li>The stack trace will be in your CLI (through <code>System.err</code>) and not in a GUI.</li>
  <li>The code will not work if you build the projects into jar files, but it is written to work for plain class files into directories, for simplicity. For example, in NetBeans, don't build into jar files, but rather click <em>Run File</em> for each entry point.</li>
  <li>The built classes should have their file's <code>.class</code> extention in lower case.</li>
  </ol>
  <h2>Warning:</h2>
  <p>If you put the source files in different packages but in the same project, or even worse in the same package, then the class loading might succeed without errors because:</p>
  <ol>
  <li>I am using the default constructor of the <code>ClassLoader</code> class in my <code>LiveClientTest.MemoryClassLoader</code> class, which means the parent class loader is the system class loader.</li>
  <li>The <code>LiveClientTest.MemoryClassLoader.findClass</code> method first searches the parent <code>ClassLoader</code> and then, if that fails, it searches the downloaded classes. To my knowledge, this is the suggested way of implementing this, mainly because the <code>ClassLoader</code> class (which is the parent class of my <code>LiveClientTest.MemoryClassLoader</code> class) caches already defined classes.</li>
  </ol>
  <h2>References:</h2>
  <ol>
  <li><a href="https://stackoverflow.com/questions/60764/how-to-load-jar-files-dynamically-at-runtime">How to load JAR files dynamically at Runtime?</a></li>
  <li><a href="https://stackoverflow.com/questions/19594929/in-which-scenarios-the-remote-class-loading-are-needed">In which scenarios the remote class loading are needed?</a></li>
  <li><a href="https://stackoverflow.com/questions/24538509/does-the-java-classloader-load-inner-classes">Does the Java ClassLoader load inner classes?</a></li>
  <li><a href="https://stackoverflow.com/questions/2548384/java-get-a-list-of-all-classes-loaded-in-the-jvm">Java - Get a list of all Classes loaded in the JVM</a></li>
  <li><a href="https://stackoverflow.com/questions/2868337/java-how-to-load-a-class-and-its-inner-classes-that-is-already-on-the-class-p">Java: How to load a class (and its inner classes) that is already on the class path?</a></li>
  <li><a href="https://stackoverflow.com/questions/9819318/create-new-classloader-to-reload-class">Create new ClassLoader to reload Class</a></li>
  <li><a href="https://stackoverflow.com/questions/41963273/how-to-use-classloader-to-load-class-file-from-server-to-client">How to use classloader to load class file from server to client</a></li>
  <li><a href="https://stackoverflow.com/questions/5436410/custom-java-classloader-and-inner-classes">Custom Java classloader and inner classes</a></li>
  <li><a href="https://stackoverflow.com/questions/24580370/classformaterror-in-java-8">ClassFormatError in java 8?</a></li>
  <li><a href="https://stackoverflow.com/questions/30017525/jvm-invalid-nested-class-name">JVM Invalid Nested Class Name?</a></li>
  <li><a href="https://bugs.openjdk.java.net/browse/JDK-8145051" rel="nofollow noreferrer">https://bugs.openjdk.java.net/browse/JDK-8145051</a></li>
  <li><a href="https://www.programming-books.io/essential/java/implementing-a-custom-classloader-0f0fe95cf7224c668e631a671eef3b94" rel="nofollow noreferrer">https://www.programming-books.io/essential/java/implementing-a-custom-classloader-0f0fe95cf7224c668e631a671eef3b94</a></li>
  <li><a href="https://www.baeldung.com/java-classloaders" rel="nofollow noreferrer">https://www.baeldung.com/java-classloaders</a></li>
  <li><a href="https://www.infoworld.com/article/2077260/learn-java-the-basics-of-java-class-loaders.html" rel="nofollow noreferrer">https://www.infoworld.com/article/2077260/learn-java-the-basics-of-java-class-loaders.html</a></li>
  <li><a href="https://www.oracle.com/technical-resources/articles/javase/classloaders.html" rel="nofollow noreferrer">https://www.oracle.com/technical-resources/articles/javase/classloaders.html</a></li>
  <li><a href="https://www.ibm.com/developerworks/library/j-onejar/index.html" rel="nofollow noreferrer">https://www.ibm.com/developerworks/library/j-onejar/index.html</a></li>
  </ol>
  <p>I am new to class loading, so please don't take my words for granted.</p>
  <p>Huge post, because of the divided code of the MRE. Sorry. I tried to make the code as minimal as possible.</p>