title: How to create a lightweight C code sandbox?
body: <p>I'd like to build a C pre-processor / compiler that allows functions to be collected from local and online sources. ie:</p>
  
  <pre><code>#fetch MP3FileBuilder http://scripts.com/MP3Builder.gz
  #fetch IpodDeviceReader http://apple.com/modules/MP3Builder.gz
  
  void mymodule_main() {
    MP3FileBuilder(&amp;some_data);
  }
  </code></pre>
  
  <p>That's the easy part. </p>
  
  <p>The hard part is <strong>I need a reliable way to "sandbox" the imported code from direct or unrestricted access to disk or system resources (including memory allocation and the stack)</strong>. I want a way to safely run <strong>small snippets of untrusted C code</strong> (modules) without the overhead of putting them in separate process, VM or interpreter (a separate thread would be acceptable though).</p>
  
  <p><strong>REQUIREMENTS</strong></p>
  
  <ul>
  <li>I'd need to put quotas on its access to data and resources including CPU time. </li>
  <li>I will block direct access to the standard libraries</li>
  <li>I want to stop malicious code that creates endless recursion</li>
  <li>I want to limit static and dynamic allocation to specific limits</li>
  <li>I want to catch all exceptions the module may raise (like divide by 0).</li>
  <li>Modules may only interact with other modules via core interfaces</li>
  <li>Modules may only interact with the system (I/O etc..) via core interfaces</li>
  <li>Modules must allow bit ops, maths, arrays, enums, loops and branching.</li>
  <li>Modules cannot use ASM</li>
  <li>I want to limit pointer and array access to memory reserved for the module (via a custom safe_malloc())</li>
  <li>Must support ANSI C or a subset (see below)</li>
  <li>The system must be lightweight and cross-platform (including embedded systems). </li>
  <li>The system must be GPL or LGPL compatible.</li>
  </ul>
  
  <p>I'm happy to settle for a subset of C. I don't need things like templates or classes. I'm primarily interested in the things high-level languages don't do well like fast maths, bit operations, and the searching and processing of binary data.</p>
  
  <p>It is <strong>not</strong> the intention that existing C code can be reused without modification to create a module. The intention is that modules would be required to conform to a set of rules and limitations designed to limit the module to basic logic and transformation operations (like a video transcode or compression operations for example).</p>
  
  <p>The theoretical input to such a compiler/pre-processor would be a single ANSI C file (or safe subset) with a module_main function, NO includes or pre-processor directives, no ASM, It would allow loops, branching, function calls, pointer maths (restricted to a range allocated to the module), bit-shifting, bitfields, casts, enums, arrays, ints, floats, strings and maths. Anything else is optional.</p>
  
  <p><strong>EXAMPLE IMPLEMENTATION</strong></p>
  
  <p>Here's a pseudo-code snippet to explain this better. Here a module exceeds it's memory allocation quota and also creates infinite recursion.</p>
  
  <pre><code>buffer* transcodeToAVI_main( &amp;in_buffer ) {
      int buffer[1000000000]; // allocation exceeding quota
      while(true) {} // infinite loop
      return buffer;
  }
  </code></pre>
  
  <p>Here's a transformed version where our preprocessor has added watchpoints to check for memory usage and recursion and wrapped the whole thing in an exception handler.</p>
  
  <pre><code>buffer* transcodeToAVI_main( &amp;in_buffer ) {
      try {
          core_funcStart(__FILE__,__FUNC__); // tell core we're executing this function
          buffer = core_newArray(1000000000, __FILE__, __FUNC__); // memory allocation from quota
          while(true) {
             core_checkLoop(__FILE__, __FUNC__, __LINE__) &amp;&amp; break; // break loop on recursion limit
          } 
          core_moduleEnd(__FILE__,__FUNC__);
      } catch {
          core_exceptionHandler(__FILE__, __FUNC__);
      }
      return buffer;
  }
  </code></pre>
  
  <p>I realise performing these checks impact the module performance but I suspect it will still outperform high-level or VM languages for the tasks it is intended to solve. I'm not trying to stop modules doing dangerous things outright, I'm just trying to force those dangerous things to happen in a controlled way (like via user feedback). ie: "Module X has exceeded it's memory allocation, continue or abort?".</p>
  
  <p><strong>UPDATE</strong></p>
  
  <p>The best I've got so far is to use a custom compiler (Like a hacked TCC) with bounds checking and some custom function and looping code to catch recursions. I'd still like to hear thoughts on what else I need to check for or what solutions are out there. I imagine that removing ASM and checking pointers before use solves a lot of the concerns expressed in previous answers below. I added a bounty to pry some more feedback out of the SO community. </p>
  
  <p><strong>For the bounty I'm looking for:</strong></p>
  
  <ul>
  <li>Details of potential exploits against the theoretical system defined above</li>
  <li>Possible optimisations over checking pointers on each access</li>
  <li>Experimental open-source implementations of the concepts (like Google Native Client)</li>
  <li>Solutions that support a wide range of OS and devices (no OS/hardware based solutions)</li>
  <li>Solutions that support the most C operations, or even C++ (if that's possible)</li>
  </ul>
  
  <p>Extra credit for a method that can work with GCC (ie, a pre-processor or <em>small</em> GCC patch).</p>
  
  <p>I'll also give consideration to anyone who can conclusively prove what I'm attempting cannot be done at all. You will need to be pretty convincing though because none of the objections so far have really nailed the technical aspects of why they think it's impossible. In the defence of those who said no this question was originally posed as a way to safely run C++. I have now scaled back the requirement to a limited subset of C.</p>
  
  <p>My understanding of C could be classed as "intermediate", my understanding of PC hardware is maybe a step below "advanced". Try to coach your answers for that level if you can. Since I'm no C expert I'll be going largely based on votes given to an answer as well as how closely the answer comes to my requirements. You can assist by providing sufficient evidence for your claims (respondents) and by voting (everyone else). I'll assign an answer once the bounty countdown reaches 6 hours.</p>
  
  <p>Finally, I believe solving this problem would be a major step towards maintaining C's relevance in an increasingly networked and paranoid world. As other languages close the gap performance-wise and computing power grows it will be harder and harder to justify the added risk of C development (as it is now with ASM). I believe your answers will have a much greater relevance than scoring a few SO points so please contribute what you can, even if the bounty has expired.</p>