Richard Kallos: Posts tagged 'debugging' Richard Kallos: Posts tagged 'debugging' Sun, 22 Apr 2018 19:00:00 UT Sun, 22 Apr 2018 19:00:00 UT 1800 Erlang Debug VMs: What, Why and How urn:https-www-rkallos-com:-blog-2018-04-22-erlang-debug-vms-what-why-and-how Sun, 22 Apr 2018 19:00:00 UT Richard Kallos <p>Erlang can load and run C libraries. This can be useful if you want to speed up parts of your code, or if you want to interact with some OS-specific features. These special C libraries are written specially to work with the Erlang VM, and can&rsquo;t easily be run in isolation without extra code. This makes these libraries difficult to debug. In this post, I introduce Erlang debug VMs, explain how to build them, and show why they can be tremendously useful for debugging linked-in drivers and NIFs.</p> <!-- more--> <p>Once you load and run arbitrary C code, the Erlang VM becomes very easy to kill.</p> <pre><code>$ erl Erlang/OTP 19 [erts-8.2.2] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] Eshell V8.2.2 (abort with ^G) 1&gt; test_nif:do_something_cool(). Segmentation fault $ :(</code></pre> <!-- TODO: Shrink font, or remove line numbers--> <p>There are so many great tools for debugging Erlang code (<a href="">dbg</a>, <a href="">redbug</a>, <a href="">erlyberly</a>, <a href="">recon</a>) as well as excellent tools for debugging C code (<a href="">gdb</a>, <a href="">valgind</a>, <a href="">afl-fuzz</a>). However, there is a bit of a shortage of good tooling for debugging C code that gets executed by the BEAM. There&rsquo;s <a href="">niffy</a>, which is a neat tool that acts like a reasonable subset of the Erlang VM, letting you load and call NIFs. Niffy and your NIF can then be run through something like afl-fuzz or your own test suite to try and uncover bugs. As far as I am aware, no such tool exists for Port Drivers.</p> <h1 id="enter-debug-vms">Enter Debug VMs</h1> <p>I&rsquo;ve gotten into the habit of building Erlang/OTP from source. This affords me more flexibility than using a package manager. One particularly nice thing about building Erlang from source is that you can compile <em>debug</em> VMs, modified BEAM binaries intended to be run inside of tools like GDB and Valgrind. This lets us debug any C code we&rsquo;ve written that gets executed by BEAM. In addition, there is also a bonus set of GDB scripts called <a href="">ETP</a> which can pretty-print Erlang terms from inside of gdb! Debug VM binaries and ETP make the ordeal of digging through core dumps of crashed BEAM processes much easier, as you can poke around various parts of memory, and print the Erlang terms they correspond to.</p> <h1 id="how-i-build-my-debug-vms">How I Build My Debug VMs</h1> <p>If you want to build some debug VMs for yourself, the first thing you need to do is clone the Erlang/OTP git repository. (For managing software compiled from source, I <em>highly</em> recommend <a href="">GNU Stow</a>).</p> <p>Here&rsquo;s the general workflow of building and installing Erlang from source, using the git repository. Please note that these commands are somewhat symbolic; if you copy and paste them into your terminal, they may not work.</p> <ol> <li>Clone <a href=""></a></li> <li><code>export ERL_TOP="$(pwd)/otp"; cd $ERL_TOP</code></li> <li><code>git checkout OTP-X.Y.Z</code></li> <li><code>$PAGER HOWTO/</code></li> <li><code>./otp_build autoconf</code></li> <li><code>./configure --help | $PAGER</code></li> <li>Figure out which options I want from step 6.</li> <li><code>./configure --prefix=/usr/local/stow/erlang-X.Y.Z --and-other-opts</code></li> <li><code>make &amp;&amp; make docs</code></li></ol> <p>The following steps are optional:</p> <ol> <li><code>sudo make install install-docs</code></li> <li><code>cd /usr/local/stow</code></li> <li><code>readlink $(which erl)</code> to figure out the current version in use</li> <li><code>sudo stow -D erlang-X.(Y-1).W</code> to unlink the old version</li> <li><code>sudo stow erlang-X.Y.Z</code> to link the new version</li></ol> <p>We haven&rsquo;t built any debug VMs yet. Let&rsquo;s do that now. There are quite a couple of different VMs you can build. The ones I use the most are the <code>debug</code> and <code>valgrind</code> variants.</p> <pre><code>$ cd $ERL_TOP/erts/emulator $ make debug valgrind gcov gprof lcnt icount</code></pre> <p>After this, you should have a shell script called <code>cerl</code> in <code>$ERL_TOP/bin</code>. This is the script to invoke when you want to run a special BEAM VM. Adding that script to your path will let you invoke a debug VM whenever you like!</p> <h1 id="invoking-cerl">Invoking <code>cerl</code></h1> <p>Now that you have built debug VMs, and have <a href=""><code>cerl</code></a> somewhere in your path, we can start the BEAM inside of gdb or valgrind using <code>cerl -rgdb</code> or <code>cerl -valgrind</code>. After that, you can pass arguments to the script as if you were passing them to <code>erl</code>. <br /><strong>NOTE:</strong> If you type <code>cerl -gdb</code>, the script will start Emacs to control your gdb session, which can be surprising!</p> <h1 id="conclusion">Conclusion</h1> <p>I hope this inspires some of you to play around with Erlang debug VMs! There&rsquo;s a lot of great stuff to be discovered! In the near future, I hope to publish another post showing off some use cases of debug VMs!</p>