Take the 2-minute tour ×

Short Question
Is there an accepted way to run compilers / linkers for embedded software projects on a remote server and still be able to program and debug the software on local machine.

Note I know every IDE will be different, so what I am after is how to define a work flow to accomplish this task assuming that the IDE can be ran using the .o/.elf files built from the remote server.

Areas of Concern
1) Networking to a virtual Windows machine.
2) How / when to transfer the source code to the server to build.

Each family of microprocessor that our software team works with requires it's own compiler, IDE, and programmer. This, overtime, creates many difficulties to overcome.

1) Each developer requires it's own, often pricey, license.
2) To pick up a project that another developer started requires extra care to make sure all of the compiler settings are the same.
3) Supporting legacy software may require an old compilers that conflict with the currently installed compiler.
... the list goes on and on.

Edit: 7-10-2011 1:30 PM CST
1) The compilers I am speaking of are indeed cross compilers
2) A short list of processor families this system ideally would support: Motorola Coldfire, PIC and STM8.
3) Our Coldfire compiler is a variant of GCC, but we have to support multiple versions of it. All other compilers use a target specific compiler that does not provide a floating license.
4) To address littleadv, what I would like to accomplish is an external build server.
5) We currently use a combination of SVN and GIT hosted on an online repository for version control. This is in fact how I thought I would be transferring files to the build server.
6) We are stuck with Windows for most of the compilers.

I now believe that the direction to go is an external build server. There a few obstacles to over come yet. I will assume that we will have to transfer the source files to the server via version control software. Seeing how multiple product lines require access to the same compilers, having an instance for each project does not seem practical.

Would it make sense to create a repository for each compiler that would include folders for build, source, include, output, etc... then have scripts on the users' end that takes care of moving files form the IDE's file structure to the required structure for the compiler? This approach would keep the project repository from being thrashed and give a since of how many times a compiler has been used. Thanks for all of the great responses so far!

share|improve this question
add comment

5 Answers

up vote 5 down vote accepted

In my opinion implementing an automated build server would be the cleanest solution to what you're trying to achieve. With an additional benefit...continuous integration! (I'll touch on CI a bit later).

There are plenty of tools out there to use. @Clifford has already mentioned CMake. But some others are:

  • Hudson (Open Source)
  • CruiseControl (Open Source)
  • TeamCity (Commercial - But it has a fairly generous free version that allows up to 3 build agents and 20 build configurations. The enterprise version of TeamCity is what my company uses so my answer will be geared towards this as it's what I know but concepts will likely apply across multiple tools)

So first of all I'll try to explain what we do and suggest how this might work for you. I don't suggest this is the accepted way to do things but it has worked for us. As I mentioned we use TeamCity for our build server. Each software project is added into TeamCity and build configurations are set up. The build configurations tell TeamCity when to build, how to build and where your project's SCM repository is. We use two different build configurations for each project, one we call "integration" which monitors the project's SCM repository and triggers an incremental build when a check-in is detected. The other configuration we call "nightly" which triggers at a set time every night and performs a completely clean build.

Incidentally just a quick note regarding SCM. For this to work cleanest I think the SCM for each project should be used in a stable trunk topology. If your developers all work from their own branches you'd probably need separate build configurations for each developer which I think would get unnecessarily messy. We've set up our build server with its own SCM user account but with read-only access.

So when a build is triggered for a particular build configuration the server grabs the latest files from the repository and sends them to a "build agent" which executes the build using a build script. We've used Rake to script our builds and automated testing but you can use whatever. The build agent can be on the same PC as the server but in our case we have a separate PC because our build server is centrally located with the ICT department whereas we need our build agent to be physically located with my team (for automated on-target testing). So the toolchains that you use are installed on your build agent.

How could this work for you?

Lets say you work for TidyDog and you have two projects on the go:

  1. "PoopScoop" is based on a PIC18F target compiled using the C18 compiler has its trunk located in your SCM at //PoopScoop/TRUNK/
  2. "PoopBag" is based on a ColdFire target compiled with GCC has its trunk located at //PoopBag/TRUNK/

The compilers that you need in order to build all projects are installed on your build agent (We'll call it TidyDogBuilder). Whether that's the same PC that's running the build server or a separate box is dependent on your situation. Each project has it's own build script (e.g. //PoopScoop/Rakefile.rb and //PoopBag/Rakefile.rb) which handles source file dependencies and invocation of the appropriate compilers. You could for example go to //PoopScoop/ in command prompt, enter rake and the build script would take care of compiling the PoopScoop project within the command prompt.

You then have your build configurations set up on the build server. A build configuration for PoopScoop for example would specify what SCM tool you're using and the repository location (e.g. //PoopScoop/TRUNK/), specify which build agent to use (e.g. TidyDogBuilder), specify where to find the appropriate build script and any necessary command to use (e.g. //PoopScoop/Rakefile.rb invoked with rake incremental:build) and specify what event triggers a build (e.g. Detection of a check-in to //PoopScoop/TRUNK/). So the idea is that if someone submits a change to //PoopScoop/TRUNK/Source/Scooper.c the build server detects this change, grabs the latest revisions of the source files from the repository and sends them to the build agent to be compiled using the build script and in the end emails every developer that has a change in build with the build result.

If your projects need to be compiled for multiple targets you would just modify the project's build script to handle this (e.g. You might have commands like rake build:PIC18 or rake build:Coldfire) and set up a separate build configuration on the build server for each target.

Continuous Integration

So with this system you get continuous integration up and running. Modify your build scripts to run unit tests as well as compile your project and you can have your unit testing being performed automatically after every change. The motive for this is to try to pick up problems as early as possible, as you're developing rather than being surprised during verification activities.

Closing Thoughts

  • Developers not having all toolchain installations would be somewhat dependent on what work they do most often. If it was me and my work tended to be mostly low level, interacting a lot with the hardware, not having the compilers on my workstation would annoy the bejeezes out of me. If on the other hand I was mostly working at an application level and could stub out hardware dependencies it may not be such a problem.
  • TeamCity has a plugin for Eclipse with quite a cool feature. You can do personal builds which means that developers can run build of a pending changelist against any given build configuration. This means that the developer initiate a build of pre-commited code on the build server without having to actually submit their code to SCM. We use this to trial changes against our current unit tests and static analysis as our expensive test tools are only installed on the build agent.
  • With regards to accessing build artifacts when "on the road" I agree, something like a VPN into your intranet is probably the easiest option.
share|improve this answer
+1 Great information. I believe this level of build server is what our lead engineer is wanting to accomplish. I will go through and check out all of these options and discuss with our IT group as well on how to try out some of these ideas. Regarding you closing thought: bullet 1; I am always talking to hardware, so this is indeed an issue that we will need to over come. I will try and post our solution to this problem (assuming we can find one) –  Adam Lewis Jul 12 '11 at 6:04
@Adam It seems like a whole lot of work but it's really not that difficult. My advice is just to bite it off in chunks. –  Jonathan Thomson Jul 12 '11 at 7:03
@Adam: Start with your build script. I like Rake, it's all in Ruby. Someone recommended MSBuild to me once but it's xml format makes my eyes bleed. The build script is useful even without a build server. e.g. Microchip MPLAB is fugly, so the script allows us to build from within Eclipse. Then sort out the build server so it's just building your project (Took me less than a day with TeamCity). Then when you're ready start hooking up unit tests to the system too. We're still improving, our next step is to automate as many of our system acceptance tests as possible. –  Jonathan Thomson Jul 12 '11 at 7:13
Thanks for the starting off point! On a side note; I agree MPLAB 8.X is fugly, but MPLAB X is now in beta and is based off of Netbeans. I haven't gotten much time to play with it, but so far it's worlds better. –  Adam Lewis Jul 12 '11 at 13:53
add comment

One part of your question that hasn't been addressed very much in the other answers (at least as of the time when I'm writing this) is how to transfer the files to the build server. I can provide some experience on that point, as my own development process is reasonably close to that part of your situation.

In my case, I use the "Unison" utility to mirror a section of my home directory on my development laptop to a section of home directory on the build servers. From a programmer's point of view, Unison is basically a wrapper around rsync, with a stored set of checksums to determine whether files on either end of the connection have been modified. It uses this to do a bi-directional synchronization; any modifications I make locally get transferred across to the remote end, and vice-versa. The usual mode of operation is to ask for confirmation of all transfers; you can turn that off, but I find it handy as a check to make sure I changed what I think I changed.

So, my usual workflow is:

  • Edit files locally on my development machine.
  • Run "unison" to synchronize those edits up to the build server.
  • In an ssh connection to the build server, run the compiler.
  • Also in that ssh connection, run the program, producing an output file. (This is of course a little different from your situation.)
  • Run "unison" again to synchronize the changed output file back to my development machine. (You'd be synchronizing the compiled program here.)
  • Look at the output results on my development machine.

It's not quite as fast as the edit-compile-run loop on a local machine, but I find that it's still fast enough not to be annoying. And it's much less heavyweight than using source control as an intermediary; you don't need to check in every edit you make and record it for posterity.

Also, Unison has the advantage of working pretty well across platforms; you can use it on Windows (most easily with Cygwin, though that's not required), and it can either tunnel over an SSH connection if you run an SSH server on your Windows machine, run its own connection service on the build server, or simply use a Windows file share and treat the build server as a "local" file. (Or you could put the server-side files on a Linux-based Samba share in the server farm, and mount that on your Windows build VMs; that may be easier than having "local" files in the VMs.)

Edit: Actually, this is sort of a modification of option 2 in littleadv's discussion of file transferring; it takes the place of editing the files directly on the server via Samba/NFS shares. And it works reasonably well in parallel with that -- I find that having a local cache of the files is ideal and avoids network-lag issues when working remotely, but other engineers at my company prefer something like sshfs (and, of course, on-site using Samba or NFS is fine). It all produces the same result from the point of view of the build server.

share|improve this answer
add comment

I'm not sure I understand what you mean, but I'll try to answer what I think that the question is :-)

First of all, you're talking about cross-compilers, right? You're compiling on one system a code to be run on another system.

Second, you look for a "floating" license model, instead of having a dedicated compiler license per developer.

Third, you're looking to have a build machine where everyone will compile, instead of each developer compiling on his own machine.

These issues are not the same. I'll try to cover them:

  1. Cross compilers - some are free, some are licensed. Some come with IDE, some are just command line compilers that can be integrated into Eclipse/VS/SlickEdit/vi/whatelse. I don't know which one you're using, so let's take a look at Tornado (VxWorks compiler). It has it's own horrible IDE, but can be integrated into others (I've used SlickEdit with it's project files directly, works like a charm). The Tornado compiler requires a license and has several different models, so we'll look at it for the next two points.

  2. Floating licenses. Tornado, as an example, can come with either single license per installation model, or a floating license which will allocate licenses per request. If you're using a build machine - you'll need either a single license (in this case you can only run one instance at a time, which defeats the purpose), or floating license to run several instances at a time. Some cross compilers/libraries don't require licenses at all (various GCC flavors for example).

  3. Build machine - I've experienced using the VxWorks Tornado both as a dedicated compiler on my PC, and as a build-machine installation.

For a build machine you need a way to pass the code to build. That is a problem #2 for you:


Sharing over the network is not a good idea as the network latency will make the compilation times unbearable. Instead, do one of these:

  1. Install the source control on the build machine, have the developers pass the code through the source control. That poses the risk of trashing the source control, so what we did was this:

  2. All the developer's checked out files on the build machine directly (a big UNIX system with a lot of storage), and each developer would edit the files using the network share (SAMBA or NFS), which, on the 1GB company LAN was fine, and compile locally on the build machine. Some would edit directly on the Unix system using the vi/emacs/Unix version of Tornado IDE, which I hated. That also answers your problem #1:

1) Networking to a virtual Windows machine.

(It doesn't have to be a virtual Windows machine, Linux can work too if you have the cross-compiler from Linux to your embedded system).

Hope that helps.

share|improve this answer
+1 for lots of good points. Check out my edit to answer some of your questions / comments. –  Adam Lewis Jul 10 '11 at 18:48
@Adam - Would it make sense to create a repository for each compiler that would include folders for build, source, include, output, etc... then have scripts on the users' end that takes care of moving files form the IDE's file structure to the required structure for the compiler? This approach would keep the project repository from being thrashed and give a since of how many times a compiler has been used. - yes, I think it would. The more automation - the less place for mistakes. –  littleadv Jul 10 '11 at 19:14
add comment

I'm no expert here, but would it be possible to create a virtual machine for every target environment. Then a developer could start the needed VM image on his local machine.

share|improve this answer
That poses licensing issues. You're basically creating different copies of the same machine. –  littleadv Jul 10 '11 at 8:12
add comment

It seems that what you are trying to do is a "distributed build" where there is only a single build server. This may be possible with something like CMake. However while that may "economise" your use of compiler licences, for many tool-chains the single-user licence also (or in some cases exclusively) applies to the debugger; it may not be particularly useful if your developers cannot debug and test their code.

One solution is perhaps to use a single machine with the necessary development tools, and target hardware, and access it from multiple workstations via Remote desktop, or even Telnet if you only need a command line interface. Remote Desktop will only allow one user access at a time, but multiple simultaneous access will probably be contrary to the EULA in many cases anyway.

An DIY solution is to create your own TCP/IP app that takes build commands and files, executes the local compiler or linker and returns the results (build log, and/or object file etc.).

It may be possible to avoid the problem altogether by moving all your development to open-source tools such as GCC or SDCC; that would depend on what targets you need to support.

share|improve this answer
In general our team uses open source whenever possible, but many are not available for the target systems we are working on. Regarding having a dedicated machine that talks to the hardware is not feasible for us at all. We all have laptops and often have to program/debug while in the field (during testing). –  Adam Lewis Jul 10 '11 at 18:01
@Adam: Well the problem of in-the-field development probably exists in any case if the "external server" is not accessible to perform the build. You might consider floating or dongle licensing where available; but availability will vary from vendor to vendor; IAR for example offer both as options. Keil's floating license manager allows you to manually check-out a licence for a defined period so that you can work while disconnected from the license server. The point is you can have more developers than you have licences so long as not all developers will require a license at the same time. –  Clifford Jul 11 '11 at 17:04
The solution I was looking would require an internet connection (we use mobile hot spots when on the road). With this connection I would see either talking to a server on say something like Slice Host, or VPN into our own intranet. Floating dongles don't exist for our compilers :(. Thanks again for your input! –  Adam Lewis Jul 11 '11 at 19:13
Talk to your compiler vendor would be my advice; you may be able to get a concession. Many fixed licences allow two installations (nominally one on a desktop and one on a laptop) for exactly this purpose. Check your EULA carefully. –  Clifford Jul 12 '11 at 19:29
add comment

Your Answer


