user35: When I compile this sketch for the Yún:
int led = 7;

void setup() {                
  pinMode(led, OUTPUT);     
}

void loop() {
  digitalWrite(led, HIGH);
}

I get:

Sketch uses 5,098 bytes (17%) of program storage space. 
Maximum is
  28,672 bytes. Global variables use 153 bytes (5%) of dynamic memory,
  leaving 2,407 bytes for local variables. Maximum is 2,560 bytes.

Even when I compile the BareMinimum sketch:
void setup() {                
  // setup    
}

void loop() {
  // loop
}

I get:

Sketch uses 4,548 bytes (15%) of program storage space.
Maximum is
  28,672 bytes. Global variables use 151 bytes (5%) of dynamic memory,
  leaving 2,409 bytes for local variables. Maximum is 2,560 bytes.

Why does a bare minimum sketch take up 15% of the program storage space allocated? And why does a very simple sketch take up 17% of the program storage space? According to the Arduino website:

It's easy to use it all up by having lots of strings in your program.
  For example, a declaration like: char message[] = "I support the Cape
  Wind project."; puts 33 bytes into SRAM (each character takes a byte,
  plus the '\0' terminator).

However, there aren't any strings declared in either of these sketches.
It seems as if they might import or use other libraries/classes that I don't specify. Maybe it imports a system default library? Or is it something else?

user10794: I wrote a post about Why does it take 1000 bytes to blink one LED?.
The brief answer is: "It doesn't take 2000 bytes to blink two LEDs!"
The longer answer is that the standard Arduino libraries (which you don't have to use if you don't want to) have some nice functionality to simplify your life. For example, you can address pins by number at runtime, where the library converts (say) pin 8 to the correct port and correct bit number. If you hard-code port access you can save that overhead.
Even if you don't use them, the standard libraries include code to count "ticks" so you can find out the current "time" (by calling millis()). To do this it has to add the overhead of some interrupt service routines.
If you simplify down (on the Arduino Uno) to this sketch you get the program memory usage down to 178 bytes (on IDE 1.0.6):
int main ()
  {
  DDRB = bit (5);
  while (true)
    PINB = bit (5);
  }

OK, 178 bytes isn't that much, and of that the first 104 bytes are the hardware interrupt vectors (4 bytes each, for 26 vectors).
So arguably, there are only 74 bytes required to blink an LED. And of that 74 bytes most are really the code generated by the compiler to initialize global memory. If you add enough code to blink two LEDs:
int main ()
  {
  DDRB = bit (5);  // pin 13
  DDRB |= bit (4);  // pin 12

  while (true)
    {
    PINB = bit (5); // pin 13
    PINB = bit (4); // pin 12
    }
  }

Then the code size increases to 186 bytes. So therefore you could argue that it only takes 186 - 178 = 8 bytes to blink an LED.
So, 8 bytes to blink an LED. Sounds pretty efficient to me.

In case you are tempted to try this at home, I should point out that whilst the posted code above blinks two LEDs, it does so very rapidly indeed. In fact, they blink at 2 MHz - see screenshot. Channel 1 (yellow) is pin 12, channel 2 (cyan) is pin 13.

As you can see, the output pins have a square wave with a frequency of 2 MHz. Pin 13 changes state 62.5 ns (one clock cycle) before pin 12, due to the order of the toggling of the pins in the code.
So unless you have much better eyes than mine, you won't actually see any blinking effect.

As an amusing extra, you can actually toggle two pins in the same amount of program space as toggling one pin.
int main ()
  {
  DDRB = bit (4) | bit (5);  // set pins 12 and 13 to output

  while (true)
    PINB =  bit (4) | bit (5); // toggle pins 12 and 13
  } // end of main

That compiles into 178 bytes.
This gives you a higher frequency:

Now we are up to 2.66 MHz.

user7508: You already have some perfectly good answers. I am posting this only to
share some stats I did one day I asked myself the same sort of
questions: What is taking so much space on a minimal sketch? What is the
minimum needed to achieve the same functionality?
Below are three versions of a minimal blinky program that toggles the
LED on pin 13 every second. All three versions have been compiled for an
Uno (no USB involved) using avr-gcc 4.8.2, avr-libc 1.8.0 and
arduino-core 1.0.5 (I do not use the Arduino IDE).
First, the standard Arduino way:
const uint8_t ledPin = 13;

void setup() {
    pinMode(ledPin, OUTPUT);
}

void loop() {
    digitalWrite(ledPin, HIGH);
    delay(1000);
    digitalWrite(ledPin, LOW);
    delay(1000);
}

This compiles to 1018 bytes. Using both avr-nm and
disassembly,
I broke down that size into individual functions. From
largest to smallest:
 148 A ISR(TIMER0_OVF_vect)
 118 A init
 114 A pinMode
 108 A digitalWrite
 104 C vector table
  82 A turnOffPWM
  76 A delay
  70 A micros
  40 U loop
  26 A main
  20 A digital_pin_to_timer_PGM
  20 A digital_pin_to_port_PGM
  20 A digital_pin_to_bit_mask_PGM
  16 C __do_clear_bss
  12 C __init
  10 A port_to_output_PGM
  10 A port_to_mode_PGM
   8 U setup
   8 C .init9 (call main, jmp exit)
   4 C __bad_interrupt
   4 C _exit
-----------------------------------
1018   TOTAL

In the list above, the first column is the size in bytes, and the second
column tells whether the code comes from the Arduino core library (A,
822 bytes total), the C runtime (C, 148 bytes) or the user (U, 48
bytes).
As can be seen in this list, the largest function is the routine
servicing the timer 0 overflow interrupt. This routine is responsible of
tracking time, and is needed by millis(), micros() and delay().
The second largest function is init(), which sets the hardware timers
for PWM, enables TIMER0_OVF interrupt and disconnects the USART (that
was used by the bootloader). Both this and the previous function are
defined in
<Arduino directory>/hardware/arduino/cores/arduino/wiring.c.
Next is the C + avr-libc version:
#include <avr/io.h>
#include <util/delay.h>

int main(void)
{
    DDRB |= _BV(PB5);     /* set pin PB5 as output */
    for (;;) {
        PINB = _BV(PB5);  /* toggle PB5 */
        _delay_ms(1000);
    }
}

The break-down of the individual sizes:
104 C vector table
 26 U main
 12 C __init
  8 C .init9 (call main, jmp exit)
  4 C __bad_interrupt
  4 C _exit
----------------------------------
158   TOTAL

This is 132 bytes for the C runtime and 26 bytes of user code, including
the inlined function _delay_ms().
It could be noted that, since this program does not use interrupts,
the interrupt vector table is not needed, and regular user code
could be put in its place. The following assembly version does exactly
that:
#include <avr/io.h>
#define io(reg) _SFR_IO_ADDR(reg)

    sbi io(DDRB), 5  ; set PB5 as output
loop:
    sbi io(PINB), 5  ; toggle PB5
    ldi r26, 49      ; delay for 49 * 2^16 * 5 cycles
delay:
    sbiw r24, 1
    sbci r26, 0
    brne delay
    rjmp loop

This is assembled (with avr-gcc -nostdlib) into only 14 bytes,
most of which are used to delay the toggles so that the blink is
visible. If you remove that delay loop, you end up with a 6-byte
program that blinks too fast to be seen (at 2 MHz):
    sbi io(DDRB), 5  ; set PB5 as output
loop:
    sbi io(PINB), 5  ; toggle PB5
    rjmp loop


user60: The YUN is a combo. Part Arduino and Part OpenWRT(Linux). Your question is in reference to the Arduino. Where this is actually a ATmega32u4 similar to a Leonardo and not an UNO(ATmega328p). The 32u4(Leo) communicates via Virtual Serial Ports over the USB (short answer: this needs to be supported) , where the UNO has a real Serial Port (aka UART). Below are builds statistics of the different boards types for the AVR processors.
Note on the UNO there is an external chip that converts USB to the Serial port's DTR pin which toggles the ATmega328's reset pin when connected causing a reboot to the bootloader. In contrast the Leo/Yun's USB to Serial is in implemented in the firmware of the 32u4. Hence in order to remotely reboot the Leo or YUN's 32u4 chip the firmware loaded must always support the USB client side driver. Which consumes approximately 4K. 
If the USB was NOT needed and no other library resources were called as in the case of BareMinimum.ino on an UNO, only approximately 466 bytes are needed for the core Arduino Library.
compile stats of BareMinimum.ino on a UNO(ATmega328p)
Sketch uses 466 bytes (1%) of program storage space. Maximum is 32,256 bytes.
Global variables use 9 bytes (0%) of dynamic memory, leaving 2,039 bytes for local variables. Maximum is 2,048 bytes.

compile stats of BareMinimum.ino on a Leonardo(ATmega32u4)
Sketch uses 4,554 bytes (15%) of program storage space. Maximum is 28,672 bytes.
Global variables use 151 bytes (5%) of dynamic memory, leaving 2,409 bytes for local variables. Maximum is 2,560 bytes.

compile stats of BareMinimum.ino on a Yun(ATmega32u4)
Sketch uses 4,548 bytes (15%) of program storage space. Maximum is 28,672 bytes.
Global variables use 151 bytes (5%) of dynamic memory, leaving 2,409 bytes for local variables. Maximum is 2,560 bytes.


user83: Arduino compiles in a lot of standard libraries, interrupts, ... etc. For example the pinMode and digitalWrite functions use a lookup table to figure out at run time which GPIO registers to write data to. Another example is that Arduino keeps track of time, it defines some interrupts by default and all this functionality requires some space. You'll notice that if you extend the program, the foot print will only change slightly.
I personally like to program controllers with bare minimum, without "bloat", but you'll quickly enter the world of EE.SE and SO because several easy to use functions will no longer work out of the box. There are some alternative libraries for pinMode and digitalWrite that compile into a smaller footprint, but come with other disadvantages like for example static compiled pins (where led cannot be a variable, but is a constant).

35: <p>When I compile this sketch for the Yún:</p>

<pre><code>int led = 7;

void setup() {                
  pinMode(led, OUTPUT);     
}

void loop() {
  digitalWrite(led, HIGH);
}
</code></pre>

<p>I get:</p>

<blockquote>
  <p>Sketch uses 5,098 bytes (17%) of program storage space. </p>
  
  <p>Maximum is
  28,672 bytes. Global variables use 153 bytes (5%) of dynamic memory,
  leaving 2,407 bytes for local variables. Maximum is 2,560 bytes.</p>
</blockquote>

<p>Even when I compile the BareMinimum sketch:</p>

<pre><code>void setup() {                
  // setup    
}

void loop() {
  // loop
}
</code></pre>

<p>I get:</p>

<blockquote>
  <p>Sketch uses 4,548 bytes (15%) of program storage space.</p>
  
  <p>Maximum is
  28,672 bytes. Global variables use 151 bytes (5%) of dynamic memory,
  leaving 2,409 bytes for local variables. Maximum is 2,560 bytes.</p>
</blockquote>

<p>Why does a bare minimum sketch take up 15% of the program storage space allocated? And why does a very simple sketch take up 17% of the program storage space? According to <a href="http://arduino.cc/en/Tutorial/Memory#.UwfEavkiEw8">the Arduino website</a>:</p>

<blockquote>
  <p>It's easy to use it all up by having lots of strings in your program.
  For example, a declaration like: <code>char message[] = "I support the Cape
  Wind project.";</code> puts 33 bytes into SRAM (each character takes a byte,
  plus the '\0' terminator).</p>
</blockquote>

<p>However, there aren't any strings declared in either of these sketches.</p>

<p>It seems as if they might import or use other libraries/classes that I don't specify. Maybe it imports a system default library? Or is it something else?</p>
 10794: <p>I wrote a post about <a href="http://www.gammon.com.au/forum/?id=12173" rel="nofollow noreferrer">Why does it take 1000 bytes to blink one LED?</a>.</p>

<p>The brief answer is: "It doesn't take 2000 bytes to blink <strong>two</strong> LEDs!"</p>

<p>The longer answer is that the standard Arduino libraries (which you don't have to use if you don't want to) have some nice functionality to simplify your life. For example, you can address pins by number at runtime, where the library converts (say) pin 8 to the correct port and correct bit number. If you hard-code port access you can save that overhead.</p>

<p>Even if you don't use them, the standard libraries include code to count "ticks" so you can find out the current "time" (by calling <code>millis()</code>). To do this it has to add the overhead of some interrupt service routines.</p>

<p>If you simplify down (on the Arduino Uno) to this sketch you get the program memory usage down to 178 bytes (on IDE 1.0.6):</p>

<pre><code>int main ()
  {
  DDRB = bit (5);
  while (true)
    PINB = bit (5);
  }
</code></pre>

<p>OK, 178 bytes isn't that much, and of that the first 104 bytes are the hardware interrupt vectors (4 bytes each, for 26 vectors).</p>

<p>So arguably, there are only 74 bytes required to blink an LED. And of that 74 bytes most are really the code generated by the compiler to initialize global memory. If you add enough code to blink two LEDs:</p>

<pre><code>int main ()
  {
  DDRB = bit (5);  // pin 13
  DDRB |= bit (4);  // pin 12

  while (true)
    {
    PINB = bit (5); // pin 13
    PINB = bit (4); // pin 12
    }
  }
</code></pre>

<p>Then the code size increases to 186 bytes. So therefore you could argue that it only takes <code>186 - 178 = 8</code> bytes to blink an LED.</p>

<p>So, 8 bytes to blink an LED. Sounds pretty efficient to me.</p>

<hr>

<p>In case you are tempted to try this at home, I should point out that whilst the posted code above blinks two LEDs, it does so very rapidly indeed. In fact, they blink at 2 MHz - see screenshot. Channel 1 (yellow) is pin 12, channel 2 (cyan) is pin 13.</p>

<p><img src="https://i.stack.imgur.com/mPd81.png" alt="Rapid blinking of pins 12 and 13"></p>

<p>As you can see, the output pins have a square wave with a frequency of 2 MHz. Pin 13 changes state 62.5 ns (one clock cycle) before pin 12, due to the order of the toggling of the pins in the code.</p>

<p>So unless you have much better eyes than mine, you won't actually see any blinking effect.</p>

<hr>

<p>As an amusing extra, you can actually toggle <strong>two</strong> pins in the <em>same</em> amount of program space as toggling one pin.</p>

<pre><code>int main ()
  {
  DDRB = bit (4) | bit (5);  // set pins 12 and 13 to output

  while (true)
    PINB =  bit (4) | bit (5); // toggle pins 12 and 13
  } // end of main
</code></pre>

<p>That compiles into 178 bytes.</p>

<p>This gives you a higher frequency:</p>

<p><img src="https://i.stack.imgur.com/VdQde.png" alt="Very rapid blinking of pins 12 and 13"></p>

<p>Now we are up to 2.66 MHz.</p>
 7508: <p>You already have some perfectly good answers. I am posting this only to
share some stats I did one day I asked myself the same sort of
questions: What is taking so much space on a minimal sketch? What is the
minimum needed to achieve the same functionality?</p>

<p>Below are three versions of a minimal blinky program that toggles the
LED on pin 13 every second. All three versions have been compiled for an
Uno (no USB involved) using avr-gcc 4.8.2, avr-libc 1.8.0 and
arduino-core 1.0.5 (I do not use the Arduino IDE).</p>

<p>First, the standard Arduino way:</p>

<pre class="lang-c++ prettyprint-override"><code>const uint8_t ledPin = 13;

void setup() {
    pinMode(ledPin, OUTPUT);
}

void loop() {
    digitalWrite(ledPin, HIGH);
    delay(1000);
    digitalWrite(ledPin, LOW);
    delay(1000);
}
</code></pre>

<p>This compiles to 1018 bytes. <a href="https://arduino.stackexchange.com/a/12684">Using both <code>avr-nm</code> and
disassembly</a>,
I broke down that size into individual functions. From
largest to smallest:</p>

<pre><code> 148 A ISR(TIMER0_OVF_vect)
 118 A init
 114 A pinMode
 108 A digitalWrite
 104 C vector table
  82 A turnOffPWM
  76 A delay
  70 A micros
  40 U loop
  26 A main
  20 A digital_pin_to_timer_PGM
  20 A digital_pin_to_port_PGM
  20 A digital_pin_to_bit_mask_PGM
  16 C __do_clear_bss
  12 C __init
  10 A port_to_output_PGM
  10 A port_to_mode_PGM
   8 U setup
   8 C .init9 (call main, jmp exit)
   4 C __bad_interrupt
   4 C _exit
-----------------------------------
1018   TOTAL
</code></pre>

<p>In the list above, the first column is the size in bytes, and the second
column tells whether the code comes from the Arduino core library (A,
822 bytes total), the C runtime (C, 148 bytes) or the user (U, 48
bytes).</p>

<p>As can be seen in this list, the largest function is the routine
servicing the timer 0 overflow interrupt. This routine is responsible of
tracking time, and is needed by <code>millis()</code>, <code>micros()</code> and <code>delay()</code>.
The second largest function is <code>init()</code>, which sets the hardware timers
for PWM, enables TIMER0_OVF interrupt and disconnects the USART (that
was used by the bootloader). Both this and the previous function are
defined in
<code>&lt;Arduino directory&gt;/hardware/arduino/cores/arduino/wiring.c</code>.</p>

<p>Next is the C + avr-libc version:</p>

<pre class="lang-c prettyprint-override"><code>#include &lt;avr/io.h&gt;
#include &lt;util/delay.h&gt;

int main(void)
{
    DDRB |= _BV(PB5);     /* set pin PB5 as output */
    for (;;) {
        PINB = _BV(PB5);  /* toggle PB5 */
        _delay_ms(1000);
    }
}
</code></pre>

<p>The break-down of the individual sizes:</p>

<pre><code>104 C vector table
 26 U main
 12 C __init
  8 C .init9 (call main, jmp exit)
  4 C __bad_interrupt
  4 C _exit
----------------------------------
158   TOTAL
</code></pre>

<p>This is 132 bytes for the C runtime and 26 bytes of user code, including
the inlined function <code>_delay_ms()</code>.</p>

<p>It could be noted that, since this program does not use interrupts,
the interrupt vector table is not needed, and regular user code
could be put in its place. The following assembly version does exactly
that:</p>

<pre><code>#include &lt;avr/io.h&gt;
#define io(reg) _SFR_IO_ADDR(reg)

    sbi io(DDRB), 5  ; set PB5 as output
loop:
    sbi io(PINB), 5  ; toggle PB5
    ldi r26, 49      ; delay for 49 * 2^16 * 5 cycles
delay:
    sbiw r24, 1
    sbci r26, 0
    brne delay
    rjmp loop
</code></pre>

<p>This is assembled (with <code>avr-gcc -nostdlib</code>) into only 14 bytes,
most of which are used to delay the toggles so that the blink is
visible. If you remove that delay loop, you end up with a 6-byte
program that blinks too fast to be seen (at 2&nbsp;MHz):</p>

<pre><code>    sbi io(DDRB), 5  ; set PB5 as output
loop:
    sbi io(PINB), 5  ; toggle PB5
    rjmp loop
</code></pre>
 60: <p>The YUN is a combo. Part Arduino and Part OpenWRT(Linux). Your question is in reference to the Arduino. Where this is actually a ATmega32u4 similar to a Leonardo and not an UNO(ATmega328p). The 32u4(Leo) communicates via Virtual Serial Ports over the USB <em>(short answer: this needs to be supported)</em> , where the UNO has a real Serial Port (aka UART). Below are builds statistics of the different boards types for the AVR processors.</p>

<p>Note on the UNO there is an external chip that converts USB to the Serial port's DTR pin which toggles the ATmega328's reset pin when connected causing a reboot to the bootloader. In contrast the Leo/Yun's USB to Serial is in implemented in the firmware of the 32u4. Hence in order to remotely reboot the Leo or YUN's 32u4 chip the firmware loaded must always support the USB client side driver. Which consumes approximately 4K. </p>

<p>If the USB was NOT needed and no other library resources were called as in the case of BareMinimum.ino on an UNO, only approximately 466 bytes are needed for the core Arduino Library.</p>

<p>compile stats of BareMinimum.ino on a UNO(ATmega328p)</p>

<pre><code>Sketch uses 466 bytes (1%) of program storage space. Maximum is 32,256 bytes.
Global variables use 9 bytes (0%) of dynamic memory, leaving 2,039 bytes for local variables. Maximum is 2,048 bytes.
</code></pre>

<p>compile stats of BareMinimum.ino on a Leonardo(ATmega32u4)</p>

<pre><code>Sketch uses 4,554 bytes (15%) of program storage space. Maximum is 28,672 bytes.
Global variables use 151 bytes (5%) of dynamic memory, leaving 2,409 bytes for local variables. Maximum is 2,560 bytes.
</code></pre>

<p>compile stats of BareMinimum.ino on a Yun(ATmega32u4)</p>

<pre><code>Sketch uses 4,548 bytes (15%) of program storage space. Maximum is 28,672 bytes.
Global variables use 151 bytes (5%) of dynamic memory, leaving 2,409 bytes for local variables. Maximum is 2,560 bytes.
</code></pre>
 83: <p>Arduino compiles in a lot of standard libraries, interrupts, ... etc. For example the pinMode and digitalWrite functions use a lookup table to figure out at run time which GPIO registers to write data to. Another example is that Arduino keeps track of time, it defines some interrupts by default and all this functionality requires some space. You'll notice that if you extend the program, the foot print will only change slightly.</p>

<p>I personally like to program controllers with bare minimum, without "bloat", but you'll quickly enter the world of EE.SE and SO because several easy to use functions will no longer work out of the box. There are some alternative libraries for pinMode and digitalWrite that compile into a smaller footprint, but come with other disadvantages like for example static compiled pins (where <code>led</code> cannot be a variable, but is a constant).</p>
