title: ARM: Start/Wakeup/Bringup the other CPU cores/APs and pass execution start address?
body: <p>I've been banging my head with this for the last 3-4 days and I can't find a DECENT explanatory documentation (from ARM or unofficial) to help me.
  I've got an <strong>ODROID-XU board (big.LITTLE 2 x Cortex-A15 + 2 x Cortex-A7)</strong> board and I'm trying to understand a bit more about the ARM architecture. In my "experimenting" code I've now arrived at the stage where I want to <strong>WAKE UP THE OTHER CORES FROM THEIR WFI (wait-for-interrupt) state.</strong></p>
  
  <p>The missing information I'm still trying to find is:</p>
  
  <p><strong>1.</strong> When getting the base address of the memory-mapped GIC I understand that I need to read CBAR; But no piece of documentation explains how the bits in CBAR (the 2 PERIPHBASE values) should be arranged to get to the final GIC base address</p>
  
  <p><strong>2.</strong> When sending an SGI through the GICD_SGIR register, what interrupt ID between 0 and 15 should I choose? Does it matter?</p>
  
  <p><strong>3.</strong> When sending an SGI through the GICD_SGIR register, how can I tell the other cores <strong>WHERE TO START EXECUTION FROM</strong>?</p>
  
  <p><strong>4.</strong> How does the fact that <strong>my code is loaded by the U-BOOT bootloader</strong> affect this context?</p>
  
  <p>The <strong>Cortex-A Series Programmer's Guide v3.0</strong> (found here: <a href="http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.den0013c/index.html">link</a>) states the following in <strong>section 22.5.2</strong> (SMP boot in Linux, page <strong>271</strong>):</p>
  
  <blockquote>
    <p><em><strong>While the primary core is booting, the secondary cores will be held in a standby state, using the 
    WFI instruction. It (the primary core) will provide a startup address to the secondary cores and wake them using an 
    Inter-Processor Interrupt(IPI), meaning an SGI signalled through the GIC</em></strong></p>
  </blockquote>
  
  <p>How does Linux do that? The documentation-<strong>S</strong> don't give any other details regarding "<strong>It will provide a startup address to the secondary cores</strong>".</p>
  
  <p>My frustration is growing and I'd be very grateful for answers.
  Thank you very much in advance!</p>
  
  <p><strong>EXTRA DETAILS</strong></p>
  
  <p>Documentation I use:</p>
  
  <ul>
  <li><strong>ARMv7-A&amp;R Architecture Reference Manual</strong></li>
  <li><strong>Cortex-A15 TRM (Technical Reference Manual)</strong></li>
  <li><strong>Cortex-A15 MPCore TRM</strong></li>
  <li><strong>Cortex-A Series Programmer's Guide v3.0</strong></li>
  <li><strong>GICv2 Architecture Specification</strong></li>
  </ul>
  
  <p>What I've done by now:</p>
  
  <ul>
  <li><strong>UBOOT loads me at 0x40008000; I've set-up Translation Tables (TTBs), written TTBR0 and TTBCR accordingly and mapped 0x40008000 to 0x8000_0000 (2GB), so I also enabled the MMU</strong></li>
  <li><strong>Set-up exception handlers of my own</strong></li>
  <li><strong>I've got Printf functionality over the serial (UART2 on ODROID-XU)</strong></li>
  </ul>
  
  <p>All the above seems to work properly.</p>
  
  <p>What I'm trying to do now:</p>
  
  <ul>
  <li><strong>Get the GIC base address => at the moment I read CBAR and I simply AND (&amp;) its value with 0xFFFF8000 and use this as the GIC base address, although I'm almost sure this ain't right</strong></li>
  <li><strong>Enable the GIC distributor (at offset 0x1000 from GIC base address?), by writting GICD_CTLR with the value 0x1</strong></li>
  <li><strong>Construct an SGI with the following params: Group = 0, ID = 0, TargetListFilter = "All CPUs Except Me" and send it (write it) through the GICD_SGIR GIC register</strong></li>
  <li>Since I haven't passed any execution start address for the other cores, nothing happens after all this</li>
  </ul>
  
  <p><strong>....UPDATE....</strong></p>
  
  <p>I've started looking at the Linux kernel and QEMU source codes in search for an answer. Here's what I found out (please correct me if I'm wrong):</p>
  
  <ul>
  <li><strong>When powering up the board ALL THE CORES start executing from the reset vector</strong></li>
  <li><strong>A <em>software (firmware)</em> component executes WFI on the secondary cores and some other code that will act as a protocol between these secondary cores and the primary core, when the latter wants to wake them up again</strong></li>
  <li><strong>For example, the protocol used on the <em>EnergyCore ECX-1000 (Highbank)</em> board is as follows:</strong></li>
  </ul>
  
  <p><code>**(1)** the secondary cores enter WFI and when</code></p>
  
  <p><code>**(2)** the primary core sends an SGI to wake them up</code></p>
  
  <p><code>**(3)** they check if the value at address (0x40 + 0x10 * coreid) is non-null;</code></p>
  
  <p><code>**(4)** if it is non-null, they use it as an address to jump to (execute a BX)</code></p>
  
  <p><code>**(5)** otherwise, they re-enter standby state, by re-executing WFI</code></p>
  
  <p><code>**(6)** So, if I had an EnergyCore ECX-1000 board, I should write (0x40 + 0x10 * coreid) with the address I want each of the cores to jump to and send an SGI</code></p>
  
  <p>Questions:</p>
  
  <ul>
  <li><strong>1. What is the software component that does this? Is it the BL1 binary I've written on the SD Card, or is it U-BOOT?</strong></li>
  <li><strong>2. From what I understand, this software protocol differs from board to board. Is it so, or does it only depend on the underlying processor?</strong></li>
  <li><strong>3. Where can I find information about this protocol for a pick-one ARM board? - can I find it on the official ARM website or on the board webpage?</strong></li>
  </ul>