Document Sample
CELL CULTURE Powered By Docstoc
					Programming                      Programming the Cell

                                                                                                                                         Dmitry Sunagatov, Fotolia
Application development for the Cell processor

The Cell architecπture is finding its way into a vast range of computer systems – from huge supercomputers
to inauspicious Playstation game consoles. We’ll show you around the Cell and take a look at a sample Cell

         ony Computer Entertainment,           In addition to its power and flexibility,     While the operating system on the PPE
         Toshiba, and IBM started devel-     the Cell is also known for energy effi-       manages system resources, the SPEs
         oping the innovative Cell Broad-    ciency. Cell-based systems currently hold     handle algebraic operations. Their 128-
band Engine Architecture (CBEA)              the top seven spots in the Green 500 List     bit registers either manipulate four 32-bit
around 2001. The Cell architecture spe-      [2] of the most energy-efficient super-       numbers per operation (short integers or
cializes in efficient processing of large    computers. In this article, I explore the     single-precision floating points), or two
data streams, such as the streams that       Cell architecture and describe an exam-       64-bit figures (long integers, or double-
occur in multimedia applications or          ple application that will help you get        precision floating points). This SIMD ar-
computer games. The first implementa-        started with programming for the Cell.        chitecture (Single Instruction, Multiple
tion of the Cell architecture is the Cell      The Cell computer specializes in han-       Data) is similar to the PC processor’s
Broadband Engine, also known as the          dling problems that need a large amount       MMX extension.
Cell processor, which dates back to 2005     of computer power but are easily split          One special thing about the SPEs is
(Figure 1). Since it was introduced as the   into separate tasks. The individual Cell      that they only work with code and data
processor for the Sony PlayStation 3, the    processor cores then process these sepa-      stored in their local memory; they do not
Cell CPU has attracted much attention.       rate tasks in parallel.                       access main memory or peripherals. Ap-
Although the Playstation (Figure 2) is         The Cell processor consists of a con-       plications must ensure that the right
certainly the most widespread applica-       ventional processor core (Power Process-      code and data are available locally. The
tion of the Cell architecture, the most      ing Element, PPE) with 64-bit IBM             data transfer operations between main
spectacular application has to be the        Power Architecture and eight Synergistic      and local SPE memory are organized by
Roadrunner (Figure 3), which uses more       Processing Elements (SPE; see Figure 4).      the SPE’s DMA controllers and do not
than 12,000 Cell processors [1].             Each of the eight SPEs has 256KB of           cause SPU overhead.
   Cell blades are available from both       local memory and a DMA controller
IBM and Mercury Computer Systems.            (Memory Flow Controller, MFC). All            Developing for the Cell
Mercury has even built a PCI Express         nine processors are linked by a data bus      Of course, developing applications for
card with a full-fledged Cell processor      (Element Interconnect Bus, EIB) to each       the Cell processor is more appealing for
computer. Toshiba uses a variant of the      other, the main memory, and the periph-       those who have access to a Cell-based
Cell processor in its Qosmio notebooks.      eral devices.                                 machine. If you work on a Cell blade

74        ISSUE 99                           FEBRuARy 2009
                                                                          Programming the Cell                                        Programming

                                                   According to IBM, the minimum hard-
                                                ware requirement is an Intel Pentium 4
                                                with 2GHz clock speed or an AMD
                                                Socket F Opteron. On top of this, the
                                                SDK needs 1GB RAM and 5GB free disk
                                                space. To install the Cell SDK on Linux,
                                                you also need the rsync, sed, TCL, and
                                                wget packages. Because the installation
                                                script downloads various packages from
                                                the Barcelona Supercomputer Center [5],
                                                you will need continuous Internet access
                                                throughout the installation.
                                                   The cell‑install‑3.1.0‑0.0.‑noarch.‑rpm
                                                RPM creates an /opt/cell directory for the
                                                developer environment and documenta-

                                                tion. The installation script expects the
Figure 1: The Cell CPU is manufactured using    path to the CD images as an option:
a Silicon On Insulator (SOI) approach.                                                                                  Figure 3: The Roadrunner supercomputer by
                                                 /opt/cell/cellsdk    U                                                 IBM, which currently holds the number one
server, you will probably develop your           ‑‑iso path install                                                     spot in the Top 500 list, uses around 12000
applications directly on the Cell plat-                                                                                 units of the Cell chip.
form. If you have a Playstation 3, it           This variant has the advantage that you
makes more sense to use a Linux PC as           can install the content of both images in                               runs on the Power PC core (PPE pro-
your development platform. The Playsta-         a single process. If you have the Cell                                  gram), and at least one program that
tion doesn’t have much in the way of            SDK CD images on separate CDs, you                                      keeps the SPEs busy (SPE program). To
RAM – just 256MB – and the low mem-             need to insert the Developer and Extras                                 allow the PPE program to control the
ory becomes fairly obvious when you             CD one after another and launch the in-                                 SPE software, the PPE source code must
work with an X11 interface.                     stallation separately by typing /opt‑/cell/                             include the libspe2.h header file from the
   IBM provides a free Software Develop-        cellsdk install. If you have installed the                              Libspe2 library. The SPE program con-
ment Kit (SDK) for the Cell architecture        system simulator, you can initialize it                                 tains the actual calculating routines. An
[3]. The Cell SDK will run on the x86,          using the /opt/cell/cellsdk_sync_simula‑                                SPE program must include the spu_
x86_64, and PowerPC platforms, as well          tor script, which installs some required                                intrinsics.h and spu_mfcio.h header files
as on Cell-based Linux machines. The            SDK elements. The ISOs contain several                                  for SIMD calculation functions and for
latest version of the Cell SDK (3.1) sup-       libraries that are not open source. For                                 communication with the PPE and the
ports Fedora 9 and Red Hat Enterprise           installation documentation, check out                                   other SPEs.
Linux 5.2. The kit includes the Developer       /opt/cell/sdk/docs/install.                                                The example program described in
and Extras CD images and an RPM pack-                                                                                   this article provides an approximation of
age with the installation script. Up to         π on the Cell                                                           π using the Shotgun algorithm (see the
version 3.0, the Cell SDK for Fedora in-        Applications for the Cell processor con-                                box titled “The Mathematical Shotgun”).
cluded a system simulator, which would          sist of at least two parts: a program that                              The program expects command-line pa-
let programmers test and optimize appli-
cations without physical Cell hardware.                                                                                 The Mathematical Shotgun
As of Version 3.1, the simulator is avail-
                                                                                                                         Several methods exist for calculating an
able separately from the IBM website                                                                                     approximate value for π. The Shotgun
[4]. The new Version 3.1 is still in beta,                                                                               algorithm involves the computer calcu-
                                                                                          Sony Computer Entertainment

but it works perfectly on Fedora 9.                                                                                      lating pairs of random numbers be-
                                                                                                                         tween 0 and 1 (Figure 5). Each pair rep-
   Linux on the Playstation                                                                                              resents a point in a square with an edge
                                                                                                                         length of 1, where the bottom left corner
 In contrast to other console manufactur-                                                                                has the coordinates (0,0) and the top
 ers, Sony officially supports the installa-                                                                             right corner the coordinates (1,1).
 tion of Linux on the Playstation, and you                                                                               Assuming that the dots are spread
 will find many howtos on the web [6].                                                                                   evenly across the square, the ratio
 There are two things to note about run-                                                                                 between the number of dots that lie
 ning Linux on the PS 3. First, direct ac-                                                                               inside a circle of radius 1 and the total
 cess to the hardware is not supported;                                                                                  number of dots is approximately equal
 to protect its proprietary firmware, Sony                                                                               to the ratio between the areas of a quar-
 added a virtualization layer. Second,                                                                                   ter circle with a radius of 1 and a square
 only six of the Cell processor’s eight                                                                                  with an edge length of 1, which is ex-
                                                Figure 2: The most popular application for
 SPEs are available to Linux programs.                                                                                   actly π/4.
                                                the Cell processor is Sony's Playstation 3.

                                                                           FEBRuARy 2009                                                       ISSUE 99         75
Programming                              Programming the Cell

                                                                                                                            a program on an SPE,
      SPE           SPE            SPE           SPE             SPE          SPE           SPE           SPE
                                                                                                                            three steps are required.
       SPU               SPU              SPU      SPU             SPU           SPU              SPU         SPU           First, the spe_context_
                                                                                                                            create() function (line 7)
         LS               LS               LS       LS              LS            LS               LS          LS           needs to create an SPE
         MFC             MFC              MFC      MFC              MFC           MFC              MFC         MFC          context. Second, the spe_
                                                                                                                            program_load() function
                                                                                                                            (line 8) needs to specify
                                               Element Interconnect Bus (EIB)                                               the program to execute;
                                                                                                                            the programmer needs to
                                                                                                                            declare the spe_program_
     Level-2-Cache                                                              MIC                         BIC             handle_t variable in the
                                                                                                                            PPE program header for
                                                                                                                            this. This variable is al-
      Level-1-Cache               PPE                                         Memory                         I/O            ways declared externally,
                                                                                                                            that is, outside of the
     SPE:       Synergistic Processing Element                         PPE:    Power Processing Element
     SPU:       Synergistic Processing Unit                            MIC:    Memory Interface Controller                  function. The name is
     LS:        Local Store                                            BIC:    Bus Interface Controller                     identical to the name that
     MFC:       Memory Flow Controller                                 MFC:    Memory Flow Controller
                                                                                                                            the SPE program will be
Figure 4: Each Synergistic Processing Element (SPE) has a Synergistic Processing Unit (SPU), local memory                   given later when you
(LM), and a Memory Flow Controller (MFC). A Memory Interface Controller (MIC) sits in front of the main                     compile it.
memory, and a Bus Interface Controller (BIC) is in front of the input/output interface.                                        The third step is for the
                                                                                                                            spe_context_run() func-
rameters for the number of random pairs               integer multiples of 128 for best possible           tion to launch the program you want to
of figures to generate and the number of              data transfer. Programmers can achieve               execute. Normally, this function would
SPEs. After the main() routine in pi_lib‑             this by using the posix_memalign()                   block the PPE program while the SPE
spe_ppe.c has parsed the command line                 function instead of the conventional                 program is running, thus preventing any
for this information, the program dy-                 malloc(). The size of the individual                 other SPE programs from launching par-
namically allocates three memory areas.               blocks exchanged by the PPE and SPE                  allel to it. A Posix thread helps to avoid
The first array stores a structure with the           also must be a multiple of 16. If inexpli-           this by executing the spu_pthread()
parameters that the PPE and SPE ex-                   cable bus errors occur when you test the             function (line 10), which in turn
change for each SPE. The spe_par_t SPE                application, this is often a result of in-           launches an SPE program without inter-
structure type is declared in the pi_                 correct start address alignment or illegal           rupting the PPE program flow.
libspe.h header (Listing 1). The second               block sizes in the data blocks trans-                    Now the SPE program needs to know
array stores a structure with the SPE                 ferred. The third array is only used inter-          where the parameters for the forthcom-
context for each SPE. This data contains              nally by the PPE program and does not                ing calculations are located. Each SPE
everything the PPE needs to know about                have to fulfill any special requirements             has a mailbox for incoming messages
a program running on an SPE. The data                 with regard to start addresses or sizes.             (four 32-bit words) and a mailbox for
type for this is declared in libspe2.h.                                                                    outgoing messages (one 32-bit word).
                                                      random                                               Another mailbox triggers a software in-
addressing                                            The PPE program contains a loop (List-               terrupt when data is available. In this
The start addresses for variables that the            ing 2), which distributes the workload               case, the PPE program calls spe_in_
PPE and SPE need to exchange later                    over the SPEs involved and sets a seed               mbox_write() (line 13) to pass in the
must be integer multiples of 16, or even              for creating pseudo random numbers                   start address of the array in which the
                                                      from the current system time. To launch              parameters for the calculations are
      Listing 1: Header File
            pi_libspe.h                                               Listing 2: For Loop for Controlling the SPEs
 01 #ifndef PI_LIBSPE_H_                               01 for ( i = 0; i < numspe; i++ ) {              09

 02 #define PI_LIBSPE_H_                               02    spe_par[i].rounds = rounds /               10      pthread_create( &spe_thread_
                                                            numspe;                                          handle[i], NULL, &spu_pthread, &spe_
                                                       03    gettimeofday( &tv, NULL );                      ctx[i] );
 04 typedef struct {
                                                       04    spe_par[i].seed = tv.tv_sec *              11
 05          float value;
                                                            1000000 + tv.tv_usec;                       12      myaddr = (uint64_t) &spe_par[i];
 06          uint64_t rounds;
                                                       05    spe_par[i].value = 0.0;                    13      spe_in_mbox_write( spe_ctx[i], (
 07          uint64_t seed;
                                                       06                                                    unsigned int * ) &myaddr, 2, SPE_
 08          char reserved[4];                                                                               MBOX_ANY_NONBLOCKING );
                                                       07    spe_ctx[i] = spe_context_
 09 } spe_par_t;                                                                                        14 }
                                                       08    spe_program_load( spe_ctx[i], &pi_
 11 #endif /*PI_LIBSPE_H_*/                                 libspe_spe );

76          ISSUE 99                                FEBRuARy 2009
                                                                        Programming the Cell                  Programming

                                                              between main and local memory      The .spuo suffix indicates an object file
(0,1 )                                        (1 ,1 )
                                                              (line 10). An SPE can manage up    based on the SPE instruction set. To cre-
                                                              to 32 tag IDs. Following this, the ate a single executable, the ppu‑embed‑
                                                              spu_mfcdma64() function trans-     spu tool converts the SPE program’s ob-
                                                              fers the parameter block that      ject code into a format that the PPE can
                                                              points to the main memory ad-      read:
                                                              dress previously retrieved from
                                                              the mailbox to the spe_par vari-     /opt/cell/toolchain/bin/U
                                                              able in local memory (line 12).      ppu‑embedspu pi_libspe_spe U
                                                              This function can handle both        pi_libspe_spe.spuo U
                                                              read and write DMA transfer. The     pi_libspe_spe.o
                                                              sixth argument defines the trans-
                                                              fer direction, as a comparison     The first parameter is the name used by
                                                              with line 18 shows.                the PPE to address the SPE program; it
 (0,0)                                            (1 ,0)         The spu_mfcdma64() func-        is identical to the name of the spe_pro‑
Figure 5: Approximating π with the Shotgun algo-              tion does not wait for the mem-    gram_handle_t type variable, which is de-
rithm: each red dot represents a pair of random fig-          ory transfer to complete. To en-   clared in the pi_libspe_ppe.c source file.
ures. If you count the dots in the blue circle and divide     sure data integrity, the SPE pro-     The second parameter is the name of
this number by the total number of dots, the result is        gram must wait until the DMA       the file containing the SPE object code,
an approximate value for π/4.                                 controller (Memory Flow Con-       and the third refers to the file where
                                                              troller, MFC) has finished; the    ppu‑embedspu will write the PPE-read-
stored. The SPE context defines which               mfc_read_tag_status_all() (line 14)          able object code. Finally, the developer
SPE receives the message; its start ad-             makes sure of this. The mfc_write_tag_       must link the PPE and SPE programs
dress is the first function argument.               mask() function (lines 19 and 20) tells      with the libspe2 library to create an
   When all SPE programs have termi-                us which of the 32 possible parallel DMA     executable:
nated, the PPE program releases the                 transfers it is waiting for.
memory for the SPE context in question.                Now the calculations can start, and         /opt/cell/toolchain/bin/U
Finally, the PPE program outputs the                the results, which are again stored in the     ppu‑gcc ‑o pi_libspe U
SPE’s results on the console.                       spe_par structure, make their way back         pi_libspe_ppe.o U
                                                    into main memory. Finally, line 22 re-         pi_libspe_spe.o ‑lspe2
SPE Culture                                         leases the tag ID.
The SPE’s work starts with the compute_                                                          If you have access to a computer with
pi() function (Listing 3). compute_pi()             instilling Life                              Cell hardware, you can simply copy the
expects a seed as an argument, which it             Creating the object files is the next step.  pi_libspe executable to it and execute the
will use to generate random numbers,                Because the PPE SPE processor cores use      program. If you are using the simulator,
and the number of pairs of numbers to               different instruction sets, two different    you will need to take a small detour.
calculate. The function returns an ap-              compilers must be used to build the
proximate value for π as a function                 source:                                      Simulated Entity
value. To allow this to happen, the                                                              Before you can launch the Cell Full Sys-
main() function (Listing 4) reads the                 /opt/cell/toolchain/bin/U                  tem Simulator, you must store the path
main memory address at which the                      spu‑gcc ‑o U                               to the simulator in the SYSTEMSIM_TOP
structure with the parameters for the                 pi_libspe_spe.spuo U                       environmental variable, which is /opt/
current SPE program is located. This ad-              pi_libspe_spe.c                            ibm/systemsim‑cell by default.
dress is also referred to as an effective             /opt/cell/toolchain/bin/U                     The following command wakes up the
address.                                              ppu‑gcc ‑c pi_libspe_ppe.c                 simulator:
   Because the spu_read_in_mbox()
function can only read single 32-bit                                       Listing 3: compute_pi Function
words, it must be called twice to retrieve            01 float compute_pi( long int seed,        11     x = (float) lrand48()/RAND_MAX;
the full 64-bit address (lines 7 and 8).                 uint64_t rounds )                       12     y = (float) lrand48()/RAND_MAX;
The variables declared inside the SPE                 02 {                                       13
program all lie within the SPE’s local                03   uint64_t i;                           14     if (( x * x + y * y ) < 1.0 ) {
memory space. Pointers also reference                 04   uint64_t in = 0;                      15        in++;
memory addresses in the local memory.                 05   float x, y;                           16     }
Because the Cell processor uses Big En-               06   unsigned long int h;                  17   }
dian architecture, the first word contains            07                                         18
the higher value, and the second word                 08   srand48( seed );                      19   return ( float ) 4.0 * in / rounds;
contains the lower value bits.                        09                                         20 }
   Next, the SPE program must reserve a
                                                      10   for ( i = 0; i < rounds; i++ ) {
tag ID to distinguish DMA data transfers

                                                                          FEBRuARy 2009                               ISSUE 99        77
Programming                       Programming the Cell

                                                                                            cision with which the result matches the
                                                                                            accepted value of π depends on the
                                                                                            quality of the pseudo-random numbers,
                                                                                            but also on the number of attempts. The
                                                                                            statistical error is approximately identi-
                                                                                            cal to the reciprocal value of the square
                                                                                            root of the number of attempts. Given
                                                                                            one-million attempts, the deviation be-
                                                                                            tween the approximated value and the
                                                                                            actual value of π is about one thou-
                                                                                            sandth, that is, about 0.003.
                                                                                               Another programming tool is the Data
                                                                                            Communication and Synchronization
                                                                                            (DaCS) library. Dacs abstracts a number
                                                                                            of the Cell processor’s special features,
                                                                                            which means that it potentially could be
                                                                                            ported to other accelerator architectures.
                                                                                            In contrast to this, the Accelerator Li-
                                                                                            brary Framework (ALF) implements a
                                                                                            programming model that swaps out indi-
                                                                                            vidual functions to the SPEs. DaCS and
                                                                                            ALF are included in the IBM developer
Figure 6: The Cell Full System Simulator by IBM makes physical Cell hardware unnecessary.   environment.
                                                                                               The Multicore Application Runtime
 /opt/ibm/systemsim‑cell/U                     text. To import an executable file stored    System (Mars) is an open source project
 bin/systemsim ‑g                              in the path /tmp/pi_libspe on the physi-     spearheaded by Sony [7]. Mars installs
                                               cal machine, use the command:                miniature kernels on the SPEs, and the
The ‑g option launches a Tcl/Tk-based                                                       kernels autonomously manage the exe-
graphical interface (Figure 6). To see the       callthru source   U                        cution of programs on “their” SPEs. Re-
various modes the simulator offers,              /tmp/pi_libspe > pi_libspe                 leased in November 2008, version 1.0.1
press the Mode button. For a simple                                                         is available as either an RPM package or
function test, Fast Mode is probably your      After modifying the permissions, as in       Tar archive. n
best choice. Clicking Go launches the          chmod u+x pi_libspe, you can then fi-
simulator. Now the console window will         nally launch the program:                                             INFO
show you the operating system booting                                                         [1] Top 500:
on the simulated Cell machine.                   ./pi_libspe 1000000 8
                                                                                              [2] Green 500:
  To load the program you want to run
                                                                                              [3] Cell SDK:
on the simulator, use the callthru com-        Running the program tells the simulation
mand. If you run the command without           machine to create a million pairs of ran-
any parameters, it will just show a help       dom numbers using eight SPEs. The pre-         [4] Cell system simulator:
            Listing 4: Main Function in the SPE Program
 01 int main ()                                14     mfc_read_tag_status_all();
                                                                                              [5] Barcelona Supercomputer Center:
 02 {                                          15
                                                                                              [6] Linux on the PS 3:
 03     uint32_t ea_block_h, ea_block_l;       16     spe_par.value = compute_pi( spe_
                                                    par.seed, spe_par.rounds );
 04     uint32_t tag_id;
 05     spe_par_t spe_par __attribute__        17

      ((aligned(16)));                         18     spu_mfcdma64( &spe_par, ea_block_h,
                                                                                              [7] Mars software and documentation:
                                                    ea_block_l, sizeof( spe_par_t ),    
                                                    tag_id, MFC_PUT_CMD );                        Sony‑PS3/mars
 07     ea_block_h = spu_read_in_mbox();
                                               19     mfc_write_tag_mask( 1 << tag_id );
 08     ea_block_l = spu_read_in_mbox();
                                               20     mfc_read_tag_status_all();                         Professor Peter Väterlein teaches at
                                                                                            THE AUTHOR

                                               21                                                        the university of Esslingen’s Faculty
 10     tag_id = mfc_tag_reserve();                                                                      of Information Technology. His spe-
                                               22     mfc_tag_release( tag_id );
 11                                                                                                      cialties are operating systems – pref-
                                               23                                                        erably Linux – and parallel computat-
 12     spu_mfcdma64( &spe_par, ea_block_h,
      ea_block_l, sizeof( spe_par_t ),         24     return 0;                                          ing from multicore processors to grid
      tag_id, MFC_GET_CMD );                   25 }                                                      computing. His homepage is http://
 13     mfc_write_tag_mask( 1 << tag_id );

78          ISSUE 99                           FEBRuARy 2009

Shared By: