The Eucalyptus Open-source Cloud-computing Eucalyptus Open-source Cloud-computing System ... sic principles of ... lutions within the EUCALYPTUS software framework. 2 Related Work Machine virtualization ...

  • Published on

  • View

  • Download

Embed Size (px)


<ul><li><p>The Eucalyptus Open-source Cloud-computing System</p><p>Daniel Nurmi, Rich Wolski, Chris GrzegorczykGraziano Obertelli, Sunil Soman, Lamia Youseff, Dmitrii Zagorodnov</p><p>Computer Science DepartmentUniversity of California, Santa Barbara</p><p>Santa Barbara, California 93106</p><p>AbstractCloud computing systems fundamentally provide ac-</p><p>cess to large pools of data and computational resourcesthrough a variety of interfaces similar in spirit to exist-ing grid and HPC resource management and program-ming systems. These types of systems offer a new pro-gramming target for scalable application developers andhave gained popularity over the past few years. However,most cloud computing systems in operation today are pro-prietary, rely upon infrastructure that is invisible to theresearch community, or are not explicitly designed to beinstrumented and modified by systems researchers.</p><p>In this work, we present EUCALYPTUS an open-source software framework for cloud computing that im-plements what is commonly referred to as Infrastructureas a Service (IaaS); systems that give users the ability torun and control entire virtual machine instances deployedacross a variety physical resources. We outline the ba-sic principles of the EUCALYPTUS design, detail impor-tant operational aspects of the system, and discuss archi-tectural trade-offs that we have made in order to allowEucalyptus to be portable, modular and simple to use oninfrastructure commonly found within academic settings.Finally, we provide evidence that EUCALYPTUS enablesusers familiar with existing Grid and HPC systems to ex-plore new cloud computing functionality while maintain-ing access to existing, familiar application developmentsoftware and Grid middle-ware.</p><p>1 IntroductionThere are many ways in which computational power</p><p>and data storage facilities are provided to users, rang-ing from a user accessing a single laptop to the alloca-tion of thousands of compute nodes distributed around theworld. Users generally locate resources based on a va-riety of characteristics, including the hardware architec-ture, memory and storage capacity, network connectivityand, occasionally, geographic location. Usually this re-source location process involves a mix of resource avail-</p><p>ability, application performance profiling, software ser-vice requirements, and administrative connections. Whilegreat strides have been made in the HPC and Grid Com-puting communities [15, 7] toward the creation of resourceprovisioning standards [14, 18, 33, 38], this process re-mains somewhat cumbersome for a user with complex re-source requirements.</p><p>For example, a user that requires a large number ofcomputational resources might have to contact severaldifferent resource providers in order to satisfy her re-quirements. When the pool of resources is finally deliv-ered, it is often heterogeneous, making the task of per-formance profiling and efficient use of the resources diffi-cult. While some users have the expertise required to ex-ploit resource heterogeneity, many prefer an environmentwhere resource hardware, software stacks, and program-ming environments are uniform. Such uniformity makesthe task of large-scale application development and de-ployment more accessible.</p><p>Recently, a number of systems have arisen that at-tempt to convert what is essentially a manual large-scaleresource provisioning and programming problem into amore abstract notion commonly referred to as elastic, util-ity, or cloud computing (we use the term cloud com-puting to refer to these systems in the remainder of thiswork). As the number and scale of cloud-computing sys-tems continues to grow, significant study is required to de-termine directions we can pursue toward the goal of mak-ing future cloud computing platforms successful. Cur-rently, most existing cloud-computing offerings are eitherproprietary or depend on software that is not amenableto experimentation or instrumentation. Researchers in-terested in pursuing cloud-computing infrastructure ques-tions have few tools with which to work.</p><p>The lack of research tools is unfortunate given thateven the most fundamental questions are still unanswered:what is the right distributed architecture for a cloud-computing system? What resource characteristics mustVM instance schedulers consider to make most efficientuse of the resources? How do we construct VM instance</p><p>1</p></li><li><p>networks that are flexible, well-performing, and secure?In addition, questions regarding the benefits of cloud com-puting remain difficult to address. Which application do-mains can benefit most from cloud computing systems andwhat interfaces are appropriate? What types of servicelevel agreements should cloud computing provide? Howcan cloud-computing systems be merged with more com-mon resource provisioning systems already deployed?</p><p>Cloud computing systems provide a wide varietyof interfaces and abstractions ranging from the abilityto dynamically provision entire virtual machines (i.e.,Infrastructure-as-a-Service systems such as Amazon EC2and others [4, 12, 27, 9, 30]) to flexible access to hostedsoftware services (i.e. Software-as-a-Service systemssuch as and others [37, 20, 21, 29]). All,however, share the notion that delivered resources shouldbe well defined, provide reasonably deterministic perfor-mance, and can be allocated and de-allocated on demand.We have focused our efforts on the lowest layer of cloudcomputing systems (IaaS) because here we can providea solid foundation on top of which language-, service-,and application-level cloud-computing systems can be ex-plored and developed.</p><p>In this work, we present EUCALYPTUS: an open-sourcecloud-computing framework that uses computational andstorage infrastructure commonly available to academic re-search groups to provide a platform that is modular andopen to experimental instrumentation and study. With EU-CALYPTUS, we intend to address open questions in cloudcomputing while providing a common open-source frame-work around which we hope a development communitywill arise. EUCALYPTUS is composed of several compo-nents that interact with one another through well-definedinterfaces, inviting researchers to replace our implementa-tions with their own or to modify existing modules. Here,we address several crucial cloud computing questions, in-cluding VM instance scheduling, VM and user data stor-age, cloud computing administrative interfaces, construc-tion of virtual networks, definition and execution of ser-vice level agreements (cloud/user and cloud/cloud), andcloud computing user interfaces. In this work, we willdiscuss each of these topics in more detail and provide afull description of our own initial implementations of so-lutions within the EUCALYPTUS software framework.</p><p>2 Related WorkMachine virtualization projects producing Virtual Ma-</p><p>chine (VM) hypervisor software [5, 6, 25, 40] have en-abled new mechanisms for providing resources to users.In particular, these efforts have influenced hardware de-sign [3, 22, 26] to support transparent operating sys-tem hosting. The right virtualization architecture re-mains an open field of study [2]): analyzing, optimiz-ing, and understanding the performance of virtualized sys-tems [23, 24, 31, 32, 41] is an active area of research.EUCALYPTUS implements a cloud computing operating</p><p>system that is, by design, hypervisor agnostic. However,the current implementation of the system uses Xen-basedvirtualization as its initial target hypervisor.</p><p>Grid computing must be acknowledged as an intellec-tual sibling of, if not ancestor to, cloud computing [7, 15,33, 38]. The original metaphor for a computational utility,in fact, gives grid computing its name. While grid com-puting and cloud computing share a services oriented ap-proach [16, 17] and may appeal to some of the same users(e.g., researchers and analysts performing loosely-coupledparallel computations), they differ in two key ways. First,grid systems are architected so that individual user re-quests can (and should) consume large fractions of thetotal resource pool [34]. Cloud systems often limit thesize of an individual request to be tiny fraction of the to-tal available capacity [4] and, instead, focus on scaling tosupport large numbers of users.</p><p>A second key difference concerns federation. From itsinception, grid computing took a middleware-based ap-proach as a way of promoting resource federation amongcooperating, but separate, administrative domains [14].Cloud service venues, to date, are unfederated. That is, acloud system is typically operated by a single (potentiallylarge) entity with the administrative authority to mandateuniform configuration, scheduling policies, etc. WhileEUCALYPTUS is designed to manage and control large col-lections of distributed resources, it conforms to the designconstraints governing cloud systems with regards to fed-eration of administrative authority and resource allocationpolicies.</p><p>Thanks in part to the new facilities provided by virtu-alization platforms, a large number of systems have beenbuilt using these technologies for providing scalable In-ternet services [4, 1, 8, 10, 11, 19, 37], that share in com-mon many system characteristics: they must be able torapidly scale up and down as workload fluctuates, supporta large number of users requiring resources on-demand,and provide stable access to provided resources over thepublic Internet. While the details of the underlying re-source architectures on which these systems operate arenot commonly published, EUCALYPTUS is almost cer-tainly shares some architectural features with these sys-tems due to shared objectives and design goals.</p><p>In addition to the commercial cloud computing of-ferings mentioned above (Amazon EC2/S3, Google Ap-pEngine,, etc.), which maintain a propri-etary infrastructure with open interfaces, there are open-source projects aimed at resource provisioning with thehelp of virtualization. Usher [30] is a modular open-source virtual machine management framework fromacademia. Enomalism [12] is an open-source cloud soft-ware infrastructure from a start-up company. VirtualWorkspaces [27] is a Globus-based [14] system for pro-visioning workspaces (i.e., VMs), which leverages sev-eral pre-existing solutions developed in the grid comput-</p><p>2</p></li><li><p>ing arena. The Cluster-on-demand [9] project focuses onthe provisioning of virtual machines for scientific com-puting applications. oVirt [35] is a Web-based virtual ma-chine management console.</p><p>While these projects produced software artifacts thatare similar to EUCALYPTUS, there are several differences.First, EUCALYPTUS was designed from the ground up tobe as easy to install and as non-intrusive as possible, with-out requiring sites to dedicate resources to it exclusively(one can even install it on a laptop for experimentation.)Second, the EUCALYPTUS software framework is highlymodular, with industry-standard, language-agnostic com-munication mechanisms, which we hope will encouragethird-party extensions to the system and community de-velopment. Third, the external interface to EUCALYPTUSis based on an already popular API developed by Amazon.Finally, EUCALYPTUS is unique among the open-sourceofferings in providing a virtual network overlay that bothisolates network traffic of different users and allows two ormore clusters to appear to belong to the same Local AreaNetwork (LAN).</p><p>Overall, we find that there are a great number of cloudcomputing systems in design and operation today that ex-pose interfaces to proprietary and closed software and re-sources, a smaller number of open-source cloud comput-ing offerings that typically require substantial effort and/ordedication of resources in order to use, and no system an-tecedent to EUCALYPTUS that has been designed specif-ically with support academic exploration and communityinvolvement as fundamental design goals.</p><p>3 EUCALYPTUS DesignThe architecture of the EUCALYPTUS system is simple,</p><p>flexible and modular with a hierarchical design reflectingcommon resource environments found in many academicsettings. In essence, the system allows users to start, con-trol, access, and terminate entire virtual machines usingan emulation of Amazon EC2s SOAP and Query in-terfaces. That is, users of EUCALYPTUS interact with thesystem using the exact same tools and interfaces that theyuse to interact with Amazon EC2. Currently, we supportVMs that run atop the Xen [5] hypervisor, but plan to addsupport for KVM/QEMU [6], VMware [40], and others inthe near future.</p><p>We have chosen to implement each high-level systemcomponent as a stand-alone Web service. This has thefollowing benefits: first, each Web service exposes a well-defined language-agnostic API in the form of a WSDLdocument containing both operations that the service canperform and input/output data structures. Second, wecan leverage existing Web-service features such as WS-Security policies for secure communication between com-ponents. There are four high-level components, each withits own Web-service interface, that comprise a EUCALYP-TUS installation:</p><p>CLCandWalrus</p><p>ClusterB</p><p>CC</p><p>NC</p><p>NC</p><p>NC</p><p>NC</p><p>Privatenetwork</p><p>ClusterA</p><p>CC</p><p>NC</p><p>NC</p><p>NC</p><p>NC</p><p>Privatenetwork</p><p>Publicnetwork</p><p>Figure 1. EUCALYPTUS employs a hierarchical de-sign to reflect underlying resource topologies.</p><p> Node Controller controls the execution, inspection,and terminating of VM instances on the host where itruns.</p><p> Cluster Controller gathers information about andschedules VM execution on specific node controllers,as well as manages virtual instance network.</p><p> Storage Controller (Walrus) is a put/get storageservice that implements Amazons S3 interface, pro-viding a mechanism for storing and accessing virtualmachine images and user data.</p><p> Cloud Controller is the entry-point into the cloudfor users and administrators. It queries node man-agers for information about resources, makes high-level scheduling decisions, and implements them bymaking requests to cluster controllers.</p><p>The relationships and deployment locations of eachcomponent within a typical small cluster setting are shownin Figure 1.</p><p>Node ControllerAn Node Controller (NC) executes on every node that</p><p>is designated for hosting VM instances. An NC queriesand controls the system software on its node (i.e., thehost operating system and the hypervisor) in response toqueries and control requests from its Cluster Controller.</p><p>An NC makes queries to discover the nodes physicalresources the number of cores, the size of memory, theavailable disk space as well as to learn about the state ofVM instances on the node (although an NC keeps track ofthe instances that it controls, instances may be started andstopped through mechanisms beyond NCs control). Theinformation thus collected is propagated up to the ClusterController in responses to describeResource and describe-Instances requests.</p><p>3</p></li><li><p>Cluster Controllers control VM instances on a node bymaking runInstance and terminateInstance requests to thenodes NC. Upon verifying the authorization e.g., onlythe owner of an instance or an...</p></li></ul>