The workflow of Rainbow is shown in **Figure 3**. Multiple data drives are shipped to Amazon. After the BAM or FASTQ files have been uploaded to S3, large FASTQ files are split into smaller files in parallel. Then multiple clusters are launched in the cloud, with each cluster processing a single sample. Crossbow is responsible for mapping reads to the reference sequence and for SNP calling. The SNPs for all samples are then combined by a Perl script. When the analysis is complete, the results can either be downloaded directly or exported via Amazon Export. We applied Rainbow to analyze the 44 subjects, with 0.55–1 billion pairended 100 bp short reads per sample. The running environments were as follows. For step #1 in **Figure 3**, we chose the Amazon m1.large instance, which has two CPUs, 7.5 GB memory, and two 420 GB instance drives. For Crossbow run, each compute cluster has 40 c1.xlarge nodes as recommended by the Crossbow developers. Each c1.xlarge node has 8 CPUs, 7 GB memory, and 1690 GB instance storage.

The performance of Rainbow is summarized in **Figure 4**. In a 320-CPU (=40 × 8) cluster, the alignment of billions of reads takes between 0.8 and 1.6 h. The linear relationship shown in **Figure 4** indicates that the sequence data blocks in the Hadoop distributed file system (HDFS) are physically local to the nodes that processed them, which reduces virtual I/O delays. The SOAPsnp running time ranges from 1 to 1.8 h, which takes a little longer than the alignment. All EC2 instances and clusters are terminated immediately after the jobs are finished. On average, it costs less than 120 US dollars to analyze each subject, and the total cost for analyzing those 44 subjects was 5800 US dollars, including data import. More important than the cost is the ability to scale Rainbow up or down, so that the analyses can be accomplished in a reasonably short amount time, regardless of sample size. No upfront investment in

**Figure 3.** The Rainbow pipeline. S3 centralizes data storage, including inputs, intermediate results, and outputs. Alignment and SNP call are performed by Crossbow in a cluster with multiple nodes.

**Figure 4.** Top panel: running time of bowtie alignment vs. the number of paired sequence reads. Bottom panel: running time of SOAPsnp vs. the number of paired mapped reads. Note: each cluster consists of 40 c1.xlarge instances (8 CPUs per c1.xlarge instance).

infrastructure is required and there is no additional administrative costs involved using Amazon cloud. Rainbow is a scalable, cost-effective, and open-source tool for large-scale WGS data analysis. It is available for third-party implementation and use, and can be downloaded from the Rainbow website.

In order to access the Rainbow cloud pipeline, the user must first set up an AWS account (http://aws.amazon.com/). Once registered, the user needs to sign up for Amazon EC2, S3, EMR, and SES services. The user can then start an instance based on the public AMI: ami-0f1f9866 in US-East (N. Virginia); or ami-b6bc89f3 in US-West (N. California). All required software is already preinstalled and configured in the AMI. Then, the user can connect to the instance and configure EC2, EMR, and S3cmd command-line tools. After a successful connection to the instance has been established, the user needs to prepare a sample manifest file in order to run Rainbow.  Each subject has a corresponding entry in the manifest file and each entry consists of three fields separated by spaces or tabs: (1) a unique identifier; (2) locations of the raw reads in S3; and (3) an output folder in S3. Each individual step in the Rainbow workflow uses this same manifest file as input, thus all output files are named and stored consistently. After the creation of the manifest file, the user just needs to run a couple of command lines and all the analyses will be done automatically in the cloud.

Analyzing large datasets in the cloud is different from performing the same analysis in a local environment. When developing Rainbow, we have learned the following lessons.

#### **5. Cloud computing hurdles**

**Figure 3.** The Rainbow pipeline. S3 centralizes data storage, including inputs, intermediate results, and outputs.

**Figure 4.** Top panel: running time of bowtie alignment vs. the number of paired sequence reads. Bottom panel: running time of SOAPsnp vs. the number of paired mapped reads. Note: each cluster consists of 40 c1.xlarge instances (8 CPUs

per c1.xlarge instance).

Alignment and SNP call are performed by Crossbow in a cluster with multiple nodes.

40 Cloud Computing - Architecture and Applications

Albeit relatively new, cloud computing holds great promise in effectively addressing big data storage and analysis problems in NGS data analysis. Despite the potential gains achieved, there are also several important issues that need to be addressed. Below, we present the main hurdles on the adoption of cloud computing.

#### **5.1. Big data transfer**

To analyze the NGS data in the cloud, data have to be transferred across the wired network and uploaded onto AWS. The volume and complexity of NGS data have exponentially increased, giving rise to issues related to data analysis, management, and transfer to the cloud [64–66]. For example, WGS of 400 subjects at 30× coverage will generate approximately 100 TB raw sequence reads in FASTQ format. In the future, more and more sequencing projects would generate ultra-large volumes of biological data and thus require bioinformatics clouds for big data storage, sharing, and analysis. One of the most challenging issues of cloud computing is data transfer. Transferring vast amounts of biological data to the cloud is a significant bottleneck in cloud computing.

The speed of data transfer is usually slow and at present there are not many solutions available for moving the huge amount of data to cloud. Therefore, we need more efficient data transfer technologies in cloud computing. According to CloudHarmoy's report on download speed relative to the year 2010 (http://blog.cloudharmony.com/2010/02/cloud-speed-test-results. html), the download speed from Amazon AWS EC2 in North Virginia (U.S.) was 2.95 Mb/s, which corresponds to downloading a 10 GB file in 29,116 s (\* 8 h). Therefore, data transfer is a serious bottleneck in NGS data analysis on cloud service. To deal with the data transfer issue, Aspera (http://www.asperasoft.com/) has developed the fast and secure protocol (FASP) for data transfer with a speed of up to 5 GB/s. Ideally, using FASP, the user can download a 10 GB file in 17.2 s, which is a revolutionary improvement. But still it cannot transfer data at the TB scale. Alternatively, sequencing service providers such as BGI and Illumina offer a service in which they deliver a hard disk drive (HDD) containing the sequencing data. However, the shipping has limitations aside from the time taken up by travel, and once a package (of hard drives) is stamped and sealed, researchers cannot control when (and sometimes if) the parcel arrived or what condition it arrived. This service cannot guarantee safe transfer of data, since the HDD can get lost, stolen, or physical errors can occur.

#### **5.2. Most bioinformatics tools are not cloud-aware**

Most bioinformatics software tools are written for desktop (rather than cloud) applications and are therefore not provided as cloud-based web services accessible via the Web, making it infeasible to perform complex bioinformatics tasks in the cloud. For instance, bowtie [61] is one of the most popular mapping algorithms, but it requires that input files are stored on local disk when mapping reads and is not compatible with Amazon S3. Even if you run bowtie in an EC2 instance, the raw FASTQ files have to be fetched to an elastic block store (EBS) volume that is attached to the EC2 instance. In other words, bowtie does not have built-in support for S3. Spliced transcripts alignment to a reference (STAR) [67–69] is a popular RNA-seq mapper that performs highly accurate spliced sequence alignment at an ultrafast speed. However, it is not cloud-friendly either. Like bowtie, STAR does not take advantage of AWS cloud services, and cannot work with S3 either. Unfortunately, the majority of bioinformatics tools are developed without native support for cloud computing.

MapReduce [70, 71], developed by Google, is an easy-to-use and general-purpose parallel programming model that is suitable for large data set analysis on a commodity hardware cluster. MapReduce is a software framework, written in Java, designed to run over a cluster of machines in a distributed way. A MapReduce program is composed of a user-defined map function and a reduce function. When a program that is implemented with the map and reduce functions has been launched, the map function processes each key/value pair and produces a list of intermediate key/value pairs, while the reduce function aggregates all the intermediate values with the same keys. MapReduce is an important advancement in cloud computing because it can process huge data sets quickly and safely using commodity hardware.

Hadoop, comprised of MapReduce and the Hadoop distributed file system (HDFS), is based on a strategy of colocating data and processing to significantly accelerate computing performance [60]. Hadoop allows for the distributed processing of large datasets across multiple computer nodes, supports big data scaling, and enables fault-tolerant parallel analysis. The Hadoop framework has been recently deemed as the most suitable method for handling bioinformatics data [70]. Unfortunately, many traditional bioinformatics tools and algorithms have to be redesigned and implemented in order to support and benefit from Hadoop MapReduce infrastructure. Even with the help of the corresponding developers, it will take a while for most bioinformatics tools currently available to add this feature.

Apache Spark™ (https://spark.apache.org/) is a fast and general engine for large-scale data processing, natively supported in Amazon EMR. Apache Spark supports a variety of languages, including Java, Scala, and Python, for developers to build applications. Hadoop and Apache Spark are both big data frameworks, but they do not really serve the same purposes. Hadoop is essentially a distributed data infrastructure. It distributes massive data collections across multiple nodes within a cluster of commodity servers, Spark, on the other hand, is a data-processing tool that operates on those distributed data collections; it does not do distributed storage. To study the utility of Apache Spark in the genomic context, SparkSeq was created [72]. It is a general-purpose, flexible, and easily extendable library for genomic cloud computing, and can be used to build genomic analysis pipelines in Scala and run them in an interactive way. Recently, SparkBWA [73] was introduced; a new tool that exploits Spark to boost the performance of one of the most widely adopted sequence aligner, the Burrows-Wheeler Aligner (BWA). It is hoped more Apache Spark-based bioinformatics algorithms will be developed for large-scale genomic data analysis in the future.

#### **5.3. Open clouds for bioinformatics**

**5.1. Big data transfer**

42 Cloud Computing - Architecture and Applications

neck in cloud computing.

To analyze the NGS data in the cloud, data have to be transferred across the wired network and uploaded onto AWS. The volume and complexity of NGS data have exponentially increased, giving rise to issues related to data analysis, management, and transfer to the cloud [64–66]. For example, WGS of 400 subjects at 30× coverage will generate approximately 100 TB raw sequence reads in FASTQ format. In the future, more and more sequencing projects would generate ultra-large volumes of biological data and thus require bioinformatics clouds for big data storage, sharing, and analysis. One of the most challenging issues of cloud computing is data transfer. Transferring vast amounts of biological data to the cloud is a significant bottle-

The speed of data transfer is usually slow and at present there are not many solutions available for moving the huge amount of data to cloud. Therefore, we need more efficient data transfer technologies in cloud computing. According to CloudHarmoy's report on download speed relative to the year 2010 (http://blog.cloudharmony.com/2010/02/cloud-speed-test-results. html), the download speed from Amazon AWS EC2 in North Virginia (U.S.) was 2.95 Mb/s,

serious bottleneck in NGS data analysis on cloud service. To deal with the data transfer issue, Aspera (http://www.asperasoft.com/) has developed the fast and secure protocol (FASP) for data transfer with a speed of up to 5 GB/s. Ideally, using FASP, the user can download a 10 GB file in 17.2 s, which is a revolutionary improvement. But still it cannot transfer data at the TB scale. Alternatively, sequencing service providers such as BGI and Illumina offer a service in which they deliver a hard disk drive (HDD) containing the sequencing data. However, the shipping has limitations aside from the time taken up by travel, and once a package (of hard drives) is stamped and sealed, researchers cannot control when (and sometimes if) the parcel arrived or what condition it arrived. This service cannot guarantee safe transfer of data, since

Most bioinformatics software tools are written for desktop (rather than cloud) applications and are therefore not provided as cloud-based web services accessible via the Web, making it infeasible to perform complex bioinformatics tasks in the cloud. For instance, bowtie [61] is one of the most popular mapping algorithms, but it requires that input files are stored on local disk when mapping reads and is not compatible with Amazon S3. Even if you run bowtie in an EC2 instance, the raw FASTQ files have to be fetched to an elastic block store (EBS) volume that is attached to the EC2 instance. In other words, bowtie does not have built-in support for S3. Spliced transcripts alignment to a reference (STAR) [67–69] is a popular RNA-seq mapper that performs highly accurate spliced sequence alignment at an ultrafast speed. However, it is not cloud-friendly either. Like bowtie, STAR does not take advantage of AWS cloud services, and cannot work with S3 either. Unfortunately, the majority of bioinformatics tools are devel-

MapReduce [70, 71], developed by Google, is an easy-to-use and general-purpose parallel programming model that is suitable for large data set analysis on a commodity hardware

8 h). Therefore, data transfer is a

which corresponds to downloading a 10 GB file in 29,116 s (\*

the HDD can get lost, stolen, or physical errors can occur.

**5.2. Most bioinformatics tools are not cloud-aware**

oped without native support for cloud computing.

Currently, the largest cloud computing provider is Amazon, which provides commercial clouds for processing big data. Additionally, Google also provides a cloud platform to allow users to develop and host applications, and to store and analyze data. However, commercial clouds are not yet able to provide ample data and software for bioinformatics analysis. By placing public biological database and software into the cloud and delivering them as services, data and software can be seamlessly and easily integrated into the cloud. AWS hosts a variety of public data sets for free access (https://aws.amazon.com/public-data-sets/). All public datasets in AWS are delivered as services. Previously, large data sets, such as the mapping of the human genome, required hours or days to locate, download, customize, and analyze. Now, anyone can access these data sets via the AWS centralized data repository from any Amazon EC2 instance or Amazon EMR cluster. Google Genomics also helps the life science community organize the world's genomic data and make them accessible and useful.

In the era of big data, however, only a tiny amount of biological data is accessible in the cloud at present (only AWS, including GenBank [35], Ensembl [74], 1000 Genomes [20], etc.) and the vast majority of data are still deposited in conventional biological databases. It is difficult for commercial clouds to keep pace with the emerging needs from academic research, opening up the demand for specific open clouds for bioinformatics studies. Needless to say, open access and public availability of data and software are of great significance to biological science. To satisfy the need for big data storage, sharing, and analysis with lower cost and higher efficiency, it is essential that a large number of biological data as well as a wide variety of bioinformatics tools should be publicly accessible in the cloud and delivered as services. Therefore, future efforts should be devoted to building open bioinformatics clouds for the bioinformatics community. GenomeSpace [75] is a cloud-based, cooperative community resource that currently supports the streamlined interaction of 20 bioinformatics tools and data resources. To facilitate integrative analysis by nonprogrammers, it offers a growing set of 'recipes', short workflows to guide investigators through high-utility analysis tasks. The potential benefits of open bioinformatics clouds include maximizing the scope for data sharing, easing large-scale data integration, and harnessing collective intelligence for knowledge discovery.

#### **5.4. Security and privacy**

The many characteristics of cloud computing have made the long-dreamed vision of "computing as a utility" a reality. The cloud computing offers scalable and competitively priced computing resources for the analysis and storage of data from large-scale genomics studies, but it must also ensure that genetic data coming from human subjects are hosted in a context that is both secure and compliant with regulations [76]. When deciding whether to move the analyses into the cloud or not, potential cloud users need to weigh all the factors including system performance, service availability, cost, and most importantly, data security. Genomics data extracted from clinical samples are sensitive data and present unprecedented requirements of privacy and security [11, 77]. In general, there are concerns that genomics and clinical data managed through a cloud are susceptible to loss, leakage, theft, unauthorized access, and attacks. The centralized storage and shared tenancy of physical storage space means the cloud users are at higher risk of disclosure of their sensitive data to unwanted parties. A secure protection scheme will be necessary to protect the sensitive information from medical records. There is considerable amount of work to enforce data protection against security attacks.

However, the question of security in cloud computing is intrinsically complicated. Cloud computing is built on the top of existing architectures and techniques such as SaaS and distributed computing. When combining all the benefits of these architectures and techniques, cloud computing also inherits almost all of their security issues at various levels of the system stack. When cloud users move their applications from within their enterprise/organization boundary into the open cloud, they will lose physical control over their data, and traditional security protection mechanisms such as firewalls are no longer applicable to cloud applications. As a result, cloud users have to heavily rely upon the service providers for data privacy and security protection. In cloud computing, the data and applications from different customers reside on the same physical computing resources. This fact will inevitably bring forth more security risks in the sense that any intentional or inadvertent misbehavior by one cloud user would make other coresidences victims.

#### **6. Conclusion**

Now, anyone can access these data sets via the AWS centralized data repository from any Amazon EC2 instance or Amazon EMR cluster. Google Genomics also helps the life science

In the era of big data, however, only a tiny amount of biological data is accessible in the cloud at present (only AWS, including GenBank [35], Ensembl [74], 1000 Genomes [20], etc.) and the vast majority of data are still deposited in conventional biological databases. It is difficult for commercial clouds to keep pace with the emerging needs from academic research, opening up the demand for specific open clouds for bioinformatics studies. Needless to say, open access and public availability of data and software are of great significance to biological science. To satisfy the need for big data storage, sharing, and analysis with lower cost and higher efficiency, it is essential that a large number of biological data as well as a wide variety of bioinformatics tools should be publicly accessible in the cloud and delivered as services. Therefore, future efforts should be devoted to building open bioinformatics clouds for the bioinformatics community. GenomeSpace [75] is a cloud-based, cooperative community resource that currently supports the streamlined interaction of 20 bioinformatics tools and data resources. To facilitate integrative analysis by nonprogrammers, it offers a growing set of 'recipes', short workflows to guide investigators through high-utility analysis tasks. The potential benefits of open bioinformatics clouds include maximizing the scope for data sharing, easing large-scale

community organize the world's genomic data and make them accessible and useful.

data integration, and harnessing collective intelligence for knowledge discovery.

The many characteristics of cloud computing have made the long-dreamed vision of "computing as a utility" a reality. The cloud computing offers scalable and competitively priced computing resources for the analysis and storage of data from large-scale genomics studies, but it must also ensure that genetic data coming from human subjects are hosted in a context that is both secure and compliant with regulations [76]. When deciding whether to move the analyses into the cloud or not, potential cloud users need to weigh all the factors including system performance, service availability, cost, and most importantly, data security. Genomics data extracted from clinical samples are sensitive data and present unprecedented requirements of privacy and security [11, 77]. In general, there are concerns that genomics and clinical data managed through a cloud are susceptible to loss, leakage, theft, unauthorized access, and attacks. The centralized storage and shared tenancy of physical storage space means the cloud users are at higher risk of disclosure of their sensitive data to unwanted parties. A secure protection scheme will be necessary to protect the sensitive information from medical records. There is considerable amount of work to enforce data protection against security attacks.

However, the question of security in cloud computing is intrinsically complicated. Cloud computing is built on the top of existing architectures and techniques such as SaaS and distributed computing. When combining all the benefits of these architectures and techniques, cloud computing also inherits almost all of their security issues at various levels of the system stack. When cloud users move their applications from within their enterprise/organization boundary into the open cloud, they will lose physical control over their data, and traditional security protection mechanisms such as firewalls are no longer applicable to cloud applications. As a

**5.4. Security and privacy**

44 Cloud Computing - Architecture and Applications

Pharmacogenomics is an important branch of genomics that studies the impact of SNP on drug response in patients, the toxicity or efficacy of a drug, and the development of diseases [78, 79]. Pharmacogenomics aims to improve drug therapy with respect to the patients' genotype, to ensure maximum efficacy with minimal adverse effects, and is at the basis of the idea of "precision medicine" where drugs are chosen or optimized to meet the genetic profile of each patient. Thus, the presence (or the absence) of specific SNPs may be used as a clinical marker to predict drug effectiveness, and to foresee the drug responses of individuals with different SNPs. Precision medicine also opens up the possibility to treat diseases based on the genetic makeup of the patient. Identification of the genetic defect underlying early-onset diabetes is important for determining the specific diabetes subtype, providing personalized treatment. The study by Artuso et al. [14] has revealed that NGS provides a highly sensitive method for identification of variants in a new-set of "driver genes" causing diabetes. NGS can be used to analyze the comprehensive landscape of genetic alterations, including known disease-causing gene fusions in transcripts, which brings new insights to study diseases with a highly complex and heterogeneous genetic composition such as cancer [19]. Therefore, NGS facilitates precision medicine and changes the paradigm of cancer therapy, and holds expanded promise for its diagnostic, prognostic, and therapeutic applicability in various diseases.

The substantial decrease in the cost of NGS techniques in the past decade has dramatically reshaped the biomedical research and has led to its rapid adoption in biological research and drug development [3, 4]. Nowadays, massive amount of data, targeting a variety of biological questions, can be generated quickly using NGS platforms. These data range from the function and regulation of genes, the clinical diagnosis and treatment of diseases, to the omics profiling of individual patients for precision medicine. To better understand the association between SNPs and diseases, and to gain deeper insights into the relation between drug response and genetic variations, large-scale sequencing projects are continuously being initiated in research institutes and pharmaceutical companies. The availability of NGS and the genomics studies of large populations are producing an increasing amount of data. However, the storage, preprocessing, and analysis of NGS data are becoming the main bottleneck in the analysis pipeline. With the exponential increase in volume and complexity of NGS data, cluster or high performance computing (HPC) systems are essential for the analysis of large amounts of NGS data. But the associated costs with the infrastructure itself and the maintenance personnel will likely be prohibitive for small institutions or laboratories, and even too much to swallow for big institutions and pharmaceutical companies.

This chapter provides a useful perspective on cloud computing and helps researchers, who plan to analyze their NGS data on cloud services, to gain an understanding of the basic concept. As discussed earlier, cloud computing drives down infrastructure costs both up-front and on an on-going basis. It offers many operational advantages, such as fast completion of massive computational projects and infrastructures can be set up in minutes rather than months. There are growing demands for cloud-based NGS data analysis, which constitutes a good alternative for researchers with little interest in investing in a cluster system. Cloud computing is becoming a technology mature enough for its use in genomic researches. The use of large datasets, the demanding analysis algorithms, and the urgent need for computational resources, make large-scale sequencing projects an attractive test-case for cloud computing. It is likely that in the future, researchers will be able to analyze their NGS data on cloud services for a cost low enough to eliminate the need to introduce clusters into their laboratories. However, we must emphasize that there are a number of open issues and problems associated with the use of cloud, such as privacy and security, especially when managing patients' clinical data. Data transfer of large amounts of NGS data to the cloud remains a challenge and bottleneck in NGS data analysis. Furthermore, many traditional NGS tools are not designed to use in cloud environment, and thus cannot harness the benefits of cloud services. With the ever-increasing demand, commercial companies and bioinformaticians will develop more NGS analysis tools, using APIs offered by service providers, in the foreseeable future, making NGS data analysis in the cloud more efficient, friendly, and secure.

#### **Author details**

Shanrong Zhao1 \*, Kirk Watrous<sup>2</sup> , Chi Zhang1 and Baohong Zhang1

\*Address all correspondence to<EMAIL_ADDRESS>
1 Early Clinical Development, Pfizer Worldwide Research and Development, Cambridge, MA, USA

2 Business Technology, Pfizer Worldwide Research and Development, Groton, Connecticut, USA

#### **References**

[4] Yadav NK, Shukla P, Omer A, Pareek S, Srivastava AK, Bansode FW, et al. Next generation sequencing: potential and application in drug discovery. Sci World J. 2014;2014:802437.

This chapter provides a useful perspective on cloud computing and helps researchers, who plan to analyze their NGS data on cloud services, to gain an understanding of the basic concept. As discussed earlier, cloud computing drives down infrastructure costs both up-front and on an on-going basis. It offers many operational advantages, such as fast completion of massive computational projects and infrastructures can be set up in minutes rather than months. There are growing demands for cloud-based NGS data analysis, which constitutes a good alternative for researchers with little interest in investing in a cluster system. Cloud computing is becoming a technology mature enough for its use in genomic researches. The use of large datasets, the demanding analysis algorithms, and the urgent need for computational resources, make large-scale sequencing projects an attractive test-case for cloud computing. It is likely that in the future, researchers will be able to analyze their NGS data on cloud services for a cost low enough to eliminate the need to introduce clusters into their laboratories. However, we must emphasize that there are a number of open issues and problems associated with the use of cloud, such as privacy and security, especially when managing patients' clinical data. Data transfer of large amounts of NGS data to the cloud remains a challenge and bottleneck in NGS data analysis. Furthermore, many traditional NGS tools are not designed to use in cloud environment, and thus cannot harness the benefits of cloud services. With the ever-increasing demand, commercial companies and bioinformaticians will develop more NGS analysis tools, using APIs offered by service providers, in the foreseeable future, making

NGS data analysis in the cloud more efficient, friendly, and secure.

, Chi Zhang1

sound barrier of human genetics. Mutagenesis. 2014;29(5):303-10.

sequencing technologies. Nat Rev Genet. 2016;17(6):333-51.

Drug Discov Today. 2011;16(11-12):512-9.

1 Early Clinical Development, Pfizer Worldwide Research and Development, Cambridge,

2 Business Technology, Pfizer Worldwide Research and Development, Groton, Connecticut, USA

[1] Bahassi el M, Stambrook PJ. Next-generation sequencing technologies: breaking the

[2] Goodwin S, McPherson JD, McCombie WR. Coming of age: ten years of next-generation

[3] Woollard PM, Mehta NA, Vamathevan JJ, Van Horn S, Bonde BK, Dow DJ. The application of next-generation sequencing technologies to drug discovery and development.

and Baohong Zhang1

\*, Kirk Watrous<sup>2</sup>

46 Cloud Computing - Architecture and Applications

\*Address all correspondence to<EMAIL_ADDRESS>
**Author details**

Shanrong Zhao1

MA, USA

**References**

[35] Clark K, Karsch-Mizrachi I, Lipman DJ, Ostell J, Sayers EW. GenBank. Nucleic Acids Res. 2016;44(D1):D67-72.

[20] Zheng-Bradley X, Flicek P. Applications of the 1000 genomes project resources. Brief

[21] Shringarpure SS, Carroll A, De La Vega FM, Bustamante CD. Inexpensive and highly reproducible cloud-based variant calling of 2,535 human genomes. PLoS One.

[22] Nagalakshmi U, Waern K, Snyder M. RNA-seq: a method for comprehensive transcriptome analysis. Curr Protoc Mol Biol (edited by Frederick M Ausubel [et al]). 2010;Chapter

[23] Mutz KO, Heilkenbrinker A, Lonne M, Walter JG, Stahl F. Transcriptome analysis using

[24] Zhao S, Zhang B, Zhang Y, Gordon W, Du S, Paradis T, et al. Bioinformatics for RNA-Seq Data Analysis. In: Abdurakhmonov I, editor. Bioinformatics—Updated Features and

[25] Picelli S. Single-cell RNA-sequencing: the future of genome biology is now. RNA

[26] Zhao S, Fung-Leung WP, Bittner A, Ngo K, Liu X. Comparison of RNA-seq and microarray in transcriptome profiling of activated T cells. PLoS One. 2014;9(1):e78644.

[27] Byron SA, Van Keuren-Jensen KR, Engelthaler DM, Carpten JD, Craig DW. Translating RNA sequencing into clinical diagnostics: opportunities and challenges. Nat Rev Genet.

[28] Zhao S, Xi L, Quan J, Xi H, Zhang Y, von Schack D, et al. QuickRNASeq lifts large-scale RNA-seq data analyses to the next level of automation and interactive visualization.

[29] Hoeijmakers WA, Bartfai R, Stunnenberg HG. Transcriptome analysis using RNA-seq.

[30] Stegle O, Teichmann SA, Marioni JC. Computational and analytical challenges in single-

[31] Jeong HM, Lee S, Chae H, Kim R, Kwon MJ, Oh E, et al. Efficiency of methylated DNA immunoprecipitation bisulphite sequencing for whole-genome DNA methylation anal-

[32] Li N, Ye M, Li Y, Yan Z, Butcher LM, Sun J, et al. Whole genome DNA methylation analysis based on high throughput sequencing technology. Methods. 2010;52(3):203-12.

[33] Massie CE, Mills IG. Mapping protein-DNA interactions using ChIP-sequencing. Meth

[34] Nakato R, Shirahige K. Recent advances in ChIP-seq analysis: from quality management to whole-genome annotation. Brief Bioinformatics. 2016; pii: bbw023. [Epub

next-generation sequencing. Curr Opin Biotechnol. 2013;24(1):22-30.

Funct Genomics. 2016; pii: elw027. [Epub ahead of print]

2015;10(6):e0129277.

48 Cloud Computing - Architecture and Applications

Applications: InTech; 2016. pp. 125-49.

4:Unit 4.11.1-3.

Biol.2016;21:1-14.

2016;17(5):257-71.

BMC Genomics. 2016;17:39.

Methods Mol Biol. 2013;923:221-39.

ysis. Epigenomics. 2016;8(8):1061-77.

Mol Biol. 2012;809:157-73.

ahead of print]

cell transcriptomics. Nat Rev Genet. 2015;16(3):133-45.

[68] Dobin A, Gingeras TR. Mapping RNA-seq Reads with STAR. Curr Protoc Bioinformatics (edited by Andreas D Baxevanis [et al]). 2015;51:11.4.1-9.

[51] Chae H, Rhee S, Nephew KP, Kim S. BioVLAB-MMIA-NGS: microRNA-mRNA integrated analysis using high-throughput sequencing data. Bioinformatics. 2015;31(2):265-7.

[52] Lee H, Yang Y, Chae H, Nam S, Choi D, Tangchaisin P, et al. BioVLAB-MMIA: a cloud environment for microRNA and mRNA integrated analysis (MMIA) on Amazon EC2.

[53] Naccache SN, Federman S, Veeraraghavan N, Zaharia M, Lee D, Samayoa E, et al. A cloud-compatible bioinformatics pipeline for ultrarapid pathogen identification from

[55] McKenna A, Hanna M, Banks E, Sivachenko A, Cibulskis K, Kernytsky A, et al. The genome analysis toolkit: a MapReduce framework for analyzing next-generation DNA

[56] Krampis K, Booth T, Chapman B, Tiwari B, Bicak M, Field D, et al. CloudBioLinux: preconfigured and on-demand bioinformatics computing for the genomics community.

[57] Afgan E, Baker D, Coraor N, Chapman B, Nekrutenko A, Taylor J. Galaxy CloudMan: delivering cloud compute clusters. BMC Bioinformatics. 2010;11(Suppl 12):S4.

[58] Bhuvaneshwar K, Sulakhe D, Gauba R, Rodriguez A, Madduri R, Dave U, et al. A case study for cloud based high throughput analysis of NGS data using the globus genomics

[59] Madduri RK, Sulakhe D, Lacinski L, Liu B, Rodriguez A, Chard K, et al. Experiences building globus genomics: a next-generation sequencing analysis service using Galaxy,

[60] Taylor RC. An overview of the Hadoop/MapReduce/HBase framework and its current

[61] Langmead B, Trapnell C, Pop M, Salzberg SL. Ultrafast and memory-efficient alignment of short DNA sequences to the human genome. Genome Biol. 2009;10(3):R25.

[62] Li R, Li Y, Fang X, Yang H, Wang J, Kristiansen K, et al. SNP detection for massively

[63] Bragg L, Tyson GW. Metagenomics using next-generation sequencing. Meth Mol Biol.

Globus, and Amazon Web Services. Concurr Comput. 2014;26(13):2266-79.

applications in bioinformatics. BMC Bioinformatics. 2010;11(Suppl 12):S1.

parallel whole-genome resequencing. Genome Res. 2009;19(6):1124-32.

[64] Marx V. Biology: the big challenges of big data. Nature. 2013;498(7453):255-60.

[66] Mardis ER. The challenges of big data. Dis Models Mech. 2016;9(5):483-5.

universal RNA-seq aligner. Bioinformatics. 2013;29(1):15-21.

[65] Fan J, Han F, Liu H. Challenges of big data analysis. Natl Sci Rev. 2014;1(2):293-314.

[67] Dobin A, Davis CA, Schlesinger F, Drenkow J, Zaleski C, Jha S, et al. STAR: ultrafast

next-generation sequencing of clinical samples. Genome Res. 2014;24(7):1180-92. [54] Li H, Durbin R. Fast and accurate long-read alignment with Burrows-Wheeler trans-

IEEE Trans Nanobioscience. 2012;11(3):266-72.

50 Cloud Computing - Architecture and Applications

form. Bioinformatics. 2010;26(5):589-95.

BMC Bioinformatics. 2012;13:42.

2014;1096:183-201.

sequencing data. Genome Res. 2010;20(9):1297-303.

system. Comput Struct Biotechnol J. 2015;13:64-74.

## **Green-Aware Virtual Machine Migration Strategy in Sustainable Cloud Computing Environments**

Xiaoying Wang, Guojing Zhang, Mengqin Yang and Lei Zhang

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/67350

#### **Abstract**

As cloud computing develops rapidly, the energy consumption of large-scale datacenters becomes unneglectable, and thus renewable energy is considered as the extra supply for building sustainable cloud infrastructures. In this chapter, we present a green-aware virtual machine (VM) migration strategy in such datacenters powered by sustainable energy sources, considering the power consumption of both IT functional devices and cooling devices. We define an overall optimization problem from an energy-aware point of view and try to solve it using statistical searching approaches. The purpose is to utilize green energy sufficiently while guaranteeing the performance of applications hosted by the datacenter. Evaluation experiments are conducted under realistic workload traces and solar energy generation data in order to validate the feasibility. Results show that the green energy utilization increases remarkably, and more overall revenues could be achieved.

**Keywords:** virtual machine migration, resource management, power management, renewable energy aware

#### **1. Introduction**

Large-scale datacenters, as the key infrastructure of cloud environments, usually own massive computing and storage resources in order to provide online services for thousands of millions of customers simultaneously. This leads to significant energy consumption, and thus high carbon footprint will be produced. Recent reports estimate that the emissions brought by information and computing technologies grow from 2% in 2010 [1] to 8% in 2016 and will grow to 13% by 2027 [2]. Hence, considering the heavy emissions and increasing impact on

© 2017 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

climate change, governments, organizations, and also IT enterprises are trying to find cleaner ways to manage the datacenters, for example, exploiting renewable energy such as wind, solar, and tidal.

However, the intermittency and the instability of the renewable energy sources make it difficult to efficiently utilize them. Fortunately, we know that the datacenter workloads are usually variable, which give us opportunities to find ways to manage the resources and power together inside the datacenters to utilize renewable energy sources more efficiently. On the other hand, to provide guaranteed services for third-party applications, the datacenter is responsible of keeping the quality of service (QoS) at a certain level, subject to the service level agreements (SLAs) [3].

In modern datacenters, applications are often deployed in virtual machines (VMs). By virtualization mechanisms, VMs are flexible and easy to migrate across different servers in the datacenter. In this chapter, we attempt to conduct research on energy-aware virtual machine migration methods for power and resource management in hybrid energy-powered datacenters. Especially, we also employ thermal-aware ideas when designing VM migration approaches. The holistic framework is described, then the model is established, and heuristic and stochastic strategies are presented in detail. Experimental results show the effectivity and feasibility of the proposed strategies. We hope that this chapter would be helpful for researchers to study the features of VM workloads in the datacenter and find ways to utilize more green energy than traditional brown energy.

The remainder of this chapter is organized as follows. Section 2 introduces some relevant prior work in the field of energy-aware and thermal-aware resource and power management. Section 3 presents the entire system architecture we discuss in this chapter. Section 4 formulates the optimization problem corresponding to the issue we need to address. Section 5 describes the methods and strategies we designed to solve the problem. Section 6 illustrates the experimental results by comparing three different strategies, and finally conclusion is given out in Section 7, in which we also discuss about some of the possible future work.

#### **2. Literature review**

This section reviews the literature in the area of energy-aware resource management, thermal-aware power management, and green energy utilization in datacenters.

In the recent decade, many researchers started to focus on power-aware management methods to manage workload fluctuation and search trade-off between performance and power consumption. Sharma et al. [4] have developed adaptive algorithms using a feedback loop that regulates CPU frequency and voltage levels in order to minimize the power consumption. Tanelli et al. [5] controlled CPUs by dynamic voltage scaling techniques in Web servers, aiming at decreasing their power consumption. Berl et al. [6] reviewed the current best practice and progress of the energy efficient technology and summarized the remaining key challenges in the future. Urgaonkar et al. [7] employed queuing theory to make decision aiming at optimizing the application throughput and minimizing the overall energy costs. The above work attempts to reduce the power consumption while guaranteeing the system performance. On the basis of such ideas, we incorporate the usage of renewable energy into the optimization model, which might support performance improvement when the green energy is sufficient enough.

Besides, thermal-aware resource management approaches also attracted some interest of researchers recently. For example, Mukherjee et al. [8] developed two kinds of temperatureaware algorithms to minimize the maximum temperature in order to avoid hot spots. Tang et al. [9] proposed *XInt* which can schedule tasks to minimize the inlet temperatures and also to reduce the cooling energy costs. Pakbaznia et al. [10] combined chassis consolidation and efficient cooling together to save the power consumption while keeping the maximum temperature under a controlled level. Wang et al. [11] designed two kinds of thermal-aware algorithms aiming at lowering the temperatures and minimizing the cooling system power consumption. Islam et al. [12] proposed DREAM which can manage the resources to control allocate capacity to servers and distribute load considering temperature situations. Similarly, we consider the impact of temperature on two kinds of cooling devices in this chapter, which directly decide the cooling power consumption.

As renewable energy becomes more widely used in datacenters, corresponding research starts to put insights into green energy–oriented approaches for managing the resources and power. Deng et al. [13] treated carbon-heavy energy as a primary cost and designed some mechanisms to allocate resources on demand. Goiri et al. designed *GreenSlot* [14] aiming at scheduling batch workloads and *GreenHadoop* [15] which could deal with MapReduce-based tasks. Both of them tried to efficiently utilize green energy to improve the application performance. Li et al. [16] proposed *iSwitch*, which can switch the power supply between wind power and utility grid according to the renewable power variation. Arlitt et al. [17] defined the "Net-Zero energy" datacenter, which needs on-site renewable generators to offset the usage of power coming from the electricity grid. Deng et al. also conducted research on Datacenter Power Supply System (DPSS) and proposed an efficient, online control algorithm SmartDPSS [18] helping to make online decisions in order to fully leverage the available renewable energy and varying electricity prices from the grid markets, for minimum operational cost. Zhenhua et al. [19] presented a holistic approach that integrates renewable energy supply, dynamic pricing, cooling supply, and workload planning to improve the overall attainability of the datacenter.

Upon the basic concepts of these work, we exploit the possibility and of efficient VM migration management toward sufficiently utilizing renewable energy supply, incorporating the flexibility of transactional workloads, cooling power consumption, and the amount of available green energy.

#### **3. Datacenter architecture**

climate change, governments, organizations, and also IT enterprises are trying to find cleaner ways to manage the datacenters, for example, exploiting renewable energy such as wind,

However, the intermittency and the instability of the renewable energy sources make it difficult to efficiently utilize them. Fortunately, we know that the datacenter workloads are usually variable, which give us opportunities to find ways to manage the resources and power together inside the datacenters to utilize renewable energy sources more efficiently. On the other hand, to provide guaranteed services for third-party applications, the datacenter is responsible of keeping the quality of service (QoS) at a certain level, subject to the service level agreements (SLAs) [3]. In modern datacenters, applications are often deployed in virtual machines (VMs). By virtualization mechanisms, VMs are flexible and easy to migrate across different servers in the datacenter. In this chapter, we attempt to conduct research on energy-aware virtual machine migration methods for power and resource management in hybrid energy-powered datacenters. Especially, we also employ thermal-aware ideas when designing VM migration approaches. The holistic framework is described, then the model is established, and heuristic and stochastic strategies are presented in detail. Experimental results show the effectivity and feasibility of the proposed strategies. We hope that this chapter would be helpful for researchers to study the features of VM workloads in the datacenter and find ways to utilize more

The remainder of this chapter is organized as follows. Section 2 introduces some relevant prior work in the field of energy-aware and thermal-aware resource and power management. Section 3 presents the entire system architecture we discuss in this chapter. Section 4 formulates the optimization problem corresponding to the issue we need to address. Section 5 describes the methods and strategies we designed to solve the problem. Section 6 illustrates the experimental results by comparing three different strategies, and finally conclusion is given out in Section 7, in which we also discuss about some of the possible future

This section reviews the literature in the area of energy-aware resource management, ther-

In the recent decade, many researchers started to focus on power-aware management methods to manage workload fluctuation and search trade-off between performance and power consumption. Sharma et al. [4] have developed adaptive algorithms using a feedback loop that regulates CPU frequency and voltage levels in order to minimize the power consumption. Tanelli et al. [5] controlled CPUs by dynamic voltage scaling techniques in Web servers, aiming at decreasing their power consumption. Berl et al. [6] reviewed the current best practice and progress of the energy efficient technology and summarized the remaining key challenges in the future. Urgaonkar et al. [7] employed queuing theory to make decision aiming at optimizing the application throughput and minimizing the overall energy costs. The above work attempts

mal-aware power management, and green energy utilization in datacenters.

solar, and tidal.

54 Cloud Computing - Architecture and Applications

work.

**2. Literature review**

green energy than traditional brown energy.

This section describes the datacenter architecture, including the hybrid power supply and virtualization infrastructure.

**Figure 1** shows the system architecture of the sustainable datacenter powered by both renewable energy and traditional energy supplies. The grid utility and renewable energy are combined together by the automatic transfer switch (ATS) in order to provide power supply for the datacenter. Both functional devices and cooling devices have to consume power, as shown in the bottom part of the figure.

**Figure 1.** Architecture of the sustainable datacenter powered by both green energy and brown energy.

**Figure 2** illustrates the infrastructure of virtualized cloud datacenter. As shown, the underlying infrastructure of the datacenter is comprised of many physical machines (PMs), which are placed onto groups of racks. The utility grid bus and the renewable energy bus are connected together to supply power for the datacenter devices. Renewable sources will be used first, and the grid power will be leveraged as the supplementary energy supply.

As mentioned before, virtual machines (VMs) are running on the underlying infrastructure as used to host multiple applications, as shown in the virtualization layer in **Figure 2**. Different VMs on the same PM might serve for different applications. In this chapter, we mainly discuss about transactional applications which needs CPU resources mostly, other than other types of resources.

Green-Aware Virtual Machine Migration Strategy in Sustainable Cloud Computing Environments http://dx.doi.org/10.5772/67350 57

**Figure 2.** Virtualized datacenter.

combined together by the automatic transfer switch (ATS) in order to provide power supply for the datacenter. Both functional devices and cooling devices have to consume power, as

**Figure 2** illustrates the infrastructure of virtualized cloud datacenter. As shown, the underlying infrastructure of the datacenter is comprised of many physical machines (PMs), which are placed onto groups of racks. The utility grid bus and the renewable energy bus are connected together to supply power for the datacenter devices. Renewable sources will be used first, and

**Figure 1.** Architecture of the sustainable datacenter powered by both green energy and brown energy.

As mentioned before, virtual machines (VMs) are running on the underlying infrastructure as used to host multiple applications, as shown in the virtualization layer in **Figure 2**. Different VMs on the same PM might serve for different applications. In this chapter, we mainly discuss about transactional applications which needs CPU resources mostly, other than other types

the grid power will be leveraged as the supplementary energy supply.

of resources.

shown in the bottom part of the figure.

56 Cloud Computing - Architecture and Applications

#### **4. Problem definition**

This section defines necessary variables and also the problem we need to solve throughout this chapter.

#### **4.1. Model of computing and service units**

In the target problem, there are *N* heterogeneous physical machines in the virtualized cloud environment, and the available CPU resource capacity of PM *i* is denoted as *Ф<sup>i</sup>* . The entire environment is hosting *M* kinds of different applications, deployed on *M* different VMs. Denote the *j*th VM as *VMj* . Then, denote *xj* as the index of the PM which is hosting *VMj* . Denote *φi* as the allocated CPU capacity to *VMj* and *di* as the demanded CPU capacity of application *j* at the current time slot.

#### **4.2. Power consumption model**

According to the mechanisms of dynamic voltage and frequency scaling (DVFS) techniques, here we use a simple power model which assumes that the power consumption of other components in the PM correlate well with CPU [20]. Denote *pi* as the power consumption of PM *i* in each time slot and *pi MAX* as the maximum power consumption of PM *i* (100% occupied by workloads). Then, the following equation can be used to compute the PM power consumption:

$$p\_i = \ p\_i^{\text{MAX}} \cdot \left(c + (1 - c) \cdot \Theta\right) \tag{1}$$

where *c* is a constant number representing the ratio of the idle-state power consumption of a PM compared to the full-utilized-state power consumption [21] and *θ<sup>i</sup>* is the current CPU utilization of PM *i*.

Besides, we also consider the power cost spent on cooling devices when establishing the power model, which is usually much related to temperature. The cooling system we discuss here consists of both the traditional computer room air conditioning (CRAC) unit and the air economizer. According to relevant studies [10], the coefficient of performance (CoP) is often used to indicate the efficiency of a cooling system, which can be computed by

$$\stackrel{\circ}{\text{used to indicate the efficiency of a cooling system, which can be computed by}}$$

$$\text{CoP} = \begin{cases} 1/\text{k}(T\_{\text{sup}} - T\_{\text{out}}) & \text{when } T\_{\text{out}} \le T\_{\text{sup}} \\ 0.0068 \, T\_{\text{sup}}^2 + 0.0008 \, T\_{\text{sup}} + 0.458, & \text{otherwise} \end{cases} \tag{2}$$

where *k* is a factor reflecting the difference between outside air and target temperature, *Tsup* is the target supply temperature, and *Tout* is the outside temperature. As it can be observed, Eq. (2) contains two parts, corresponding to the situation whether the CRAC or the air economizer will be used for cooling, respectively.

Hence, the total power consumed by both functional devices and cooling devices can be calculated by

$$p\_{\rm DC} = \left(1 + \frac{1}{\rm CoP}\right) \cdot \sum\_{i=1}^{N} p\_i \tag{3}$$

Furthermore, considering the impact of environmental temperature inside the datacenter, we also tried to exploit thermal-aware VM migration strategies. The power consumption of the servers will make the surrounding environmental temperature increase, due to the dissipated heat. Prior studies [11] provided ways to model the vector of inlet temperatures *T***in** as

$$T\_{\rm in} = T\_s + \mathbf{D} \bullet \mathbf{p} \tag{4}$$

where *D* is the heat transferring matrix, *p* is the power consumption vector, and *T***<sup>s</sup>** is the supplied air temperature vector.

The thermal-aware strategy tries to reduce the cooling power by balancing the temperature over the servers. Accordingly, the workload on different PMs should also be maintained balanced. Denote *Tsafe* as the safe outlet temperature and *Tserver* as the outlet temperature of the hottest server. In order to lower the server temperature to the safe level, the output temperature of cooling devices should be adjusted by *Tadj* = *Tsafe* − *Tserver*. Then, the output temperature after adjusted will be *Tnew* = *Tsup* + *Tadj*. Hereafter, the *CoP* value can be determined by *Tnew* and *Tout* [22].

#### **4.3. Modeling overhead and delay**

To reduce the power consumption of the PM, it can be switched to sleeping state which can help save energy as much as possible. In addition, the operational costs also include the VM migration costs, since migrating VMs dynamically will definitely lead to some overhead. Denote *ai* as the flag recording whether PM *i* is active or sleeping. Denote *cA* as the cost for activating a PM from sleeping state and *cMIG* as the cost for migrating a VM from one PM to another. Besides, the time delay is also considered and integrated into the experiments in Section 6 for waking up a PM and migrating a VM.

#### **4.4. Optimization problem formulation**

where *c* is a constant number representing the ratio of the idle-state power consumption of

Besides, we also consider the power cost spent on cooling devices when establishing the power model, which is usually much related to temperature. The cooling system we discuss here consists of both the traditional computer room air conditioning (CRAC) unit and the air economizer. According to relevant studies [10], the coefficient of performance (CoP) is often

where *k* is a factor reflecting the difference between outside air and target temperature, *Tsup* is the target supply temperature, and *Tout* is the outside temperature. As it can be observed, Eq. (2) contains two parts, corresponding to the situation whether the CRAC or the air econo-

Hence, the total power consumed by both functional devices and cooling devices can be

Furthermore, considering the impact of environmental temperature inside the datacenter, we also tried to exploit thermal-aware VM migration strategies. The power consumption of the servers will make the surrounding environmental temperature increase, due to the dissipated

*T*in = *Ts* + *D* • *p* (4)

The thermal-aware strategy tries to reduce the cooling power by balancing the temperature over the servers. Accordingly, the workload on different PMs should also be maintained balanced. Denote *Tsafe* as the safe outlet temperature and *Tserver* as the outlet temperature of the hottest server. In order to lower the server temperature to the safe level, the output temperature of cooling devices should be adjusted by *Tadj* = *Tsafe* − *Tserver*. Then, the output temperature after adjusted will be *Tnew* = *Tsup* + *Tadj*. Hereafter, the *CoP* value can be determined by *Tnew* and *Tout* [22].

To reduce the power consumption of the PM, it can be switched to sleeping state which can help save energy as much as possible. In addition, the operational costs also include the VM migration costs, since migrating VMs dynamically will definitely lead to some overhead.

activating a PM from sleeping state and *cMIG* as the cost for migrating a VM from one PM to another. Besides, the time delay is also considered and integrated into the experiments in

as the flag recording whether PM *i* is active or sleeping. Denote *cA* as the cost for

heat. Prior studies [11] provided ways to model the vector of inlet temperatures *T***in** as

where *D* is the heat transferring matrix, *p* is the power consumption vector, and *T***<sup>s</sup>**

CoP) ∙ ∑*<sup>i</sup>*=1

*k*(*T*sup‒*T*out), when *T*out ≤ *T*sup 0.0068 *<sup>T</sup>*sup

<sup>2</sup> <sup>+</sup> 0.0008 *<sup>T</sup>*sup <sup>+</sup> 0.458, otherwise (2)

*<sup>N</sup> pi* (3)

is the sup-

is the current CPU

a PM compared to the full-utilized-state power consumption [21] and *θ<sup>i</sup>*

used to indicate the efficiency of a cooling system, which can be computed by

1 /

utilization of PM *i*.

calculated by

CoP <sup>=</sup> {

58 Cloud Computing - Architecture and Applications

plied air temperature vector.

**4.3. Modeling overhead and delay**

Section 6 for waking up a PM and migrating a VM.

Denote *ai*

mizer will be used for cooling, respectively.

*pDC* <sup>=</sup> (<sup>1</sup> <sup>+</sup> \_\_\_\_ <sup>1</sup>

From the resource providers' point of view, the objective should be maximizing the total revenues by meeting the requirements of the hosted applications while minimizing the consumed power and other costs. Usually, the revenues from hosting the applications are related to service quality and the predefined level in the SLA. Assume here that the service quality is reflected by the CPU capacity scheduled to the target application. Denote *dj* as the demanded CPU capacity of APP *j* and *φ<sup>j</sup>* as the CPU capacity amount scheduled to APP *j*. Denote *Ω<sup>j</sup> (*•*)* as the profit model for APP *j*, which gives the actual revenue by serving APP *j* at a certain quality level.

Since the dynamic action decisions are made during constant time periods, denote *τ* as the length of one time slot. Denote *t* as the current time slot, and then in time slot *t*+*1*, the goal is to maximize the net revenue subject to various constraints. Denote *xj* as the index of PM currently hosting VM *j*, and then the VM placement vector *X* can be denoted as

$$\mathbf{X} = (\mathbf{x}\_{\mathbf{'}'} \mathbf{x}\_{\mathbf{'}'} \dots \mathbf{x}\_{\mathbf{'}'} \mathbf{x}\_{\mathbf{'}'} \dots \mathbf{x}\_{\mathbf{M}}) \tag{5}$$

Hence, the optimizing objective of the defined problem can be expressed as

$$\begin{aligned} \text{Hence, the optimization objective of the defined problem can be expressed as} \\ \sum\_{j=1}^{M} \Omega(d\_j \, \varphi) - \mathbf{c}^p \cdot \mathbf{p}\_{\text{xc}} \\ \max - \mathbf{c}^\lambda \cdot \sum\_{i=1}^{N} \max\{0, \mathbf{a}\_i(t+1) - \mathbf{a}\_i(t)\} \\ - \mathbf{c}^{\text{MG}} \cdot \sum\_{j=1}^{M} \left(\mathbf{x}\_j(t+1) - \mathbf{x}\_j(t)\right) \end{aligned} \tag{6}$$

where the first term is the total revenue summarized over all of the hosted applications, the second term represents the power consumption costs of the entire datacenter, the third term is the PM wake-up cost, and the last term represents the VM migration cost.

With respect to the objective defined above, the constraints could be expressed as

$$\sum\_{x \neq l} \varphi\_j \le \Phi\_l \cdot a\_l \tag{7}$$

$$0 \le \varphi\_{\mid} \le d\_{\forall} \mathbf{j} = \mathbf{1}, \mathbf{2}, \dots, \mathbf{M} \tag{8}$$

$$a\_i \in \{0, 1\}, x\_j \in \{1, N\} \tag{9}$$

where Eq. (7) means that the allocated capacity cannot exceed the PM CPU capacity, Eq. (8) means that the CPU scheduled to a VM should be less than its demanded value, and Eq. (9) gives the validated ranges of the defined variables.

#### **5. Methods and strategies**

In this section, we design some heuristic methods and also the joint hybrid strategy, and describe the ideas in detail.

#### **5.1. Dynamic load balancing** *(DLB)*

The idea of the *DLB* strategy is to make the workload on different PMs balanced by dynamically placing VMs. To achieve the balancing effect, if one PM is detected to be more utilized than the specified upper threshold, some VMs on this PM will be chosen to migrate otherwhere. As a result, the PM utilization ratio will be controlled in a certain range, and there will be as few overloaded PMs as possible.

#### **5.2. Dynamic VM consolidation (***DVMC***)**

According to the features of virtualization techniques, VMs could be consolidated together onto a few PMs to make other PMs zero loaded. Hence, the main idea of the *DVMC* strategy is to consolidate VMs as much as possible aiming at saving more power. Both the upper threshold and the lower threshold of the PM utilization level are defined. If one PM is light loaded enough that its utilization is less than the lower threshold, the VM consolidation process will be triggered. After this process, VMs upon underutilized PM will be migrated onto other PMs. Finally, zero-loaded PMs could be turned into inactivate state in order to save more power.

#### **5.3. Joint optimal planning (***JOP)*

The *JOP* strategy aims to optimize the VM placement scheme with the objective of sufficiently utilizing the renewable energy and reducing the total costs.

#### *5.3.1. Renewable energy forecasting*

Since renewable energy is used as one source of power supply, we have to forecast the input power value in the next time slot. Here the *k*-nearest neighbor (*k*-NN) algorithm is adopted. A distance weight function is designed to calculate the distance each solar radiation values, as follows:

$$\mathbf{w}\_{\rangle} = \langle \mathbf{1}/\mathbf{d}^{\dagger} \rangle \langle \mathbf{1}/\mathbf{d}^{1} + \mathbf{1}/\mathbf{d}^{2} + \dots + \mathbf{1}/\mathbf{d}^{\dagger} \rangle \tag{10}$$

where *di* is the distance between the *i*th neighbor and the current point.

**Figure 3** shows the forecasting effect on one day in October 2013. The data were measured and collected in Qinghai University, Xining, Qinghai Province of China. By analyzing the data points, the allowed absolute percentage errors (AAPE) of 97.01% data are less than 30%. The accuracy of the prediction method depends on the similar weather conditions in the recent past and may be affected by weather forecast data.

#### *5.3.2. Stochastic search*

In order to look for the best scheme of VM placement, we use stochastic search to do the optimization. Specifically, the genetic algorithm (GA) is modified and employed as follows:

For a typical genetic algorithm, there are two basic items as follows:

**1.** A genetic representation of solution space

Here, for this problem, the decision variable is the vector of VM placement, which can be denoted as **X = (x**<sup>1</sup> **, x**<sup>2</sup> **…, x**M**)**.

Green-Aware Virtual Machine Migration Strategy in Sustainable Cloud Computing Environments http://dx.doi.org/10.5772/67350 61

**Figure 3.** An example of renewable energy forecasting effect.

#### **2.** A fitness function to compute the value of each solution

As described, the objective function defined by Eq. (6) could be used as the fitness function. It is functional in measuring the quality of a certain solution. Hereafter, the fitness function will be denoted as *F(X)*.

The procedure of genetic algorithm can be divided into following steps:

i. Initialization

than the specified upper threshold, some VMs on this PM will be chosen to migrate otherwhere. As a result, the PM utilization ratio will be controlled in a certain range, and there will

According to the features of virtualization techniques, VMs could be consolidated together onto a few PMs to make other PMs zero loaded. Hence, the main idea of the *DVMC* strategy is to consolidate VMs as much as possible aiming at saving more power. Both the upper threshold and the lower threshold of the PM utilization level are defined. If one PM is light loaded enough that its utilization is less than the lower threshold, the VM consolidation process will be triggered. After this process, VMs upon underutilized PM will be migrated onto other PMs. Finally, zero-loaded PMs could be turned into inactivate state in order to

The *JOP* strategy aims to optimize the VM placement scheme with the objective of sufficiently

Since renewable energy is used as one source of power supply, we have to forecast the input power value in the next time slot. Here the *k*-nearest neighbor (*k*-NN) algorithm is adopted. A distance weight function is designed to calculate the distance each solar radiation values,

**Figure 3** shows the forecasting effect on one day in October 2013. The data were measured and collected in Qinghai University, Xining, Qinghai Province of China. By analyzing the data points, the allowed absolute percentage errors (AAPE) of 97.01% data are less than 30%. The accuracy of the prediction method depends on the similar weather conditions in the recent

In order to look for the best scheme of VM placement, we use stochastic search to do the optimization. Specifically, the genetic algorithm (GA) is modified and employed as follows:

Here, for this problem, the decision variable is the vector of VM placement, which can be

(1/ d1 + 1/ d2 + …+1/ d*<sup>k</sup>*) (10)

) /

is the distance between the *i*th neighbor and the current point.

be as few overloaded PMs as possible.

60 Cloud Computing - Architecture and Applications

save more power.

as follows:

where *di*

*5.3.2. Stochastic search*

denoted as **X = (x**<sup>1</sup>

**5.3. Joint optimal planning (***JOP)*

*5.3.1. Renewable energy forecasting*

w*<sup>i</sup>* = (1/ d*<sup>i</sup>*

past and may be affected by weather forecast data.

**1.** A genetic representation of solution space

**, x**<sup>2</sup> **…, x**M**)**.

For a typical genetic algorithm, there are two basic items as follows:

utilizing the renewable energy and reducing the total costs.

**5.2. Dynamic VM consolidation (***DVMC***)**

First, we add the current configuration vector in the last time epoch into the initial generation. Besides, a fixed number (denoted as *ng* ) of individual solutions will be randomly generated. Specifically, a part of the elements of each solution will be generated randomly, in the range of 0~*N*−1.

ii. Selection

After initialization, the generations will be produced successively. For each generation, *nb* best-ranking individuals from the current and past population will be selected to breed a new generation. Then, in order to keep the population constant, the remained individuals will either be removed or replicated based on its quality level. The selection procedure is conducted based on fitness, which means that solutions with higher fitness values are more prone to be selected.

According to such concepts, the probability to select an individual *X<sup>i</sup>* can be calculated as

$$\mathbf{P}\{\mathbf{X}\_{\mathsf{i}}\} = \frac{\mathbf{F}\{\mathbf{X}\_{\mathsf{i}}\}}{\sum\_{l=1}^{n}\mathbf{F}\{\mathbf{X}\_{\mathsf{i}}\}} \tag{11}$$

In this way, less fit solutions will be less likely to be selected, and this helps to keep the diversity of the population and to keep away from premature convergences on poor solutions.

#### **1.** Reproduction

After selection, a second generation of population should be generated from those selected solutions through two kinds of genetic operators: crossover and mutation.

The crossover operator first selects two different individuals, denoted as *X*<sup>1</sup> <sup>=</sup> (*x*<sup>1</sup> 1 , *x*2 1 , …,*xM* 1 ) and *<sup>X</sup>*<sup>2</sup> <sup>=</sup> (*x*<sup>1</sup> 2 , *x*2 2 , …,*xM* 2 ). Then, a cutoff point *k* is set from the range 1~*M*. Both *X<sup>1</sup>* and *X<sup>2</sup>* are divided into two halves, and the second half of them will be swapped and then *X*1' <sup>=</sup> (*x*<sup>1</sup> 1 , *x*2 1 , …*xk* 1 , *xk*+1 <sup>2</sup> , …,*xM* 2 ) and *X*<sup>2</sup>' <sup>=</sup> (*x*<sup>1</sup> 2 , *x*2 2 , …,*xk* 2 , *xk*+1 <sup>1</sup> , …,*xM* 1 ). As a result, two new individuals will come out, which is perhaps already in the current population or not.

After crossover, the mutation operator will mutate each individual with a certain probability. The mutation process starts by randomly choosing an element in the vector and then changing its value, and then converts an individual into another.

#### **2.** Termination

This production process will repeat again and again until the number of generations reaches to a predefined level.

#### **6. Evaluation results**

This section shows our experiments comparing different strategies, and then the results and some details will be discussed.

#### **6.1. Parameter settings**

For the following experiments, we used C#.NET to develop the simulation environment and set up the prototype test bed. Specifically, a virtualized datacenter is established, comprised of 40 PMs with CPU capacity of 1500 MIPS each. For the power model, *pi* MAX is set to 259W according to Ref. [23] and, *c* is set to 66% according to Ref. [21]. Then, 100 VMs hosting different applications were simulated and put on the PMs. The workload on each VM fluctuates with time, with the value randomly generated under the uniform distribution.

**Table 1** shows all of the parameter settings in detail, and **Figure 4** shows variation of the total CPU demand summarized over all of the workloads, from which it can be seen that there are two peaks in the 24-h period.

We defined a nonlinear revenue function for each application, as mentioned in Section 4. **Figure 5** shows some three typical examples. It can be seen that the revenue of every application changes elastically in a certain range.

Green-Aware Virtual Machine Migration Strategy in Sustainable Cloud Computing Environments http://dx.doi.org/10.5772/67350 63

**Table 1.** Parameter settings for example applications.

According to such concepts, the probability to select an individual *X<sup>i</sup>*

solutions through two kinds of genetic operators: crossover and mutation.

The crossover operator first selects two different individuals, denoted as *X*<sup>1</sup> <sup>=</sup> (*x*<sup>1</sup>

into two halves, and the second half of them will be swapped and then *X*1' <sup>=</sup> (*x*<sup>1</sup>

). Then, a cutoff point *k* is set from the range 1~*M*. Both *X<sup>1</sup>*

) <sup>=</sup> <sup>F</sup>(Xi ) \_\_\_\_\_\_\_\_\_ ∑*<sup>k</sup>*=1 *<sup>n</sup>*<sup>g</sup> F(Xi

In this way, less fit solutions will be less likely to be selected, and this helps to keep the diversity of the population and to keep away from premature convergences on poor solutions.

After selection, a second generation of population should be generated from those selected

After crossover, the mutation operator will mutate each individual with a certain probability. The mutation process starts by randomly choosing an element in the vector and then chang-

This production process will repeat again and again until the number of generations reaches

This section shows our experiments comparing different strategies, and then the results and

For the following experiments, we used C#.NET to develop the simulation environment and set up the prototype test bed. Specifically, a virtualized datacenter is established, comprised

according to Ref. [23] and, *c* is set to 66% according to Ref. [21]. Then, 100 VMs hosting different applications were simulated and put on the PMs. The workload on each VM fluctuates

**Table 1** shows all of the parameter settings in detail, and **Figure 4** shows variation of the total CPU demand summarized over all of the workloads, from which it can be seen that there are

We defined a nonlinear revenue function for each application, as mentioned in Section 4. **Figure 5** shows some three typical examples. It can be seen that the revenue of every

of 40 PMs with CPU capacity of 1500 MIPS each. For the power model, *pi*

with time, with the value randomly generated under the uniform distribution.

). As a result, two new individuals will come out, which is perhaps

P(Xi

62 Cloud Computing - Architecture and Applications

already in the current population or not.

ing its value, and then converts an individual into another.

**1.** Reproduction

*<sup>X</sup>*<sup>2</sup> <sup>=</sup> (*x*<sup>1</sup> 2 , *x*2 2 , …,*xM* 2

and *X*<sup>2</sup>' <sup>=</sup> (*x*<sup>1</sup>

**2.** Termination

to a predefined level.

**6. Evaluation results**

**6.1. Parameter settings**

some details will be discussed.

two peaks in the 24-h period.

application changes elastically in a certain range.

2 , *x*2 2 , …,*xk* 2 , *xk*+1 <sup>1</sup> , …,*xM* 1

can be calculated as

1 , *x*2 1 , …,*xM* 1 ) and

are divided

MAX is set to 259W

and *X<sup>2</sup>*

1 , *x*2 1 , …*xk* 1 , *xk*+1 <sup>2</sup> , …,*xM* 2 )

) (11)

**Figure 4.** The workload variation.

**Figure 5.** The revenue functions of three example applications.

The control interval for reconfiguration actions in the experiment is set to 60 minutes. According to Refs. [24–26], we set *cP* to \$0.08, set *cA* to \$0.00024, and set *cMIG* to \$0.00012. The VM migration delay is set to 5s, and the PM wakeup delay is set to 15s. The total experiment time is set to 1440 minutes. The temperature data used in the experiments come from the realistic data measured on 4 October 2013, recorded in the campus of the Qinghai University, Xining, Qinghai Province, China, as shown in **Figure 6**.

**Figure 6.** Variation of the outside temperature in 1440 min.

#### **6.2. Results**

In order to investigate the effectiveness of the proposed strategy, we will compare the performance among three different strategies – *DLB*, *DVMC*, and *JOP*, as stated in Section 5.

#### *6.2.1. Revenues*

As described in Section 4, the net revenue is a main optimizing objective in our problem. **Figure 7** shows the total accumulated net revenues throughout the 1440-min experiment time. It can be observed that the *JOP* strategy can keep the net revenue relatively higher than other ones. Moreover, the *DVMC* approach behaves relatively better than *DLB* since it can save more power by VM consolidation. By examining the detailed data, we found that *JOP* could make the gains 38.2 and 24.2% higher than *DLB* and *DVMC*, respectively, with respect to the accumulated revenue.

**Figure 7.** Accumulated net revenue variation.

#### *6.2.2. Power consumption*

The control interval for reconfiguration actions in the experiment is set to 60 minutes. According to Refs. [24–26], we set *cP* to \$0.08, set *cA* to \$0.00024, and set *cMIG* to \$0.00012. The VM migration delay is set to 5s, and the PM wakeup delay is set to 15s. The total experiment time is set to 1440 minutes. The temperature data used in the experiments come from the realistic data measured on 4 October 2013, recorded in the campus of the Qinghai University,

In order to investigate the effectiveness of the proposed strategy, we will compare the performance among three different strategies – *DLB*, *DVMC*, and *JOP*, as stated in Section 5.

As described in Section 4, the net revenue is a main optimizing objective in our problem. **Figure 7** shows the total accumulated net revenues throughout the 1440-min experiment time. It can be observed that the *JOP* strategy can keep the net revenue relatively higher than other ones. Moreover, the *DVMC* approach behaves relatively better than *DLB* since it can save more power by VM consolidation. By examining the detailed data, we found that *JOP* could make the gains 38.2 and 24.2% higher than *DLB* and *DVMC*, respectively, with respect to the

Xining, Qinghai Province, China, as shown in **Figure 6**.

64 Cloud Computing - Architecture and Applications

**6.2. Results**

**Figure 6.** Variation of the outside temperature in 1440 min.

*6.2.1. Revenues*

accumulated revenue.

Now we intend to investigate the power consumption in detail when using *JOP*, as **Figure 8** illustrates. It can be observed from the figure that *JOP* is able to follow the solar energy variation quite well. When the solar power drops to insufficient level, *JOP* is prone to degrade the application performance to save more power. On the contrary, when the solar power arises, *JOP* allows both functional and cooling devices to consume more power, under the constraints of the input power. Interestingly, we can see that the temperature varies more or less in coincidence with solar energy generation, which implies that thermal-aware coscheduling of energy supply and consumption might be promising, since the temperature also affects energy consumption to some extent.

#### *6.2.3. PM Management*

**Figure 9** shows the number of active servers when using the three different strategies. We can see that *JOP* can increase or decrease the number of active servers according to the variation of the solar power generation amount. Under the *DLB* strategy, all PMs are kept active so that the system-wide workload could be balanced. Comparatively, *DVMC* uses much fewer active PMs than *DLB* due to VM consolidation. However, it still uses more PMs at night time because it cannot effectively deal with the relationship of revenues and costs. Overall, *JOP* tries to manage PMs dynamically toward the optimization objective and thus can keep the number of active PMs as needed.

#### *6.2.4. Energy for cooling*

The cooling energy consumption is also investigated when using the three different strategies, as shown in **Figure 10**. As illustrated, *JOP* allows cooling devices to consume more power until after 18:00, showing its capability of catching the solar energy variation. By forecasting the solar power generation amount, *JOP* is able to make better decisions for migrating VMs according to the optimized scheme.

**Figure 8.** Power consumption details under *JOP* strategy.

**Figure 9.** Variation of the number of activated PMs.

Green-Aware Virtual Machine Migration Strategy in Sustainable Cloud Computing Environments http://dx.doi.org/10.5772/67350 67

**Figure 10.** Cooling energy consumption under three different strategies.

#### **7. Conclusion and future work**

until after 18:00, showing its capability of catching the solar energy variation. By forecasting the solar power generation amount, *JOP* is able to make better decisions for migrating VMs

according to the optimized scheme.

66 Cloud Computing - Architecture and Applications

**Figure 8.** Power consumption details under *JOP* strategy.

**Figure 9.** Variation of the number of activated PMs.

As the energy consumption of large-scale datacenters becomes significant and attracts more attentions, renewable energy is being exploited by more enterprises and cloud providers to be used as a supplement of traditional brown energy. In this chapter, we introduced the target system environment using hybrid energy supply mixed with both grid energy and renewables. From the datacenter's own point of view, the optimization problem was defined aiming at maximizing net revenues. Accordingly, three different strategies were designed to migrate VMs across different PMs dynamically, among which the *JOP* strategy could leverage stochastic search to help the optimization process. Results illustrate the feasibility and effectiveness of the proposed strategy and further investigation about the accumulated revenues, PM states, and cooling power consumption helps us to see more details of the working mechanisms of the proposed strategy.

As datacenters become larger and larger and thus enormous amount of energy is still needed to power these datacenters, it can be expected that green sources of energy will attract more insights to provide power supplies instead of traditional brown energy. Our work tries to explore some strategies to migrate VMs inside a datacenter in a green-aware way. Nevertheless, there are still a lot of challenges in the field of leveraging sustainable energy to power the datacenters. On one hand, more kinds of clean energy sources besides wind and solar could be exploited, such as hydrogen and fuel cell, and their features should be studied and developed. On the other hand, how to synthetically utilize the battery, utility grid, and datacenter loads to solve the intermittency and fluctuation problems of the energy sources remains a difficult problem for system designers. In addition, it is also necessary and interesting to conduct some research on the air flow characteristics among racks and server nodes inside the datacenter room and develop some thermal-aware scheduling approaches correspondingly.

### **Acknowledgements**

This work is partially supported in part by National Natural Science Foundation of China (No. 61363019, No. 61563044, and No. 61640206) and National Natural Science Foundation of Qinghai Province (No. 2014-ZJ-718, No. 2015-ZJ-725).

#### **Author details**

Xiaoying Wang1 \*, Guojing Zhang<sup>1</sup> , Mengqin Yang1 and Lei Zhang1,2

\*Address all correspondence to<EMAIL_ADDRESS>
1 Department of Computer Technology and Application, State Key Laboratory of Plateau Ecology and Agriculture, Qinghai University, Xining, China

2 College of Computer Science, Sichuan University, Chengdu, China

#### **References**

[9] Q. Tang, S. Gupta, G. Varsamopoulos. Thermal-aware task scheduling for datacenters through minimizing heat recirculation. In: 2007 IEEE International Conference on Cluster Computing; Austin, TX, USA; IEEE Computer Society; 2007. pp. 129–138.

**Acknowledgements**

68 Cloud Computing - Architecture and Applications

**Author details**

Xiaoying Wang1

**References**

legacy/Global/

2003. p. 63.

J. 2010;**53**(7):1045.

Qinghai Province (No. 2014-ZJ-718, No. 2015-ZJ-725).

\*, Guojing Zhang<sup>1</sup>

\*Address all correspondence to<EMAIL_ADDRESS>
Ecology and Agriculture, Qinghai University, Xining, China

2 College of Computer Science, Sichuan University, Chengdu, China

giswatch.org/thematic-report/sustainability-climatechange/

Workshops at OOPSLA09; Orlando, Florida; ACM; 2009; pp. 1–10.

Computing; Sydney, Australia; Springer; 2008. pp. 599–606.

Symposium (NOMS); Osaka, Japan; IEEE; 2010. pp. 479–486.

Conference; Anaheim, CA, USA; ACM; 2005. pp. 196–201.

This work is partially supported in part by National Natural Science Foundation of China (No. 61363019, No. 61563044, and No. 61640206) and National Natural Science Foundation of

1 Department of Computer Technology and Application, State Key Laboratory of Plateau

[1] H.S. Dunn. The carbon footprint of ICTs. [Internet]. 2010. Available from: https://www.

[2] Greenpeace. Clicking clean: how companies are creating a green internet: 2015 update. [Internet]. 2015. Available from: http://www.greenpeace.org/usa/wp-content/uploads/

[3] P. Patel, A. Ranabahu, A. Sheth. Service level agreement in cloud computing. In: Cloud

[4] V. Sharma, A. Thomas, T. Abdelzaher, et al. Power-aware QoS management in web servers. In: 24th IEEE International Real-Time Systems Symposium; Cancun, Mexico; IEEE;

[5] M. Tanelli, D. Ardagna, M. Lovera, et al. Model identification for energy-aware management of web service systems. In: the 6th International Conference on Service-Oriented

[6] A. Berl, E. Gelenbe, M. Di Girolamo, et al.. Energy-efficient cloud computing. Comput.

[7] R. Urgaonkar, U.C. Kozat, K. Igarashi, et al. Dynamic resource allocation and power management in virtualized datacenters. In: 2010 IEEE Network Operations and Management

[8] R. Mukherjee, S.O. Memik, G. Memik. Temperature-aware resource allocation and binding in high-level synthesis. In: Proceedings of the 42nd annual Design Automation

and Lei Zhang1,2

, Mengqin Yang1

**Chapter 4**

### **M-ary Optical Computing**

Jian Wang and Yun Long

[21] A. Beloglazov, J. Abawajy, R. Buyya. Energy-aware resource allocation heuristics for efficient management of datacenters for cloud computing. Future Gener. Comput. Syst.

[22] F. Ahmad, T. Vijaykumar. Joint optimization of idle and cooling power in datacenters while maintaining response time. In: Proceedings of the fifteenth edition of ASPLOS;

[23] K.-D. Lange. Identifying shades of green: the SPECpower benchmarks. Computer.

[24] D. Ardagna, B. Panicucci, M. Trubian, et al. Energy-aware autonomic resource allocation in multi-tier virtualized environments. IEEE Trans. Serv. Comput. 2010; 5(1): 2–19. [25] D. Kusic, J.O. Kephart, J.E. Hanson, et al. Power and performance management of virtualized computing environments via lookahead control. Cluster Comput. 2009;**12**(1):1-15.

[26] M. Steinder, I. Whalley, D. Carrera, et al. Server virtualization in autonomic management of heterogeneous workloads. In: 10th IFIP/IEEE International Symposium on Integrated

Network Management; Munich, Germany; IEEE; 2007. pp. 139–148.

2012;**28**(5):755–768.

70 Cloud Computing - Architecture and Applications

2009;**42**(3):95–97.

Pittsburgh, PA, USA; ACM; 2010. pp. 243–256.

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/67351

#### **Abstract**

The era of cloud computing has fuelled the increasing demand on data centers for highperformance, high-speed data storage and computing. Digital signal processing may find applications in future cloud computing networks containing a large sum of data centers. Addition and subtraction are considered to be fundamental building blocks of digital signal processing which are ubiquitous in microprocessors for arithmetic operations. However, the processing speed is limited by the electronic bottleneck. It might be valuable to implement high-speed arithmetic operations of addition and subtraction in the optical domain. In this chapter, recent results of M-ary optical arithmetic operations for high base numbers are presented. By exploiting degenerate and nondegenerate four-wave mixing (FWM) in highly nonlinear fibers (HNLFs), graphene-assisted optical devices, and silicon waveguide devices, various types of two-/three-input high-speed quaternary/octal/decimal/hexadecimal optical computing operations have been demonstrated. Operation speed up to 50 Gbaud of this computing approach is experimentally examined. The demonstrated M-ary optical computing using high base numbers may facilitate advanced data management and superior network performance.

**Keywords:** high-base optical signal processing, multilevel modulation format, fourwave mixing, wavelength conversion, optical computing

#### **1. Introduction**

The great progress of fiber-optic communication has driven the success in transmitting/receiving very high-speed data signals in optical fiber links [1–5]. Recently, the era of cloud computing has fuelled the increasing demand on data centers for high-performance, high-speed data storage and computing. Optical interconnection is considered to be a promising technology for data interconnection in data centers. In future cloud computing networks containing a large sum of data centers, optical technology will play an important part [6–8]. For inter-data

© 2017 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

center communication, modern optical communication links will be used. Advanced modulation formats and wavelength division multiplex (WDM) can be used to enhance the transmission capacity of inter-data center links. And for intra-data center links, low-cost short-reach optical interconnection technologies, such as vertical-cavity surface-emitting laser (VCSEL) and multimode fiber, will be adopted. The rapid development of optical interconnection in data centers has also promoted increasing interest for digital signal processing used in data centers for wavelength management or routing. Among various digital signal processing operations, two important arithmetic modules, i.e., addition and subtraction, are considered to be fundamental building blocks of digital signal processing which are ubiquitous in microprocessors for arithmetic operations. However, the processing speed is limited by the electronic bottleneck. It might be valuable to implement high-speed arithmetic operations of addition and subtraction in the optical domain.

Remarkably, nonlinear optics has offered great potential to develop high-speed optical signal processing using optical nonlinearities [9–21]. Multitudinous optical signal processing functionalities have been demonstrated. Commonly used optical signal processing functionalities include wavelength (de)multiplexing, wavelength conversion, data exchange, optical addressing, optical switching, optical logic gate and computing, optical format conversion, optical equalization, tunable optical delay, optical regeneration, optical coding/decoding, and more [22–54]. As depicted in **Figure 1**, the material platforms for nonlinear optical signal processing mainly include highly nonlinear fiber (HNLF) [51, 55–57], semiconductor optical amplifier (SOA) [58–60], periodically poled lithium niobate (PPLN) waveguide [32, 35, 36, 61, 62], chalcogenide (As<sup>2</sup> S3 ) waveguide [63], silicon waveguide [64–66], and graphene-assisted device [67]. Previously, optical arithmetic or optical logic operations have been reported in these material systems. It is noted that most of previous research efforts are dedicated to optical computing for binary modulation formats such as on-off keying (OOK) and binary phaseshift keying (BPSK). Despite favorable operation performance achieved for binary operation, it suffers the limited bit rate and low spectral efficiency because each symbol for binary modulation formats only carries single-bit information.

**Figure 1.** Material classification for nonlinear optical signal processing.

The use of M-ary phase-shift keying (m-PSK) and M-ary quadrature amplitude modulation (m-QAM) in coherent systems has become a key technique for efficient increase of the transmission capacity and spectral efficiency of optical communication systems. For instance, quadrature phase-shift keying (QPSK) with 2-bit information in one symbol has been extensively used in high-speed optical fiber transmission systems [68, 69]. Multilevel modulation format containing multiple constellation points in the constellation diagram can also be used to represent M-ary numbers. Taking QPSK as an example, four constellation points (i.e., four-phase levels) in the constellation diagram of QPSK signal can donate a quaternary base number (i.e., 0, 1, 2, 3), as shown in **Figure 2**. Similarly, 8 PSK (16 PSK) signal which has 8 (16) points in its constellation plane can represent an octal (hexadecimal) base number. The related optical signal processing functions to multilevel modulation formats could be addition and subtraction of high base numbers. In this scenario, a laudable goal would be to perform addition and subtraction of high base numbers because (i) high capacities might be achievable, (ii) optical spectra might be utilized efficiently, and (iii) processing throughput might be improved.

center communication, modern optical communication links will be used. Advanced modulation formats and wavelength division multiplex (WDM) can be used to enhance the transmission capacity of inter-data center links. And for intra-data center links, low-cost short-reach optical interconnection technologies, such as vertical-cavity surface-emitting laser (VCSEL) and multimode fiber, will be adopted. The rapid development of optical interconnection in data centers has also promoted increasing interest for digital signal processing used in data centers for wavelength management or routing. Among various digital signal processing operations, two important arithmetic modules, i.e., addition and subtraction, are considered to be fundamental building blocks of digital signal processing which are ubiquitous in microprocessors for arithmetic operations. However, the processing speed is limited by the electronic bottleneck. It might be valuable to implement high-speed arithmetic operations of

Remarkably, nonlinear optics has offered great potential to develop high-speed optical signal processing using optical nonlinearities [9–21]. Multitudinous optical signal processing functionalities have been demonstrated. Commonly used optical signal processing functionalities include wavelength (de)multiplexing, wavelength conversion, data exchange, optical addressing, optical switching, optical logic gate and computing, optical format conversion, optical equalization, tunable optical delay, optical regeneration, optical coding/decoding, and more [22–54]. As depicted in **Figure 1**, the material platforms for nonlinear optical signal processing mainly include highly nonlinear fiber (HNLF) [51, 55–57], semiconductor optical amplifier (SOA) [58–60], periodically poled lithium niobate (PPLN) waveguide [32, 35, 36, 61,

device [67]. Previously, optical arithmetic or optical logic operations have been reported in these material systems. It is noted that most of previous research efforts are dedicated to optical computing for binary modulation formats such as on-off keying (OOK) and binary phaseshift keying (BPSK). Despite favorable operation performance achieved for binary operation, it suffers the limited bit rate and low spectral efficiency because each symbol for binary modu-

) waveguide [63], silicon waveguide [64–66], and graphene-assisted

addition and subtraction in the optical domain.

72 Cloud Computing - Architecture and Applications

S3

lation formats only carries single-bit information.

**Figure 1.** Material classification for nonlinear optical signal processing.

62], chalcogenide (As<sup>2</sup>

In this chapter, we tend to provide a comprehensive report of our recent research works on M-ary optical computing for multilevel modulation formats by exploiting optical nonlinearities [70–75]. Various material platforms, including HNLFs, graphene-assisted optical devices, and silicon waveguide devices, are adopted to performing high-speed M-ary addition and subtraction. First, we report the experimental results of optical addition and subtraction using HNLFs. Functionalities of quaternary addition/subtraction are examined. Second, we show the grapheneenhanced optical nonlinearities in graphene-assisted optical devices and its application in optical computing. Finally, we present the latest results of high-speed optical computing using ultracompact on-chip silicon waveguides. Quaternary/hexadecimal hybrid optical computing is suc-

**Figure 2.** Schematic constellations of advanced multilevel modulation formats representing M-ary (quaternary, octal, hexadecimal) numbers (QPSK, 8PSK, 16PSK).

cessfully demonstrated in a complementary metal oxide semiconductor (CMOS)-compatible platform, which can be potentially integrated with standard CMOS large-scale integrated circuit.

#### **2. Binary optical logic**

In the last two decades, binary optical computing has been widely studied. Up to now, many schemes have been demonstrated to realize various elementary optical logic operations, including AND, OR, NOT, XOR, XNOR, NAND, and NOR [32, 55, 61, 62, 76–85]. By combining multiple elementary optical logic operations, advanced logic operations such as half-adder, half-subtractor, full-adder, and full-subtractor have also been proposed and demonstrated [36, 86–91]. **Figure 3** shows an example of simultaneous half-adder, half-subtractor, and OR logic gate [36].

**Figure 3.** Digital gate-level diagram and logical truth table for simultaneous half-adder, half-subtractor, and OR logic gate.

Despite favorable operation performance of the binary operation, it still suffers from the limited bit rate and low spectral efficiency. Owing to the great success of advanced modulation format and coherent detection in optical communication, the implementation of M-ary optical computing becomes possible. Since the multiple constellation points in the complex plane of multilevel modulation format can be used to represent M-ary numbers, it is easy to extend binary optical computing to M-ary.

#### **3. M-ary optical computing using HNLF**

We propose and demonstrate M-ary optical computing of advanced multilevel modulation signals based on degenerate/nondegenerate FWM in HNLFs.

We first demonstrate high-speed two-input high-base optical computing (addition/subtraction/complement/doubling) of quaternary numbers using optical nonlinearities and (differential) quadrature phase-shift keying ((D)QPSK) signals. **Figure 4** illustrates the concept and

cessfully demonstrated in a complementary metal oxide semiconductor (CMOS)-compatible platform, which can be potentially integrated with standard CMOS large-scale integrated circuit.

In the last two decades, binary optical computing has been widely studied. Up to now, many schemes have been demonstrated to realize various elementary optical logic operations, including AND, OR, NOT, XOR, XNOR, NAND, and NOR [32, 55, 61, 62, 76–85]. By combining multiple elementary optical logic operations, advanced logic operations such as half-adder, half-subtractor, full-adder, and full-subtractor have also been proposed and demonstrated [36, 86–91]. **Figure 3** shows an example of simultaneous half-adder, half-subtractor,

Despite favorable operation performance of the binary operation, it still suffers from the limited bit rate and low spectral efficiency. Owing to the great success of advanced modulation format and coherent detection in optical communication, the implementation of M-ary optical computing becomes possible. Since the multiple constellation points in the complex plane of multilevel modulation format can be used to represent M-ary numbers, it is easy to extend

**Figure 3.** Digital gate-level diagram and logical truth table for simultaneous half-adder, half-subtractor, and OR logic

We propose and demonstrate M-ary optical computing of advanced multilevel modulation

We first demonstrate high-speed two-input high-base optical computing (addition/subtraction/complement/doubling) of quaternary numbers using optical nonlinearities and (differential) quadrature phase-shift keying ((D)QPSK) signals. **Figure 4** illustrates the concept and

**2. Binary optical logic**

74 Cloud Computing - Architecture and Applications

and OR logic gate [36].

gate.

binary optical computing to M-ary.

**3. M-ary optical computing using HNLF**

signals based on degenerate/nondegenerate FWM in HNLFs.

**Figure 4.** (a) Concept and (b) principle of two-input high-base optical computing (quaternary addition/subtraction/ complement/doubling) using a single nonlinear device and (D)QPSK signals.

operation principle of the proposed quaternary addition/subtraction/complement/doubling. Four-phase levels of (D)QPSK signal represent quaternary numbers. Three nondegenerate FWMs and three degenerate FWMs in an HNLF are exploited to simultaneously implement multiple arithmetic functions. The input of the HNLF contains two (D)QPSK signals (*A*, *B*) and one continuous wave (CW) pump. Six converted idlers (idlers 1–6) are generated by three nondegenerate FWMs (idlers 1–3) and three degenerate FWMs (idlers 4–6). The relationships between the electrical field (E) and optical phase (Φ) under non-depletion approximation are expressed as

$$E\_{\rm in} \propto E\_A \cdot E\_\mathbf{g} \cdot E\_{\rm CW}^\* \quad \mathfrak{O}\_\mathbf{n} = \mathfrak{O}\_A + \mathfrak{O}\_\mathbf{g} - \mathfrak{O}\_{\rm CW} \tag{1}$$

$$E\_{\rm 2} \propto E\_A \cdot E\_{\rm 3}^\* \cdot E\_{\rm CW'} \quad \bullet\_{\rm 2} = \bullet\_A - \bullet\_{\rm 3} + \bullet\_{\rm CW} \tag{2}$$

$$E\_{\rm 3} \propto E\_A^\* \cdot E\_{\rm 3} \cdot E\_{\rm CW'} \quad \mathfrak{O}\_{\rm 3} = \left. \mathfrak{O}\_{\rm 3} - \mathfrak{O}\_A + \mathfrak{O}\_{\rm CW} \right. \tag{3}$$

$$E\_{\kappa} \propto E\_{\mathfrak{g}} \cdot E\_{\mathfrak{g}} \cdot E\_{\text{CW}'}^{\*} \quad \mathfrak{O}\_{\kappa} = \mathfrak{D} \cdot \mathfrak{O}\_{\text{CW}} \tag{6}$$

Owing to the phase wrap characteristic with a periodicity of 2π, it is implied from Eqs. (1) to (6) that idlers 1–6 carry out modulo four operations of quaternary addition (A + B), dual-directional subtraction (A − B, B − A), complement (−A, −B), and doubling (2B), respectively.

Shown in **Figure 5** are measured spectra. Two 100-Gbit/s 2<sup>7</sup> -1 RZ-(D)QPSK signals (A, 1546.6 nm; B, 1555.5 nm), and a CW pump (1553.2 nm), are launched into a 460-m HNLF. The low and flat dispersion of HNLF enables multiple FWM processes, and thus six idlers are obtained. The six idlers correspond to addition (A + B), subtraction (A − B, B − A), complement (−A, −B), and doubling (2B) of quaternary numbers (A, B), respectively.

We measured waveforms and balanced eyes of the demodulated in-phase (I) and quadrature (Q) components of two input 100-Gbit/s (D)QPSK signals and six converted idlers. The 100-Gbit/s (D)QPSK signal is demodulated using a delay-line interferometer (DLI) with a 20 ps delay difference between two arms. The obtained results are shown in **Figures 6** and **7**, which confirm the successful implementation of 50-Gbaud quaternary addition (A+B), dualdirectional subtraction (A−B, B−A), complement (−A, −B), and doubling (2B) based on FWM in an HNLF.

**Figure 8** shows the bit error rate (BER) curves. The power penalty is about 4 dB for addition, while 3 dB for subtraction, 2 dB for complement, and 3.1 dB for doubling at a BER of 10−9. The measured constellations using an optical complex spectrum analyzer are shown in **Figure 9**. One can clearly see that addition (A+B) and subtraction (A−B, B−A) have four-phase levels (0, π/2, π, 3π/2), while doubling (2B) has only two-phase levels (0, π).

**Figure 5.** Measured spectra for 50-Gbaud two-input quaternary addition/subtraction/complement/doubling (a) before and (b) after HNLF.

*Ei*<sup>6</sup> ∝ *EB* · *EB* · *ECW*

76 Cloud Computing - Architecture and Applications

Shown in **Figure 5** are measured spectra. Two 100-Gbit/s 2<sup>7</sup>

and doubling (2B) of quaternary numbers (A, B), respectively.

π/2, π, 3π/2), while doubling (2B) has only two-phase levels (0, π).

respectively.

in an HNLF.

and (b) after HNLF.

\* , *Φ<sup>i</sup>*<sup>6</sup> = 2 *Φ<sup>B</sup>* − *ΦCW* (6)

Owing to the phase wrap characteristic with a periodicity of 2π, it is implied from Eqs. (1) to (6) that idlers 1–6 carry out modulo four operations of quaternary addition (A + B), dual-directional subtraction (A − B, B − A), complement (−A, −B), and doubling (2B),

nm; B, 1555.5 nm), and a CW pump (1553.2 nm), are launched into a 460-m HNLF. The low and flat dispersion of HNLF enables multiple FWM processes, and thus six idlers are obtained. The six idlers correspond to addition (A + B), subtraction (A − B, B − A), complement (−A, −B),

We measured waveforms and balanced eyes of the demodulated in-phase (I) and quadrature (Q) components of two input 100-Gbit/s (D)QPSK signals and six converted idlers. The 100-Gbit/s (D)QPSK signal is demodulated using a delay-line interferometer (DLI) with a 20 ps delay difference between two arms. The obtained results are shown in **Figures 6** and **7**, which confirm the successful implementation of 50-Gbaud quaternary addition (A+B), dualdirectional subtraction (A−B, B−A), complement (−A, −B), and doubling (2B) based on FWM

**Figure 8** shows the bit error rate (BER) curves. The power penalty is about 4 dB for addition, while 3 dB for subtraction, 2 dB for complement, and 3.1 dB for doubling at a BER of 10−9. The measured constellations using an optical complex spectrum analyzer are shown in **Figure 9**. One can clearly see that addition (A+B) and subtraction (A−B, B−A) have four-phase levels (0,

**Figure 5.** Measured spectra for 50-Gbaud two-input quaternary addition/subtraction/complement/doubling (a) before

**Figure 6.** Demodulated waveforms and balanced eyes for 50-Gbaud two-input quaternary addition and dual-directional subtraction using 100-Gbit/s (D)QPSK signals.

**Figure 7.** Demodulated waveforms and balanced eyes for 50-Gbaud quaternary complement and doubling using 100-Gbit/s (D)QPSK signals.

**Figure 8.** Measured BER curves for input/output signals (A, B), quaternary addition (A+B), dual-directional subtraction (A−B, B−A), complement (−A, −B), and doubling (2B).

**Figure 9.** Measured constellations for 50-Gbaud two-input quaternary addition, dual-directional subtraction, complement, and doubling using 100-Gbit/s (D)QPSK signals.

**Figure 7.** Demodulated waveforms and balanced eyes for 50-Gbaud quaternary complement and doubling using

**Figure 8.** Measured BER curves for input/output signals (A, B), quaternary addition (A+B), dual-directional subtraction

100-Gbit/s (D)QPSK signals.

78 Cloud Computing - Architecture and Applications

(A−B, B−A), complement (−A, −B), and doubling (2B).

#### **4. Graphene-enhanced optical nonlinearity for M-ary optical computing**

Graphene as a purely two-dimensional material with only one-carbon-atom thickness has received great interest since it features many interesting and useful electrical, optical, chemical, and mechanical properties [92, 93]. Over the last decade, many remarkable optical properties of graphene have been discovered, such as self-luminosity, tunable optical absorption, strong nonlinearity, saturable absorption, etc. [94–96]. Recently, optical nonlinearities have been observed in graphene in various configurations, e.g., slow-light graphene-silicon photonic crystal waveguide [97], graphene optically deposited onto fiber ferrules [98], and graphene based on microfiber [99]. The large absorption and Pauli blocking effect in graphene, together with the ultrafast carrier dynamics and strong optical nonlinearity with a fast response time, make graphene-based photonic devices suitable for performing efficient nonlinear functions. Very recently, an experimental observation of FWM-based wavelength conversion of a 10-Gb/s nonreturn-to-zero (NRZ) signal was reported [100]. In this section, we introduce our recent progress in optical M-ary computing functions using a graphene-assisted nonlinear optical device.

**Figure 10** illustrates the fabrication process of the graphene-assisted nonlinear optical device. First, a monolayer graphene was grown on a Cu foil by the chemical vapor deposition (CVD) method. Poly(methyl methacrylate) (PMMA) film was next spin coated on the surface

**Figure 10.** Fabrication process of the graphene-assisted nonlinear optical device.

of the graphene-deposited Cu foil, and the Cu foil was etched away with 1 M FeCl<sup>3</sup> solution. The resultant PMMA/graphene film (5 mm × 5 mm) was then washed in deionized water several times and transferred to deionized water solution or Si/SiO<sup>2</sup> substrate. Then, the floating PMMA/graphene sheet was mechanically transferred onto the fiber pigtail cross section and dried in a cabinet. After drying at room temperature for about 24 hours, the carbon atoms could be self-assembled onto the fiber end facet. The PMMA layer was finally removed by boiling acetone. By connecting this graphene-on-fiber component with another clean and dry fiber connector, the nonlinear optical device was thereby constructed for nonlinear optical signal processing applications.

**Figure 11(a)** depicts the optical microscope (OM) image of the grown graphene film transferred on a 300-nm SiO<sup>2</sup> /Si substrate. **Figure 11(b)** shows a scanning electron microscopy (SEM) image of the graphene sheet transferred on silicon-on-insulator (SOI). One can clearly see the evidence of the uniformity of the graphene. The Raman spectrum of the graphene, as displayed in **Figure 11(c)**, shows a weak D peak and a strong 2D peak. The D to G peak intensity ratio is ~0.08, which indicates that the graphene formed on a SiO<sup>2</sup> /Si substrate was almost defect-free.

**Figure 11.** (a) Optical microscope (OM) image of graphene transferred on a SiO<sup>2</sup> /Si substrate. (b) SEM image of graphene transferred on silicon-on-insulator (SOI). (c) Typical Raman spectrum of single-layer graphene on a SiO<sup>2</sup> /Si substrate (excitation wavelength: 532 nm).

We first examine the wavelength conversion of the graphene-assisted nonlinear optical device. **Figure 12(a)** shows a typical output FWM spectrum obtained after the CVD single-layer graphene-coated fiber device. In the experiment, the signal wavelength is fixed at 1550.12 nm. A newly converted idler at 1546.88 nm is generated when the pump is set to be 1548.49 nm. We also measure the output spectrum without graphene for reference under the same experimental conditions. As clearly shown in the inset of **Figure 12(a)**, the power of converted idler without graphene is observed to be ~5.5 dB lower than the one with graphene. That is, under the same experimental conditions, the converted idler without graphene is ~71.9% lower than the one with graphene. Hence, the degenerate FWM in graphene contributes more in the wavelength conversion process. The insets of **Figure 12(a)** also depict measured QPSK constellations of the converted idler and the input signal. We also present a comparison of the FWM conversion efficiency as a function of the pump power with and without graphene. As shown in **Figure 12(b)**, the pump wavelength is fixed at λpump = 1548.49 nm and the signal is λsignal = 1550.12 nm. One can clearly see that the conversion efficiency increases with the pump power. When the pump power varies from 23 dBm to 33 dBm, the enhanced FWM conversion efficiency by graphene changes from 4.7 dB to 7.5 dB.

**Figure 13(a)** plots the converted idler wavelength as a function of the pump wavelength when the pump power is fixed at 31 dBm. A linear wavelength relationship between the converted idler and pump is observed. The measured FWM conversion efficiency of tunable wavelength conversion with and without graphene is shown in **Figure 13(b)**. The signal wavelength is fixed at 1550.12 nm and the pump wavelength is tuned from 1547 to 1553 nm. When using graphene-coated fiber device, the conversion efficiency varies about 1.7 dB within a ~6 nm wavelength range. By comparing the measured pump wavelength-dependent conversion efficiency with and without graphene, one can clearly see that the FWM conversion efficiency with graphene is enhanced more than 5 dB within the tuning range of pump wavelength.

of the graphene-deposited Cu foil, and the Cu foil was etched away with 1 M FeCl<sup>3</sup>

cal device was thereby constructed for nonlinear optical signal processing applications.

intensity ratio is ~0.08, which indicates that the graphene formed on a SiO<sup>2</sup>

**Figure 11.** (a) Optical microscope (OM) image of graphene transferred on a SiO<sup>2</sup>

transferred on silicon-on-insulator (SOI). (c) Typical Raman spectrum of single-layer graphene on a SiO<sup>2</sup>

and transferred to deionized water solution or Si/SiO<sup>2</sup>

80 Cloud Computing - Architecture and Applications

**Figure 10.** Fabrication process of the graphene-assisted nonlinear optical device.

ferred on a 300-nm SiO<sup>2</sup>

(excitation wavelength: 532 nm).

almost defect-free.

resultant PMMA/graphene film (5 mm × 5 mm) was then washed in deionized water several times

phene sheet was mechanically transferred onto the fiber pigtail cross section and dried in a cabinet. After drying at room temperature for about 24 hours, the carbon atoms could be self-assembled onto the fiber end facet. The PMMA layer was finally removed by boiling acetone. By connecting this graphene-on-fiber component with another clean and dry fiber connector, the nonlinear opti-

**Figure 11(a)** depicts the optical microscope (OM) image of the grown graphene film trans-

(SEM) image of the graphene sheet transferred on silicon-on-insulator (SOI). One can clearly see the evidence of the uniformity of the graphene. The Raman spectrum of the graphene, as displayed in **Figure 11(c)**, shows a weak D peak and a strong 2D peak. The D to G peak

/Si substrate. **Figure 11(b)** shows a scanning electron microscopy

solution. The

/Si substrate was

/Si substrate. (b) SEM image of graphene

/Si substrate

substrate. Then, the floating PMMA/gra-

To characterize the performance of QPSK wavelength conversion, we further measure the BER curve as a function of the received observed OSNR for B-to-B signal and converted idler.

**Figure 12.** (a) Measured FWM spectra with (circle) and without (square) graphene. (b) Measured conversion efficiency of FWM with and without graphene when pump power is tuned from 23 to 33 dBm.

**Figure 13.** (a) Converted idler wavelength versus pump wavelength. (b) Measured FWM conversion efficiency with and without graphene when pump wavelength is tuned from 1547 to 1553 nm. Pump power: 31 dBm.

**Figure 14** plots measured BER performance for tunable QPSK wavelength conversion with the converted idler generated at 1546.88, 1539.92, and 1557.90 nm, respectively. The measured conversion efficiencies for converted idlers at 1546.88, 1539.92, and 1557.90 nm are −36.2, −48.2, and −39.8 dB, respectively. As shown in **Figure 14**, the observed OSNR penalty is around 1 dB at a BER of 1×10−3 (7% forward error correction (FEC) threshold) for QPSK wavelength conversion with the converted idler at 1546.88 nm. The received OSNR penalties of ∼2.2 dB at a BER of 1×10−3 are observed for converted idlers at 1539.92 and 1557.90 nm. The increased OSNR penalty is mainly due to the reduced conversion efficiency for converted idlers at 1539.92 and 1557.90 nm. The right insets of **Figure 14** depict corresponding constellations of the B-to-B signals and converted idlers. The obtained results shown in **Figures 11**–**14** imply favorable performance achieved for tunable wavelength conversion of QPSK signal using a fiber pigtail cross section coated with a single-layer graphene.

**Figure 14.** Measured BER versus received OSNR for wavelength conversion of QPSK signal. Insets show constellations of QPSK.

We then show the results of optical computing based on the fabricated graphene-assisted nonlinear optical device. **Figure 15** illustrates the concept and principle of two-input hybrid quaternary arithmetic functions. From the constellation in the complex plane (**Figure 15(a)**), it is clear that one can use four-phase levels (π/4, 3π/4, 5π/4, 7π/4) of (D)QPSK to represent quaternary base numbers (0, 1, 2, 3). To implement two-input hybrid quaternary arithmetic functions, the aforementioned graphene-assisted nonlinear optical device is employed. Two-input quaternary numbers (A, B) are coupled into the nonlinear device, and then two converted idlers (idler 1, idler 2) are simultaneously generated by two degenerate FWM processes. **Figure 15(b)** illustrates the degenerate FWM process. We derive the electrical field (E) and optical phase (*Ф*) relationships of two degenerate FWM processes under the pump non-depletion approximation expressed as

$$E\_{\rm in} \propto E\_{\rm A} \cdot E\_{\rm A} \cdot E\_{\rm y}^\* \quad \mathfrak{O}\_{\rm in} = \mathfrak{O}\_{\rm A} + \mathfrak{O}\_{\rm A} - \mathfrak{O}\_{\rm B} \tag{7}$$

$$E\_{\rm 2} \propto E\_{\rm 3} \cdot E\_{\rm 3} \cdot E\_{A'}^\* \quad \mathfrak{O}\_{\rm 2} = \mathfrak{O}\_{\rm 3} + \mathfrak{O}\_{\rm 3} - \mathfrak{O}\_A \tag{8}$$

where the subscripts A, B, i1, and i2 denote input signal A, signal B, converted idler 1, and idler 2, respectively. Owing to the phase wrap characteristic with a periodicity of 2π, it is implied from the linear phase relationships in Eqs. (7) and (8) that idler 1 and idler 2 carry out modulo 4 operations of hybrid quaternary arithmetic functions of doubling and subtraction (2A−B, 2B−A).

**Figure 14** plots measured BER performance for tunable QPSK wavelength conversion with the converted idler generated at 1546.88, 1539.92, and 1557.90 nm, respectively. The measured conversion efficiencies for converted idlers at 1546.88, 1539.92, and 1557.90 nm are −36.2, −48.2, and −39.8 dB, respectively. As shown in **Figure 14**, the observed OSNR penalty is around 1 dB at a BER of 1×10−3 (7% forward error correction (FEC) threshold) for QPSK wavelength conversion with the converted idler at 1546.88 nm. The received OSNR penalties of ∼2.2 dB at a BER of 1×10−3 are observed for converted idlers at 1539.92 and 1557.90 nm. The increased OSNR penalty is mainly due to the reduced conversion efficiency for converted idlers at 1539.92 and 1557.90 nm. The right insets of **Figure 14** depict corresponding constellations of the B-to-B signals and converted idlers. The obtained results shown in **Figures 11**–**14** imply favorable performance achieved for tunable wavelength conversion of QPSK signal using a fiber pigtail

**Figure 14.** Measured BER versus received OSNR for wavelength conversion of QPSK signal. Insets show constellations

**Figure 13.** (a) Converted idler wavelength versus pump wavelength. (b) Measured FWM conversion efficiency with and

without graphene when pump wavelength is tuned from 1547 to 1553 nm. Pump power: 31 dBm.

cross section coated with a single-layer graphene.

82 Cloud Computing - Architecture and Applications

of QPSK.

**Figure 16** depicts measured typical spectrum obtained after the CVD single-layer graphenecoated fiber device. Two 10-Gbaud NRZ-(D)QPSK signals at 1550.10 (A) and 1553.60 nm (B) are employed as two inputs. The power of two input signals (A, B) is about 32 dBm. The conversion efficiency is measured to be around −36 dB. One can clearly see that two converted idlers are obtained by two degenerate FWM processes with idler 1 at 1546.60 nm (2A−B) and idler 2 at 1557.20 nm (2B−A). The resolution of the measured spectrum is set to 0.02 nm. The steps in the measured spectrum are actually the modulation sidebands of two NRZ-(D)QPSK carrying signals. In order to verify the hybrid quaternary arithmetic functions, we measure the phase of symbol sequence for two input signals and two converted idlers, as shown in **Figure 17**. By carefully comparing the quaternary base numbers for two input signals and two converted idlers, one can confirm the successful implementation of two-input hybrid quaternary arithmetic functions of 2A−B and 2B−A.

**Figure 15.** (a) Concept and (b) principle of hybrid quaternary arithmetic functions (2A−B, 2B−A) using degenerate FWM and (D)QPSK signals.

**Figure 16.** Measured spectrum for 10-Gbaud two-input hybrid quaternary arithmetic functions.

We further investigate the BER performance for the proposed optical two-input hybrid quaternary arithmetic functions. The OSNR penalties at a BER of 2×10−3 for hybrid quaternary arithmetic functions are measured to be about 7.4 dB for 2A−B and 7.0 dB for 2B−A. The insets in **Figure 18(a)** show constellations of the last point of the BER curves of output Sig. B and 2A−B. The constellation of Sig. B is measured under an OSNR of 12.6 dB, while the constellation of 2A−B is observed under an OSNR of 19.6 dB. To clearly show the differences between these two constellations, we also assess the EVM of these two constellations, i.e., EVM = 27.61% for output Sig. B and EVM = 30.09% for output 2A−B. The significant performance degradations for the two-input hybrid quaternary arithmetic functions (2A−B, 2B−A) might

**Figure 17.** Measured phase of symbol sequence with coherent detection for 10-Gbaud two-input hybrid quaternary arithmetic functions.

**Figure 18.** Measured BER curves for two-input hybrid quaternary arithmetic functions of 2A−B and 2B−A; (b) BER versus signal offset.

We further investigate the BER performance for the proposed optical two-input hybrid quaternary arithmetic functions. The OSNR penalties at a BER of 2×10−3 for hybrid quaternary arithmetic functions are measured to be about 7.4 dB for 2A−B and 7.0 dB for 2B−A. The insets in **Figure 18(a)** show constellations of the last point of the BER curves of output Sig. B and 2A−B. The constellation of Sig. B is measured under an OSNR of 12.6 dB, while the constellation of 2A−B is observed under an OSNR of 19.6 dB. To clearly show the differences between these two constellations, we also assess the EVM of these two constellations, i.e., EVM = 27.61% for output Sig. B and EVM = 30.09% for output 2A−B. The significant performance degradations for the two-input hybrid quaternary arithmetic functions (2A−B, 2B−A) might

**Figure 17.** Measured phase of symbol sequence with coherent detection for 10-Gbaud two-input hybrid quaternary

arithmetic functions.

**Figure 16.** Measured spectrum for 10-Gbaud two-input hybrid quaternary arithmetic functions.

84 Cloud Computing - Architecture and Applications

be ascribed to the relatively low conversion efficiency for two converted idlers at 1546.60 nm and 1557.20 nm and accumulated distortions transferred from two input signals (A, B). It is possible to further enhance the conversion efficiency by appropriately increasing the number of graphene layers employed in the experiment. **Figure 18(b)** depicts the BER performance as a function of the relative time offset between two signals (signal offset) under an OSNR of ~20 dB. It is found that the BER is kept below enhanced forward error correction (EFEC) threshold when the signal offset/symbol time is within 15 ps, which indicates a favorable tolerance to the signal offset.

We also propose an approach to performing three-input optical addition and subtraction of quaternary base numbers using multiple nondegenerate FWM processes based on grapheneassisted device.

**Figure 19** illustrates the concept and working principle of the proposed graphene-assisted three-input high-base optical computing. Three input (D)QPSK signals (A, B, C) are launched into the nonlinear device, in which three converted idlers (idler 1, idler 2, idler 3) are simultaneously generated by three nondegenerate FWM processes. Quaternary hybrid addition and subtraction of A+B−C, A+C−B, and B+C−A are obtained simultaneously.

In the experiment, the wavelengths of three input signals A, B, and C are fixed at 1548.52, 1550.12, and 1552.52 nm, respectively. **Figure 20** depicts measured typical optical spectrum obtained after the single-layer graphene-coated fiber device. One can clearly see that three converted idlers are generated by three nondegenerate FWM processes with idler 1 at 1546.13 nm (A+B−C), idler 2 at 1550.92 nm (A+C−B), and idler 3 at 1554.13 nm (B+C−A), respectively. The conversion efficiencies of three nondegenerate FWM processes are measured to be larger than −34 dB. In order to verify the quaternary optical computing functions, we measure the phase of symbol sequence for three input signals and three converted idlers, as shown in **Figure 21**. By carefully comparing the quaternary base numbers for three input signals and three converted idlers, one can confirm the successful implementation of graphene-assisted three-input quaternary optical computing (i.e., quaternary hybrid addition and subtraction) functions of A+B−C, A+C−B, and A+C−B.

To characterize the performance of the proposed graphene-assisted three-input high-base optical computing functions, we further measure the BER curves as a function of the received OSNR for B-to-B signals and three converted idlers. **Figure 22** depicts measured BER curves for 10-Gbaud three-input quaternary hybrid addition and subtraction of A+B−C, A+C−B, and B+C−A. As shown in **Figure 22**, the observed OSNR penalties of three-input quaternary hybrid addition and subtraction are accessed to be less than 7 dB at a BER of 2×10−3 (7% EFEC threshold). The increased OSNR penalties might be mainly due to the relatively low conversion efficiency for converted idlers and accumulated distortions transferred from three input signals (A, B, C). The insets in **Figure 22** depict corresponding constellations of the B-to-B signals and converted idlers. The BER curves and constellations of three output signals (A, B, C) after graphene are also shown in **Figure 22** for reference.

**Figure 19.** (a) Concept and (b) principle of graphene-assisted three-input (A, B, C) quaternary hybrid addition and subtraction (A+B−C, A+C−B, B+C−A) using nondegenerate FWM and (D)QPSK signals.

**Figure 20.** Measured spectrum for 10-Gbaud three-input quaternary hybrid addition and subtraction.

**Figure 21**. By carefully comparing the quaternary base numbers for three input signals and three converted idlers, one can confirm the successful implementation of graphene-assisted three-input quaternary optical computing (i.e., quaternary hybrid addition and subtraction)