Datacenter Rack-scale Capability Architecture (RCA)

For the last couple of years there has been a resurgent theme cropping up regarding disaggregation of components within a rack to support hyperscale datacenters. Back in 2013 Facebook, as founder of the Open Compute Foundation, and Intel announced their collaboration on future data center rack technologies.

This architecture deals basically with the type of business that Facebook itself is running and as such is very component focused such that compute, storage and network components are disaggregated across trays, trays being interconnected with a silicon photonic internal network fabric.

This has the advantage for hyperscale datacenters of modularity allowing components such as CPU to be swapped out individually as opposed to the entire server construct. Intel, presenting at Interop 2013, had an excellent presentation on the architecture outlining various advantages. This architecture is indeed being used by Baidu, Alibaba Group, Tencent and China Telecom (according to Intel).

This in itself is not earth shattering, but seems to lack the added "magic sauce". As it stands this is simply a re-jigging of the arrangement of the form factor but in itself does not do anything to really enhance workload density outside of the consolidation of large numbers of components in the physical rack footprint (i.e. more core, RAM, network bandwidth).

Principally it is aimed at reducing cable, switch clutter, associated power requirements and upgrade modularity, essentially this increases the compute surface per rack. These are fine advantages when dealing with hyperscale datacenters as they represent considerable capital expenditure as outlined in the Moor Insights & Strategy paper on the subject.

Tying into the themes in my previous blog regarding the "Future of the Datacenter" there is a densifying effect taking place affecting the current datacenter network architecture as aptly shown in the Moor study:

Examining this architecture, the following points stand out:

  • Rack level architecture is essential in creating economies of scale for hyper-scale and private enterprise datacenters
  • East-West traffic is coming to front-and-center whilst most datacenters are still continuing in North-South network investment with monolithic switch topologies
  • Simply increasing the number of cores and RAM within a rack does not itself increase the workload density (load/unit)
  • Workload consolidation is more complex than this architecture indicates utilizing multiples components at different times under different loading
  • Many approaches are already available using an aggregation architecture (HP Moonshot, Calxeda ARM Architecture, even SoCs)

There is a lot of added value to be derived for an enterprise datacenter using some of these "integrated-disaggregated" concepts, but competing with and surviving in spite of hyperscale datacenters requires additional innovative approaches to be taken by the enterprise.

Enterprises that have taken on board the "rack as a computer" paradigm have annealed this with capability architecture to drive density increases up to and exceeding 10x over simple consolidation within a more capable physical rack:

  • General purpose usage can be well serviced with integrated/(hyper)converged architectures (e.g. Oracle's Private Cloud Appliance, VCE Vblock, Tintri, Nutanix)
  • Big Data architectures use a similar architecture but have the magic sauce embedded in the way that the Hadoop cluster software itself works
  • Oracle's Engineered Systems further up the ante in that they start to add magic sauce to the hardware mix and the software smarts – hence engineered rather than simply integrated. Other examples are available from Teradata, Microsoft and its appliance partners)
  • In particular, the entire rack needs to be thought of as a workload capability server:
    • If database capability is required then everything in that rack should be geared to that workload
    • In-Platform (in the database engine itself) capability used above general purpose virtualization to drive hyper-tenancy
    • Private networking fabric (Infiniband in the case of Oracle Exadata and most high-end appliances)
    • Storage should be modular and intelligent, offloading not just storage block I/O but also being able to deal with part of the SQL Database Workload itself whilst providing the usual complement of thin/sparse-provisioning, deduplication and compression
    • The whole of database workload consolidation is many times the sum of parts in the rack
  • The datacenter becomes a grouping of these hyper-dense intelligent capability rack-scale servers
    • Intelligent provisioning is used to "throw" the workload type onto the best place for doing it at scale, lowest overall cost and still deliver world-class performance and security
    • Integrate into the overall information management architecture of the enterprise
    • Ensure that as new paradigms related to Big Data Analytics and the tsunami of information expected from Internet-of-Things that they can be delivered in the rack scale computer form but with additional "smarts" to further increase value being delivered as well as provide agility to the business.

The enterprise Datacenter can deliver value beyond a hyper-scale datacenter through thinking about continuous consolidation in line with the business not just IT needs that need to be delivered. Such platform and rack-scale capability architecture (RCA) has been proven to provide massive agility to organizations and indeed prepares them for new technologies such that they can behave like "start-ups" with a fast-low-cost-to-fail mentality to power iterative innovation cycles.

Opportunities for the CIO

The CIO and senior team have a concrete opportunity here to steal a march on the Public Cloud vendors by providing hyper-efficient capability architectures for their business in re-thinking the datacenter rack through RCA paradigms.

Not only will this massively reduce footprint in the existing premises and costs, but focuses IT on how best to serve the business through augmentation with hybrid Cloud scenarios.

The industry has more or less spoken about the need for hybrid Cloud scenarios where private on-premise cloud is augmented with public cloud capabilities. Further today's announcement with regards to the EU making "Safe Harbour" data treaty effectively invalid should put organizational IT on point about how to rapidly deal with these changes.

Industry thinking indicates that enterprise private datacenters will shrink, and the CIO team can already ensure they are "thinking" that way and making concrete steps to realize compact ultra-dense datacenters.

A hyper-scale datacenter can't really move this quickly or be that agile as their operating scale inhibits this nimble thinking that should be the hallmark of the CIO of the 2020s.

In the 2020s perhaps nano- and pico-datacenters may be of more interest to enterprises as way of competing for business budgetary investment as post-silicon graphene compute substrates running at 400GHz room become the new norm!

The Future of the Datacenter

The IT industry is characterized by loops of iterative innovation – hence such adages as "back to the mainframe again". However, recent reports regarding the potential future design of datacenters from Emerson Network PowerData Center 2025: Exploring the Possibilities – really outline the state of the industry in relation to drivers from cloud technologies on the inside of the business as well as externally as public cloud providers.

The report highlights differing opinions naturally, however, there are some rather radical thought paradigms in place that may well entail a whole new way of internal IT thinking on behalf of the CIO and infrastructure management:

  • 43% likelihood of Private Power Generation in Hyperscale Data Centers in 2025
  • Average power density of data center expected to reach 52kW/rack 
  • 67% of survey participants believe at least 60% of computing will be cloud-based by 2025
  • Growth of cloud will ultimately result in the decline of the 1-2MW enterprise data center 
  • 58% believe that data centers will be 50% smaller than current facilities 
  • 41% see a combination of air and liquid being used for thermal management 
  • Data centers will be highly automated – all devices auto-identify and operated without/minimum human resources with utilization rates expected of 60-80% 
  • 50% expect server lifecycles to stay in 3-6year range (as today) with Asia Pacific projecting 7+ years


Figure 1: Overall Response from Participants

In short, some very serious implications for anyone that is running a private datacenter today. The relatively conservative pace of IT innovation within the datacenter is set to change based on new computational and storage needs with their implicit effect on networking and physical infrastructure.

In the same report, Andy Lawrence, vice president at 451 Research summarizes nicely by saying:

"The data center of 2025 certainly won't be one data center. The analogy I like to use is to transport. On the road, we see sports cars and family cars; we see buses and we see trucks. They have different kinds of engines, different types of seating and different characteristics in terms of energy consumptions and reliability. We are going to see something similar to that in the data center world. In fact that is already happening, and I expect it to continue."

Datacenter management needs to consider how such a metaphor can be instantiated within their own "bricks and mortar stores" to be enable a bridge to the future right-sizing of datacenters taking place.

Opportunities for the CIO


Even within the confines of this report and it predictions, there are areas of innovation to still explore. The need to have smaller more ultra-dense datacenters provides the following advantages for enterprises to utilize fast nimble techniques that are simply not economical for a large public cloud provider.

  • Service quality excellence
    In a world of "public" public cloud downtimes announced within a fraction of a second by those relying on these constructs through social networks, service quality excellence and superior uptime remain the hall mark by which business will measure the value of internal IT.


  • Vertical capability integration
    With much of the heavy network and storage traffic being East-West in a classic datacenter, vertical capability-focused integration (as opposed to simple infrastructure integration) will provide ultra-dense configurations handling superior numbers of concurrent workloads whilst maintaining service quality excellence. Typical workloads such as database would be able to take advantage of much faster networking (typically Infiniband 40-56Gbps vs 10GbE in North-South directions) with local modular storage and usually results in massive reductions of footprint, licensing and facility needs. Interlinkage with other capabilities (such as middleware or analytics) can be facilitated over the same high-speed fabric rather than inefficiently "hop-scotching" over the entire classic datacenter.


  • "Disposable" Private Cloud Datacenter Containers
    Ultra-dense capability computing will again be a workable idea using the state of the art in air/liquid cooling. These can basically strip out the entire traditional costs of building out facilities and are simply more flexible for location. Further, they can utilize low factor costs in geographies where a traditional datacenter can't be built without extreme cost being incurred. Full automation of the container allows a notion of "decentralized centralized IT" to still be put in place. The idea of a "disposable" container – literally swapping in/out as newer technology merits has also been bandied about as a way of extracting micro-location financial yields through asset sweating.


  • Urban Datacenter Grids & Locality
    Datacenter containers allow enterprises to physically locate "their" private clouds close to urban business concentrations allowing superior service quality, low latency and ability to differentiate service offerings at software and hardware level. Trends in creating datacenter container parks may well arise that allow a further sharing of common estate costs.


  • Intra-Inter Rack Form Factor Design
    Ultra-density imposes challenges on form factors of traditional rack-and-stack configurations. Many initiatives are underway to further optimize within the rack itself using rack-level disaggregation combined with a rack-level component fabric. Intel's Rack Scale Architecture (RSA) is currently gaining most media attention, but is by no means the first. Indeed Oracle Engineered Systems are architected in a similar manner as are appliances from Teradata and others.


The CIO of today is constantly bombarded with messages from the business and the press about moving all to the "Cloud". However, the potential for enterprise IT innovation remains essentially quite strong.

Complete and realistic business case calculation of public cloud providers, including opex costs for storage, compute and importantly network can be used to justify and finance an innovative cost-effective/cost-neutral enterprise datacenter strategy with much higher standards of security and service quality. The public cloud is providing a generational jump from IaaS in the forms of PaaS and functional SaaS capabilities. However, they do not cater for all the security and uptime quality needs of all enterprises.

To differentiate and capitalize on existing talent and resources, the CIO may need to buck the trend in keeping a fleet of smaller enterprise datacenter containers that are used to extend the "commodity" functionality in the Public Cloud PaaS and SaaS to in/on premise IT as a means of innovating beyond the cycle of the Public Cloud industry.

After all – if everyone does the same thing – then we should expect a similar level of result albeit more efficient than today!

In Memory Computing (IMC) Cloud - so what about the Memory?

There's been a lot of talk in 2013 about in-memory computing (IMC), with Gartner indicating strategic significance in 2012. Very little has been said about the memory needed for IMC!

IMC is claimed to be "new", "radical", "never before done in the industry" etc. Much of this has been from SAP's HANA marketing amongst others. The discussion is IT industry relevant and all workloads delivered locally or through IMC-enabled Clouds!

Larry Page, after proposing holding the Internet in memory in 2000 at the Intel Developer Forum,  moved forward with this idea of holding the Internet in memory -  resulting in Google. He had only 2,400 computers in the Google datacenter then!

The industry has responded in like kind - by stating large amounts of memory have been available in platforms for nigh on a decade. Indeed, the latest Oracle Exadata X3-8 Engineered System has 4TB of RAM and a 22TB of PCIe Flash - non-volatile RAM.


So IMC is not new in the sense SAP and others would have you believe. It is a natural evolution of economies of scale bringing price/GB down accompanied by technological speed & capacity innovations.

A purist approach based on DRAM (nanosecond access) alone has a vast cost difference from NAND Flash (microseconds - 1,000x slower) and spinning disk (milliseconds) technolgies today - whilst being volatile - data gone on power cycling! Economically speaking - a hybrid approach has to be taken as a road to the IMC-Cloud!

Amongst the characteristics facilitating wholesale transformation to full IMC (hardware, software, application architectures) are:

  1. Performance - nanosecond to microseconds as DRAM/Flash currently
  2. Capacity -Terabytes initially and then Petabytes as Flash/Disk currently
  3. Volatility - non-volatile on power cycling much in the same way as Flash/Disk today
  4. Locality - as close as possible to the CPU, but needs to be manageable at cloud scale!

Individually, each of these characteristics has been achieved. Combined, they are technically challenging. Many promising technologies are evolving aiming to solve this quandry and change the face of computing as we know it forever. Basically Massive, Low-power consuming non-volatile RAM!  

Advances are being made in all areas:

  • HMC - Hybrid Memory Cube using stackable DRAM chips resulting in 320GB/s throughput (vs. DDR3 maxing out at 24GB/S) and 90% less space with 70% less energy. Still volatile though!
  • Phase-change memory (PCM/PRAM) producing non-volatile RAM. Micron already has this shipping in 1GB chips (in 2012). This does not require erase-before-rewriting cycles like Flash so potentially much faster. Current speeds are 400MB/s. 
  • HDIMM - Hybrid DIMMs - combine high-speed DRAM with non-volatile NAND storage (Flash). Micron (with DDR4) and Viking Technology (DDR3 NVDIMM) have these technologies with latencies of 25 nanoseconds.
  • NRAM - Carbon nanotube based non-volatile RAM. Nantero and Belgium's IMEC are working jointly to create this alternative to DRAM and scaling below 18nm sizes. Stackable like HMC and all non-volatile.
  • Graphene (single layer of carbon atoms) based non-volatile RAM such as efforts in 2013 in Lausanne, CH.

Which of these will be driving future architectures remains to be seen in the sensitive price-capacity-performance markets. Post-silicon era options based on carbon/graphene nanotube technology would of course power the next wave of compute as well as memory structures.

Questioning "Accepted Wisdom"

So In-Memory Computing is coming - has been for the last decade or so. So, are datacenter infrastructure and application architectures keeping pace or at least preparing for this "age of abundance" where non-volatile RAM is concerned?

In discussions with folks from IT and industry colleagues there is a clear focus on procurement at low price points with IT simply saying "everything is commodity"! This is like saying two cars of identical model/make/engine with different chip management software are the same - one clearly perfoms better than the other! The software magic in these hardware and software stacks makes them anything but commodity.

Many IT shops still think a centralized array of storage is the only way to go. They basically change media within the array to change storage characteristics - 7.2/10/15K RPM spinning disks to SSD drives. That is where their thinking essentially stops!

This short-term thinking will effectively result in the next wave of infrastructure and application sprawl OR revolution through IMC-Cloud enabled vectors turning IT on its colective head.

This would simply be too slow as a model for IMC Clouds. There are some clear trends emerging indicating how CIO/CTO and CFOs can prepare for IMC based datacenters of the future to drastically increase capability while changing the procurement equations:

  • Modular storage containers located close to processor & RAM driving the move away from islands of central/massive/SAN infrastructures.
  • Internetworking needs to be way faster to leverage IMC capabilities. Think 2013 for 40Gbps GbE/Infiniband now. Think 2016 for PCIe4 at 512Gb/s (x16 lane duplex). That speed is needed at least at the intersection points of compute/RAM/Storage!
  • Engineered (hardware and software optimized) for entire platforms. Simply not worth focusing all IT effort on individual best-of-breed components when "the whole needs to be greater than the sum of parts!".
  • Backup architectures need to keep up. Tape remains a cost-effective media for inactive/data-backup data sets, particularly when in open source Linear Tape File System (LTFS) format. A great blog on that from Oracle-StorageTek's Rick Ramsay.
  • Application architectures need to move away from bottleneck-resolution thinking! Most developers don't know what to do with Tbytes of RAM! Applications need to be massively parallel patterns where possible. Developers need to deliver data in real-time!
2013-2016 will see the strong rise of non-volatile memory technology and architectures. CIO/CTOs should be thinking about how they will leverage these capabilities. IT philosophers need to discuss and map out implications before handing over to Enterprise Architectes to enact.

Why is this important for the CIO, CTO & CFO?

Simple server consolidation has had its day! Most IT shops have used server virtualization in one form or another. The early fast returns are almost exhausted! Continuous workload consolidation needs to take center-front stage again- think thousands of workloads per server not 20-40!

Private IMC-Clouds provide an ability for CIOs to keep in-house IT relevant to the business.

CTOs should be thinking about how IMC-Clouds can power the next wave of innovative applications-services-products in an increasingly interconnected always-on manner. Scaling, performance, resiliency to failure should be designed into application platforms - NOT applications themselves. Fast moving application development can then proceed without recreating these features in every app.

For the CFO, IMC-enabled Private Clouds represent dramatic lowering of all costs associated with IT to the business. Consolidating massive chunks of datacenter infrastructure, decommission datacenters, simplify demands on CFO resources for more performance/capacity will allow CFOs to free trapped financial value that can be used directly by the business. Tech-refresh cycles may need to be shortened to bring this vision to fruition earlier!

IMC-enabled Clouds, combined with Intelligent Storage will allow fundamental transformations to take place at paces exceeding even those of hyper-Cloud providers such as Amazon. Business IT can choose to transform


The opinions expressed here are my personal opinions. Content published here is not read or approved in advance by my current employer and does not necessarily reflect the views and opinions of my employer.

Storage Intelligence - about time!

I was reading recently an article about Backblaze releasing storage designs. This is a 180TB NAS device in 4U! Absolutely huge! A 42U rack would be able to have around 1.8Petabyte in a single rack.


When thinking about Petabytes, one thinks about the big players in storage, EMC/NetApp/HDS, selling tens of storage racks covering substantial parts of the datacenter floor space and offering a fraction of this capability.

Vmax_images Fas_index

Clearly, the storage profile of what the large monolithic enterprise arrays offer is different. However, Backblaze clearly highlights the ability to get conventional "dumb" storage easily and at low cost! Packing some flash  cache or SSD in front would already bring these boxes to the same I/O capacity;-)

This makes the case that storage per se is not really a challenge anymore. However, making storage aid in the overall performance equation; making sure that storage helps in specific workload acceleration is going to be critical going forward. Basically Intelligent Storage!

Questioning "Accepted Wisdom"

Many IT shops still think of storage as a separate part of their estate. It should simply store data and provide it back rapidly when asked - politely. The continuing stifling of innovation in datacenters due to having a single answer for all questions - namely VMware/hypervisors and server virtualization - tends to stop any innovative thinking that may actually aid an organisation to accelerate those parts of the application landscape leveraging revenue.

Some questions that came to mind and also echoed by clients are:

  • Disk is cheap now. SSD answers my performance needs for storage access. Is there something that together with software actually increases the efficiency of how I do things in the business?

  • For whole classes of typical applications, structured data persistence pools, web servers etc what would "intelligent" storage do for the physical estate and the business consumers of this resource?

  • How can enterprise architecture concepts be overlaid to intelligent storage? What will this mean to how future change programmes or business initiatives are structured and architected?

  • How can current concepts of intelligent storage be used in the current datacenter landscape?

We are seeing the first impact of this type of thinking in the structured data / database world. By combining the database workload with storage and through software enablement we get  intelligent acceleration of store/retrieval operations. This is very akin to having map-reduce concepts within the relational database world.

Further combining storage processing, with CPU/RAM/Networking offload of workload specific storage requests, facilitatest unprecedented scale-out, performance and data compression capabilities.

Oracle's Engineered Systems, the Exadata Database Machine in particular, represents this intelligent storage concept, amongst other innovations, for accelerating the Oracle database workload.

These workload specific constructs foster security of precious data assets, physically and logically. This is increasingly important when one considers that organisations are using shared "dumb" storage for virtual machines, general data assets and application working data sets.

In the general marketplace other vendors (IBM PureSystems + DB2, Teradata, SAP HANA etc) starting to use variations of the technologies for intelligent storage. The level of maturity varies dramatically, with Oracle having a substantial time advantage as first mover.

2013-2015 will see more workload focused solutions materializing, replacing substantial swathes of datacenter assets built using the traditional storage view.

Why is this important for the CIO, CTO & CFO?

Intelligent workload-focused storage solutions are allowing CIO/CTOs to do things that were not easily implemented within solutions based on server virtualization technology using shared monolithic storage arrays - dumb storage - such as in the VMware enabled VCE Vblock and HP CloudSystem Matrix - which are effectively only IaaS solutions.

Workload specific storage solutions are allowing much greater consolidation ratios. Forget the 20-30-40 Virtual Machines per physical server. Think 100s of workloads per intelligent construct! An improvement of 100s of percent over the current situation!

It is important to verify how intelligent storage solutions can be a part of the CIO/CTO's product mix to support the business aspirations as well as simplify the IT landscape. Financing options are also vastly simplified with a direct link between business performance and physical asset procurement/leasing:

  • Intelligent storage removes architectural storage bottlenecks and really shares the compute/IO/networking load more fully.

  • Intelligent storage ensures those workloads supporting the business revenue generating activities are accelerated. Acceleration is linked to the cost of underlying storage assets. As cost of NAND flash, SSDs and rotating disks drop, more is automatically brought into the storage mix to reduce overall costs without disrupting the IT landscape.

  • Greater volumes of historic data are accessible thanks to the huge level of context sensitive, workload-specific data compression technologies. Big data analytics can be powered from here, as well as enterprise datawarehouse needs. This goes beyond simple static storage tiering and deduplication technologies that are unaware of WHAT they are storing!
  • Workload-specific stacking supports much higher levels of consolidation than simple server virtualization. The positive side effects of technologies such as Exadata include the rationalization of datacenter workload estates in terms of variety, operating systems can be rationalized and generally have net-net healthier estate. This means big savings for the CFO!

Intelligent storage within vertically engineered workload specific constructs, what Gartner calls Fabric Based Infrastructure present a more cogent vision of optimizing the organizational's IT capability. It provides a higher level of understanding how precious funding from CFOs is invested to those programmes necessary for the welfare of the concern.

CIO/CTOs still talking about x86 and server virtualization as the means to tackle every Business IT challenge would be well advised to keep an eye on this development.

Intelligent storage will be a fundamental part of the IT landscape allowing effective competition with hyperscale Cloud Providers such as Google/Amazon and curtailing the funding leakage from the business to external providers.



Desktop of the Future? What Desktop? - Shake up of the Digital Workspace!

I have been saying for some time that CIO/CTOs should be taking a serious look at the "desktop" of the future. What type of applications are needed, where they are sourced from, form factor and the impact on workplace design.

Tablets are already firmly entrenched in consumer hands, and indeed the hands of professionals. However, they fail to make a dent against the estate of desktop/laptop devices and the associated workplace implications.

Google Glass perhaps represents the next evolution of the desktop->laptop->netbook->tablet saga. This not only challenges the existing notions of mobility, but adds to the rich experience that you and I are demanding.


Google-glass-patent-2-21-13-01This is not necessarily a new idea. We have been seeing this in sci-fi movies for decades. Indeed, Sony also had a similar ambition around a decade ago.

However, this design, with modern materials and the massive backing of Google may well succeed en masse compared to previous designs.


The ability to link into Cloud based backends, stream applications magically/wirelessly, as well as surely providing input devices of corresponding smartness - maybe something tracking eye movements as we see for disabled individuals.

The really interesting feature that is involved is augmented reality for everyone. Google has been investing heavily in technology for mapping the real virtual in their 3D Google Maps. This investment is really going to pay off by providing a compelling lead.

Augmented reality

The practical application of information gathered all around us, and mapped onto our line of vision will increase our ability to navigate an increasingly complex world.

Such a concept could be easily translated to our business or indeed consumer worlds. Why can a stock trader not look around, cuk in relevant information about world events, annotate verbally on impact on stock price trends, and then have an automated trading engine supporting the actual trading desk.

A consumer could walk into a shopping center and automatically be presented with details of competitive offers in the area/online, trace the production chain to check the articles green credentials, and indeed order online from the same shop that the person is currently in - reducing the need to keep large quantities of stock at hand. And the effect on advertising.....well!

An office or knowledge worker would be literally able to grab data, manipulate it, enter data and essentially carry out any other task without the need for the traditional stuffy workplace environments we are surrounded by. Yuo never know, perhaps we could end up with workplaces that look like....(film: Matrix Trilogy - Zion control room)

Questioning "Accepted Wisdom"

Traditional ideas of the workspace environment being provided for employees and partners should seriously be challenged. It is not necessarily about whether a Microsoft 8 or Ubuntu will be rolled out. It is very much about providing a safe secure working environment, protecting data assets of an organization and increasing efficiency dramatically.

  • Challenge the idea of a desk! Find out what types of work environments are conducive for your staff, ranging from highly creative to highly task oriented individuals.

  • Understand how the walls of the organization can be made safely permeable. How can the technology enable an individual to be out there - physically, socially, online etc? What if voice recognition and real-time language translation capabilities are added?

  • What applications classes are being used? Where are they being consumed, in which form, are they on-premise sourced or from a SaaS/Public Cloud provider?

  • Understand the impact on the physical workspace, its constituent parts, how work and play can be mixed? Cost control and making creative workplaces can be benefit dramatically in this mindshift!
The way the physical world and information is being represented as digital assets that can be manipulated without regard to time and space constraints represents a dramatic opportunity for organization willing to challenge the current accepted wisdom.

Why is this important for the CIO, CTO & CFO?

CIO/CTOs are responsible for the overall digital enterprise architecture and workplace environment. The CFO has the fun work of ensuring sufficient funding whilst maintaining cash flow and a generally healthy financial posture. 

Rather than simply accepting what your IT staff are telling you regarding options for workspace computing based on a legacy view of software and physical computing devices, challenge what could be done by leaping forward.

  • How would this improve the financial situation regarding licensing, physical desktop estate, building space, furniture and employee productivity?

  • Understand what is needed from a human capital management perspective? Legislation and health may well be key drivers or blockers. What would such a workplace mean for talent management and making your organisation a 1st class address for potential graduates/employees?

  • From an enterprise architecture (a middle up-down architectural approach) what would be needed to service such a workplace environment and the implications on business stakeholder aspirations as they expressed currently?

  • Security, physical and digital, needs to be more pervasive but also more invisible. Data assets may needs to be stored in more secure ways in corporate datacenters. Databases are fine, but database firewalls, controlling who is doing what and why, would be required. Ditto for other areas powering the enterprise.
  • Database silos, using workload stacking and virtualization makes sense. However, combining them into the general compute environments together with other general data assets is probably not secure enough. A level of physical security should be ensured.
  • Understand if current investment in virtual desktop infrastructure (VDI) makes sense. Does it capture new digital realities? Understand new demands on mobility. Are you investing in something that literally will disappear?

Many organisations are experimenting at scale with full mobile (as we understand it today - usually tablets, smart phones, mobile video conferencing) technology.

They are experiencing the implications on the cost of providing IT capability, the impact on how employees are exploiting the technology, and overall performance of the organisation.Subtle areas such as training, motivation, social contact implications if working away from a "traditional office environment" need to be factored in.

This is a great chance to innovate and basically get the organization in top form for globalisation where labour factor costs are continually driving employment further offshore (until it comes full circle - the earth is indeed round and not flat) Smileyearth2



The Shape of Things to Come!

A lot of what I do involves talking with thought leaders from organizations keen to transform how they do business. In some cases, they espouse thoughts moving along general industry lines or marketing. However, in some cases, there is real innovative thought taking place. I believe firmly innovation starts with questioning the current status quo.

We are bombarded by Intel x86 as the ultimate in commodity processor offering everything one could possibly imagine on the one side, and public cloud on the other hand as the doom of in-house IT centers. It is incumbent on all in this industry to think beyond even the cloud as we know it today.

Questioning "Datacenter Wisdom"

This blog entry in entitled the Shape of Things to come with a clear series of ideas in mind:

  • System-on-a-Chip (SOC) are getting very powerful indeed. At what point are these so powerful that they represent the same order of magnitude as an entire hyperscale datacenter from Google or Amazon with a million machines inside?

  • Why does in-house IT have to move out to the cloud? Why could hyperscale clouds not be built up from capacity that organizations are already putting in place? This would be akin to the electricity grid as the transport for capacity created from multiple providers. Borrowing capacity could be done in-industry or across-industries.

  • Why is there disaggregation of all components at a physical datacenter level (CPU, RAM, storage, networking etc) rather than having assembly lines with appliances/constructs hyper-efficient at a particular task within the enterprise portfolios of services and applications?

  • Why are servers still in the same form factor of compute, memory, networking and power supply? Indeed why are racks still square and datacenter space management almost a 2-dimensional activity? When we have too many people living in a limited space we tend to build upwards, with lifts and stairs to transport people. Why not the same for the datacenter?

I 'm not the only one asking these questions. Indeed, in the industry the next wave of physical manifestation of new concepts is taking place albeit slowly. I wanted to share some industry insight as examples to whet the appetite.

  • At Cornell University a great whitepaper on cyclindrical racks using 60GHz wireless transceivers for interconnects within the rack show a massively efficient model for ultrascale computing.
  • RoundWirelessServerack

  • Potentially the server container would be based on a wheel with servers as cake slice wedges plugged into the central tube core. Wheels would be stacked vertically. Although they suggest wireless connectivity, there is no reason why the central core of the tube could not carry power, networking and indeed coolant. Indeed the entire tube could be made to move upwards and downwards - think tubes kept in fridge like housings (like in the film Minority Report!)
  • MinorityReport

  • One client suggested that CPUs should be placed into ultracooled trays that can use the material of the racks as conductors and transport to other trays full of RAM. We do this with hard disks using enclosures. Indeed Intel does 3D chip stacking already!
    • Taking the Intel 22nm Xeons with 10 cores or indeed Oracle's own SPARC T5 at 28nm and 16 cores as building blocks
    • A 2U CPU tray would allow say 200 such processor packages. This is an enormous capability! For the SPARC T5 this would be 3200 cores, 25600 threads and 11Thz of aggregate power!
    • Effectively, you could provide capacity on the side to Google!
    • A RAM tray would basically allow you to provide 20TB+ depending on how it is implemented (based on current PCIe based SSD cards).
  • Fit-for-purpose components for particular workloads as general assembly lines within an organization would fit in well with the mass-scale concepts that the industrial and indeed digital revolutions promoted.
    • If we know that we will be persisting structured data within some form of relational database, then why not use the best construct for that. Oracle's Engineered Systems paved the way forward for this construct.
    • Others are following with their own engineered stacks.
    • The tuning of all components and the software to do a specific task that will be used for years to come is the key point!

So the technical components in this radical shake up of the datacenter are materializing. We haven't even started to talk about some of te work happening in material science providing unparalleled changes in CPUs (up to 300GHz at room temperature) or even non-volatile RAM totally replacing spinning disk and possibly SSD and DRAM.

Why is this important for the CIO, CTO & CFO?

Customers typically ask whether they should move everything out to cloud providers such as Google/Amazon or private cloud hosters such as CSC/ATOS/T-Systems. Well looking at the nexus of technological change that is almost upon us, I would say that at some level it might make sense to evaluate the mix of on-premise and off-premise resource.

The Cloud is effectively a delivery model - some applications such as email clearly can be in the public cloud - bearing in mind privacy issues. However the capabilities needed for an organization to thrive as expressed in Enterprise Architecture in order to exploit market forces can be expressed in other ways.

  • Server virtualization relies on workloads not taking all the resources of a physical server. You should be questioning why the software, the most expensive components, is not being used to its maximum? Solving server acquisition costs does not reduce costs for you in a meaningful way.

  • Entertain the idea that with acceleration at every level of the stack, information requests may be serviced in near-realtime! The business should be asking what it would do with that capability? What would you do differently?

  • Datacenter infrastructure may change radically. It may well be that the entire datacenter is replaced by a tube stacked vertically that can do the job of the current football field sized datacenter. How can you exploit assembly line strategies that will already start to radically reduce the physical datacenter estate? Oracle's Engineered Systems are one approach for this for certain workloads, replacing huge swathes of racks, storage arrays and network switches of equipment.

  • Verify if notions of desktops are still valid. If everything is accessible with web based technologies, including interactive applications such as Microsoft Office, then why not ensure that virtual desktops are proactively made obsolete, and simply provide viewing/input devices to those interactive web pages.

  • Middleware may well represent a vastly unexplored ecosystem for reducing physical datacenter footprints and drastically reducing costs.
    • Networking at 100+Gbps already enables bringing your applications/web powered effective desktops with interaction to the users' viewing devices wherever they are.
    • Use intra-application constructs to insulate from the technical capability below. Java applications have this feature built-in, being cross platform by nature. This is a more relevant level of virtualization than the entire physical server.

  • Security should be enabled at all layers, and not rely on some magic from switch vendors in the form of firewalls. It should be in the middleware platforms to support application encapsulation techniques, as well as within pools of data persistence (databases, filesytems etc).
Enterprise architecture is fueling a new examination of how business defines the IT capabilities it needs to thrive and power growth. This architecture is showing the greater reliance on data integration technologies, speed to market and indeed the need to persist greater volumes of data for longer periods of time.

It may well be incumbent on the CIO/CTO/CFO to pave the way for this brave new world! They need to be already ensuring that people understand what is impossible now, technically or financially, will sort itself out. The business needs to be challenged on what it would do in a world without frontiers or computational/storage limitations?

If millions of users can be serviced per square round meter of datacenter space using a cylindrical server tube wedge/slice - why not do it? This is not the time for fanatics within the datacenter that are railroading discussions to what they are currently using - or to provide the universal answer "server virtualization from VMware is the answer, and what is the question?".

Brave thinking is required. Be prepared to know what to do when the power is in you hands. The competitive challenges of our time require drastic changes. Witness what is happening in the financial services world with traders being replaced by automated programs. This requires serious resources. Changes in technology will allow this to be performed effortlessly with the entire stock market data kept in memory, and a billion risk simulations run per second!



A Resurgent SPARC platform for Enterprise Cloud Workloads (Part 2) - SPARC T5

Some time ago, I blogged about the resurgence of the SPARC platform. The then newly designed SPARC T4 was showing tremendous promise in its own write to be able to take up its former mantle of being an innovation leader running extreme workloads with the Solaris 11 operating system.

Indeed, it was used as the driving engine of the SPARC Supercluster for dealing with not just massive acceleration of Oracle database workloads using the Exadata Storage Cell technology, but the ability to combine firmware embedded near-zero overhead virtualization concepts for electrically separate logical domains, carving up the physical hardware, and Solaris zones which allow near-native "virtual machines" sharing an installed Solaris operating system.

Up to 128 virtual machines (zones) supported on a system - a vast improvement over the 20-30 one gets under VMware-like hypervisors typically!

This welcome addition to the wider Oracle engineered systems family allowed the missing parts of the datacenter to be consolidated - these being typically glossed over or totally skipped when virtualization with VMware-like hypervisors was discussed. Customers were aware that their mission critical workloads were not always able to run with an x86 platform which was then further reduced in performance using a hypervisor to support large data set manipulation.

Well the rumor mills have started as the run up to Oracle Openworld 2012 at the end of September. One of the interesting areas is the "possible" announcement of the SPARC T5 processor. This is interesting in its own right as we have steadily been seeing the SPARC T4 and now the T5 having ever greater embedded capability in silicon to drive database consolidation and indeed the entire WebLogic middleware stack together with high-end vertical applications such as SAP, EBusiness Suite, Siebel CRM and so on.

Speculating on what "rumors" and the Oracle SPARC public roadmap, I'd like to indicate where I see this new chip making inroads in those extreme cloud workload environments whilst maintaining the paradigm of continuous consolidation. This paradigm that I outlined in a blog in 2010 is still very relevant - the SPARC T5 providing alternative avenues than simply following the crowd on x86.

Questioning "Datacenter Wisdom"

The new SPARC T5 will have, according to the roadmap the following features and technologies included:

  • Increasing System-on-a-Chip (SOC) orientation providing ever more enhanced silicon accelerators for offloading tasks that software typically struggles with at cloud scale. This combines cores, memory controllers, I/O ports, accelerators and network interface controllers providing a very utilitarian design.
  • 16 cores from the T4's 8-core. This takes them right up to the top end in core terms.
  • 8 threads per core - giving 128 threads of execution per processor providing exceptional performance for threaded applications such as with Java and indeed the entire SOA environment
  • Core speeds of 3.6GHz providing exceptional single threaded performance as well as the intelligence to detect thread workloads dynamically (think chip level thread workload elasticity)
  • Move to 28nm from 40nm - continuous consolidation paradigm being applied at silicon level
  • Crossbar bandwidth of 1TB/s (twice that of the T4) providing exceptional straight line scaling for applications as well as supporting the glueless NUMA design of the T5
  • Move to PCIe Generation 3 and 1TB/s memory bandwidth using 1GHz DDR3 memory chips will start to provide the means of creating very large memory server configuration (think double-digit TB of RAM for all in-memory workload processing)
  • QDR (40Gbps) Infiniband private networking
  • 10GbE Public networking
  • Database workload stacking becomes even more capable and effective than simple hypervisor based virtualization for datacenter estate consolidation at multiple levels (storage, server, network and licensed core levels)

This in itself at the processor level is really impressive, but the features that are on the roadmap aligned to the T5 possibly are the real crown jewels:

  •  on-die crypto accelerators for encryption (RSA, DH, DSA, ECC, AES, DES,3DES, Camellia, Kasum) providing excellent performance through offloading. This is particularly relevant in multi-tenant Cloud based environments
  • on-die message digest and hashing accelerators (CRC32c, MD5, SHA-1, SHA-224, SHA-256, SHA-384, SHA-512) providing excellent security offloading. Again particularly relevant in multi-tenant environments
  • on-die accelerator for random number generation
  • PCIe Generation 3 opens the door to even faster Infiniband networking (56Gbps instead of the current 40Gbps - with active-active links being possible to drive at wire speed)
  • Hardware based compression which will seriously reduce the storage footprint of databases. This will provide further consolidation and optimization of database information architectures.
  • Columnar database acceleration and Oracle number acceleration will provide extremely fast access to structured information. Further, when combined with in-memory structures, the database will literally be roaring !

Indeed when we think that the Exadata Storage cells will also be enhanced to support new chip generations, flash density as well as other optimizations, the next SPARC Supercluster (which has the embedded Exadata storage cells) will literally be one of the best performing database platforms on the planet!

To ignore the new SPARC T5 (whenever it arrives) is to really miss a trick. The embedded technology provides true sticky competitive advantage to anyone that running a database workload or indeed multi-threaded applications. As a Java platform, middleware and SOA platform as well as vertical application platform, the enterprise can seriously benefit from this new innovation.

Why is this important for the CIO & CFO?

CIOs and CFOs are constantly being bombarded with messages from IT that x86 is the only way to go, that Linux is the only way to go, that VMware is the only way to go. As most CFOs will have noted by now:

  • Financially speaking - the x86 servers may have been cheaper per unit, but the number of units is so large to get the job done that any financial advantage that might have been there has evaporated!
  • Overall end-2-end costs for those services that the CIO/CFO signed off on are never really well calculated for the current environment.
  • Focused investment on those activities that support revenue streams and those technologies that will continue to do that for at least the next decade with capacity upgrades of course
  • There must be other ways of doing things that make life easier and more predictable

Well Engineered Systems with the new SPARC T5 represent a way for the CIO/CFO to be able to power those projects that need investment which in turn drive revenue and value. The ability to literally roll the SPARC SuperCluster or any other Engineered System is going to be instrumental in:

  • Shortening project cycles at the infrastructure level
    • don't lose 6 months on a critical ERP/CRM/Custom application project in provisioning hardware, getting unexpected billing for general infrastructure layers such as networking that have nothing to do with this project, IT trying to tune and assemble, getting stuck in multi-vendor contract and support negotiations etc.
    • That time can be literally worth millions - why lose that value?
  • Concentrate valuable and sparse investment strategies literally to the last square meter in the datacenter!
    • If that next project is a risk management platform, then IT should be able to give exactly to the last datacenter floor tile the resources that are needed for that one project alone and the cost
    • Project based or zero-budgetting will allow projects to come online faster, predictably, reuse of existing platforms dealing with the load as well as supporting continuous workload consolidation paradigms
    • Finance enterprise architecture projects that put in the enabling conditions to support faster turnaround for critical revenue focused/margin increasing project activity
Engineered systems are already using the technologies that the rest of the industry is trying to re-package to meet the challenges customers are facing now and in the coming years.The lead is not just in technology but also the approach that customers are demanding - specific investments balanced with specific revenue generating high-yield business returns.

As a CIO it is important to recognize the value that Engineered Systems and the SPARC platform, as part of an overall datacenter landscape, bring in addressing key business requirements and ensure an overall simplification of the Datacenter challenge and large CAPEX requirements in general.

As Oracle and others proceed in acquiring or organically developing new capabilities in customer facing technologies, managing exabyte data sets it becomes strategically important to understand how that can be dealt with.

Hardware alone is not the only answer. Operating systems need to be able to deal with big thinking and big strategy as do applications and the hardware. By creating balanced designs that can then scale-out a consistent effective execution strategy can be managed at the CIO/CTO/CFO levels to ensure that business is not hindered but encouraged to the maximum through removing barriers that IT may well have propagated with the state of the art many years ago.

Engineered Systems enable and weaponize the datacenter to directly handle the real-time enterprise. High-end operating systems such as Solaris and the SPARC processor roadmap are dealing with the notions of having terabyte datasets, millions of execution threads and thousands of logical domains with hundreds of zones (virtual machines) each per purchased core.

Simply carving up a physical server's resources to make up for the deficiencies of operating system/application in dealing with workloads can't be an answer by itself. This is what is also fueling the Platform-as-a-Service strategies partly. How to get systems working cooperatively together to deal with more of the same workload (e.g. database access/web server content for millions of users) or indeed different workloads spread across systems transparently is the question!

High performance computing fields have been doing just this with stunning results albeit at extreme cost conditions and limited workloads. Engineered systems are facilitating this thinking at scale with relatively modest investment for the workloads being supported.

It is this big thinking from organizations such as Oracle and others, who are used to dealing with petabytes of data, and millions of concurrent users that can fulfill  requirements expressed by the CIO/CTO/CFO teams. If millions of users needing web/content/database/analytics/billing can be serviced per square meter of datacenter space - why not do it?


The opinions expressed here are my personal opinions. Content published here is not read or approved in advance by Oracle and does not necessarily reflect the views and opinions of Oracle.

Inflexibility of Datacenter Culture - 'The way we do things round here' & Engineered Systems

With a focus on large enterprise and service provider datacenter infrastructures, I get the chance to regularly meet with senior executives as well as rank and file administrators - top-2-bottom view.

One of the things that as always struck me as rather strange is the relative "inflexible" nature of datacenters and their management operations.

As an anecdotal example, I recall one organization with a heavy focus on cost cutting. At the same time the datacenter management staff decided that they would standardize on all grey racks from the firm Rittal. Nothing wrong here - a very respectable vendor.

The challenge arising was:

  • The selected Rittal racks at that time were around 12,000 Euro each approximately
  • The racks that came from suppliers such as Dell, Fujitsu, Sun etc were around 6,000 Euro each

See the problem? 50% savings thrown literally out of the window because someone wanted all grey racks. When we are talking about a couple of racks - that is no big deal. With say 1000 racks we are looking at over-expense of 6 million Euro - before anything has been put into those racks!

Upon asking why it was necessary to standardize the racks, the answers I got were:

  • independence from server manufacturers
  • create rack and cabling rows before servers arrive to facilitate provisioning
  • simpler ordering
  • perceived "better-ness" as enclosures is a Rittal specializ

Sounds reasonable at first glance - until we see that racks are all engineered t support certain loads, and typically optimized for what they will eventually contain. Ordering was also not really simpler - as the server vendors made that a "no brainer". Perception of quality was not validated either - just a gut feel.

The heart of the problem as came out later was that the datacenter would benefit from having everything homogenous. Homogenous = standardized for datacenter staff.

The problem with this is that datacenters are not flexible at all, they have a focus on homogeneity and ultimately cost lots of money to the business financing them.

In an age where flexibility and agility means the literal difference between life and death for an IT organization, it is incumbent on management to ensure that datacenter culture allows the rapid adoption of competitive technologies within the datacenter confines.

Standardized does not mean that everything needs to be physically the same. It involves the processes o dealing with change in such as way that it can be effected quickly, easily and effectively to derive key value!

I indicated the recent trend of CIOs reporting to CFOs and this would have provided financial stewardship and accountability in this case - getting staff and managers to really examine their decision in the light of what was best for the organization.

Questioning "Datacenter Wisdom"

The focus on homogenous environments has become so strong that everything is being made to equate a zero-sum game. This happens in cycles in the industry. We had mainframes in the past, Unix, Linux (which is Unix basically for x86), Windows - and the latest is VMware vSphere and all x86!

Don't get me wrong here - as a former EMC employee - I have the greatest respect for VMware and indeed the potential x86 cost savings.

What is a concern is this is translated to "strategy". In this case the approach has been selected without understanding why! It is a patch to cover profligate past spending and hoped that magically all possible issues will be solved.

After all - it is homogenous. All x86, all VMware, all virtualized. Must be good - everyone says it is good!

See what I mean - the thinking and strategizing has been pushed to the side. Apparently there is no time to do that. That is really hard to believe, as this is one of those areas that fall squarely into the CIO/CTO/CFO's collective lap.

There are other approaches, and indeed they are not mutually exclusive. Rather they stem from understanding underlying challenges - and verifying if there are now solutions to tackle those challenges head-on.

Why is this important for the CIO?

At a time of crisis and oversight, it is incumbent on the CIO to question the approach put on his/her table for Datacenter Infrastructure transformation.

The CIO has the authority to question what is happening in his/her turf.

At countless organisations, I have performed strategic analysis of macro challenges mapped to the IT infrastructure capability of an organization to deal with those changes. Time and again, when discussing with the techies and managers (who were from a technical background but seem to struggle with strategy formulation itself) it was shown that the marginal differences in technologies were not enough to justify the additional expenditure - or that there were other approaches.

Engineered Systems, in Oracle parlance, are one such challenge. They do not "fit" the datacenter culture. They can't be taken apart and then distributed into the slots that are free in racks spread over the datacenter.

From a strategy perspective, a great opportunity is not being exploited here. Engineered systems, such as Exadata, Exalogic, SPARC SuperCluster, Exalytics,  and Oracle Data Appliance represent the chance to change the Datacenter culture and indeed make the whole datacenter more flexible and agile.

They force a mindset change - that the datacenter is a housing environment that contains multiple datacenters within it. Those mini-datacenters each represent unique capabilities within the IT landscape. They just need to be brought in and then cabled up to the datacenter network, power, cooling and space cabilities of the housing datacenter.

There are other assets like this in the datacenter already - enterprise tape libraries provinding their unique capability to the entire datacenter. Nobody tries to take a tape drive or cartridge out and place it physically somewhere else!

Engineered Systems are like that too. Taking Exadata as an example. This is clearly assembled and tuned to do database work with the Oracle Database 11gR2 platform. It is tuned and tweaked to do that extremely well. It breaks down some of the traditional barriers of datawarehouse and OLTP workloads and indeed allows database workloads to be "stacked".

Taking the idea of what a datacenter really should be (facilities for storing and running IT infrastructure) and being flexible - Exadata should literally be placed on the floor, cabled to the main LAN and power conduits and the database infrastructure platform is in place. After that databases can be created in this mini-Datacenter. The capability is literally available immediately.

Contrast this with creating lots of racks in rows where it is not certain what will be in those racks, put VMware everywhere, add lots of SAN cabling as I/O will always be an issue - and then spend ages in tuning performance to make sure it works well.

The CIO should identify this as a waste of time and resources. These are clever people who should be doing clever things for the benefit of an organisation. It would be similar to the idea of buying a car to drive around in OR getting the techies to buy all the components of the car and trying to assemble.

This loss of value inherent in the Exadata when taking a x86/hypervisor route to creating many virtual virtual machines running each a separate database make no real sense in the case of databases.

The CIO can use valuable organizational knowledge gained over many years regarding the functions the business needs. If, as in this example, it is the ability to store/retrieve/manage structured information at scale - the answer should literally be to bring in that platform and leverage the cascading value it provides to the business.

Neither x86 or a certain operating system is "strategically" relevant - in that it is a platform - and the normal DBAs can manage this using tools they already know. This mini-datacenter concept can be used in extremely effective ways and supports the notion of continuous consolidation.

CIOs can get very rapid quick-wins for an organization in this way. Datacenter infrastructure management and strategy should be considered in terms of bringing in platforms that can do their job well with software and hardware well tuned to run together. Further, they should reduce "other" assets and software that is needed.

Exadata does with not needing a SAN switch/cabling infrastructure - it encapsulates paradigms of virtualization, cloud and continuous consolidation. This will drive deep savings and allow value to be derived rapidly.

Challenge datacenter ideas and culture in particular. Agility requires being prepared to change things and being equipped to absorb change quickly!



Cloud Transformatics - CIOs/CTOs/CFOs Challenging Industry "Wisdom"

Hi Folks. After a long absence from blogging attributed to a move of apartment, job and some grandchildren, I thought it was time to take up the proverbial pen.

One of the big changes that took place was a change of jobs - in my case moving from EMC to Oracle - Blue to Red literally. In making that move there was some substantial levels of orientation and technology to absorb.

Let's face it, it is not every day that you are expected to learn about all those high-end enterprise applications. This takes time. You probably know what I am talking about - assets absorbed from Sun, StorageTek and a host of other acquisitions based in the Cloud coupled with databases, big data, data architecture strategies and the list goes on.

One of things that did come up was to understand what enterprise and industry wisdom mean to people. At my previous employer, enterprise applications were big complex constructions using by association big complex storage arrays, with big complex SANs and big complex backup environments. If possible, this should all be wrapped in a big complex virtual infrastructure based on VMware, preferably using Cisco UCS server blades and switches. Fine. No problem - as there was also value to be derived for the client.

However, the nature of complex applications at EMC was at that time looking at things like large Microsoft Exchange deployments, or database environments supporting ERP, Microsoft SharePoint and indeed mainly looking at the storage ramifications of thousands of users hitting disk in parallel.

However, the enterprise is vastly more complex than that. The ability to interact and manage the user experience of customers through sophisticated CRM offerings, the ability to manage oceans of structured data using massively parallel databases and indeed prepare for new techs like big data is a daunting task.

It is no wonder that the world of x86 server virtualization only penetrated to a certain level of an organization. The industry wisdom of "it must be on x86", "must use horizontal pools of infrastructure resources", "must use consolidated centralized storage" and "must be virtualized in VMware or some other hypervisor to be Cloud ready" is not a sufficiently complete answer to tackle these themes.

That did not stop every Tom, Dick and Harry from trying to sell a hypervisor of some sort with all the associated paraphernalia. This in turn generated a spree of IT organizations trying to recreate what they already had but just delivered "slightly differently".

Let's face it folks, Cloud is not "new" in the technology sense - no matter how many times the marketing departments of certain organizations try to convince you. More accessible -yes, easier to use - yes, new - no!

It is no wonder then that it has taken the IT industry some time to grapple with what Cloud and this "industry wisdom" all means for them. This has led to some important questions arising -

  • "must it always be x86?",
  • "should I forget all the last 30 years of IT knowledge?",
  • "what is the role of IT in an organization?",
  • "Dare a different approach be used not based on x86 exclusively?",
  • "can a normal Unix be used in conjunction to a Linux/Windows on non-86?".
  • "can I get sustained competitive advantage be being only commodity x86/Linux based?"

With the current industry trends and wisdom, I believe that it is incumbent on CFOs, CIOs and indeed CTOs to take a little step back. CTOs have the chance to examine technology for 'fit for purpose' and innovation certainly, whilst the CIO needs to ensure its practical application for the good of the organization.

The recent trend of CIOs reporting to CFOs is an interesting change. The CFO represents the ability of an organization to challenge what it is doing and why it is doing this. Innovation and creative thinking driven by financial stewardship!

CFO Trigger to Innovate and Question "Industry Wisdom"

Rather than viewing CFO oversight as a burden, it may well be that this "get back to basics" is a blessing in disguise. IT organizations get entrenched in their ways. There are many reasons to continue doing some things as they are - but plenty that need to change in light of CFO oversight.

However, there are some things that simply need to have a much higher level of efficiency. I have been meeting many customers using the Oracle database, and am surprised to see how few of them really use all the features that are available.

Indeed, the IT organization says "thou be too expensive matey", and then goes off on a spending binge to bring in a complete new stack of VMware with SAN storage, switches, servers and what have you - whilst reinventing ITIL so that it fits VMware operations. Ummm. That is not necessarily cheaper.

Don't get me wrong. This is not a mistake - but it is based on a pattern of break it all up so that it fits on a small x86 server and then virtualize it - how else do you sell a VMware/chargeable hypervisor license?

Well there are some workloads that fill up the entire machine - databases supporting large ERP installations. They can be broken up - but certain features like live VM migration of a running DB instance to another machine don't make a lot of sense when the VM has hundreds of GB/TB storage and 1/2 a terabyte of RAM.

In this case, the reverse pattern - antipatterns -  is perhaps a better one - literally using machines in an aggregation to service the workload - spreading the DB instance across multiple machines - this already exists in Oracle RAC. This is not new - and works pretty well.

We see other patterns like this in the Google datacenters - where search workloads are "spread" across servers working in concert - wow sounds like a cluster!

The CFO can and indeed should work with CIO/CTOs to start questioning investment decisions. The one thing I saw with IT organizations using VMware/SANs/SANswitches/etc was that from a financial perspective something was not quite right:

  • IT Investment that was leveraged on the back of a genuine critical business project was spread around the datacenter infrastructure
  • Storage arrays were expanded to accommodate the project, new switches were acquired, new licenses purchased etc.
  • Everything was mixed up - to the point that IT could not say where the money went - and the  business could not verify that it had received what it actually wanted.
  • Billing and chargeback were either non-existent or so primitively applied that it failed to gain traction

This was all justified through the IT organization rejigging its numbers. Not a good sign! The strength of the original investment was diluted through spreading it around such that the business unit really needing those resources was effectively "short changed" and trying to simply "make do".

Why is this important for the CIO?

At a time of crisis and oversight, it is incumbent on the CIO to question the approach put on his/her table for transformation or rejuvenation projects.

Verify whether there is another approach. Verify the patterns that are emerging. Remember, the reason we had wholesale x86 virtualization was that the servers running their particular workloads were not using hardware to its maximum capability or IT had simply decided for isolation reasons to add more servers etc.

  • If servers are being used to their practical maximums - does the IT organization still need to invest in a chargeable hypervisor?
  • If isolation can be provided differently than by encapsulating an entire operating system stack (a VM) - does the IT organization need to still make virtual machines and acquire more licenses for a hypervisor and more resources for server/SAN etc?

x86 was also the commodity route - costs less - doesn't matter what the workload it - just get ton loads of them - we'll worry about the datacenter space/power we need later. Well mainframes do a hell of a lot of work - albeit complex to manage. Big iron Unix boxes were simpler to operate but also doing a lot of complex workloads in parallel.

  • If a larger server can intrinsically do the job of many smaller servers with high levels of reliability, isolation and performance - do we need to invest in many smaller servers? - do we need to get x86 only? - must it be Linux?

If we examine the world of tomorrow - the idea of holding structured information of some sort even if acquired from unstructured data sources as in the case of big data is still valid. Logical isolation is still needed and we do not need to throw out the last 3 decades of database innovation just because some marketing scheme is based on that.

The same and more applies for applications - value added apps are available to an increasingly mobile customer base demanding access 24x7 and from any device. The web and indeed technologies that encapsulate applications like Java are being used to address these needs. SOA architectures were developed to decompose applications to reusable components that could be recomposed to add more value and shorter time-2-value.

These are the real enterprise applications that drive value. That is where the investment should flow. Investment needs to be concentrated into those areas to make the dent in the business revenue that is needed. This implies a "vertical" architecture model of investment.

However, IT organizations need to manage large estates. They think "across" those estates - a "horizontal" pattern of activity. Good for IT - bad for concentrated vertical investment based on projects.

However, I believe that horizontal manageability can be achieved through management tools whilst allowing investment to be applied vertically for maximum results. The CIO can create the conditions where this is encouraged and create a level of transparency in datacenters and IT organizations that has hitherto been lacking.

This gradual transition to running IT as a business itself with the same level of fiduciary responsibility that the rest of the business is subjected to can and should be the driver for the recomposition of IT services and the new role that IT needs to play in ever tighter markets.

In the next series of blogs I will take some of these themes and see how that can be done from an architectural pattern and technology point of view for CIOs in the context of business driven projects.



Cloud Transformatics - Rethink Your Storage Strategy (SAN)

One of the great things about the Cloud is the ability to stimulate new/different ways of thinking about your datacenter and IT operations in general. This is particularly valid in the areas of service delivery and quality management for an ever more demanding consumer population!

This is what I call Cloud Transformatics - the vectors and drivers of Cloud induced change in traditional IT operations and service delivery. I will be writing a mini-series of blogs on this theme, starting with SAN storage.

One of the tenants that the Cloud has really hit on the head, is the notion of massive old-fashioned monolithic arrays. The simple idea that using smaller lower-cost building blocks can actually reduce overall cost while giving more reliability, performance and ultimately service quality swept through the storage industry through the last decade.

Most storage manufacturers moved to disguising their systems as modular and scalable storage systems and only under innovation pressure from competitors started putting in features that cut margins and overall sales volume - think deduplication, virtual LUNs, compression, unified storage, storage tiering, thin storage etc.

Indeed even the very storage protocols became a religious debate. SAN FC vs iSCSI, SAN vs NAS, CIFS vs NFS, converged cabling etc. Many examples of this spring to mind, the NetApps and 3PARs of this world. Many of these were simply acquired by those companies that were their competitors. 3PAR went to HP, Equallogic to Dell, Isilon/Data Domain to EMC and Pillar Axiom to Oracle.

Curiously, in the IT market, good-innovative products don't always survive. Isilon was struggling in the market against massive competition - and was acquired and survived and thrived. That is great news - without the acquisition the industry may have lost the important innovations they bring to market. The same goes for 3PAR and Data Domain. 

I have been watching Pillar Axiom for some time. It struck me when verifying its architecture, market penetration, sentiment analysis and "cool" factor that this was a very promising technical architecture directly tackling issues typical in customer datacenters and storage services delivery in particular!

Firstly, there is the idea of SLAMMERS - the protocol and CPU heart of the Axiom Storage system. These are modular, built on standard industry CPUs (AMD) and are easily ramped up for roaring performance. A simple CPU upgrade from dual to quad core delivered vastly more vroom and correspondingly better service management. Imagine what an AMD Bulldozer (16 cores) or a corresponding Intel chip would deliver!

Secondly, there is the idea of BRICKS - which are dual RAID controller disk enclosures providing up to 24TB of raw storage using 12 disks, with a separate hot spare - all in a 2U package. This is what I call modular. These enclosures have their own cache and offload disk controller operations away from the slammers. Local hot spares all ensure rapid rebuild operations localised in the brick. Oracle claim 128 RAID controllers operating in parallel in 64 separate bricks per axiom storage system.

This is a great level of parallelism and modularity. Again, industry standard components are used, and scalability through component upgrades (CPU, RAM, backplane, interconnects, disks) can yield surprisingly large gains in performance and capacity!

Last, but not least, there is the PILOT - the hardware appliance providing out-of-band management features. This provides in a simple intuitive interface the management cockpit of the axiom storage system. It has features you would expect such as managing disk resources and pools. It also provides features that are definitely in the enterprise class league:

  1. Application aware storage profiles
  2. Policy based resource control
  3. Spindle striping levels
  4. IO Priority Queues
  5. Disk stroking (using inner or outer spindles for performance)
  6. Network bandwidth control
  7. Multi-tenancy with Quality of Service profiles
  8. Thin storage, storage tiering, distributed RAID for linear scaling

Pillar was typically put in the mid-range league, probably due to not supporting mainframes. However, the feature set is clearly enterprise worthy. Recently Oracle has started to add support for Hybrid Columnar Compression which further adds value to this device to provide SAN enabled storage support for Oracle 11g environments over and above what is available in any other high-end storage array currently.

Looking under the covers, and extrapolating somewhat, we have a SAN storage array with enterprise features. It has a modular slammer structure that corresponds to the EMC VMAX with its engines model. The Axiom also offers 8 engines, but provides, through its bricks, a linear scalability. If Oracle inserts Infiniband interconnects (40Gbps links) inside the axiom, then this will be a screamer of a system!

I have often seen clients who exhaust the high-end arrays in I/O well before the claimed 2000+ disks are reached. So top end capacity is nice on paper, but in reality it is rarely reached in intense environments.

The Axiom with its odd brick and slammer architecture addresses one critical concern that I have noticed with the traditional gamut of high-end arrays - namely the posturing that this is the last word in storage and why one would possibly need anything else! Well, Cloud Transformatics clearly shows that true modular systems are winning out in the real world and provide dramatically better performance and value.

If one needs more performance than available in a single Axiom system, then put a second Axiom system in place! That is how Cloud storage works - ramp up IO and Capacity through adding another module. That we can do this in a SAN is definitely worth taking a look at!  

The Race to Business Value

The Axiom already had many of the features associated with high-end SAN storage, but available in a pay-as-you grow model through adding additional modular slammers and bricks! This is remarkable value! Further, the Axiom also provides, through its architecture, NAS services. 

With the current shortfall of global disk production due to the tragic flooding in Thailand, every storage manufacturer is projecting steep price increases or delayed delivery. This is the perfect time to re-evaluate the SAN storage strategy. Oracle is providing substantial intellectual property integration into the platform to further deliver real Cloud derived value (CVD).

This platform in its architecture shares similarities with scale-out NAS systems such as Isilon, providing infrastructure for big data. It does not take a big leap in imagination to see such functionality coming to this important datacenter building block.

Why is this important for the CIO?

One would not typically think of Oracle as an enterprise SAN storage provider - but with the Pillar Axiom acquisition, they are squarely set to disrupt this space. Customers are using these systems to great effect. Axiom represents an inflexion point in how one things of SAN storage. That it can be cost effectively implemented, high end features provided and direct links to enterprise software stacks of database/ERP systems (SAP/Oracle) make this a potent platform at dramatically lower prices!

CIOs setting their IT agendas in 2012 should take into account that there is a new disruptive player in the datacenter offering storage services. Oracle can not really be disregarded in this respect. Time will tell how clients react to the exciting potential of the Pillar Axiom platform now that it has some enterprise backing through Oracle. 

CIOs looking to secure deep cost savings and efficiency should note the potential of this SAN storage platform. Clearly, if there is a use case in their organisations for enterprise software stacks (databases, messaging systems, ERP, CRM etc) then Oracle is a major player. Oracle is already probably used in most enterprise shops, and further deep practical value (not just paper savings) can be accrued from using the Axiom SAN storage platform.      


