Archive for October, 2014

DevOps and Databases?

October 16th, 2014

I’m super looking forward to next weeks DevOps Enterprise Summit in San Francisco. You might ask “Why?!” since I’m a database guy and not a DevOps guy.  I can understand that reaction.

I don’t know about you, but when I hear the term DevOps I have to roll my eyes and think “oh, the latest greatest tech industry buzzword.” So why would I as a DBA care about DevOps? With DevOps, as with most tech industry buzzwords, there is actually a worthy idea at the kernel. The kernel of DevOps has two parts.

  1. communication
  2. automation

The first part is changing company culture seeking to improve communication, understanding, empathy and create bridges between teams in different silos in development and operations to improve efficiencies. The second part of DevOps is using the best methods and tools to enable automation. With better communication between groups including fast feedback loops to track the impact of changes quickly and automated tools to rollout changes quickly DevOps can profoundly improve the efficiency of companies.

How can a DBA help bring DevOps culture to a company 

It’s true though, that the best tools will have no effect if there is not a culture in place that can adopt the new tools. On the other hand the best culture can only go so far with out the best tools. Introducing tools and methods is often the best way to facilitate making changes in culture. What is the best tool that a DBA can bring to his organization that facilitates not only automation but to improve communication and culture. That tool is data virtualization.

Why is data virtualization key to DevOps and databases?

First what is the goal of DevOps? It’s to improve efficiencies by bringing development closer to operations, i.e. IT. Bringing Dev and Ops closer together means better communication, better understanding and more efficient interactions. What kind of interactions do Dev and Ops have? Well developers are creating new applications and features that have to be deployed into production. In order to create and deploy applications, developers require copies of the production environment. The hardest part of creating a copy of the production environment is creating a copy of the production database. The database is the largest amount of data in the environments and also the most complicated part to copy as it requires special tools and procedures to copy. The copies often require masking which leads to more complexity. All of this means that making copies of databases becomes the bottleneck in development and production deployment. To deploy applications into production requires thorough QA testing of code and not only testing of code but testing of the actual rollout process. All of this QA and testing requires more environments and again.

As Gene Kim author on The Phoenix Project said, the number one bottleneck in application development is supplying development and QA environments.

What if with one simple stroke one could provide all the environments they wanted at the push of a button for almost no storage and the interface was so simple with secure logons that developers and QA could provision, rollback, refresh, bookmark, rewind full environments themselves. Thats what Delphix does. Simply bringing in Delphix makes a paradigm shift easy. Yes, Dev and Ops still have to work together to adopt the processes that Delphix provides but when the processes are so powerful and  so easy then shifting company culture becomes possible. Shifting Dev and Ops to DevOps even becomes possible in the largest of enterprise companies which are the most daunting cultures to bring into the DevOps movement.

After bring in Delphix to a 100 of the Fortune 500 we’ve seen case after case of application development teams doubling output, reducing bugs and creating higher quality code.

If you are at DevOps Enterprise Summit next week come visit the Delphix booth and stop by and say “hi” and find out how Delphix can eliminate the biggest constraints in application development and put a jet pack on your projects.

Kicking off the summit will be Gene Kim (IT Revolution) + Steve Brodie (Electric Cloud)!


Screen Shot 2015-11-17 at 2.02.21 PM


Delphix and Oracle 12c partnership

October 10th, 2014

More and more people have been asking me how Delphix compares with Oracle EM 12c Cloud Control with Snap Clone.

Delphix compliments Oracle. Oracle and Delphix are partners.  Delphix is an Oracle Gold ISV partner and we collaborated at Oracle Open World on hosting hands on Delphix labs.

Delphix adds value to Oracle by enabling Oracle customers to more efficiently and easily make copies of there databases. Databases are typically copied from a production system to development, QA, backups, reporting, sandboxes, UAT, forensics etc. Across these copies 90-99% of the data blocks are duplicate. Instead of making duplicate copies, Delphix enables the duplicate blocks to be shared across all copies.  Blocks are shared across the copies which are from the same point in time but more importantly blocks are shared even by copies coming for any point in time thanks to a technology called time flow that collects and tracks datablock changes from a source database. No other technology  in the industry has time flow technology. Time flow, data block version management, source database syncing and provisioning of thin clones, aka virtual databases is handled seamlessly and automatically by Delphix and interacting with Delphix is as simple as clicking some buttons in a beautiful UI.

One of the most complementary value adds is enabling the cloning of PDBs in the same container or remote container in minutes for almost no storage no matter the size. The size of PDBs can be a impediment to making copies of PDBs but with Delphix all duplicate blocks are shared on disk and in storage cache thanks to the Delphix technology. The shared block storage and the shared blocks in page caching complements the SGA and processes sharing that PDB technology enables reducing memory usage and allow more databases per given set of hardware.

Oracle EM 12c Snap Clone is a nice feature that allows Oracle customers to leverage existing Netapp or ZFS storage hardware more efficiently but Delphix is software that installs on commodity hardware, allows customers to sync with source data be it Oracle, other databases, application stacks, software distributions, track changes in those sources and provision our space efficient copies in minutes for almost no storage usage to target machines through out customers IT centers or even into the cloud including Amazon AWS.

The bottom line is that most of the functionality that Delphix brings is not available with any other technology in the industry such as

  • Data Protection – 30 days of backups and recovery down to the second stored in the same storage size as the original database and accessible in minutes. Fine grain RPO and fast RTO. (see graphic at bottom of post)
  • Live Archive – archiving and providing point in time versions of a database in minutes for a fraction of the storage otherwise required. Important for supporting audits like SOX, CCAR, Dodd Frank etc
  • Business Intelligence – 24×7 ETL windows & fast refresh of new data in minutes.
  • Synchronized Cloning – cloning multiple related databases at the same exact point in time.
  • Branching – clones can be cloned themselves in minutes, thus a development team can clone their database and give a copy in minutes to QA
  • EBS cloning – push button cloning of the whole EBS stack, binaries, application stack and database
  • Clone data protection – clones by default have data protection and if a developer messes up his copy, a new copy can be made in minutes just before the bad operation happened
  • Cross platform thin cloning – thin cloning of Unix sources to Linux VDBs
  • Data Center Migration – Delphix supports active replication between Delphix appliances making cloud migration simple, efficient and continually synced between in house and cloud.
  • Super caching – sharing of duplicate data blocks not only on disk but also in cache on Delphix which looks and acts just like a NAS, aka an NFS file server, from the point of view of the the virtual databases

Not to mention that Delphix supports full automated source syncing and virtual database provisioning other databases beside Oracle such as SQL Server, Sybase, Postgres. Other databases such as DB2 can be virtualized as well with some additional manual steps.

Delphix is software that installs anywhere where as Oracle EM 12c Snap Clone is a  feature that is a simple and nice enhancement for the usage of specialized existing hardware, either Netapp or ZFS storage to make static thin clones at one point in time.

Features such as in-memory option are supported by Delphix.  In-memory is transparent to Delphix whether the virtual database uses in-memory. In-memory is a memory feature and Delphix is only at the storage level.

On the other hand pluggable database support changes a lot of things.  Cloning a single PDB in a CDB involves some complexities and Delphix fully supports PDB cloning. Here is a video I made demonstrating this

Those are all black and white features that Delphix supports but just as a demonstration of this power the following video shows linking to a RAC database and provisioning a RAC clone in minutes:


Another example, mentioned in point one above is the power of Delphix as  backup & data protection. The graph below shows the classic Oracle backup routine of a full backup on the weekends and increment backups daily. The chart below represents at 9TB database with 1TB change rate per day (a huge change rate). At such a change rate, an Oracle backup would be 9TB on the weekend and a daily backup of 1TB of changes. Recovery of such a database on a Friday for example would be  entail copying the full 9TB database to the recovery location, then replaying 5 incremental backups each of 1TB on top of the full backup (starting with a full Sunday backup and applying incremental from Mon,Tues,Wed, Thurs, Fri). Such a restore would be a massive operation.

With Delphix, the first link to the source would only be 3TB thanks to compression. Each incremental backup would only be 1/3 TB. Incremental backups are immediately applied and available. Recovering from an incremental on any day would only take minutes with no data movement. A full backup is NEVER taken again. Only incrementals are taken. Thanks to this time flow technology and compression, one can store almost 4 weeks of backups in the size of the original database! and one can access those database version in minutes at the push of a button with no data movement. Recovering is just a matter of pushing a button and Delphix handles mounting the data at that point in time over NFS to a target machine and Delphix starts up and finishes the recovery of that database point in time.

Screen Shot 2014-10-09 at 4.31.12 PM

Girls with Heads Together Hugging