<![CDATA[Cirrostratus]]>https://cirrostratus.cloudRSS for NodeSat, 21 May 2022 03:11:31 GMT60<![CDATA[RDS Aurora Serverless v2 is finally here, let's migrate!]]>After being in preview for what felt like an eternity, AWS has finally released RDS Aurora Serverless v2 on April 21, 2022.

RDS Aurora Serverless v2 is available for both Mysql 8 and Postgres 13 and will be a game changer for many organizations that struggle with "spiky" database loads. According to AWS, RDS Aurora Serverless v2 delivers "instant" scaling of the capacity of your database cluster, varying between 0.5 "Aurora Capacity Units" to 128 of those things.

An Aurora Capacity Unit, or ACU, provides 2 GiB of memory and "corresponding compute and networking". The minimum allocation is 0.5 ACU, or 1GB of memory, and your cluster could scale all the way up to 128ACUs, or 256GiB of memory.

The difference between Aurora v1 and v2 is small, but with huge impact. Where v1 scaled by doubling the ACU's, v2 will scale incrementally. The scaling policies currently available are very limited: there's a lower and upper limit of ACUs, and that's it. Also the "autopause" feature as well as the data API have not made it to Serverless v2 (yet).

Very Cool, let's migrate!

RDS Aurora Serverless v2 is only available on the latest versions of both MySQL and Postgres. That means your cluster must be on at least on Aurora MySQL 3.02.0 or Aurora PostgreSQL 13.6.

As a test, I've provisioned a normal RDS Aurora MySQL 8 cluster with a provisioned instance:


Note that there's no in-place upgrade from MySQL 5.7 to 8, so if you're still on 5.7, you'll need to launch a new cluster from a snapshot.

First, let's upgrade to the latest version, by modifying the version of our cluster:



Oops! First hurdle. It appears Performance Insights can not be enabled during upgrade. The error message is wrong; Performance Insights is actually supported. If you've got Performance Insights enabled for any of your instances, disable it first to continue.


After fixing the above, the cluster started upgrading. This takes about 10 minutes for an empty cluster, so your mileage will vary.


Once your cluster is available again, we need to add a new reader to the cluster:


On the Add Reader screen, select "Serverless v2" as your DB instance class, and specify a capacity range:


Now we have a serverless reader added to our cluster. If your application is read-heavy and supports splitting reads and writes, this setup can be awesome as well. Use a small provisioned writer, and have Serverless v2 automatically manage reader capacity.


For this article however, we want to end up with a completely serverless cluster. So let's continue by migrating the writer role to the serverless "Instance". To do that, select the serverless instance, go to actions, and press "Failover"


Then, press Failover again.


Note that after the failover was successful, our serverless instance now has the Writer role:


To complete the migration, just delete the old provisioned instance that now has the reader role:


And that's it, we're completely serverless!


After the migration, Performance Insights can be enabled for the serverless instance again. Also, it's wise to test your application thoroughly, both because of the new MySQL/PostgreSQL version, and to verify if the scaling of the serverless instance does not affect your application in any way. The upgrade/migrate process will cause downtime, so when doing this in production, schedule the migration in a maintenance window.

I'm curious to see how much cost and performance benefits serverless v2 will have in the real world. You now know how to get there!

Thanks for reading this article!

https://cirrostratus.cloud/rds-aurora-serverless-v2-is-finally-here-lets-migratehttps://cirrostratus.cloud/rds-aurora-serverless-v2-is-finally-here-lets-migrateSat, 23 Apr 2022 11:31:06 GMThttps://cdn.hashnode.com/res/hashnode/image/upload/v1650713587330/3IIxEZ3og.png
<![CDATA[Deploying (CDK) to AWS using Bitbucket Pipelines and OIDC]]>image.png

The other day I was browsing Reddit and a user asked for help regarding deploying CDK with Bitbucket Pipelines, using OIDC. I assumed that this was straight forward and helped the user by pointing him to some articles. A few days later, the same use case appeared on one of our backlogs, so I took the opportunity to test if it actually was straight forward. As things go; it was, after a few hours of debugging.

In this article I'll try to explain what I've done to get this working, but also try to help you understand what we're actually doing.

I'm in no way an authority on CDK, Bitbucket Pipelines or OIDC, but I know enough to stick them all together to work in a nice way!

The What and Why

If you're reading this, chances are that you're already using both Bitbucket pipelines and CDK, perhaps you're even already deploying CDK using Bitbucket pipelines. Then you also know that traditionally we've always used IAM access keys to authenticate to AWS from Bitbucket pipelines. There are several ways to secure those access keys when stored in Bitbucket, but that just solves one part of the problem. Access keys have some inherent flaws, as they can be used by anyone, from anywhere. Leaking access keys is the number one way to compromise an AWS account. If you can help it; don't use access keys.

A better way, which more and more platforms support, is telling AWS that you trust the OIDC provider that comes with your Bitbucket workspace, and allow it to assume IAM roles on your AWS account.

There actually is a pretty good article on atlassian.com that outlines most of the process, but it skips some important steps to get CDK to work.

The Warning

Even though I'm convinced using the solution below is far superior to using IAM access keys, there are some things to take into account. First; anyone with enough permissions in your Bitbucket workspace can use the roles that Bitbucket can. This is also true for access keys, but it's good to be aware of. Secondly, this article does not go in to limiting which repo and branch can actually use an IAM role. Since the whole workspace is trusted, it's good practice to limit to a repo and branch. How to do this is shared in this article on atlassian.com.


Since we're going to deploy CDK, we're assuming you have already bootstrapped the AWS account and region where you want to deploy your app. We're also going to assume that you can access the AWS account, as well as create IAM Identity Providers and roles. Administrative privileges in your Bitbucket workspace would also be nice.

Getting started with OIDC

First, we will need to configure AWS to trust the OIDC provider that comes with your Bitbucket workspace. To do that, we require the Identity Provider URL and the Audience. Both can be accessed by browsing to Repository settings -> OpenID Connect or at https://bitbucket.org/YOURWORKSPACE/YOURREPO/admin/addon/admin/pipelines/openid-connect.

The Identity provider URL will look something like this:


And the Audience will look something like:


IAM Identity Provider and IAM Role

Next, log in to your AWS account and browse to the identity providers screen in the IAM service.

Once there, click Add Provider


Then, select OpenID Connect and enter the Provider URL and Audience from the previous step. Click Get thumbprint and then Add Provider.


If successful, you'll see that you now have an additional identity provider registered. Next, browse to Roles and select Create Role.

Then select Web Identity and select the Identity provider and audience from the drop down, and click next.


In the next screen, select one ore more Permissions policies. It's good practice to pick and choose what your pipeline will need, in stead of going for AdministratorAccess right off the bat.

After choosing the applicable policies, give your role a name and select Create role to finish the process. Once the role is created, find and open the role and note the ARN, because we'll need it in the next step.

Bitbucket Pipelines

Using Bitbucket Pipelines is fairly simple. You create a file named bitbucket-pipelines.yml in the root of your repo and specify which actions to take when. Most actions are the same as on your local machine; just simple commands.

The syntax that Bitbucket Pipelines uses is well documented and can be quite complex. Today we'll do some simple stuff, so it's more like a getting started.

So; create a file named bitbucket-pipelines.yml in the root of your repository and copy over my example from Github. I'll go through what does what next.


The pipeline runs in a docker image and so we must choose one that fits our needs with little to no install or updates of packages. I chose the node:16.3.0 image since it has everything I need to deploy CDK apps.

image: node:16.3.0


In the next few lines, we start the actual definition of the pipeline, and here I've chosen to run this pipeline for the main branch only.



Once we've defined that we want to run a pipeline for our main branch, next we need to define which steps the pipeline has. For CDK I've chosen a "Synth" and a "Deploy" step.

First, we define our Synth step and tell Bitbucket to use OIDC:

     - step:
          name: CDK Synth
          oidc: true

Then, we configure a cache for node packages, this will speed up the steps in the future:

            - node

Next, we define the script that must run. Change the AWS_REGION and AWS_ROLE_ARN to your Region and to the ARN from the role that we made earlier. The other commands in this script are static.

Warning; the AWS_REGION export is required! Even though CDK might say that it will use it's own ENV files to define a region, the variable is used to assume the IAM role. Removing it will break the process.

            - export AWS_REGION=eu-west-1
            - export AWS_ROLE_ARN=arn:aws:iam::012345678901:role/IAMRoleForBitBucket
            - export AWS_WEB_IDENTITY_TOKEN_FILE=$(pwd)/web-identity-token
            - echo $BITBUCKET_STEP_OIDC_TOKEN > $(pwd)/web-identity-token
            - npm install
            - npx cdk synth

As a last step, we define which artifacts we want to share with the next steps, in our case it's the output of CDK synth:

            - cdk.out/**

And that's our first step! The next step, CDK Deploy, is identical to the Synth step, but we use the npx cdk deploy command to start the actual deployment.

We leave out the artifacts part since this is the last step and we don't any files to transition. In stead, depending on your deployment, we can add docker support. In my case I was deploying a task to ECS, and CDK needs docker to be able to build the image:

            - docker

Testing, 1-2-3.

And that's it! Commit bitbucket-pipelines.yml to the branch that you've defined and watch your pipeline deploy your app! (or fail miserably)


Closing thoughts

As mentioned earlier, this example is meant to get you a working example and by no means covers all use cases. Also, for the title, I've chosen to put CDK in parentheses since this solution should work for Terraform, Pulumi, CloudFormation or any other IaC tool. Note that some might require you to use the AWS CLI to script the "assume role with web identity" part, since some tools don't support it yet.

Thank you for reading this article!

https://cirrostratus.cloud/deploying-cdk-to-aws-using-bitbucket-pipelines-and-oidchttps://cirrostratus.cloud/deploying-cdk-to-aws-using-bitbucket-pipelines-and-oidcSun, 13 Mar 2022 09:13:25 GMT
<![CDATA[Mindbloggeling AWS!]]>It's been a while since I wrote about my work, but since I've been working on a small article that's scheduled to be published on our company website, I thought it a good idea to re:Start myself as well!

Nothing more to say today, first things first!

https://cirrostratus.cloud/mindbloggeling-awshttps://cirrostratus.cloud/mindbloggeling-awsThu, 18 Nov 2021 22:01:50 GMT