Introducing Client-side Evaluation for Amazon CloudWatch Evidently
AWS Architecture BlogIntroducing Client-side Evaluation for Amazon CloudWatch Evidentlyby Cole Thienes | on27 FEB 2023| inAmazon CloudWatch,Amazon Elastic Container Service,Architecture,AWS CloudFormation,Technical How-to|Permalink|ShareAmazon CloudWatch Evidentlyenables developers to test new features on a small percentage of traffic and gauge the outcome before rolling it out to the rest of their users. Evidently feature flags are defined ahead of your release and, at runtime, your application code queries a remote service to determine whether to show the new feature to a given user. The remote call to fetch feature flags for a user is susceptible tonetwork latency, adding several hundred milliseconds of delay in bad cases. Any additional latency added to fetching feature flags can directly impact the speed of a web page, where milliseconds matter. Our solution:client-side evaluation for Amazon CloudWatch Evidently. With client-side evaluation, developers can significantly decrease latency by fetching feature flags locally and avoiding network overhead altogether.The term “client-side” does not refer to the browser in this case, but the operation taking place on your container application rather than through a remote API call. This removes the need for a network call by fetching feature flags for a user from theAWS AppConfig agent—a sidecar container running alongside your container application backend. The agent enables container runtimes to leverageAWS AppConfig, a service allowing customers to change the way an application behaves while running without deploying new code. In this post, we’ll walk through the solution architecture and how to instrument client-side evaluation in anAmazon Elastic Container Service(Amazon ECS) application.Overview of solutionFigure 1 illustrates how client-side evaluation works in an application running on Amazon ECS. The webpage makes a call to the webpage backend to determine which website features to show an end-user. Let’s explore how this works.Figure 1. High-level architecture of client-side evaluation for Amazon CloudWatch EvidentlyCreate an Evidently project, feature, and launch through theAWS Console Mobile Application, API, orAWS CloudFormation.Create an Amazon ECS task for the backend application container and attach an AWS AppConfig agent container to the task. At runtime, the application container will invoke theEvaluateFeatureAPI to fetch feature flags. Without client-side evaluation, this API call would perform a remote call to the Evidently cloud service.With client-side evaluation, the API call is made from the application container to the AWS AppConfig agent container onlocalhost, short-circuiting the network overhead.Evidently maintains a synchronized copy of the Evidently features in anAWS AppConfig configurationwithin your AWS account. When subsequent changes are made to the features, the configuration is updated (usually within a minute).When the backend application initializes, the agent fetches the necessary configuration profile and caches it, polling periodically to refresh the cache. When the AWS AppConfig agent is invoked from the backend application, it evaluates the requested feature flag using the cached data.After each successfulEvaluateFeaturecall, a transaction record is generated, called anevaluation event. This useful bookkeeping mechanism helps developers view data to tell which of their users saw what feature and when. As the evaluation events are generated, they are placed in a buffer within the agent. Once the buffer reaches a certain size or age, events in the buffer are uploaded to Evidently via thePutProjectEventsAPI.The evaluation events are then available for offline analysis indeveloper-configured storage, including CloudWatch Logs,Amazon Simple Storage Service(Amazon S3), and CloudWatch Metrics.WalkthroughLet’s take a practical example to demonstrate client-side evaluation. I have a simple webpage with a search bar on it. I’ve implemented a newer, fancier search bar, but I only want to show it to 10 percent of my visitors to make sure it doesn’t cause issues on my existing webpage before rolling it out to everyone, as in Figure 2.Figure 2. A webpage experience where 10% of users see the new search barWe could set up the necessary AWS resources by hand, but let’s use a pre-builtAWS Cloud Development Kit(AWS CDK) example to save time. The sample code for this example is available on GitHub. Here are the high-level steps:Provision the infrastructure. The infrastructure will consist of:An ECS service with aVirtual Private Cloud(Amazon VPC) to serve as our backend application and return the search bar variationAn Evidently launch to split the traffic between the two search barsAn AWS AppConfig environment, which the AWS AppConfig agent will fetch Evidently data fromTest our webpage. Once our code is deployed, we will visit our webpage to fetch feature flags using client-side evaluation.Clean up by removing our infrastructure.PrerequisitesInstallGitInstallNode and npmCreate anAWS account.Install theAWS CDK Toolkit. Ensure the AWS CDK Toolkit is updated to the latest version.InstallDocker. AWS CDK will not work without Docker installed.StepsClone the repositoryFirst, clone the official AWS CDK example repository:git clone https://github.com/aws-samples/aws-cdk-examplesThe repository has many examples for setting up AWS infrastructure in CDK. Let’s go to a client-side evaluation example.Code explanationLet’s take a look at the code example. When we visit our web page, a request will be routed to our application deployed onAWS Fargate, which allows us to run containers directly using ECS without having to manageElastic Compute Cloud(Amazon EC2) instances. The application code is written inNode.js with Typescriptand leverages theExpress framework:<p><code>// local-image/app.ts</code></p><p><code>import * as express from 'express';</code><br /><code>import {Evidently} from '@aws-sdk/client-evidently';</code></p><p><code>const app = express();</code></p><p><code>const evidently = new Evidently({</code><br /><code>    endpoint: 'http://localhost:2772',</code><br /><code>    disableHostPrefix: true</code><br /><code>});</code></p><p><code>app.get("/", async (_, res) =&gt; {</code><br /><code>    try {</code><br /><code>        console.time('latency')</code><br /><code>        const evaluation = await evidently.evaluateFeature({</code><br /><code>            project: 'WebPage',</code><br /><code>            feature: 'SearchBar',</code><br /><code>            entityId: 'WebPageVisitor43'</code><br /><code>        })</code><br /><code>        console.timeEnd('latency')</code><br /><code>        res.send(evaluation.variation)</code><br /><code>    } catch (err: any) {</code><br /><code>        console.timeEnd('latency')</code><br /><code>        res.send(err.toString())</code><br /><code>    }</code><br /><code>});</code></p>The container application will invoke theEvaluateFeatureAPI using theAWS SDK for Javascriptand return thesearch bar variation, either the old or new search bar. Here we also log thelatencyof the operation. TheEvaluateFeaturerequest is forwarded to theendpointwe configure for the Evidently client: http://localhost:2772. This is the local address where the AWS AppConfig agent can be reached. To make this possible, we add the AWS AppConfig agent as a container to theAmazon ECS task definition:// index.tsservice.taskDefinition.addContainer('AppConfigAgent', {image: ecs.ContainerImage.fromRegistry('public.ecr.aws/aws-appconfig/aws-appconfig-agent:2.x'),portMappings: [{containerPort: 2772}]})We also need to set up anAppConfig environmentfor the Evidently project. This tells Evidently where to create the configuration to keep a synchronized copy of the features in the project:// index.tsconst application = new appconfig.CfnApplication(this,'AppConfigApplication', {name: 'MyApplication'});const environment = new appconfig.CfnEnvironment(this, 'AppConfigEnvironment', {applicationId: application.ref,name: 'MyEnvironment'});const project = new evidently.CfnProject(this, 'EvidentlyProject', {name: 'WebPage',appConfigResource: {applicationId: application.ref,environmentId: environment.ref}});Finally, we set up an Evidently feature and launch that ensures only 10 percent of traffic receives the new search bar:// index.tsconst launch = new evidently.CfnLaunch(this, 'EvidentlyLaunch', {project: project.name,name: 'MyLaunch',executionStatus: {status: 'START'},groups: [{feature: feature.name,variation: OLD_SEARCH_BAR,groupName: OLD_SEARCH_BAR},{feature: feature.name,variation: NEW_SEARCH_BAR,groupName: NEW_SEARCH_BAR}],scheduledSplitsConfig: [{startTime: '2022-01-01T00:00:00Z',groupWeights: [{groupName: OLD_SEARCH_BAR,splitWeight: 90000},{groupName: NEW_SEARCH_BAR,splitWeight: 10000}]}]})We start the launch immediately by settingexecutionStatustoSTARTandstartTimeto a timestamp in the past. If you want to wait to show the new search bar, we can specify a future start time.Install dependenciesInstall the necessaryNode modules:npm installBuild and deployBuild the AWS CDK template from the source code:npm run buildBefore deploying the app:Ensure that you set upAWS credentialsin your environment.The AWS CDK Toolkit isbootstrapped in your AWS account.Confirm the max number of VPCs has not been reached in your AWS account; the Amazon ECS service will deploy an Amazon VPC.After that, you can deploy the AWS CDK template to your AWS account:cdk deployTest the webpageAfter the previous step, you should see the following output in your console:Figure 3. Console output showing a successful AWS CDK deploymentIn your browser, visit the URL specified by theFargateServiceServiceURLoutput above and you will seeoldSearchBar. We can visit ourCloudWatch Logs from the Amazon ECS taskto see our application logs. Go to the AWS console and visit the CloudWatch log groups page and visit the log group with the prefixEvidentlyClientSideEvaluationEcs. There, we can see that fetching feature flags took under two milliseconds, as in Figure 4.Figure 4. CloudWatch Logs for the Amazon ECS task showing an EvaluateFeature latency of 1.238 millisecondsAdditionally, we can see how visitors have seen each version of the search bar. On the AWS console, visit the CloudWatch metrics page and see the Evidently metrics underAll>Evidently>Feature, Project, Variation, as in Figure 5:Figure 5. CloudWatch metrics showing the VariationCount (the number of times each feature flag variation was fetched)We can increase or decrease the percentage of visitors seeing the new search bar at any time. On the AWS console, go to the CloudWatch Evidently page and go toProjects>WebPage>Launches>MyLaunch>Modify launch trafficand adjust theTraffic percentage, as in Figure 6.Figure 6. Adjusting the traffic percentage of an Evidently launchCleaning upTo avoid incurring future charges, delete the resources. Let’s run:cdk destroyYou can confirm the removal by going into CloudFormation and confirming the resources were deleted.ConclusionIn this blog post, we learned how to set up a web page backend in Amazon ECS with client-side evaluation for Amazon CloudWatch Evidently. We easily deployed the example CloudFormation stack with AWS CDK Toolkit. Then we visited the example webpage and demonstrated the improved speed of fetching feature flags with client-side evaluation. If you’re interested in using client-side evaluation withAWS Lambdainstead of Amazon ECS, check out this AWS CDK example.Cole ThienesCole Thienes is a Software Engineer with Amazon CloudWatch. He focuses on developing solutions that help customers monitor their infrastructure and safely manage their applications.
Amazon CloudWatch Evidently enables developers to test new features on a small percentage of traffic and gauge the outcome before rolling it out to the rest of their users. Evidently feature flags are defined ahead of your release and, at runtime, your application code queries a remote service to determine whether to show the new […]
Cole Thienes
Amazon CloudWatch Amazon Elastic Container Service Architecture AWS CloudFormation Technical How-to