Managing data confidentiality for Scope 3 emissions using AWS Clean Rooms
AWS Architecture BlogManaging data confidentiality for Scope 3 emissions using AWS Clean Roomsby Sundeep Ramachandran and Marco Masciola | on31 MAY 2023| inAmazon Simple Storage Service (S3),Architecture,AWS Clean Rooms,AWS Glue,Sustainability,Technical How-to|Permalink|ShareScope 3 emissions are indirect greenhouse gas emissions that are a result of a company’s activities, but occuroutsidethe company’s direct control or ownership. Measuring these emissions requires collecting data from a wide range of external sources, like raw material suppliers, transportation providers, and other third parties. One of the main challenges with Scope 3 data collection is ensuring data confidentiality when sharing proprietary information between third-party suppliers. Organizations are hesitant to share information that could potentially be used by competitors. This can make it difficult for companies to accurately measure and report on their Scope 3 emissions. And the result is that it limits their ability to manage climate-related impacts and risks.In this blog, we show how to useAWS Clean Roomsto share Scope 3 emissions data between a reporting company and two of their value chain partners (a raw material purchased goods supplier and a transportation provider). Data confidentially requirements are specified by each organization before participating in the data AWS Clean Rooms collaboration (see Figure 1).Figure 1. Data confidentiality requirements of reporting company and value chain partnersEach account has confidential data described as follows:Column 1 lists theraw materialRegion of origin. This is business confidential information for supplier.Column 2 lists theemission factorsat the raw material level. This is sensitive information for the supplier.Column 3 lists themode of transportation. This is business confidential information for the transportation provider.Column 4 lists theemissions in transporting individual items. This is sensitive information for the transportation provider.Rows in column 5 list theproduct recipeat the ingredient level. This is trade secret information for the reporting company.Overview of solutionIn this architecture, AWS Clean Rooms is used to analyze and collaborate on emission datasets without sharing, moving, or revealing underlying data to collaborators (shown in Figure 2).Figure 2. Architecture for AWS Clean Rooms Scope 3 collaborationThree AWS accounts are used to demonstrate this approach. TheReporting Accountcreates a collaboration in AWS Clean Rooms and invites thePurchased Goods AccountandTransportation Accountto join as members. All accounts can protect their underlying data with privacy-enhancing controls to contribute data directly fromAmazon Simple Storage Service (S3)usingAWS Gluetables.ThePurchased Goods Accountincludes users who can update the purchased goods bucket. Similarly, theTransportation Accounthas users who can update the transportation bucket. TheReporting Accountcan run SQL queries on the configured tables. AWS Clean Rooms only returns results complying with the analysis rules set by all participating accounts.PrerequisitesFor this walkthrough, you should have the following prerequisites:ThreeAWS accountsin the same AWS RegionAnAmazon S3bucket in each account with emissions data (see Figure 1)AnAWS Glue Data Catalogfor the emissions data stored in each S3 bucketAlthough Amazon S3 and AWS Clean Rooms arefree-tiereligible, a low fee applies toAWS Glue. Clean-up actions are provided later in this blog post to minimize costs.ConfigurationWe configured the S3 buckets for each AWS account as follows:Reporting Account:reportingcompany.csvPurchased Goods Account:purchasedgood.csvTransportation Account:transportation.csvCreate an AWS Glue Data Catalog for each S3 data source following the method in theGlue Data Catalog Developer Guide. The AWS Glue tables should match the schema detailed previously in Figure 1, for each respective account (see Figure 3).Figure 3. Configured AWS Glue table for ‘Purchased Goods’Data consumers can be configured to ingest, analyze, and visualize queries (refer back to Figure 2). We will tag theReporting AccountGlue Database as “reporting-db” and the Glue Table as “reporting.” Likewise, thePurchased Goods Accountwill have “purchase-db” and “purchase” tags.SecurityAdditional actions are recommended to secure each account in a production environment. To configure encryption, review theFurther Readingsection at the end of this post,AWS Identity and Access Management (IAM)roles, andAmazon CloudWatch.WalkthroughThis walkthrough consists of four steps:TheReporting Accountcreates the AWS Clean Rooms collaboration and invites thePurchased Goods AccountandTransportation Accountto share data.ThePurchased Goods AccountandTransportation Accountaccepts this invitation.Rules are applied for each collaboration account restricting how data is shared between AWS Clean Rooms collaboration accounts.The SQL query is created and run in theReporting Account.1. Create the AWS Clean Rooms collaboration in the Reporting Account(The steps covered in this section require you to be logged into theReporting Account.)Navigate to the AWS Clean Rooms console and clickCreate collaboration.In theDetailssection, type “Scope 3 Clean Room Collaboration” in theNamefield.Scroll to theMember 1section. Enter “Reporting Account” in theMember display namefield.InMember 2section, enter “Purchased Goods Account” for your first collaboration member name, with their account number in theMember AWS account IDbox.ClickAdd another memberand add “Transportation Account” as the third collaborator with their AWS account number.Choose the “Reporting Account” as the Member who can query and receive result in theMember abilitiessection. ClickNext.SelectYes, join by creating membership now. ClickNext.Verify the collaboration settings on theReview and Createpage, then selectCreate and join collaboration and create membership.Both accounts will then receive an invitation to accept the collaboration (see Figure 4). The console reveals each member status as “Invited” until accepted. Next, we will show how the invited members apply query restrictions on their data.Figure 4. New collaboration created in AWS Clean Rooms2. Accept invitations and configure table collaboration rulesSteps in this section are applied to thePurchased Goods AccountandTransportation Accountfollowing collaboration environment setup. For brevity, we will demonstrate steps using thePurchased Goods Account. Differences for theTransportation Accountare noted.Log in to the AWS account owning thePurchased Goods Accountand accept the collaboration invitation.Open the AWS Clean Rooms console and selectCollaborationson the left-hand navigation pane, then clickAvailable to join.You will see an invitation from theScope 3 Clean Room Collaboration. Click onScope 3 Clean Room Collaborationand thenCreate membership.SelectTables, thenAssociate table. ClickConfigure new table.The next action is to associate the Glue table created from thepurchasedgoods.csvfile. This sequence restricts access to theorigin_regioncolumn (transportation_modefor theTransportation Accounttable) in the collaboration.In theScope 3 Clean Room Collaboration, selectConfigured tablesin the left-hand pane, thenConfigure new table. Select the AWS Glue table associated withpurchasedgoods.csv(shown in Figure 5).Select theAWS Glue Database(purchase-db) andAWS Glue Table(purchase).Verify the correct table section by togglingView schemafrom the AWS Glue slider bar.In theColumns allowed in collaborationsection, select all fields except fororigin_region. This action prevents theorigin_regioncolumn being accessed and viewed in the collaboration.Complete this step by selectingConfigure new table.Figure 5. Purchased Goods account table configurationSelectConfigure analysis rule(see Figure 6).SelectAggregationtype thenNext.SelectSUMas theAggregate functionands3_upstream_purchased_goodfor the column.Under Join controls, select Specify Join column. Select “item” from the list of options. This permits SQL join queries to execute on the “item” column. ClickNext.Figure 6. Table rules for the Purchased Goods accountThe next page specifies the minimum number of unique rows to aggregate for the “join” command. Select “item” forColumn nameand “2” for theMinimum number of distinct values. ClickNext.To confirm the table configuration query rules, clickConfigure analysis rule.The final step is to clickAssociate to collaborationand selectScope 3 Clean Room Collaborationin the pulldown menu. SelectAssociate tableafter page refresh.The procedure in this section is repeated for theTransportation Account, with the following exceptions:The columns shared in this collaboration areitem,s3_upstream_transportation, andunit.TheAggregation functionis aSUMapplied on thes3_upstream_transportationcolumn.Theitemcolumn has anAggregation constraintminimum of two distinct values.3. Configure table collaboration rules inside the Reporting AccountAt this stage, member account tables are created and shared in the collaboration. The next step is to configure the Reporting Account tables in theReporting Account’s AWS account.Navigate to AWS Clean Rooms. SelectConfigured tables, thenConfigure new table.Select the Glue database and table associated with the filereportingcompany.csv.UnderColumns allowed in collaboration, selectAll columns, thenConfigure new table.Configure collaboration rules by clickingConfigure analysis ruleusing theGuided workflow.SelectAggregationtype, thenNext.SelectSUMas theAggregate functionandingredientfor the column (see Figure 7).Only SQL join queries can be executed on theingredientcolumn by selecting it in theSpecify join columnssection.In the Dimension controls, selectproduct. This option permits grouping by product name in the SQL query. SelectNext.SelectNonein theScalar functionssection. ClickNext. Read more about scalar functions in theAWS Clean Rooms User Guide.Figure 7. Table rules for the Reporting accountOn the next page, selectingredientforColumn nameand2for theMinimum number of distinct values. ClickNext. To confirm query control submission, selectConfigure analysis ruleon the next page.Validate the setting in theReview and Configurewindow, then selectNext.Inside theConfigured tablestab, selectAssociate to collaboration. Assign the table to theScope 3 Clean Rooms Collaboration.Select theScope 3 Clean Room Collaborationin the dropdown menu. SelectChoose collaboration.On theScope 3 Clean Room Collaborationpage, selectreporting, thenAssociate table.4. Create and run the SQL queryQueries can now be run inside theReporting Account(shown in Figure 8).Figure 8. Query results in the Clean Rooms Reporting AccountSelect an S3 destination to output the query results. SelectAction, thenSet results settings.Enter the S3 bucket name, then clickSave changes.Paste this SQL snippet inside the query text editor (see Figure 8):SELECTr.productAS“Product”,SUM(p.s3_upstream_purchased_good)AS“Scope_3_Purchased_Goods_Emissions”,SUM(t.s3_upstream_transportation)AS“Scope_3_Transportation_Emissions”FROMreporting rINNER JOINpurchase pON r.ingredient = p.itemINNER JOINtransportation tON p.item = t.itemGROUP BYr.productClickRun query. The query results should appear after a few minutes on the initial query, but will take less time for subsequent queries.ConclusionThis example shows how Clean Rooms can aggregate data across collaborators to produce total Scope 3 emissions for each product from purchased goods and transportation. This query was performed between three organizations without revealing underlying emission factors or proprietary product recipe to one another. This alleviates data confidentially concerns and improves sustainability reporting transparency.Clean UpThe following steps are taken to clean up all resources created in this walkthrough:Member and Collaboration Accounts:AWS Clean Rooms: Disassociate and delete collaboration tablesAWS Clean Rooms: Remove member account in the collaborationAWS Glue: Delete the crawler, database, and tablesAWS IAM: Delete the AWS Clean Rooms service policyAmazon S3: Delete the CSV file storage buckets·Collaboration Account only:Amazon S3: delete the SQL query bucketAWS Clean Rooms: delete theScope 3 Clean Room CollaborationFurther Reading:Greenhouse Gas Protocol Scope 3 StandardAWS Clean RoomsAWS Clean Rooms User GuideGetting started with the AWS Glue Data CatalogAnalyzing Data in S3 using Amazon AthenaSecurity PracticesAWS Clean Rooms securitypracticesCreateIAM Rolesto enforce S3 and Clean Rooms access policies throughleast privilege permissionsApplyencryption ruleson S3 buckets with key rotationUseCloudTrail to record AWS Clean Rooms API callsEncrypt Glue tablesusing KMS keysSundeep RamachandranSundeep Ramachandran is a Senior Partner Solutions Architect and technical lead for the Global Sustainability Partner Segment at AWS. He works as an industry specialist and trusted advisor for AWS Partners tackling challenges like emissions management, supply chain traceability, decarbonization and net zero transition, material circularity, climate resilience, optimizing food-energy-water systems, and tracking ESG factors across multiple industries.Marco MasciolaMarco Masciola is a Senior Partner Solutions Architect for the Global Sustainability Partner Segment at Amazon Web Services. Marco helps AWS partners build and deliver their sustainability solutions through the AWS cloud.
Scope 3 emissions are indirect greenhouse gas emissions that are a result of a company’s activities, but occur outside the company’s direct control or ownership. Measuring these emissions requires collecting data from a wide range of external sources, like raw material suppliers, transportation providers, and other third parties. One of the main challenges with Scope […]
Sundeep Ramachandran
Amazon Simple Storage Service (S3) Architecture AWS Clean Rooms AWS Glue Sustainability Technical How-to