Convert and Watermark Documents Automatically with Amazon S3 Object Lambda
AWS Architecture BlogConvert and Watermark Documents Automatically with Amazon S3 Object Lambdaby Joseph Simon, Arnab Ghosh, and David Desroches | on23 AUG 2021| inAmazon API Gateway,Amazon Cognito,Amazon DynamoDB,Amazon Simple Storage Service (S3),Architecture,AWS Lambda|Permalink|ShareWhen you provide access to a sensitive document to someone outside of your organization, you likely need to ensure that the document is read-only. In this case, your document should be associated with a specific user in case it is shared.For example, authors often embed user-specific watermarks into their ebooks. This way, if their ebook gets posted to a file-sharing site, they can prevent the purchaser from downloading copies of the ebook in the future.In this blog post, we provide you a cost-efficient, scalable, and secure solution to efficiently generate user-specific versions of sensitive documents. This solution helps users track who their documents are shared with. This helps prevent fraud and ensure that private information isn’t leaked. Our solution uses a RESTful API, which usesAmazon S3 Object Lambdato convert documents to PDF and apply a watermark based on the requesting user. It also provides a method for authentication and tracks access to the original document.Architectural overviewS3 Object Lambda processes and transforms data that is requested fromAmazon Simple Storage Service (Amazon S3)before it’s sent back to a client. TheAWS Lambdafunction is invoked inline via a standard S3 GET request. It can return different results from the same document based on parameters, such as who is requesting the document. Figure 1 provides a high-level view of the different components that make up the solution.Figure 1. Document processing architectural diagramAuthenticating users with Amazon CognitoThis architecture defines a RESTful API, but users will likely be using a mobile or web application that calls the API. Thus, the application will first need to authenticate users. We do this viaAmazon Cognito, which functions as its own identity provider (IdP). You could also use anexternal IdP, including those that supportOpenID Connectand SAML.Validating the JSON Web Token with API GatewayOnce the user is successfully authenticated with Amazon Cognito, the application will be sent a JSON Web Token (JWT). This JWT contains information about the user and will be used in subsequent requests to the API.Now that the application has a token, it will make a request to the API, which is provided byAmazon API Gateway. API Gateway provides a secure, scalable entryway into your application. The API Gateway validates the JWT sent from the client with Amazon Cognito to make sure it is valid. If it is validated, the request is accepted and sent on to the Lambda API Handler. If it’s not, the client gets rejected and sent an error code.Storing user data with DynamoDBWhen the Lambda API Handler receives the request, it parses the JWT to extract the user making the request. It then logs that user, file, and access time intoAmazon DynamoDB. Optionally, you may use DynamoDB to store an encoded string that will be used as the watermark, rather than something in plaintext, like user name or email.Generating the PDF and user-specific watermarkAt this point, the Lambda API Handler sends an S3 GET request. However, instead of going to Amazon S3 directly, it goes to a different endpoint that invokes the S3 Object Lambda function. This endpoint is called anS3 Object Lambda Access Point. The S3 GET request contains the original file name and the string that will be used for the watermark.The S3 Object Lambda function transforms the original file that it downloads from its source S3 bucket. It uses the open-source office suiteLibreOffice(and specificallythis Lambda layer) to convert the source document to PDF. Once it is converted, a JavaScript library (PDF-Lib) embeds the watermark into the PDF before it’s sent back to the Lambda API Handler function.The Lambda API Handler stores the converted file in a temporary S3 bucket, generates apresigned URL, and sends that URL back to the client as a 302 redirect. Then the client sends a request to that presigned URL to get the converted file.To keep the temporary S3 bucket tidy, we use anS3 lifecycle configurationwith anexpiration policy.Figure 2. Process workflow for document transformationAlternate approachBefore S3 Object Lambda was available,Lambda@Edgewas used. However, there are three main issues with using Lambda@Edge instead of S3 Object Lambda:It is designed to run code closer to the end user to decrease latency, but in this case, latency is not a major concern.It requires using anAmazon CloudFrontdistribution, and the single-download pattern described here will not take advantage of Lamda@Edge’s caching.It hasquotason memory that don’t lend themselves to complex libraries like OfficeLibre.Extending this solutionThis blog post describes the basic building blocks for the solution, but it can be extended relatively easily. For example, you could add another function to the API that would convert, resize, and watermark images. To do this, create an S3 Object Lambda function to perform those tasks. Then, add an S3 Object Lambda Access Point to invoke it based on a different API call.API Gateway has many built-insecurity features, but you may want to enhance the security of your RESTful API. To do this, add enhanced security rules viaAWS WAF.Integrating your IdP into Amazon Cognitocan give you a single place to manage your users.Monitoring any solution is critical, and understanding how an application is behaving end to end can greatly benefit optimization and troubleshooting. AddingAWS X-RayandAmazon CloudWatch Lambda Insightswill show you how functions and their interactions are performing.Should you decide to extend this architecture, follow the architectural principles defined inAWS Well-Architected, and pay particular attention to theServerless Application Lens.Figure 3. Example expanded document processing architectureConclusionYou can implement this solution in a number of ways. However, by using S3 Object Lambda, you can transform documents without needing intermediary storage. S3 Object Lambda will also decouple your file logic from the rest of the application.TheServerless on AWScomponents mentioned in this post allow you to reduce administrative overhead, saving you time and money.Finally, the extensible nature of this architecture allows you to add functionality easily as your organization’s needs grow and change.The following links provide more information on how to use S3 Object Lambda in your architectures:S3 Object Lambda AnnouncementS3 Object Lambda User GuideJoseph SimonJoseph is an Enterprise Solutions Architect helping sports and media customers realize the potential of cloud computing on AWS. Outside of work, Joseph enjoys fitness and traveling.Arnab GhoshArnab Ghosh is a Solutions Architect for AWS in North America helping enterprise customers build resilient and cost-efficient architectures. He has over 14 years of experience in architecting, designing, and developing enterprise applications solving complex business problems.David DesrochesDave Desroches is a Solutions Architect supporting Independent Software Vendor accounts in the Northeast. He has over 25 years of experience in helping customers around the world achieve their technical goals.
When you provide access to a sensitive document to someone outside of your organization, you likely need to ensure that the document is read-only. In this case, your document should be associated with a specific user in case it is shared. For example, authors often embed user-specific watermarks into their ebooks. This way, if their […]
Joseph Simon
Amazon API Gateway Amazon Cognito Amazon DynamoDB Amazon Simple Storage Service (S3) Architecture AWS Lambda