Analyze Fraud Transactions using Amazon Fraud Detector and Amazon Athena
AWS Architecture BlogAnalyze Fraud Transactions using Amazon Fraud Detector and Amazon Athenaby Raghavarao Sodabathina | on26 JUL 2021| inAmazon Athena,Amazon Fraud Detector,Amazon QuickSight,Amazon Simple Storage Service (S3),Architecture,Intermediate (200)|Permalink|ShareOrganizations with online businesses have to be on guard constantly for fraudulent activity, such as fake accounts or payments made with stolen credit cards. One way they try to identify fraudsters is by using fraud detection applications. Some of these applications use machine learning (ML).A common challenge with ML is the need for a large, labeled dataset to create ML models to detect fraud. You will also need the skill set and infrastructure to build, train, deploy, and scale your ML model.In this post, I discuss how to perform fraud detection on a batch of many events usingAmazon Fraud Detector. Amazon Fraud Detector is a fully managed service that can identify potentially fraudulent online activities. These can be situations such as the creation of fake accounts or online payment fraud. Unlike general-purpose ML packages, Amazon Fraud Detector is designed specifically to detect fraud. You can analyze fraud transaction prediction results by usingAmazon AthenaandAmazon QuickSight. I will explain how to review fraud using Amazon Fraud Detector andAmazon SageMakerbuilt-in algorithms.Batch fraud prediction use casesYou can use abatch predictionsjob in Amazon Fraud Detector to get predictions for a set of events that do not require real-time scoring. You may want to generate fraud predictions for a batch of events. These might be payment fraud, account take over or compromise, and free tier misuse while performing an offline proof-of-concept. You can also use batch predictions to evaluate the risk of events on an hourly, daily, or weekly basis depending upon your business need.Batch fraud insights using Amazon Fraud DetectorOrganizations such as ecommerce companies and credit card companies use ML to detect the fraud. Some of the most common types of fraud include email account compromise (personal or business), new account fraud, and non-payment or non-delivery (which includes compromised card numbers).Amazon Fraud Detector automates the time-consuming and expensive steps to build, train, and deploy an ML model for fraud detection. Amazon Fraud Detector customizes each model it creates to your dataset, making the accuracy of models higher than current one-size-fits-all ML solutions. And because you pay only for what you use, you can avoid large upfront expenses.If you want to analyze fraud transactions after the fact, you can perform batch fraud predictions using Amazon Fraud Detector. Then you can store fraud prediction results in an Amazon S3 bucket. Amazon Athena helps you analyze the fraud prediction results. You can create fraud prediction visualization dashboards using Amazon QuickSight.The following diagram illustrates how to perform fraud predictions for a batch of events and analyze them using Amazon Athena.Figure 1. Example architecture for analyzing fraud transactions using Amazon Fraud Detector and Amazon AthenaThe architecture flow follows these general steps:Create and publish a detector.First create and publish a detector using Amazon Fraud Detector. It should contain your fraud prediction model and rules. For additional details, seeGet started (console).Create an input Amazon S3 bucket and upload your CSV file.Prepare a CSV file that contains the events you want to evaluate. Then upload your CSV file into the input S3 bucket. In this file, include a column for each variable in the event type associated with your detector. In addition, include columns for EVENT_ID, ENTITY_ID, EVENT_TIMESTAMP, ENTITY_TYPE. Refer toAmazon Fraud Detector batch input and output filesfor more details. ReadCreate a variablefor additional information on Amazon Fraud Detector variable data types and formatting.Create an output Amazon S3 bucket.Create an output Amazon S3 bucket to store your Amazon Fraud Detector prediction results.Perform a batch prediction.You can use a batch predictions job in Amazon Fraud Detector to get predictions for a set of events that do not require real-time scoring. Read more here aboutBatch predictions.Review your prediction results.Review your results in the CSV file that is generated and stored in the Amazon S3 output bucket.Analyze your fraud prediction results.After creating a Data Catalog by using AWS Glue, you can use Amazon Athena to analyze your fraud prediction results with standard SQL.You can develop user-friendly dashboards to analyze fraud prediction results using Amazon QuickSight by creating new datasets with Amazon Athena as your data source.Fraud detection using Amazon SageMakerTheAmazon Web Services (AWS)Solutions Implementation,Fraud Detection Using Machine Learning, enables you to run automated transaction processing. This can be on an example dataset or your own dataset. The included ML model detects potentially fraudulent activity and flags that activity for review. The diagram following presents the architecture you can automatically deploy using the solution’s implementation guide and accompanyingAWS CloudFormationtemplate.SageMaker provides several built-in machine learning algorithms that you can use for a variety of problem types. This solution leverages the built-in Random Cut Forest algorithm for unsupervised learning and the built-in XGBoost algorithm for supervised learning. In the SageMaker Developer Guide, you can see howRandom Cut ForestandXGBoostalgorithms work.Figure 2. Fraud detection using machine learning architecture on AWSThis architecture can be segmented into three phases.Develop a fraud prediction machine learning model.The AWS CloudFormation template deploys an example dataset of credit card transactions contained in anAmazon Simple Storage Service (Amazon S3)bucket. An Amazon SageMaker notebook instance with different ML models will be trained on the dataset.Perform fraud prediction.The solution also deploys anAWS Lambdafunction that processes transactions from the example dataset. It invokes the two SageMaker endpoints that assign anomaly scores and classification scores to incoming data points. AnAmazon API GatewayREST API initiates predictions using signed HTTP requests. AnAmazon Kinesis Data Firehosedelivery stream loads the processed transactions into another Amazon S3 bucket for storage. The solution also provides an example of how to invoke the prediction REST API as part of the Amazon SageMaker notebook.Analyze fraud transactions.Once the transactions have been loaded into Amazon S3, you can use analytics tools and services for visualization, reporting, ad-hoc queries, and more detailed analysis.By default, the solution is configured to process transactions from the example dataset. To use your own dataset, you must modify the solution. For more information, seeCustomization.ConclusionIn this post, we showed you how to analyze fraud transactions using Amazon Fraud Detector and Amazon Athena. You can build fraud insights using Amazon Fraud Detector and Amazon SageMaker built-in algorithms Random Cut Forest and XGBoost. With the information in this post, you can build your own fraud insights models on AWS. You’ll be able to detect fraud faster. Finally, you’ll be able to solve a variety of fraud types. These can be new account fraud, online transaction fraud, and fake reviews, among others.Read more and get started on building fraud detection models on AWS.Build an Amazon Fraud Detection ModelBatch Fraud predictions with Amazon Fraud DetectorOnline Fraud predictions with Amazon Fraud DetectorReviewing online fraud using Amazon Fraud Detector and Amazon A2IAmazon Athena user guide & Amazon QuickSight user guideRaghavarao SodabathinaRaghavarao Sodabathina is a Principal Solutions Architect at AWS, focusing on Data Analytics, AI/ML, and Serverless Platform. He engages with customers to create innovative solutions that address customer business problems and to accelerate the adoption of AWS services. In his spare time, Raghavarao enjoys spending time with his family, reading books, and watching movies.
Organizations with online businesses have to be on guard constantly for fraudulent activity, such as fake accounts or payments made with stolen credit cards. One way they try to identify fraudsters is by using fraud detection applications. Some of these applications use machine learning (ML). A common challenge with ML is the need for a […]
Raghavarao Sodabathina
Amazon Athena Amazon Fraud Detector Amazon QuickSight Amazon Simple Storage Service (S3) Architecture Intermediate (200)