Securely Ingest Industrial Data to AWS via Machine to Cloud Solution
AWS Architecture BlogSecurely Ingest Industrial Data to AWS via Machine to Cloud Solutionby Ajay Swamy and Erin McGill | on08 SEP 2021| inAmazon CloudFront,Amazon DynamoDB,Amazon Simple Storage Service (S3),Architecture,AWS CloudFormation,AWS IoT Core,AWS IoT Greengrass,AWS IoT SiteWise,AWS Lambda,AWS Solutions Implementations,Manufacturing|Permalink|ShareAs a manufacturing enterprise, maximizing your operational efficiency and optimizing output are critical factors in this competitive global market. However, many manufacturers are unable to frequently collect data, link data together, and generate insights to help them optimize performance. Furthermore, decades of competing standards for connectivity have resulted in the lack of universal protocols to connect underlying equipment and assets.Machine to Cloud Connectivity Framework (M2C2)is anAmazon Web Services (AWS)Solution that provides the secure ingestion of equipment telemetry data to the AWS Cloud. This allows you to use AWS services to conduct analysis on your equipment data, instead of managing underlying infrastructure operations. The solution allows for robust data ingestion from industrial equipment that use OPC Data Access (OPC DA) and OPC Unified Access (OPC UA) protocols.Secure, automated configuration and ingestion of industrial dataM2C2 allows manufacturers to ingest their shop floor data into various data destinations in AWS. These includeAWS IoT SiteWise,AWS IoT Core,Amazon Kinesis Data Streams, andAmazon Simple Storage Service (S3). The solution is integrated with AWS IoT SiteWise so you can store, organize, and monitor data from your factory equipment at scale. Additionally, the solution provides customers an intuitive user interface to create, configure, monitor, and manage connections.Automated setup and configurationFigure 1. Automatically create and configure connectionsWith M2C2, you can connect to your operational technology assets (see Figure 1). The solution automatically creates AWS IoT certificates, keys, and configuration files forAWS IoT Greengrass. This allows you to set up Greengrass to run on your industrial gateway. It also automates the deployment of any Greengrass group configuration changes required by the solution. You can define a connection with the interface, and specify attributes about equipment, tags, protocols, and read frequency for equipment data.Figure 2. Send data to different destinations in the AWS CloudOnce the connection details have been specified, you can send data to different destinations in AWS Cloud (see Figure 2). M2C2 provides capability to ingest data from industrial equipment using OPC-DA and OPC-UA protocols. The solutioncollectsthe data, and thenpublishesthe data to AWS IoT SiteWise, AWS IoT Core, or Kinesis Data Streams.Publishing data to AWS IoT SiteWise allows for end-to-end modeling and monitoring of your factory floor assets. When using the default solution configuration, publishing data to Kinesis Data Streams allows for ingesting and storing data in an Amazon S3 bucket. This gives you the capability for custom advanced analytics use cases and reporting.You can choose to create multiple connections, and specify sites, areas, processes, and machines, by using the setup UI.Management of connections and messagesFigure 3. Manage your connectionsM2C2 provides a straightforward connections screen (see Figure 3), where production managers can monitor and review the current state of connections. You can start and stop connections, view messages and errors, and gain connectivity across different areas of your factory floor. TheManage connectionsUI allows you to holistically manage data connectivity from a centralized place. You can then make changes and corrections as needed.Architecture and workflowFigure 4. Machine to Cloud Connectivity (M2C2) Framework architectureTheAWS CloudFormationtemplate deploys the following infrastructure, shown in Figure 4:AnAmazon CloudFrontuser interface that deploys into an Amazon S3 bucket configured for web hosting.AnAmazon API GatewayAPI provides the user interface for client requests.AnAmazon Cognitouser pool authenticates the API requests.AWS Lambdafunctions power the user interface, in addition to the configuration and deployment mechanism for AWS IoT Greengrass and AWS IoT SiteWise gateway resources.Amazon DynamoDBtables store the connection metadata.An AWS IoT SiteWise gateway configuration can be used for any OPC UA data sources.An Amazon Kinesis Data Streams data stream,Amazon Kinesis Data Firehose, and Amazon S3 bucket to store telemetry data.AWS IoT Greengrass is installed and used on an on-premises industrial gateway to runprotocol connector Lambda functions. These connect and read telemetry data from your OPC UA and OPC DA servers.Lambda functions are deployed onto AWS IoT Greengrass Core software on the industrial gateway. They connect to the servers and send the data to one or more configured destinations.Lambda functions that collect the telemetry data write toAWS IoT Greengrass stream managerstreams. ThepublisherLambda functions read from the streams.Publisher Lambda functions forward the data to the appropriate endpoint.Data collectionThe Machine to Cloud Connectivity solution uses Lambda functions running on Greengrass to connect to your on-premises OPC-DA and OPC-UA industrial devices. When you deploy a connection for an OPC-DA device, the solution configures a connection-specific OPC-DA connector Lambda. When you deploy a connection for an OPC-UA device, the solution uses the AWS IoT SiteWise Greengrass connector to collect the data.Regardless of protocol, the solution configures a publisher Lambda function, which takes care of sending your streaming data to one or more desired destinations.Stream Managerenables the reading and writing of stream data from multiple sources and to multiple destinations within the Greengrass core. This enables each configured collector to write data to a stream. The publisher reads from that stream and sends the data to your desired AWS resource.ConclusionMachine to Cloud Connectivity (M2C2) Framework is a self-deployable solution that provides secure connectivity between your technology (OT) assets and the AWS Cloud. With M2C2, you can send data to AWS IoT Core or AWS IoT SiteWise for analytics and monitoring. You can store your data in an industrial data lake using Kinesis Data Streams and Amazon S3. Get started withMachine to Cloud Connectivity (M2C2) Frameworktoday.Ajay SwamyAjay Swamy is Senior Product Lead at AWS Solutions Group. He is responsible for managing a portfolio of AWS Solutions that offers quick, easy to deploy cloud-based solutions for complex technical and business problems. He is based in NYC.Erin McGillErin McGill is a Software Development Engineer on the AWS Solutions team. She has contributed to the updates to Machine to Cloud Connectivity Framework. She is based in New York.
As a manufacturing enterprise, maximizing your operational efficiency and optimizing output are critical factors in this competitive global market. However, many manufacturers are unable to frequently collect data, link data together, and generate insights to help them optimize performance. Furthermore, decades of competing standards for connectivity have resulted in the lack of universal protocols to […]
Ajay Swamy
Amazon CloudFront Amazon DynamoDB Amazon Simple Storage Service (S3) Architecture AWS CloudFormation AWS IoT Core AWS IoT Greengrass AWS IoT SiteWise AWS Lambda AWS Solutions Implementations Manufacturing