Field Notes: Launch a Fully Configured AWS Deep Learning Desktop with NICE DCV
AWS Architecture BlogField Notes: Launch a Fully Configured AWS Deep Learning Desktop with NICE DCVby Ajay Vohra and Dean Phillips | on22 JUL 2021| inAmazon SageMaker,Architecture,AWS Deep Learning AMIs,Technical How-to|Permalink|ShareYou want to start quickly when doing deep learning usingGPU-activated Elastic Compute Cloud (Amazon EC2) instancesin the AWS Cloud. Although AWS provides end-to-end machine learning (ML) inAmazon SageMaker, working at the deep learning frameworks level, the quickest way to start is withAWS Deep Learning AMIs (DLAMIs), which provide preconfigured Conda environments for most of the popular frameworks.DLAMIs make it straightforward to launchAmazon EC2instances, but these instances do not automatically provide the high-performance graphics visualization required during deep learning research. Additionally, they are not preconfigured to use AWS storage services or SageMaker. This post explains how you can launch a fully-configured deep learning desktop in the AWS Cloud. Not only is this desktop preconfigured with the popular frameworks such asTensorFlow,PyTorch, andApache MXNet, but it is also enabled for high-performance graphics visualization.NICE DCVis the remote display protocol used for visualization. In addition, it is preconfigured to use AWS storage services and SageMaker.Overview of the SolutionThe deep learning desktop described in this solution is ready for research and development of deep neural networks (DNNs) and their visualization. You no longer need to set up low-level drivers, libraries, and frameworks, or configure secure access to AWS storage services and SageMaker. The desktop has preconfigured access to your data in aSimple Storage Service (Amazon S3)bucket, and a sharedAmazon Elastic File System (Amazon EFS)is automatically attached to the desktop. It is automatically configured to accessSageMakerfor ML services, and provides you with the ability to prepare the data needed for deep learning, and to research, develop, build, and debug your DNNs. You can use all the advanced capabilities of SageMaker from your deep learning desktop. The following diagram shows the reference architecture for this solution.Figure 1 – Architecture overview of the solution to launch a fully configured AWS Deep Learning Desktop with NICE DCVThe deep learning desktop solution discussed in this post is contained in a singleAWS CloudFormationtemplate. To launch the solution, you create aCloudFormation stackfrom the template. Before we provide a detailed walkthrough for the solution, let us review the key benefits.DNN Research and DevelopmentDuring the DNN research phase, there is iterative exploration until you choose the preferred DNN architecture. During this phase, you may prefer to work in an isolated environment (for example, a dedicated desktop) with your favorite integrated development environment (IDE) (for example,Visual Studio CodeorPyCharm). Developers like the ability to step through code the IDE Debugger. With the increasing support for imperative programming in modern ML frameworks, the ability to step through code in the research phase can accelerate DNN development.The DLAMIs are preconfigured with NVIDIA GPU drivers, NVIDIA CUDA Toolkit, and low-level libraries such as Deep Neural Network library (cuDNN). Deep learning ML frameworks such as TensorFlow, PyTorch, and Apache MXNet are preconfigured.After you launch the deep learning desktop, you need to install and open your favorite IDE, clone your GitHub repository, and you can start researching, developing, debugging, and visualizing your DNN. The acceleration of DNN research and development is the first key benefit for the solution described in this post.Figure 2 – Developing on deep learning desktop with Visual Studio Code IDEElasticity in number of GPUsDuring the research phase, you need to debug any issues using a single GPU. However, as the DNN is stabilized, you horizontally scale across multiple GPUs in a single machine, followed by scaling across multiple machines.Most modern deep learning frameworks support distributed training across multiple GPUs in a single machine, and also across multiple machines. However, when you use a single GPU in an on-premises desktop equipped with multiple GPUs, the idle GPUs are wasted. With the deep learning desktop solution described in this post, you can stop the deep learning desktop instance, change its Amazon EC2 instance type to another compatible type, restart the desktop, and get the exact number of GPUs you need at the moment. The elasticity in the number of GPUs in the deep learning desktop is the second key benefit for the solution described in this post.Integrated access to storage servicesSince the deep learning desktop is running in AWS Cloud, you have access to all of the AWS data storage options, including the S3 object store, the Amazon EFS, and theAmazon FSx file system for Lustre. You can build your favorite data pipeline and it will be supported by one or more data storage options. You can also easily useML-IOlibrary, which is a high-performance data access library for ML tasks with support for multiple data formats. The integrated access to highly durable and scalable object and file system storage services for accessing ML data is the third key benefit for the solution described in this post.Integrated access to SageMakerOnce you have a stable version of your DNN, you need to find the right hyperparameters that lead to model convergence during training. Having tuned the hyperparameters, you need to run multiple trials over permutations of datasets and hyperparameters to fine-tune your models. Finally, you may need to prune and compile the models to optimize inference. To compress the training time, you may need to dodistributed data parallel trainingacross multiple GPUs in multiple machines. For all of these activities, the deep learning desktop is preconfigured to use SageMaker. You can usejupyter-labnotebooks running on the desktop to launch SageMaker training jobs for distributed training in infrastructure automatically managed by SageMaker.Figure 3 – Submitting a SageMaker training job from deep learning desktop using Jupyter Lab notebookThe SageMaker training logs,TensorBoardsummaries, and model checkpoints can be configured to be written to the Amazon EFS attached to the deep learning desktop. You can use the Linux command tail to monitor the logs, or start a TensorBoard server from the Conda environment on the deep learning desktop, and monitor the progress of your SageMaker training jobs. You can use a Jupyter Lab notebook running on the deep learning desktop to load a specific model checkpoint available on the Amazon EFS, and visualize the predictions from the model checkpoint, even while the SageMaker training job is still running.Figure 4 – Locally monitoring the TensorBoard summaries from SageMaker training jobSageMaker offers many advanced capabilities, such as profiling ML training jobs usingAmazon SageMaker Debugger, and these services are easily accessible from the deep learning desktop. You can manage the training input data, training model checkpoints, training logs, and TensorBoard summaries of your local iterative development, in addition to the distributed SageMaker training jobs, all from your deep learning desktop. The integrated access to SageMaker services is the fourth key use case for the solution described in this post.PrerequisitesTo get started, complete the following steps:If you do not have an AWS Account,create a new AWS account.You must haveAWS Identity and Access Management (IAM)access consistent withadministrator job function.In theAWS Management Console, select a supportedRegion(review the prerequisites in the read me file within the GitHubrepositoryfor the list of supported Regions).Subscribe toDLAMI (Ubuntu 18.04).Create a new Amazon EC2 key pair, or use an existing key pair.Create a new S3 bucketin the selected AWS Region, or use an existing bucket. The bucket can be empty.Clone the GitHubrepositoryon your laptop using theGit clonecommand.WalkthroughThe complete source and reference documentation for this solution is available in therepositoryaccompanying this post. Following is a walkthrough of the steps.Create a CloudFormation stackCreate a stack on the CloudFormation consolein your selected AWS Region using theCloudFormation templatein your cloned GitHub repository. This CloudFormation stack creates IAM resources. When you are creating a CloudFormation stack using the console, you must confirm:I acknowledge that AWS CloudFormation might create IAM resources.To create the CloudFormation stack, you must specify values for the following input parameters (for the rest of the input parameters, default values are recommended):DesktopAccessCIDR– Use thepublic internet addressof your laptop as the base value for theCIDR.DesktopInstanceType– For deep leaning, the recommended value for this parameter is p3.2xlarge, or larger.DesktopVpcId– Select an Amazon Virtual Private Cloud (VPC) with at least one public subnet.DesktopVpcSubnetId– Select a public subnet in your VPC.DesktopSecurityGroupId– The specified security group must allow inbound access over ports 22 (SSH) and 8443 (NICE DCV) from yourDesktopAccessCIDR, and must allow inbound access from within the security group to port 2049 and all network ports required for distributed SageMaker training in your subnet.If you leave it blank, the automatically-created security group allows inbound access for SSH, and NICE DCV from yourDesktopAccessCIDR, and allows inbound access to all ports from within the security group.KeyName– Select your SSH key pair name.S3Bucket– Specify your S3 bucket name. The bucket can be empty.Visit the documentation onall the input parameters.Connect to the deep learning desktopWhen the status for the stack in the CloudFormation console isCREATE_COMPLETE, find the deep learning desktop instance launched in your stack in the Amazon EC2 console,Connect to the instance using SSHas user ubuntu, using your SSH key pair. When you connect using SSH, if you see the message,“Cloud init in progress. Machine will REBOOT after cloud init is complete!!”, disconnect and try again in about 15 minutes.The desktop installs the NICE DCV server on first-time startup, and automatically reboots after the install is complete. If instead you see the message,“NICE DCV server is enabled!”, the desktop is ready for use.Before you can connect to the desktop using theNICE DCV client, you need to set a new password for user ubuntu using the Bash command:sudo passwd ubuntuAfter you successfully set the new password for user ubuntu, exit the SSH connection. You are now ready to connect to the desktop using a suitableNICE DCV client(a non–web browser client is recommended) using the userubuntu, and the new password.NICE DCV client asks you to specify the server host and port to connect. For the server host, use the public IPv4 DNS address of the desktop Amazon EC2 instance available in Amazon EC2 console.You do not need to specify the port, because the desktop is configured to use the default NICE DCV server port of 8443.When you first login to the desktop using the NICE DCV client, you will be asked if you would like to upgrade the OS version.Do not upgrade the OS version!Develop on the deep learning desktopWhen you are connected to the desktop using the NICE DCV client, use theUbuntu Software Centerto installVisual Studio Code, or your favorite IDE. To view the available Conda environments containing the popular deep learning frameworks preconfigured on the desktop, open a desktop terminal, and run the Bash command:conda env listThe deep learning desktop instance has secure access to the S3 bucket you specified when you created the CloudFormation stack. You can verify access to the S3 bucket by running the Bash command (replace ‘your-bucket-name’ following with your S3 bucket name):aws s3 ls your-bucket-nameIf your bucket is empty, a successful initiation of the previous command will produce no output, which is normal.AnAmazon Elastic Block Store (Amazon EBS)root volume is attached to the instance. In addition, an Amazon EFS is mounted on the desktop at the value ofEFSMountPathinput parameter, which by default is/home/ubuntu/efs. You can use the Amazon EFS for staging deep learning input and output data.Use SageMaker from the deep learning desktopThe deep learning desktop is preconfigured to use SageMaker. To get started with SageMaker examples in a JupyterLab notebook, launch the following Bash commands in a desktop terminal:mkdir ~/git
cd ~/git
git clone https://github.com/aws/amazon-sagemaker-examples.git
jupyter-labThis will start a ‘jupyter-lab’ notebook server in the terminal, and open a tab in your web browser. You can explore any of the SageMaker example notebooks. We recommend starting with the exampleDistributed Training of Mask-RCNN in SageMaker using Amazon EFSfound at the following path in the cloned repository:advanced_functionality/distributed_tensorflow_mask_rcnn/mask-rcnn-scriptmode-efs.ipynbThe preceding SageMaker example requires you to specify a subnet and a security group. Use the preconfigured OS environment variables as follows:security_group_ids = [ os.environ['desktop_sg_id'] ] 
subnets = [ os.environ['desktop_subnet_id' ] ]Stopping and restarting the desktopYou may safely reboot, stop, and restart the desktop instance at any time. The desktop will automatically mount the Amazon EFS at restart.Clean UpWhen you no longer need the deep learning desktop, you may delete the CloudFormation stack from the CloudFormation console. Deleting the stack will shut down the desktop instance, and delete the root Amazon EBS volume attached to the desktop. The Amazon EFS is not automatically deleted when you delete the stack.ConclusionIn this post, we showed how to launch a desktop pre-configured with the popular machine learning frameworks for research and development of deep learning neural networks.  NICE-DCV was used for high performance visualization related to deep learning. AWS storage services were used for highly scalable access to deep learning data.  Finally, Amazon SageMaker was used for the distributed training of deep learning data.Field Notes provides hands-on technical guidance from AWS Solutions Architects, consultants, and technical account managers, based on their experiences in the field solving real-world business problems for customers.TAGS:Field NotesAjay VohraAjay Vohra is a Principal Prototyping Architect specializing in perception machine learning for autonomous vehicle development. Prior to Amazon, Ajay worked in the area of massively parallel grid-computing for financial risk modeling.Dean PhillipsDean is Worldwide Technical Leader for Automotive at Amazon Web Services. As a technical executive and strategist, Dean leads a community of automotive Solution Architects who apply artificial intelligence, machine learning and internet of things technologies to build connected mobility, autonomous vehicle, and Smart Factory solutions with their customers.
You want to start quickly when doing deep learning using GPU-activated Elastic Compute Cloud (Amazon EC2) instances in the AWS Cloud. Although AWS provides end-to-end machine learning (ML) in Amazon SageMaker, working at the deep learning frameworks level, the quickest way to start is with AWS Deep Learning AMIs (DLAMIs), which provide preconfigured Conda environments for […]
Ajay Vohra
Amazon SageMaker Architecture AWS Deep Learning AMIs Technical How-to