GPUs are an important enabler for many Edge Computing Applications, especially to reduce compute latency for many use cases. For a broader overview on how to deploy GPU applications, you can watch our session at Nvidia GTC, where we show how to deploy a GPU docker compose deployment on MobiledgeX.
Some important considerations when deploying GPU Applications :
Not all cloudlets support GPU applications. You can verify whether a cloudlet has GPU resources from the Cloudlets tab in the MobiledgeX Console.
GPUs are a very limited resource for most cloudlets. If you are run into a resource error with a GPU flavor, it is likely that all GPUs are being used for that cloudlet. Please contact MobiledgeX Support if you run into such limitations.
GPU support is available for Docker and Virtual Machine deployments.
In regions where GPUs are deployed, a set of GPU-specific flavors are available. These flavors contain the string GPU, for example, m4.medium-gpu.
GPU-enabled Docker Deployment
Note: GPU support for Docker and Docker Compose is still actively under development. Please check back periodically for updates to GPU support.
You can deploy Docker containers to GPU-enabled hardware in the following ways:
As a Docker image
As Docker Compose file
Deploy a GPU-enabled Docker image
If you would like to use a Docker image for your GPU deployment, select a GPU-enabled Flavor for the application if you are using auto-clustering, or choose a Flavor for manually created clusters. Then, follow the steps as described within Upload images to MobiledgeX registries to upload your Docker image.
Deploy a GPU-enabled Docker Compose file
While the easiest way to deploy a GPU-enabled container is to use a Docker image, as described above, you can also use Docker Compose. Simply follow the steps in How to use a Docker Compose file as input, and remember to identify the following in your Docker Compose file:
Specify the Compose version as 2.3
Specify the option runtime-nvidia in your service definition
Add NVIDIA_VISIBLE_DEVICES=all to your environment section
Ensure you correctly specify your arguments in the yaml file, as shown in the example above.
Make sure you select the appropriate GPU-enabled flavor.
If you received errors indicating that "no host is available," the cloudlet you are attempting to use may not have any free GPU resources available. If that is the case, try deploying to a different cloudlet.
If problems continue to persist and you cannot deploy your Docker container to a GPU-enabled hardware, contact the Support Team for further assistance.