Xavier Setup and requirements

• Flashed with JetPack 4.4 (L4T R32.4.3) with all ML and CV tools (including nvidia-docker)
• Samsung NVMe to store docker images and serve as location for Blob Storage data
• 16 GB swap file on NVMe mount
• Azure CLI for pushing image to Azure Container Registry
• [Optional] Docker may be configured to run with non-root user as in Manage Docker as a non-root user allowing the omission of using sudo with docker
• As Azure IoT Edge device (will need Azure IoT Hub for this)

Azure requirements

Create the following resources:

1. Azure Storage Account
2. Azure Container Registry
3. Azure IoT Hub
4. Azure Media Services Account

Note: All of these resources should be created in the same resource group to make it easier to clean-up later. Also, all of these resources may be create in the Portal or with the Azure CLI on the command line.

Build docker AI image

The following instructions will enable you to build a docker container with a YOLOv4 (tiny) TensorFlow Lite model using nginx, gunicorn, flask, and runit. The app code is based on the tensorflow-yolov4-tflite project. This project uses TensorFlow v2.

Note: References to third-party software in this repo are for informational and convenience purposes only. Microsoft does not endorse nor provide rights for the third-party software. For more information on third-party software please see the links provided above.

Prerequisites for building image

1. Ensure NVIDIA Docker on your Jetson
2. Install curl
3. Install Azure CLI to be able to push image to Azure Container Registry (ACR)

Preparing for using Blob Storage on the Edge

1. You will need to create a local key for the Blob Edge module. Generate the local storage key with the scripts/generate_key_for_local.py or a tool like GeneratePlus.
2. Create a file in the app/ folder called .env with the following contents:

LOCAL_STORAGE_ACCOUNT_NAME=<Name for local blob storage in IoT edge Blob Storage module (default: annotatedimageslocal)>
LOCAL_STORAGE_ACCOUNT_KEY=<Key you generated for local IoT edge Blob Storage module in double quotes>

Note:

• After the Edge deployment outlined below, two containers will be created in Azure Blob Storage:
  • annotated-images-yolo4
  • lowconf-images-yolo4
• Refer to Deploy the Azure Blob Storage on IoT Edge module to your device for general information

Building the docker container

1. Create a new directory on your machine and copy all the files (including the sub-folders) from this GitHub repo to that directory.
2. Build the container image (will take several minutes) by running the following docker command from a terminal window in that directory. Use your ACR URL (i.e. .azurecr.io) as part of the tag in the command below to make it easier to push to this ACR.

sudo nvidia-docker build . -t <your ACR URL>/tiny-yolov4-tflite:arm64v8-cuda-cudnn -f arm64v8-gpu-cudnn.dockerfile

Upload docker image to Azure Container Registry

Log in to ACR with the Azure CLI (also may use docker login):

az acr login --name <name of your ACR user>

Push the image to ACR:

docker push <your ACR URL>/tiny-yolov4-tflite:arm64v8-cuda-cudnn

Note:

• More instruction at Push and Pull Docker images - Azure Container Registry to save your image for later use on another machine.
• IMPORTANT: Docker may need to be configured to run with non-root user as in Manage Docker as a non-root user.

Deploy as an edge module for Live Video Analytics

Visual Studio Code (VSCode) will be used for the following. The following will be needed:

1. Azure IoT Tools extension installed (search in VSCode for this)
2. Deployment manifest file from this repo (edge-module/deployment.yolov4.gpu.jetson.blob.template.json)
3. LVA Python console app or LVA .NET console app
4. IDs and keys associated with Azure Resources

When the Live Video Analytics on Edge direct methods are invoked (through the LVA console app) on device with the console app, images will appear in a folder with the name of your local container e.g. /media/nvme/blob_storage/BlockBlob/annotatedimageslocal and with default deployment manifest, will stick around on device for 60 minutes as well as being uploaded to the cloud Blob Storage container (in this example, called annotated-images-yolo4).

Steps:

1. Associate the device with IoT Hub if haven't already
2. Create a Service Principal associated with the AMS account (Manage your Azure AD app and secret)
3. Create a .env parameters file within the edge-module folder with the following contents:

SUBSCRIPTION_ID_FOR_AMS=<Azure subscription associated with the AMS account>
RESOURCE_GROUP_FOR_AMS=<Resource Group where the AMS account resides>
AMS_ACCOUNT=<AMS account name>
IOTHUB_CONNECTION_STRING="<IoT Hub connection string>"
AAD_TENANT_ID=<Active Directory tenant ID>
AAD_SERVICE_PRINCIPAL_ID=<Active Directory Service Principal ID associated with AMS>
AAD_SERVICE_PRINCIPAL_SECRET="<Service Principal secret/key associated with AMS>"
INPUT_VIDEO_FOLDER_ON_DEVICE="<input video folder on device for the media server>"
OUTPUT_VIDEO_FOLDER_ON_DEVICE="<output video folder on device to sink video clips from LVA>"
OUTPUT_BLOB_FOLDER_ON_DEVICE="<output folder for Blob Storage data>"
APPDATA_FOLDER_ON_DEVICE="/var/lib/azuremediaservices"
CONTAINER_REGISTRY_USERNAME_myacr=<ACR username>
CONTAINER_REGISTRY_PASSWORD_myacr=<ACR key>
CLOUD_STORAGE_CONN_STRING="<Azure Blob Storage cloud connection string>"
LOCAL_STORAGE_ACCOUNT_NAME="<local Edge Storage account name>"
LOCAL_STORAGE_ACCOUNT_KEY="<local Edge Storage account key>"

These variables, from the .env, will be automatically populated into the IoT Edge deployment manifest file after a deployment manifest file is generated from the template.

4. Generate the deployment manifest and deploy with VSCode or the Azure CLI
5. Run the LVA console app (Python or .NET)
6. Monitor the device for messages from VSCode

From VSCode, messages to IoT Hub should look similar to:

[IoTHubMonitor] [1:11:41 PM] Message received from [xavier-yolov4/lvaEdge]:
{
  "inferences": [
    {
      "type": "entity",
      "entity": {
        "tag": {
          "value": "car",
          "confidence": "0.43346554"
        },
        "box": {
          "l": "0.6137574",
          "t": "0.5797131",
          "w": "0.05888599",
          "h": "0.047415733"
        }
      }
    },
    {
      "type": "entity",
      "entity": {
        "tag": {
          "value": "truck",
          "confidence": "0.33760804"
        },
        "box": {
          "l": "0.6137574",
          "t": "0.5797131",
          "w": "0.05888599",
          "h": "0.047415733"
        }
      }
    }
  ]
}

Notes:

• Learn more about Edge deployments with Understand IoT Edge automatic deployments for single devices or at scale
• Review LVA through the guide Quickstart: Get started - Live Video Analytics on IoT Edge

Flask Python web app dashboard

Requirments

1. Python 3.6+

Get started

1. Navigate to the web-app directory.

2. Create your environment variables as show in the example below and load them into the cu