[Jasper/PyT] Updated notebooks

Author: nvpstr

PyTorch/SpeechRecognition/Jasper/notebooks/JasperTRT.ipynb

@@ -26,7 +26,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "<img src=\"img/nvidia_logo.png\" style=\"width: 90px; float: right;\">\n",
+    "<img src=http://developer.download.nvidia.com/compute/machine-learning/frameworks/nvidia_logo.png style=\"width: 90px; float: right;\">\n",
     "\n",
     "# Jasper Inference For TensorRT 6\n",
     "This Jupyter notebook provides scripts to perform high-performance inference using NVIDIA TensorRT. \n",
@@ -88,17 +88,6 @@
     "### 3.1  Start a detached session in the NGC container"
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "DATA_DIR=\"$PWD/../data\" # replace with absolute path to dataset root folder that contains various datasets. E.g. this path should contain LibriSpeech as subfolder\n",
-    "CHECKPOINT_DIR=\"$PWD/../checkpoints\" # replace with absolute path to checkpoint folder. Following code assumes this folder to contain 'jasper_fp16.pt'\n",
-    "RESULT_DIR=\"$PWD/../results\" # replace with absolute path to result folder, where log files and prediction files will be saved after inference."
-   ]
-  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -110,17 +99,18 @@
     "  --shm-size=4g \\\n",
     "  --ulimit memlock=-1 \\\n",
     "  --ulimit stack=67108864 \\\n",
-    "  -v $DATA_DIR:/datasets \\\n",
-    "  -v $CHECKPOINT_DIR:/checkpoints/ \\\n",
-    "  -v $RESULT_DIR:/results/ \\\n",
+    "  -v $PWD/data:/datasets \\\n",
+    "  -v $PWD/checkpoint:/checkpoints/ \\\n",
+    "  -v $PWD/result:/results/ \\\n",
+    "  -v $PWD:/workspace/jasper/ \\\n",
     "  jasper:trt6 bash"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "You can also specify the GPU to run the container on by adding \\\"NV_GPU\\\" before the \\\"nvidia-docker run\\\" command. For example, to specify GPU ID 0 to run the container, add \\\"NV_GPU=0\\\" before the \\\"nvidia-docker run\\\" command. You can use the command \\\"nvidia-smi\\\" to check your GPU IDs and utilization"
+    "You can also specify single or multiple GPUs to run the container by adding \"NV_GPU\" before the \"nvidia-docker run\" command. For example, to specify GPU ID 2 to run the container, add \"NV_GPU=2\" before the \"nvidia-docker run\" command. You can use the command \"nvidia-smi\" to check your GPU IDs and utilization."
    ]
   },
   {
@@ -129,22 +119,17 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!NV_GPU=0 nvidia-docker run -it -d --rm --name \"JasperTRT\" \\\n",
-    "  --runtime=nvidia \\\n",
-    "  --shm-size=4g \\\n",
-    "  --ulimit memlock=-1 \\\n",
-    "  --ulimit stack=67108864 \\\n",
-    "  -v $DATA_DIR:/datasets \\\n",
-    "  -v $CHECKPOINT_DIR:/checkpoints/ \\\n",
-    "  -v $RESULT_DIR:/results/ \\\n",
-    "  jasper:trt6 bash"
+    "#check the container that you just started\n",
+    "!docker ps -a"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "### 3.2 Download and preprocess the dataset.\n",
+    "You will not need to download the dataset if you directly go to Section 5 to play with audio examples.\n",
+    "\n",
     "If LibriSpeech http://www.openslr.org/12 has already been downloaded and preprocessed, no further steps in this subsection need to be taken.\n",
     "If LibriSpeech has not been downloaded already, note that only a subset of LibriSpeech is typically used for inference (dev-* and test-*). LibriSpeech contains 1000 hours of 16kHz read English speech derived from public domain audiobooks from LibriVox project and has been carefully segmented and aligned. For more information, see paper [LIBRISPEECH: AN ASR CORPUS BASED ON PUBLIC DOMAIN AUDIO BOOKS paper](http://www.danielpovey.com/files/2015_icassp_librispeech.pdf).\n",
     "To acquire the inference subset of LibriSpeech run (does not require GPU):"
@@ -205,16 +190,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### 3.3 Download pretrained model checkpoint\n",
-    "A pretrained model checkpoint can be downloaded from NGC model repository https://ngc.nvidia.com/catalog/models/nvidia:jasperpyt_fp16\n",
-    "        \n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### 3.4. Start TensorRT inference prediction\n",
+    "### 3.3. Start TensorRT inference prediction\n",
     "\n",
     "Inside the container, use the following script to run inference with TensorRT.\n",
     "You will need to set the parameters such as: \n",
@@ -244,7 +220,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    " ### 3.5. Start TensorRT Inference Benchmark\n",
+    " ### 3.4. Start TensorRT Inference Benchmark\n",
     "\n",
     "Run the following commmand to run inference benchmark with TensorRT inside the container.\n",
     "\n",
@@ -299,14 +275,14 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!nvidia-docker exec -it -e CHECKPOINT=/checkpoints/jasper_fp16.pt -e TRT_PREDICTION_PATH=/results/benchmark.txt -e TRT_PRECISION=fp16 -e PYTORCH_PRECISION=fp16 -e CSV_PATH=/results/res_fp16.csv JasperTRT bash trt/scripts/trt_inference_benchmark.sh"
+    "!nvidia-docker exec -it -e CHECKPOINT=/checkpoints/jasper_fp16.pt -e TRT_PREDICTION_PATH=/results/benchmark.txt -e TRT_PRECISION=fp16 -e PYTORCH_PRECISION=fp16 -e CSV_PATH=/result/res_fp16.csv JasperTRT bash trt/scripts/trt_inference_benchmark.sh"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "From the performance metrics that you get from res.csv (fp32) and res_fp16.csv (automatic mixed precision) files, you can see that automatic mixed precision can speedup the inference efficiently compared to fp32."
+    "From the performance metrics (pyt_infer) that you get from res.csv (for fp32) and res_fp16.csv (for automatic mixed precision) files, you can see that automatic mixed precision can speedup the inference efficiently compared to fp32."
    ]
   },
   {
@@ -315,7 +291,7 @@
    "source": [
     "### 5. Play with audio examples\n",
     "\n",
-    "You can perform inference using pre-trained checkpoints which takes audio file (in .wav format) as input, and produces the corresponding text file. You can customize the content of the text file. For example, there are examples of audio files in \\\"notebooks\\\" directory and we can listen to keynote.wav"
+    "You can perform inference using pre-trained checkpoints which takes audio file (in .wav format) as input, and produces the corresponding text file. You can customize the content of the text file. For example, there are several examples of input files at \"notebooks\" dirctory and we can listen to example1.wav:"
    ]
   },
   {
@@ -325,7 +301,7 @@
    "outputs": [],
    "source": [
     "import IPython.display as ipd\n",
-    "ipd.Audio('keynote.wav', rate=22050)"
+    "ipd.Audio('notebooks/example1.wav', rate=22050)"
    ]
   },
   {
@@ -334,9 +310,9 @@
    "source": [
     "You can run inference using the trt/perf.py script:\n",
     "* the checkpoint is passed as `--ckpt` argument \n",
-    "* `--model_toml` specifies the path to network configuration file (see examples in \\\"config\\\" directory)\n",
+    "* `--model_toml` specifies the path to network configuration file (see examples in \"config\" directory)\n",
     "* `--make_onnx` exports to ONNX file at the path if set\n",
-    "* `--engine_path` saves the engine (*.plan) file.\n",
+    "* `--engine_path` saves the engine file (*.plan) \n",
     "\n",
     "To create a new engine file (jasper.plan) for TensorRT and run it using fp32 (building the engine for the first time can take several minutes):"
    ]
@@ -347,7 +323,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!nvidia-docker exec -it JasperTRT python trt/perf.py --ckpt_path /checkpoints/jasper_fp16.pt --wav=notebooks/keynote.wav --model_toml=configs/jasper10x5dr_nomask.toml --make_onnx --onnx_path jasper.onnx --engine_path jasper.plan"
+    "!nvidia-docker exec -it JasperTRT python trt/perf.py --ckpt_path /checkpoints/jasper_fp16.pt --wav=notebooks/example1.wav --model_toml=configs/jasper10x5dr_nomask.toml --make_onnx --onnx_path jasper.onnx --engine_path jasper.plan"
    ]
   },
   {
@@ -363,7 +339,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!nvidia-docker exec -it JasperTRT python trt/perf.py --wav=notebooks/keynote.wav --model_toml=configs/jasper10x5dr_nomask.toml --use_existing_engine --engine_path jasper.plan --trt_fp16"
+    "!nvidia-docker exec -it JasperTRT python trt/perf.py --wav=notebooks/example1.wav --model_toml=configs/jasper10x5dr_nomask.toml --use_existing_engine --engine_path jasper.plan --trt_fp16"
    ]
   },
   {
@@ -379,7 +355,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!nvidia-docker exec -it JasperTRT python trt/perf.py --ckpt_path /checkpoints/jasper_fp16.pt --wav=notebooks/keynote.wav --model_toml=configs/jasper10x5dr_nomask.toml --make_onnx --onnx_path jasper.onnx --engine_path jasper_fp16.plan --trt_fp16"
+    "!nvidia-docker exec -it JasperTRT python trt/perf.py --ckpt_path /checkpoints/jasper_fp16.pt --wav=notebooks/example1.wav --model_toml=configs/jasper10x5dr_nomask.toml --make_onnx --onnx_path jasper.onnx --engine_path jasper_fp16.plan --trt_fp16"
    ]
   },
   {
@@ -395,14 +371,14 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!nvidia-docker exec -it JasperTRT python trt/perf.py --wav=notebooks/keynote.wav --model_toml=configs/jasper10x5dr_nomask.toml --use_existing_engine --engine_path jasper_fp16.plan --trt_fp16"
+    "!nvidia-docker exec -it JasperTRT python trt/perf.py --wav=notebooks/example1.wav --model_toml=configs/jasper10x5dr_nomask.toml --use_existing_engine --engine_path jasper_fp16.plan --trt_fp16"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "You can also input your own audio files and generate the output text files in this way.\n",
+    "You can play with other examples at \"notebooks\" dirctory. You can also input your own audio files and generate the output text files in this way.\n",
     "\n",
     "For more information about TensorRT and building an engine file in Python, please see: https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#python_topics"
    ]
@@ -448,7 +424,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.3"
+   "version": "3.6.8"
   }
  },
  "nbformat": 4,

PyTorch/SpeechRecognition/Jasper/notebooks/README.md

@@ -1,63 +1,74 @@
+# Jasper notebook
+
 ## Overview
 
-This notebook provides scripts for you to run Jasper with TRT for inference step by step. You can run inference using either LibriSpeech dataset or your own audio input in .wav format, to generate the corresponding text file for the audio file. 
+This notebook provides scripts for you to run Jasper with TRT for inference step by step. You can run inference using either LibriSpeech dataset or your own audio input in .wav format, to generate the corresponding text file for the audio file.
 
 ## Requirements
 
 This repository contains a Dockerfile which extends the PyTorch 19.09-py3 NGC container and encapsulates some dependencies. Aside from these dependencies, ensure you have the following components:
-    
-* [NVIDIA Docker](https://github.com/NVIDIA/nvidia-docker)
-* [PyTorch 19.09-py3 NGC container](https://ngc.nvidia.com/catalog/containers/nvidia:pytorch)
-* [NVIDIA Turing](https://www.nvidia.com/en-us/geforce/turing/) or [Volta](https://www.nvidia.com/en-us/data-center/volta-gpu-architecture/) based GPU
-* [Pretrained Jasper Model Checkpoint](https://ngc.nvidia.com/catalog/models/nvidia:jasperpyt_fp16)
+
+-    NVIDIA Docker
+-    PyTorch 19.09-py3 NGC container
+-    NVIDIA Turing or Volta based GPU
+-    Pretrained Jasper Model Checkpoint
 
 ## Quick Start Guide
 
 Running the following scripts will build and launch the container containing all required dependencies for both TensorRT as well as native PyTorch. This is necessary for using inference with TensorRT and can also be used for data download, processing and training of the model.
 
-1. Clone the repository.
+#### 1. Clone the repository.
 
-```bash
+```
 git clone https://github.com/NVIDIA/DeepLearningExamples
 cd DeepLearningExamples/PyTorch/SpeechRecognition/Jasper
 ```
-2. Build the Jasper PyTorch with TRT 6 container:
 
-```bash
+#### 2. Build the Jasper PyTorch with TRT 6 container:
+
+```
 bash trt/scripts/docker/trt_build.sh
 ```
-3. Prepare to start a detached session in the NGC container
+
+#### 3. Create directories
+Prepare to start a detached session in the NGC container.
 Create three directories on your local machine for dataset, checkpoint, and result, respectively, naming "data" "checkpoint" "result":
 
-```bash
+```
 mkdir data checkpoint result
 ```
-Download the checkpoint file `jasperpyt_fp16` to the directory `checkpoint` from NGC Model Repository: https://ngc.nvidia.com/catalog/models/nvidia:jasperpyt_fp16
-Assume you will download the dataset to /dev/sdb and mount the data on /dev/sdb to "data", please replace "/dev/sdb" with your own directories if you use other directories:
 
-```bash
-sudo mount /dev/sdb data
-```
-The Jasper PyTorch container will be launched in the Jupyter notebook. Within the container, the contents of the root repository will be copied to the /workspace/jasper directory. The /datasets, /checkpoints, /results directories are mounted as volumes and mapped to the corresponding directories "data" "checkpoint" "result" on the host.
+#### 4. Download the checkpoint
+Download the checkpoint file jasperpyt_fp16 from NGC Model Repository:  
+- https://ngc.nvidia.com/catalog/models/nvidia:jasperpyt_fp16
+
+to the directory: _checkpoint_
+
+The Jasper PyTorch container will be launched in the Jupyter notebook. Within the container, the contents of the root repository will be copied to the /workspace/jasper directory.
 
-Copy the JasperTRT.ipynb notebook to the root directory of Jasper:
-```bash
+The /datasets, /checkpoints, /results directories are mounted as volumes and mapped to the corresponding directories "data" "checkpoint" "result" on the host.
+
+#### 5. Copy the notebook to the root
+
+Copy the notebook to the root directory of Jasper:
+
+```
 cp notebooks/JasperTRT.ipynb .
 ```
 
+#### 6. Run the notebook
 For running the notebook on your local machine, run:
 
-```bash
+```
 jupyter notebook JasperTRT.ipynb
 ```
-For running the notebook on another machine remotely, run: 
 
-```bash
+For running the notebook on another machine remotely, run:
+
+```
 jupyter notebook --ip=0.0.0.0 --allow-root
 ```
-And navigate a web browser to the IP address or hostname of the host machine at port 8888: http://[host machine]:8888
-
-Use the token listed in the output from running the jupyter command to log in, for example: http://[host machine]:8888/?token=aae96ae9387cd28151868fee318c3b3581a2d794f3b25c6b
-
 
+And navigate a web browser to the IP address or hostname of the host machine at port 8888: `http://[host machine]:8888`
 
+Use the token listed in the output from running the jupyter command to log in, for example: `http://[host machine]:8888/?token=aae96ae9387cd28151868fee318c3b3581a2d794f3b25c6b`

PyTorch/SpeechRecognition/Jasper/notebooks/example1.wav

@@ -1,4 +1,3 @@
-
 ARG FROM_IMAGE_NAME=nvcr.io/nvidia/pytorch:19.09-py3
 FROM ${FROM_IMAGE_NAME}