{
  "cells": [
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "aGYwt_UjIrqp"
      },
      "source": [
        "# Instance Segmentation\n",
        "\n",
        "In this tutorial, you will learn:\n",
        "- the basic structure of Mask R-CNN.\n",
        "- to perform inference with a MMDetection detector.\n",
        "- to train a new instance segmentation model with a new dataset.\n",
        "\n",
        "Let's start!\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6hD0mmMixT0p",
        "outputId": "3fdfddc5-9314-4d11-ed2b-2833795e1cb6"
      },
      "outputs": [],
      "source": [
        "# Check Pytorch installation\n",
        "import torch, torchvision\n",
        "print(\"torch version:\",torch.__version__, \"cuda:\",torch.cuda.is_available())\n",
        "\n",
        "# Check MMDetection installation\n",
        "import mmdet\n",
        "print(\"mmdetection:\",mmdet.__version__)\n",
        "\n",
        "# Check mmcv installation\n",
        "import mmcv\n",
        "print(\"mmcv:\",mmcv.__version__)\n",
        "\n",
        "# Check mmengine installation\n",
        "import mmengine\n",
        "print(\"mmengine:\",mmengine.__version__)\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "gi9zw03oM4CH"
      },
      "source": [
        "## Perform Inference with An MMDetection Detector"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "3pFYLerc0we1"
      },
      "source": [
        "### A two-stage detector\n",
        "\n",
        "In this tutorial, we use Mask R-CNN, a simple two-stage detector as an example.\n",
        "\n",
        "The high-level architecture of Mask R-CNN is shown in the following picture. More details can be found in the [paper](https://arxiv.org/abs/1703.06870).\n",
        "\n",
        "<img src=\"https://user-images.githubusercontent.com/40661020/143967081-c2552bed-9af2-46c4-ae44-5b3b74e5679f.png\" alt=\"mask rcnn\" align=\"bottom\" />\n",
        "\n",
        "Mask R-CNN adds a mask branch based on the original Faster R-CNN. It also uses RoIAlign, a more precise version of RoIPooling for RoI feature extraction to improve the performance.\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "!mim download mmdet --config mask-rcnn_r50-caffe_fpn_ms-poly-3x_coco --dest ./checkpoints"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8M5KUnX7Np3h",
        "outputId": "ef343a81-a46b-4041-8f6c-a4049a5c8a4e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "local loads checkpoint from path: checkpoints/mask_rcnn_r50_caffe_fpn_mstrain-poly_3x_coco_bbox_mAP-0.408__segm_mAP-0.37_20200504_163245-42aa3d00.pth\n"
          ]
        }
      ],
      "source": [
        "import mmcv\n",
        "import mmengine\n",
        "from mmdet.apis import init_detector, inference_detector\n",
        "from mmdet.utils import register_all_modules\n",
        "# Choose to use a config and initialize the detector\n",
        "config_file = 'configs/mask_rcnn/mask-rcnn_r50-caffe_fpn_ms-poly-3x_coco.py'\n",
        "# Setup a checkpoint file to load\n",
        "checkpoint_file = 'checkpoints/mask_rcnn_r50_caffe_fpn_mstrain-poly_3x_coco_bbox_mAP-0.408__segm_mAP-0.37_20200504_163245-42aa3d00.pth'\n",
        "\n",
        "# register all modules in mmdet into the registries\n",
        "register_all_modules()\n",
        "\n",
        "# build the model from a config file and a checkpoint file\n",
        "model = init_detector(config_file, checkpoint_file, device='cuda:0')  # or device='cuda:0'\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "pVqDQAOiKkJK"
      },
      "source": [
        "From the printed model, we will find that the model does consist of the components that we described earlier. It uses ResNet as its CNN backbone, and has a RPN head and RoI Head. \n",
        "The RoI Head includes box head and mask head. In addition, the model has a neural network module, named neck, directly after the CNN backbone. It is a [feature pyramid network (FPN)](https://arxiv.org/abs/1612.03144) for enhancing the multi-scale features.\n",
        "\n",
        "\n",
        "### Inference with the detector\n",
        "\n",
        "The model is successfully created and loaded, let's see how good it is. We use the high-level API `inference_detector` implemented in the MMDetection. This API is created to ease the inference process. The details of the codes can be found [here](https://github.com/open-mmlab/mmdetection/blob/master/mmdet/apis/inference.py#L15)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Wi6DRpsQPEmV",
        "outputId": "8670eb7c-7e35-4c6d-edf8-9599c296fd01"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "<DetDataSample(\n",
            "\n",
            "    META INFORMATION\n",
            "    img_path: None\n",
            "    img_id: 0\n",
            "    ori_shape: (427, 640)\n",
            "    pad_shape: (800, 1216)\n",
            "    img_shape: (800, 1199)\n",
            "    batch_input_shape: (800, 1216)\n",
            "    scale_factor: (1.8734375, 1.873536299765808)\n",
            "\n",
            "    DATA FIELDS\n",
            "    ignored_instances: <InstanceData(\n",
            "        \n",
            "            META INFORMATION\n",
            "        \n",
            "            DATA FIELDS\n",
            "        ) at 0x7f2e2e6bcf50>\n",
            "    pred_instances: <InstanceData(\n",
            "        \n",
            "            META INFORMATION\n",
            "        \n",
            "            DATA FIELDS\n",
            "            bboxes: tensor([[219.8669, 177.0847, 458.3748, 382.1474],\n",
            "                        [480.5577, 110.4770, 523.1896, 130.3325],\n",
            "                        [167.2964, 108.4705, 219.3719, 139.7076],\n",
            "                        [432.7686, 104.8358, 482.8203, 132.2971],\n",
            "                        [293.9351, 116.1539, 381.3922, 150.1960],\n",
            "                        [189.7073, 107.6949, 296.3810, 154.8893],\n",
            "                        [608.6339, 111.2585, 635.6986, 136.8761],\n",
            "                        [  0.0000, 112.1333,  64.1112, 145.1969],\n",
            "                        [396.6749, 110.8233, 434.0646, 133.4602],\n",
            "                        [558.0623, 109.3838, 593.6662, 127.6788],\n",
            "                        [590.7148, 109.8769, 620.7405, 126.5420],\n",
            "                        [269.1914, 105.4428, 327.0159, 127.3173],\n",
            "                        [ 85.7338,  89.2483,  99.3804,  98.0981],\n",
            "                        [ 98.4362,  89.3782, 118.6272, 100.8053],\n",
            "                        [372.5269, 136.1227, 432.7708, 187.9508],\n",
            "                        [377.2284, 135.4568, 432.1865, 187.9847],\n",
            "                        [553.8429, 110.4870, 562.0100, 126.6810],\n",
            "                        [604.7444, 109.7502, 630.5547, 127.8575],\n",
            "                        [186.7552, 109.0126, 228.5790, 145.0180],\n",
            "                        [535.8762, 108.3895, 545.3394, 124.3338],\n",
            "                        [254.0456, 103.2392, 263.8701, 113.5147],\n",
            "                        [177.0460, 105.5745, 257.9588, 144.5202],\n",
            "                        [375.6874, 119.4976, 382.5284, 132.6479],\n",
            "                        [609.1522, 113.8937, 624.6566, 125.1477],\n",
            "                        [375.7222, 119.5245, 382.4897, 133.1029],\n",
            "                        [222.0861,  98.0716, 251.4668, 107.5865],\n",
            "                        [100.2691,  88.5256, 117.7642,  95.0901],\n",
            "                        [614.0441, 102.3211, 634.2105, 112.2726],\n",
            "                        [532.6127, 108.7434, 541.8550, 125.0906],\n",
            "                        [541.6937, 111.1456, 561.5138, 126.8486],\n",
            "                        [533.5305, 108.6224, 545.8924, 136.2011],\n",
            "                        [135.4246,  89.6802, 152.1010,  96.3840],\n",
            "                        [ 98.6437,  92.1219, 110.0702, 100.7781],\n",
            "                        [168.5462,  96.7544, 182.6949, 105.4325],\n",
            "                        [373.9462, 112.1712, 415.0881, 133.9640],\n",
            "                        [555.3751, 110.0361, 574.9877, 127.4093],\n",
            "                        [227.8020, 115.6859, 237.1718, 123.8733],\n",
            "                        [430.8841, 104.9890, 484.4784, 131.9014],\n",
            "                        [294.1376, 116.5310, 378.7437, 149.9074],\n",
            "                        [  7.9081, 101.6706, 121.0289, 142.4942],\n",
            "                        [364.8197, 108.6048, 437.1257, 133.0375],\n",
            "                        [392.4489, 111.1929, 434.3052, 133.6733],\n",
            "                        [187.0611, 108.6241, 300.6114, 155.1898],\n",
            "                        [ 13.0592,  93.1397,  25.3220, 126.1789],\n",
            "                        [ 80.9496,  89.6315,  92.6623,  97.9697],\n",
            "                        [241.6728, 116.3161, 252.0896, 125.3812],\n",
            "                        [598.0181, 103.6448, 634.9662, 124.7286],\n",
            "                        [ 82.1132,  89.2355,  99.1312,  97.9048],\n",
            "                        [ 98.8391,  89.3512, 119.2116, 101.0520],\n",
            "                        [  9.4260, 106.1665, 107.9734, 144.0200],\n",
            "                        [134.5091,  90.4124, 152.7241, 100.8206],\n",
            "                        [ 43.4657, 105.9954, 114.2389, 128.5451]], device='cuda:0')\n",
            "            masks: tensor([[[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        ...,\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]]], device='cuda:0')\n",
            "            scores: tensor([0.9990, 0.9938, 0.9878, 0.9854, 0.9836, 0.9828, 0.9781, 0.9778, 0.9731,\n",
            "                        0.9605, 0.9476, 0.9366, 0.6861, 0.6647, 0.6537, 0.5689, 0.4984, 0.3680,\n",
            "                        0.3508, 0.2958, 0.2663, 0.2558, 0.2349, 0.2339, 0.2218, 0.2080, 0.2002,\n",
            "                        0.1949, 0.1807, 0.1676, 0.1652, 0.1503, 0.1207, 0.1145, 0.1064, 0.1062,\n",
            "                        0.0810, 0.0799, 0.0776, 0.0751, 0.0731, 0.0699, 0.0686, 0.0655, 0.0637,\n",
            "                        0.0590, 0.0583, 0.0572, 0.0539, 0.0532, 0.0527, 0.0514],\n",
            "                       device='cuda:0')\n",
            "            labels: tensor([13,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2, 13, 56,  2,  2,\n",
            "                         2,  0,  0,  2,  2,  2,  0,  2,  2,  2,  0,  2,  0,  2,  2,  2,  2,  2,\n",
            "                         0,  7,  7,  7,  2,  7,  7, 25,  2,  0,  2,  7,  7,  2,  2,  7],\n",
            "                       device='cuda:0')\n",
            "        ) at 0x7f2e3a16cc10>\n",
            "    _pred_instances: <InstanceData(\n",
            "        \n",
            "            META INFORMATION\n",
            "        \n",
            "            DATA FIELDS\n",
            "            bboxes: tensor([[219.8669, 177.0847, 458.3748, 382.1474],\n",
            "                        [480.5577, 110.4770, 523.1896, 130.3325],\n",
            "                        [167.2964, 108.4705, 219.3719, 139.7076],\n",
            "                        [432.7686, 104.8358, 482.8203, 132.2971],\n",
            "                        [293.9351, 116.1539, 381.3922, 150.1960],\n",
            "                        [189.7073, 107.6949, 296.3810, 154.8893],\n",
            "                        [608.6339, 111.2585, 635.6986, 136.8761],\n",
            "                        [  0.0000, 112.1333,  64.1112, 145.1969],\n",
            "                        [396.6749, 110.8233, 434.0646, 133.4602],\n",
            "                        [558.0623, 109.3838, 593.6662, 127.6788],\n",
            "                        [590.7148, 109.8769, 620.7405, 126.5420],\n",
            "                        [269.1914, 105.4428, 327.0159, 127.3173],\n",
            "                        [ 85.7338,  89.2483,  99.3804,  98.0981],\n",
            "                        [ 98.4362,  89.3782, 118.6272, 100.8053],\n",
            "                        [372.5269, 136.1227, 432.7708, 187.9508],\n",
            "                        [377.2284, 135.4568, 432.1865, 187.9847],\n",
            "                        [553.8429, 110.4870, 562.0100, 126.6810],\n",
            "                        [604.7444, 109.7502, 630.5547, 127.8575],\n",
            "                        [186.7552, 109.0126, 228.5790, 145.0180],\n",
            "                        [535.8762, 108.3895, 545.3394, 124.3338],\n",
            "                        [254.0456, 103.2392, 263.8701, 113.5147],\n",
            "                        [177.0460, 105.5745, 257.9588, 144.5202],\n",
            "                        [375.6874, 119.4976, 382.5284, 132.6479],\n",
            "                        [609.1522, 113.8937, 624.6566, 125.1477],\n",
            "                        [375.7222, 119.5245, 382.4897, 133.1029],\n",
            "                        [222.0861,  98.0716, 251.4668, 107.5865],\n",
            "                        [100.2691,  88.5256, 117.7642,  95.0901],\n",
            "                        [614.0441, 102.3211, 634.2105, 112.2726],\n",
            "                        [532.6127, 108.7434, 541.8550, 125.0906],\n",
            "                        [541.6937, 111.1456, 561.5138, 126.8486],\n",
            "                        [533.5305, 108.6224, 545.8924, 136.2011],\n",
            "                        [135.4246,  89.6802, 152.1010,  96.3840],\n",
            "                        [ 98.6437,  92.1219, 110.0702, 100.7781],\n",
            "                        [168.5462,  96.7544, 182.6949, 105.4325],\n",
            "                        [373.9462, 112.1712, 415.0881, 133.9640],\n",
            "                        [555.3751, 110.0361, 574.9877, 127.4093],\n",
            "                        [227.8020, 115.6859, 237.1718, 123.8733],\n",
            "                        [430.8841, 104.9890, 484.4784, 131.9014],\n",
            "                        [294.1376, 116.5310, 378.7437, 149.9074],\n",
            "                        [  7.9081, 101.6706, 121.0289, 142.4942],\n",
            "                        [364.8197, 108.6048, 437.1257, 133.0375],\n",
            "                        [392.4489, 111.1929, 434.3052, 133.6733],\n",
            "                        [187.0611, 108.6241, 300.6114, 155.1898],\n",
            "                        [ 13.0592,  93.1397,  25.3220, 126.1789],\n",
            "                        [ 80.9496,  89.6315,  92.6623,  97.9697],\n",
            "                        [241.6728, 116.3161, 252.0896, 125.3812],\n",
            "                        [598.0181, 103.6448, 634.9662, 124.7286],\n",
            "                        [ 82.1132,  89.2355,  99.1312,  97.9048],\n",
            "                        [ 98.8391,  89.3512, 119.2116, 101.0520],\n",
            "                        [  9.4260, 106.1665, 107.9734, 144.0200],\n",
            "                        [134.5091,  90.4124, 152.7241, 100.8206],\n",
            "                        [ 43.4657, 105.9954, 114.2389, 128.5451]], device='cuda:0')\n",
            "            masks: tensor([[[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        ...,\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]]], device='cuda:0')\n",
            "            scores: tensor([0.9990, 0.9938, 0.9878, 0.9854, 0.9836, 0.9828, 0.9781, 0.9778, 0.9731,\n",
            "                        0.9605, 0.9476, 0.9366, 0.6861, 0.6647, 0.6537, 0.5689, 0.4984, 0.3680,\n",
            "                        0.3508, 0.2958, 0.2663, 0.2558, 0.2349, 0.2339, 0.2218, 0.2080, 0.2002,\n",
            "                        0.1949, 0.1807, 0.1676, 0.1652, 0.1503, 0.1207, 0.1145, 0.1064, 0.1062,\n",
            "                        0.0810, 0.0799, 0.0776, 0.0751, 0.0731, 0.0699, 0.0686, 0.0655, 0.0637,\n",
            "                        0.0590, 0.0583, 0.0572, 0.0539, 0.0532, 0.0527, 0.0514],\n",
            "                       device='cuda:0')\n",
            "            labels: tensor([13,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2, 13, 56,  2,  2,\n",
            "                         2,  0,  0,  2,  2,  2,  0,  2,  2,  2,  0,  2,  0,  2,  2,  2,  2,  2,\n",
            "                         0,  7,  7,  7,  2,  7,  7, 25,  2,  0,  2,  7,  7,  2,  2,  7],\n",
            "                       device='cuda:0')\n",
            "        ) at 0x7f2e3a16cc10>\n",
            "    gt_instances: <InstanceData(\n",
            "        \n",
            "            META INFORMATION\n",
            "        \n",
            "            DATA FIELDS\n",
            "        ) at 0x7f2e2e6bc4d0>\n",
            "    _gt_instances: <InstanceData(\n",
            "        \n",
            "            META INFORMATION\n",
            "        \n",
            "            DATA FIELDS\n",
            "        ) at 0x7f2e2e6bc4d0>\n",
            "    _ignored_instances: <InstanceData(\n",
            "        \n",
            "            META INFORMATION\n",
            "        \n",
            "            DATA FIELDS\n",
            "        ) at 0x7f2e2e6bcf50>\n",
            ") at 0x7f2e39221d90>\n"
          ]
        }
      ],
      "source": [
        "# Use the detector to do inference\n",
        "image = mmcv.imread('demo/demo.jpg',channel_order='rgb')\n",
        "result = inference_detector(model, image)\n",
        "print(result)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Let's plot the result"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/home/sanbu/anaconda3/envs/mmlab2/lib/python3.7/site-packages/mmengine/visualization/visualizer.py:170: UserWarning: `Visualizer` backend is not initialized because save_dir is None.\n",
            "  warnings.warn('`Visualizer` backend is not initialized '\n"
          ]
        }
      ],
      "source": [
        "from mmdet.registry import VISUALIZERS\n",
        "# init visualizer(run the block only once in jupyter notebook)\n",
        "visualizer = VISUALIZERS.build(model.cfg.visualizer)\n",
        "# the dataset_meta is loaded from the checkpoint and\n",
        "# then pass to the model in init_detector\n",
        "visualizer.dataset_meta = model.dataset_meta"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "image/png": "[encoded data, 909732 characters]",
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ]
          },
          "execution_count": 16,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# show the results\n",
        "visualizer.add_datasample(\n",
        "    'result',\n",
        "    image,\n",
        "    data_sample=result,\n",
        "    draw_gt = None,\n",
        "    wait_time=0,\n",
        ")\n",
        "visualizer.show()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "7GrWIJywLV-V"
      },
      "source": [
        "## Train a Detector on A Customized Dataset\n",
        "\n",
        "To train a new detector, there are usually three things to do:\n",
        "1. Support a new dataset\n",
        "2. Modify the config\n",
        "3. Train a new detector\n",
        "\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "E73y5Lru-wBx"
      },
      "source": [
        "### Support a new dataset\n",
        "\n",
        "There are three ways to support a new dataset in MMDetection: \n",
        "  1. Reorganize the dataset into a COCO format\n",
        "  2. Reorganize the dataset into a middle format\n",
        "  3. Implement a new dataset\n",
        "\n",
        "We recommend the first two methods, as they are usually easier than the third.\n",
        "\n",
        "In this tutorial, we give an example that converts the data into COCO format because MMDetection **only support evaluating mask AP of dataset in COCO format for now**. Other methods and more advanced usages can be found in the [doc](https://mmdetection.readthedocs.io/en/latest/advanced_guides/customize_dataset.html).\n",
        "\n",
        "First, let's download the [the balloon dataset](https://github.com/matterport/Mask_RCNN/tree/master/samples/balloon)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rHnw5Q_nARXq",
        "outputId": "5993532c-3a6f-46d2-e9ad-428cf44dae60"
      },
      "outputs": [],
      "source": [
        "# download and unzip the data\n",
        "!wget -c https://github.com/matterport/Mask_RCNN/releases/download/v2.1/balloon_dataset.zip\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "!unzip balloon_dataset.zip -d ./ballondatasets/"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Check the directory structure of the tiny data\n",
        "\n",
        "# Install tree first in your terminal(linux)\n",
        "sudo apt-get -q install tree\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Wuwxw1oZRtVZ",
        "outputId": "4ee508e8-5acb-450d-c06b-69ceffdc85dd"
      },
      "outputs": [],
      "source": [
        "!tree ballondatasets"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 594
        },
        "id": "YnQQqzOWzE91",
        "outputId": "befa7aae-a21b-42c8-c3ee-5cb4f5bf3e57"
      },
      "outputs": [
        {
          "data": {
            "image/png": "[encoded data, 581596 characters]",
            "text/plain": [
              "<Figure size 1500x1000 with 1 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Let's take a look at the dataset image\n",
        "import mmcv\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "img = mmcv.imread('ballondatasets/balloon/train/10464445726_6f1e3bbe6a_k.jpg')\n",
        "plt.figure(figsize=(15, 10))\n",
        "plt.imshow(mmcv.bgr2rgb(img))\n",
        "plt.show()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "PMZvtSIl71qi"
      },
      "source": [
        "After downloading the data, we need to implement a function to convert the annotation format into the COCO format. Then we can use implemented `COCODataset` to load the data and perform training and evaluation.\n",
        "Let's take a look at the annotation json file.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "n7rwalnPd6e1"
      },
      "outputs": [],
      "source": [
        "# Check the label of a single image\n",
        "import mmengine\n",
        "\n",
        "annotation = mmengine.load('./ballondatasets/balloon/train/via_region_data.json')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "keLW7uqJM54Y",
        "outputId": "d71b98a7-516b-48d4-852d-373f33b881f4"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'fileref': '',\n",
              " 'size': 1115004,\n",
              " 'filename': '34020010494_e5cb88e1c4_k.jpg',\n",
              " 'base64_img_data': '',\n",
              " 'file_attributes': {},\n",
              " 'regions': {'0': {'shape_attributes': {'name': 'polygon',\n",
              "    'all_points_x': [1020,\n",
              "     1000,\n",
              "     994,\n",
              "     1003,\n",
              "     1023,\n",
              "     1050,\n",
              "     1089,\n",
              "     1134,\n",
              "     1190,\n",
              "     1265,\n",
              "     1321,\n",
              "     1361,\n",
              "     1403,\n",
              "     1428,\n",
              "     1442,\n",
              "     1445,\n",
              "     1441,\n",
              "     1427,\n",
              "     1400,\n",
              "     1361,\n",
              "     1316,\n",
              "     1269,\n",
              "     1228,\n",
              "     1198,\n",
              "     1207,\n",
              "     1210,\n",
              "     1190,\n",
              "     1177,\n",
              "     1172,\n",
              "     1174,\n",
              "     1170,\n",
              "     1153,\n",
              "     1127,\n",
              "     1104,\n",
              "     1061,\n",
              "     1032,\n",
              "     1020],\n",
              "    'all_points_y': [963,\n",
              "     899,\n",
              "     841,\n",
              "     787,\n",
              "     738,\n",
              "     700,\n",
              "     663,\n",
              "     638,\n",
              "     621,\n",
              "     619,\n",
              "     643,\n",
              "     672,\n",
              "     720,\n",
              "     765,\n",
              "     800,\n",
              "     860,\n",
              "     896,\n",
              "     942,\n",
              "     990,\n",
              "     1035,\n",
              "     1079,\n",
              "     1112,\n",
              "     1129,\n",
              "     1134,\n",
              "     1144,\n",
              "     1153,\n",
              "     1166,\n",
              "     1166,\n",
              "     1150,\n",
              "     1136,\n",
              "     1129,\n",
              "     1122,\n",
              "     1112,\n",
              "     1084,\n",
              "     1037,\n",
              "     989,\n",
              "     963]},\n",
              "   'region_attributes': {}}}}"
            ]
          },
          "execution_count": 18,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# The annotation is a dict, and its values looks like the following\n",
        "annotation['34020010494_e5cb88e1c4_k.jpg1115004']"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "QA1pFg-FeO3l"
      },
      "source": [
        "According to the above observation, each single image has a corresponding annotation containing keys `filename` and `regions` that are necessary for training.\n",
        "We need to read annotations of each image and convert them into COCO format as below:\n",
        "\n",
        "```python\n",
        "{\n",
        "    \"images\": [image],\n",
        "    \"annotations\": [annotation], \n",
        "    \"categories\": [category]\n",
        "}\n",
        "\n",
        "\n",
        "image = {\n",
        "    \"id\": int,\n",
        "    \"width\": int,\n",
        "    \"height\": int,\n",
        "    \"file_name\": str,\n",
        "}\n",
        "\n",
        "annotation = {\n",
        "    \"id\": int,\n",
        "    \"image_id\": int,\n",
        "    \"category_id\": int,\n",
        "    \"segmentation\": RLE or [polygon],\n",
        "    \"area\": float,\n",
        "    \"bbox\": [x,y,width,height],\n",
        "    \"iscrowd\": 0 or 1,\n",
        "}\n",
        "\n",
        "categories = [{\n",
        "    \"id\": int,\n",
        "    \"name\": str,\n",
        "    \"supercategory\": str,\n",
        "}]\n",
        "```\n",
        "**Note**: We only list the necessary keys for training, as shown above. For a full COCO format, please see [here](https://cocodataset.org/#format-data)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "GdSaB2ad0EdX"
      },
      "outputs": [],
      "source": [
        "import os.path as osp\n",
        "\n",
        "def convert_balloon_to_coco(ann_file, out_file, image_prefix):\n",
        "    data_infos = mmengine.load(ann_file)\n",
        "\n",
        "    annotations = []\n",
        "    images = []\n",
        "    obj_count = 0\n",
        "    for idx, v in enumerate(mmengine.track_iter_progress(data_infos.values())):\n",
        "        filename = v['filename']\n",
        "        img_path = osp.join(image_prefix, filename)\n",
        "        height, width = mmcv.imread(img_path).shape[:2]\n",
        "\n",
        "        images.append(dict(\n",
        "            id=idx,\n",
        "            file_name=filename,\n",
        "            height=height,\n",
        "            width=width))\n",
        "\n",
        "        bboxes = []\n",
        "        labels = []\n",
        "        masks = []\n",
        "        for _, obj in v['regions'].items():\n",
        "            assert not obj['region_attributes']\n",
        "            obj = obj['shape_attributes']\n",
        "            px = obj['all_points_x']\n",
        "            py = obj['all_points_y']\n",
        "            poly = [(x + 0.5, y + 0.5) for x, y in zip(px, py)]\n",
        "            poly = [p for x in poly for p in x]\n",
        "\n",
        "            x_min, y_min, x_max, y_max = (\n",
        "                min(px), min(py), max(px), max(py))\n",
        "\n",
        "\n",
        "            data_anno = dict(\n",
        "                image_id=idx,\n",
        "                id=obj_count,\n",
        "                category_id=0,\n",
        "                bbox=[x_min, y_min, x_max - x_min, y_max - y_min],\n",
        "                area=(x_max - x_min) * (y_max - y_min),\n",
        "                segmentation=[poly],\n",
        "                iscrowd=0)\n",
        "            annotations.append(data_anno)\n",
        "            obj_count += 1\n",
        "\n",
        "    coco_format_json = dict(\n",
        "        images=images,\n",
        "        annotations=annotations,\n",
        "        categories=[{'id':0, 'name': 'balloon'}])\n",
        "    mmengine.dump(coco_format_json, out_file)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "G3xV5ktqlpFu",
        "outputId": "af264997-25d1-4fc1-91bb-f9f1ff2c68c9"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 61/61, 61.7 task/s, elapsed: 1s, ETA:     0s\n",
            "[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 13/13, 59.5 task/s, elapsed: 0s, ETA:     0s\n"
          ]
        }
      ],
      "source": [
        "convert_balloon_to_coco(\n",
        "    './ballondatasets/balloon/train/via_region_data.json',\n",
        "    './ballondatasets/balloon/train/annotation_coco.json',\n",
        "    './ballondatasets/balloon/train/')\n",
        "convert_balloon_to_coco(\n",
        "    './ballondatasets/balloon/val/via_region_data.json',\n",
        "    './ballondatasets/balloon/val/annotation_coco.json',\n",
        "    './ballondatasets/balloon/val/')"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "PwqJOpBe-bMj"
      },
      "source": [
        "### Modify the config\n",
        "\n",
        "In the next step, we need to modify the config for the training.\n",
        "To accelerate the process, we finetune a detector using a pre-trained detector."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "hamZrlnH-YDD"
      },
      "outputs": [],
      "source": [
        "from mmengine import Config\n",
        "cfg = Config.fromfile('./configs/mask_rcnn/mask-rcnn_r50-caffe_fpn_ms-poly-1x_coco.py')"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "HntziLGq-92Z"
      },
      "source": [
        "Given a config that trains a Mask R-CNN on COCO dataset, we need to modify some values to use it for training on the balloon dataset."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pUbwD8uV0PR8",
        "outputId": "0c9ba286-1111-407d-bda4-14d6a262a3e3"
      },
      "outputs": [],
      "source": [
        "from mmengine.runner import set_random_seed\n",
        "\n",
        "# Modify dataset classes and color\n",
        "cfg.metainfo = {\n",
        "    'CLASSES': ('balloon', ),\n",
        "    'PALETTE': [\n",
        "        (220, 20, 60),\n",
        "    ]\n",
        "}\n",
        "\n",
        "# Modify dataset type and path\n",
        "cfg.data_root = './ballondatasets/balloon'\n",
        "\n",
        "cfg.train_dataloader.dataset.ann_file = 'train/annotation_coco.json'\n",
        "cfg.train_dataloader.dataset.data_root = cfg.data_root\n",
        "cfg.train_dataloader.dataset.data_prefix.img = 'train/'\n",
        "cfg.train_dataloader.dataset.metainfo = cfg.metainfo\n",
        "\n",
        "cfg.val_dataloader.dataset.ann_file = 'val/annotation_coco.json'\n",
        "cfg.val_dataloader.dataset.data_root = cfg.data_root\n",
        "cfg.val_dataloader.dataset.data_prefix.img = 'val/'\n",
        "cfg.val_dataloader.dataset.metainfo = cfg.metainfo\n",
        "\n",
        "cfg.test_dataloader = cfg.val_dataloader\n",
        "\n",
        "# Modify metric config\n",
        "cfg.val_evaluator.ann_file = cfg.data_root+'/'+'val/annotation_coco.json'\n",
        "cfg.test_evaluator = cfg.val_evaluator\n",
        "\n",
        "# Modify num classes of the model in box head and mask head\n",
        "cfg.model.roi_head.bbox_head.num_classes = 1\n",
        "cfg.model.roi_head.mask_head.num_classes = 1\n",
        "\n",
        "# We can still the pre-trained Mask RCNN model to obtain a higher performance\n",
        "cfg.load_from = 'checkpoints/mask_rcnn_r50_caffe_fpn_mstrain-poly_3x_coco_bbox_mAP-0.408__segm_mAP-0.37_20200504_163245-42aa3d00.pth'\n",
        "\n",
        "# Set up working dir to save files and logs.\n",
        "cfg.work_dir = './tutorial_exps'\n",
        "\n",
        "\n",
        "# We can set the evaluation interval to reduce the evaluation times\n",
        "cfg.train_cfg.val_interval = 3\n",
        "# We can set the checkpoint saving interval to reduce the storage cost\n",
        "cfg.default_hooks.checkpoint.interval = 3\n",
        "\n",
        "# The original learning rate (LR) is set for 8-GPU training.\n",
        "# We divide it by 8 since we only use one GPU.\n",
        "cfg.optim_wrapper.optimizer.lr = 0.02 / 8\n",
        "cfg.default_hooks.logger.interval = 10\n",
        "\n",
        "\n",
        "# Set seed thus the results are more reproducible\n",
        "# cfg.seed = 0\n",
        "set_random_seed(0, deterministic=False)\n",
        "\n",
        "# We can also use tensorboard to log the training process\n",
        "cfg.visualizer.vis_backends.append({\"type\":'TensorboardVisBackend'})"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "111W_oZV_3wa"
      },
      "source": [
        "### Train a new detector\n",
        "\n",
        "Finally, lets initialize the dataset and detector, then train a new detector!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "11/23 10:45:03 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
            "------------------------------------------------------------\n",
            "System environment:\n",
            "    sys.platform: linux\n",
            "    Python: 3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 06:08:21) [GCC 9.4.0]\n",
            "    CUDA available: True\n",
            "    numpy_random_seed: 209652396\n",
            "    GPU 0: NVIDIA GeForce RTX 3080\n",
            "    CUDA_HOME: /usr/local/cuda-11.5\n",
            "    NVCC: Cuda compilation tools, release 11.5, V11.5.50\n",
            "    GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0\n",
            "    PyTorch: 1.12.1+cu113\n",
            "    PyTorch compiling details: PyTorch built with:\n",
            "  - GCC 9.3\n",
            "  - C++ Version: 201402\n",
            "  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications\n",
            "  - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)\n",
            "  - OpenMP 201511 (a.k.a. OpenMP 4.5)\n",
            "  - LAPACK is enabled (usually provided by MKL)\n",
            "  - NNPACK is enabled\n",
            "  - CPU capability usage: AVX2\n",
            "  - CUDA Runtime 11.3\n",
            "  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86\n",
            "  - CuDNN 8.3.2  (built against CUDA 11.5)\n",
            "  - Magma 2.5.2\n",
            "  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.3.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, \n",
            "\n",
            "    TorchVision: 0.13.1+cu113\n",
            "    OpenCV: 4.6.0\n",
            "    MMEngine: 0.3.0\n",
            "\n",
            "Runtime environment:\n",
            "    cudnn_benchmark: False\n",
            "    mp_cfg: {'mp_start_method': 'fork', 'opencv_num_threads': 0}\n",
            "    dist_cfg: {'backend': 'nccl'}\n",
            "    seed: None\n",
            "    Distributed launcher: none\n",
            "    Distributed training: False\n",
            "    GPU number: 1\n",
            "------------------------------------------------------------\n",
            "\n",
            "11/23 10:45:04 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Config:\n",
            "model = dict(\n",
            "    type='MaskRCNN',\n",
            "    data_preprocessor=dict(\n",
            "        type='DetDataPreprocessor',\n",
            "        mean=[103.53, 116.28, 123.675],\n",
            "        std=[1.0, 1.0, 1.0],\n",
            "        bgr_to_rgb=False,\n",
            "        pad_mask=True,\n",
            "        pad_size_divisor=32),\n",
            "    backbone=dict(\n",
            "        type='ResNet',\n",
            "        depth=50,\n",
            "        num_stages=4,\n",
            "        out_indices=(0, 1, 2, 3),\n",
            "        frozen_stages=1,\n",
            "        norm_cfg=dict(type='BN', requires_grad=False),\n",
            "        norm_eval=True,\n",
            "        style='caffe',\n",
            "        init_cfg=dict(\n",
            "            type='Pretrained',\n",
            "            checkpoint='open-mmlab://detectron2/resnet50_caffe')),\n",
            "    neck=dict(\n",
            "        type='FPN',\n",
            "        in_channels=[256, 512, 1024, 2048],\n",
            "        out_channels=256,\n",
            "        num_outs=5),\n",
            "    rpn_head=dict(\n",
            "        type='RPNHead',\n",
            "        in_channels=256,\n",
            "        feat_channels=256,\n",
            "        anchor_generator=dict(\n",
            "            type='AnchorGenerator',\n",
            "            scales=[8],\n",
            "            ratios=[0.5, 1.0, 2.0],\n",
            "            strides=[4, 8, 16, 32, 64]),\n",
            "        bbox_coder=dict(\n",
            "            type='DeltaXYWHBBoxCoder',\n",
            "            target_means=[0.0, 0.0, 0.0, 0.0],\n",
            "            target_stds=[1.0, 1.0, 1.0, 1.0]),\n",
            "        loss_cls=dict(\n",
            "            type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),\n",
            "        loss_bbox=dict(type='L1Loss', loss_weight=1.0)),\n",
            "    roi_head=dict(\n",
            "        type='StandardRoIHead',\n",
            "        bbox_roi_extractor=dict(\n",
            "            type='SingleRoIExtractor',\n",
            "            roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0),\n",
            "            out_channels=256,\n",
            "            featmap_strides=[4, 8, 16, 32]),\n",
            "        bbox_head=dict(\n",
            "            type='Shared2FCBBoxHead',\n",
            "            in_channels=256,\n",
            "            fc_out_channels=1024,\n",
            "            roi_feat_size=7,\n",
            "            num_classes=1,\n",
            "            bbox_coder=dict(\n",
            "                type='DeltaXYWHBBoxCoder',\n",
            "                target_means=[0.0, 0.0, 0.0, 0.0],\n",
            "                target_stds=[0.1, 0.1, 0.2, 0.2]),\n",
            "            reg_class_agnostic=False,\n",
            "            loss_cls=dict(\n",
            "                type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),\n",
            "            loss_bbox=dict(type='L1Loss', loss_weight=1.0)),\n",
            "        mask_roi_extractor=dict(\n",
            "            type='SingleRoIExtractor',\n",
            "            roi_layer=dict(type='RoIAlign', output_size=14, sampling_ratio=0),\n",
            "            out_channels=256,\n",
            "            featmap_strides=[4, 8, 16, 32]),\n",
            "        mask_head=dict(\n",
            "            type='FCNMaskHead',\n",
            "            num_convs=4,\n",
            "            in_channels=256,\n",
            "            conv_out_channels=256,\n",
            "            num_classes=1,\n",
            "            loss_mask=dict(\n",
            "                type='CrossEntropyLoss', use_mask=True, loss_weight=1.0))),\n",
            "    train_cfg=dict(\n",
            "        rpn=dict(\n",
            "            assigner=dict(\n",
            "                type='MaxIoUAssigner',\n",
            "                pos_iou_thr=0.7,\n",
            "                neg_iou_thr=0.3,\n",
            "                min_pos_iou=0.3,\n",
            "                match_low_quality=True,\n",
            "                ignore_iof_thr=-1),\n",
            "            sampler=dict(\n",
            "                type='RandomSampler',\n",
            "                num=256,\n",
            "                pos_fraction=0.5,\n",
            "                neg_pos_ub=-1,\n",
            "                add_gt_as_proposals=False),\n",
            "            allowed_border=-1,\n",
            "            pos_weight=-1,\n",
            "            debug=False),\n",
            "        rpn_proposal=dict(\n",
            "            nms_pre=2000,\n",
            "            max_per_img=1000,\n",
            "            nms=dict(type='nms', iou_threshold=0.7),\n",
            "            min_bbox_size=0),\n",
            "        rcnn=dict(\n",
            "            assigner=dict(\n",
            "                type='MaxIoUAssigner',\n",
            "                pos_iou_thr=0.5,\n",
            "                neg_iou_thr=0.5,\n",
            "                min_pos_iou=0.5,\n",
            "                match_low_quality=True,\n",
            "                ignore_iof_thr=-1),\n",
            "            sampler=dict(\n",
            "                type='RandomSampler',\n",
            "                num=512,\n",
            "                pos_fraction=0.25,\n",
            "                neg_pos_ub=-1,\n",
            "                add_gt_as_proposals=True),\n",
            "            mask_size=28,\n",
            "            pos_weight=-1,\n",
            "            debug=False)),\n",
            "    test_cfg=dict(\n",
            "        rpn=dict(\n",
            "            nms_pre=1000,\n",
            "            max_per_img=1000,\n",
            "            nms=dict(type='nms', iou_threshold=0.7),\n",
            "            min_bbox_size=0),\n",
            "        rcnn=dict(\n",
            "            score_thr=0.05,\n",
            "            nms=dict(type='nms', iou_threshold=0.5),\n",
            "            max_per_img=100,\n",
            "            mask_thr_binary=0.5)))\n",
            "dataset_type = 'CocoDataset'\n",
            "data_root = './ballondatasets/balloon'\n",
            "file_client_args = dict(backend='disk')\n",
            "train_pipeline = [\n",
            "    dict(type='LoadImageFromFile', file_client_args=dict(backend='disk')),\n",
            "    dict(\n",
            "        type='LoadAnnotations',\n",
            "        with_bbox=True,\n",
            "        with_mask=True,\n",
            "        poly2mask=False),\n",
            "    dict(\n",
            "        type='RandomChoiceResize',\n",
            "        scales=[(1333, 640), (1333, 672), (1333, 704), (1333, 736),\n",
            "                (1333, 768), (1333, 800)],\n",
            "        keep_ratio=True),\n",
            "    dict(type='RandomFlip', prob=0.5),\n",
            "    dict(type='PackDetInputs')\n",
            "]\n",
            "test_pipeline = [\n",
            "    dict(type='LoadImageFromFile', file_client_args=dict(backend='disk')),\n",
            "    dict(type='Resize', scale=(1333, 800), keep_ratio=True),\n",
            "    dict(type='LoadAnnotations', with_bbox=True, with_mask=True),\n",
            "    dict(\n",
            "        type='PackDetInputs',\n",
            "        meta_keys=('img_id', 'img_path', 'ori_shape', 'img_shape',\n",
            "                   'scale_factor'))\n",
            "]\n",
            "train_dataloader = dict(\n",
            "    batch_size=2,\n",
            "    num_workers=2,\n",
            "    persistent_workers=True,\n",
            "    sampler=dict(type='DefaultSampler', shuffle=True),\n",
            "    batch_sampler=dict(type='AspectRatioBatchSampler'),\n",
            "    dataset=dict(\n",
            "        type='CocoDataset',\n",
            "        data_root='./ballondatasets/balloon',\n",
            "        ann_file='train/annotation_coco.json',\n",
            "        data_prefix=dict(img='train/'),\n",
            "        filter_cfg=dict(filter_empty_gt=True, min_size=32),\n",
            "        pipeline=[\n",
            "            dict(\n",
            "                type='LoadImageFromFile',\n",
            "                file_client_args=dict(backend='disk')),\n",
            "            dict(\n",
            "                type='LoadAnnotations',\n",
            "                with_bbox=True,\n",
            "                with_mask=True,\n",
            "                poly2mask=False),\n",
            "            dict(\n",
            "                type='RandomChoiceResize',\n",
            "                scales=[(1333, 640), (1333, 672), (1333, 704), (1333, 736),\n",
            "                        (1333, 768), (1333, 800)],\n",
            "                keep_ratio=True),\n",
            "            dict(type='RandomFlip', prob=0.5),\n",
            "            dict(type='PackDetInputs')\n",
            "        ],\n",
            "        metainfo=dict(CLASSES=('balloon', ), PALETTE=[(220, 20, 60)])))\n",
            "val_dataloader = dict(\n",
            "    batch_size=1,\n",
            "    num_workers=2,\n",
            "    persistent_workers=True,\n",
            "    drop_last=False,\n",
            "    sampler=dict(type='DefaultSampler', shuffle=False),\n",
            "    dataset=dict(\n",
            "        type='CocoDataset',\n",
            "        data_root='./ballondatasets/balloon',\n",
            "        ann_file='val/annotation_coco.json',\n",
            "        data_prefix=dict(img='val/'),\n",
            "        test_mode=True,\n",
            "        pipeline=[\n",
            "            dict(\n",
            "                type='LoadImageFromFile',\n",
            "                file_client_args=dict(backend='disk')),\n",
            "            dict(type='Resize', scale=(1333, 800), keep_ratio=True),\n",
            "            dict(type='LoadAnnotations', with_bbox=True, with_mask=True),\n",
            "            dict(\n",
            "                type='PackDetInputs',\n",
            "                meta_keys=('img_id', 'img_path', 'ori_shape', 'img_shape',\n",
            "                           'scale_factor'))\n",
            "        ],\n",
            "        metainfo=dict(CLASSES=('balloon', ), PALETTE=[(220, 20, 60)])))\n",
            "test_dataloader = dict(\n",
            "    batch_size=1,\n",
            "    num_workers=2,\n",
            "    persistent_workers=True,\n",
            "    drop_last=False,\n",
            "    sampler=dict(type='DefaultSampler', shuffle=False),\n",
            "    dataset=dict(\n",
            "        type='CocoDataset',\n",
            "        data_root='./ballondatasets/balloon',\n",
            "        ann_file='val/annotation_coco.json',\n",
            "        data_prefix=dict(img='val/'),\n",
            "        test_mode=True,\n",
            "        pipeline=[\n",
            "            dict(\n",
            "                type='LoadImageFromFile',\n",
            "                file_client_args=dict(backend='disk')),\n",
            "            dict(type='Resize', scale=(1333, 800), keep_ratio=True),\n",
            "            dict(type='LoadAnnotations', with_bbox=True, with_mask=True),\n",
            "            dict(\n",
            "                type='PackDetInputs',\n",
            "                meta_keys=('img_id', 'img_path', 'ori_shape', 'img_shape',\n",
            "                           'scale_factor'))\n",
            "        ],\n",
            "        metainfo=dict(CLASSES=('balloon', ), PALETTE=[(220, 20, 60)])))\n",
            "val_evaluator = dict(\n",
            "    type='CocoMetric',\n",
            "    ann_file='./ballondatasets/balloon/val/annotation_coco.json',\n",
            "    metric=['bbox', 'segm'],\n",
            "    format_only=False)\n",
            "test_evaluator = dict(\n",
            "    type='CocoMetric',\n",
            "    ann_file='./ballondatasets/balloon/val/annotation_coco.json',\n",
            "    metric=['bbox', 'segm'],\n",
            "    format_only=False)\n",
            "train_cfg = dict(type='EpochBasedTrainLoop', max_epochs=12, val_interval=3)\n",
            "val_cfg = dict(type='ValLoop')\n",
            "test_cfg = dict(type='TestLoop')\n",
            "param_scheduler = [\n",
            "    dict(\n",
            "        type='LinearLR', start_factor=0.001, by_epoch=False, begin=0, end=500),\n",
            "    dict(\n",
            "        type='MultiStepLR',\n",
            "        begin=0,\n",
            "        end=12,\n",
            "        by_epoch=True,\n",
            "        milestones=[8, 11],\n",
            "        gamma=0.1)\n",
            "]\n",
            "optim_wrapper = dict(\n",
            "    type='OptimWrapper',\n",
            "    optimizer=dict(type='SGD', lr=0.0025, momentum=0.9, weight_decay=0.0001))\n",
            "auto_scale_lr = dict(enable=False, base_batch_size=16)\n",
            "default_scope = 'mmdet'\n",
            "default_hooks = dict(\n",
            "    timer=dict(type='IterTimerHook'),\n",
            "    logger=dict(type='LoggerHook', interval=10),\n",
            "    param_scheduler=dict(type='ParamSchedulerHook'),\n",
            "    checkpoint=dict(type='CheckpointHook', interval=3),\n",
            "    sampler_seed=dict(type='DistSamplerSeedHook'),\n",
            "    visualization=dict(type='DetVisualizationHook'))\n",
            "env_cfg = dict(\n",
            "    cudnn_benchmark=False,\n",
            "    mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0),\n",
            "    dist_cfg=dict(backend='nccl'))\n",
            "vis_backends = [dict(type='LocalVisBackend')]\n",
            "visualizer = dict(\n",
            "    type='DetLocalVisualizer',\n",
            "    vis_backends=[\n",
            "        dict(type='LocalVisBackend'),\n",
            "        dict(type='TensorboardVisBackend')\n",
            "    ],\n",
            "    name='visualizer')\n",
            "log_processor = dict(type='LogProcessor', window_size=50, by_epoch=True)\n",
            "log_level = 'INFO'\n",
            "load_from = 'checkpoints/mask_rcnn_r50_caffe_fpn_mstrain-poly_3x_coco_bbox_mAP-0.408__segm_mAP-0.37_20200504_163245-42aa3d00.pth'\n",
            "resume = False\n",
            "metainfo = dict(CLASSES=('balloon', ), PALETTE=[(220, 20, 60)])\n",
            "work_dir = './tutorial_exps'\n",
            "\n",
            "Result has been saved to /home/sanbu/mmdetection3.x/mmdetection3/tutorial_exps/modules_statistic_results.json\n",
            "11/23 10:45:06 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Distributed training is not used, all SyncBatchNorm (SyncBN) layers in the model will be automatically reverted to BatchNormXd layers if they are used.\n"
          ]
        }
      ],
      "source": [
        "from mmdet.datasets import build_dataset\n",
        "from mmdet.models import build_detector\n",
        "from mmengine.runner import Runner\n",
        "\n",
        "# build the runner from config\n",
        "runner = Runner.from_cfg(cfg)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "loading annotations into memory...\n",
            "Done (t=0.01s)\n",
            "creating index...\n",
            "index created!\n",
            "loading annotations into memory...\n",
            "Done (t=0.00s)\n",
            "creating index...\n",
            "index created!\n",
            "loading annotations into memory...\n",
            "Done (t=0.00s)\n",
            "creating index...\n",
            "index created!\n",
            "11/04 09:53:22 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - load model from: open-mmlab://detectron2/resnet50_caffe\n",
            "11/04 09:53:22 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - openmmlab loads checkpoint from path: open-mmlab://detectron2/resnet50_caffe\n",
            "11/04 09:53:22 - mmengine - \u001b[5m\u001b[4m\u001b[33mWARNING\u001b[0m - The model and loaded state dict do not match exactly\n",
            "\n",
            "unexpected key in source state_dict: conv1.bias\n",
            "\n",
            "local loads checkpoint from path: checkpoints/mask_rcnn_r50_caffe_fpn_mstrain-poly_3x_coco_bbox_mAP-0.408__segm_mAP-0.37_20200504_163245-42aa3d00.pth\n",
            "The model and loaded state dict do not match exactly\n",
            "\n",
            "size mismatch for roi_head.bbox_head.fc_cls.weight: copying a param with shape torch.Size([81, 1024]) from checkpoint, the shape in current model is torch.Size([2, 1024]).\n",
            "size mismatch for roi_head.bbox_head.fc_cls.bias: copying a param with shape torch.Size([81]) from checkpoint, the shape in current model is torch.Size([2]).\n",
            "size mismatch for roi_head.bbox_head.fc_reg.weight: copying a param with shape torch.Size([320, 1024]) from checkpoint, the shape in current model is torch.Size([4, 1024]).\n",
            "size mismatch for roi_head.bbox_head.fc_reg.bias: copying a param with shape torch.Size([320]) from checkpoint, the shape in current model is torch.Size([4]).\n",
            "size mismatch for roi_head.mask_head.conv_logits.weight: copying a param with shape torch.Size([80, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([1, 256, 1, 1]).\n",
            "size mismatch for roi_head.mask_head.conv_logits.bias: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([1]).\n",
            "11/04 09:53:23 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Load checkpoint from checkpoints/mask_rcnn_r50_caffe_fpn_mstrain-poly_3x_coco_bbox_mAP-0.408__segm_mAP-0.37_20200504_163245-42aa3d00.pth\n",
            "11/04 09:53:23 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Checkpoints will be saved to /home/sanbu/mmdetection3.x/mmdetection/tutorial_exps.\n",
            "11/04 09:53:26 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][10/31]  lr: 4.7545e-05  memory: 3365  data_time: 0.0250  loss: 14.0508  loss_rpn_cls: 0.1337  loss_rpn_bbox: 0.0135  loss_cls: 0.8340  acc: 29.5898  loss_bbox: 0.2582  loss_mask: 12.8114  time: 0.3857\n",
            "11/04 09:53:29 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][20/31]  lr: 9.7595e-05  memory: 3365  data_time: 0.0171  loss: 9.3286  loss_rpn_cls: 0.1047  loss_rpn_bbox: 0.0160  loss_cls: 0.7288  acc: 82.9102  loss_bbox: 0.2851  loss_mask: 8.1940  time: 0.2973\n",
            "11/04 09:53:30 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][30/31]  lr: 1.4765e-04  memory: 3367  data_time: 0.0125  loss: 6.6740  loss_rpn_cls: 0.1221  loss_rpn_bbox: 0.0173  loss_cls: 0.6059  acc: 94.7266  loss_bbox: 0.2559  loss_mask: 5.6728  time: 0.2623\n",
            "11/04 09:53:31 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Exp name: mask-rcnn_r50-caffe_fpn_ms-poly-1x_coco_20221104_095316\n",
            "11/04 09:53:33 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][10/31]  lr: 2.0270e-04  memory: 3365  data_time: 0.0106  loss: 5.2092  loss_rpn_cls: 0.1266  loss_rpn_bbox: 0.0210  loss_cls: 0.5266  acc: 95.7031  loss_bbox: 0.2635  loss_mask: 4.2715  time: 0.2441\n",
            "11/04 09:53:35 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][20/31]  lr: 2.5275e-04  memory: 3365  data_time: 0.0052  loss: 4.1211  loss_rpn_cls: 0.1078  loss_rpn_bbox: 0.0194  loss_cls: 0.4638  acc: 95.8984  loss_bbox: 0.2648  loss_mask: 3.2653  time: 0.1960\n",
            "11/04 09:53:36 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][30/31]  lr: 3.0280e-04  memory: 3183  data_time: 0.0037  loss: 1.5987  loss_rpn_cls: 0.0902  loss_rpn_bbox: 0.0195  loss_cls: 0.3427  acc: 84.9609  loss_bbox: 0.2771  loss_mask: 0.8692  time: 0.1959\n",
            "11/04 09:53:37 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Exp name: mask-rcnn_r50-caffe_fpn_ms-poly-1x_coco_20221104_095316\n",
            "11/04 09:53:39 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [3][10/31]  lr: 3.5786e-04  memory: 3143  data_time: 0.0040  loss: 0.9650  loss_rpn_cls: 0.0816  loss_rpn_bbox: 0.0182  loss_cls: 0.2597  acc: 95.8984  loss_bbox: 0.2766  loss_mask: 0.3288  time: 0.1931\n",
            "11/04 09:53:41 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [3][20/31]  lr: 4.0791e-04  memory: 3365  data_time: 0.0035  loss: 0.8676  loss_rpn_cls: 0.0514  loss_rpn_bbox: 0.0153  loss_cls: 0.2326  acc: 86.5234  loss_bbox: 0.3119  loss_mask: 0.2563  time: 0.1960\n",
            "11/04 09:53:43 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [3][30/31]  lr: 4.5796e-04  memory: 3365  data_time: 0.0035  loss: 0.7866  loss_rpn_cls: 0.0340  loss_rpn_bbox: 0.0134  loss_cls: 0.2125  acc: 80.8594  loss_bbox: 0.3319  loss_mask: 0.1948  time: 0.1969\n",
            "11/04 09:53:43 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Exp name: mask-rcnn_r50-caffe_fpn_ms-poly-1x_coco_20221104_095316\n",
            "11/04 09:53:43 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Saving checkpoint at 3 epochs\n",
            "11/04 09:53:48 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [3][10/13]    memory: 3932  \n",
            "11/04 09:53:49 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Evaluating bbox...\n",
            "Loading and preparing results...\n",
            "DONE (t=0.00s)\n",
            "creating index...\n",
            "index created!\n",
            "Running per image evaluation...\n",
            "Evaluate annotation type *bbox*\n",
            "DONE (t=0.06s).\n",
            "Accumulating evaluation results...\n",
            "DONE (t=0.01s).\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.547\n",
            " Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.735\n",
            " Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.682\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.011\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.496\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.602\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.674\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.674\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.674\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.250\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.625\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.714\n",
            "11/04 09:53:49 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - bbox_mAP_copypaste: 0.547 0.735 0.682 0.011 0.496 0.602\n",
            "11/04 09:53:49 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Evaluating segm...\n",
            "Loading and preparing results...\n",
            "DONE (t=0.01s)\n",
            "creating index...\n",
            "index created!\n",
            "Running per image evaluation...\n",
            "Evaluate annotation type *segm*\n",
            "DONE (t=0.07s).\n",
            "Accumulating evaluation results...\n",
            "DONE (t=0.01s).\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.631\n",
            " Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.734\n",
            " Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.730\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.022\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.563\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.691\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.792\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.792\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.792\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.600\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.750\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.817\n",
            "11/04 09:53:49 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - segm_mAP_copypaste: 0.631 0.734 0.730 0.022 0.563 0.691\n",
            "11/04 09:53:49 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [3][13/13]  coco/bbox_mAP: 0.5470  coco/bbox_mAP_50: 0.7350  coco/bbox_mAP_75: 0.6820  coco/bbox_mAP_s: 0.0110  coco/bbox_mAP_m: 0.4960  coco/bbox_mAP_l: 0.6020  coco/segm_mAP: 0.6310  coco/segm_mAP_50: 0.7340  coco/segm_mAP_75: 0.7300  coco/segm_mAP_s: 0.0220  coco/segm_mAP_m: 0.5630  coco/segm_mAP_l: 0.6910\n",
            "11/04 09:53:51 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [4][10/31]  lr: 5.1301e-04  memory: 3365  data_time: 0.0040  loss: 0.7791  loss_rpn_cls: 0.0324  loss_rpn_bbox: 0.0134  loss_cls: 0.2024  acc: 97.2656  loss_bbox: 0.3607  loss_mask: 0.1702  time: 0.1965\n",
            "11/04 09:53:53 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [4][20/31]  lr: 5.6306e-04  memory: 3345  data_time: 0.0036  loss: 0.7585  loss_rpn_cls: 0.0305  loss_rpn_bbox: 0.0136  loss_cls: 0.1872  acc: 99.2188  loss_bbox: 0.3721  loss_mask: 0.1552  time: 0.2002\n",
            "11/04 09:53:55 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [4][30/31]  lr: 6.1311e-04  memory: 3367  data_time: 0.0037  loss: 0.7116  loss_rpn_cls: 0.0260  loss_rpn_bbox: 0.0127  loss_cls: 0.1689  acc: 93.4570  loss_bbox: 0.3631  loss_mask: 0.1409  time: 0.2001\n",
            "11/04 09:53:55 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Exp name: mask-rcnn_r50-caffe_fpn_ms-poly-1x_coco_20221104_095316\n",
            "11/04 09:53:57 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [5][10/31]  lr: 6.6817e-04  memory: 3365  data_time: 0.0045  loss: 0.6867  loss_rpn_cls: 0.0231  loss_rpn_bbox: 0.0143  loss_cls: 0.1581  acc: 93.7500  loss_bbox: 0.3642  loss_mask: 0.1270  time: 0.1991\n",
            "11/04 09:53:59 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [5][20/31]  lr: 7.1822e-04  memory: 3541  data_time: 0.0041  loss: 0.5979  loss_rpn_cls: 0.0193  loss_rpn_bbox: 0.0132  loss_cls: 0.1305  acc: 99.0234  loss_bbox: 0.3157  loss_mask: 0.1193  time: 0.1998\n",
            "11/04 09:54:01 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [5][30/31]  lr: 7.6827e-04  memory: 3366  data_time: 0.0040  loss: 0.5307  loss_rpn_cls: 0.0172  loss_rpn_bbox: 0.0129  loss_cls: 0.1162  acc: 94.6289  loss_bbox: 0.2745  loss_mask: 0.1099  time: 0.2006\n",
            "11/04 09:54:01 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Exp name: mask-rcnn_r50-caffe_fpn_ms-poly-1x_coco_20221104_095316\n",
            "11/04 09:54:04 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [6][10/31]  lr: 8.2332e-04  memory: 3541  data_time: 0.0045  loss: 0.4656  loss_rpn_cls: 0.0151  loss_rpn_bbox: 0.0122  loss_cls: 0.1042  acc: 98.4375  loss_bbox: 0.2295  loss_mask: 0.1046  time: 0.2012\n",
            "11/04 09:54:05 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [6][20/31]  lr: 8.7337e-04  memory: 3365  data_time: 0.0035  loss: 0.3887  loss_rpn_cls: 0.0132  loss_rpn_bbox: 0.0101  loss_cls: 0.0865  acc: 98.2422  loss_bbox: 0.1784  loss_mask: 0.1004  time: 0.1981\n",
            "11/04 09:54:07 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [6][30/31]  lr: 9.2342e-04  memory: 3524  data_time: 0.0035  loss: 0.3397  loss_rpn_cls: 0.0126  loss_rpn_bbox: 0.0094  loss_cls: 0.0800  acc: 94.8242  loss_bbox: 0.1421  loss_mask: 0.0957  time: 0.1982\n",
            "11/04 09:54:08 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Exp name: mask-rcnn_r50-caffe_fpn_ms-poly-1x_coco_20221104_095316\n",
            "11/04 09:54:08 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Saving checkpoint at 6 epochs\n",
            "11/04 09:54:10 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [6][10/13]    memory: 2361  \n",
            "11/04 09:54:10 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Evaluating bbox...\n",
            "Loading and preparing results...\n",
            "DONE (t=0.00s)\n",
            "creating index...\n",
            "index created!\n",
            "Running per image evaluation...\n",
            "Evaluate annotation type *bbox*\n",
            "DONE (t=0.02s).\n",
            "Accumulating evaluation results...\n",
            "DONE (t=0.01s).\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.743\n",
            " Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.877\n",
            " Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.837\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.067\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.672\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.797\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.786\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.786\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.786\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.200\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.742\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.833\n",
            "11/04 09:54:10 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - bbox_mAP_copypaste: 0.743 0.877 0.837 0.067 0.672 0.797\n",
            "11/04 09:54:10 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Evaluating segm...\n",
            "Loading and preparing results...\n",
            "DONE (t=0.00s)\n",
            "creating index...\n",
            "index created!\n",
            "Running per image evaluation...\n",
            "Evaluate annotation type *segm*\n",
            "DONE (t=0.02s).\n",
            "Accumulating evaluation results...\n",
            "DONE (t=0.01s).\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.788\n",
            " Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.858\n",
            " Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.858\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.050\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.673\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.846\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.828\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.828\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.828\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.350\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.783\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.869\n",
            "11/04 09:54:10 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - segm_mAP_copypaste: 0.788 0.858 0.858 0.050 0.673 0.846\n",
            "11/04 09:54:10 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [6][13/13]  coco/bbox_mAP: 0.7430  coco/bbox_mAP_50: 0.8770  coco/bbox_mAP_75: 0.8370  coco/bbox_mAP_s: 0.0670  coco/bbox_mAP_m: 0.6720  coco/bbox_mAP_l: 0.7970  coco/segm_mAP: 0.7880  coco/segm_mAP_50: 0.8580  coco/segm_mAP_75: 0.8580  coco/segm_mAP_s: 0.0500  coco/segm_mAP_m: 0.6730  coco/segm_mAP_l: 0.8460\n",
            "11/04 09:54:12 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [7][10/31]  lr: 9.7848e-04  memory: 3367  data_time: 0.0042  loss: 0.3144  loss_rpn_cls: 0.0118  loss_rpn_bbox: 0.0090  loss_cls: 0.0772  acc: 98.8281  loss_bbox: 0.1217  loss_mask: 0.0947  time: 0.1963\n",
            "11/04 09:54:14 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [7][20/31]  lr: 1.0285e-03  memory: 3366  data_time: 0.0037  loss: 0.3031  loss_rpn_cls: 0.0093  loss_rpn_bbox: 0.0091  loss_cls: 0.0761  acc: 98.9258  loss_bbox: 0.1127  loss_mask: 0.0958  time: 0.1968\n",
            "11/04 09:54:16 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [7][30/31]  lr: 1.0786e-03  memory: 3367  data_time: 0.0037  loss: 0.2942  loss_rpn_cls: 0.0092  loss_rpn_bbox: 0.0095  loss_cls: 0.0720  acc: 98.9258  loss_bbox: 0.1051  loss_mask: 0.0984  time: 0.1959\n",
            "11/04 09:54:17 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Exp name: mask-rcnn_r50-caffe_fpn_ms-poly-1x_coco_20221104_095316\n",
            "11/04 09:54:19 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [8][10/31]  lr: 1.1336e-03  memory: 3365  data_time: 0.0050  loss: 0.2925  loss_rpn_cls: 0.0090  loss_rpn_bbox: 0.0100  loss_cls: 0.0724  acc: 99.0234  loss_bbox: 0.1047  loss_mask: 0.0963  time: 0.1991\n",
            "11/04 09:54:21 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [8][20/31]  lr: 1.1837e-03  memory: 3365  data_time: 0.0044  loss: 0.2771  loss_rpn_cls: 0.0078  loss_rpn_bbox: 0.0095  loss_cls: 0.0688  acc: 97.9492  loss_bbox: 0.0963  loss_mask: 0.0946  time: 0.2001\n",
            "11/04 09:54:23 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [8][30/31]  lr: 1.2337e-03  memory: 3182  data_time: 0.0044  loss: 0.2633  loss_rpn_cls: 0.0071  loss_rpn_bbox: 0.0097  loss_cls: 0.0650  acc: 98.2422  loss_bbox: 0.0919  loss_mask: 0.0896  time: 0.2004\n",
            "11/04 09:54:23 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Exp name: mask-rcnn_r50-caffe_fpn_ms-poly-1x_coco_20221104_095316\n",
            "11/04 09:54:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [9][10/31]  lr: 1.2888e-04  memory: 3367  data_time: 0.0055  loss: 0.2546  loss_rpn_cls: 0.0069  loss_rpn_bbox: 0.0095  loss_cls: 0.0635  acc: 97.0703  loss_bbox: 0.0876  loss_mask: 0.0872  time: 0.2001\n",
            "11/04 09:54:27 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [9][20/31]  lr: 1.3388e-04  memory: 3345  data_time: 0.0041  loss: 0.2377  loss_rpn_cls: 0.0055  loss_rpn_bbox: 0.0090  loss_cls: 0.0601  acc: 97.7539  loss_bbox: 0.0824  loss_mask: 0.0808  time: 0.1994\n",
            "11/04 09:54:29 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [9][30/31]  lr: 1.3889e-04  memory: 3423  data_time: 0.0042  loss: 0.2358  loss_rpn_cls: 0.0062  loss_rpn_bbox: 0.0097  loss_cls: 0.0603  acc: 99.1211  loss_bbox: 0.0810  loss_mask: 0.0787  time: 0.2002\n",
            "11/04 09:54:29 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Exp name: mask-rcnn_r50-caffe_fpn_ms-poly-1x_coco_20221104_095316\n",
            "11/04 09:54:29 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Saving checkpoint at 9 epochs\n",
            "11/04 09:54:31 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [9][10/13]    memory: 1681  \n",
            "11/04 09:54:31 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Evaluating bbox...\n",
            "Loading and preparing results...\n",
            "DONE (t=0.00s)\n",
            "creating index...\n",
            "index created!\n",
            "Running per image evaluation...\n",
            "Evaluate annotation type *bbox*\n",
            "DONE (t=0.01s).\n",
            "Accumulating evaluation results...\n",
            "DONE (t=0.01s).\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.764\n",
            " Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.879\n",
            " Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.845\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.202\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.645\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.821\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.808\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.808\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.808\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.400\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.750\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.850\n",
            "11/04 09:54:31 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - bbox_mAP_copypaste: 0.764 0.879 0.845 0.202 0.645 0.821\n",
            "11/04 09:54:31 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Evaluating segm...\n",
            "Loading and preparing results...\n",
            "DONE (t=0.00s)\n",
            "creating index...\n",
            "index created!\n",
            "Running per image evaluation...\n",
            "Evaluate annotation type *segm*\n",
            "DONE (t=0.02s).\n",
            "Accumulating evaluation results...\n",
            "DONE (t=0.01s).\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.795\n",
            " Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.861\n",
            " Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.861\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.118\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.664\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.852\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.834\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.834\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.834\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.350\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.792\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.875\n",
            "11/04 09:54:31 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - segm_mAP_copypaste: 0.795 0.861 0.861 0.118 0.664 0.852\n",
            "11/04 09:54:31 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [9][13/13]  coco/bbox_mAP: 0.7640  coco/bbox_mAP_50: 0.8790  coco/bbox_mAP_75: 0.8450  coco/bbox_mAP_s: 0.2020  coco/bbox_mAP_m: 0.6450  coco/bbox_mAP_l: 0.8210  coco/segm_mAP: 0.7950  coco/segm_mAP_50: 0.8610  coco/segm_mAP_75: 0.8610  coco/segm_mAP_s: 0.1180  coco/segm_mAP_m: 0.6640  coco/segm_mAP_l: 0.8520\n",
            "11/04 09:54:33 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [10][10/31]  lr: 1.4439e-04  memory: 3366  data_time: 0.0050  loss: 0.2297  loss_rpn_cls: 0.0055  loss_rpn_bbox: 0.0094  loss_cls: 0.0580  acc: 98.5352  loss_bbox: 0.0791  loss_mask: 0.0777  time: 0.1991\n",
            "11/04 09:54:36 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [10][20/31]  lr: 1.4940e-04  memory: 3366  data_time: 0.0038  loss: 0.2331  loss_rpn_cls: 0.0056  loss_rpn_bbox: 0.0095  loss_cls: 0.0586  acc: 99.7070  loss_bbox: 0.0800  loss_mask: 0.0795  time: 0.2006\n",
            "11/04 09:54:37 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [10][30/31]  lr: 1.5440e-04  memory: 3366  data_time: 0.0039  loss: 0.2143  loss_rpn_cls: 0.0057  loss_rpn_bbox: 0.0086  loss_cls: 0.0540  acc: 98.9258  loss_bbox: 0.0724  loss_mask: 0.0736  time: 0.1982\n",
            "11/04 09:54:38 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Exp name: mask-rcnn_r50-caffe_fpn_ms-poly-1x_coco_20221104_095316\n",
            "11/04 09:54:40 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [11][10/31]  lr: 1.5991e-04  memory: 3534  data_time: 0.0043  loss: 0.2142  loss_rpn_cls: 0.0060  loss_rpn_bbox: 0.0080  loss_cls: 0.0562  acc: 99.4141  loss_bbox: 0.0723  loss_mask: 0.0716  time: 0.1963\n",
            "11/04 09:54:42 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [11][20/31]  lr: 1.6491e-04  memory: 3366  data_time: 0.0035  loss: 0.2125  loss_rpn_cls: 0.0050  loss_rpn_bbox: 0.0070  loss_cls: 0.0567  acc: 94.1406  loss_bbox: 0.0720  loss_mask: 0.0718  time: 0.1956\n",
            "11/04 09:54:44 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [11][30/31]  lr: 1.6992e-04  memory: 3541  data_time: 0.0035  loss: 0.2068  loss_rpn_cls: 0.0055  loss_rpn_bbox: 0.0073  loss_cls: 0.0554  acc: 96.8750  loss_bbox: 0.0695  loss_mask: 0.0691  time: 0.1975\n",
            "11/04 09:54:44 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Exp name: mask-rcnn_r50-caffe_fpn_ms-poly-1x_coco_20221104_095316\n",
            "11/04 09:54:46 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [12][10/31]  lr: 1.7543e-05  memory: 3182  data_time: 0.0040  loss: 0.1914  loss_rpn_cls: 0.0053  loss_rpn_bbox: 0.0063  loss_cls: 0.0518  acc: 94.4336  loss_bbox: 0.0641  loss_mask: 0.0638  time: 0.1957\n",
            "11/04 09:54:48 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [12][20/31]  lr: 1.8043e-05  memory: 3183  data_time: 0.0035  loss: 0.1919  loss_rpn_cls: 0.0052  loss_rpn_bbox: 0.0061  loss_cls: 0.0521  acc: 98.6328  loss_bbox: 0.0647  loss_mask: 0.0638  time: 0.1974\n",
            "11/04 09:54:50 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [12][30/31]  lr: 1.8544e-05  memory: 3367  data_time: 0.0035  loss: 0.2058  loss_rpn_cls: 0.0055  loss_rpn_bbox: 0.0074  loss_cls: 0.0530  acc: 98.7305  loss_bbox: 0.0692  loss_mask: 0.0707  time: 0.1979\n",
            "11/04 09:54:50 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Exp name: mask-rcnn_r50-caffe_fpn_ms-poly-1x_coco_20221104_095316\n",
            "11/04 09:54:50 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Saving checkpoint at 12 epochs\n",
            "11/04 09:54:52 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [12][10/13]    memory: 1803  \n",
            "11/04 09:54:52 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Evaluating bbox...\n",
            "Loading and preparing results...\n",
            "DONE (t=0.00s)\n",
            "creating index...\n",
            "index created!\n",
            "Running per image evaluation...\n",
            "Evaluate annotation type *bbox*\n",
            "DONE (t=0.01s).\n",
            "Accumulating evaluation results...\n",
            "DONE (t=0.01s).\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.775\n",
            " Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.879\n",
            " Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.853\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.126\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.635\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.841\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.812\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.812\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.812\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.250\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.742\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.867\n",
            "11/04 09:54:52 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - bbox_mAP_copypaste: 0.775 0.879 0.853 0.126 0.635 0.841\n",
            "11/04 09:54:52 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Evaluating segm...\n",
            "Loading and preparing results...\n",
            "DONE (t=0.00s)\n",
            "creating index...\n",
            "index created!\n",
            "Running per image evaluation...\n",
            "Evaluate annotation type *segm*\n",
            "DONE (t=0.02s).\n",
            "Accumulating evaluation results...\n",
            "DONE (t=0.01s).\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.791\n",
            " Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.861\n",
            " Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.861\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.088\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.661\n",
            " Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.846\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.830\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.830\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.830\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.350\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.792\n",
            " Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.869\n",
            "11/04 09:54:52 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - segm_mAP_copypaste: 0.791 0.861 0.861 0.088 0.661 0.846\n",
            "11/04 09:54:52 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [12][13/13]  coco/bbox_mAP: 0.7750  coco/bbox_mAP_50: 0.8790  coco/bbox_mAP_75: 0.8530  coco/bbox_mAP_s: 0.1260  coco/bbox_mAP_m: 0.6350  coco/bbox_mAP_l: 0.8410  coco/segm_mAP: 0.7910  coco/segm_mAP_50: 0.8610  coco/segm_mAP_75: 0.8610  coco/segm_mAP_s: 0.0880  coco/segm_mAP_m: 0.6610  coco/segm_mAP_l: 0.8460\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "MaskRCNN(\n",
              "  (data_preprocessor): DetDataPreprocessor()\n",
              "  (backbone): ResNet(\n",
              "    (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)\n",
              "    (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "    (relu): ReLU(inplace=True)\n",
              "    (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)\n",
              "    (layer1): ResLayer(\n",
              "      (0): Bottleneck(\n",
              "        (conv1): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (relu): ReLU(inplace=True)\n",
              "        (downsample): Sequential(\n",
              "          (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        )\n",
              "      )\n",
              "      (1): Bottleneck(\n",
              "        (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (relu): ReLU(inplace=True)\n",
              "      )\n",
              "      (2): Bottleneck(\n",
              "        (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (relu): ReLU(inplace=True)\n",
              "      )\n",
              "    )\n",
              "    (layer2): ResLayer(\n",
              "      (0): Bottleneck(\n",
              "        (conv1): Conv2d(256, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
              "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (relu): ReLU(inplace=True)\n",
              "        (downsample): Sequential(\n",
              "          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
              "          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        )\n",
              "      )\n",
              "      (1): Bottleneck(\n",
              "        (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (relu): ReLU(inplace=True)\n",
              "      )\n",
              "      (2): Bottleneck(\n",
              "        (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (relu): ReLU(inplace=True)\n",
              "      )\n",
              "      (3): Bottleneck(\n",
              "        (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (relu): ReLU(inplace=True)\n",
              "      )\n",
              "    )\n",
              "    (layer3): ResLayer(\n",
              "      (0): Bottleneck(\n",
              "        (conv1): Conv2d(512, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
              "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (relu): ReLU(inplace=True)\n",
              "        (downsample): Sequential(\n",
              "          (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
              "          (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        )\n",
              "      )\n",
              "      (1): Bottleneck(\n",
              "        (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (relu): ReLU(inplace=True)\n",
              "      )\n",
              "      (2): Bottleneck(\n",
              "        (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (relu): ReLU(inplace=True)\n",
              "      )\n",
              "      (3): Bottleneck(\n",
              "        (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (relu): ReLU(inplace=True)\n",
              "      )\n",
              "      (4): Bottleneck(\n",
              "        (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (relu): ReLU(inplace=True)\n",
              "      )\n",
              "      (5): Bottleneck(\n",
              "        (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (relu): ReLU(inplace=True)\n",
              "      )\n",
              "    )\n",
              "    (layer4): ResLayer(\n",
              "      (0): Bottleneck(\n",
              "        (conv1): Conv2d(1024, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
              "        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (relu): ReLU(inplace=True)\n",
              "        (downsample): Sequential(\n",
              "          (0): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
              "          (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        )\n",
              "      )\n",
              "      (1): Bottleneck(\n",
              "        (conv1): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (relu): ReLU(inplace=True)\n",
              "      )\n",
              "      (2): Bottleneck(\n",
              "        (conv1): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
              "        (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (relu): ReLU(inplace=True)\n",
              "      )\n",
              "    )\n",
              "  )\n",
              "  init_cfg={'type': 'Pretrained', 'checkpoint': 'open-mmlab://detectron2/resnet50_caffe'}\n",
              "  (neck): FPN(\n",
              "    (lateral_convs): ModuleList(\n",
              "      (0): ConvModule(\n",
              "        (conv): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1))\n",
              "      )\n",
              "      (1): ConvModule(\n",
              "        (conv): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1))\n",
              "      )\n",
              "      (2): ConvModule(\n",
              "        (conv): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1))\n",
              "      )\n",
              "      (3): ConvModule(\n",
              "        (conv): Conv2d(2048, 256, kernel_size=(1, 1), stride=(1, 1))\n",
              "      )\n",
              "    )\n",
              "    (fpn_convs): ModuleList(\n",
              "      (0): ConvModule(\n",
              "        (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
              "      )\n",
              "      (1): ConvModule(\n",
              "        (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
              "      )\n",
              "      (2): ConvModule(\n",
              "        (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
              "      )\n",
              "      (3): ConvModule(\n",
              "        (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
              "      )\n",
              "    )\n",
              "  )\n",
              "  init_cfg={'type': 'Xavier', 'layer': 'Conv2d', 'distribution': 'uniform'}\n",
              "  (rpn_head): RPNHead(\n",
              "    (loss_cls): CrossEntropyLoss(avg_non_ignore=False)\n",
              "    (loss_bbox): L1Loss()\n",
              "    (rpn_conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
              "    (rpn_cls): Conv2d(256, 3, kernel_size=(1, 1), stride=(1, 1))\n",
              "    (rpn_reg): Conv2d(256, 12, kernel_size=(1, 1), stride=(1, 1))\n",
              "  )\n",
              "  init_cfg={'type': 'Normal', 'layer': 'Conv2d', 'std': 0.01}\n",
              "  (roi_head): StandardRoIHead(\n",
              "    (bbox_roi_extractor): SingleRoIExtractor(\n",
              "      (roi_layers): ModuleList(\n",
              "        (0): RoIAlign(output_size=(7, 7), spatial_scale=0.25, sampling_ratio=0, pool_mode=avg, aligned=True, use_torchvision=False)\n",
              "        (1): RoIAlign(output_size=(7, 7), spatial_scale=0.125, sampling_ratio=0, pool_mode=avg, aligned=True, use_torchvision=False)\n",
              "        (2): RoIAlign(output_size=(7, 7), spatial_scale=0.0625, sampling_ratio=0, pool_mode=avg, aligned=True, use_torchvision=False)\n",
              "        (3): RoIAlign(output_size=(7, 7), spatial_scale=0.03125, sampling_ratio=0, pool_mode=avg, aligned=True, use_torchvision=False)\n",
              "      )\n",
              "    )\n",
              "    (bbox_head): Shared2FCBBoxHead(\n",
              "      (loss_cls): CrossEntropyLoss(avg_non_ignore=False)\n",
              "      (loss_bbox): L1Loss()\n",
              "      (fc_cls): Linear(in_features=1024, out_features=2, bias=True)\n",
              "      (fc_reg): Linear(in_features=1024, out_features=4, bias=True)\n",
              "      (shared_convs): ModuleList()\n",
              "      (shared_fcs): ModuleList(\n",
              "        (0): Linear(in_features=12544, out_features=1024, bias=True)\n",
              "        (1): Linear(in_features=1024, out_features=1024, bias=True)\n",
              "      )\n",
              "      (cls_convs): ModuleList()\n",
              "      (cls_fcs): ModuleList()\n",
              "      (reg_convs): ModuleList()\n",
              "      (reg_fcs): ModuleList()\n",
              "      (relu): ReLU(inplace=True)\n",
              "    )\n",
              "    init_cfg=[{'type': 'Normal', 'std': 0.01, 'override': {'name': 'fc_cls'}}, {'type': 'Normal', 'std': 0.001, 'override': {'name': 'fc_reg'}}, {'type': 'Xavier', 'distribution': 'uniform', 'override': [{'name': 'shared_fcs'}, {'name': 'cls_fcs'}, {'name': 'reg_fcs'}]}]\n",
              "    (mask_roi_extractor): SingleRoIExtractor(\n",
              "      (roi_layers): ModuleList(\n",
              "        (0): RoIAlign(output_size=(14, 14), spatial_scale=0.25, sampling_ratio=0, pool_mode=avg, aligned=True, use_torchvision=False)\n",
              "        (1): RoIAlign(output_size=(14, 14), spatial_scale=0.125, sampling_ratio=0, pool_mode=avg, aligned=True, use_torchvision=False)\n",
              "        (2): RoIAlign(output_size=(14, 14), spatial_scale=0.0625, sampling_ratio=0, pool_mode=avg, aligned=True, use_torchvision=False)\n",
              "        (3): RoIAlign(output_size=(14, 14), spatial_scale=0.03125, sampling_ratio=0, pool_mode=avg, aligned=True, use_torchvision=False)\n",
              "      )\n",
              "    )\n",
              "    (mask_head): FCNMaskHead(\n",
              "      (loss_mask): CrossEntropyLoss(avg_non_ignore=False)\n",
              "      (convs): ModuleList(\n",
              "        (0): ConvModule(\n",
              "          (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
              "          (activate): ReLU(inplace=True)\n",
              "        )\n",
              "        (1): ConvModule(\n",
              "          (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
              "          (activate): ReLU(inplace=True)\n",
              "        )\n",
              "        (2): ConvModule(\n",
              "          (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
              "          (activate): ReLU(inplace=True)\n",
              "        )\n",
              "        (3): ConvModule(\n",
              "          (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
              "          (activate): ReLU(inplace=True)\n",
              "        )\n",
              "      )\n",
              "      (upsample): ConvTranspose2d(256, 256, kernel_size=(2, 2), stride=(2, 2))\n",
              "      (conv_logits): Conv2d(256, 1, kernel_size=(1, 1), stride=(1, 1))\n",
              "      (relu): ReLU(inplace=True)\n",
              "    )\n",
              "  )\n",
              ")"
            ]
          },
          "execution_count": 4,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# start training\n",
        "runner.train()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "_vYQF5K2NqqI"
      },
      "source": [
        "### Understand the log\n",
        "From the log, we can have a basic understanding on the training process and know how well the detector is trained.\n",
        "\n",
        "First, since the dataset we are using is small, we loaded a Mask R-CNN model and finetune it for detection. Because the original Mask R-CNN is trained on COCO dataset that contains 80 classes but KITTI Tiny dataset only have 3 classes. Therefore, the last FC layers of the pre-trained Mask R-CNN for classification and regression have different weight shape and are not used. The pre-trained weights of mask prediction layer `mask_head.conv_logits` also does not matches the current model and is not used due to similar reason.\n",
        "\n",
        "Third, after training, the detector is evaluated by the default COCO-style evaluation. The results show that the detector achieves 79.6 bbox AP and 81.5 mask AP on the val dataset, not bad!\n",
        "\n",
        " We can also check the tensorboard to see the curves."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "%pip install tensorboard  -i https://mirrors.ustc.edu.cn/pypi/web/simple"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 840,
          "resources": {
            "https://localhost:6006/?tensorboardColab=true": {
              "data": "[encoded data, 252024 characters]",
              "headers": [
                [
                  "content-type",
                  "text/html; charset=utf-8"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/environment": {
              "data": "eyJ2ZXJzaW9uIjogIjIuNy4wIiwgImRhdGFfbG9jYXRpb24iOiAiLi90dXRvcmlhbF9leHBzIiwgIndpbmRvd190aXRsZSI6ICIiLCAiZXhwZXJpbWVudF9uYW1lIjogIiIsICJleHBlcmltZW50X2Rlc2NyaXB0aW9uIjogIiIsICJjcmVhdGlvbl90aW1lIjogMC4wLCAiZGVidWciOiB7ImRhdGFfcHJvdmlkZXIiOiAiR3JwY0RhdGFQcm92aWRlcihhZGRyPSdsb2NhbGhvc3Q6MzMwNzcnKSIsICJmbGFncyI6IHsibG9nZGlyIjogIi4vdHV0b3JpYWxfZXhwcyIsICJsb2dkaXJfc3BlYyI6ICIiLCAiaG9zdCI6IG51bGwsICJiaW5kX2FsbCI6IGZhbHNlLCAicG9ydCI6IG51bGwsICJyZXVzZV9wb3J0IjogZmFsc2UsICJsb2FkX2Zhc3QiOiAiYXV0byIsICJleHRyYV9kYXRhX3NlcnZlcl9mbGFncyI6ICIiLCAiZ3JwY19jcmVkc190eXBlIjogImxvY2FsIiwgImdycGNfZGF0YV9wcm92aWRlciI6ICIiLCAicHVyZ2Vfb3JwaGFuZWRfZGF0YSI6IHRydWUsICJkYiI6ICIiLCAiZGJfaW1wb3J0IjogZmFsc2UsICJpbnNwZWN0IjogZmFsc2UsICJ2ZXJzaW9uX3RiIjogZmFsc2UsICJ0YWciOiAiIiwgImV2ZW50X2ZpbGUiOiAiIiwgInBhdGhfcHJlZml4IjogIiIsICJ3aW5kb3dfdGl0bGUiOiAiIiwgIm1heF9yZWxvYWRfdGhyZWFkcyI6IDEsICJyZWxvYWRfaW50ZXJ2YWwiOiA1LjAsICJyZWxvYWRfdGFzayI6ICJhdXRvIiwgInJlbG9hZF9tdWx0aWZpbGUiOiBudWxsLCAicmVsb2FkX211bHRpZmlsZV9pbmFjdGl2ZV9zZWNzIjogODY0MDAsICJnZW5lcmljX2RhdGEiOiAiYXV0byIsICJzYW1wbGVzX3Blcl9wbHVnaW4iOiB7fSwgImN1c3RvbV9wcmVkaWN0X2ZuIjogIiIsICJ3aXRfZGF0YV9kaXIiOiAiIiwgIl9fdGVuc29yYm9hcmRfc3ViY29tbWFuZCI6ICJzZXJ2ZSJ9fX0=",
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugin/scalars/tags": {
              "data": "e30=",
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/plugins_listing": {
              "data": "eyJzY2FsYXJzIjogeyJkaXNhYmxlX3JlbG9hZCI6IGZhbHNlLCAiZW5hYmxlZCI6IGZhbHNlLCAicmVtb3ZlX2RvbSI6IGZhbHNlLCAidGFiX25hbWUiOiAic2NhbGFycyIsICJsb2FkaW5nX21lY2hhbmlzbSI6IHsidHlwZSI6ICJDVVNUT01fRUxFTUVOVCIsICJlbGVtZW50X25hbWUiOiAidGYtc2NhbGFyLWRhc2hib2FyZCJ9fSwgImN1c3RvbV9zY2FsYXJzIjogeyJkaXNhYmxlX3JlbG9hZCI6IGZhbHNlLCAiZW5hYmxlZCI6IGZhbHNlLCAicmVtb3ZlX2RvbSI6IGZhbHNlLCAidGFiX25hbWUiOiAiQ3VzdG9tIFNjYWxhcnMiLCAibG9hZGluZ19tZWNoYW5pc20iOiB7InR5cGUiOiAiQ1VTVE9NX0VMRU1FTlQiLCAiZWxlbWVudF9uYW1lIjogInRmLWN1c3RvbS1zY2FsYXItZGFzaGJvYXJkIn19LCAiaW1hZ2VzIjogeyJkaXNhYmxlX3JlbG9hZCI6IGZhbHNlLCAiZW5hYmxlZCI6IGZhbHNlLCAicmVtb3ZlX2RvbSI6IGZhbHNlLCAidGFiX25hbWUiOiAiaW1hZ2VzIiwgImxvYWRpbmdfbWVjaGFuaXNtIjogeyJ0eXBlIjogIkNVU1RPTV9FTEVNRU5UIiwgImVsZW1lbnRfbmFtZSI6ICJ0Zi1pbWFnZS1kYXNoYm9hcmQifX0sICJhdWRpbyI6IHsiZGlzYWJsZV9yZWxvYWQiOiBmYWxzZSwgImVuYWJsZWQiOiBmYWxzZSwgInJlbW92ZV9kb20iOiBmYWxzZSwgInRhYl9uYW1lIjogImF1ZGlvIiwgImxvYWRpbmdfbWVjaGFuaXNtIjogeyJ0eXBlIjogIkNVU1RPTV9FTEVNRU5UIiwgImVsZW1lbnRfbmFtZSI6ICJ0Zi1hdWRpby1kYXNoYm9hcmQifX0sICJkZWJ1Z2dlci12MiI6IHsiZGlzYWJsZV9yZWxvYWQiOiBmYWxzZSwgImVuYWJsZWQiOiBmYWxzZSwgInJlbW92ZV9kb20iOiBmYWxzZSwgInRhYl9uYW1lIjogIkRlYnVnZ2VyIFYyIiwgImxvYWRpbmdfbWVjaGFuaXNtIjogeyJ0eXBlIjogIk5HX0NPTVBPTkVOVCJ9fSwgImdyYXBocyI6IHsiZGlzYWJsZV9yZWxvYWQiOiB0cnVlLCAiZW5hYmxlZCI6IGZhbHNlLCAicmVtb3ZlX2RvbSI6IGZhbHNlLCAidGFiX25hbWUiOiAiZ3JhcGhzIiwgImxvYWRpbmdfbWVjaGFuaXNtIjogeyJ0eXBlIjogIkNVU1RPTV9FTEVNRU5UIiwgImVsZW1lbnRfbmFtZSI6ICJ0Zi1ncmFwaC1kYXNoYm9hcmQifX0sICJkaXN0cmlidXRpb25zIjogeyJkaXNhYmxlX3JlbG9hZCI6IGZhbHNlLCAiZW5hYmxlZCI6IGZhbHNlLCAicmVtb3ZlX2RvbSI6IGZhbHNlLCAidGFiX25hbWUiOiAiZGlzdHJpYnV0aW9ucyIsICJsb2FkaW5nX21lY2hhbmlzbSI6IHsidHlwZSI6ICJDVVNUT01fRUxFTUVOVCIsICJlbGVtZW50X25hbWUiOiAidGYtZGlzdHJpYnV0aW9uLWRhc2hib2FyZCJ9fSwgImhpc3RvZ3JhbXMiOiB7ImRpc2FibGVfcmVsb2FkIjogZmFsc2UsICJlbmFibGVkIjogZmFsc2UsICJyZW1vdmVfZG9tIjogZmFsc2UsICJ0YWJfbmFtZSI6ICJoaXN0b2dyYW1zIiwgImxvYWRpbmdfbWVjaGFuaXNtIjogeyJ0eXBlIjogIkNVU1RPTV9FTEVNRU5UIiwgImVsZW1lbnRfbmFtZSI6ICJ0Zi1oaXN0b2dyYW0tZGFzaGJvYXJkIn19LCAidGV4dCI6IHsiZGlzYWJsZV9yZWxvYWQiOiBmYWxzZSwgImVuYWJsZWQiOiBmYWxzZSwgInJlbW92ZV9kb20iOiBmYWxzZSwgInRhYl9uYW1lIjogInRleHQiLCAibG9hZGluZ19tZWNoYW5pc20iOiB7InR5cGUiOiAiQ1VTVE9NX0VMRU1FTlQiLCAiZWxlbWVudF9uYW1lIjogInRmLXRleHQtZGFzaGJvYXJkIn19LCAicHJfY3VydmVzIjogeyJkaXNhYmxlX3JlbG9hZCI6IGZhbHNlLCAiZW5hYmxlZCI6IGZhbHNlLCAicmVtb3ZlX2RvbSI6IGZhbHNlLCAidGFiX25hbWUiOiAiUFIgQ3VydmVzIiwgImxvYWRpbmdfbWVjaGFuaXNtIjogeyJ0eXBlIjogIkNVU1RPTV9FTEVNRU5UIiwgImVsZW1lbnRfbmFtZSI6ICJ0Zi1wci1jdXJ2ZS1kYXNoYm9hcmQifX0sICJwcm9maWxlX3JlZGlyZWN0IjogeyJkaXNhYmxlX3JlbG9hZCI6IGZhbHNlLCAiZW5hYmxlZCI6IGZhbHNlLCAicmVtb3ZlX2RvbSI6IGZhbHNlLCAidGFiX25hbWUiOiAiUHJvZmlsZSIsICJsb2FkaW5nX21lY2hhbmlzbSI6IHsidHlwZSI6ICJDVVNUT01fRUxFTUVOVCIsICJlbGVtZW50X25hbWUiOiAidGYtcHJvZmlsZS1yZWRpcmVjdC1kYXNoYm9hcmQifX0sICJocGFyYW1zIjogeyJkaXNhYmxlX3JlbG9hZCI6IGZhbHNlLCAiZW5hYmxlZCI6IGZhbHNlLCAicmVtb3ZlX2RvbSI6IGZhbHNlLCAidGFiX25hbWUiOiAiaHBhcmFtcyIsICJsb2FkaW5nX21lY2hhbmlzbSI6IHsidHlwZSI6ICJDVVNUT01fRUxFTUVOVCIsICJlbGVtZW50X25hbWUiOiAidGYtaHBhcmFtcy1kYXNoYm9hcmQifX0sICJtZXNoIjogeyJkaXNhYmxlX3JlbG9hZCI6IGZhbHNlLCAiZW5hYmxlZCI6IGZhbHNlLCAicmVtb3ZlX2RvbSI6IGZhbHNlLCAidGFiX25hbWUiOiAibWVzaCIsICJsb2FkaW5nX21lY2hhbmlzbSI6IHsidHlwZSI6ICJDVVNUT01fRUxFTUVOVCIsICJlbGVtZW50X25hbWUiOiAibWVzaC1kYXNoYm9hcmQifX0sICJ0aW1lc2VyaWVzIjogeyJkaXNhYmxlX3JlbG9hZCI6IGZhbHNlLCAiZW5hYmxlZCI6IGZhbHNlLCAicmVtb3ZlX2RvbSI6IGZhbHNlLCAidGFiX25hbWUiOiAiVGltZSBTZXJpZXMiLCAibG9hZGluZ19tZWNoYW5pc20iOiB7InR5cGUiOiAiTkdfQ09NUE9ORU5UIn19LCAicHJvamVjdG9yIjogeyJkaXNhYmxlX3JlbG9hZCI6IHRydWUsICJlbmFibGVkIjogZmFsc2UsICJyZW1vdmVfZG9tIjogZmFsc2UsICJ0YWJfbmFtZSI6ICJwcm9qZWN0b3IiLCAibG9hZGluZ19tZWNoYW5pc20iOiB7InR5cGUiOiAiSUZSQU1FIiwgIm1vZHVsZV9wYXRoIjogIi9kYXRhL3BsdWdpbi9wcm9qZWN0b3IvaW5kZXguanMifX0sICJ3aGF0aWYiOiB7ImRpc2FibGVfcmVsb2FkIjogZmFsc2UsICJlbmFibGVkIjogZmFsc2UsICJyZW1vdmVfZG9tIjogZmFsc2UsICJ0YWJfbmFtZSI6ICJXaGF0LUlmIFRvb2wiLCAibG9hZGluZ19tZWNoYW5pc20iOiB7InR5cGUiOiAiSUZSQU1FIiwgIm1vZHVsZV9wYXRoIjogIi9kYXRhL3BsdWdpbi93aGF0aWYvaW5kZXguanMifX19",
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/data/runs": {
              "data": "W10=",
              "headers": [
                [
                  "content-type",
                  "application/json"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/font-roboto/RxZJdnzeo3R5zSexge8UUZBw1xU1rKptJj_0jans920.woff2": {
              "data": "[encoded data, 14384 characters]",
              "headers": [
                [
                  "content-type",
                  "font/woff2"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/font-roboto/d-6IYplOFocCacKzxwXSOJBw1xU1rKptJj_0jans920.woff2": {
              "data": "[encoded data, 14352 characters]",
              "headers": [
                [
                  "content-type",
                  "font/woff2"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/font-roboto/oMMgfZMQthOryQo9n22dcuvvDin1pK8aKteLpeZ5c0A.woff2": {
              "data": "[encoded data, 14332 characters]",
              "headers": [
                [
                  "content-type",
                  "font/woff2"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/font-roboto/vPcynSL0qHq_6dX7lKVByXYhjbSpvc47ee6xR_80Hnw.woff2": {
              "data": "[encoded data, 16472 characters]",
              "headers": [
                [
                  "content-type",
                  "font/woff2"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/icon_bundle.svg": {
              "data": "[encoded data, 18556 characters]",
              "headers": [
                [
                  "content-type",
                  "image/svg+xml; charset=utf-8"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            },
            "https://localhost:6006/index.js?_file_hash=29a7d03a": {
              "data": "[encoded data, 10086504 characters]",
              "headers": [
                [
                  "content-type",
                  "application/javascript; charset=utf-8"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            }
          }
        },
        "id": "PW2NAam_7irv",
        "outputId": "38b8995a-c139-441c-e393-56c11f214655"
      },
      "outputs": [],
      "source": [
        "# load tensorboard in jupyter notebook\n",
        "%load_ext tensorboard"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# see curves in tensorboard\n",
        "# if you see <IPython.core.display.HTML object> please run it again\n",
        "%tensorboard --logdir tutorial_exps/"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "MfQ-yspZLuuI"
      },
      "source": [
        "## Test the Trained Detector\n",
        "\n",
        "After finetuning the detector, let's visualize the prediction results!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 854
        },
        "id": "_MuZurfGLq0p",
        "outputId": "5df4dc5f-5b90-46c3-9aeb-a4d0ac13564a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "local loads checkpoint from path: tutorial_exps/epoch_12.pth\n",
            "<DetDataSample(\n",
            "\n",
            "    META INFORMATION\n",
            "    pad_shape: (800, 1216)\n",
            "    ori_shape: (1360, 2048)\n",
            "    img_id: 0\n",
            "    img_shape: (800, 1205)\n",
            "    scale_factor: (0.58837890625, 0.5882352941176471)\n",
            "    img_path: None\n",
            "    batch_input_shape: (800, 1216)\n",
            "\n",
            "    DATA FIELDS\n",
            "    _ignored_instances: <InstanceData(\n",
            "        \n",
            "            META INFORMATION\n",
            "        \n",
            "            DATA FIELDS\n",
            "            labels: tensor([], dtype=torch.int64)\n",
            "            bboxes: tensor([], size=(0, 4))\n",
            "            masks: BitmapMasks(num_masks=0, height=1360, width=2048)\n",
            "        ) at 0x7f8ded7a5f90>\n",
            "    _pred_instances: <InstanceData(\n",
            "        \n",
            "            META INFORMATION\n",
            "        \n",
            "            DATA FIELDS\n",
            "            labels: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])\n",
            "            bboxes: tensor([[6.5086e+02, 2.3730e+02, 1.3999e+03, 1.0022e+03],\n",
            "                        [1.2901e+03, 4.1324e+02, 1.6869e+03, 9.3037e+02],\n",
            "                        [5.8837e+02, 4.2606e+02, 7.3247e+02, 8.2997e+02],\n",
            "                        [1.0073e+03, 4.6810e+00, 1.4482e+03, 1.6622e+02],\n",
            "                        [4.9131e+02, 4.0341e+02, 6.8386e+02, 5.2800e+02],\n",
            "                        [1.7873e+03, 2.0464e+02, 1.8131e+03, 2.3014e+02],\n",
            "                        [1.0289e+03, 1.1672e+03, 1.1495e+03, 1.2271e+03],\n",
            "                        [6.9832e+00, 1.2445e+03, 2.7969e+02, 1.3579e+03],\n",
            "                        [1.1418e+03, 9.0589e+02, 1.3853e+03, 1.3376e+03],\n",
            "                        [7.2739e-01, 1.1680e+03, 7.1160e+01, 1.2572e+03],\n",
            "                        [4.1536e+02, 1.1588e+02, 5.5911e+02, 3.8368e+02],\n",
            "                        [5.2834e+02, 5.7182e+02, 5.7518e+02, 6.1491e+02],\n",
            "                        [1.0190e+03, 1.3105e+03, 1.1955e+03, 1.3599e+03],\n",
            "                        [1.3140e+03, 1.1050e+03, 1.5466e+03, 1.3433e+03],\n",
            "                        [6.1398e+02, 7.6313e+02, 6.7014e+02, 8.2759e+02],\n",
            "                        [9.8987e+02, 1.7473e+01, 1.5034e+03, 3.9971e+02],\n",
            "                        [4.6703e+02, 3.3207e+02, 7.0845e+02, 5.2375e+02],\n",
            "                        [1.5542e+00, 1.0101e+03, 5.8170e+01, 1.1833e+03],\n",
            "                        [1.0516e+03, 3.4101e+02, 1.1707e+03, 4.5346e+02],\n",
            "                        [3.8340e+01, 1.1175e+03, 1.1662e+02, 1.1785e+03],\n",
            "                        [3.5384e+02, 6.0554e+02, 5.0771e+02, 1.0169e+03],\n",
            "                        [1.0586e+03, 3.3332e+02, 1.6495e+03, 9.5863e+02],\n",
            "                        [1.3338e+03, 4.2638e+00, 1.4954e+03, 1.4791e+02]])\n",
            "            scores: tensor([0.9967, 0.9822, 0.9776, 0.9207, 0.8162, 0.7081, 0.5568, 0.2580, 0.2516,\n",
            "                        0.2240, 0.2226, 0.1996, 0.1893, 0.1731, 0.1476, 0.1446, 0.1104, 0.0907,\n",
            "                        0.0905, 0.0834, 0.0673, 0.0588, 0.0554])\n",
            "            masks: tensor([[[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        ...,\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]]])\n",
            "        ) at 0x7f8ded7a54d0>\n",
            "    ignored_instances: <InstanceData(\n",
            "        \n",
            "            META INFORMATION\n",
            "        \n",
            "            DATA FIELDS\n",
            "            labels: tensor([], dtype=torch.int64)\n",
            "            bboxes: tensor([], size=(0, 4))\n",
            "            masks: BitmapMasks(num_masks=0, height=1360, width=2048)\n",
            "        ) at 0x7f8ded7a5f90>\n",
            "    gt_instances: <InstanceData(\n",
            "        \n",
            "            META INFORMATION\n",
            "        \n",
            "            DATA FIELDS\n",
            "            labels: tensor([], dtype=torch.int64)\n",
            "            bboxes: tensor([], size=(0, 4))\n",
            "            masks: BitmapMasks(num_masks=0, height=1360, width=2048)\n",
            "        ) at 0x7f8ded7a58d0>\n",
            "    _gt_instances: <InstanceData(\n",
            "        \n",
            "            META INFORMATION\n",
            "        \n",
            "            DATA FIELDS\n",
            "            labels: tensor([], dtype=torch.int64)\n",
            "            bboxes: tensor([], size=(0, 4))\n",
            "            masks: BitmapMasks(num_masks=0, height=1360, width=2048)\n",
            "        ) at 0x7f8ded7a58d0>\n",
            "    pred_instances: <InstanceData(\n",
            "        \n",
            "            META INFORMATION\n",
            "        \n",
            "            DATA FIELDS\n",
            "            labels: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])\n",
            "            bboxes: tensor([[6.5086e+02, 2.3730e+02, 1.3999e+03, 1.0022e+03],\n",
            "                        [1.2901e+03, 4.1324e+02, 1.6869e+03, 9.3037e+02],\n",
            "                        [5.8837e+02, 4.2606e+02, 7.3247e+02, 8.2997e+02],\n",
            "                        [1.0073e+03, 4.6810e+00, 1.4482e+03, 1.6622e+02],\n",
            "                        [4.9131e+02, 4.0341e+02, 6.8386e+02, 5.2800e+02],\n",
            "                        [1.7873e+03, 2.0464e+02, 1.8131e+03, 2.3014e+02],\n",
            "                        [1.0289e+03, 1.1672e+03, 1.1495e+03, 1.2271e+03],\n",
            "                        [6.9832e+00, 1.2445e+03, 2.7969e+02, 1.3579e+03],\n",
            "                        [1.1418e+03, 9.0589e+02, 1.3853e+03, 1.3376e+03],\n",
            "                        [7.2739e-01, 1.1680e+03, 7.1160e+01, 1.2572e+03],\n",
            "                        [4.1536e+02, 1.1588e+02, 5.5911e+02, 3.8368e+02],\n",
            "                        [5.2834e+02, 5.7182e+02, 5.7518e+02, 6.1491e+02],\n",
            "                        [1.0190e+03, 1.3105e+03, 1.1955e+03, 1.3599e+03],\n",
            "                        [1.3140e+03, 1.1050e+03, 1.5466e+03, 1.3433e+03],\n",
            "                        [6.1398e+02, 7.6313e+02, 6.7014e+02, 8.2759e+02],\n",
            "                        [9.8987e+02, 1.7473e+01, 1.5034e+03, 3.9971e+02],\n",
            "                        [4.6703e+02, 3.3207e+02, 7.0845e+02, 5.2375e+02],\n",
            "                        [1.5542e+00, 1.0101e+03, 5.8170e+01, 1.1833e+03],\n",
            "                        [1.0516e+03, 3.4101e+02, 1.1707e+03, 4.5346e+02],\n",
            "                        [3.8340e+01, 1.1175e+03, 1.1662e+02, 1.1785e+03],\n",
            "                        [3.5384e+02, 6.0554e+02, 5.0771e+02, 1.0169e+03],\n",
            "                        [1.0586e+03, 3.3332e+02, 1.6495e+03, 9.5863e+02],\n",
            "                        [1.3338e+03, 4.2638e+00, 1.4954e+03, 1.4791e+02]])\n",
            "            scores: tensor([0.9967, 0.9822, 0.9776, 0.9207, 0.8162, 0.7081, 0.5568, 0.2580, 0.2516,\n",
            "                        0.2240, 0.2226, 0.1996, 0.1893, 0.1731, 0.1476, 0.1446, 0.1104, 0.0907,\n",
            "                        0.0905, 0.0834, 0.0673, 0.0588, 0.0554])\n",
            "            masks: tensor([[[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        ...,\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]],\n",
            "                \n",
            "                        [[False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         ...,\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False],\n",
            "                         [False, False, False,  ..., False, False, False]]])\n",
            "        ) at 0x7f8ded7a54d0>\n",
            ") at 0x7f8ded7a5d10>\n"
          ]
        }
      ],
      "source": [
        "import mmcv\n",
        "from mmdet.apis import init_detector, inference_detector\n",
        "img = mmcv.imread('./ballondatasets/balloon/train/7178882742_f090f3ce56_k.jpg',channel_order='rgb')\n",
        "checkpoint_file = 'tutorial_exps/epoch_12.pth'\n",
        "model = init_detector(cfg, checkpoint_file, device='cpu')\n",
        "new_result = inference_detector(model, img)\n",
        "print(new_result)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "image/png": "[encoded data, 605944 characters]",
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from mmengine.visualization import Visualizer\n",
        "# get built visualizer\n",
        "visualizer_now = Visualizer.get_current_instance()\n",
        "# the dataset_meta is loaded from the checkpoint and\n",
        "# then pass to the model in init_detector\n",
        "visualizer_now.dataset_meta = model.dataset_meta\n",
        "# show the results\n",
        "visualizer_now.add_datasample(\n",
        "    'new_result',\n",
        "    img,\n",
        "    data_sample=new_result,\n",
        "    draw_gt=False,\n",
        "    wait_time=0,\n",
        "    out_file=None,\n",
        "    pred_score_thr=0.5\n",
        ")\n",
        "visualizer_now.show()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "6rzruCwFgPXm"
      },
      "source": [
        "## What to Do Next?\n",
        "\n",
        "So far, we have learnt how to test and train Mask R-CNN. To further explore the segmentation task, you could do several other things as shown below:\n",
        "\n",
        "- Try cascade methods, e.g., [Cascade Mask R-CNN](https://github.com/open-mmlab/mmdetection/tree/master/configs/cascade_rcnn) and [HTC](https://github.com/open-mmlab/mmdetection/tree/master/configs/htc) in [MMDetection model zoo](https://github.com/open-mmlab/mmdetection/blob/master/docs/en/model_zoo.md). They are powerful detectors that are ranked high in many benchmarks, e.g., COCO dataset.\n",
        "- Try single-stage methods, e.g., [K-Net](https://github.com/ZwwWayne/K-Net) and [Dense-RepPoints](https://github.com/justimyhxu/Dense-RepPoints). These two algorithms are based on MMDetection. Box-free instance segmentation is a new trend in the instance segmentation community.\n",
        "- Try semantic segmentation. Semantic segmentation is also a popular task with wide applications. You can explore [MMSegmentation](https://github.com/open-mmlab/mmsegmentation/); we also provide a [colab tutorial](https://github.com/open-mmlab/mmsegmentation/blob/master/demo/MMSegmentation_Tutorial.ipynb) for semantic segmentation using MMSegmentation.\n"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "name": "tutorial_03_image_segmentation_final.ipynb",
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3.7.12 ('mmlab2')",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.15"
    },
    "vscode": {
      "interpreter": {
        "hash": "8868640c17582ff5a3e06365ba2fb344ce697cf42d4745ae8b85a9738303c037"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
