clearml/examples/reporting/Allegro_Trains_logging_example.ipynb

{
 "nbformat": 4,
 "nbformat_minor": 0,
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "colab": {
   "name": "Allegro Trains logging example.ipynb",
   "provenance": [],
   "collapsed_sections": []
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RZiRah3QiR_G",
    "colab_type": "text"
   },
   "source": [
    "# Allegro Trains logging example\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/allegroai/trains/blob/master/examples/reporting/Allegro_Trains_logging_example.ipynb)\n",
    "\n",
    "This tutorial introduce [Trains](https://github.com/allegroai/trains) logger functionality. You can find more reporting examples [here](https://github.com/allegroai/trains/tree/master/examples/reporting)."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "RbESVEV0jl3c",
    "colab_type": "code",
    "colab": {}
   },
   "source": [
    "!pip install trains\n",
    "!pip install numpy"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8p9tkR5wue2x",
    "colab_type": "text"
   },
   "source": [
    "### Create a new task\n",
    "Create a new task and get the logger object for this task.\n",
    "\n",
    "Task object should be provided `project_name` (the project name for the experiment) and `task_name` (the name of the experiment).\n",
    "A link to the newly generated task will be printed and the task will be updated real time in the Trains demo server.\n",
    "\n",
    "Logger is the Trains console log and metric interface.\n",
    "You can read about the logger in the [docs](https://allegro.ai/docs/logger.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "885DHN5SjsHy",
    "colab_type": "code",
    "colab": {}
   },
   "source": [
    "import numpy as np\n",
    "\n",
    "from trains import Task\n",
    "\n",
    "# Start a new task\n",
    "task = Task.init(project_name=\"Colab notebooks\", task_name=\"Explicit Logging\")\n",
    "\n",
    "# Get the task logger,\n",
    "# You can also call Task.current_task().get_logger() from anywhere in your code.\n",
    "logger = task.get_logger()"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SADmtLH8uwhw",
    "colab_type": "text"
   },
   "source": [
    "### Explicit scalar logging\n",
    "Explicit scalar logging. Data available in the task 'Results' section.\n",
    "For more [usage examples](https://allegro.ai/docs/logger.html#trains.logger.Logger.report_scalar)"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "E6YH4lNLjLs8",
    "colab_type": "code",
    "colab": {}
   },
   "source": [
    "# report two scalar series on the same graph\n",
    "for i in range(10):\n",
    "    logger.report_scalar(\"unified graph\", \"series A\", iteration=i, value=1./(i+1))\n",
    "    logger.report_scalar(\"unified graph\", \"series B\", iteration=i, value=10./(i+1))\n",
    "\n",
    "# report two scalar series on two different graphs\n",
    "for i in range(10):\n",
    "    logger.report_scalar(\"graph A\", \"series A\", iteration=i, value=1./(i+1))\n",
    "    logger.report_scalar(\"graph B\", \"series B\", iteration=i, value=10./(i+1))"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8iqvizEzwRtk",
    "colab_type": "text"
   },
   "source": [
    "### Explicit histogram logging\n",
    "Explicit histogram logging. Data available in the task 'Results' section.\n",
    "You can report histograms, matrix, 3D scatter diagrams and surface.\n",
    "For more examples check [here](https://allegro.ai/docs/logger.html#trains.logger.Logger.report_histogram)\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "_rWW7HTYjLtK",
    "colab_type": "code",
    "colab": {}
   },
   "source": [
    "iteration = 100\n",
    "\n",
    "# report a single histogram\n",
    "histogram = np.random.randint(10, size=10)\n",
    "logger.report_histogram(\n",
    "    \"single_histogram\",\n",
    "    \"random histogram\",\n",
    "    iteration=iteration,\n",
    "    values=histogram,\n",
    "    xaxis=\"title x\",\n",
    "    yaxis=\"title y\",\n",
    ")\n",
    "\n",
    "# report a two histograms on the same plot\n",
    "histogram1 = np.random.randint(13, size=10)\n",
    "histogram2 = histogram * 0.75\n",
    "logger.report_histogram(\n",
    "    \"two_histogram\",\n",
    "    \"series 1\",\n",
    "    iteration=iteration,\n",
    "    values=histogram1,\n",
    "    xaxis=\"title x\",\n",
    "    yaxis=\"title y\",\n",
    ")\n",
    "logger.report_histogram(\n",
    "    \"two_histogram\",\n",
    "    \"series 2\",\n",
    "    iteration=iteration,\n",
    "    values=histogram2,\n",
    "    xaxis=\"title x\",\n",
    "    yaxis=\"title y\",\n",
    ")\n",
    "\n",
    "# report confusion matrix\n",
    "confusion = np.random.randint(10, size=(10, 10))\n",
    "logger.report_matrix(\n",
    "    \"example_confusion\",\n",
    "    \"ignored\",\n",
    "    iteration=iteration,\n",
    "    matrix=confusion,\n",
    "    xaxis=\"title X\",\n",
    "    yaxis=\"title Y\",\n",
    ")\n",
    "\n",
    "scatter2d = np.hstack(\n",
    "    (np.atleast_2d(np.arange(0, 10)).T, np.random.randint(10, size=(10, 1)))\n",
    ")\n",
    "# report 2d scatter plot with markers\n",
    "logger.report_scatter2d(\n",
    "    \"example_scatter\",\n",
    "    \"series_lines+markers\",\n",
    "    iteration=iteration,\n",
    "    scatter=scatter2d,\n",
    "    xaxis=\"title x\",\n",
    "    yaxis=\"title y\",\n",
    "    mode='lines+markers'\n",
    ")\n",
    "\n",
    "# report 3d surface\n",
    "surface = np.random.randint(10, size=(10, 10))\n",
    "logger.report_surface(\n",
    "    \"example_surface\",\n",
    "    \"series1\",\n",
    "    iteration=iteration,\n",
    "    matrix=surface,\n",
    "    xaxis=\"title X\",\n",
    "    yaxis=\"title Y\",\n",
    "    zaxis=\"title Z\",\n",
    ")\n",
    "\n",
    "# report 3d scatter plot\n",
    "scatter3d = np.random.randint(10, size=(10, 3))\n",
    "logger.report_scatter3d(\n",
    "    \"example_scatter_3d\",\n",
    "    \"series_xyz\",\n",
    "    iteration=iteration,\n",
    "    scatter=scatter3d,\n",
    "    xaxis=\"title x\",\n",
    "    yaxis=\"title y\",\n",
    "    zaxis=\"title z\",\n",
    ")"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OZqPEaFRwcVY",
    "colab_type": "text"
   },
   "source": [
    "### Explicit debug samples reporting\n",
    "Explicit debug samples reporting. Data available in the task 'Results' section\n",
    "\n",
    "We are using StorageManager to download a local copy of the files. \n",
    "You can use the StorageManager immediately, you only need to provide the url. \n",
    "Cache is enabled by default for all downloaded remote urls/files.\n",
    "\n",
    "For more information, you can read about the storage manager [here](https://allegro.ai/docs/storage.html#trains.storage.manager.StorageManager)\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "s4rf0ap0jLtb",
    "colab_type": "code",
    "colab": {}
   },
   "source": [
    "from trains.storage import StorageManager\n",
    "image_local_copy = StorageManager.get_local_copy(\n",
    "    remote_url=\"https://pytorch.org/tutorials/_static/img/neural-style/picasso.jpg\",\n",
    "    name=\"picasso.jpg\"\n",
    ")\n",
    "\n",
    "print(\"Image location: {}\".format(image_local_copy))\n",
    "\n",
    "video_local_copy = StorageManager.get_local_copy(\n",
    "    remote_url=\"https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/720/Big_Buck_Bunny_720_10s_1MB.mp4\",\n",
    "    name=\"Big_Buck_Bunny_720_10s_1MB.mp4\"\n",
    ")\n",
    "\n",
    "print(\"Video location: {}\".format(video_local_copy))\n",
    "\n",
    "audio_local_copy = StorageManager.get_local_copy(\n",
    "    remote_url=\"https://www2.cs.uic.edu/~i101/SoundFiles/PinkPanther30.wav\",\n",
    "    name=\"PinkPanther30.wav\"\n",
    ")\n",
    "\n",
    "print(\"Audio location: {}\".format(audio_local_copy))\n"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vydmi7HWw0gS",
    "colab_type": "text"
   },
   "source": [
    "### Report images and media\n",
    "Reporting the downloaded samples. Data available in the task 'Results' section.\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "MtEhbE4S_P66",
    "colab_type": "code",
    "colab": {}
   },
   "source": [
    "logger.report_image(\"image\", \"image from url\", iteration=100, local_path=image_local_copy)\n",
    "\n",
    "# Image can be uploaded via 'report_media' too\n",
    "# report video, an already uploaded video media (url)\n",
    "logger.report_media(\n",
    "    'video', 'big bunny', iteration=1, local_path=video_local_copy)\n",
    "\n",
    "\n",
    "# This will actually use the cache and will not download the file again.\n",
    "audio_local_copy_cache = StorageManager.get_local_copy(\n",
    "    remote_url=\"https://www2.cs.uic.edu/~i101/SoundFiles/PinkPanther30.wav\",\n",
    "    name=\"PinkPanther30.wav\"\n",
    ")\n",
    "\n",
    "# report audio, report an already uploaded audio media (url)\n",
    "logger.report_media(\n",
    "    'audio', 'pink panther', iteration=1, local_path=audio_local_copy)\n",
    "\n",
    "# reporting html from url to debug samples section\n",
    "logger.report_media(\"html\", \"url_html\", iteration=1, url=\"https://allegro.ai/docs/index.html\")"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "uDrcRCJxxCQP",
    "colab_type": "text"
   },
   "source": [
    "### Explicit text logging\n",
    "Explicit text logging. Data available in the task 'Results' section.\n",
    "For more examples check [here](https://allegro.ai/docs/logger.html?highlight=report_text#trains.logger.Logger.report_text)\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "a2UlIETAjLtk",
    "colab_type": "code",
    "colab": {}
   },
   "source": [
    "# report text\n",
    "logger.report_text(\"hello, this is plain text\")\n"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aNFDbjZ7xNco",
    "colab_type": "text"
   },
   "source": [
    "### Flushing the reports\n",
    "If flush is not called, reports are flushed in the background every couple of seconds, \n",
    "and at the end of the process execution.\n",
    "\n",
    "More information can be found [here](https://allegro.ai/docs/logger.html?highlight=report_text#trains.logger.Logger.flush)\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "SSyGUskJjLtr",
    "colab_type": "code",
    "colab": {}
   },
   "source": [
    "logger.flush()"
   ],
   "execution_count": null,
   "outputs": []
  }
 ]
}