mirror of
https://github.com/clearml/clearml
synced 2025-01-31 09:07:00 +00:00
391 lines
12 KiB
Plaintext
391 lines
12 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"colab_type": "text",
|
|
"id": "RZiRah3QiR_G"
|
|
},
|
|
"source": [
|
|
"# Allegro ClearML logging example\n",
|
|
"\n",
|
|
"[](https://colab.research.google.com/github/allegroai/clearml/blob/master/examples/reporting/Allegro_Trains_logging_example.ipynb)\n",
|
|
"\n",
|
|
"This example introduces ClearML [Logger](https://allegro.ai/docs/logger.html) functionality. Logger is the ClearML console log and metric interface.\n",
|
|
"\n",
|
|
"You can find more reporting examples [here](https://github.com/allegroai/clearml/tree/master/examples/reporting)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {},
|
|
"colab_type": "code",
|
|
"id": "RbESVEV0jl3c"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"!pip install clearml\n",
|
|
"!pip install numpy"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"colab_type": "text",
|
|
"id": "8p9tkR5wue2x"
|
|
},
|
|
"source": [
|
|
"### Create a new Task\n",
|
|
"Create a new Task and get a Logger object for the Task.\n",
|
|
"\n",
|
|
"To create a new Task object, call the `Task.init` method providing it with `project_name` (the project name for the experiment) and `task_name` (the name of the experiment). When `Task.init` executes, a link to the Web UI Results page for the newly generated Task will be printed, and the Task will be updated in real time in the ClearML demo server.\n",
|
|
"\n",
|
|
"You can read about the `Task` class in the docs [here](https://allegro.ai/docs/task.html).\n",
|
|
"\n",
|
|
"After the Task is created, get a Logger for it."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {},
|
|
"colab_type": "code",
|
|
"id": "885DHN5SjsHy"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import numpy as np\n",
|
|
"\n",
|
|
"from clearml import Task\n",
|
|
"\n",
|
|
"# Start a new task\n",
|
|
"task = Task.init(project_name=\"Colab notebooks\", task_name=\"Explicit Logging\")\n",
|
|
"\n",
|
|
"# Get the task logger,\n",
|
|
"# You can also call Task.current_task().get_logger() from anywhere in your code.\n",
|
|
"logger = task.get_logger()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"colab_type": "text",
|
|
"id": "SADmtLH8uwhw"
|
|
},
|
|
"source": [
|
|
"### Explicit scalar logging\n",
|
|
"Use the [Logger.report_scalar](https://allegro.ai/docs/logger.html#clearml.logger.Logger.report_scalar) method to explicitly log scalars. Scalar plots appear in the Web UI, Results > Scalars tab."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {},
|
|
"colab_type": "code",
|
|
"id": "E6YH4lNLjLs8"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# report two scalar series on the same graph\n",
|
|
"for i in range(10):\n",
|
|
" logger.report_scalar(\"unified graph\", \"series A\", iteration=i, value=1./(i+1))\n",
|
|
" logger.report_scalar(\"unified graph\", \"series B\", iteration=i, value=10./(i+1))\n",
|
|
"\n",
|
|
"# report two scalar series on two different graphs\n",
|
|
"for i in range(10):\n",
|
|
" logger.report_scalar(\"graph A\", \"series A\", iteration=i, value=1./(i+1))\n",
|
|
" logger.report_scalar(\"graph B\", \"series B\", iteration=i, value=10./(i+1))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"colab_type": "text",
|
|
"id": "8iqvizEzwRtk"
|
|
},
|
|
"source": [
|
|
"### Explicit logging of other data\n",
|
|
"\n",
|
|
"You can log other data and report the data in a variety of plot types, including histograms, confusion matrices, 2D and 3D scatter diagrams, and surface diagrams. They appear in the Results > Plots tab.\n",
|
|
"\n",
|
|
"For information about the methods to report each type of plot, see the [Logger](https://allegro.ai/docs/logger.html) module.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {},
|
|
"colab_type": "code",
|
|
"id": "_rWW7HTYjLtK"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"iteration = 100\n",
|
|
"\n",
|
|
"# report a single histogram\n",
|
|
"histogram = np.random.randint(10, size=10)\n",
|
|
"logger.report_histogram(\n",
|
|
" \"single_histogram\",\n",
|
|
" \"random histogram\",\n",
|
|
" iteration=iteration,\n",
|
|
" values=histogram,\n",
|
|
" xaxis=\"title x\",\n",
|
|
" yaxis=\"title y\",\n",
|
|
")\n",
|
|
"\n",
|
|
"# report a two histograms on the same plot\n",
|
|
"histogram1 = np.random.randint(13, size=10)\n",
|
|
"histogram2 = histogram * 0.75\n",
|
|
"logger.report_histogram(\n",
|
|
" \"two_histogram\",\n",
|
|
" \"series 1\",\n",
|
|
" iteration=iteration,\n",
|
|
" values=histogram1,\n",
|
|
" xaxis=\"title x\",\n",
|
|
" yaxis=\"title y\",\n",
|
|
")\n",
|
|
"logger.report_histogram(\n",
|
|
" \"two_histogram\",\n",
|
|
" \"series 2\",\n",
|
|
" iteration=iteration,\n",
|
|
" values=histogram2,\n",
|
|
" xaxis=\"title x\",\n",
|
|
" yaxis=\"title y\",\n",
|
|
")\n",
|
|
"\n",
|
|
"# report confusion matrix\n",
|
|
"confusion = np.random.randint(10, size=(10, 10))\n",
|
|
"logger.report_matrix(\n",
|
|
" \"example_confusion\",\n",
|
|
" \"ignored\",\n",
|
|
" iteration=iteration,\n",
|
|
" matrix=confusion,\n",
|
|
" xaxis=\"title X\",\n",
|
|
" yaxis=\"title Y\",\n",
|
|
")\n",
|
|
"\n",
|
|
"scatter2d = np.hstack(\n",
|
|
" (np.atleast_2d(np.arange(0, 10)).T, np.random.randint(10, size=(10, 1)))\n",
|
|
")\n",
|
|
"# report 2d scatter plot with markers\n",
|
|
"logger.report_scatter2d(\n",
|
|
" \"example_scatter\",\n",
|
|
" \"series_lines+markers\",\n",
|
|
" iteration=iteration,\n",
|
|
" scatter=scatter2d,\n",
|
|
" xaxis=\"title x\",\n",
|
|
" yaxis=\"title y\",\n",
|
|
" mode='lines+markers'\n",
|
|
")\n",
|
|
"\n",
|
|
"# report 3d surface\n",
|
|
"surface = np.random.randint(10, size=(10, 10))\n",
|
|
"logger.report_surface(\n",
|
|
" \"example_surface\",\n",
|
|
" \"series1\",\n",
|
|
" iteration=iteration,\n",
|
|
" matrix=surface,\n",
|
|
" xaxis=\"title X\",\n",
|
|
" yaxis=\"title Y\",\n",
|
|
" zaxis=\"title Z\",\n",
|
|
")\n",
|
|
"\n",
|
|
"# report 3d scatter plot\n",
|
|
"scatter3d = np.random.randint(10, size=(10, 3))\n",
|
|
"logger.report_scatter3d(\n",
|
|
" \"example_scatter_3d\",\n",
|
|
" \"series_xyz\",\n",
|
|
" iteration=iteration,\n",
|
|
" scatter=scatter3d,\n",
|
|
" xaxis=\"title x\",\n",
|
|
" yaxis=\"title y\",\n",
|
|
" zaxis=\"title z\",\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"colab_type": "text",
|
|
"id": "OZqPEaFRwcVY"
|
|
},
|
|
"source": [
|
|
"### Explicit debug samples reporting\n",
|
|
"\n",
|
|
"Explicitly report debug samples, including images, audio, and video."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### Downloading the files\n",
|
|
"\n",
|
|
"We use StorageManager to download a local copy of the files. You can use it immediately. Just provide the URL. Cache is enabled by default for all downloaded remote URLs/files.\n",
|
|
"\n",
|
|
"For more information, you can read about the storage manager [here](https://allegro.ai/docs/storage_manager_storagemanager.html)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {},
|
|
"colab_type": "code",
|
|
"id": "s4rf0ap0jLtb"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from clearml.storage import StorageManager\n",
|
|
"image_local_copy = StorageManager.get_local_copy(\n",
|
|
" remote_url=\"https://pytorch.org/tutorials/_static/img/neural-style/picasso.jpg\",\n",
|
|
" name=\"picasso.jpg\"\n",
|
|
")\n",
|
|
"\n",
|
|
"print(\"Image location: {}\".format(image_local_copy))\n",
|
|
"\n",
|
|
"video_local_copy = StorageManager.get_local_copy(\n",
|
|
" remote_url=\"https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/720/Big_Buck_Bunny_720_10s_1MB.mp4\",\n",
|
|
" name=\"Big_Buck_Bunny_720_10s_1MB.mp4\"\n",
|
|
")\n",
|
|
"\n",
|
|
"print(\"Video location: {}\".format(video_local_copy))\n",
|
|
"\n",
|
|
"audio_local_copy = StorageManager.get_local_copy(\n",
|
|
" remote_url=\"https://www2.cs.uic.edu/~i101/SoundFiles/PinkPanther30.wav\",\n",
|
|
" name=\"PinkPanther30.wav\"\n",
|
|
")\n",
|
|
"\n",
|
|
"print(\"Audio location: {}\".format(audio_local_copy))\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"colab_type": "text",
|
|
"id": "vydmi7HWw0gS"
|
|
},
|
|
"source": [
|
|
"#### Report images and media\n",
|
|
"\n",
|
|
"Use [Logger.report_image](https://allegro.ai/docs/logger.html?highlight=report_image#clearml.logger.Logger.report_image) and [Logger.report_media](https://allegro.ai/docs/logger.html?highlight=report_media#clearml.logger.Logger.report_media) to report the downloaded samples. The debug samples appear in the Results > Debug Samples tab."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {},
|
|
"colab_type": "code",
|
|
"id": "MtEhbE4S_P66"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"logger.report_image(\"image\", \"image from url\", iteration=100, local_path=image_local_copy)\n",
|
|
"\n",
|
|
"# Image can be uploaded via 'report_media' too\n",
|
|
"# report video, an already uploaded video media (url)\n",
|
|
"logger.report_media(\n",
|
|
" 'video', 'big bunny', iteration=1, local_path=video_local_copy)\n",
|
|
"\n",
|
|
"\n",
|
|
"# This will actually use the cache and will not download the file again.\n",
|
|
"audio_local_copy_cache = StorageManager.get_local_copy(\n",
|
|
" remote_url=\"https://www2.cs.uic.edu/~i101/SoundFiles/PinkPanther30.wav\",\n",
|
|
" name=\"PinkPanther30.wav\"\n",
|
|
")\n",
|
|
"\n",
|
|
"# report audio, report an already uploaded audio media (url)\n",
|
|
"logger.report_media(\n",
|
|
" 'audio', 'pink panther', iteration=1, local_path=audio_local_copy)\n",
|
|
"\n",
|
|
"# reporting html from url to debug samples section\n",
|
|
"logger.report_media(\"html\", \"url_html\", iteration=1, url=\"https://allegro.ai/docs/index.html\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"colab_type": "text",
|
|
"id": "uDrcRCJxxCQP"
|
|
},
|
|
"source": [
|
|
"### Explicit text logging\n",
|
|
"Use [Logger.report_text](https://allegro.ai/docs/logger.html?highlight=report_text#clearml.logger.Logger.report_text) to log text message. They appear in Results > Log."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {},
|
|
"colab_type": "code",
|
|
"id": "a2UlIETAjLtk"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# report text\n",
|
|
"logger.report_text(\"hello, this is plain text\")\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"colab_type": "text",
|
|
"id": "aNFDbjZ7xNco"
|
|
},
|
|
"source": [
|
|
"### Flushing the reports\n",
|
|
"\n",
|
|
"Reports are flushed in the background every couple of seconds, and at the end of the process execution.\n",
|
|
"\n",
|
|
"Or, flush the Logger by calling [Logger.flush](https://allegro.ai/docs/logger.html?highlight=report_text#clearml.logger.Logger.flush)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {},
|
|
"colab_type": "code",
|
|
"id": "SSyGUskJjLtr"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"logger.flush()"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"colab": {
|
|
"collapsed_sections": [],
|
|
"name": "Allegro ClearML logging example.ipynb",
|
|
"provenance": []
|
|
},
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.6.5"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 1
|
|
}
|