mirror of
https://github.com/clearml/clearml-docs
synced 2025-05-12 00:21:49 +00:00
Merge branch 'main' of https://github.com/allegroai/clearml-docs
This commit is contained in:
commit
d09a09a71c
@ -63,13 +63,13 @@ To configure a new app instance, click `Launch New` <img src="/docs/latest/icons
|
|||||||
to open the app's configuration form.
|
to open the app's configuration form.
|
||||||
|
|
||||||
### Configuration Options
|
### Configuration Options
|
||||||
* Import Configuration - Import an app instance configuration file. This will fill the configuration form with the
|
* **Import Configuration** - Import an app instance configuration file. This will fill the configuration form with the
|
||||||
values from the file, which can be modified before launching the app instance
|
values from the file, which can be modified before launching the app instance
|
||||||
* Project name - ClearML Project where your Embedding Model Deployment app instance will be stored
|
* **Project name** - ClearML Project where your Embedding Model Deployment app instance will be stored
|
||||||
* Task name - Name of ClearML Task for your Embedding Model Deployment app instance
|
* **Task name** - Name of ClearML Task for your Embedding Model Deployment app instance
|
||||||
* Queue - The [ClearML Queue](../../fundamentals/agents_and_queues.md#what-is-a-queue) to which the Embedding Model
|
* **Queue** - The [ClearML Queue](../../fundamentals/agents_and_queues.md#what-is-a-queue) to which the Embedding Model
|
||||||
Deployment app instance task will be enqueued (make sure an agent is assigned to it)
|
Deployment app instance task will be enqueued (make sure an agent is assigned to it)
|
||||||
* Model Configuration
|
* **Model Configuration**
|
||||||
* Model - A ClearML Model ID or a Hugging Face model name (e.g. `openai-community/gpt2`)
|
* Model - A ClearML Model ID or a Hugging Face model name (e.g. `openai-community/gpt2`)
|
||||||
* Revision - The specific Hugging Face version of the model you want to use. You can use a specific commit ID or a
|
* Revision - The specific Hugging Face version of the model you want to use. You can use a specific commit ID or a
|
||||||
branch like `refs/pr/2`
|
branch like `refs/pr/2`
|
||||||
@ -85,9 +85,9 @@ Deployment app instance task will be enqueued (make sure an agent is assigned to
|
|||||||
Transformer models
|
Transformer models
|
||||||
* \+ Add item - Add another model endpoint. Each model will be accessible through the same base URL, with the model
|
* \+ Add item - Add another model endpoint. Each model will be accessible through the same base URL, with the model
|
||||||
name appended to the URL.
|
name appended to the URL.
|
||||||
* Hugging Face Token - Token for accessing Hugging Face models that require authentication
|
* **Hugging Face Token** - Token for accessing Hugging Face models that require authentication
|
||||||
* Idle Time Limit (Hours) - Maximum idle time after which the app instance will shut down
|
* **Idle Time Limit** (Hours) - Maximum idle time after which the app instance will shut down
|
||||||
* Export Configuration - Export the app instance configuration as a JSON file, which you can later import to create a
|
* **Export Configuration** - Export the app instance configuration as a JSON file, which you can later import to create a
|
||||||
new instance with the same configuration
|
new instance with the same configuration
|
||||||
|
|
||||||

|

|
@ -27,7 +27,7 @@ Once you have launched an app instance, you can view the following information i
|
|||||||
|
|
||||||

|

|
||||||
|
|
||||||
## App Instance Configuration
|
## Jupyter Lab Instance Configuration
|
||||||
When configuring a new Jupyter Lab instance, you can fill in the required parameters or reuse the configuration of
|
When configuring a new Jupyter Lab instance, you can fill in the required parameters or reuse the configuration of
|
||||||
a previously launched instance.
|
a previously launched instance.
|
||||||
|
|
||||||
|
@ -45,7 +45,7 @@ After starting a llama.cpp Model Deployment instance, you can view the following
|
|||||||
|
|
||||||

|

|
||||||
|
|
||||||
## Llama.cpp Model Deployment Instance Configuration
|
### Llama.cpp Model Deployment Instance Configuration
|
||||||
|
|
||||||
When configuring a new llama.cpp Model Deployment instance, you can fill in the required parameters or reuse the
|
When configuring a new llama.cpp Model Deployment instance, you can fill in the required parameters or reuse the
|
||||||
configuration of a previously launched instance.
|
configuration of a previously launched instance.
|
||||||
@ -60,19 +60,19 @@ To configure a new app instance, click `Launch New` <img src="/docs/latest/icons
|
|||||||
to open the app's configuration form.
|
to open the app's configuration form.
|
||||||
|
|
||||||
## Configuration Options
|
## Configuration Options
|
||||||
* Import Configuration - Import an app instance configuration file. This will fill the configuration form with the
|
* **Import Configuration** - Import an app instance configuration file. This will fill the configuration form with the
|
||||||
values from the file, which can be modified before launching the app instance
|
values from the file, which can be modified before launching the app instance
|
||||||
* Project name - ClearML Project where your llama.cpp Model Deployment app instance will be stored
|
* **Project name** - ClearML Project where your llama.cpp Model Deployment app instance will be stored
|
||||||
* Task name - Name of [ClearML Task](../../fundamentals/task.md) for your llama.cpp Model Deployment app instance
|
* **Task name** - Name of [ClearML Task](../../fundamentals/task.md) for your llama.cpp Model Deployment app instance
|
||||||
* Queue - The [ClearML Queue](../../fundamentals/agents_and_queues.md#agent-and-queue-workflow) to which the
|
* **Queue** - The [ClearML Queue](../../fundamentals/agents_and_queues.md#agent-and-queue-workflow) to which the
|
||||||
llama.cpp Model Deployment app instance task will be enqueued (make sure an agent is assigned to it)
|
llama.cpp Model Deployment app instance task will be enqueued (make sure an agent is assigned to it)
|
||||||
* Model - A ClearML Model ID or a Hugging Face model. The model must be in GGUF format. If you are using a
|
* **Model** - A ClearML Model ID or a Hugging Face model. The model must be in GGUF format. If you are using a
|
||||||
HuggingFace model, make sure to pass the path to the GGUF file. For example: `provider/repo/path/to/model.gguf`
|
HuggingFace model, make sure to pass the path to the GGUF file. For example: `provider/repo/path/to/model.gguf`
|
||||||
* General
|
* **General**
|
||||||
* Hugging Face Token - Token for accessing Hugging Face models that require authentication
|
* Hugging Face Token - Token for accessing Hugging Face models that require authentication
|
||||||
* Number of GPU Layers - Number of layers to store in VRAM. `9999` indicates that all layers should be loaded in
|
* Number of GPU Layers - Number of layers to store in VRAM. `9999` indicates that all layers should be loaded in
|
||||||
VRAM. Used to offload the model on the CPU RAM
|
VRAM. Used to offload the model on the CPU RAM
|
||||||
* Advanced Options
|
* **Advanced Options**
|
||||||
* Idle Time Limit (Hours) - Maximum idle time after which the app instance will shut down
|
* Idle Time Limit (Hours) - Maximum idle time after which the app instance will shut down
|
||||||
* Last Action Report Interval (Seconds) - The frequency at which the last activity made by the application is reported.
|
* Last Action Report Interval (Seconds) - The frequency at which the last activity made by the application is reported.
|
||||||
Used to stop the application from entering an idle state when the machine metrics are low but the application is
|
Used to stop the application from entering an idle state when the machine metrics are low but the application is
|
||||||
|
@ -64,14 +64,14 @@ To configure a new app instance, click `Launch New` <img src="/docs/latest/icons
|
|||||||
to open the app's configuration form.
|
to open the app's configuration form.
|
||||||
|
|
||||||
### Configuration Options
|
### Configuration Options
|
||||||
* Import Configuration - Import an app instance configuration file. This will fill the instance launch form with the
|
* **Import Configuration** - Import an app instance configuration file. This will fill the instance launch form with the
|
||||||
values from the file, which can be modified before launching the app instance
|
values from the file, which can be modified before launching the app instance
|
||||||
* Project name - ClearML Project Name
|
* **Project name** - ClearML Project Name
|
||||||
* Task name - Name of ClearML Task for your Model Deployment app instance
|
* **Task name** - Name of ClearML Task for your Model Deployment app instance
|
||||||
* Queue - The [ClearML Queue](../../fundamentals/agents_and_queues.md#what-is-a-queue) to which the Model Deployment app
|
* **Queue** - The [ClearML Queue](../../fundamentals/agents_and_queues.md#what-is-a-queue) to which the Model Deployment app
|
||||||
instance task will be enqueued (make sure an agent is assigned to that queue)
|
instance task will be enqueued (make sure an agent is assigned to that queue)
|
||||||
* Model - A ClearML Model ID or a HuggingFace model name (e.g. `openai-community/gpt2`)
|
* **Model** - A ClearML Model ID or a HuggingFace model name (e.g. `openai-community/gpt2`)
|
||||||
* Model Configuration
|
* **Model Configuration**
|
||||||
* Trust Remote Code - Select to set Hugging Face [`trust_remote_code`](https://huggingface.co/docs/text-generation-inference/main/en/reference/launcher#trustremotecode)
|
* Trust Remote Code - Select to set Hugging Face [`trust_remote_code`](https://huggingface.co/docs/text-generation-inference/main/en/reference/launcher#trustremotecode)
|
||||||
to `true`.
|
to `true`.
|
||||||
* Revision - The specific Hugging Face version of the model (i.e. weights) you want to use. You
|
* Revision - The specific Hugging Face version of the model (i.e. weights) you want to use. You
|
||||||
@ -85,7 +85,7 @@ instance task will be enqueued (make sure an agent is assigned to that queue)
|
|||||||
* Tokenizer Mode - Select the tokenizer mode:
|
* Tokenizer Mode - Select the tokenizer mode:
|
||||||
* `auto` - Uses the fast tokenizer if available
|
* `auto` - Uses the fast tokenizer if available
|
||||||
* `slow` - Uses the slow tokenizer.
|
* `slow` - Uses the slow tokenizer.
|
||||||
* LoRA Configuration
|
* **LoRA Configuration**
|
||||||
* Enable LoRA - If checked, enable handling of [LoRA adapters](https://huggingface.co/docs/diffusers/en/training/lora#lora).
|
* Enable LoRA - If checked, enable handling of [LoRA adapters](https://huggingface.co/docs/diffusers/en/training/lora#lora).
|
||||||
* LoRA Modules - LoRA module configurations in the format `name=path`. Multiple modules can be specified.
|
* LoRA Modules - LoRA module configurations in the format `name=path`. Multiple modules can be specified.
|
||||||
* Max LoRAs - Max number of LoRAs in a single batch.
|
* Max LoRAs - Max number of LoRAs in a single batch.
|
||||||
@ -98,7 +98,7 @@ instance task will be enqueued (make sure an agent is assigned to that queue)
|
|||||||
* `float32`
|
* `float32`
|
||||||
* Max CPU LoRAs - Maximum number of LoRAs to store in CPU memory. Must be greater or equal to the
|
* Max CPU LoRAs - Maximum number of LoRAs to store in CPU memory. Must be greater or equal to the
|
||||||
`Max Number of Sequences` field in the General section below. Defaults to `Max Number of Sequences`.
|
`Max Number of Sequences` field in the General section below. Defaults to `Max Number of Sequences`.
|
||||||
* General
|
* **General**
|
||||||
* Disable Log Stats - Disable logging statistics
|
* Disable Log Stats - Disable logging statistics
|
||||||
* Enforce Eager - Always use eager-mode PyTorch. If False, a hybrid of eager mode and CUDA graph will be used for
|
* Enforce Eager - Always use eager-mode PyTorch. If False, a hybrid of eager mode and CUDA graph will be used for
|
||||||
maximal performance and flexibility.
|
maximal performance and flexibility.
|
||||||
@ -140,8 +140,8 @@ instance task will be enqueued (make sure an agent is assigned to that queue)
|
|||||||
* Max Context Length to Capture - Maximum context length covered by CUDA graphs. When a sequence has context length
|
* Max Context Length to Capture - Maximum context length covered by CUDA graphs. When a sequence has context length
|
||||||
larger than this, we fall back to eager mode.
|
larger than this, we fall back to eager mode.
|
||||||
* Max Log Length - Max number of prompt characters or prompt ID numbers being printed in log. Default: unlimited
|
* Max Log Length - Max number of prompt characters or prompt ID numbers being printed in log. Default: unlimited
|
||||||
* Idle Time Limit (Hours) - Maximum idle time after which the app instance will shut down
|
* **Idle Time Limit** (Hours) - Maximum idle time after which the app instance will shut down
|
||||||
* Export Configuration - Export the app instance configuration as a JSON file, which you can later import to create a
|
* **Export Configuration** - Export the app instance configuration as a JSON file, which you can later import to create a
|
||||||
new instance with the same configuration
|
new instance with the same configuration
|
||||||
|
|
||||||

|

|
@ -61,25 +61,25 @@ to open the app's instance launch form.
|
|||||||
|
|
||||||
|
|
||||||
### Configuration Options
|
### Configuration Options
|
||||||
* Import Configuration - Import an app instance configuration file. This will fill the instance launch form with the
|
* **Import Configuration** - Import an app instance configuration file. This will fill the instance launch form with the
|
||||||
values from the file, which can be modified before launching the app instance
|
values from the file, which can be modified before launching the app instance
|
||||||
* Git - The details for a git repository to optionally clone into your remote environment:
|
* **Git** - The details for a git repository to optionally clone into your remote environment:
|
||||||
* Repository
|
* Repository
|
||||||
* Branch
|
* Branch
|
||||||
* Commit
|
* Commit
|
||||||
* Docker - Input details to run the session in Docker container
|
* **Docker** - Input details to run the session in Docker container
|
||||||
* Image - Docker image to launch
|
* Image - Docker image to launch
|
||||||
* Docker Arguments - Additional arguments for the Docker image
|
* Docker Arguments - Additional arguments for the Docker image
|
||||||
* Init Script - Bash script that is executed upon container boot (comments are supported only at the beginning of the
|
* Init Script - Bash script that is executed upon container boot (comments are supported only at the beginning of the
|
||||||
line)
|
line)
|
||||||
* Extra Packages - Specify Python packages to install when setting up the remote environment
|
* **Extra Packages** - Specify Python packages to install when setting up the remote environment
|
||||||
* Persistent Workspace Path - Specify your workspace root directory, it will be automatically stored when the session is
|
* **Persistent Workspace Path** - Specify your workspace root directory, it will be automatically stored when the session is
|
||||||
closed and restored into a new instance when the session app instance is cloned (example: `~/workspace`)
|
closed and restored into a new instance when the session app instance is cloned (example: `~/workspace`)
|
||||||
* Queue - The [ClearML Queue](../../fundamentals/agents_and_queues.md#what-is-a-queue) to which the SSH Session app
|
* **Queue** - The [ClearML Queue](../../fundamentals/agents_and_queues.md#what-is-a-queue) to which the SSH Session app
|
||||||
instance task will be enqueued (make sure an agent is assigned to that queue)
|
instance task will be enqueued (make sure an agent is assigned to that queue)
|
||||||
* Maximum idle time (in hours) - Maximum idle time after which the app instance will shut down
|
* **Maximum idle time** (in hours) - Maximum idle time after which the app instance will shut down
|
||||||
* Interactive Session Name - Name your current interactive session
|
* **Interactive Session Name** - Name your current interactive session
|
||||||
* Advanced Options
|
* **Advanced Options**
|
||||||
* Interactive Session Project - The project in which the interactive session is created. If left empty, the default
|
* Interactive Session Project - The project in which the interactive session is created. If left empty, the default
|
||||||
project `Interactive Session` is used
|
project `Interactive Session` is used
|
||||||
* Interactive Session Tags - Comma separated list of tags to add to your interactive session
|
* Interactive Session Tags - Comma separated list of tags to add to your interactive session
|
||||||
@ -89,7 +89,7 @@ closed and restored into a new instance when the session app instance is cloned
|
|||||||
* Idle Network Threshold (MB) - Network throughput under which the session will be considered idle
|
* Idle Network Threshold (MB) - Network throughput under which the session will be considered idle
|
||||||
* Idle CPU Threshold (%) - CPU utilization under which the session will be considered idle
|
* Idle CPU Threshold (%) - CPU utilization under which the session will be considered idle
|
||||||
* Idle GPU Threshold (%) - GPU utilization under which the session will be considered idle
|
* Idle GPU Threshold (%) - GPU utilization under which the session will be considered idle
|
||||||
* Export Configuration - Export the app instance configuration as a JSON file, which you can later import to create a
|
* **Export Configuration** - Export the app instance configuration as a JSON file, which you can later import to create a
|
||||||
new instance with the same configuration
|
new instance with the same configuration
|
||||||
|
|
||||||

|

|
Loading…
Reference in New Issue
Block a user