mirror of
https://github.com/clearml/clearml-docs
synced 2025-01-30 22:18:02 +00:00
This commit is contained in:
parent
10c44dd4a1
commit
7388083880
@ -63,13 +63,13 @@ To configure a new app instance, click `Launch New` <img src="/docs/latest/icons
|
||||
to open the app's configuration form.
|
||||
|
||||
### Configuration Options
|
||||
* Import Configuration - Import an app instance configuration file. This will fill the configuration form with the
|
||||
* **Import Configuration** - Import an app instance configuration file. This will fill the configuration form with the
|
||||
values from the file, which can be modified before launching the app instance
|
||||
* Project name - ClearML Project where your Embedding Model Deployment app instance will be stored
|
||||
* Task name - Name of ClearML Task for your Embedding Model Deployment app instance
|
||||
* Queue - The [ClearML Queue](../../fundamentals/agents_and_queues.md#what-is-a-queue) to which the Embedding Model
|
||||
* **Project name** - ClearML Project where your Embedding Model Deployment app instance will be stored
|
||||
* **Task name** - Name of ClearML Task for your Embedding Model Deployment app instance
|
||||
* **Queue** - The [ClearML Queue](../../fundamentals/agents_and_queues.md#what-is-a-queue) to which the Embedding Model
|
||||
Deployment app instance task will be enqueued (make sure an agent is assigned to it)
|
||||
* Model Configuration
|
||||
* **Model Configuration**
|
||||
* Model - A ClearML Model ID or a Hugging Face model name (e.g. `openai-community/gpt2`)
|
||||
* Revision - The specific Hugging Face version of the model you want to use. You can use a specific commit ID or a
|
||||
branch like `refs/pr/2`
|
||||
@ -85,9 +85,9 @@ Deployment app instance task will be enqueued (make sure an agent is assigned to
|
||||
Transformer models
|
||||
* \+ Add item - Add another model endpoint. Each model will be accessible through the same base URL, with the model
|
||||
name appended to the URL.
|
||||
* Hugging Face Token - Token for accessing Hugging Face models that require authentication
|
||||
* Idle Time Limit (Hours) - Maximum idle time after which the app instance will shut down
|
||||
* Export Configuration - Export the app instance configuration as a JSON file, which you can later import to create a
|
||||
* **Hugging Face Token** - Token for accessing Hugging Face models that require authentication
|
||||
* **Idle Time Limit** (Hours) - Maximum idle time after which the app instance will shut down
|
||||
* **Export Configuration** - Export the app instance configuration as a JSON file, which you can later import to create a
|
||||
new instance with the same configuration
|
||||
|
||||
![Embedding Model Deployment form](../../img/apps_embedding_model_deployment_form.png)
|
@ -27,7 +27,7 @@ Once you have launched an app instance, you can view the following information i
|
||||
|
||||
![Jupyter Lab Dashboard](../../img/apps_jupyter_lab.png)
|
||||
|
||||
## App Instance Configuration
|
||||
## Jupyter Lab Instance Configuration
|
||||
When configuring a new Jupyter Lab instance, you can fill in the required parameters or reuse the configuration of
|
||||
a previously launched instance.
|
||||
|
||||
|
@ -45,7 +45,7 @@ After starting a llama.cpp Model Deployment instance, you can view the following
|
||||
|
||||
![llama deployment dashboard](../../img/apps_llama_dashboard.png)
|
||||
|
||||
## Llama.cpp Model Deployment Instance Configuration
|
||||
### Llama.cpp Model Deployment Instance Configuration
|
||||
|
||||
When configuring a new llama.cpp Model Deployment instance, you can fill in the required parameters or reuse the
|
||||
configuration of a previously launched instance.
|
||||
@ -60,19 +60,19 @@ To configure a new app instance, click `Launch New` <img src="/docs/latest/icons
|
||||
to open the app's configuration form.
|
||||
|
||||
## Configuration Options
|
||||
* Import Configuration - Import an app instance configuration file. This will fill the configuration form with the
|
||||
* **Import Configuration** - Import an app instance configuration file. This will fill the configuration form with the
|
||||
values from the file, which can be modified before launching the app instance
|
||||
* Project name - ClearML Project where your llama.cpp Model Deployment app instance will be stored
|
||||
* Task name - Name of [ClearML Task](../../fundamentals/task.md) for your llama.cpp Model Deployment app instance
|
||||
* Queue - The [ClearML Queue](../../fundamentals/agents_and_queues.md#agent-and-queue-workflow) to which the
|
||||
* **Project name** - ClearML Project where your llama.cpp Model Deployment app instance will be stored
|
||||
* **Task name** - Name of [ClearML Task](../../fundamentals/task.md) for your llama.cpp Model Deployment app instance
|
||||
* **Queue** - The [ClearML Queue](../../fundamentals/agents_and_queues.md#agent-and-queue-workflow) to which the
|
||||
llama.cpp Model Deployment app instance task will be enqueued (make sure an agent is assigned to it)
|
||||
* Model - A ClearML Model ID or a Hugging Face model. The model must be in GGUF format. If you are using a
|
||||
* **Model** - A ClearML Model ID or a Hugging Face model. The model must be in GGUF format. If you are using a
|
||||
HuggingFace model, make sure to pass the path to the GGUF file. For example: `provider/repo/path/to/model.gguf`
|
||||
* General
|
||||
* **General**
|
||||
* Hugging Face Token - Token for accessing Hugging Face models that require authentication
|
||||
* Number of GPU Layers - Number of layers to store in VRAM. `9999` indicates that all layers should be loaded in
|
||||
VRAM. Used to offload the model on the CPU RAM
|
||||
* Advanced Options
|
||||
* **Advanced Options**
|
||||
* Idle Time Limit (Hours) - Maximum idle time after which the app instance will shut down
|
||||
* Last Action Report Interval (Seconds) - The frequency at which the last activity made by the application is reported.
|
||||
Used to stop the application from entering an idle state when the machine metrics are low but the application is
|
||||
|
@ -64,14 +64,14 @@ To configure a new app instance, click `Launch New` <img src="/docs/latest/icons
|
||||
to open the app's configuration form.
|
||||
|
||||
### Configuration Options
|
||||
* Import Configuration - Import an app instance configuration file. This will fill the instance launch form with the
|
||||
* **Import Configuration** - Import an app instance configuration file. This will fill the instance launch form with the
|
||||
values from the file, which can be modified before launching the app instance
|
||||
* Project name - ClearML Project Name
|
||||
* Task name - Name of ClearML Task for your Model Deployment app instance
|
||||
* Queue - The [ClearML Queue](../../fundamentals/agents_and_queues.md#what-is-a-queue) to which the Model Deployment app
|
||||
* **Project name** - ClearML Project Name
|
||||
* **Task name** - Name of ClearML Task for your Model Deployment app instance
|
||||
* **Queue** - The [ClearML Queue](../../fundamentals/agents_and_queues.md#what-is-a-queue) to which the Model Deployment app
|
||||
instance task will be enqueued (make sure an agent is assigned to that queue)
|
||||
* Model - A ClearML Model ID or a HuggingFace model name (e.g. `openai-community/gpt2`)
|
||||
* Model Configuration
|
||||
* **Model** - A ClearML Model ID or a HuggingFace model name (e.g. `openai-community/gpt2`)
|
||||
* **Model Configuration**
|
||||
* Trust Remote Code - Select to set Hugging Face [`trust_remote_code`](https://huggingface.co/docs/text-generation-inference/main/en/reference/launcher#trustremotecode)
|
||||
to `true`.
|
||||
* Revision - The specific Hugging Face version of the model (i.e. weights) you want to use. You
|
||||
@ -85,7 +85,7 @@ instance task will be enqueued (make sure an agent is assigned to that queue)
|
||||
* Tokenizer Mode - Select the tokenizer mode:
|
||||
* `auto` - Uses the fast tokenizer if available
|
||||
* `slow` - Uses the slow tokenizer.
|
||||
* LoRA Configuration
|
||||
* **LoRA Configuration**
|
||||
* Enable LoRA - If checked, enable handling of [LoRA adapters](https://huggingface.co/docs/diffusers/en/training/lora#lora).
|
||||
* LoRA Modules - LoRA module configurations in the format `name=path`. Multiple modules can be specified.
|
||||
* Max LoRAs - Max number of LoRAs in a single batch.
|
||||
@ -98,7 +98,7 @@ instance task will be enqueued (make sure an agent is assigned to that queue)
|
||||
* `float32`
|
||||
* Max CPU LoRAs - Maximum number of LoRAs to store in CPU memory. Must be greater or equal to the
|
||||
`Max Number of Sequences` field in the General section below. Defaults to `Max Number of Sequences`.
|
||||
* General
|
||||
* **General**
|
||||
* Disable Log Stats - Disable logging statistics
|
||||
* Enforce Eager - Always use eager-mode PyTorch. If False, a hybrid of eager mode and CUDA graph will be used for
|
||||
maximal performance and flexibility.
|
||||
@ -140,8 +140,8 @@ instance task will be enqueued (make sure an agent is assigned to that queue)
|
||||
* Max Context Length to Capture - Maximum context length covered by CUDA graphs. When a sequence has context length
|
||||
larger than this, we fall back to eager mode.
|
||||
* Max Log Length - Max number of prompt characters or prompt ID numbers being printed in log. Default: unlimited
|
||||
* Idle Time Limit (Hours) - Maximum idle time after which the app instance will shut down
|
||||
* Export Configuration - Export the app instance configuration as a JSON file, which you can later import to create a
|
||||
* **Idle Time Limit** (Hours) - Maximum idle time after which the app instance will shut down
|
||||
* **Export Configuration** - Export the app instance configuration as a JSON file, which you can later import to create a
|
||||
new instance with the same configuration
|
||||
|
||||
![Model Deployment app form](../../img/apps_model_deployment_form.png)
|
@ -61,25 +61,25 @@ to open the app's instance launch form.
|
||||
|
||||
|
||||
### Configuration Options
|
||||
* Import Configuration - Import an app instance configuration file. This will fill the instance launch form with the
|
||||
* **Import Configuration** - Import an app instance configuration file. This will fill the instance launch form with the
|
||||
values from the file, which can be modified before launching the app instance
|
||||
* Git - The details for a git repository to optionally clone into your remote environment:
|
||||
* **Git** - The details for a git repository to optionally clone into your remote environment:
|
||||
* Repository
|
||||
* Branch
|
||||
* Commit
|
||||
* Docker - Input details to run the session in Docker container
|
||||
* **Docker** - Input details to run the session in Docker container
|
||||
* Image - Docker image to launch
|
||||
* Docker Arguments - Additional arguments for the Docker image
|
||||
* Init Script - Bash script that is executed upon container boot (comments are supported only at the beginning of the
|
||||
line)
|
||||
* Extra Packages - Specify Python packages to install when setting up the remote environment
|
||||
* Persistent Workspace Path - Specify your workspace root directory, it will be automatically stored when the session is
|
||||
* **Extra Packages** - Specify Python packages to install when setting up the remote environment
|
||||
* **Persistent Workspace Path** - Specify your workspace root directory, it will be automatically stored when the session is
|
||||
closed and restored into a new instance when the session app instance is cloned (example: `~/workspace`)
|
||||
* Queue - The [ClearML Queue](../../fundamentals/agents_and_queues.md#what-is-a-queue) to which the SSH Session app
|
||||
* **Queue** - The [ClearML Queue](../../fundamentals/agents_and_queues.md#what-is-a-queue) to which the SSH Session app
|
||||
instance task will be enqueued (make sure an agent is assigned to that queue)
|
||||
* Maximum idle time (in hours) - Maximum idle time after which the app instance will shut down
|
||||
* Interactive Session Name - Name your current interactive session
|
||||
* Advanced Options
|
||||
* **Maximum idle time** (in hours) - Maximum idle time after which the app instance will shut down
|
||||
* **Interactive Session Name** - Name your current interactive session
|
||||
* **Advanced Options**
|
||||
* Interactive Session Project - The project in which the interactive session is created. If left empty, the default
|
||||
project `Interactive Session` is used
|
||||
* Interactive Session Tags - Comma separated list of tags to add to your interactive session
|
||||
@ -89,7 +89,7 @@ closed and restored into a new instance when the session app instance is cloned
|
||||
* Idle Network Threshold (MB) - Network throughput under which the session will be considered idle
|
||||
* Idle CPU Threshold (%) - CPU utilization under which the session will be considered idle
|
||||
* Idle GPU Threshold (%) - GPU utilization under which the session will be considered idle
|
||||
* Export Configuration - Export the app instance configuration as a JSON file, which you can later import to create a
|
||||
* **Export Configuration** - Export the app instance configuration as a JSON file, which you can later import to create a
|
||||
new instance with the same configuration
|
||||
|
||||
![SSH Session form](../../img/apps_ssh_session_wizard.png)
|
Loading…
Reference in New Issue
Block a user