ClearML/clearml-server
1
0
Fork 0
mirror of https://github.com/clearml/clearml-server synced 2025-06-23 08:45:30 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Upgrade to v0.12

Browse Source
This commit is contained in:
allegroai 2019-10-29 20:43:46 +02:00
parent 18c8dd449d
commit 1e701becd3
3 changed files with 157 additions and 96 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

69
README.md
View File

@ -77,6 +77,7 @@ Latest docker images can be found [here](https://hub.docker.com/r/allegroai/trai
sudo mkdir -p /opt/trains/data/elastic sudo mkdir -p /opt/trains/data/elastic
sudo mkdir -p /opt/trains/data/mongo/db sudo mkdir -p /opt/trains/data/mongo/db
sudo mkdir -p /opt/trains/data/mongo/configdb sudo mkdir -p /opt/trains/data/mongo/configdb
sudo mkdir -p /opt/trains/data/redis
sudo mkdir -p /opt/trains/logs sudo mkdir -p /opt/trains/logs
sudo mkdir -p /opt/trains/data/fileserver sudo mkdir -p /opt/trains/data/fileserver
sudo mkdir -p /opt/trains/config sudo mkdir -p /opt/trains/config
@ -221,15 +222,29 @@ We are constantly updating, improving and adding to the **trains-server**.
New releases will include new pre-built Docker images. New releases will include new pre-built Docker images.
When we release a new version and include a new pre-built Docker image for it, upgrade as follows: When we release a new version and include a new pre-built Docker image for it, upgrade as follows:
1. Shut down and remove each of your Docker instances using the following commands: * Upgrading your docker-compose installation
* Using Docker-Compose * Shut down the docker containers
```bash
$ docker-compose down
```
```bash * We highly recommend backing up your data directory before upgrading
$ docker-compose down (see **Step ii** in the Manual Docker upgrade)
```
* Manual Docker launching * Spin up the docker containers, it will automatically pull the latest trains-server build
```bash
$ docker-compose up
```
* In case of a docker error: "... The container name "/trains-???" is already in use by ..."
Try removing deprecated images with:
```bash
$ docker rm -f $(docker ps -a -q)
```
* Manual Docker upgrade
1. Shut down and remove each of your Docker instances using the following commands:
```bash ```bash
$ sudo docker stop <docker-name> $ sudo docker stop <docker-name>
@ -240,37 +255,39 @@ When we release a new version and include a new pre-built Docker image for it, u
* `trains-elastic` * `trains-elastic`
* `trains-mongo` * `trains-mongo`
* `trains-redis`
* `trains-fileserver` * `trains-fileserver`
* `trains-apiserver` * `trains-apiserver`
* `trains-webserver` * `trains-webserver`
2. We highly recommend backing up your data directory!. A simple way to do that is using `tar`: 2. We highly recommend backing up your data directory!. A simple way to do that is using `tar`:
For example, if your data directory is `/opt/trains`, use the following command: For example, if your data directory is `/opt/trains`, use the following command:
```bash ```bash
$ sudo tar czvf ~/trains_backup.tgz /opt/trains/data $ sudo tar czvf ~/trains_backup.tgz /opt/trains/data
``` ```
This backups all data to an archive in your home directory. This backups all data to an archive in your home directory.
To restore this example backup, use the following command: To restore this example backup, use the following command:
```bash ```bash
$ sudo rm -R /opt/trains/data $ sudo rm -R /opt/trains/data
$ sudo tar -xzf ~/trains_backup.tgz -C /opt/trains/data $ sudo tar -xzf ~/trains_backup.tgz -C /opt/trains/data
``` ```
3. Pull the new **trains-server** docker image using the following command: 3. Pull the new **trains-server** docker image using the following command:
```bash ```bash
$ sudo docker pull allegroai/trains:latest $ sudo docker pull allegroai/trains:latest
``` ```
If you wish to pull a different version, replace `latest` with the required version number, for example: If you wish to pull a different version, replace `latest` with the required version number, for example:
```bash ```bash
$ sudo docker pull allegroai/trains:0.10.1 $ sudo docker pull allegroai/trains:0.11.0
``` ```
4. Launch the newly released Docker image (see [Launching Docker Containers](#launch-docker)).
4. Launch the newly released Docker image (see [Launching Docker Containers](#launch-docker)).
## Community & Support ## Community & Support

5
docs/docker_setup.md
View File

@ -61,6 +61,7 @@ For example, if your data directory is `/opt/trains`, then use the following com
sudo mkdir -p /opt/trains/data/elastic sudo mkdir -p /opt/trains/data/elastic
sudo mkdir -p /opt/trains/data/mongo/db sudo mkdir -p /opt/trains/data/mongo/db
sudo mkdir -p /opt/trains/data/mongo/configdb sudo mkdir -p /opt/trains/data/mongo/configdb
sudo mkdir -p /opt/trains/data/redis
sudo mkdir -p /opt/trains/logs sudo mkdir -p /opt/trains/logs
sudo mkdir -p /opt/trains/data/fileserver sudo mkdir -p /opt/trains/data/fileserver
sudo mkdir -p /opt/trains/config sudo mkdir -p /opt/trains/config
@ -82,6 +83,10 @@ If your data directory is not `/opt/trains`, then in the five `docker run` comma
sudo docker run -d --restart="always" --name="trains-mongo" -v /opt/trains/data/mongo/db:/data/db -v /opt/trains/data/mongo/configdb:/data/configdb --network="host" mongo:3.6.5 sudo docker run -d --restart="always" --name="trains-mongo" -v /opt/trains/data/mongo/db:/data/db -v /opt/trains/data/mongo/configdb:/data/configdb --network="host" mongo:3.6.5
1. Launch the **trains-redis** Docker container.
sudo docker run -d --restart="always" --name="trains-redis" -v /opt/trains/data/redis:/data --network="host" redis:5.0
1. Launch the **trains-fileserver** Docker container. 1. Launch the **trains-fileserver** Docker container.
sudo docker run -d --restart="always" --name="trains-fileserver" --network="host" -v /opt/trains/logs:/var/log/trains -v /opt/trains/data/fileserver:/mnt/fileserver allegroai/trains:latest fileserver sudo docker run -d --restart="always" --name="trains-fileserver" --network="host" -v /opt/trains/logs:/var/log/trains -v /opt/trains/data/fileserver:/mnt/fileserver allegroai/trains:latest fileserver

71
docs/install_aws.md
View File

@ -21,6 +21,27 @@ The minimum recommended instance type is **t3a.large**
In order to upgrade **trains-server** on an existing EC2 instance based on one of these AMIs, SSH into the instance and follow the [upgrade instructions](../README.md#upgrade) for **trains-server**. In order to upgrade **trains-server** on an existing EC2 instance based on one of these AMIs, SSH into the instance and follow the [upgrade instructions](../README.md#upgrade) for **trains-server**.
### Upgrading AMI's to v0.12
**Including the automatically updated AMI**
Version 0.12 introduced an additional REDIS docker to the trains-server setup.
AMI upgrading instructions:
1. SSH to the EC2 machine running one of the `Latest Version AMI's`
2. Execute the following bash commands
```bash
sudo bash
echo "" >> /usr/bin/start_or_update_server.sh
echo "sudo mkdir -p \${datadir}/redis" >> /usr/bin/start_or_update_server.sh
echo "sudo docker stop trains-redis || true && sudo docker rm -v trains-redis || true" >> /usr/bin/start_or_update_server.sh
echo "echo never | sudo tee -a /sys/kernel/mm/transparent_hugepage/enabled" >> /usr/bin/start_or_update_server.sh
echo "sudo sysctl vm.overcommit_memory=1" >> /usr/bin/start_or_update_server.sh
echo "sudo docker run -d --restart=always --name=trains-redis -v \${datadir}/redis:/data --network=host redis:5 redis-server" >> /usr/bin/start_or_update_server.sh
```
3. Reboot the EC2 machine
## Released versions ## Released versions
The following sections provide a list containing AMI Image ID per region for each released **trains-server** version. The following sections provide a list containing AMI Image ID per region for each released **trains-server** version.
@ -28,22 +49,40 @@ The following sections provide a list containing AMI Image ID per region for eac
### Latest Version AMI <a name="autoupdate"></a> ### Latest Version AMI <a name="autoupdate"></a>
**For easier upgrades: The following AMI automatically update to the latest release every reboot** **For easier upgrades: The following AMI automatically update to the latest release every reboot**
* **eu-north-1** : ami-05d0d39ba39c93781 * **eu-north-1** : ami-072aef14041e70651
* **ap-south-1** : ami-01ae99e1c27e0490a * **ap-south-1** : ami-08032d881daca4de1
* **eu-west-3** : ami-01b156f8c7dd38121 * **eu-west-3** : ami-0b39c123d4343d408
* **eu-west-2** : ami-01b80d5a23b8847fb * **eu-west-2** : ami-0e0fe6fd14b2e9029
* **eu-west-1** : ami-0524891495168c944 * **eu-west-1** : ami-087c81e06d722e938
* **ap-northeast-2** : ami-0594f00619bea922f * **ap-northeast-2** : ami-0caf74f03322b994c
* **ap-northeast-1** : ami-0d97b860be6f71a9f * **ap-northeast-1** : ami-0f723b3d49c0f2749
* **sa-east-1** : ami-0b0889651918730b8 * **sa-east-1** : ami-0ac5595ad0e106502
* **ca-central-1** : ami-040c641b2f71082b1 * **ca-central-1** : ami-053049b463869469a
* **ap-southeast-1** : ami-00a57be01d39ff964 * **ap-southeast-1** : ami-0b440ec389d6ff541
* **ap-southeast-2** : ami-066dcf2cc155b6ec1 * **ap-southeast-2** : ami-02af978ddc2c15b71
* **eu-central-1** : ami-0bb64c4bdecebc0a9 * **eu-central-1** : ami-09ef364aa8df29760
* **us-east-2** : ami-04addd0766ebb8f46 * **us-east-2** : ami-02e33f8ab77071509
* **us-west-1** : ami-0ea895789568bb537 * **us-west-1** : ami-0ff33f256907fd460
* **us-west-2** : ami-07ae3d0dedfdb2278 * **us-west-2** : ami-0387728fb09c8cda7
* **us-east-1** : ami-07fe3993427800995 * **us-east-1** : ami-02c47c5233eed7f88
### v0.12.0
* **eu-north-1** : ami-0ebb4bb8637d0da65
* **ap-south-1** : ami-0fb3c89eb8a8fc294
* **eu-west-3** : ami-0b55ea4a6698d5875
* **eu-west-2** : ami-02979b6d77856b842
* **eu-west-1** : ami-07f4c17a636489574
* **ap-northeast-2** : ami-06071092427dd5ab4
* **ap-northeast-1** : ami-0fbacddfc0e8d2651
* **sa-east-1** : ami-073590d3b3e6f4cfd
* **ca-central-1** : ami-0839610fc0101e41c
* **ap-southeast-1** : ami-0ff0adeef7f9fa879
* **ap-southeast-2** : ami-03ed15d31bfc2844c
* **eu-central-1** : ami-0813c06d8b2462c39
* **us-east-2** : ami-07c593425f988b054
* **us-west-1** : ami-0eb0e13b1f06c03c0
* **us-west-2** : ami-000568ca142798412
* **us-east-1** : ami-062d9da44f96c8a87
### v0.11.0 ### v0.11.0
* **eu-north-1** : ami-0cbe338f058018c97 * **eu-north-1** : ami-0cbe338f058018c97