--- url: /en/self_host/docker/migration-docker-kubernetes.md --- # Migrating from Docker Version to K8S Version ![migration from docker to kubernetes](https://swanlab-docs-1301372061.cos.ap-beijing.myqcloud.com/assets/en/guide_cloud/self_host/kubernetes/migration.png) This guide is for migrating data from the SwanLab Docker version to the **SwanLab Kubernetes (K8S)** version. It **only applies to scenarios where external services are integrated into the SwanLab Kubernetes (K8S)** version (refer to [Customizing Basic Service Resources](../kubernetes/deploy.md#_3-fill-in-values)). If you wish to migrate to managed services provided by cloud vendors or to self-deployed cloud-native high-availability services, this guide can serve as a reference. Please also refer to the official migration documentation of the respective cloud vendor or cloud-native project, and ensure database names, table names, and object storage bucket names are correct during migration. \[\[toc]] *** **This solution requires:** 1. Migrating data first, then deploying the service. 2. You use the [Customizing Basic Service Resources](../kubernetes/deploy.md#_3-fill-in-values) feature. 3. You have a busybox image for implementing migration tasks. 4. Ensure your Storage Class's reclaim policy does **not** delete data when Pods are removed. **We pre-create PersistentVolumeClaims by packaging data, downloading it, and storing it in a volume, then mount these volume resources**. *** **Please identify the resources you need to migrate:** 1. **PostgreSQL Single Instance**: Used to store SwanLab's core business data. 2. **Clickhouse Single Instance**: Used to store metric data. 3. **MinIO Single Instance**: Used to store media resources. 4. **Redis Single Instance**: Cache service. Among these, SwanLab-House does **not** require data migration. :::warning Before migrating, please ensure: 1. Your Docker service has been stopped, or you need to ensure the migrated data is extracted from a specific storage snapshot. 2. You have located the data volume path mounted by the SwanLab Docker version. By default, it should be in the `docker/swanlab/data` directory of the [self-hosted](https://github.com/SwanHubX/self-hosted) project. If you have forgotten the storage path, you can find the volume mount location for the corresponding service container using the `docker inspect` command. 3. You have found the `docker-compose.yaml` file generated by swanlab. This is primarily for migrating account passwords. If you have forgotten the location of the `docker-compose.yaml` file, you can still find the corresponding account password environment variables using `docker inspect`. ::: For convenience, the following Docker-related commands are based on `self-hosted/docker/swanlab/`. Please adjust the paths according to your actual situation. Additionally, the account passwords mentioned in this guide are for reference only; please adjust them according to your actual configuration. Please identify the storage locations of various basic service resources in the Docker version: 1. PostgreSQL data is in `self-hosted/docker/swanlab/data/postgres` 2. Clickhouse data is stored in `self-hosted/docker/swanlab/data/clickhouse` 3. Minio data is stored in `self-hosted/docker/swanlab/data/minio` 4. Redis data is stored in `self-hosted/docker/swanlab/data/redis` *** **Glossary:** * `self-hosted`: The deployed SwanLab Kubernetes cluster. ## 1. Migrate PostgreSQL * Connection string example: `postgresql://swanlab:1uuYNzZa4p@postgres:5432/app?schema=public` ### 1.1 Package and Upload the Archive :::warning Note: For PostgreSQL, its data directory is located at `/var/lib/postgresql/data`, corresponding to the external data volume at `/data/postgres/data`. However, PostgreSQL does not allow any folders other than `data` under the `/var/lib/postgresql` directory. Therefore, for PG, the final archive is a `data` directory. ::: Use the following command to package the pg data: ```bash tar -czvf postgres-data.tar.gz -C data/postgres/ . ``` Then upload it to object storage or any network storage service accessible by the cluster. In this example, we upload it to Aliyun Object Storage. The file link example is: ``` https://xxx.oss-cn-beijing.aliyuncs.com/self-hosted/docker/postgres-data.tar.gz ``` ### 1.2 Copy Data to the Persistent Volume Reference configuration: ```yaml apiVersion: batch/v1 kind: Job metadata: name: postgres-migrate labels: swanlab: postgres spec: template: spec: restartPolicy: OnFailure containers: - name: postgres-migrate image: busybox:1.37.0 imagePullPolicy: IfNotPresent volumeMounts: - name: postgres-volume mountPath: /data env: - name: FILE_URL value: "https://xxx.oss-cn-beijing.aliyuncs.com/self-hosted/docker/postgres-data.tar.gz" command: - /bin/sh - -c - | wget $FILE_URL -O /tmp/postgres-data.tar.gz tar -xzvf /tmp/postgres-data.tar.gz -C /data volumes: - name: postgres-volume persistentVolumeClaim: claimName: postgres-docker-pvc ``` ### 1.3 Clarify Your Configuration You need to modify the corresponding resources under `dependencies.postgres` to bind the PVC and correctly set the corresponding account passwords. Example: ```yaml dependencies: postgres: username: "swanlab" password: "1uuYNzZa4p" persistence: existingClaim: "postgres-docker-pvc" ``` > `self-hosted` itself will create a Secret resource based on `username` and `password` and will not store them in plain text. ## 2. Migrate Redis In the SwanLab Docker version, the Redis connection string is the default `redis://default@redis:6379`. ### 2.1 Package and Upload the Archive Use the following command to package the redis data: ```bash tar -czvf redis-data.tar.gz -C data/redis/ . ``` Then upload it to object storage or any network storage service accessible by the cluster. In this example, we upload it to Aliyun Object Storage. The file link example is: ``` https://xxx.oss-cn-beijing.aliyuncs.com/self-hosted/docker/redis-data.tar.gz ``` ### 2.2 Copy Data to the Persistent Volume Reference configuration: ```yaml apiVersion: batch/v1 kind: Job metadata: name: redis-migrate labels: swanlab: redis spec: template: spec: restartPolicy: OnFailure containers: - name: redis-migrate image: busybox:1.37.0 imagePullPolicy: IfNotPresent volumeMounts: - name: redis-volume mountPath: /data env: - name: FILE_URL value: "https://xxx.oss-cn-beijing.aliyuncs.com/self-hosted/docker/redis-data.tar.gz" command: - /bin/sh - -c - | wget $FILE_URL -O /tmp/redis-data.tar.gz tar -xzvf /tmp/redis-data.tar.gz -C /data volumes: - name: redis-volume persistentVolumeClaim: claimName: redis-docker-pvc ``` Note the PVC name, and run it after confirming it's correct. ### 2.3 Clarify Your Configuration You need to modify the corresponding resources under `dependencies.redis` to bind the PVC. Example: ```yaml dependencies: redis: persistence: existingClaim: "redis-docker-pvc" ``` ## 3. Migrate Clickhouse ### 3.1 Package and Upload the Archive * Connection string: `tcp://swanlab:2jwnZiojEV@clickhouse-docker:9000/app` * HTTP port: `8123` Use the following command to package the clickhouse data: ```bash tar -czvf clickhouse-data.tar.gz -C data/clickhouse/ . ``` Then upload it to object storage or any network storage service accessible by the cluster. In this example, we upload it to Aliyun Object Storage. The file link example is: ``` https://xxxx.oss-cn-beijing.aliyuncs.com/self-hosted/docker/clickhouse-data.tar.gz ``` ### 3.2 Copy Data to the Persistent Volume Reference configuration: ```yaml apiVersion: batch/v1 kind: Job metadata: name: clickhouse-migrate labels: swanlab: clickhouse spec: template: spec: restartPolicy: OnFailure containers: - name: clickhouse-migrate image: busybox:latest imagePullPolicy: IfNotPresent volumeMounts: - name: clickhouse-volume mountPath: /data env: - name: FILE_URL value: "https://xxx.oss-cn-beijing.aliyuncs.com/self-hosted/docker/clickhouse-data.tar.gz" command: - /bin/sh - -c - | wget $FILE_URL -O /tmp/clickhouse-data.tar.gz tar -xzvf /tmp/clickhouse-data.tar.gz -C /data volumes: - name: clickhouse-volume persistentVolumeClaim: claimName: clickhouse-docker-pvc ``` ### 3.3 Clarify Your Configuration You need to modify the corresponding resources under `dependencies.clickhouse` to bind the PVC and correctly set the corresponding account passwords. Example: ```yaml dependencies: clickhouse: username: "swanlab" password: "2jwnZiojEV" persistence: existingClaim: "clickhouse-docker-pvc" ``` > `self-hosted` itself will create a Secret resource based on `username` and `password` and will not store them in plain text. ## 4. Migrate Minio * accessKey: `swanlab` * accessSecret: `qtllV4B9KZ` ### 4.1 Package and Upload the Archive Use the following command to package the minio data: ```bash tar -czvf minio-data.tar.gz -C data/minio/ . ``` Then upload it to object storage or any network storage service accessible by the cluster. In this example, we upload it to Aliyun Object Storage. The file link example is: ``` https://xxx.oss-cn-beijing.aliyuncs.com/self-hosted/docker/minio-data.tar.gz ``` ### 4.2 Copy Data to the Persistent Volume Reference configuration: ```yaml apiVersion: batch/v1 kind: Job metadata: name: minio-migrate labels: swanlab: minio spec: template: spec: restartPolicy: OnFailure containers: - name: minio-migrate image: busybox:latest imagePullPolicy: IfNotPresent volumeMounts: - name: minio-volume mountPath: /data env: - name: FILE_URL value: "https://xxx.oss-cn-beijing.aliyuncs.com/self-hosted/docker/minio-data.tar.gz" command: - /bin/sh - -c - | wget $FILE_URL -O /tmp/minio-data.tar.gz tar -xzvf /tmp/minio-data.tar.gz -C /data volumes: - name: minio-volume persistentVolumeClaim: claimName: minio-docker-pvc ``` ### 4.3 Clarify Your Configuration You need to modify the corresponding resources under `dependencies.minio` to bind the PVC and correctly set the corresponding account passwords. Example: ```yaml dependencies: s3: accessKey: "swanlab" secretKey: "qtllV4B9KZ" persistence: existingClaim: "minio-docker-pvc" ``` ## 5. Deploy the Service After completing the above four steps, you can begin deploying the SwanLab Kubernetes service. > For basic operations on deploying the Kubernetes service, see: [Deploying with Kubernetes](../kubernetes/deploy.md). You only need to modify the `dependencies` section based on the original [values.yaml](https://github.com/SwanHubX/charts/blob/main/charts/self-hosted/values.yaml). Reference values (please ensure your credentials are correct): ```yaml dependencies: postgres: username: "swanlab" password: "1uuYNzZa4p" persistence: existingClaim: "postgres-docker-pvc" redis: persistence: existingClaim: "redis-docker-pvc" clickhouse: username: "swanlab" password: "2jwnZiojEV" persistence: existingClaim: "clickhouse-docker-pvc" s3: accessKey: "swanlab" secretKey: "qtllV4B9KZ" persistence: existingClaim: "minio-docker-pvc" ``` After making the modifications, execute the following command to deploy the SwanLab Kubernetes service: ```bash helm install swanlab-self-hosted swanlab/self-hosted -f values.yaml ``` > After deployment, if you are already logged into the **SwanLab Docker version** in your browser and the domain name remains the same before and after migration, you do not need to log in again. For more detailed operations on Kubernetes deployment, please refer to: [Deploying with Kubernetes](../kubernetes/deploy.md).