Flexible Deployment

This feature is available only to enterprise users. Authentication process: Console -> Left-hand menu -> Account Security -> Real-Name Authentication. API documentation: Read the API documentation

Support for reusing stopped containers is now available, which can significantly reduce container creation time

Elastic Deployment is a feature designed specifically for enterprise AIGC and similar scenarios that enables batch scheduling and launching of multiple containers, as well as managing their lifecycles.

Introduction to Scheduling Modes

Computing Power Scheduling Unit

The system resource pool manages N physical hosts with different configurations. The hosts differ in GPU model and quantity, CPU model and quantity, and memory size. The system divides the hosts’ CPU and memory resources proportionally into N computing power scheduling units based on the number of GPUs (N). Modifying or splitting these scheduling units is not supported.

For example, if Host A is configured with 8 RTX 3090 GPUs, a 128-core CPU, and 720 GB of memory, then each computing power scheduling unit on that host is: 3090 * 1；16 vCPU；90GB内存. When creating a container on this host, the container’s computing power configuration must be between 1 and 8 times that of the scheduling unit; the ratio of CPU, memory, and GPU resources cannot be adjusted.

Container Scheduling and Startup

Containers are created based on the scheduling conditions you specify. Due to the indivisibility of the computing units mentioned above and the differences in computing units across different hosts, the specified scheduling conditions may include: GPU model and quantity, CPU quantity range, memory size range, price range, and so on.When a specific host meets these scheduling conditions, the container is created and started on that host. For example:

There are machines A, B, and C. Their available computing power is as follows: 8卡3090、128核CPU、720GB内存, 8卡3090、64核CPU、720GB内存, 8卡3090、128核CPU、360GB内存

Set the scheduling conditions as follows: Number of GPUs = 8, CPU count range = [100, 200], memory size range = [224, 1024].

The system will ultimately create an instance on either Host A or Host C. If the instance is created on Host A, its configuration specifications will be 8卡3090、128核CPU、720GB内存; if it is created on Host C, its configuration specifications will be 8卡3090、128核CPU、360GB内存

Scheduling for ReplicaSet Types

In WebCal, a ReplicaSet refers to the creation and maintenance of a stable set of containers that remains operational at all times and maintains the specified number of replicas. Each container replica is scheduled and launched according to the container scheduling conditions you set.

If you modify the scheduling conditions, the system will terminate any existing running containers that no longer meet the conditions and start new containers that do meet them. Changing the number of container replicas will immediately create new containers or terminate existing running containers until the number of running containers equals the number of replicas you specified.

Scheduling by Job Type

A WebCal Job refers to the creation of one or more containers until the specified number of containers have completed execution and exited.Unlike a ReplicaSet, a Job does not launch new containers to maintain a specified number of container replicas once a container has finished (new containers are launched only because the number of completed containers has not yet reached the target number). Instead, once a container finishes, it is marked as complete. The scheduling process continues until the number of completed containers reaches the target value, at which point the entire scheduling process ends and no new containers are launched.

Scheduling of Container Types

In WebCal, "Container" refers to creating a single container until it terminates and exits, at which point scheduling ends. This is equivalent to a job scheduled with a target number of containers set to 1.

Container Lifecycle

The container’s lifecycle depends on the execution lifecycle you set for the cmd command.Once the cmd execution completes, the container will exit and shut down. Therefore, if the command to start the application in your cmd runs in the background, add sleep infinity at the end of the cmd to prevent the container from shutting down and all other processes from terminating when the parent process exits. Both of the following methods are acceptable (the specific commands below are for illustrative purposes only):

# 方式一：
python app.py

# 方式二：
nohup python app.py & && sleep infinity

In Method 2 above, the application runs in the background and prevents the cmd from terminating by using sleep infinity. This makes the lifecycle of the app.py application independent of the container’s lifecycle, requiring you to manage the application’s lifecycle separately (i.e., even if app.py has finished running, the container may still be running normally, and you cannot infer the application’s status based on the container’s status).If you need to manage the application’s lifecycle yourself, we recommend using this method; otherwise, we recommend Method 1. In addition, you can stop a container by calling the API to stop the elastic deployment or the API to stop a specific container.

Billing

Please refer to the Billing Documentation

The Difference Between Containers and Container Instances in Elastic Deployment

Difference 1: Different data retention rules

For container instances rented on the computing power marketplace, data is retained for a certain period after the instance is shut down (see documentation). For elastically deployed containers, data is released immediately upon shutdown and is not retained.

Difference 2: Differences in data disk expansion

Container instances rented on the computing power marketplace support data disk expansion, while containers deployed via elastic deployment do not support data disk expansion; that is, the default data disk size is 50 GB.

Difference 3: Different Restart Methods

Container instances rented on the computing power marketplace can be restarted as long as they have not been released; however, containers in an elastic deployment cannot be restarted once the deployment has been stopped. If the deployment has been stopped, you can create a new deployment. If the deployment has not been stopped, you can adjust the number of running containers by setting the number of replicas.

Difference 4: Different Methods for Accessing the Container

Elastic Deployment containers do not have a JupyterLab entry point. If you need to access the container, you can use the SSH commands and password provided to log in. See Documentation