[AI Cog] 想要运营AI业务，但没有GPU？环境搞不定？使用Cog帮您轻松将业务部署上云

当你想开展AI业务，却没有GPU，你该怎么办？

可以考虑用Cog，将AI服务部署在云上，serverless。

我们来看下，如何用Cog将其上云。

找一台开发服务器

Cog

安装

1
2


sudo curl -o /usr/local/bin/cog -L https://github.com/replicate/cog/releases/latest/download/cog_`uname -s`_`uname -m`
sudo chmod +x /usr/local/bin/cog

验证

这一步可以省略，非必须。主要用于验证你的环境是否ok。

1

sudo cog predict r8.im/stability-ai/stable-diffusion@sha256:f178fa7a1ae43a9a9af01b833b9d2ecf97b1bcb0acfd2dc5dd04895e042863f1 -i prompt="a pot of gold"

初始化

1

cog init

生成主要文件

1
2


├── cog.yaml # 类似 docker file，定义环境
├── predict.py # 推理代码

写代码

修改代码如下

cog.yaml 类似 docker file，定义环境

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28


# Configuration for Cog ⚙️
# Reference: https://cog.run/yaml

build:
  # set to true if your model requires a GPU
  gpu: false

  # a list of ubuntu apt packages to install
  # system_packages:
  #   - "libgl1-mesa-glx"
  #   - "libglib2.0-0"

  # python version in the form '3.11' or '3.11.4'
  python_version: "3.10"

  # a list of packages in the format <package-name>==<version>
  # python_packages:
  #   - "numpy==1.19.4"
  #   - "torch==1.8.0"
  #   - "torchvision==0.9.0"

  # commands run after the environment is setup
  # run:
  #   - "echo env is ready!"
  #   - "echo another command if needed"

# predict.py defines how predictions are run on your model
predict: "predict.py:Predictor"

predict.py 定义了输入(name: str, scale: float)，输出(str)，推理过程

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


# Prediction interface for Cog ⚙️
# https://cog.run/python

from cog import BasePredictor, Input, Path

class Predictor(BasePredictor):
    def setup(self) -> None:
        """Load the model into memory to make running multiple predictions efficient"""
        # self.model = torch.load("./weights.pth")

    def predict(
        self,
        name: str = Input(description="Your name"),
        # image: Path = Input(description="Grayscale input image"),
        scale: float = Input(
            description="Factor to scale image by", ge=0, le=10, default=1.5
        ),
    ) -> str:
        """Run a single prediction on the model"""
        # processed_input = preprocess(image)
        # output = self.model(processed_image, scale)
        # return postprocess(output)
        return "hello " + name + " and scale " + str(scale)

本地测试

测试一下

1

cog predict -i name=从零开始学AI

输出

1
2
3


Starting Docker image cog-git-base and running setup()...
Running prediction...
hello 从零开始学AI and scale 1.5

部署

在云上 create model

push model 到云上

1
2


cog login
cog push r8.im/<your-username>/<your-model-name>

云上测试

cog-input

cog-output

测试成功！

之后，就可以用 api 调用

结论

本文主要演示如何用 Cog 上云的整个流程。

文中的例子，未使用 GPU 。如有需要，可查文档。