Local-Paddle-Serving

Veröffentlicht am | Edited on | in | Views:

本地paddle serving部署

参考

参考

TEST

UCI_housing

下载模型

1
2
wget --no-check-certificate https://paddleserving.bj.bcebos.com/uci_housing.tar.gz
tar -xzf uci_housing.tar.gz

由于直接在windows本机上部署，所以不需要上述命令，直接下载文件解压

安装paddle serving

1
2
pip install paddle_serving_server  //在CPU环境上安装paddle_serving_server
pip install paddke_serving_server_gpu   //在GPU环境上安装paddle_serving_server

直接在windows上安装会报错，因为fnctl这个库只在linux上，所以用WSL

应该是用WSL的原因，安装GPU版本的显示找不到GPU，还是用的CPU

  • 报错sentencepiece安装失败：Failed to find sentencepiece pkg-config

    去官网下载 sentencepiece ，版本：sentencepiece-0.1.94-cp36-cp36m-manylinux2014_x86_64

    安装：pip3 install sentencepiece-0.1.94-cp36-cp36m-manylinux2014_x86_64

  • WSL自带python2、python3.6，默认用python2，切换：

    1
    2
    3
    4
    sudo update-alternatives --install /usr/bin/python python /usr/bin/python2 100
    sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 150

    需要切换到python2时：sudo update-alternatives --config python

启动服务

1
python -m paddle_serving_server.serve --model uci_housing_model/ --thread 10 --port 9292 --name uci
  • model：Server 端配置与模型文件所在目录。
  • thread：Server 端线程数。
  • port：Server 端预测服务端口号。
  • name：HTTP 预测服务的的名称，如果不指定 name，则会启动 RPC 预测服务。

测试

同样使用WSL测试

1
curl -H "Content-Type:application/json" -X POST -d '{"feed":[{"x": [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332]}], "fetch":["price"]}' http://127.0.0.1:9292/uci/prediction

image-20201107182344856

文本

语义模型

Bert

下载模型

目标是给定输入一个句子，推理服务可以将句子表示成一个语义向量返回给客户端。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import paddlehub as hub
model_name = "bert_chinese_L-12_H-768_A-12"
#获取模型文件
module = hub.Module(model_name)
#获取模型的输入输出信息以及program
inputs, outputs, program = module.context(
    trainable=True, max_seq_len=20)
#将输入输出的名称与模型中的输入输出变量一一映射
feed_keys = ["input_ids", "position_ids", "segment_ids",
             "input_mask", ]
fetch_keys = ["pooled_output", "sequence_output"]
feed_dict = dict(zip(feed_keys, [inputs[x] for x in feed_keys]))
fetch_dict = dict(zip(fetch_keys, [outputs[x] for x in fetch_keys]))
#保存serving需要的模型与配置文件，参数1是server端模型与配置文件保存的目录，参数2是client端配置文件保存的目录，参数3是输入dict，参数4是输出dict，参数5是模型的program
import paddle_serving_client.io as serving_io
serving_io.save_model("bert_seq20_model", "bert_seq20_client",
                      feed_dict, fetch_dict, program)

报错：cublas64_100.dll not found

下载相应的文件 地址，放到 C:\Windows\System32 文件下

模型下载后可以移到当前文件夹下，然后用 module = hub.Module(directory="")加载

  • 也可使用 bert 这里的 prepare_model.py

启动服务

在WSL下：python -m paddle_serving_server.serve --model bert_seq128_model/ --thread 10 --port 9292

这之后 参考 这个网站上的代码有些问题

参考 bert python -m paddle_serving_server.serve --model bert_seq128_model/ --port 9292

这个项目里用的长度为128，但模型都是一样的，名字都是 bert_chinese_L-12_H-768_A-12

测试

首先用WSL运行getdata.sh，脚本将下载中文词典vocab.txt和中文样例数据data-c.txt ，然后测试：

1
2
3
4
head data-c.txt | python bert_client.py --model bert_seq128_client/serving_client_conf.prototxt

可以用head -n 2 data-c.txt指定只传入前几行
也可以 echo "xxxx"传入指定字符串
  • ImportError: libSM.so.6: cannot open shared object file: No such file or directory

参考

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Python3环境下，import cv2 提示 

ImportError: libSM.so.6: cannot open shared object file: No such file or directory

解决步骤

sudo apt-get update

sudo apt-get install -y python-qt4

import cv2 提示 

 ImportError: No module named cv2

sudo pip3 install opencv-python

以及 

ImportError: libgthread-2.0.so.0: cannot open shared object file: No such file or directory
apt-get update

apt-get install -y libglib2.0-dev
  • 最后一行的结果

image-20201107215218113

词法分词

LAC

下载模型

1
hub install lac==2.2.0

命令行预测：

1
2
3
4
5
6
7
8
9
10
11
12
hub run lac --input_text "今天是个好日子"
hub run lac --input_file test.txt --user_dict user.dict

# test.txt 存放待分词文本
今天是个好日子     
今天天气晴朗   

# user.dict为用户自定义词典，可以不指定，当指定自定义词典时，可以干预默认分词结果
春天/SEASON
花/n 开/v
秋天的风
落 阳

image-20201109195508700

本地运行

1
2
3
4
5
6
7
8
9
10
import paddlehub as hub

lac = hub.Module(name="lac")
test_text = ["今天是个好日子", "天气预报说今天要下雨", "下一班地铁马上就要到了"]

results = lac.cut(text=test_text, use_gpu=False, batch_size=1, return_tag=True)

for result in results:
    print(result['word'])
    print(result['tag'])

启动服务

1
hub serving start -m lac -p 9292

测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import requests
import json

# 待预测数据
text = ["今天是个好日子", "天气预报说今天要下雨"]

# 设置运行配置
# 对应本地预测lac.cut(text=text, batch_size=1)
data = {"text": text, "batch_size": 1}

# 指定预测方法为lac并发送post请求，content-type类型应指定json方式
# HOST_IP为服务器IP
url = "http://127.0.0.1:9292/predict/lac"
headers = {"Content-Type": "application/json"}
r = requests.post(url=url, headers=headers, data=json.dumps(data))

# 打印预测结果
print(json.dumps(r.json(), indent=4, ensure_ascii=False))

图像

人脸检测

pyramidbox

下载模型

hub install pyramidbox_face_detection==1.1.0

测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
import paddlehub as hub
import os


def run(img_name):
    face_detector = hub.Module(name="./pyramidbox_face_detection")
    path = './images/'
    result = face_detector.face_detection(paths=[os.path.join(path, img_name)], visualization=True)
    print(result)
    return result


if __name__ == "__main__":
    print(run('1.jpg'))

图像生成

stylepro_artistic

下载模型

hub install stylepro_artistic==1.0.0

测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import paddlehub as hub
import os


def run(input_img_name, style_img_name):
    stylepro_artistic = hub.Module(directory="stylepro_artistic")
    path = './images/'
    result = stylepro_artistic.style_transfer(
        paths=[{
            'content': os.path.join(path, input_img_name),
            'styles': [os.path.join(path, style_img_name)]
        }] , visualization=True)
    print(result[0]['save_path'])
    return result[0]['save_path']


if __name__ == "__main__":
    print(run('1.jpg', 'picasso.png'))

目标检测

SSD

参考

下载模型

可以通过代码下载

1
ssd = hub.Module(name="ssd_mobilenet_v1_pascal")

也可以安装

1
hub install ssd_mobilenet_v1_pascal==1.1.1

如果通过代码进行测试，可以更改模型位置，用 ssd = hub.Module(directory = "") 加载即可，在当前文件夹下用 hub run ssd_mobilenet_v1_pascal --input_path "./images/test_img_cat.jpg" 也可

如果要启用serving服务，不能指定模型位置，比如 hub serving start -m ./ssd_mobilenet_v1_pascal 会找不到位置

启动服务

hub serving start -m ssd_mobilenet_v1_pascal -p 9292

测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import requests
import json
import cv2
import base64
import os


def cv2_to_base64(image):
    data = cv2.imencode('.jpg', image)[1]
    return base64.b64encode(data.tostring()).decode('utf8')


path = "./images"
images = os.listdir(path)


# 发送HTTP请求
data = {'images': [cv2_to_base64(cv2.imread(os.path.join(path, image))) for image in images]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:9292/predict/ssd_mobilenet_v1_pascal"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
# 打印预测结果
print(r.json()["results"])

fix

如果是多张图片会出现在同一张图片上绘制检测框的问题：

源码中processor，修改以下代码

1
2
3
# Fixed the following one line of code
# org_img = images[index - unhandled_paths_num]
org_img = images[handle_id]

image-20201108151114536

图像分类

EfficientNet b7

下载模型

hub install efficientnetb7_imagenet==1.1.0

本地运行

1
2
3
4
5
6
7
8
9
10
11
import paddlehub as hub
import cv2
import os

classifier = hub.Module(directory="efficientnetb7_imagenet")
path = './images/'
images = os.listdir(path)

results = classifier.classify(images=[cv2.imread(os.path.join(path, image)) for image in images])
for result in results:
    print(result)

启动服务

1
hub serving start -m efficientnetb7_imagenet -p 9292

测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import requests
import json
import cv2
import base64
import os

def cv2_to_base64(image):
    data = cv2.imencode('.jpg', image)[1]
    return base64.b64encode(data.tostring()).decode('utf8')


path = "./images/"
images = os.listdir(path)

# 发送HTTP请求
data = {'images':[cv2_to_base64(cv2.imread(os.path.join(path, image))) for image in images]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:9292/predict/efficientnetb7_imagenet"
r = requests.post(url=url, headers=headers, data=json.dumps(data))

# 打印预测结果
print(r.json()["results"])

image-20201108170957240

文字识别

Chinese_text_detection

下载模型

1
$ hub install chinese_text_detection_db_server==1.0.2

命令行预测：hub run chinese_text_detection_db_server --input_path "/PATH/TO/IMAGE"

本地运行

  • visualization = True
1
2
3
4
5
6
7
8
9
10
11
12
13
import paddlehub as hub
import cv2
import os

path = './images/'
images = os.listdir(path)

text_detector = hub.Module(name="chinese_text_detection_db_server")
results = text_detector.detect_text(images=[cv2.imread(os.path.join(path, image)) for image in images], visualization = True)
for result in results:
    print(result)
# or
# result = ocr.recognize_text(paths=['/PATH/TO/IMAGE'])

image-20201108180527140

  • 报错 from shapely.geos import lgeos：

image-20201108180007036

  1. conda install -c conda-forge shapely

启动服务

1
hub serving start -m chinese_text_detection_db_server -p 9292

测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import requests
import json
import cv2
import base64
import os


def cv2_to_base64(image):
    data = cv2.imencode('.jpg', image)[1]
    return base64.b64encode(data.tostring()).decode('utf8')


path = './images/'
images = os.listdir(path)
# 发送HTTP请求
data = {'images': [cv2_to_base64(cv2.imread(os.path.join(path, image))) for image in images], 'visualization':True}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:9292/predict/chinese_text_detection_db_server"
r = requests.post(url=url, headers=headers, data=json.dumps(data))

# 打印预测结果
print(r.json()["results"])

image-20201108181155305

Chinese OCR Mobile

chinese_ocr_db_crnn_server 运行中需要的内存资源太大了

下载的是 chinese_ocr_db_crnn_mobile

下载模型

1
hub install chinese_ocr_db_crnn_mobile==1.1.0

命令行测试： hub run chinese_ocr_db_crnn_mobile --input_path './images/3.jpg'

image-20201108182035187

本地运行

  • visualization = True
1
2
3
4
5
6
7
8
9
10
11
import paddlehub as hub
import cv2
import os

path = './images/'
images = os.listdir(path)

ocr = hub.Module(name="chinese_ocr_db_crnn_mobile")
results = ocr.recognize_text(images=[cv2.imread(os.path.join(path, image)) for image in images], visualization=True)
for result in results:
    print(result)

image-20201108182442180

启动服务

1
hub serving start -m chinese_ocr_db_crnn_mobile -p 9292

测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import requests
import json
import cv2
import base64
import os


def cv2_to_base64(image):
    data = cv2.imencode('.jpg', image)[1]
    return base64.b64encode(data.tostring()).decode('utf8')

path = './images'
images = os.listdir(path)

# 发送HTTP请求
data = {'images': [cv2_to_base64(cv2.imread(os.path.join(path, image))) for image in images], 'visualization':True}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:9292/predict/chinese_ocr_db_crnn_mobile"
r = requests.post(url=url, headers=headers, data=json.dumps(data))

# 打印预测结果
print(r.json()["results"])

图像分割

Humanseg Mobile

下载模型

1
hub install humanseg_mobile==1.1.0

命令行运行：hub run humanseg_mobile --input_path "/PATH/TO/IMAGE"

image-20201108212156637

本地运行

1
2
3
4
5
6
7
8
9
10
11
12
13
14
import cv2
import paddlehub as hub
import os

path = './images'
images = os.listdir(path)

human_seg = hub.Module(directory='humanseg_mobile')
# visualization=True可以用于查看超分图片效果，可设置为False提升运行速度。
res = human_seg.segment(images=[cv2.imread(os.path.join(path, image)) for image in images], visualization=True)
for r in res:
    print(r['save_path'])
# human_seg.video_segment('/PATH/TO/VIDEO')
# human_seg.save_inference_model('/PATH/TO/SAVE/MODEL')

直接运行的话，一个文件夹中的全部图片只有一部分能得到结果，因为源码中按照时间戳命名保存图片，运行的太快会覆盖掉之前的。

image-20201108214401230

修改process中get_save_image_name，直接使用随机数

1
2
3
4
5
6
7
8
ext = '.png'
t = time.strftime("%Y%m%d_%H%M%S_", time.localtime())
rand = random.randint(0, 1000000)
save_im_path = os.path.join(output_dir, t + str(rand) + ext)
while os.path.exists(save_im_path):
    save_im_path = os.path.join(output_dir, t + str(random.randint(0, 1000000)) + ext)

return save_im_path

启动服务

1
hub serving start -m humanseg_mobile -p 9292

测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
import requests
import json
import base64

import cv2
import numpy as np
import os

def cv2_to_base64(image):
    data = cv2.imencode('.jpg', image)[1]
    return base64.b64encode(data.tostring()).decode('utf8')


def base64_to_cv2(b64str):
    data = base64.b64decode(b64str.encode('utf8'))
    data = np.fromstring(data, np.uint8)
    data = cv2.imdecode(data, cv2.IMREAD_COLOR)
    return data


# 发送HTTP请求
path = './images'
images = os.listdir(path)
outpath = './serving_image_results/'
if not os.path.exists(outpath):os.mkdir(outpath)

data = {'images': [cv2_to_base64(cv2.imread(os.path.join(path, image))) for image in images]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:9292/predict/humanseg_mobile"
rs = requests.post(url=url, headers=headers, data=json.dumps(data))

imagesdata = rs.json()['results']
for i in range(len(imagesdata)):
    mask = cv2.cvtColor(base64_to_cv2(imagesdata[i]['data']), cv2.COLOR_BGR2GRAY)
    org_im = cv2.imread(os.path.join(path, images[i]))
    rgba = np.concatenate((org_im, np.expand_dims(mask, axis=2)), axis=2)
    print(outpath+str(i)+".png")
    cv2.imwrite(outpath+str(i)+".png", rgba)