记录一次服务器跑大模型经历

2024/07/31

由于网络问题，速度慢或者无法直连hugging-face docker hub pipy

设置系统代理

# set proxy config via profie.d - should apply for all users 
export http_proxy="http://127.0.0.1:10000/" 
export https_proxy="http://127.0.0.1:10000/" localhost"

# 注意，jupyter是不会直接调用你的bash设置的，所以你需要在cell先设置一下系统代理，这样jupyter笔记本就会调用你的代理了

Pipy 使用国内镜像，我用的清华镜像，也可以用其他的

# pipy 如果你配置了代理，就可以不用设置这个源了
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple some-package

Docker 主要是连接docker hub 比较吃力 可以选1panel的镜像

sudo mkdir -p /etc/docker sudo tee /etc/docker/daemon.json <<-'EOF' { "registry-mirrors": \[ "https://dockerproxy.com", "https://docker.mirrors.ustc.edu.cn", "https://docker.nju.edu.cn", docker.1panel.live \] } EOF sudo systemctl daemon-reload sudo systemctl restart docker

深度学习环境配置 这里有两种方案 #### 方案一-只有pytorch+jupyter > refer -> https://hub.docker.com/r/tverous/pytorch-notebook

这里面是没有nvcc的需要在以下网站里安装 > refer -> https://developer.nvidia.com/cuda-downloads

docker run --rm -it \\ --gpus all \\ -p 8888:8888 \\ -e JUPYTER\_TOKEN=passwd \\ -v /local\_vol:/docker\_vol \\ tverous/pytorch-notebook:latest

也可以把jupyter 跑在后台，然后每次通过服务器的http://ip:8888访问

方案二-可选pytorch+tensorflow

模型下载

设置huggingface 凭证

token hf_token > 参考https://www.cnblogs.com/ghj1976/p/18336819/hugging-face-access-tokens-si-zhong-yong-fa

from transformers import AutoProcessor, PaliGemmaForConditionalGeneration access\_token="您的Hugging Face的Access Token，一般hf\_开头" model\_id = "google/paligemma-3b-pt-224" processor\_id = "google/paligemma-3b-pt-224" model = PaliGemmaForConditionalGeneration.from\_pretrained(model\_id,token=access\_token) processor = AutoProcessor.from\_pretrained(processor\_id,token=access\_token)

同样两种方案

方案一-hugging-face镜像 > refer -> https://hf-mirror.com/

方案二-cloudflare worker 自建反向代理 > refer -> https://www.zouht.com/3740.html

然后每次使用

wget https://yourdomain.com/https://blockdomain.com

Enjoy your learning !