lxc搭建自己的对象储存s3 mino over ssl/tls

场景

对象储存是一个简单的网络文件储存协议，Simple Storage Service 基于http。性能要超过webdav。
因为云厂商的s3价格比较贵，免费的限制很多。所以在部分情况下，还是用自己的搭建的。

minio

golang写的绿色环保轻量高性能开源免费

环境准备

我这里是pve平台，采用lxc alpine搭建。创建以一个ct容器 id 9000 ，镜像选择alpine3.17 硬盘8g，另外直通一个物理盘进来，lxc直通教程请站内搜索。 ip 固定10.1.1.90
alpine 启动后，修改源市区

1
2
3
4
5
6
sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories
apk add tzdata 
cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
echo "Asia/Shanghai" > /etc/timezone
apk del tzdata
rm -rf /var/cache/apk/*

安装minio

1
2
3
apk add wget
wget https://dl.min.io/server/minio/release/linux-amd64/minio  #活跃软件更新比较频繁
chmod +x minio

启动测试

1
2
3
4
MINIO_ROOT_USER=admin MINIO_ROOT_PASSWORD=123456 ./minio server /mnt/data --console-address ":9001"
# http://10.1.1.90:9001
# 用户名：admin
# 密码： 123456

ctrl -c 结束

配置证书

我的证书文件均由 github action 托管，所以我这里直接拉取回来即可。详情
minio要求私钥为private.key，公共证书为public.crt。配置自动更新证书的脚本

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
cat > auto-updatessl-form-github.sh   << EOF
cd /root
rm ssl.zip*
wget https://cdn.jsdelivr.net/gh/XXXX/sXXXX.zip #https://github.com/XXXXXXXXXXXXX.zip
if [ ! -f "/root/ssl.zip" ]; then
 wget -O/dev/null -q https://api.day.app/你的key/SSL-update-err/[home-mino]?sound=birdsong&amp;isArchive=1
else
  rm -rf ssl
  7z x ssl.zip  -p密码  #密码和p之间不能有空格
  supervisorctl restart minio
  nginx -S reload
fi
EOF
echo "15 8 * * * sh /root/auto-updatessl-form-github.sh" >> /etc/crontabs/root  && cat /etc/crontabs/root

启用证书

minio 支持tls通讯，但是理解起来有一点困难，本着能不用脑子的时候，绝对不用的前提。我还是选用nginx来搞定,下面操作依旧是基于alpine 其他linux系统注意一下路径

aio环境也可以集中到 nginxWebUI处理

1
2
3
4
apk add nginx
rc-update add nginx boot
mv /etc/nginx/http.d/default.conf  /etc/nginx/http.d/default.conf-bak
vi /etc/nginx/http.d/mino.conf 

内容,

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
server {
  server_name s3.jia.leiyanhui.com; #管理面板
  listen 51443 ssl http2;
  ssl_certificate /root/ssl/XXXX.cer;
  ssl_certificate_key /root/ssl/XXX.key;
  ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
  listen 51442;
  if ($scheme = http) {
    return 301 https://$host:51443$request_uri;
  }
  error_page 497 https://$host:51443;
  # Allow special characters in headers
   ignore_invalid_headers off;
   # Allow any size file to be uploaded.
   # Set to a value such as 1000m; to restrict file size to a specific value
   client_max_body_size 0;
   # Disable buffering
   proxy_buffering off;
   proxy_request_buffering off;
   location / {
      proxy_set_header Host $http_host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header X-NginX-Proxy true;
      # This is necessary to pass the correct IP to be hashed
      real_ip_header X-Real-IP;
      proxy_connect_timeout 300;
      # To support websocket
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";
      chunked_transfer_encoding off;
      proxy_pass http://10.1.1.90:9001/; # This uses the upstream directive definition to load balance and assumes a static Console port of 9001
   }
}
server {
  server_name s3api.域名.com; #api
  listen 51443 ssl http2;
  ssl_certificate /root/ssl/XXXX.cer;
  ssl_certificate_key /root/ssl/XXX.key;
  ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
  listen 51442;
  if ($scheme = http) {
    return 301 https://$host:51443$request_uri;
  }
  error_page 497 https://$host:51443;
  # Allow special characters in headers
   ignore_invalid_headers off;
   # Allow any size file to be uploaded.
   # Set to a value such as 1000m; to restrict file size to a specific value
   client_max_body_size 0;
   # Disable buffering
   proxy_buffering off;
   proxy_request_buffering off;

   location / {
      proxy_set_header Host $http_host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;

      proxy_connect_timeout 300;
      # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
      proxy_http_version 1.1;
      proxy_set_header Connection "";
      chunked_transfer_encoding off;

      proxy_pass http://10.1.1.90:9000/; # This uses the upstream directive definition to load balance
   }
}

上文参考：https://min.io/docs/minio/linux/integrations/setup-nginx-proxy-with-minio.html
去掉了负载均衡部分

测试ssl

1
2
service nginx start
MINIO_ROOT_USER=admin MINIO_ROOT_PASSWORD=12345678 ./minio server /mnt/data --console-address ":9001"

写入服务

配置和环境变量

通常都是使用 supervisorctl，使用也比较简单。但是今天准备尝试用 alpine的rc服务管理器来处理。 minio 已经不建议用 –config-dir 来管理配置文件，而是把配置文件放到了目录的 .minio.sys

1
ls /mnt/data/.minio.sys

我这里直接export 放到 start_pre ~~ 我这里直接改环境变量 echo “export MINIO_ROOT_USER=admin”»/etc/profile
echo “export MINIO_ROOT_PASSWORD=123456”»/etc/profile
source /etc/profile ~~

添加系统服务

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
vi /etc/init.d/minio 
-----------------------
#!/sbin/openrc-run

name="minio"
command="/root/minio "
command_args="server /mnt/data --console-address ':9001'"
command_background="yes"
pidfile="/run/$RC_SVCNAME.pid"
start_pre() {
    export MINIO_ROOT_USER=admin
    export MINIO_ROOT_PASSWORD=12345678
}
depend() {
        need localmount
        need logger
}

此处有一个小坑，密码长度不能低于8位用户名不能低于三位。另外这个账户密码是面板以及 mc的管理里面。面板可以禁用 mc的不确定，所以密码强度最好高一点

1
2
3
4
chmod +x /etc/init.d/minio
rc-service --list
rc-update add minio boot
service minio start