本篇主要记录创建社区版 JuiceFS PVC 的脚本，方便快速配置。组件部署可以参考 使用 Fluid 和 JuiceFS 在 Kubernetes 管理数据 。
1. 设置环境变量
1
2
3
4
5
6
| export ACCESS_KEY=
export SECRET_KEY=
export BUCKET=
export ENDPOINT=ks3-cn-beijing-internal.ksyun.com
export BUCKET_ENPOINT=$BUCKET.$ENDPOINT
export PROVIDER=ks3
|
1
2
3
4
5
| export NAMESPACE=
export PVC_NAME=
export NODE_SELECTOR_KEY=
export NODE_SELECTOR_VALUE=
|
export JUICEFS_IMAGE=juicedata/juicefs-fuse
export DEMO_IMAGE=shaowenchen/demo:ubuntu
如果是 Redis 配置
1
2
3
4
| export REDIS_PASSWORD=
#ip:port/database
export REDIS_ENDPOINT=
export META_SERVER=redis://:$REDIS_PASSWORD@$REDIS_ENDPOINT
|
如果是 PostgreSQL 配置
1
2
3
4
5
6
| export POSTGRES_USER=admin
export POSTGRES_PASSWORD=
export POSTGRES_IP=
export POSTGRES_PORT=5432
export POSTGRES_DB=
export META_SERVER=postgres://$POSTGRES_USER:$POSTGRES_PASSWORD@$POSTGRES_IP:$POSTGRES_PORT/$POSTGRES_DB?sslmode=disable
|
1
| psql -U $POSTGRES_USER -h $POSTGRES_IP -p $POSTGRES_PORT -d postgres -c "CREATE DATABASE $POSTGRES_DB;"
|
2. 初始化 JuiceFS
1
2
3
4
5
6
| juicefs format \
--trash-days=1 \
--storage $PROVIDER \
--bucket $BUCKET_ENPOINT \
$META_SERVER \
$BUCKET
|
trash-days 为 0 表示禁用回收站。
3. 创建 Secret
1
2
3
4
5
6
7
8
9
10
11
12
| kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
name: $PVC_NAME
namespace: $NAMESPACE
type: Opaque
stringData:
metaurl: $META_SERVER
access-key: $ACCESS_KEY
secret-key: $SECRET_KEY
EOF
|
4. 创建 Dataset
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
| kubectl apply -f - <<EOF
apiVersion: data.fluid.io/v1alpha1
kind: Dataset
metadata:
name: $PVC_NAME
namespace: $NAMESPACE
spec:
accessModes:
- ReadWriteMany
mounts:
- name: $BUCKET
mountPoint: "juicefs:///"
options:
bucket: $BUCKET_ENPOINT
storage: $PROVIDER
encryptOptions:
- name: metaurl
valueFrom:
secretKeyRef:
name: $PVC_NAME
key: metaurl
- name: access-key
valueFrom:
secretKeyRef:
name: $PVC_NAME
key: access-key
- name: secret-key
valueFrom:
secretKeyRef:
name: $PVC_NAME
key: secret-key
EOF
|
5. 创建 JuiceFSRuntime
注意修改 nodeSelector，因为 worker 会占用大量资源。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
| kubectl apply -f - <<EOF
apiVersion: data.fluid.io/v1alpha1
kind: JuiceFSRuntime
metadata:
name: $PVC_NAME
namespace: $NAMESPACE
spec:
replicas: 1
juicefsVersion:
image: $JUICEFS_IMAGE
imageTag: ce-v1.1.0
fuse:
image: $JUICEFS_IMAGE
imageTag: ce-v1.1.0
cleanPolicy: OnDemand
worker:
nodeSelector:
$NODE_SELECTOR_KEY: "$NODE_SELECTOR_VALUE"
resources:
limits:
cpu: 15
memory: 100Gi
requests:
cpu: 1
memory: 10Gi
tieredstore:
levels:
- mediumtype: SSD
path: /data/jfs/cache
quota: 40960 # 40GiB
EOF
|
如果是企业版的 JuiceFS 可以在 fuse 和 worker 部分配置 cache-group。
1
2
| options:
"cache-group": "$PVC_NAME"
|
需要注意，对于同一个高速网络下的节点，才能用同一个 cache-group。
6. 创建测试 Pod
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
| kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
name: ${PVC_NAME}-deploy
namespace: $NAMESPACE
spec:
replicas: 1
selector:
matchLabels:
app: ${PVC_NAME}-demo
template:
metadata:
labels:
app: ${PVC_NAME}-demo
spec:
containers:
- name: demo
image: $DEMO_IMAGE
volumeMounts:
- mountPath: /data/jfs
name: data
nodeSelector:
$NODE_SELECTOR_KEY: "$NODE_SELECTOR_VALUE"
volumes:
- name: data
persistentVolumeClaim:
claimName: $PVC_NAME
EOF
|
7. 性能测试
1
| kubectl -n $NAMESPACE exec -it $PVC_NAME -- bash
|
1
| curl -sSL https://d.juicefs.com/install | sh -
|
