软件部署及教程

生产node2环境记录

zhanglu — Fri, 25 Jul 2025 06:59:56 GMT

sda 8:0 0 3.7T 0 disk
├─sda1 8:1 0 487M 0 part /boot/efi
├─sda2 8:2 0 3.7G 0 part
├─sda3 8:3 0 465.7G 0 part /home
├─sda4 8:4 0 465.7G 0 part /var
├─sda5 8:5 0 465.7G 0 part /usr
├─sda6 8:6 0 465.7G 0 part /opt
└─sda7 8:7 0 1.8T 0 part /
sdb 8:16 0 8.9T 0 disk
└─sdb1 8:17 0 8.9T 0 part /ceph_disk1
sdc 8:32 0 8.9T 0 disk
└─sdc3 8:35 0 3.1G 0 part
sdd 8:48 0 8.9T 0 disk
└─sdd1 8:49 0 8.9T 0 part
sde 8:64 0 8.9T 0 disk
└─sde3 8:67 0 3.1G 0 part
sdf 8:80 0 8.9T 0 disk
└─sdf3 8:83 0 3.1G 0 part
sdg 8:96 0 8.9T 0 disk
└─sdg1 8:97 0 8.9T 0 part
sdh 8:112 0 8.9T 0 disk
└─sdh3 8:115 0 3.1G 0 part
sdi 8:128 0 8.9T 0 disk
└─sdi3 8:131 0 3.1G 0 part
sdj 8:144 0 8.9T 0 disk
└─sdj3 8:147 0 3.1G 0 part

生产node3磁盘挂载记录

zhanglu — Mon, 02 Dec 2024 08:34:33 GMT

sda 8:0 0 10.7T 0 disk
sdb 8:16 0 10.7T 0 disk
sdc 8:32 0 10.7T 0 disk
sdd 8:48 0 10.7T 0 disk
sde 8:64 0 10.7T 0 disk
sdf 8:80 0 10.7T 0 disk
sdg 8:96 0 10.7T 0 disk
sdh 8:112 0 10.7T 0 disk
sdi 8:128 0 10.7T 0 disk
sdj 8:144 0 10.7T 0 disk
sdk 8:160 0 10.7T 0 disk
sdl 8:176 0 10.7T 0 disk
sdm 8:192 0 10.7T 0 disk
sdn 8:208 0 10.7T 0 disk
sdo 8:224 0 10.7T 0 disk
sdp 8:240 0 10.7T 0 disk
sdq 65:0 0 10.7T 0 disk
sdr 65:16 0 10.7T 0 disk
sds 65:32 0 10.7T 0 disk
sdt 65:48 0 10.7T 0 disk
└─sdt1 65:49 0 10.7T 0 part /public5
sdu 65:64 0 10.7T 0 disk
└─sdu1 65:65 0 10.7T 0 part /public4
sdv 65:80 0 10.7T 0 disk
└─sdv1 65:81 0 10.7T 0 part /public3
sdw 65:96 0 10.7T 0 disk
└─sdw1 65:97 0 10.7T 0 part /public2
sdx 65:112 0 10.7T 0 disk /public1
sdy 65:128 0 10.7T 0 disk
├─sdy1 65:129 0 512M 0 part /boot/efi
├─sdy2 65:130 0 500M 0 part /boot
├─sdy3 65:131 0 32G 0 part
└─sdy4 65:132 0 10.7T 0 part /

cirrocumulus 安装说明

zhanglu — Wed, 27 Nov 2024 11:11:53 GMT

@zhanglu dockerhub.genostack.com:8090/library/mongo:latest

blast_gpu

zhanglu — Fri, 27 Sep 2024 03:40:22 GMT

测试

gpu time /cephfs_data/app/blast_gpu/ncbi-blast-2.2.28+-src/c++/GCC480-ReleaseMT64/bin/blastp -query /cephfs_data/app/blast_gpu/database/test2.fa -db /cephfs_data/app/blast_gpu/database/nr -gpu t -num_threads 30 real 13m42.160s user 38m52.082s sys 0m13.170s cpu

time /cephfs_data/app/blast_gpu/ncbi-blast-2.2.28+-src/c++/GCC480-ReleaseMT64/bin/blastp -query /cephfs_data/app/blast_gpu/database/test2.fa -db /cephfs_data/app/blast_gpu/database/nr -gpu f -num_threads 30

real 23m23.076s
user 695m50.953s
sys 1m35.377s

ubuntu 22.04 docker 安装

zhanglu — Wed, 17 Jul 2024 07:11:06 GMT

https://blog.csdn.net/qq_26328861/article/details/136205861

novnc-web控制桌面

zhangfanglin — Mon, 20 May 2024 03:40:09 GMT

启动x11vnc 最后一个链接断开时，不停止vnc

-forever

x11vnc -forever

kubesphere安装问题记录

zhanglu — Sat, 11 May 2024 08:13:51 GMT

63343368-47a1-427b-a972-d398f02c244e-image.png

mountvolume.setup failed for volume "kubesphere-config" : configmap "kubesphere-config" not found

查询报错信息，显示没有kubesphere-config这个configmap。应该去创建kubesphere-config这个configmap。但是在kubesphere的文档上看到有人调整了ks-apiserver的limit memory就好了，我查看ks-apiserver默认的limit memorr设置的是1G，我调整成4G,没想到ks-apiserver服务还真运行起来了。我也很纳闷，configmap跟内存有啥关系，先不管了。继续处理问题。

genostack_k8s_imageslist

zhanglu — Wed, 24 Apr 2024 10:46:55 GMT

docker pull dockerhub.genostack.com:8090/rook/cephcsi/cephcsi:v3.7.2
docker pull dockerhub.genostack.com:8090/rook/csi-node-driver-registrar:v2.5.1
docker pull dockerhub.genostack.com:8090/rook/csi-resizer:v1.6.0
docker pull dockerhub.genostack.com:8090/rook/csi-provisioner:v3.3.0
docker pull dockerhub.genostack.com:8090/rook/csi-snapshotter:v6.1.0
docker pull dockerhub.genostack.com:8090/rook/csi-attacher:v4.0.0
docker pull dockerhub.genostack.com:8090/rook/ceph/ceph:v17.2.5

docker tag dockerhub.genostack.com:8090/rook/cephcsi/cephcsi:v3.7.2 quay.io/cephcsi/cephcsi:v3.7.2
docker tag dockerhub.genostack.com:8090/rook/csi-node-driver-registrar:v2.5.1 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1
docker tag dockerhub.genostack.com:8090/rook/csi-resizer:v1.6.0 registry.k8s.io/sig-storage/csi-resizer:v1.6.0
docker tag dockerhub.genostack.com:8090/rook/csi-provisioner:v3.3.0 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0
docker tag dockerhub.genostack.com:8090/rook/csi-snapshotter:v6.1.0 registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0
docker tag dockerhub.genostack.com:8090/rook/csi-attacher:v4.0.0 registry.k8s.io/sig-storage/csi-attacher:v4.0.0
docker tag dockerhub.genostack.com:8090/rook/ceph/ceph:v17.2.5 quay.io/ceph/ceph:v17.2.5

Nextflow流程

mengpf — Thu, 22 Feb 2024 02:25:39 GMT

quantms

定量质谱工作流程。目前支持具有 DDA-LFQ、DDA-Isobaric 和 DIA-LFQ 定量复杂实验设计的蛋白质组学实验。

nextflow run quantms/main.nf \ --outdir result \ --labelling_type "label free sample" \ --input '/ceph_disk3/mengpf/work/Project/Nextflow/quantms/data/BSA_design_urls.tsv' \ --database '/ceph_disk3/mengpf/work/Project/Nextflow/quantms/data/18Protein_SoCe_Tr_detergents_trace_target_decoy.fasta' \ --posterior_probabilities "fit_distributions" \ --search_engines "msgf,comet" \ --decoy_string "rev" \ --add_triqler_output true \ --protein_level_fdr_cutoff 1.0 \ --psm_level_fdr_cutoff 1.0 \ --acquisition_method "dda" \ --quantify_decoys true \ -profile docker

大服务器新增节点，搭建新版本k8s集群记录

zhanglu — Mon, 11 Dec 2023 10:27:08 GMT

/opt/app/elasticsearch/data.json 备份到node3:/public3/node1/elasticsearch

大服务器服务部署命令清单

zhanglu — Wed, 06 Dec 2023 03:58:43 GMT

@zhanglu /opt/app/genostack_v3_service/genostack_tool/amplicon_analysis_16s.yaml

rsync server 配置

anneng — Wed, 22 Nov 2023 16:27:19 GMT

sudo systemctl start rsync.service
sudo systemctl status rsync.service
sudo systemctl enable rsync.service

nexus 部署

zhanglu — Mon, 20 Nov 2023 11:03:11 GMT

docker run -d --network=host -v /ceph_disk2/app/nexus:/nexus-data sonatype/nexus3

Ubuntu安装向日葵并且设置自启动

zhanglu — Fri, 01 Sep 2023 04:53:12 GMT

https://blog.csdn.net/matlab001/article/details/127621176

单机管理工具portainer 安装

zhanglu — Wed, 30 Aug 2023 04:07:00 GMT

docker run -d --restart=always --name=portainer -p 9090:9000 -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data dockerhub.genostack.com:8090/library/portainer

docker volume rm portainer_data && docker volume create portainer_data

snakemake流程-CellChat

zhanglu — Fri, 18 Aug 2023 10:56:05 GMT

https://github.com/sqjin/CellChat

redash安装

zhangfanglin — Wed, 09 Aug 2023 09:00:56 GMT

首先，需要克隆 Git 存储库:


git clone https://github.com/getredash/redash.git
cd redash/

在根目录下创建一个. env 文件，并设置需要的任何环境变量。

touch .env

环境变量设置

REDASH_REDIS_URL
REDASH_DISABLE_PUBLIC_URLS
REDASH_BLOCKED_DOMAINS
REDASH_PROXIES_COUNT
REDASH_STATSD_HOST
REDASH_STATSD_PORT
REDASH_STATSD_PREFIX
REDASH_STATSD_USE_TAGS
REDASH_DATABASE_URL
REDASH_QUERY_RESULTS_CLEANUP_ENABLED
REDASH_QUERY_RESULTS_CLEANUP_COUNT
REDASH_QUERY_RESULTS_CLEANUP_MAX_AGE
REDASH_SCHEMAS_REFRESH_SCHEDULE
REDASH_AUTH_TYPE
REDASH_ENFORCE_HTTPS
REDASH_INVITATION_TOKEN_MAX_AGE
REDASH_MULTI_ORG
REDASH_GOOGLE_CLIENT_ID
REDASH_GOOGLE_CLIENT_SECRET
REDASH_REMOTE_USER_LOGIN_ENABLED
REDASH_REMOTE_USER_HEADER
REDASH_LDAP_LOGIN_ENABLED
REDASH_LDAP_URL
REDASH_LDAP_BIND_DN
REDASH_LDAP_BIND_DN_PASSWORD
REDASH_LDAP_DISPLAY_NAME_KEY
REDASH_LDAP_EMAIL_KEY
REDASH_LDAP_CUSTOM_USERNAME_PROMPT
REDASH_LDAP_SEARCH_TEMPLATE
REDASH_LDAP_SEARCH_DN
REDASH_STATIC_ASSETS_PATH
REDASH_JOB_EXPIRY_TIME
REDASH_COOKIE_SECRET
REDASH_LOG_LEVEL
REDASH_MAIL_SERVER
REDASH_MAIL_PORT
REDASH_MAIL_USE_TLS
REDASH_MAIL_USE_SSL
REDASH_MAIL_USERNAME
REDASH_MAIL_PASSWORD
REDASH_MAIL_DEFAULT_SENDER
REDASH_MAIL_MAX_EMAILS
REDASH_MAIL_ASCII_ATTACHMENTS
REDASH_HOST
REDASH_ALERTS_DEFAULT_MAIL_SUBJECT_TEMPLATE
REDASH_ALERTS_DEFAULT_MAIL_BODY_TEMPLATE_FILE
REDASH_THROTTLE_LOGIN_PATTERN
REDASH_LIMITER_STORAGE
REDASH_CORS_ACCESS_CONTROL_ALLOW_ORIGIN
REDASH_CORS_ACCESS_CONTROL_ALLOW_CREDENTIALS
REDASH_CORS_ACCESS_CONTROL_REQUEST_METHOD
REDASH_CORS_ACCESS_CONTROL_ALLOW_HEADERS
REDASH_ENABLED_QUERY_RUNNERS
REDASH_ADDITIONAL_QUERY_RUNNERS
REDASH_DISABLED_QUERY_RUNNERS
REDASH_SCHEDULED_QUERY_TIME_LIMIT
REDASH_ADHOC_QUERY_TIME_LIMIT
REDASH_ENABLED_DESTINATIONS
REDASH_ADDITIONAL_DESTINATIONS
REDASH_EVENT_REPORTING_WEBHOOKS
REDASH_SENTRY_DSN
REDASH_ALLOW_SCRIPTS_IN_USER_INPUT
REDASH_DASHBOARD_REFRESH_INTERVALS
REDASH_QUERY_REFRESH_INTERVALS
REDASH_PASSWORD_LOGIN_ENABLED
REDASH_SAML_METADATA_URL
REDASH_SAML_ENTITY_ID
REDASH_SAML_NAMEID_FORMAT
REDASH_DATE_FORMAT
REDASH_JWT_LOGIN_ENABLED
REDASH_JWT_AUTH_ISSUER
REDASH_JWT_AUTH_PUBLIC_CERTS_URL
REDASH_JWT_AUTH_AUDIENCE
REDASH_JWT_AUTH_ALGORITHMS
REDASH_JWT_AUTH_COOKIE_NAME
REDASH_JWT_AUTH_HEADER_NAME
REDASH_FEATURE_SHOW_QUERY_RESULTS_COUNT
REDASH_VERSION_CHECK
REDASH_FEATURE_DISABLE_REFRESH_QUERIES
REDASH_FEATURE_SHOW_PERMISSIONS_CONTROL
REDASH_FEATURE_ALLOW_CUSTOM_JS_VISUALIZATIONS
REDASH_FEATURE_AUTO_PUBLISH_NAMED_QUERIES
REDASH_BIGQUERY_HTTP_TIMEOUT
REDASH_SCHEMA_RUN_TABLE_SIZE_CALCULATIONS
REDASH_WEB_WORKERS
REDASH_QUERY_RESULTS_EXPIRED_TTL_ENABLED
REDASH_QUERY_RESULTS_EXPIRED_TTL

生成Redash秘钥
Flask recommends using the secrets built-in Python module:



$ python -c 'import secrets; print(secrets.token_hex())'

'192b9bdd22ab9ed4d12e236c78afcb9a393ec15f71bbf5dc987d54727823bcbf'

Our cloud images use the CLI tool pwgen:


$ pwgen -1s 64
QinPGTd7Ulec03lar0vkI9ojqmXsuw4VOyirnC5NuvEdJSCwLwesmknNygXITunT
$ vi .env
REDASH_COOKIE_SECRET=d4taScEBEW7MwBgb8ZHUL18M6ldfZKNq44H4xd1ik1ZiZ42Ba8SfMo23AjBqC1oB

数据库初始化

#修改env配置(数据库链接)
$ vi .env
REDASH_DATABASE_URL=postgresql://postgres:pwd@192.168.1.2:5435/redash

$ docker-compose run server create_db

git仓库中没有前端的样式等文件--处理方式

#下载线上的包
docker pull redash/redash
docker run -it redash/redash bash
#将包中的dist文件夹复制出来-放置redash项目的client中
docker cp 84c31ea798a4:/app/client/dist /var/www/html/docker_app/redash/redash/client

启动服务

docker-compose up -d

RATH服务化部署

zhangfanglin — Fri, 04 Aug 2023 08:09:43 GMT

本地：

下载代码

git clone https://github.com/Kanaries/Rath.git && cd Rath

在本地设置RATH客户端：

yarn install
yarn workspace rath-client build

启动RATH客户端：

yarn workspace rath-client start

如果默认端口（localhost:3000）不可用，RATH将自动提示一个新的端口建议。输入y并按下Enter键启动RATH。

docker

dockerfile

FROM ubuntu:20.04
USER root
RUN apt-get update && apt-get install supervisor nginx vim net-tools  curl gnupg2 -y

RUN curl -s https://deb.nodesource.com/setup_16.x | bash
RUN apt-get update && apt-get install nodejs -y


RUN  apt-get remove cmdtest
RUN  apt-get remove yarn

RUN  curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg |  apt-key add -
RUN  echo "deb https://dl.yarnpkg.com/debian/ stable main" |  tee /etc/apt/sources.list.d/yarn.list
RUN  apt-get update && apt-get install yarn -y

ADD Rath /var/www/html/Rath
WORKDIR /var/www/html/Rath
RUN node -v
RUN yarn install
RUN yarn workspace rath-client build
#RUN yarn workspace rath-client start

ADD supervisord.conf /etc/supervisor/conf.d/
ENTRYPOINT supervisord -c /etc/supervisor/supervisord.conf

supervisord配置

[supervisord]
nodaemon=true
logfile=/dev/null
logfile_maxbytes=0
pidfile=/run/supervisord.pid


[program:rath]
command=yarn workspace rath-client start
stdout_logfile=/var/log/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/var/log/stderr
stderr_logfile_maxbytes=0
autorestart=false
startretries=0

Superset安装

zhangfanglin — Fri, 04 Aug 2023 03:05:14 GMT

根据设置外网不能访问配置方式：

#修改配置 vi docker/.env SESSION_COOKIE_SAMESITE=None SESSION_COOKIE_SECURE=False SESSION_COOKIE_HTTPONLY=False WTF_CSRF_ENABLED=False DATABASE_PORT= DATABASE_DIALECT= POSTGRES_DB= POSTGRES_USER= POSTGRES_PASSWORD= REDIS_HOST= REDIS_PORT= #将ports修改为:0.0.0.0:port vi docker-compose-non-dev.yml superset: env_file: docker/.env-non-dev image: *superset-image container_name: superset_app command: ["/app/docker/docker-bootstrap.sh", "app-gunicorn"] user: "root" restart: unless-stopped ports: - 0.0.0.0:6013:8088 #修改版本 TAG=2.1.0 docker-compose -f superset/docker-compose-non-dev.yml up -d

art pbsim3 模拟数据

zhanglu — Thu, 27 Jul 2023 10:27:30 GMT

二代： art

https://www.niehs.nih.gov/research/resources/software/biostatistics/art/index.cfm

Dockerfile

from ubuntu:20.04
run apt update && apt-get install -y wget && wget https://www.niehs.nih.gov/research/resources/assets/docs/artbinmountrainier2016.06.05linux64.tgz
run tar -zxvf artbinmountrainier2016.06.05linux64.tgz
run mv art_bin_MountRainier/art_illumina /usr/bin/ && rm artbinmountrainier2016.06.05linux64.tgz

运行命令：

art_illumina -sam -i seq_reference.fa -l 50 -f 10 -o ./outdir/dat_single_end
art_illumina -p -sam -i seq_reference.fa -l 50 -f 20 -m 200 -s 10 -o ./outdir/dat_paired_end

三黍lims部署记录

zhanglu — Fri, 07 Jul 2023 10:23:49 GMT

1. 启动mysql;

注意设置：

--lower-case-table-names=1
set global log_bin_trust_function_creators=TRUE;

docker run -d \
-v /cephfs_data/genostack_v3/genostack_lims/mysql/data:/var/lib/mysql \
-p 3306:3306 --name mysql -e MYSQL_ROOT_PASSWORD=super mysql:8.0.33 --lower-case-table-names=1

启动需等待几分钟后才能远程连接；

2. 启动redis:

docker run -p 6379:6379 -v /opt/genostack_v3/genostack_redis/redis-data/:/dataredis:5.0.5 redis-server

3. 启动lims

jdk1.8:
方式一：jar包：
将配置文件和jar包放置同一路径：
java -Djava.security.egd=file:/dev/./urandom -jar lims5-ms-sanshu-14.1.3.jar
方式二：docker
Dockerfile

FROM dockerhub.genostack.com/sanshu/limsbaseimage:v1.0
VOLUME /data
VOLUME /app/files

#具体项目jar名字，jar和Dockerfile在一个目录
ADD *.jar /app/lims5-ms-sanshu-14.1.3.jar

WORKDIR /app

RUN mkdir -p /app/logs
ADD application.yml /app/application.yml
ADD application-dev.yml /app/application-dev.yml

#根据项目情况实际调整，建议为项目内存的一半
ENV JAVA_OPTS="\
-Xmx9082m \
-Xms9082m"

ENTRYPOINT exec java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom  -jar lims5-ms-sanshu-14.1.3.jar.jar

cromwell + tesk 重启及gpu支持

zhanglu — Tue, 04 Jul 2023 07:05:32 GMT

cromwell 重启wdl编写和测试规范： 1. wdl数据类型：

列表文件类型：
File file = “/tmp/a.txt”

列表字符串类型：
String str = “abc”

注意点：路径参数：使用String
当task输出是一个路径时：

output { File metadata_result = "metadata.tsv" File data_dir = "data" File fqc_dir = "data1" String fqc_dir_str = "${fqc_dir}" }

先使用File data_dir = "data", 才能接收到 "data"的完整路径，但是在后续使用当中需要将File类型的路径转化成 String, String fqc_dir_str = "${fqc_dir}",

因为：在File类型里，callcaching hash不支持File类型的path,
如果在输出里用File接收，在输入里用String,使用，则会导致输入的hash值判定为新值，该步骤就无法参与重启

后续编写WDL测试规范：

功能测试：重启测试

果壳项目部署文档

zhangfanglin — Tue, 04 Jul 2023 02:23:08 GMT

数据库备份：
gk_l.sql
gk_g.sql

nextflow源码修改及部署

anneng — Wed, 28 Jun 2023 09:33:02 GMT

配置文件的优先级
Selector priority

When mixing generic process configuration and selectors the following priority rules are applied (from lower to higher):

Process generic configuration.

Process specific directive defined in the workflow script.

withLabel selector definition.

withName selector definition.

安至项目部署路径

zhanglu — Wed, 28 Jun 2023 08:51:57 GMT

部署路径：/data_raid1/genostack_v3/
启动脚本：参考，/data_raid1/genostack_v3/start.sh

吉凯部署：/data_raid1/jikai_project/genostack_v3
启动脚本：/data_raid1/jikai_project/genostack_v3/start.sh