cactus

[zhanglu]

cactus测试记录：

官网 ：https://github.com/ComparativeGenomicsToolkit/cactus

一、安装

二、运行

1. step-by-step

1. 打印分步运行的所有命令；

cactus-prepare examples/evolverMammals.txt --outDir steps-output --outSeqFile steps-output/evovlerMammals.txt --outHal steps-output/evolverMammals.hal --jobStore jobstore

2. 简化运行方式 --preprocessOnly

cactus-prepare examples/evolverMammals.txt --outDir steps-output --outSeqFile steps-output/evovlerMammals.txt --outHal steps-output/evolverMammals.hal --jobStore jobstore --preprocessOnly

3. GPU加速 --gpu

The GPU-enabled Docker releases have this turned on by default. It is also possible to manually install it from git and then enable it in cactus via the --configFile cactus option. A template can be found here, and modified to activate the GPU by setting gpuLastz="true" and realign="0" in the <caf> section

cactus-prepare examples/evolverMammals.txt --gpu --outDir steps-output --outSeqFile steps-output/evovlerMammals.txt --outHal steps-output/evolverMammals.hal --jobStore jobstore

有gpu
real 38m50.343s
无gpu
real 52m49.121s

4. 并行优化

并行：只有在独立模块，中并列的命令才可以并列运行，

例1：运行示例中Preprocessor：

1. cactus-preprocess jobstore/0 local.txt steps-output/local.txt --inputNames simHuman_chr6 simCow_chr6 simDog_chr6 --realTimeLogging --logInfo --retryCount 0
2. cactus-preprocess jobstore/1 local.txt steps-output/local.txt --inputNames simMouse_chr6 simRat_chr6 --realTimeLogging --logInfo --retryCount 0

例2：运行示例中Round 0：以下1和2中两部分分别包含三条命令，1和2是并列关系，可并列运行，但 1和2中的三条命令是串行关系，不能并列运行；

1.   cactus-blast jobstore/2 steps-output/local.txt steps-output/Anc2.cigar --root Anc2 --realTimeLogging --logInfo --retryCount 0
cactus-align jobstore/3 steps-output/local.txt steps-output/Anc2.cigar steps-output/Anc2.hal --root Anc2 --realTimeLogging --logInfo --retryCount 0 --maxCores 2
hal2fasta steps-output/Anc2.hal Anc2 --hdf5InMemory > steps-output/Anc2.fa


2.  cactus-blast jobstore/4 steps-output/local.txt steps-output/mr.cigar --root mr --realTimeLogging --logInfo --retryCount 0 
cactus-align jobstore/5 steps-output/local.txt steps-output/mr.cigar steps-output/mr.hal --root mr --realTimeLogging --logInfo --retryCount 0 --maxCores 2
hal2fasta steps-output/Anc1.hal Anc1 --hdf5InMemory > steps-output/Anc1.fa

5.资源选择依据

6. 运行示例

## Preprocessor
cactus-preprocess jobstore/0 local.txt steps-output/local.txt --inputNames simHuman_chr6 simCow_chr6 simDog_chr6 --realTimeLogging --logInfo --retryCount 0


real	0m42.069s
user	1m33.210s
sys	1m49.028s



cactus-preprocess jobstore/1 local.txt steps-output/local.txt --inputNames simMouse_chr6 simRat_chr6 --realTimeLogging --logInfo --retryCount 0 
real	0m39.688s
user	1m4.916s
sys	1m31.333s


## Alignment

### Round 0
cactus-blast jobstore/2 steps-output/local.txt steps-output/Anc2.cigar --root Anc2 --realTimeLogging --logInfo --retryCount 0
real	1m53.030s
user	2m59.048s
sys	3m34.287s

cactus-align jobstore/3 steps-output/local.txt steps-output/Anc2.cigar steps-output/Anc2.hal --root Anc2 --realTimeLogging --logInfo --retryCount 0 --maxCores 2

real	18m22.450s
user	8m34.911s
sys	9m40.047s


hal2fasta steps-output/Anc2.hal Anc2 --hdf5InMemory > steps-output/Anc2.fa
real	0m0.109s
user	0m0.011s
sys	0m0.011s


cactus-blast jobstore/4 steps-output/local.txt steps-output/mr.cigar --root mr --realTimeLogging --logInfo --retryCount 0 
real	1m51.072s
user	3m8.077s
sys	3m15.112s

cactus-align jobstore/5 steps-output/local.txt steps-output/mr.cigar steps-output/mr.hal --root mr --realTimeLogging --logInfo --retryCount 0 --maxCores 2
real	19m22.049s
user	10m10.130s
sys	11m35.772s

hal2fasta steps-output/mr.hal mr --hdf5InMemory > steps-output/mr.fa

real	0m0.019s
user	0m0.015s
sys	0m0.005s


### Round 1
cactus-blast jobstore/6 steps-output/local.txt steps-output/Anc1.cigar --root Anc1 --realTimeLogging --logInfo --retryCount 0 

real	1m19.923s
user	2m9.523s
sys	2m18.150s

-rw-r--r-- 1 root root  325612 Feb  4 06:41 Anc1.cigar.ig_coverage_0
-rw-r--r-- 1 root root  372762 Feb  4 06:41 Anc1.cigar.ig_coverage_1
-rw-r--r-- 1 root root  575910 Feb  4 06:41 Anc1.cigar.og_fragment_0
-rw-r--r-- 1 root root 1623235 Feb  4 06:41 Anc1.cigar.secondary


cactus-align jobstore/7 steps-output/local.txt steps-output/Anc1.cigar steps-output/Anc1.hal --root Anc1 --realTimeLogging --logInfo --retryCount 0 --maxCores 2

real	17m24.211s
user	8m31.867s
sys	10m25.042s

-rw-r--r-- 1 root root  882309 Feb  4 06:41 Anc1.cigar
-rw-r--r-- 1 root root  325612 Feb  4 06:41 Anc1.cigar.ig_coverage_0
-rw-r--r-- 1 root root  372762 Feb  4 06:41 Anc1.cigar.ig_coverage_1
-rw-r--r-- 1 root root  575910 Feb  4 06:41 Anc1.cigar.og_fragment_0
-rw-r--r-- 1 root root 1623235 Feb  4 06:41 Anc1.cigar.secondary
-rw-r--r-- 1 root root 1435545 Feb  4 07:00 Anc1.hal

hal2fasta steps-output/Anc1.hal Anc1 --hdf5InMemory > steps-output/Anc1.fa

real	0m0.018s
user	0m0.011s
sys	0m0.007s


### Round 2
cactus-blast jobstore/8 steps-output/local.txt steps-output/Anc0.cigar --root Anc0 --realTimeLogging --logInfo --retryCount 0 

real	1m13.752s
user	1m22.428s
sys	1m34.542s

-rw-r--r-- 1 root root  211228 Feb  4 07:06 Anc0.cigar
-rw-r--r-- 1 root root  326775 Feb  4 07:06 Anc0.cigar.secondary


cactus-align jobstore/9 steps-output/local.txt steps-output/Anc0.cigar steps-output/local.hal --root Anc0 --realTimeLogging --logInfo --retryCount 0 --maxCores 2

real	13m21.857s
user	5m41.658s
sys	9m4.849s

-rw-r--r-- 1 root root  211228 Feb  4 07:06 Anc0.cigar
-rw-r--r-- 1 root root  326775 Feb  4 07:06 Anc0.cigar.secondary



hal2fasta steps-output/local.hal Anc0 --hdf5InMemory > steps-output/Anc0.fa

real	0m0.018s
user	0m0.018s
sys	0m0.000s


## HAL merging
halAppendSubtree steps-output/local.hal steps-output/Anc1.hal Anc1 Anc1 --merge --hdf5InMemory

-rw-r--r-- 1 root root 2133914 Feb  4 07:23 local.hal

halAppendSubtree steps-output/local.hal steps-output/Anc2.hal Anc2 Anc2 --merge --hdf5InMemory
halAppendSubtree steps-output/local.hal steps-output/mr.hal mr mr --merge --hdf5InMemory

-rw-r--r-- 1 root root 4945057 Feb  4 07:24 local.hal