暗能星系
    • 登录
    • 搜索

    tmp

    张渌
    2
    262
    1755
    正在加载更多帖子
    • 从旧到新
    • 从新到旧
    • 最多赞同
    回复
    • 在新帖中回复
    登录后回复
    此主题已被删除。只有拥有主题管理权限的用户可以查看。
    • Z
      zhanglu 最后由 编辑

      [INFO] [12/18/2025 06:45:43.162] [cromwell-system-akka.dispatchers.api-dispatcher-139] [akka://cromwell-system/user/cromwell-service/WorkflowStoreActor/WorkflowStoreSubmitActor] Unspecified type (Unspecified version) workflow 5b890685-2085-4e03-8465-c317b1e1f23d submitted

      [INFO] [12/18/2025 06:47:53.027] [cromwell-system-akka.dispatchers.engine-dispatcher-57] [akka://cromwell-system/user/cromwell-service/WorkflowStoreActor/WorkflowStoreEngineActor] 1 new workflows fetched by cromid-d6f0ebc: 5b890685-2085-4e03-8465-c317b1e1f23d
      [INFO] [12/18/2025 06:47:53.027] [cromwell-system-akka.dispatchers.engine-dispatcher-57] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor] WorkflowManagerActor: Starting workflow UUID(5b890685-2085-4e03-8465-c317b1e1f23d)
      [INFO] [12/18/2025 06:47:53.027] [cromwell-system-akka.dispatchers.engine-dispatcher-57] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor] WorkflowManagerActor: Successfully started WorkflowActor-5b890685-2085-4e03-8465-c317b1e1f23d
      [INFO] [12/18/2025 06:47:53.029] [cromwell-system-akka.dispatchers.engine-dispatcher-55] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-5b890685-2085-4e03-8465-c317b1e1f23d/MaterializeWorkflowDescriptorActor] MaterializeWorkflowDescriptorActor [UUID(5b890685)]: Parsing workflow as WDL draft-2
      [INFO] [12/18/2025 06:47:53.031] [cromwell-system-akka.dispatchers.engine-dispatcher-55] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-5b890685-2085-4e03-8465-c317b1e1f23d/MaterializeWorkflowDescriptorActor] MaterializeWorkflowDescriptorActor [UUID(5b890685)]: Call-to-Backend assignments: msConvert.extractTICdata -> TESK
      [INFO] [12/18/2025 06:47:55.101] [cromwell-system-akka.dispatchers.engine-dispatcher-82] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-5b890685-2085-4e03-8465-c317b1e1f23d/WorkflowExecutionActor-5b890685-2085-4e03-8465-c317b1e1f23d] WorkflowExecutionActor-5b890685-2085-4e03-8465-c317b1e1f23d [UUID(5b890685)]: Starting msConvert.extractTICdata
      [WARN] [12/18/2025 06:48:04.714] [cromwell-system-akka.dispatchers.engine-dispatcher-55] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-5b890685-2085-4e03-8465-c317b1e1f23d/WorkflowExecutionActor-5b890685-2085-4e03-8465-c317b1e1f23d/5b890685-2085-4e03-8465-c317b1e1f23d-EngineJobExecutionActor-msConvert.extractTICdata:NA:1/BackendPreparationActor_for_5b890685:msConvert.extractTICdata👎1] BackendPreparationActor_for_5b890685:msConvert.extractTICdata👎1 [UUID(5b890685)]: Docker lookup failed
      [INFO] [12/18/2025 06:48:05.087] [cromwell-system-akka.dispatchers.backend-dispatcher-1210] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-5b890685-2085-4e03-8465-c317b1e1f23d/WorkflowExecutionActor-5b890685-2085-4e03-8465-c317b1e1f23d/5b890685-2085-4e03-8465-c317b1e1f23d-EngineJobExecutionActor-msConvert.extractTICdata:NA:1/5b890685-2085-4e03-8465-c317b1e1f23d-BackendJobExecutionActor-msConvert.extractTICdata:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(5b890685)msConvert.extractTICdata:NA:1]: echo "192.168.30.202:23099/target_meta/target_meta_dev:v3.3.7"
      [INFO] [12/18/2025 06:48:05.087] [cromwell-system-akka.dispatchers.backend-dispatcher-1210] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-5b890685-2085-4e03-8465-c317b1e1f23d/WorkflowExecutionActor-5b890685-2085-4e03-8465-c317b1e1f23d/5b890685-2085-4e03-8465-c317b1e1f23d-EngineJobExecutionActor-msConvert.extractTICdata:NA:1/5b890685-2085-4e03-8465-c317b1e1f23d-BackendJobExecutionActor-msConvert.extractTICdata:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(5b890685)msConvert.extractTICdata:NA:1]: Calculated TES outputs (found 4):
      Output(Some(rc),Some(msConvert.extractTICdata.rc),Some(/cephfs_data/genostack_v3/genostack_cromwell/cromwell-executions/msConvert/5b890685-2085-4e03-8465-c317b1e1f23d/call-extractTICdata/execution/rc),/cephfs_data/genostack_v3/genostack_cromwell/cromwell-executions/msConvert/5b890685-2085-4e03-8465-c317b1e1f23d/call-extractTICdata/execution/rc,Some(FILE))
      Output(Some(stdout),Some(msConvert.extractTICdata.stdout),Some(/cephfs_data/genostack_v3/genostack_cromwell/cromwell-executions/msConvert/5b890685-2085-4e03-8465-c317b1e1f23d/call-extractTICdata/execution/stdout),/cephfs_data/genostack_v3/genostack_cromwell/cromwell-executions/msConvert/5b890685-2085-4e03-8465-c317b1e1f23d/call-extractTICdata/execution/stdout,Some(FILE))
      Output(Some(stderr),Some(msConvert.extractTICdata.stderr),Some(/cephfs_data/genostack_v3/genostack_cromwell/cromwell-executions/msConvert/5b890685-2085-4e03-8465-c317b1e1f23d/call-extractTICdata/execution/stderr),/cephfs_data/genostack_v3/genostack_cromwell/cromwell-executions/msConvert/5b890685-2085-4e03-8465-c317b1e1f23d/call-extractTICdata/execution/stderr,Some(FILE))
      Output(Some(commandScript),Some(msConvert.extractTICdata.commandScript),Some(/cephfs_data/genostack_v3/genostack_cromwell/cromwell-executions/msConvert/5b890685-2085-4e03-8465-c317b1e1f23d/call-extractTICdata/execution/script),/cephfs_data/genostack_v3/genostack_cromwell/cromwell-executions/msConvert/5b890685-2085-4e03-8465-c317b1e1f23d/call-extractTICdata/execution/script,Some(FILE))
      [INFO] [12/18/2025 06:48:05.087] [cromwell-system-akka.dispatchers.backend-dispatcher-1210] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-5b890685-2085-4e03-8465-c317b1e1f23d/WorkflowExecutionActor-5b890685-2085-4e03-8465-c317b1e1f23d/5b890685-2085-4e03-8465-c317b1e1f23d-EngineJobExecutionActor-msConvert.extractTICdata:NA:1/5b890685-2085-4e03-8465-c317b1e1f23d-BackendJobExecutionActor-msConvert.extractTICdata:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(5b890685)msConvert.extractTICdata:NA:1]: Calculated TES inputs (found 1):
      cromwell.backend.impl.tes.Input(Some(commandScript),Some(msConvert.extractTICdata.commandScript),Some(/cephfs_data/genostack_v3/genostack_cromwell/cromwell-executions/msConvert/5b890685-2085-4e03-8465-c317b1e1f23d/call-extractTICdata/execution/script),/cephfs_data/genostack_v3/genostack_cromwell/cromwell-executions/msConvert/5b890685-2085-4e03-8465-c317b1e1f23d/call-extractTICdata/execution/script,Some(FILE),None)
      [INFO] [12/18/2025 06:48:05.087] [cromwell-system-akka.dispatchers.backend-dispatcher-1210] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-5b890685-2085-4e03-8465-c317b1e1f23d/WorkflowExecutionActor-5b890685-2085-4e03-8465-c317b1e1f23d/5b890685-2085-4e03-8465-c317b1e1f23d-EngineJobExecutionActor-msConvert.extractTICdata:NA:1/5b890685-2085-4e03-8465-c317b1e1f23d-BackendJobExecutionActor-msConvert.extractTICdata:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(5b890685)msConvert.extractTICdata:NA:1]: echo "192.168.30.202:23099/target_meta/target_meta_dev:v3.3.7"
      [INFO] [12/18/2025 06:48:09.544] [cromwell-system-akka.dispatchers.backend-dispatcher-1175] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-5b890685-2085-4e03-8465-c317b1e1f23d/WorkflowExecutionActor-5b890685-2085-4e03-8465-c317b1e1f23d/5b890685-2085-4e03-8465-c317b1e1f23d-EngineJobExecutionActor-msConvert.extractTICdata:NA:1/5b890685-2085-4e03-8465-c317b1e1f23d-BackendJobExecutionActor-msConvert.extractTICdata:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(5b890685)msConvert.extractTICdata:NA:1]: job id: task-3c5d0482
      [INFO] [12/18/2025 06:48:09.681] [cromwell-system-akka.dispatchers.backend-dispatcher-1138] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-5b890685-2085-4e03-8465-c317b1e1f23d/WorkflowExecutionActor-5b890685-2085-4e03-8465-c317b1e1f23d/5b890685-2085-4e03-8465-c317b1e1f23d-EngineJobExecutionActor-msConvert.extractTICdata:NA:1/5b890685-2085-4e03-8465-c317b1e1f23d-BackendJobExecutionActor-msConvert.extractTICdata:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(5b890685)msConvert.extractTICdata:NA:1]: Status change from - to Running
      [INFO] [12/18/2025 06:52:14.785] [cromwell-system-akka.dispatchers.backend-dispatcher-984] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-5b890685-2085-4e03-8465-c317b1e1f23d/WorkflowExecutionActor-5b890685-2085-4e03-8465-c317b1e1f23d/5b890685-2085-4e03-8465-c317b1e1f23d-EngineJobExecutionActor-msConvert.extractTICdata:NA:1/5b890685-2085-4e03-8465-c317b1e1f23d-BackendJobExecutionActor-msConvert.extractTICdata:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(5b890685)msConvert.extractTICdata:NA:1]: Job task-3c5d0482 is complete
      [INFO] [12/18/2025 06:52:14.785] [cromwell-system-akka.dispatchers.backend-dispatcher-1141] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-5b890685-2085-4e03-8465-c317b1e1f23d/WorkflowExecutionActor-5b890685-2085-4e03-8465-c317b1e1f23d/5b890685-2085-4e03-8465-c317b1e1f23d-EngineJobExecutionActor-msConvert.extractTICdata:NA:1/5b890685-2085-4e03-8465-c317b1e1f23d-BackendJobExecutionActor-msConvert.extractTICdata:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(5b890685)msConvert.extractTICdata:NA:1]: Status change from Running to Complete
      [INFO] [12/18/2025 06:52:16.222] [cromwell-system-akka.dispatchers.engine-dispatcher-112] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-5b890685-2085-4e03-8465-c317b1e1f23d/WorkflowExecutionActor-5b890685-2085-4e03-8465-c317b1e1f23d] WorkflowExecutionActor-5b890685-2085-4e03-8465-c317b1e1f23d [UUID(5b890685)]: Workflow msConvert complete. Final Outputs:
      [INFO] [12/18/2025 06:52:19.549] [cromwell-system-akka.dispatchers.engine-dispatcher-81] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor] WorkflowManagerActor: Workflow actor for 5b890685-2085-4e03-8465-c317b1e1f23d completed with status 'Succeeded'. The workflow will be removed from the workflow store.

      1 条回复 最后回复 回复 引用 0
      • Z
        zhanglu 最后由 编辑

        [INFO] [12/18/2025 07:05:32.113] [cromwell-system-akka.dispatchers.backend-dispatcher-1244] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-9c9d7421-4218-4a3f-9f46-87faa5edc47e/WorkflowExecutionActor-9c9d7421-4218-4a3f-9f46-87faa5edc47e/9c9d7421-4218-4a3f-9f46-87faa5edc47e-EngineJobExecutionActor-monoMeth.jsonFile:NA:1/9c9d7421-4218-4a3f-9f46-87faa5edc47e-BackendJobExecutionActor-monoMeth.jsonFile:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(9c9d7421)monoMeth.jsonFile:NA:1]: Job task-9a3b0fd9 is complete
        [INFO] [12/18/2025 07:05:32.113] [cromwell-system-akka.dispatchers.backend-dispatcher-1255] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-9c9d7421-4218-4a3f-9f46-87faa5edc47e/WorkflowExecutionActor-9c9d7421-4218-4a3f-9f46-87faa5edc47e/9c9d7421-4218-4a3f-9f46-87faa5edc47e-EngineJobExecutionActor-monoMeth.jsonFile:NA:1/9c9d7421-4218-4a3f-9f46-87faa5edc47e-BackendJobExecutionActor-monoMeth.jsonFile:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(9c9d7421)monoMeth.jsonFile:NA:1]: Status change from Running to Complete

        1 条回复 最后回复 回复 引用 0
        • Z
          zhanglu 最后由 编辑

          [INFO] [12/18/2025 07:26:16.905] [cromwell-system-akka.dispatchers.api-dispatcher-1503] [akka://cromwell-system/user/cromwell-service/WorkflowStoreActor/WorkflowStoreSubmitActor] Unspecified type (Unspecified version) workflow b3c8ad44-dc3b-4e87-9fe9-97c0db47031b submitted
          [INFO] [12/18/2025 07:26:31.291] [cromwell-system-akka.dispatchers.engine-dispatcher-68] [akka://cromwell-system/user/cromwell-service/WorkflowStoreActor/WorkflowStoreEngineActor] 1 new workflows fetched by cromid-247fb6c: b3c8ad44-dc3b-4e87-9fe9-97c0db47031b
          [INFO] [12/18/2025 07:26:31.291] [cromwell-system-akka.dispatchers.engine-dispatcher-68] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor] WorkflowManagerActor: Starting workflow UUID(b3c8ad44-dc3b-4e87-9fe9-97c0db47031b)
          [INFO] [12/18/2025 07:26:31.291] [cromwell-system-akka.dispatchers.engine-dispatcher-68] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor] WorkflowManagerActor: Successfully started WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b
          [INFO] [12/18/2025 07:26:31.291] [cromwell-system-akka.dispatchers.engine-dispatcher-68] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor] Retrieved 1 workflows from the WorkflowStoreActor
          [INFO] [12/18/2025 07:26:31.293] [cromwell-system-akka.dispatchers.engine-dispatcher-74] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/MaterializeWorkflowDescriptorActor] MaterializeWorkflowDescriptorActor [UUID(b3c8ad44)]: Parsing workflow as WDL draft-2
          [INFO] [12/18/2025 07:26:31.298] [cromwell-system-akka.dispatchers.engine-dispatcher-74] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/MaterializeWorkflowDescriptorActor] MaterializeWorkflowDescriptorActor [UUID(b3c8ad44)]: Call-to-Backend assignments: meta_workflow.Identify_analy -> TESK
          [INFO] [12/18/2025 07:26:33.508] [cromwell-system-akka.dispatchers.engine-dispatcher-74] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b] WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b [UUID(b3c8ad44)]: Starting meta_workflow.Identify_analy
          [INFO] [12/18/2025 07:26:41.267] [cromwell-system-akka.dispatchers.engine-dispatcher-55] [akka://cromwell-system/user/cromwell-service/JobExecutionTokenDispenser] Assigned new job execution tokens to the following groups: b3c8ad44: 1
          [WARN] [12/18/2025 07:26:41.289] [cromwell-system-akka.dispatchers.engine-dispatcher-66] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/BackendPreparationActor_for_b3c8ad44:meta_workflow.Identify_analy👎1] BackendPreparationActor_for_b3c8ad44:meta_workflow.Identify_analy👎1 [UUID(b3c8ad44)]: Docker lookup failed
          java.lang.Exception: Failed to get docker hash for dockerhub.genostack.com:8090/library/ubuntu:18.04 Error running: docker images --digests --format {{printf "%s\t%s\t%s" .Repository .Tag .Digest}}
          Exit code: 1
          Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

          at cromwell.engine.workflow.WorkflowDockerLookupActor.cromwell$engine$workflow$WorkflowDockerLookupActor$$handleLookupFailure(WorkflowDockerLookupActor.scala:279)
at cromwell.engine.workflow.WorkflowDockerLookupActor$$anonfun$3.applyOrElse(WorkflowDockerLookupActor.scala:93)
at cromwell.engine.workflow.WorkflowDockerLookupActor$$anonfun$3.applyOrElse(WorkflowDockerLookupActor.scala:78)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:35)
at akka.actor.FSM.processEvent(FSM.scala:707)
at akka.actor.FSM.processEvent$(FSM.scala:704)
at cromwell.engine.workflow.WorkflowDockerLookupActor.akka$actor$LoggingFSM$$super$processEvent(WorkflowDockerLookupActor.scala:45)
at akka.actor.LoggingFSM.processEvent(FSM.scala:847)
at akka.actor.LoggingFSM.processEvent$(FSM.scala:829)
at cromwell.engine.workflow.WorkflowDockerLookupActor.processEvent(WorkflowDockerLookupActor.scala:45)
at akka.actor.FSM.akka$actor$FSM$$processMsg(FSM.scala:701)
at akka.actor.FSM$$anonfun$receive$1.applyOrElse(FSM.scala:695)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:35)
at cromwell.docker.DockerClientHelper$$anonfun$dockerResponseReceive$1.applyOrElse(DockerClientHelper.scala:16)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270)
at akka.actor.Actor.aroundReceive(Actor.scala:539)
at akka.actor.Actor.aroundReceive$(Actor.scala:537)
at cromwell.engine.workflow.WorkflowDockerLookupActor.aroundReceive(WorkflowDockerLookupActor.scala:45)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:614)
at akka.actor.ActorCell.invoke(ActorCell.scala:583)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:268)
at akka.dispatch.Mailbox.run(Mailbox.scala:229)
at akka.dispatch.Mailbox.exec(Mailbox.scala:241)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

          [INFO] [12/18/2025 07:26:41.631] [cromwell-system-akka.dispatchers.backend-dispatcher-1467] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-BackendJobExecutionActor-meta_workflow.Identify_analy:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(b3c8ad44)meta_workflow.Identify_analy:NA:1]: echo 'DATABASE' > result.txt
          [INFO] [12/18/2025 07:26:41.632] [cromwell-system-akka.dispatchers.backend-dispatcher-1467] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-BackendJobExecutionActor-meta_workflow.Identify_analy:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(b3c8ad44)meta_workflow.Identify_analy:NA:1]: Calculated TES outputs (found 5):
          Output(Some(rc),Some(meta_workflow.Identify_analy.rc),Some(/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/rc),/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/rc,Some(FILE))
          Output(Some(stdout),Some(meta_workflow.Identify_analy.stdout),Some(/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/stdout),/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/stdout,Some(FILE))
          Output(Some(stderr),Some(meta_workflow.Identify_analy.stderr),Some(/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/stderr),/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/stderr,Some(FILE))
          Output(Some(commandScript),Some(meta_workflow.Identify_analy.commandScript),Some(/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/script),/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/script,Some(FILE))
          Output(Some(meta_workflow.Identify_analy.output.0),Some(meta_workflow.Identify_analy.output.0),Some(/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/result.txt),/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/result.txt,Some(FILE))

          [INFO] [12/18/2025 07:26:41.632] [cromwell-system-akka.dispatchers.backend-dispatcher-1467] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-BackendJobExecutionActor-meta_workflow.Identify_analy:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(b3c8ad44)meta_workflow.Identify_analy:NA:1]: Calculated TES inputs (found 1):
          cromwell.backend.impl.tes.Input(Some(commandScript),Some(meta_workflow.Identify_analy.commandScript),Some(/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/script),/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/script,Some(FILE),None)

          [INFO] [12/18/2025 07:26:41.632] [cromwell-system-akka.dispatchers.backend-dispatcher-1467] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-BackendJobExecutionActor-meta_workflow.Identify_analy:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(b3c8ad44)meta_workflow.Identify_analy:NA:1]: echo 'DATABASE' > result.txt
          [INFO] [12/18/2025 07:26:46.175] [cromwell-system-akka.dispatchers.backend-dispatcher-1467] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-BackendJobExecutionActor-meta_workflow.Identify_analy:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(b3c8ad44)meta_workflow.Identify_analy:NA:1]: job id: task-710bdfb8
          [INFO] [12/18/2025 07:26:47.921] [cromwell-system-akka.dispatchers.backend-dispatcher-1436] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-BackendJobExecutionActor-meta_workflow.Identify_analy:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(b3c8ad44)meta_workflow.Identify_analy:NA:1]: Status change from - to Running
          [INFO] [12/18/2025 07:32:14.919] [cromwell-system-akka.dispatchers.backend-dispatcher-1436] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-BackendJobExecutionActor-meta_workflow.Identify_analy:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(b3c8ad44)meta_workflow.Identify_analy:NA:1]: Job task-710bdfb8 is complete
          [INFO] [12/18/2025 07:32:14.919] [cromwell-system-akka.dispatchers.backend-dispatcher-1436] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-BackendJobExecutionActor-meta_workflow.Identify_analy:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(b3c8ad44)meta_workflow.Identify_analy:NA:1]: Status change from Running to Complete
          [INFO] [12/18/2025 07:32:16.299] [cromwell-system-akka.dispatchers.engine-dispatcher-47] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b] WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b [UUID(b3c8ad44)]: Workflow meta_workflow complete. Final Outputs:
          {
          "meta_workflow.Identify_analy.out": "/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/result.txt"
          }
          [INFO] [12/18/2025 07:32:21.193] [cromwell-system-akka.dispatchers.engine-dispatcher-82] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor] WorkflowManagerActor: Workflow actor for b3c8ad44-dc3b-4e87-9fe9-97c0db47031b completed with status 'Succeeded'. The workflow will be removed from the workflow store.

          1 条回复 最后回复 回复 引用 0
          • Z
            zhanglu 最后由 编辑

            [INFO] [12/18/2025 07:26:16.905] [cromwell-system-akka.dispatchers.api-dispatcher-1503] [akka://cromwell-system/user/cromwell-service/WorkflowStoreActor/WorkflowStoreSubmitActor] Unspecified type (Unspecified version) workflow b3c8ad44-dc3b-4e87-9fe9-97c0db47031b submitted
            [INFO] [12/18/2025 07:26:31.291] [cromwell-system-akka.dispatchers.engine-dispatcher-68] [akka://cromwell-system/user/cromwell-service/WorkflowStoreActor/WorkflowStoreEngineActor] 1 new workflows fetched by cromid-247fb6c: b3c8ad44-dc3b-4e87-9fe9-97c0db47031b
            [INFO] [12/18/2025 07:26:31.291] [cromwell-system-akka.dispatchers.engine-dispatcher-68] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor] WorkflowManagerActor: Starting workflow UUID(b3c8ad44-dc3b-4e87-9fe9-97c0db47031b)
            [INFO] [12/18/2025 07:26:31.291] [cromwell-system-akka.dispatchers.engine-dispatcher-68] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor] WorkflowManagerActor: Successfully started WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b
            [INFO] [12/18/2025 07:26:31.291] [cromwell-system-akka.dispatchers.engine-dispatcher-68] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor] Retrieved 1 workflows from the WorkflowStoreActor
            [INFO] [12/18/2025 07:26:31.293] [cromwell-system-akka.dispatchers.engine-dispatcher-74] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/MaterializeWorkflowDescriptorActor] MaterializeWorkflowDescriptorActor [UUID(b3c8ad44)]: Parsing workflow as WDL draft-2
            [INFO] [12/18/2025 07:26:31.298] [cromwell-system-akka.dispatchers.engine-dispatcher-74] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/MaterializeWorkflowDescriptorActor] MaterializeWorkflowDescriptorActor [UUID(b3c8ad44)]: Call-to-Backend assignments: meta_workflow.Identify_analy -> TESK
            [INFO] [12/18/2025 07:26:33.508] [cromwell-system-akka.dispatchers.engine-dispatcher-74] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b] WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b [UUID(b3c8ad44)]: Starting meta_workflow.Identify_analy
            [INFO] [12/18/2025 07:26:41.267] [cromwell-system-akka.dispatchers.engine-dispatcher-55] [akka://cromwell-system/user/cromwell-service/JobExecutionTokenDispenser] Assigned new job execution tokens to the following groups: b3c8ad44: 1
            [WARN] [12/18/2025 07:26:41.289] [cromwell-system-akka.dispatchers.engine-dispatcher-66] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/BackendPreparationActor_for_b3c8ad44:meta_workflow.Identify_analy👎1] BackendPreparationActor_for_b3c8ad44:meta_workflow.Identify_analy👎1 [UUID(b3c8ad44)]: Docker lookup failed
            java.lang.Exception: Failed to get docker hash for dockerhub.genostack.com:8090/library/ubuntu:18.04 Error running: docker images --digests --format {{printf "%s\t%s\t%s" .Repository .Tag .Digest}}
            Exit code: 1
            Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

            at cromwell.engine.workflow.WorkflowDockerLookupActor.cromwell$engine$workflow$WorkflowDockerLookupActor$$handleLookupFailure(WorkflowDockerLookupActor.scala:279)
at cromwell.engine.workflow.WorkflowDockerLookupActor$$anonfun$3.applyOrElse(WorkflowDockerLookupActor.scala:93)
at cromwell.engine.workflow.WorkflowDockerLookupActor$$anonfun$3.applyOrElse(WorkflowDockerLookupActor.scala:78)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:35)
at akka.actor.FSM.processEvent(FSM.scala:707)
at akka.actor.FSM.processEvent$(FSM.scala:704)
at cromwell.engine.workflow.WorkflowDockerLookupActor.akka$actor$LoggingFSM$$super$processEvent(WorkflowDockerLookupActor.scala:45)
at akka.actor.LoggingFSM.processEvent(FSM.scala:847)
at akka.actor.LoggingFSM.processEvent$(FSM.scala:829)
at cromwell.engine.workflow.WorkflowDockerLookupActor.processEvent(WorkflowDockerLookupActor.scala:45)
at akka.actor.FSM.akka$actor$FSM$$processMsg(FSM.scala:701)
at akka.actor.FSM$$anonfun$receive$1.applyOrElse(FSM.scala:695)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:35)
at cromwell.docker.DockerClientHelper$$anonfun$dockerResponseReceive$1.applyOrElse(DockerClientHelper.scala:16)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270)
at akka.actor.Actor.aroundReceive(Actor.scala:539)
at akka.actor.Actor.aroundReceive$(Actor.scala:537)
at cromwell.engine.workflow.WorkflowDockerLookupActor.aroundReceive(WorkflowDockerLookupActor.scala:45)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:614)
at akka.actor.ActorCell.invoke(ActorCell.scala:583)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:268)
at akka.dispatch.Mailbox.run(Mailbox.scala:229)
at akka.dispatch.Mailbox.exec(Mailbox.scala:241)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

            [INFO] [12/18/2025 07:26:41.631] [cromwell-system-akka.dispatchers.backend-dispatcher-1467] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-BackendJobExecutionActor-meta_workflow.Identify_analy:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(b3c8ad44)meta_workflow.Identify_analy:NA:1]: echo 'DATABASE' > result.txt
            [INFO] [12/18/2025 07:26:41.632] [cromwell-system-akka.dispatchers.backend-dispatcher-1467] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-BackendJobExecutionActor-meta_workflow.Identify_analy:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(b3c8ad44)meta_workflow.Identify_analy:NA:1]: Calculated TES outputs (found 5):
            Output(Some(rc),Some(meta_workflow.Identify_analy.rc),Some(/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/rc),/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/rc,Some(FILE))
            Output(Some(stdout),Some(meta_workflow.Identify_analy.stdout),Some(/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/stdout),/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/stdout,Some(FILE))
            Output(Some(stderr),Some(meta_workflow.Identify_analy.stderr),Some(/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/stderr),/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/stderr,Some(FILE))
            Output(Some(commandScript),Some(meta_workflow.Identify_analy.commandScript),Some(/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/script),/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/script,Some(FILE))
            Output(Some(meta_workflow.Identify_analy.output.0),Some(meta_workflow.Identify_analy.output.0),Some(/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/result.txt),/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/result.txt,Some(FILE))

            [INFO] [12/18/2025 07:26:41.632] [cromwell-system-akka.dispatchers.backend-dispatcher-1467] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-BackendJobExecutionActor-meta_workflow.Identify_analy:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(b3c8ad44)meta_workflow.Identify_analy:NA:1]: Calculated TES inputs (found 1):
            cromwell.backend.impl.tes.Input(Some(commandScript),Some(meta_workflow.Identify_analy.commandScript),Some(/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/script),/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/script,Some(FILE),None)

            [INFO] [12/18/2025 07:26:41.632] [cromwell-system-akka.dispatchers.backend-dispatcher-1467] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-BackendJobExecutionActor-meta_workflow.Identify_analy:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(b3c8ad44)meta_workflow.Identify_analy:NA:1]: echo 'DATABASE' > result.txt
            [INFO] [12/18/2025 07:26:46.175] [cromwell-system-akka.dispatchers.backend-dispatcher-1467] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-BackendJobExecutionActor-meta_workflow.Identify_analy:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(b3c8ad44)meta_workflow.Identify_analy:NA:1]: job id: task-710bdfb8
            [INFO] [12/18/2025 07:26:47.921] [cromwell-system-akka.dispatchers.backend-dispatcher-1436] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-BackendJobExecutionActor-meta_workflow.Identify_analy:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(b3c8ad44)meta_workflow.Identify_analy:NA:1]: Status change from - to Running
            [INFO] [12/18/2025 07:32:14.919] [cromwell-system-akka.dispatchers.backend-dispatcher-1436] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-BackendJobExecutionActor-meta_workflow.Identify_analy:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(b3c8ad44)meta_workflow.Identify_analy:NA:1]: Job task-710bdfb8 is complete
            [INFO] [12/18/2025 07:32:14.919] [cromwell-system-akka.dispatchers.backend-dispatcher-1436] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-EngineJobExecutionActor-meta_workflow.Identify_analy:NA:1/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b-BackendJobExecutionActor-meta_workflow.Identify_analy:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(b3c8ad44)meta_workflow.Identify_analy:NA:1]: Status change from Running to Complete
            [INFO] [12/18/2025 07:32:16.299] [cromwell-system-akka.dispatchers.engine-dispatcher-47] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b] WorkflowExecutionActor-b3c8ad44-dc3b-4e87-9fe9-97c0db47031b [UUID(b3c8ad44)]: Workflow meta_workflow complete. Final Outputs:
            {
            "meta_workflow.Identify_analy.out": "/cephfs_data/genostack_v3_dev/genostack_cromwell/cromwell-executions/meta_workflow/b3c8ad44-dc3b-4e87-9fe9-97c0db47031b/call-Identify_analy/execution/result.txt"
            }
            [INFO] [12/18/2025 07:32:21.193] [cromwell-system-akka.dispatchers.engine-dispatcher-82] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor] WorkflowManagerActor: Workflow actor for b3c8ad44-dc3b-4e87-9fe9-97c0db47031b completed with status 'Succeeded'. The workflow will be removed from the workflow store.

            1 条回复 最后回复 回复 引用 0
            • Z
              zhanglu 最后由 编辑

              [INFO] [12/18/2025 07:55:14.436] [cromwell-system-akka.dispatchers.backend-dispatcher-1296] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-6de70bcd-083a-4fd7-9198-9cbb2005dbe5/WorkflowExecutionActor-6de70bcd-083a-4fd7-9198-9cbb2005dbe5/6de70bcd-083a-4fd7-9198-9cbb2005dbe5-EngineJobExecutionActor-untarget_workflow.chemrich:NA:1/6de70bcd-083a-4fd7-9198-9cbb2005dbe5-BackendJobExecutionActor-untarget_workflow.chemrich:NA:1/TesAsyncBackendJobExecutionActor] TesAsyncBackendJobExecutionActor [UUID(6de70bcd)untarget_workflow.chemrich:NA:1]: Status change from Running to Complete

              1 条回复 最后回复 回复 引用 0
              • Z
                zhanglu 最后由 编辑

                2025-12-19 14:46:41.752 [container-3592] INFO c.a.c.redis.MessagePubListener - 消费者消费: {"workflowId":"6d8348ea-1893-4b86-b9f7-d2616826f82c[INFO] [12/19/2025 06:45:22.157] [cromwell-system-akka.dispatchers.engine-dispatcher-83] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/b16ea7b9-0d2d-4f29-9ac0-cc8a0a6722bf/WorkflowExecutionActor-b16ea7b9-0d2d-4f29-9ac0-cc8a0a6722bf] WorkflowExecutionActor-b16ea7b9-0d2d-4f29-9ac0-cc8a0a6722bf [UUID(b16ea7b9)]: Starting amplicon_meta.W01_cleandata", "taskName":"", "status":"start", "type":"workflow", "time":"2025-12-19 14:46:36"}

                [INFO] [12/19/2025 06:45:25.109] [cromwell-system-akka.dispatchers.engine-dispatcher-97] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor] WorkflowManagerActor: Successfully started WorkflowActor-6d8348ea-1893-4b86-b9f7-d2616826f82c[INFO] [12/19/2025 06:45:22.157] [cromwell-system-akka.dispatchers.engine-dispatcher-83] [akka://cromwell-system/user/cromwell-service/WorkflowManagerActor/WorkflowActor-b16ea7b9-0d2d-4f29-9ac0-cc8a0a6722bf/WorkflowExecutionActor-b16ea7b9-0d2d-4f29-9ac0-cc8a0a6722bf] WorkflowExecutionActor-b16ea7b9-0d2d-4f29-9ac0-cc8a0a6722bf [UUID(b16ea7b9)]: Starting amplicon_meta.W01_cleandata

                1 条回复 最后回复 回复 引用 0
                • Z
                  zhanglu 最后由 编辑

                  5-12-23 06:22:21.512 UTC [26] LOG: database system was interrupted; last known up at 2025-12-23 06:21:21 UTC
                  2025-12-23 06:22:21.950 UTC [26] LOG: database system was not properly shut down; automatic recovery in progress
                  2025-12-23 06:22:21.954 UTC [26] LOG: redo starts at 18/9B999808
                  2025-12-23 06:22:21.955 UTC [26] LOG: invalid record length at 18/9BA06F18: wanted 24, got 0
                  2025-12-23 06:22:21.955 UTC [26] LOG: redo done at 18/9BA06EF0
                  2025-12-23 06:22:21.955 UTC [26] LOG: last completed transaction was at log time 2025-12-23 06:22:20.084128+00
                  2025-12-23 06:22:22.075 UTC [1] LOG: database system is ready to accept connections

                  1 条回复 最后回复 回复 引用 0
                  • Z
                    zhanglu 最后由 编辑

                    cluster:
                    id: 807d820b-5c5b-451c-9f52-41b93d5d905a
                    health: HEALTH_ERR
                    1 MDSs report oversized cache
                    1 MDSs are read only
                    2 MDSs behind on trimming
                    full ratio(s) out of order
                    Low space hindering backfill (add storage if this doesn't resolve itself): 10 pgs backfill_toofull
                    94 pgs not deep-scrubbed in time
                    69 pgs not scrubbed in time

                    services:
                    mon: 3 daemons, quorum bm,bo,bp (age 13d)
                    mgr: b(active, since 2w), standbys: a
                    mds: 1/1 daemons up, 1 hot standby
                    osd: 51 osds: 51 up (since 13d), 51 in (since 2w); 67 remapped pgs

                    data:
                    volumes: 1/1 healthy
                    pools: 4 pools, 2097 pgs
                    objects: 279.98M objects, 306 TiB
                    usage: 619 TiB used, 82 TiB / 701 TiB avail
                    pgs: 3748313/601598194 objects misplaced (0.623%)
                    2024 active+clean
                    57 active+remapped+backfilling
                    10 active+remapped+backfill_toofull
                    6 active+clean+scrubbing+deep

                    io:
                    client: 1.2 MiB/s rd, 100 KiB/s wr, 183 op/s rd, 19 op/s wr
                    recovery: 52 MiB/s, 37 objects/s

                    1 条回复 最后回复 回复 引用 0
                    • Z
                      zhanglu 最后由 编辑

                      cluster:
                      id: 807d820b-5c5b-451c-9f52-41b93d5d905a
                      health: HEALTH_ERR
                      1 MDSs report oversized cache
                      1 MDSs are read only
                      2 MDSs behind on trimming
                      full ratio(s) out of order
                      Low space hindering backfill (add storage if this doesn't resolve itself): 10 pgs backfill_toofull
                      94 pgs not deep-scrubbed in time
                      69 pgs not scrubbed in time

                      services:
                      mon: 3 daemons, quorum bm,bo,bp (age 13d)
                      mgr: b(active, since 2w), standbys: a
                      mds: 1/1 daemons up, 1 hot standby
                      osd: 51 osds: 51 up (since 13d), 51 in (since 2w); 67 remapped pgs

                      data:
                      volumes: 1/1 healthy
                      pools: 4 pools, 2097 pgs
                      objects: 279.98M objects, 306 TiB
                      usage: 619 TiB used, 82 TiB / 701 TiB avail
                      pgs: 3748313/601598194 objects misplaced (0.623%)
                      2024 active+clean
                      57 active+remapped+backfilling
                      10 active+remapped+backfill_toofull
                      6 active+clean+scrubbing+deep

                      io:
                      client: 1.2 MiB/s rd, 100 KiB/s wr, 183 op/s rd, 19 op/s wr
                      recovery: 52 MiB/s, 37 objects/s

                      1 条回复 最后回复 回复 引用 0
                      • Z
                        zhanglu 最后由 编辑 

                        Image:         registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         registry.k8s.io/sig-storage/csi-attacher:v4.0.0
Image:         registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0
Image:         registry.k8s.io/sig-storage/csi-resizer:v1.6.0
Image:         registry.k8s.io/sig-storage/csi-provisioner:v3.3.0
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         registry.k8s.io/sig-storage/csi-attacher:v4.0.0
Image:         registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0
Image:         registry.k8s.io/sig-storage/csi-resizer:v1.6.0
Image:         registry.k8s.io/sig-storage/csi-provisioner:v3.3.0
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         registry.k8s.io/sig-storage/csi-provisioner:v3.3.0
Image:         registry.k8s.io/sig-storage/csi-resizer:v1.6.0
Image:         registry.k8s.io/sig-storage/csi-attacher:v4.0.0
Image:         registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         registry.k8s.io/sig-storage/csi-provisioner:v3.3.0
Image:         registry.k8s.io/sig-storage/csi-resizer:v1.6.0
Image:         registry.k8s.io/sig-storage/csi-attacher:v4.0.0
Image:         registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         rook/ceph:v1.10.8
Image:         quay.io/cephcsi/cephcsi:v3.7.2
Image:         rook/ceph:v1.10.8
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         rook/ceph:v1.10.8
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         rook/ceph:v1.10.8
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         rook/ceph:v1.10.8
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
Image:         rook/ceph:v1.10.8
Image:         quay.io/ceph/ceph:v17.2.5
Image:         quay.io/ceph/ceph:v17.2.5
                        1 条回复 最后回复 回复 引用 0
                        • Z
                          zhanglu 最后由 编辑 

                          Image:          quay.io/calico/kube-controllers:v3.16.5
Image:         quay.io/calico/cni:v3.16.5
Image:         quay.io/calico/cni:v3.16.5
Image:          quay.io/calico/node:v3.16.5
Image:         quay.io/calico/cni:v3.16.5
Image:         quay.io/calico/cni:v3.16.5
Image:          quay.io/calico/node:v3.16.5
Image:         quay.io/calico/cni:v3.16.5
Image:         quay.io/calico/cni:v3.16.5
Image:          quay.io/calico/node:v3.16.5
Image:         quay.io/calico/cni:v3.16.5
Image:         quay.io/calico/cni:v3.16.5
Image:          quay.io/calico/node:v3.16.5
Image:         quay.io/calico/cni:v3.16.5
Image:         quay.io/calico/cni:v3.16.5
Image:          quay.io/calico/node:v3.16.5
Image:         quay.io/calico/cni:v3.16.5
Image:         quay.io/calico/cni:v3.16.5
Image:          quay.io/calico/node:v3.16.5
Image:         quay.io/calico/cni:v3.16.5
Image:         quay.io/calico/cni:v3.16.5
Image:          quay.io/calico/node:v3.16.5
Image:         k8s.gcr.io/coredns:1.7.0
Image:         k8s.gcr.io/coredns:1.7.0
Image:         k8s.gcr.io/coredns:1.7.0
Image:         k8s.gcr.io/coredns:1.7.0
Image:         k8s.gcr.io/coredns:1.7.0
Image:         k8s.gcr.io/coredns:1.7.0
Image:         k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3
Image:         k8s.gcr.io/kube-apiserver:v1.19.7
Image:         k8s.gcr.io/kube-apiserver:v1.19.7
Image:         k8s.gcr.io/kube-controller-manager:v1.19.7
Image:         k8s.gcr.io/kube-controller-manager:v1.19.7
Image:         k8s.gcr.io/kube-proxy:v1.19.7
Image:         k8s.gcr.io/kube-proxy:v1.19.7
Image:         k8s.gcr.io/kube-proxy:v1.19.7
Image:         k8s.gcr.io/kube-proxy:v1.19.7
Image:         k8s.gcr.io/kube-proxy:v1.19.7
Image:         k8s.gcr.io/kube-proxy:v1.19.7
Image:         k8s.gcr.io/kube-proxy:v1.19.7
Image:         k8s.gcr.io/kube-scheduler:v1.19.7
Image:         k8s.gcr.io/kube-scheduler:v1.19.7
Image:          docker.io/library/nginx:1.19
Image:          docker.io/library/nginx:1.19
Image:          docker.io/library/nginx:1.19
Image:          docker.io/library/nginx:1.19
Image:          docker.io/library/nginx:1.19
Image:         k8s.gcr.io/dns/k8s-dns-node-cache:1.16.0
Image:         k8s.gcr.io/dns/k8s-dns-node-cache:1.16.0
Image:         k8s.gcr.io/dns/k8s-dns-node-cache:1.16.0
Image:         k8s.gcr.io/dns/k8s-dns-node-cache:1.16.0
Image:         k8s.gcr.io/dns/k8s-dns-node-cache:1.16.0
Image:         k8s.gcr.io/dns/k8s-dns-node-cache:1.16.0
Image:         k8s.gcr.io/dns/k8s-dns-node-cache:1.16.0
Image:          nvcr.io/nvidia/k8s-device-plugin:v0.17.1
Image:          nvcr.io/nvidia/k8s-device-plugin:v0.17.1
Image:          nvcr.io/nvidia/k8s-device-plugin:v0.17.1
Image:          nvcr.io/nvidia/k8s-device-plugin:v0.17.1
Image:          nvcr.io/nvidia/k8s-device-plugin:v0.17.1
Image:          nvcr.io/nvidia/k8s-device-plugin:v0.17.1
Image:          nvcr.io/nvidia/k8s-device-plugin:v0.17.1
Image:         csiplugin/snapshot-controller:v4.0.0
                          1 条回复 最后回复 回复 引用 0
                          • Z
                            zhanglu 最后由 编辑

                            docker save registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1 -o csi-node-driver-registrar.tar
                            docker save quay.io/cephcsi/cephcsi:v3.7.2 -o cephcsi.tgar
                            docker save registry.k8s.io/sig-storage/csi-attacher:v4.0.0 -o csi-attacher.tar
                            docker save registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0 -o csi-snapshotter.tar
                            docker save registry.k8s.io/sig-storage/csi-resizer:v1.6.0 -o csi-resizer.tar
                            docker save registry.k8s.io/sig-storage/csi-provisioner:v3.3.0 -o csi-provisioner.tar
                            docker save quay.io/ceph/ceph:v17.2.5 -o ceph1.tar
                            docker save rook/ceph:v1.10.8 -o ceph.tar

                            docker save quay.io/calico/kube-controllers:v3.16.5 -o kube-controllers.tar
                            docker save quay.io/calico/cni:v3.16.5 -o cni.tar
                            docker save quay.io/calico/node:v3.16.5 -o node.tar
                            docker save k8s.gcr.io/coredns:1.7.0 -o coredns.tar
                            docker save k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3 -o cluster-proportional-autoscaler-amd64.tar
                            docker save k8s.gcr.io/kube-apiserver:v1.19.7 -o apiserver.tar
                            docker save k8s.gcr.io/kube-controller-manager:v1.19.7 -o manager.tar
                            docker save k8s.gcr.io/kube-proxy:v1.19.7 -o proxy.tar
                            docker save k8s.gcr.io/kube-scheduler:v1.19.7 -o scheduler.tar
                            docker save docker.io/library/nginx:1.19 -o ngins.tar
                            docker save k8s.gcr.io/dns/k8s-dns-node-cache:1.16.0 -o cache.tar
                            docker save nvcr.io/nvidia/k8s-device-plugin:v0.17.1 -o plugin.tar
                            docker save csiplugin/snapshot-controller:v4.0.0 -o controller.tar

                            1 条回复 最后回复 回复 引用 0
                            • Z
                              zhanglu 最后由 编辑

                              docker load < csi-node-driver-registrar.tar
                              docker load < cephcsi.tgar
                              docker load <csi-attacher.tar
                              docker load < csi-snapshotter.tar
                              docker load < csi-resizer.tar
                              docker load < csi-provisioner.tar
                              docker load <-o ceph1.tar
                              docker load < -o ceph.tar

                              docker load < kube-controllers.tar
                              docker load < cni.tar
                              docker load < node.tar
                              docker load < coredns.tar
                              docker load < cluster-proportional-autoscaler-amd64.tar
                              docker load < -o apiserver.tar
                              docker load < manager.tar
                              docker load < proxy.tar
                              docker load < scheduler.tar
                              docker load < ngins.tar
                              docker load < cache.tar
                              docker load < plugin.tar
                              docker load <controller.tar

                              1 条回复 最后回复 回复 引用 0
                              • Z
                                zhanglu 最后由 编辑

                                cluster:
                                id: 807d820b-5c5b-451c-9f52-41b93d5d905a
                                health: HEALTH_ERR
                                full ratio(s) out of order
                                Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull
                                178 pgs not deep-scrubbed in time
                                117 pgs not scrubbed in time

                                services:
                                mon: 3 daemons, quorum bm,bo,bp (age 39h)
                                mgr: a(active, since 39h), standbys: b
                                mds: 1/1 daemons up, 1 hot standby
                                osd: 51 osds: 51 up (since 109m), 51 in (since 109m); 59 remapped pgs

                                data:
                                volumes: 1/1 healthy
                                pools: 4 pools, 2097 pgs
                                objects: 279.97M objects, 306 TiB
                                usage: 617 TiB used, 84 TiB / 701 TiB avail
                                pgs: 5001255/601565087 objects misplaced (0.831%)
                                2028 active+clean
                                54 active+remapped+backfilling
                                10 active+clean+scrubbing+deep
                                5 active+remapped+backfill_toofull

                                io:
                                client: 338 KiB/s rd, 8.8 MiB/s wr, 175 op/s rd, 7 op/s wr
                                recovery: 65 MiB/s, 49 objects/s

                                1 条回复 最后回复 回复 引用 0
                                • Z
                                  zhanglu 最后由 编辑

                                  datasource:
                                  # SQLite 数据库文件：项目根目录自动生成 cromwell_log.db（对应你的数据库名）
                                  url: jdbc:sqlite:/home/genostack/workspace/CromwellStatus/cromwell_log.db
                                  # SQLite 无用户名密码，留空
                                  username:
                                  password:
                                  driver-class-name: org.sqlite.JDBC
                                  initialization-mode: always # 开启自动执行脚本
                                  schema: classpath:schema.sql # 建表脚本路径

                                  1 条回复 最后回复 回复 引用 0
                                  • Z
                                    zhanglu 最后由 编辑

                                    ID HOST USED AVAIL WR OPS WR DATA RD OPS RD DATA STATE
                                    0 node1 8480G 2468G 0 0 0 0 exists,up
                                    1 node1 9556G 1392G 20 3438k 18 4128k exists,up
                                    2 node1 8933G 2015G 36 1188k 8 1328k exists,up
                                    3 node1 8795G 2153G 0 0 0 0 exists,up
                                    4 node1 9250G 1698G 0 0 0 0 exists,up
                                    5 node1 8957G 1991G 0 0 0 0 exists,up
                                    6 node1 8772G 2176G 0 2563 0 0 exists,up
                                    7 node2 9247G 1701G 1 7584 0 0 exists,up
                                    8 node2 9939G 1009G 0 0 7 20.2k exists,up
                                    9 node2 9352G 1596G 0 0 0 0 exists,up
                                    10 node2 9.87T 838G 0 2563 0 0 exists,up
                                    11 node2 9656G 1292G 0 0 0 0 exists,up
                                    12 node2 9678G 1270G 0 0 0 0 exists,up
                                    13 node2 9558G 1390G 0 2563 0 0 exists,up
                                    14 node3 9945G 1003G 0 0 0 0 exists,up
                                    15 node3 9762G 1186G 0 0 0 0 exists,up
                                    16 node3 9231G 1717G 0 0 0 0 exists,up
                                    17 node3 9239G 1709G 0 2902 0 0 exists,up
                                    18 node3 9668G 1280G 0 2563 0 0 exists,up
                                    19 node3 9551G 1397G 0 721k 0 0 exists,up
                                    20 node3 9859G 1089G 0 0 0 0 exists,up
                                    21 node8 16.1T 2062G 18 1638k 7 831k exists,up
                                    22 node7 13.1T 1450G 2 1638k 9 844k exists,up
                                    23 node7 12.8T 1780G 0 0 0 0 exists,up
                                    24 node7 13.2T 1283G 0 89.7k 0 0 exists,up
                                    25 node7 12.8T 1731G 0 2563 0 0 exists,up
                                    26 node7 13.4T 1119G 0 0 0 0 exists,up
                                    27 node5 12.5T 2039G 0 2563 0 0 exists,up
                                    28 node5 12.4T 2160G 0 0 0 0 exists,up
                                    29 node5 12.9T 1658G 4 27.4k 0 0 exists,up
                                    30 node5 12.7T 1887G 34 6468k 21 6494k exists,up
                                    31 node5 12.8T 1781G 0 0 0 0 exists,up
                                    32 node5 12.2T 2337G 0 0 0 0 exists,up
                                    33 node5 12.5T 2022G 3 819k 9 842k exists,up
                                    34 node6 13.1T 1443G 0 12.1k 0 0 exists,up
                                    35 node6 12.6T 1935G 0 0 0 0 exists,up
                                    36 node6 13.0T 1527G 1 592k 0 0 exists,up
                                    37 node6 12.5T 2095G 0 195 0 0 exists,up
                                    38 node6 12.8T 1731G 0 0 0 0 exists,up
                                    39 node6 13.4T 1174G 0 2563 0 0 exists,up
                                    40 node6 13.0T 1557G 15 2928k 19 3797k exists,up
                                    41 node8 16.4T 1814G 1 2563 7 832k exists,up
                                    42 node8 15.6T 2574G 0 819 0 0 exists,up
                                    43 node8 15.8T 2346G 0 0 0 0 exists,up
                                    44 node8 16.2T 1979G 76 1524k 9 852k exists,up
                                    45 node8 15.8T 2403G 0 0 0 0 exists,up
                                    46 node8 15.8T 2405G 8 2515k 10 2478k exists,up
                                    47 node8 16.1T 2057G 1 8408 0 0 exists,up
                                    48 node8 16.7T 1448G 0 0 0 0 exists,up
                                    49 node8 15.3T 2888G 0 0 0 0 exists,up
                                    50 node8 16.1T 2045G 0 0 0 0 exists,up

                                    1 条回复 最后回复 回复 引用 0
                                    • Z
                                      zhanglu 最后由 编辑

                                      [root@node1 prod]# kubectl get pod -n genostack-base-dev
                                      NAME READY STATUS RESTARTS AGE
                                      postgres-7cffc9f66-6ngfr 0/1 ContainerCreating 0 17m
                                      postgres-7cffc9f66-fqmzm 0/1 Terminating 0 27m
                                      redis-bd7c6cbb4-gtdz9 1/1 Running 0 30d

                                      1 条回复 最后回复 回复 引用 0
                                      • Z
                                        zhanglu 最后由 zhanglu 编辑

                                        Resource Requests Limits

                                        cpu 46867m (41%) 58400m (52%)
                                        memory 242859107584 (28%) 315783595264 (37%)
                                        ephemeral-storage 0 (0%) 0 (0%)
                                        hugepages-1Gi 0 (0%) 0 (0%)
                                        hugepages-2Mi 0 (0%) 0 (0%)
                                        Events:
                                        Type Reason Age From Message

                                        Normal NodeReady 8m (x252 over 16h) kubelet Node node1 status is now: NodeReady
                                        Normal NodeNotReady 59s (x254 over 16h) kubelet Node node1 status is now: NodeNotReady

                                        1 条回复 最后回复 回复 引用 0
                                        • Z
                                          zhanglu 最后由 编辑

                                          ps -eo pid,stat,cmd | grep -E '^.+ D'

                                          1 条回复 最后回复 回复 引用 0
                                          • Z
                                            zhanglu 最后由 编辑

                                            [root@node1 account]# ps -eo pid,stat,cmd | grep -E '^.+ D'
                                            382513 Ds /opt/gitlab/embedded/bin/postgres -D /var/opt/gitlab/postgresql/data
                                            382518 Ds postgres: autovacuum launcher process
                                            382519 Ds postgres: stats collector process
                                            385554 D svlogd -tt /var/log/gitlab/redis
                                            385557 D svlogd -tt /var/log/gitlab/sidekiq
                                            385704 D svlogd -tt /var/log/gitlab/node-exporter
                                            385705 D svlogd -tt /var/log/gitlab/gitlab-monitor
                                            385707 D svlogd -tt /var/log/gitlab/prometheus
                                            385722 Ds postgres: gitlab-psql postgres [local] SELECT
                                            385926 D tail --follow=name --retry /var/log/gitlab/nginx/error.log /var/log/gitlab/nginx/current /var/log/gitlab/nginx/gitlab_error.log /var/log/gitlab/nginx/access.log /var/log/gitlab/nginx/gitlab_access.log /var/log/gitlab/prometheus/current /var/log/gitlab/prometheus/state /var/log/gitlab/postgresql/current /var/log/gitlab/postgresql/state /var/log/gitlab/redis/current /var/log/gitlab/redis/state /var/log/gitlab/alertmanager/current /var/log/gitlab/alertmanager/state /var/log/gitlab/redis-exporter/current /var/log/gitlab/redis-exporter/state /var/log/gitlab/gitlab-shell/gitlab-shell.log /var/log/gitlab/sidekiq/current /var/log/gitlab/sidekiq/state /var/log/gitlab/gitlab-monitor/current /var/log/gitlab/gitlab-monitor/state /var/log/gitlab/sshd/current /var/log/gitlab/gitlab-workhorse/current /var/log/gitlab/gitlab-workhorse/state /var/log/gitlab/unicorn/unicorn_stderr.log /var/log/gitlab/unicorn/current /var/log/gitlab/unicorn/unicorn_stdout.log /var/log/gitlab/unicorn/state /var/log/gitlab/gitaly/current /var/log/gitlab/gitaly/state /var/log/gitlab/postgres-exporter/current /var/log/gitlab/postgres-exporter/state /var/log/gitlab/node-exporter/current /var/log/gitlab/node-exporter/state /var/log/gitlab/gitlab-rails/production.log /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2023-02-21-09-56-26.log /var/log/gitlab/gitlab-rails/application.log /var/log/gitlab/gitlab-rails/grpc.log /var/log/gitlab/gitlab-rails/production_json.log /var/log/gitlab/gitlab-rails/api_json.log /var/log/gitlab/logrotate/current /var/log/gitlab/logrotate/state

                                            1 条回复 最后回复 回复 引用 0
                                            • First post
                                              Last post
                                            Powered by 暗能星系