Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem on retryStrategy workflow when it used git artifact #9257

Open
3 tasks done
DiegoFlop opened this issue Jul 29, 2022 · 8 comments · May be fixed by #13908
Open
3 tasks done

Problem on retryStrategy workflow when it used git artifact #9257

DiegoFlop opened this issue Jul 29, 2022 · 8 comments · May be fixed by #13908
Labels
area/artifacts S3/GCP/OSS/Git/HDFS etc type/bug

Comments

@DiegoFlop
Copy link

DiegoFlop commented Jul 29, 2022 •
edited
Loading

Checklist

  • Double-checked my configuration.
  • Tested using the latest version.
  • Used the Emissary executor.

Summary

What happened/what you expected to happen?

I'm using field retryStrategy=3 at workflow template configuration

When the workflow fail on a step, this, retry the same step 3 times, if none of them finishes correctly, the workflow retry all steps since start.

This is fine, however, on the first step I use the git artifact but when retry the workflow since start. The second time workflow fail because: the files exists from first step, obviously.

Error (exit code 1): rename /mainctrfs/workspace/project.tmp /mainctrfs/workspace/project: file exists

Captura de pantalla 2022-08-01 a las 9 34 09

I would like that You add additional field for example "delete" to remove pre existing content

What version are you running?
Argo workflows: 3.3.8
Kubernetes: 1.22.8

Diagnostics

Paste the smallest workflow that reproduces the bug. We must be able to run the workflow.

apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
  name: step1

spec:
  serviceAccountName: operate-workflow-sa
  volumeClaimTemplates:
  - metadata:
      name: workdir
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 5Gi
  entrypoint: main
  templates:
    - name: main
      dag:
        tasks:
        - name: checkout
          templateRef:
            name: checkout
            template: main
          arguments:
            parameters:
              - name: git_ssh_url
                value: "git@{{= sprig.replace('net/', 'net:', sprig.substr( 8, -1, jsonpath(workflow.parameters.EVENT_DATA, '$.project.git_http_url'))) }}"
              - name: BRANCH
                value: "{{= jsonpath(workflow.parameters.EVENT_DATA, '$.object_attributes.source_branch')}}"
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: checkout
spec:
  entrypoint: main
  templates:
    - name: main
      activeDeadlineSeconds: "120"
      volumes:
        - name: config-volume
          configMap: 
            name: argo-workflows
      inputs:
        parameters:
          - name: git_ssh_url
          - name: BRANCH
        artifacts:
          - name: git-clone
            path: "/workspace/project"
            git:
              revision: "{{inputs.parameters.BRANCH}}"
              repo: "{{inputs.parameters.git_ssh_url}}"
              singleBranch: false
              sshPrivateKeySecret:
                name: github
                key: id_rsa
              insecureIgnoreHostKey: true
            optional: true
      container:
        imagePullPolicy: Always
        volumeMounts:
          - name: workdir
            mountPath: "/workspace"
          - name: config-volume
            mountPath: /utils
        image: XXXXXXX:1.0.0
        workingDir: "/workspace/project"
        command:
          - "python"
        args:
          - "xxxxx"
          - "xxxxx"
# Logs from the workflow controller:
kubectl logs -n argo deploy/workflow-controller | grep ${workflow} 

time="2022-07-29T12:45:37.398Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:45:37.437Z" level=info msg="Updated phase  -> Running" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:45:37.438Z" level=info msg="Creating pvc workflow-problem-hm68w-workdir" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:45:37.474Z" level=info msg="Retry node workflow-problem-hm68w initialized Running" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:45:37.474Z" level=info msg="DAG node workflow-problem-hm68w-2520437565 initialized Running" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:45:37.474Z" level=info msg="All of node workflow-problem-hm68w(0).checkout dependencies [] completed" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:45:37.475Z" level=info msg="Retry node workflow-problem-hm68w-4027359757 initialized Running" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:45:37.476Z" level=info msg="Pod node workflow-problem-hm68w-3269103708 initialized Pending" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:45:37.502Z" level=info msg="Created pod: workflow-problem-hm68w(0).checkout(0) (workflow-problem-hm68w-3269103708)" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:45:37.502Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:45:37.502Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:45:37.537Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83666971 workflow=workflow-problem-hm68w
time="2022-07-29T12:45:47.399Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:45:47.400Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=0 workflow=workflow-problem-hm68w
time="2022-07-29T12:45:47.401Z" level=info msg="node changed" new.message=PodInitializing new.phase=Pending new.progress=0/1 nodeID=workflow-problem-hm68w-3269103708 old.message= old.phase=Pending old.progress=0/1
time="2022-07-29T12:45:47.404Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:45:47.406Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:45:47.462Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667061 workflow=workflow-problem-hm68w
time="2022-07-29T12:45:57.443Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:45:57.445Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=1 workflow=workflow-problem-hm68w
time="2022-07-29T12:45:57.446Z" level=info msg="task-result changed" namespace=argo-events nodeID=workflow-problem-hm68w-3269103708 workflow=workflow-problem-hm68w
time="2022-07-29T12:45:57.446Z" level=info msg="node changed" new.message= new.phase=Running new.progress=0/1 nodeID=workflow-problem-hm68w-3269103708 old.message=PodInitializing old.phase=Pending old.progress=0/1
time="2022-07-29T12:45:57.449Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:45:57.449Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:45:57.472Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667135 workflow=workflow-problem-hm68w
time="2022-07-29T12:46:07.645Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:07.647Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=1 workflow=workflow-problem-hm68w
time="2022-07-29T12:46:07.647Z" level=info msg="node changed" new.message= new.phase=Succeeded new.progress=0/1 nodeID=workflow-problem-hm68w-3269103708 old.message= old.phase=Running old.progress=0/1
time="2022-07-29T12:46:07.648Z" level=info msg="node workflow-problem-hm68w-4027359757 phase Running -> Succeeded" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:07.649Z" level=info msg="node workflow-problem-hm68w-4027359757 finished: 2022-07-29 12:46:07.649002176 +0000 UTC" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:07.649Z" level=info msg="All of node workflow-problem-hm68w(0).tag dependencies [checkout] completed" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:07.649Z" level=info msg="Retry node workflow-problem-hm68w-3152904399 initialized Running" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:07.650Z" level=info msg="Pod node workflow-problem-hm68w-2551760466 initialized Pending" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:07.689Z" level=info msg="Created pod: workflow-problem-hm68w(0).tag(0) (workflow-problem-hm68w-2551760466)" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:07.689Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:07.689Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:07.724Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667212 workflow=workflow-problem-hm68w
time="2022-07-29T12:46:12.727Z" level=info msg="cleaning up pod" action=deletePod key=argo-events/workflow-problem-hm68w-3269103708/deletePod
time="2022-07-29T12:46:17.694Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:17.695Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=1 workflow=workflow-problem-hm68w
time="2022-07-29T12:46:17.696Z" level=info msg="node changed" new.message=PodInitializing new.phase=Pending new.progress=0/1 nodeID=workflow-problem-hm68w-2551760466 old.message= old.phase=Pending old.progress=0/1
time="2022-07-29T12:46:17.698Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:17.698Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:17.741Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667299 workflow=workflow-problem-hm68w
time="2022-07-29T12:46:27.735Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:27.738Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=2 workflow=workflow-problem-hm68w
time="2022-07-29T12:46:27.738Z" level=info msg="task-result changed" namespace=argo-events nodeID=workflow-problem-hm68w-2551760466 workflow=workflow-problem-hm68w
time="2022-07-29T12:46:27.741Z" level=info msg="node changed" new.message="Error (exit code 1)" new.phase=Failed new.progress=0/1 nodeID=workflow-problem-hm68w-2551760466 old.message=PodInitializing old.phase=Pending old.progress=0/1
time="2022-07-29T12:46:27.747Z" level=info msg="1 child nodes of workflow-problem-hm68w(0).tag failed. Trying again..." namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:27.753Z" level=info msg="Pod node workflow-problem-hm68w-873851471 initialized Pending" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:27.775Z" level=info msg="Created pod: workflow-problem-hm68w(0).tag(1) (workflow-problem-hm68w-873851471)" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:27.775Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:27.775Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:27.820Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667375 workflow=workflow-problem-hm68w
time="2022-07-29T12:46:27.830Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo-events/workflow-problem-hm68w-2551760466/labelPodCompleted
time="2022-07-29T12:46:37.785Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:37.786Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=2 workflow=workflow-problem-hm68w
time="2022-07-29T12:46:37.786Z" level=info msg="node changed" new.message=PodInitializing new.phase=Pending new.progress=0/1 nodeID=workflow-problem-hm68w-873851471 old.message= old.phase=Pending old.progress=0/1
time="2022-07-29T12:46:37.790Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:37.790Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:37.841Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667451 workflow=workflow-problem-hm68w
time="2022-07-29T12:46:47.845Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:47.846Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=3 workflow=workflow-problem-hm68w
time="2022-07-29T12:46:47.846Z" level=info msg="task-result changed" namespace=argo-events nodeID=workflow-problem-hm68w-873851471 workflow=workflow-problem-hm68w
time="2022-07-29T12:46:47.846Z" level=info msg="node changed" new.message="Error (exit code 1)" new.phase=Failed new.progress=0/1 nodeID=workflow-problem-hm68w-873851471 old.message=PodInitializing old.phase=Pending old.progress=0/1
time="2022-07-29T12:46:47.849Z" level=info msg="2 child nodes of workflow-problem-hm68w(0).tag failed. Trying again..." namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:47.850Z" level=info msg="Pod node workflow-problem-hm68w-3021827988 initialized Pending" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:47.890Z" level=info msg="Created pod: workflow-problem-hm68w(0).tag(2) (workflow-problem-hm68w-3021827988)" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:47.890Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:47.891Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:47.949Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667531 workflow=workflow-problem-hm68w
time="2022-07-29T12:46:47.958Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo-events/workflow-problem-hm68w-873851471/labelPodCompleted
time="2022-07-29T12:46:57.889Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:57.890Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=3 workflow=workflow-problem-hm68w
time="2022-07-29T12:46:57.890Z" level=info msg="node changed" new.message= new.phase=Running new.progress=0/1 nodeID=workflow-problem-hm68w-3021827988 old.message= old.phase=Pending old.progress=0/1
time="2022-07-29T12:46:57.892Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:57.892Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:46:57.934Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667604 workflow=workflow-problem-hm68w
time="2022-07-29T12:47:07.931Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:07.932Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=4 workflow=workflow-problem-hm68w
time="2022-07-29T12:47:07.932Z" level=info msg="task-result changed" namespace=argo-events nodeID=workflow-problem-hm68w-3021827988 workflow=workflow-problem-hm68w
time="2022-07-29T12:47:07.932Z" level=info msg="node changed" new.message="Error (exit code 1)" new.phase=Failed new.progress=0/1 nodeID=workflow-problem-hm68w-3021827988 old.message= old.phase=Running old.progress=0/1
time="2022-07-29T12:47:07.937Z" level=info msg="3 child nodes of workflow-problem-hm68w(0).tag failed. Trying again..." namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:07.938Z" level=info msg="Pod node workflow-problem-hm68w-2417686609 initialized Pending" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:08.003Z" level=info msg="Created pod: workflow-problem-hm68w(0).tag(3) (workflow-problem-hm68w-2417686609)" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:08.003Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:08.003Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:08.070Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667681 workflow=workflow-problem-hm68w
time="2022-07-29T12:47:08.079Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo-events/workflow-problem-hm68w-3021827988/labelPodCompleted
time="2022-07-29T12:47:17.992Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:17.993Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=4 workflow=workflow-problem-hm68w
time="2022-07-29T12:47:17.994Z" level=info msg="node changed" new.message= new.phase=Running new.progress=0/1 nodeID=workflow-problem-hm68w-2417686609 old.message= old.phase=Pending old.progress=0/1
time="2022-07-29T12:47:17.996Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:17.996Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:18.045Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667766 workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.033Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.034Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=5 workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.034Z" level=info msg="task-result changed" namespace=argo-events nodeID=workflow-problem-hm68w-2417686609 workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.034Z" level=info msg="node changed" new.message="Error (exit code 1)" new.phase=Failed new.progress=0/1 nodeID=workflow-problem-hm68w-2417686609 old.message= old.phase=Running old.progress=0/1
time="2022-07-29T12:47:28.042Z" level=info msg="No more retries left. Failing..." namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.042Z" level=info msg="node workflow-problem-hm68w-3152904399 phase Running -> Failed" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.042Z" level=info msg="node workflow-problem-hm68w-3152904399 message: No more retries left" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.042Z" level=info msg="node workflow-problem-hm68w-3152904399 finished: 2022-07-29 12:47:28.042507727 +0000 UTC" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.042Z" level=info msg="Outbound nodes of workflow-problem-hm68w-2520437565 set to [workflow-problem-hm68w-2417686609]" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.042Z" level=info msg="node workflow-problem-hm68w-2520437565 phase Running -> Failed" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.042Z" level=info msg="node workflow-problem-hm68w-2520437565 finished: 2022-07-29 12:47:28.04273348 +0000 UTC" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.042Z" level=info msg="Checking daemoned children of workflow-problem-hm68w-2520437565" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.044Z" level=info msg="1 child nodes of workflow-problem-hm68w failed. Trying again..." namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.044Z" level=info msg="DAG node workflow-problem-hm68w-2856137040 initialized Running" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.044Z" level=info msg="All of node workflow-problem-hm68w(1).checkout dependencies [] completed" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.046Z" level=info msg="Retry node workflow-problem-hm68w-372661854 initialized Running" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.047Z" level=info msg="Pod node workflow-problem-hm68w-24444429 initialized Pending" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.100Z" level=info msg="Created pod: workflow-problem-hm68w(1).checkout(0) (workflow-problem-hm68w-24444429)" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.100Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.100Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.154Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667844 workflow=workflow-problem-hm68w
time="2022-07-29T12:47:28.170Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo-events/workflow-problem-hm68w-2417686609/labelPodCompleted
time="2022-07-29T12:47:38.101Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:38.101Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=5 workflow=workflow-problem-hm68w
time="2022-07-29T12:47:38.102Z" level=info msg="node changed" new.message=PodInitializing new.phase=Pending new.progress=0/1 nodeID=workflow-problem-hm68w-24444429 old.message= old.phase=Pending old.progress=0/1
time="2022-07-29T12:47:38.104Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:38.104Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:38.149Z" level=info msg="Workflow update successful" namespace=argo-events phase=Running resourceVersion=83667921 workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.070Z" level=info msg="Processing workflow" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.072Z" level=info msg="Task-result reconciliation" namespace=argo-events numObjs=5 workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.072Z" level=info msg="node changed" new.message="Error (exit code 1): rename /mainctrfs/workspace/project.tmp /mainctrfs/workspace/project: file exists" new.phase=Error new.progress=0/1 nodeID=workflow-problem-hm68w-24444429 old.message=PodInitializing old.phase=Pending old.progress=0/1
time="2022-07-29T12:47:49.075Z" level=info msg="Node not set to be retried after status: Error" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.075Z" level=info msg="node workflow-problem-hm68w-372661854 phase Running -> Error" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.076Z" level=info msg="node workflow-problem-hm68w-372661854 message: Error (exit code 1): rename /mainctrfs/workspace/project.tmp /mainctrfs/workspace/project: file exists" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.076Z" level=info msg="node workflow-problem-hm68w-372661854 finished: 2022-07-29 12:47:49.076487529 +0000 UTC" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.076Z" level=info msg="Skipped node workflow-problem-hm68w-961798914 initialized Omitted (message: omitted: depends condition not met)" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.076Z" level=info msg="Outbound nodes of workflow-problem-hm68w-2856137040 set to [workflow-problem-hm68w-961798914]" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.077Z" level=info msg="node workflow-problem-hm68w-2856137040 phase Running -> Error" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.077Z" level=info msg="node workflow-problem-hm68w-2856137040 finished: 2022-07-29 12:47:49.077086018 +0000 UTC" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.077Z" level=info msg="Checking daemoned children of workflow-problem-hm68w-2856137040" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.078Z" level=info msg="Node not set to be retried after status: Error" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.078Z" level=info msg="node workflow-problem-hm68w phase Running -> Error" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.078Z" level=info msg="node workflow-problem-hm68w finished: 2022-07-29 12:47:49.078742993 +0000 UTC" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.078Z" level=info msg="TaskSet Reconciliation" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.079Z" level=info msg=reconcileAgentPod namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.079Z" level=info msg="Updated phase Running -> Error" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.079Z" level=info msg="Marking workflow completed" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.079Z" level=info msg="Marking workflow as pending archiving" namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.079Z" level=info msg="Checking daemoned children of " namespace=argo-events workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.085Z" level=info msg="cleaning up pod" action=deletePod key=argo-events/workflow-problem-hm68w-1340600742-agent/deletePod
time="2022-07-29T12:47:49.138Z" level=info msg="Workflow update successful" namespace=argo-events phase=Error resourceVersion=83668007 workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.372Z" level=info msg="archiving workflow" namespace=argo-events uid=e9a595ab-2412-4677-a413-507227dfb8e2 workflow=workflow-problem-hm68w
time="2022-07-29T12:47:49.377Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo-events/workflow-problem-hm68w-24444429/labelPodCompleted
time="2022-07-29T12:47:49.497Z" level=info msg="Queueing Error workflow argo-events/workflow-problem-hm68w for delete in 2m0s due to TTL"
time="2022-07-29T12:49:50.000Z" level=info msg="Deleting garbage collected workflow 'argo-events/workflow-problem-hm68w'"
time="2022-07-29T12:49:50.028Z" level=info msg="Successfully deleted 'argo-events/workflow-problem-hm68w'"

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.
@DiegoFlop DiegoFlop changed the title Problem when I retry workflow when it used git artifact Problem on retryStrategy workflow when it used git artifact Jul 31, 2022
@alexec alexec added the area/artifacts S3/GCP/OSS/Git/HDFS etc label Sep 5, 2022
@stale
Copy link

stale bot commented Oct 1, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is a mentoring request, please provide an update here. Thank you for your contributions.

@stale stale bot added the problem/stale This has not had a response in some time label Oct 1, 2022
@rgeraskin
Copy link

confirm, this bug still exists

@stale stale bot removed the problem/stale This has not had a response in some time label Oct 3, 2022
@stale
Copy link

stale bot commented Oct 22, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is a mentoring request, please provide an update here. Thank you for your contributions.

@stale stale bot added the problem/stale This has not had a response in some time label Oct 22, 2022
@rgeraskin
Copy link

confirm, this bug still exists

@stale stale bot removed the problem/stale This has not had a response in some time label Oct 23, 2022
@sarabala1979
Copy link
Member

@rgeraskin can you add an exit handler on the step?

@rgeraskin
Copy link

@sarabala1979 sorry I can't figure it out how it will help me? Maybe you are talking about deleting the content manually by handler?

@stale
Copy link

stale bot commented Nov 12, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is a mentoring request, please provide an update here. Thank you for your contributions.

@stale stale bot added the problem/stale This has not had a response in some time label Nov 12, 2022
@terrytangyuan terrytangyuan removed the problem/stale This has not had a response in some time label Sep 20, 2023
@tooptoop4
Copy link
Contributor

tooptoop4 commented Nov 10, 2024 •
edited
Loading

rename command in

argo-workflows/workflow/executor/executor.go

Line 249 in f470fda

err = os.Rename(tempArtPath, artPath)
to change

@tooptoop4 tooptoop4 linked a pull request Nov 15, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/artifacts S3/GCP/OSS/Git/HDFS etc type/bug
Projects
None yet
Milestone
No milestone
6 participants
@alexec @terrytangyuan @tooptoop4 @sarabala1979 @DiegoFlop @rgeraskin