Kubeflow vs Argo vs Airflow: 4 Graph Execution Gotchas

Kubeflow's DAG Isn't Always a DAG

Most tutorials show you the happy path: define dependencies, watch the pipeline execute in order. But when you start adding conditional branches and loops to Kubeflow Pipelines, you'll hit a wall the docs barely mention—cycles in your execution graph that don't throw errors until runtime.

The core issue? Kubeflow's conditional execution (dsl.Condition) and looping (dsl.ParallelFor) create dynamic subgraphs at runtime. If you're not careful with how you wire dependencies, you can create cycles that look fine in the static compilation step but fail once Argo (the underlying executor) tries to resolve the actual execution order. I spent two days debugging a pipeline that compiled successfully but hung forever on a Pending status because a downstream task inadvertently depended on its own loop output through a condition check.

Here's a simplified version of what broke:

python
import kfp
from kfp import dsl

@dsl.component
def preprocess_op(data: str) -> str:
    return f"processed_{data}"

@dsl.component
def check_quality_op(data: str) -> str:
    # Returns 'pass' or 'fail'
    return 'pass' if len(data) > 10 else 'fail'

@dsl.component
def retrain_op(data: str) -> str:
    return f"retrained_{data}"

@dsl.pipeline(name='conditional-loop-pipeline')
def my_pipeline(input_data: str):
    preprocess_task = preprocess_op(data=input_data)

    with dsl.ParallelFor(items=['batch1', 'batch2', 'batch3']) as batch:

---

*Continue reading the full article on [TildAlice](https://tildalice.io/kubeflow-argo-airflow-graph-execution-gotchas/)*