Hello everyone, I just have a quick question (hop...
# prefect-community
r
Hello everyone, I just have a quick question (hopefully). But I just took some time to finish reading the prefect core documentation from beginning to end, and there are still a few concepts I'm struggling to understand. The first thing I don't quite get is how
set_upstream
and
set_downstream
work? Here is an ETL flow I created:
Copy code
with Flow("Thinknum ETL") as flow:

    token = token_task(
        version=version,
        client_id=client_id,
        client_secret=client_secret
    )

    history = history_task(
        dataset_id=dataset_id,
        version=version,
        token=token
    )

    loaded_dates = loaded_dates_task(
        bucket_name=bucket_name,
        dataset_id=dataset_id
    )

    dates_to_load = get_dates_to_load(
        already_loaded_dates=loaded_dates,
        history=history
    )

    datasets_to_load = load_to_gcs_task.map(
        dataset_id=unmapped(dataset_id),
        date=dates_to_load,
        version=unmapped(version),
        token=unmapped(token),
        bucket_name=unmapped(bucket_name)
    )
It DOES seem to work fine, but I don't know when or if I should be applying `set_upstream`/`set_downstream`
i
@Riley Hun If your Flow has tasks that are dependent on one another (like yours does) -- Than you don't have to define an edge ( `set_upstream`/`set_downstream` ) between two tasks. If your task does not have an upstream dependency and it needs to run after a particular task do you set the edge explicitly.
upvote 1
r
Got it! Makes sense now. Thanks Itay - appreciate it!