Apologies if this has already been answered, but the docs don't make it super clear what terminology I should use so it's difficult to search for an answer. Basically, I want to know if we can have a flow where it waits for one task to complete, basically because some data isn't yet available from a provider, so it checks that for example every 15 mins and continues the flow when it's available? And importantly, is this available in the open-source version? This is the crucial reason we'd use Prefect instead of exclusively using something like

dbt

, which I think we'll use anyway.

Hi David - definitely. Here’s what I’d suggest:

from datetime import timedelta

@task(max_retries=500, retry_delay=timedelta(minutes=15))
def check_data_available():
    # hit the provider to determine whether
    # data is available
    if data_available:
        return None # can optionally return some info
    else:
        raise RuntimeError("Data not yet available.")

Thanks so much Chris!