The entrypoint is always flow_script:flow_function_name

wait -- how does the deployment call 'with tags(...)' to assign a runtime tag? the entrypoint is only starting w the call to flow

parameters can be any JSON-serializable data objects, we see most users use it for strings, int, floats, dict, lists etc

the idea is a flow takes a path to sql script as parameter, along with a date and maybe others

sounds like a perfect use case for parameters

So after reading the deployments: parameters section of the docs carefully, it looks like the entrypoint parameter cannot be an object passed from another process, but only as key-value pairs. That's reasonable. I can call the entrypoint with a string parameter variable, and initialize the object as part of a task. To solve the dynamic tagging, I think I can just use a nested flow:

@flow(name="extract write")
def extract_write(**kwargs):
    extract_config = ExtractConfig(filepath=kwargs['filename'])    

    with tags(dataset_name, "extract"):
        inner_extract_write(extract_config)

So searching the UI for flow name isn't helpful in my case. Tagging the flow with the name of the dataset makes the UI more convenient.

fully understand the problem, and I'm 100% with you on that, we would like to allow more ways to attach metadata to flow runs and deployments but for now deployment object is the best place to do that

I will keep this thread in mind when discussing this problem with the product team, if you want to, feel free to share more context of where/how you would like to organize your flow runs for better search in the UI -- this will be helpful to figure out how best we can support it this tagging functionality is still quite early and I expect it to be way nicer in the near future, your input to improve it will be valuable, thanks in advance

ohhh now I see -- deployments are not going to receive parameters from some other active process. The deployment is setting the job 'in stone' as it were. • I have a single flow • every SQL job gets its own deployment, params (

src/script1.sql

, etc) Each deployment has fixed parameters, and the tags would be set at the deployment build stage. You can change the parameters for an ad-hoc run to override the defaults, in my design this wouldn't be needed. Another way, I could have a control flow,

daily_jobs

just loop through the all the sql files and call the

extract_write

flow on each of them. In this case my

with tags...

thing would help again. I like prefect because the flexibility of how to design a system. Once I learn all of this limited set of constraints I will be able to just think about what design approach to take.

Just following up with how it worked out for me with tagging: • Using

with tags..

to tag the flow run according to the name of the dataset being processed. • entrypoint takes a string, the file path of a folder full of SQL files. • a for loop iterates through each file and sends for execution on cloud DB • this 'extract' flow is deployed for folders with daily/weekly jobs here's the entrypoint flow code:

# bulk_extract.py

@flow(name="bulk-extracts")
def bulk_extracts(directory_name):
    logger = get_run_logger()
    extract_folder_items=Path(directory_name).iterdir()

    for item in extract_folder_items:
        <http://logger.info|logger.info>(f"daily file: {item}")
        extract_config = ExtractConfig(filepath=item)
        with tags(extract_config.dataset_name, "extract"):
            extract_write(extract_config)

if __name__ == "__main__":
    bulk_extracts(Path(sys.argv[1]))

The tags are really useful in the UI, I can gather up jobs by dataset_name, distinguish between DDL/extract jobs, etc.