Pooyan
12/14/2023, 9:08 PMMarvin
12/14/2023, 9:09 PMNate
12/14/2023, 9:29 PMrun_deployment accepts a flow_run_name kwarg
• templating kwargs into a flow e.g.
@flow(flow_run_name="Fetch {table}")
def query_table(table: str):
...Pooyan
12/14/2023, 9:32 PMcreate_flow_run api endpoint?Pooyan
12/14/2023, 9:33 PMrun_deployment only?Nate
12/14/2023, 9:35 PMhanks I dont think we can make it dynamically change going with the second solution right?you mean change the template on a flow-run basis? correct, you cannot
Nate
12/14/2023, 9:37 PMPooyan
12/14/2023, 9:38 PMPooyan
12/14/2023, 9:40 PMcreate_flow_run endpoint and pass headers, payload via a rest post requestPooyan
12/14/2023, 9:40 PMNate
12/14/2023, 9:52 PMIn [1]: from prefect import flow
In [2]: from prefect.blocks.system import String
In [3]: await String(value="foo-project").save("project-name", overwrite=True)
Out[3]: UUID('18e7bf69-71ac-43af-b1ec-85b42b8345c6')
In [4]: @flow(flow_run_name=f"{String.load('project-name').value}" + " {bar}")
...: def foo(bar = 42):
...: pass
...:
In [5]: foo()Pooyan
12/14/2023, 9:56 PMNate
12/14/2023, 9:57 PMflow_run_name can be dynamically templated with whatever project name you have + parameters passed into the flow run. tldr you dont need to worry about passing the flow run name through the APINate
12/14/2023, 9:58 PMPooyan
12/14/2023, 9:59 PMPooyan
12/14/2023, 11:37 PM@flow(flow_run_name=f"{String.load('project-name').value}" + " {bar}") so instead of bar we can have whatever expression (even python) and it would be resolved at runtime? It is not exactly a jinja format so curious how it works.Pooyan
12/14/2023, 11:39 PMString.load('project-name').value it seems it can be just whatever pythonPooyan
12/14/2023, 11:44 PM@flow(name="flow-name", flow_run_name=f"{workflow_input[some_key]}")
async def my_flow(workflow_input):
.........Nate
12/14/2023, 11:46 PM"{some_flow_param}" that wouldnt be populated with the value of some_flow_param until flow runtime (when the value of that param is defined)Nate
12/14/2023, 11:47 PMflow_run_name you'd see
"foo-project {bar}"Pooyan
12/14/2023, 11:49 PMprefect.exceptions.ScriptError: Script at 'flows/my_flow.py' encountered an exception: NameError("name 'workflow_input' is not defined")Pooyan
12/14/2023, 11:49 PMNate
12/14/2023, 11:51 PM@flow(name="flow-name", flow_run_name=f"{workflow_input[some_key]}")
async def my_flow(workflow_input):
.........
this isnt the same thing as what I did
f"{workflow_input[some_key]}"
this is all an f-string, where f-strings are going to render immediately, so workflow_input needs to be a defined variable at the time you define the string, whereas something like "{workflow_input}" is a valid string, which just so happens to be a template string that you could do .format() onNate
12/14/2023, 11:53 PMMarvin
12/14/2023, 11:54 PMstring module in Python and rely on the Template class. They use placeholders defined with $ and optional curly braces ({}), for example: ${variable}. This method is considered safer for handling user-supplied input, as it avoids directly evaluating the embedded expressions.
2. F-strings: Introduced in Python 3.6 as "formatted string literals," f-strings use curly braces {} containing expressions that are replaced with their values. They use an f or F prefix, like: f"{expression}". F-strings are more concise and generally faster. They can include any valid Python expressions and inline evaluation, including method calls, mathematical operations, and even inline function definitions.
What you can put in template strings in Python:
- Template strings allow only basic variable substitutions, where the variables are placeholders written within the string with a $ prefix.
- Template strings are more limited compared to f-strings. They don't allow complex expressions or inline evaluation.
- You can pass variables or complex expressions to a template string through a dictionary by using the substitute method, which matches keys from the dictionary with placeholders in the template.
As for the excerpt mentioning a template attribute, it seems to refer to a specific context within the Prefect framework, possibly related to the configuration of some resource requirement template. This does not describe Python's general-purpose template strings but likely a Prefect-defined attribute.
In the context mentioned in the discussion about not being able to have a dictionary in a template string directly: indeed, in standard Python template strings, you cannot directly embed a dictionary. Instead, you would extract the variables you need or use the substitute method to replace placeholders with dictionary values. With f-strings, however, you can directly reference any Python expression, including dictionary lookups, within the string itself.Nate
12/14/2023, 11:55 PMIn [1]: tmpl = "{foo}"
In [2]: tmpl.format(foo="bar")
Out[2]: 'bar'
also be more concise pleaseMarvin
12/14/2023, 11:55 PMformat method allows you to create strings with placeholders enclosed in curly braces {} that can be replaced with the values of variables. You can pass these variables as arguments to format, and it will substitute the placeholders with the corresponding values based on either positional or keyword arguments.
In the example you provided, "{foo}".format(foo="bar"), foo is a placeholder within the string, and format(foo="bar") is replacing that placeholder with the string "bar". The format method is quite flexible and allows referencing attributes of objects, indexing, and even accessing items of dictionaries within the placeholders.Pooyan
12/14/2023, 11:56 PM