https://prefect.io
Join Slack
Hi, Is there any specific usage when working with dict() returned by task? below part is failing wit...
# ask-community
i
Hi, Is there any specific usage when working with dict() returned by task? below part is failing with message that 'get_connections' task class has no method get().
Copy code
token = EnvVarSecret("TOKEN")
connections = get_connections(token=token)  # this is a task that returns dict(name, connection_string)
start_task_result = start_task(connections.get('some_key'))  # i'd expect to get value from dict, not attribute error
a
No, there isn't. The issue that you see is likely due to build-time vs. runtime differences. During build-time Prefect constructs your flow graph, while at runtime it passes data between tasks. The Secret task should be only called from within a flow - do you happen to call it at the module scope?
i
This is a part of a flow. Secret is working fine and returns what I expect and get_connections() also return dict() during run_time. I need to pass only single value into next task, not whole dict()
That means that task should return single string (in my case), not a dict()?
a
I don't have enough information to give you any recommendations - could you share an example I could reproduce to understand your issue better?
i
Copy code
get_connections = K2Vault(
    name="Get connections from vault"
)

start_task = MySQLExecute(
    name='XXX',
    query=sql_start,
    commit=True
)

with Flow(name="XXX") as flow:
    token = EnvVarSecret("TOKEN")
    connections = get_connections(token=token)
    start_task_result = start_task(**eval(connections.get('aws_connection')))
Suppose that .get() and eval() will not work; that should be handled in Task.
At least kwargs unpacking should work?
Copy code
conn = get_connections_task(...)
run_result = run_task(**conn)
a
this is the part that you may need to address:
Copy code
start_task_result = start_task(**eval(connections.get('aws_connection')))
in Prefect 1.0, you can only call tasks in your Flow. You should move this part to a separate 
@task
and call it in your Flow to make it work
i
What about unpacking? Are you going to allow this in Orion?
a
100% - in Prefect 2.0 you can run any Python code you like incl. unpacking there is no more DAG requirement to package all code into tasks - you could have purely a Python script with a single 
@flow
decorator which gives Prefect an entry point to orchestration and that's all you need to turn your code into a Prefect workflow
12 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by