Hi guys. is there any way to securely pass the result of a task to the next task?
z
Zanie
12/10/2020, 3:48 PM
Hi! Could you explain in a bit more detail what you mean by securely / what your use-case is?
h
Hamed Sheykhlou
12/10/2020, 3:52 PM
consider this as an example code
Copy code
import requests
from prefect import Flow, task
@task(name='extract_task')
def extract():
res = requests.get('<https://jsonplaceholder.typicode.com/todos/1>')
res_txt = res.text
return res_txt
@task()
def load(a):
"""Print the data to indicate it was received"""
print("Here's your data1: {}".format(a))
with Flow("Callback-Example") as flow:
e = extract()
l = load(e)
flow.run()
Hi @Hamed Sheykhlou,
Where are you planning to run your Prefect Flows? Generally, we use Results to make sure that your data never leaves your infrastructure. Securing the GCS/S3 buckets where the results live is usually a matter of putting proper permissions on the execution infrastructure and the buckets.
Dylan
12/10/2020, 4:29 PM
In your example, if you were to use a Result, Prefect would do the work to ensure that
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.