https://prefect.io logo
Title
a

Andrei Tulbure

10/04/2022, 4:14 PM
He all, Quick question: If I have several tasks inside a prefect flow, and I want the downstream tasks to wait for some upstream tasks (which are mapped), what can I do? These are not sequential flows ... to be able to use the set.upstream and wait for flow. Any help is appreciated.
1
n

Nicolas Felipe

10/04/2022, 4:50 PM
Hi Andrei, not sure if may be useful the
wait_for=
on task calling.
a

Andrei Tulbure

10/04/2022, 4:52 PM
I do not use Prefect 2. I tried with upstream task, but the above task is mapped. something like
task_res = call_ts.map(task_args)
new_task_res = task2(args)
new_task_res.set_upstream(task_res)
But I am not sure it works as I get this error when testing it locally`prefect.exceptions.ClientError: [{'path': ['flow'], 'message': 'Missing Authorization header in JWT authentication mode', 'extensions': {'path': '$', 'code': 'invalid-headers', 'exception': {'message': 'Missing Authorization header in JWT authentication mode'}}}]`
z

Zanie

10/04/2022, 6:37 PM
That error looks like you’re attempting to use Prefect Cloud without logging in.
❤️ 1
a

Andrei Tulbure

10/05/2022, 4:47 PM
@Zanie thanks a lot!!!!! Question: how can I run prefect flows locally -> where is the cloud run configured ? I want to run the flows locally
Because i struggle to find why I can`t run these flow locally
z

Zanie

10/05/2022, 5:21 PM
How are you running the flow?
Have you looked at the
prefect run
command? The
--help
menu should be helpful 🙂
a

Andrei Tulbure

10/05/2022, 5:27 PM
yep, but I took over a build that somebody else built - and they run it v ia prefect cloud and ECS. The problem is that I need to add things (new tasks) or to set upstream tasks for some other tasks. In order to not mess up Prod, I want to test these locally and I am not able to run this locally ,because each time the master flow triggers running a child flow(the flow I run is triggering other flows) -> it tried to run them on prefect cloud and it was these missing auth token issues from above. How can I cancel this cloud dependency for DEV ? I am somewhat new to infra setup for prefect
z

Zanie

10/05/2022, 5:41 PM
You cannot trigger other flows without using a backing server
You can set
prefect backend server
and run
prefect server start
to get one. You’ll need to register the flows too.
1
a

Andrei Tulbure

10/05/2022, 9:54 PM
Is there a tutorial for this ?
z

Zanie

10/05/2022, 10:30 PM
@Anna Geller might have one around. There's not a great tutorial for jumping into advanced features that someone else set up 🙃 but I bet we can find some helpful resources
:thank-you: 1
1
a

Anna Geller

10/05/2022, 10:53 PM
there's a lot here, not sure which problem you are trying to solve first but here are some pointers: • Troubleshooting Prefect Server issues - list of resources you may check - Deployments & DevOps - Prefect Community - for troubleshooting Server backend issues • How to Make Your Data Pipelines More Dynamic Using Parameters in Prefect - Prefect - for prefect run CLI in v1 • Topics tagged state-dependencies - for state dependencies resources
a

Andrei Tulbure

10/06/2022, 9:24 AM
@Anna Geller @Zanie thanks a lot! The first issue I want to solve , it to test if the changes are made are ok (and I do not want to test it in prod :)) ) - so I would like to do it locally (hence have the master flow spin up the other flows using local compute, not using the prefect cloud )
a

Anna Geller

10/06/2022, 9:28 AM
why not using Prefect Cloud for development flow? as long as you don't write to some prod DB from that flow, you can run it in Cloud as well btw, you can also test your flow locally just by using:
flow.run()
in 2.0, it's even easier - I'd recommend the default version which is Prefect 2.0 docs.prefect.io if you are getting started now
a

Andrei Tulbure

10/06/2022, 12:58 PM
I am taking over something that other have written. it`s not 2.0 sadly
a

Anna Geller

10/06/2022, 1:52 PM
gotcha, still you can totally use Cloud for development flows
a

Andrei Tulbure

10/11/2022, 8:45 PM
@Zanie I still cannot schedule dependent Tasks. I want task 1 to run after flows 1,2,3 are finished, yet I am unable to do it. I scoured the whole documentation ... Do you happen to have an example or some good tutorials ?
re-birthing this....any good tutorials for scheduling dependent Tasks or flows ? I am really not managing to have an elegant solution for this 😄 @Zanie
a

Anna Geller

10/26/2022, 12:10 PM
did you start planning your migration to Prefect 2 @Andrei Tulbure? if you struggle to do it in v1, it makes sense to migrate it to the default version (Prefect 2), sooner or later this will need to happen so makes sense to do it now rather than building workarounds in v1 please don't tag Michael, he's doing a lot, better ask more specific question in a new thread
1
a

Andrei Tulbure

10/26/2022, 1:27 PM
As I said above, I cannot sadly switch to Prefect 2.x right now ... stuck with 1.x