https://prefect.io logo
m

Matt Gordon

01/22/2021, 6:46 PM
Hi Pre-friends. I wonder if I could ask for some help. I am working on a project using Prefect Core, and am trying to create a Flow-of-Flows. The documentation says of
StartFlowRun
:
Task used to kick off a flow run using Prefect Core’s server or Prefect Cloud. If multiple versions of the flow are found, this task will kick off the most recent unarchived version.
(Emphasis mine.) The problem I’m having is this: if I instantiate StartFlowRun with a
project_name
, I get:
Malformed response received from Cloud - please ensure that you have an API token properly configured.
If I instantiate it without specifying a
project_name
, at runtime I see:
raise ValueError("Must provide a project name.")
raised from
StartFlowRun
n

nicholas

01/22/2021, 6:52 PM
Hi @Matt Gordon - a couple quick questions: 1. What version of Prefect Server are you running? 2. What version of Prefect is your flow using? 3. Have you run
prefect backend server
before running your script?
m

Matt Gordon

01/22/2021, 6:54 PM
Nicholas, thanks for taking the time to answer. I’m using
prefect==0.13.19
for our execution. Up until now, I have not needed to run Prefect Server to run any of my flows. It sounds like using StartFlowRun will require me to run the backend server?
I am very new to Prefect and have not really yet quite grokked the role of the backend server.
We are planning to continue using Airflow as our topmost job orchestration for the moment.
n

nicholas

01/22/2021, 7:07 PM
Ahh gotcha. Yes, in order to use
StartFlowRun
, you'll need to be using a backend (Server or Cloud). If you're working with Core-only, there's no central arbiter of flow state. In addition,
flow.run
doesn't support concurrent flow runs so your only option is to call out to another script where a separate flow is defined (with
python other_flow.py
)
m

Matt Gordon

01/22/2021, 7:12 PM
Okay. It sounds like ,for the moment, the simplest thing for me if I just want to run a simple set of dependent flows is to just run them in order, and cache the
Result
in between them to pass the data along.
I don’t need antyhing more complicated than that at the moment
n

nicholas

01/22/2021, 7:15 PM
That sounds right - if you can get away with that locally that would be the easiest way to get started
m

Matt Gordon

01/22/2021, 7:15 PM
Great thanks
👍 1
No need to build a better mousetrap at this juncture
😄 1
2 Views