Channels
#prefect-server
Title
g
Guilherme Petris
12/06/2021, 10:13 AMHey! I’m trying to figure it out why my flows are registering the run in less than a second, but in this situation if a run the script separately they run for a couple of minutes.
Copy code
from analytics_toolbox.analytics_toolbox import *
from analytics_toolbox.analytics_toolbox import PendoUtils as pendo_utils
from analytics_toolbox.analytics_toolbox import SnowflakeUtils as snowflake_utils
from prefect import task, Flow
from prefect.utilities.notifications import slack_notifier
@task ()
def main():
......
with Flow("pendo_featureEvents") as flow:
main()
flow.register(project_name="pendo_api")a
Anna Geller
12/06/2021, 10:20 AM
@Guilherme Petris I didn’t understand the question. But perhaps the issue is that you have flow.register() in your flow definition and if you start it from the UI, it tries to reregister?
In general, you can remove the last line and use CLI instead:
Copy code
prefect register --project yourproject -p yourflow.py✅ 1
g
Guilherme Petris
12/06/2021, 10:37 AMOh! Thanks @Anna Geller! It’s exactly the issue!
🙌 1
Hey @Anna Geller another question related - when i’m running single independent task without any import from custom modules everything seems to work fine using the prefect register.
The problem is when i wanna do in bulk, when i to the prefect register without the path to the modules it works but it fails when i run the flow with the message ModuleNotFoundError
a
Anna Geller
12/06/2021, 12:08 PM
@Guilherme Petris you would need to make sure your modules are installed within the execution environment (e.g, local agent), or at least added to the PATH so that Prefect knows how to import them
g
Guilherme Petris
12/06/2021, 2:20 PMYou mean using the flow.run_config = LocalRun(PATH) ?
a
Anna Geller
12/06/2021, 2:39 PM
yes, that would be one way of doing it
g
Guilherme Petris
12/06/2021, 3:27 PMWell, i’m getting the status pending without any new inputs trying with several stuff. Already checked the prefect agent path here too.
a
Anna Geller
12/06/2021, 3:34 PM
is it stuck in a Pending state or what is the issue now?
if you don’t want to do it with paths, you can create setup.py and make your package installable - a similar issue was answered here
2 Views