https://prefect.io logo
Powered by
Title
d

Dylan

06/01/2022, 9:33 PM
Hey why would registering a flow cause the code in the flow to run?
For example, this is definitely executing the code in my flow block
with Flow(
    name=FLOW_NAME,
    storage=Local(
        add_default_labels=False,
        stored_as_script=True,
        path="flows/order_list_ingest.py",
    ),
    run_config=KubernetesRun(
        image=f"{config.ECR_REGISTRY}/{config.PROJECT}:{config.IMAGE_TAG}",
        labels=[config.ENVIRONMENT],
        job_template_path="templates/base-flow-template.yaml",
        service_account_name=config.S3_BUCKET,
    ),
    executor=LocalExecutor(),
) as flow:
    client = OrderApi()
    raw_data = get_order_list(client, **seed_params())
    save(raw_data, PREFIX)
the get_oder_list and save functions are prefect tasks
and the registration is being done with the prefect CLI. This particular time around I’m running this locally
prefect register --project parcel-planet-ingest -p flows/order_list_ingest.py
In this case, debug logging printed data from get_order_list out, much to my surprise
Actually, I think I have a bug. Turns out the code I referenced isn’t a task. Duh to me! lol
a

Anna Geller

06/01/2022, 9:42 PM
your flow gets evaluated during registration, but it's not executed - if not, something extremely rare and weird is going on are you sure you don't have flow.run() lurking around somewhere in your code?
d

Dylan

06/01/2022, 9:42 PM
right, that’s the other thing I started poking around for
I do in integration tests (that test my code outside of prefect) but that code is all orthogonal to the flow code
the int tests just pull in the tasks that compose the flow
This is still really weird. I did just get puppies so maybe I have puppy brain.
a

Anna Geller

06/01/2022, 9:53 PM
hahaha 😂
but now I really want to see a photo of your puppies
k

Kevin Kho

06/01/2022, 10:26 PM
Stupid question, but are those tasks? If they are functions they will execute eagerly
d

Dylan

06/01/2022, 10:32 PM
They’re tasks, but I’m kinda curious if there’s any sort of fuzzy matching prefect registration does, bc the function I called in the flow didn’t actually exist, but a function close to that name did, and it executed. Very odd but I’ve fixed the code so no biggie.
:thank-you: 1
❤️ 2
k

Kevin Kho

06/01/2022, 10:36 PM
Are they siblings? No that fuzzy matching is too advanced. That’s weird haha
d

Dylan

06/01/2022, 10:37 PM
Was calling 
get_order_list
in the flow when I meant to call 
api_get_order_list
, there isn’t a 
get_order_list
. So I wonder if the 
__pycache__
came into play at all. Could obvi be a rabbit hole for no good reason but it was definitely strange to see code being executed, whose function name didn’t quite exist.
Yeah last 2 pups from a litter of 5!
Girl and boy, Rosie and Wrigley (Riggs for short)
k

Kevin Kho

06/01/2022, 10:39 PM
With 
flow.run()
or Cloud flow? Could happen if there was no re-registration?
d

Dylan

06/01/2022, 10:47 PM
That’s the part that surprised me, I don’t call 
.run()
on anything in the flow itself.
and the registration command I ran locally
with the prefect cli
k

Kevin Kho

06/01/2022, 10:49 PM
That’s very mysterious lol. Really don’t know how that would happen
a

Anna Geller

06/01/2022, 11:12 PM
same - I'd try approaching it step-by-step and creating a new flow with only one task from your current flow, and then another, and then add storage, and then add run config and test it all one by one to figure out which part causes this weird behavior your puppies are the cutest! 🐶
#prefect-communityJoin Slack
Powered by