Hello team I m looking use Prefect at my company While testi Prefect Community #ask-community

Hello team! I'm looking use Prefect at my company....

Anthony L

05/02/2022, 10:02 PM

Hello team! I'm looking use Prefect at my company. While testing, I noticed that

random

generates the same values between runs. I'm using Prefect Cloud 1.0 and running a local agent (with Prefect 1.2.1 library installed). Here's my flow:

Copy code

import prefect
from prefect import task, Flow
from random import random

@task
def task1():
    logger = prefect.context.get("logger")
    random_val = random()
    <http://logger.info|logger.info>(f"{random_val=}")
    if random_val < 0.5:
        raise Exception()


with Flow("random-bug-flow") as flow:
    task1()

flow.register(project_name="ant-flows")

I manually scheduled a run of this flow in Prefect Cloud and I see the random value in the logs. However, when I schedule another run (and any run thereafter), it generates the same "random" value. Is this expected?

Anthony L

05/02/2022, 10:03 PM

I tried looking for but couldn't find anything related to this in Prefect's Github issues and Discourse threads

Kevin Kho

05/02/2022, 10:04 PM

This is not expected, but could happen maybe depending on how you trigger the runs. How are you triggering the run?

Anthony L

05/02/2022, 10:05 PM

Just through the Prefect Cloud 1.0 UI.. navigated to my flow, clicked on the "Run" tab, and scheduled it to run immediately

Kevin Kho

05/02/2022, 10:06 PM

Will try one sec

👍 1

Anthony L

05/02/2022, 10:08 PM

I even tried restarting my local agent (thinking it's related the random's seed value when the agent starts up).. but same problem when rerunning the flow on the restarted agent

Kevin Kho

05/02/2022, 10:10 PM

I can replicate. I think this has to do with serialization of the task maybe. Will look into it and likely open an issue

Anthony L

05/02/2022, 10:11 PM

Whew, I thought I was going crazy. Sweet. Thanks!

Anna Geller

05/02/2022, 11:56 PM

Yup, this is likely either extreme randomness (unlikely 😄) or a serialization issue (more likely) Can you try using script storage? If you register from CLI rather than

flow.register()

it will, by default, use script storage and should then dynamically generate the number I tried that and I'm getting different numbers at each run:

Copy code

from prefect import task, Flow
from random import random


@task(log_stdout=True)
def rand_float():
    random_val = random()
    print(f"{random_val=}")


with Flow("random_nr_flow") as flow:
    rand_float()

Anna Geller

05/02/2022, 11:57 PM

last 3rd time: again a different number

Copy code

0.26524578590511527

Copy code

(packaging-prefect-flows) ➜  community-1-0 prefect run --name random_nr_flow --watch
Looking up flow metadata... Done
Creating run for flow 'random_nr_flow'... Done
└── Name: pink-numbat
└── UUID: a2d3da01-2be6-47cc-b2f9-39b517ea320d
└── Labels: ['annas-MBP.fritz.box']
└── Parameters: {}
└── Context: {}
└── URL: <https://cloud.prefect.io/anna-prefect/flow-run/a2d3da01-2be6-47cc-b2f9-39b517ea320d>
Watching flow run execution...
└── 01:56:45 | INFO    | Entered state <Scheduled>: Flow run scheduled.
└── 01:56:54 | INFO    | Entered state <Submitted>: Submitted for execution
└── 01:56:55 | INFO    | Submitted for execution: PID: 38051
└── 01:56:55 | INFO    | Entered state <Running>: Running flow.
└── 01:56:55 | INFO    | Beginning Flow run for 'random_nr_flow'
└── 01:56:56 | INFO    | Task 'rand_float': Starting task run...
└── 01:56:57 | INFO    | random_val=0.26524578590511527
└── 01:56:57 | INFO    | Task 'rand_float': Finished task run for task with final state: 'Success'
└── 01:56:57 | INFO    | Flow run SUCCESS: all reference tasks succeeded
└── 01:56:57 | INFO    | Entered state <Success>: All reference tasks succeeded.
Flow run succeeded!

Anthony L

05/03/2022, 3:28 AM

Registering the flow via CLI instead of

flow.register(..)

fixes the issue on my end. Thanks Anna!

🙌 1

Kevin Kho

05/03/2022, 3:35 AM

Oh thanks for mentioning. Will open and include that in the issue

Kevin Kho

05/03/2022, 2:20 PM

Issue: https://github.com/PrefectHQ/prefect/issues/5744

8 Views

Open in Slack

Previous Next