Hello, a bit of an odd one, as I am not sure what ...
# prefect-community
a
Hello, a bit of an odd one, as I am not sure what I am doing wrong here. Effectively, a very simple task seems to be running forever when run via Prefect Cloud (using Local Agent, Prefect 1.0). The same flow when run on the machine that runs the local agent seems to work fine. Code:
Copy code
from prefect import task, Flow, Parameter

@task
def task_function(in_list_size=10, out_list_size=4):

    in_list = list(range(in_list_size))
    print(f">>> Input list: {in_list}")
    
    out_list = list(range(out_list_size))
    print(f">>> Output list: {out_list}")
    
    diff_list = [i for i in in_list if i not in out_list]
    print(f">>> Result list: {diff_list}")
    
    return diff_list


with Flow("Test Flow") as flow:

    in_list_size = Parameter("input_list_size", default=10)
    out_list_size = Parameter("output_list_size", default=4)
    result = task_function(in_list_size=in_list_size, out_list_size=out_list_size)


flow.register(project_name="tutorial")

if __name__ == "__main__":
    flow.run()
Trying to figure out where the problem is - guessing something is wrong on the Local Agent side?
a
Can you elaborate on how long is forever in this context?;) it's normal that the orchestrator adds some tiny extra latency as compared to running something as a standalone function, but it shouldn't be that noticeable ideally, you should remove
Copy code
flow.register(project_name="tutorial")
from your flow code and try the CLI instead
Copy code
prefect register --project xxx -p yourflow.py
and when performance and latency are critical in your use case, try Prefect 2.0 instead - this topic explains it more https://discourse.prefect.io/t/should-i-start-with-prefect-2-0-orion-skipping-prefect-1-0/544
a
Hi Anna, thank you for the reply. Apologies, I should have been clearer - it's not that it runs longer, it simply never seems to finish. I am suspecting that there is something off with python (conda) environment on the Local Agent side, the error I am getting in the console is:
Copy code
Process PID 1628 returned non-zero exit code 3221225477!
It's just a little strange, because I am able to run the flow without any issues from the command line (even using the same conda environment as the Local Agent)
a
did you remove the register line?
Copy code
from prefect import task, Flow, Parameter

@task
def task_function(in_list_size=10, out_list_size=4):

    in_list = list(range(in_list_size))
    print(f">>> Input list: {in_list}")
    
    out_list = list(range(out_list_size))
    print(f">>> Output list: {out_list}")
    
    diff_list = [i for i in in_list if i not in out_list]
    print(f">>> Result list: {diff_list}")
    
    return diff_list


with Flow("Test Flow") as flow:

    in_list_size = Parameter("input_list_size", default=10)
    out_list_size = Parameter("output_list_size", default=4)
    result = task_function(in_list_size=in_list_size, out_list_size=out_list_size)

if __name__ == "__main__":
    flow.run()
if you have some issues configuring agent and run configuration, check out this repo with examples, perhaps it helps https://github.com/anna-geller/packaging-prefect-flows/
a
Thank you, will take a look! I must be doing something stupid here 🙂
a
your flow looks totally fine apart from the registration step - how did you start that agent?
I shared the repo since storage + run configs can be set in so many ways that seeing some examples of possible configurations before registering for agent-run may be helpful
m
@Arnas Di you ever figure this out? Running into the exact same issue (that I am sure is something silly around my environment). Runs completely fine on base environment, but when I create a new environment from scratch (with most of the same libraries). I have the same issues.
a
Hey @Michael Ng. I think in my case it was the fact that I did not remove the flow.register() part from the script (pretty stupid, I know)
Also, not sure which OS you're working on - I've been unfortunate enough to be using Windows (corporate stuff...) and there were often issues when referring to mapped network drives (e.g. M:\your\path), but not sure this applies in your case
If your problem persists - let me know. It has been a while since we resolved this, but I can go through my chat logs with a colleague when we were debugging this, maybe there was something else that we changed.
🙌 1