https://prefect.io logo
#prefect-community
Title
# prefect-community
а

Андрій Демиденко

03/29/2022, 10:32 AM
Hi everybody, I'm working on a flow which should register another flow. And while flow.run() works perfectly fine for me, the running on the Cloud gives an error described in the picture. I've dived into this issue and realized that it does register another flow but outputs an error printing the success message))). So I tried changing print(msg) -> print(msg.encode('utf-8')) in the line 979 of the prefect/client/client.py and it helped me to fix this issue. Please consider adding this changing to the later versions of Prefect. Thank you in advance
a

Anna Geller

03/29/2022, 10:37 AM
Can you share the code that registers other flows? The output looks like you are using CLI rather than
prefect.Client
Also, can you explain the use case a bit more? Perhaps there is some better way of approaching this. Are you trying to build some CI/CD pipeline using this approach?
а

Андрій Демиденко

03/29/2022, 10:45 AM
Sorry, but I can't share the code. The purpose of this flow is to be able to register new flows without having direct access to the local agent. It works by the following scheme: the task calls a function from another file (in order to avoid reregistration of this flow) and the function calls another_flow.register(some_project)
To be honest I don't know the difference between CLI and prefect.Client so my approach can be wrong
a

Anna Geller

03/29/2022, 10:49 AM
Do you use some Cyrillic characters in your flow name?
а

Андрій Демиденко

03/29/2022, 10:49 AM
No
a

Anna Geller

03/29/2022, 10:51 AM
I plan to create a Github issue but want to gather more information first. Can you explain your use case?
а

Андрій Демиденко

03/29/2022, 10:53 AM
What info should I add (because I can't share any sensitive information)?
Or should I gather all the info in 1 message?
a

Anna Geller

03/29/2022, 10:56 AM
That would be great, you can redact any sensitive information. But e.g. it's useful to know WHERE do you call
flow.register()
because doing that within a Flow content may lead to unexpected results
I can imagine sth like this should work, but it also depends on the Storage of your flows
Copy code
from prefect import Client
from prefect import Flow, task


@task(log_stdout=True)
def register_other_flow():
    from yourmodule import your_flow_object
    Client().register(flow=your_flow_object, project_name="yourproject")


with Flow("xxx") as flow:
    register_other_flow()
а

Андрій Демиденко

03/29/2022, 10:59 AM
I see. I think it's within Flow context: not directly but the function which calls flow.register() is within the Flow context
a

Anna Geller

03/29/2022, 11:00 AM
but doing this is discouraged since in the Flow block you should only be calling your tasks:
Copy code
with Flow("xxx") as flow:
    your_flow_object.register("xxx")
а

Андрій Демиденко

03/29/2022, 11:03 AM
It's more closer to this main_file.py
Copy code
from prefect import Flow, task


@task(log_stdout=True)
def register_other_flow():
    from yourmodule import run
    run()


with Flow("xxx") as flow:
    register_other_flow()
yourmodule.py
Copy code
def run():
    # here we conduct the registration
    flow.register()
a

Anna Geller

03/29/2022, 11:05 AM
I see. Can you try this and see if you get the same error?
Copy code
from prefect import Client
from prefect import Flow, task


@task(log_stdout=True)
def register_other_flow():
    from yourmodule import your_flow_object
    Client().register(flow=your_flow_object, project_name="yourproject")


with Flow("xxx") as flow:
    register_other_flow()
а

Андрій Демиденко

03/29/2022, 11:05 AM
Ok.
a

Anna Geller

03/29/2022, 11:06 AM
in theory, it's calling the same register method, but dunno, could matter somehow 😄 I promise I'll open an issue if this doesn't work
а

Андрій Демиденко

03/29/2022, 11:10 AM
Is Client.register() a static method? Can Client().register(...) also work?
a

Anna Geller

03/29/2022, 11:11 AM
sorry, my mistake, it must be Client()
а

Андрій Демиденко

03/29/2022, 11:14 AM
Client().register(...) also fails. What should I specify to make it work?
a

Anna Geller

03/29/2022, 11:17 AM
thanks for trying that. Let me open an issue then 🙂 @Marvin open "Add UTF-8 encoding to the registration message"
а

Андрій Демиденко

03/29/2022, 11:18 AM
@Anna Geller Thank you very much
👍 1
a

Anna Geller

03/29/2022, 6:23 PM
@Андрій Демиденко I was trying to reproduce your issue and I couldn't. I added a comment on the issue and closed it. All strings in Python 3+ are UTF-8 encoded already and the change you suggested would convert those to bytes which wouldn't make any difference in fixing your issue. Happy to help if you are still having issues with this. Generally speaking, it's recommended (and much easier!) to register flows from the CLI. Have you tried that? This way you could register multiple flows at once without having to register flows from other flows. E.g.:
Copy code
prefect register --project yourproject -p /path/to/multiple/flows/
this way you can register both at once with no extra work
3 Views