Hello all, I am trying to understand some best practices when writing flows and tasks where the initialisation/closure of a "client" is involved. For example, let's say I have the following code:
Copy code
my_client = AwesomeService(**params)
# Initialisation
my_client.open()
# Do the task
my_client.do_the_task(abc)
# Close the session
my_client.close()
I would like to refactor the code in a way that
my_client.do_the_task
is actually a
@task
in prefect, but I don't know if is a best practice to have all this code in just one task, because if I want to use it in a concurrent manner with
submit
it would open and close multiple sessions - and ideally it would be just one "open" - do all the things you want to do - and one "close". And for this I might need to send the client as parameter to the task... right?
Don't know if someone has stumbled into this kind of refactor issue/dilemma here 😅
t
Tim Galvin
03/14/2024, 8:44 AM
I am not sure what your client is. Do you mean you want to start a client up for a sdpecific task runner?
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.