https://prefect.io
Join Slack
hello, is there a way to run a flow and it's subtask just as code? I know you can run tasks as code ...
# ask-community
j
hello, is there a way to run a flow and it's subtask just as code? I know you can run tasks as code but I was curious if there's a right way to do a flow equivalent
t
Hi John, what do you mean by running ‘just as code’?
k
I think maybe like 
my_task.fn()
?
t
If thats the case, @John O'Farrell you should be able to call flow functions as they are without useing .fn() just call any python function that has a flow decorator as you normally would.
Copy code
@flow
def my_flow():
    print('hi')

# to run
my_flow()
j
Yeah I'd like the equivalent of 
my_task.fn()
for flows. I'm trying to import the flow into a unit test file and when I call it without a decorator in the test module it still acts like a prefect flow
k
Yeah so flows do have 
.fn()
as well, but that doesn't propagate down to the task level. For example:
Copy code
from prefect import flow, task


@flow(log_prints=True)
def my_flow():
    say_hello()


@task
def say_hello():
    print(f"Hello!")


if __name__ == "__main__":
    my_flow.fn()
will land you in 
RuntimeError
territory because tasks cannot be run outside of a flow
j
yeah that's what I had noticed. I was hoping there was a way to get so I could run all the code within a flow including the tasks outside of the prefect environment
k
Have you checked out prefect_test_harness?
j
I haven't but tbh that's less appealing to me. One of my hopes was that I would be able to integrate the code from this flow into our existing python test framework without necessarily needing to build prefect-specific architecture for the testing
k
Aside from utilizing the test harness where you're testing flows, what would the major obstacles be?
j
basically I wanted to run the code without all of the API calls that prefect makes as a part of running flows. As if it's not a flow at all just a function. So using asserts to test flows doesn't change that I'm still using a flow run. honestly I was kinda surprised that the 
flow.fn()
didn't allow the tasks within flows to be run as regular code because that would have been pretty much exactly what I'm looking for
k
Gotcha. I would still recommend going with the test harness for now. It clears the API URL so the test flow runs don't make any calls to your server or cloud account, but your flow will still behave like a flow.
j
I'm assuming I need to install pytest for the test harness to work properly even if I'm using unittest as the test framework?
k
yep, pytest is a dependency for the test harness
or more accurately, the testing utilities module that has the test harness in it imports pytest. I don't know if it acutally uses pytest, but if you import the test harness you'll get a module not found error on pytest without it installed
m
Did you find a solution for this @John O'Farrell outside of using the test harness? I'm in the same boat that I don't need my flows to be run as flows for the vast majority of my tests, and tasks that have retries set up cause the tests to take, well, forever to run.
This might be of interest to you, John. It's definitely hacky, but it supports the flow it seems like we both want. When calling 
my_flow.fn()
here everything is ran as normal python functions, without involving Prefect at all. This means that run_logger, retries etc. can all be circumvented and ignored where that makes sense. I'm using pytest here, but you should be able to achieve the same thing using just the unittest library.
patched_task_decorator.py
29 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by