https://prefect.io logo
Powered by
Title
j

Jon

10/10/2022, 2:38 PM
where can I find some examples of how to pass required flow parameters for prefect v1? when i go to register a flow with required parameters, I get 
AttributeError: 'Parameter' object ...
. Why is the flow trying to run when I register it?
1️⃣ 1
the more unclear thing is that when i declare a parameter, it returns a parameter object when i really just want the value that's passed to the flow
m

Mason Menges

10/10/2022, 4:00 PM
Hey @Jon This is probably a good place to start https://docs-v1.prefect.io/core/concepts/parameters.html, Essentially you have to define your parameters within a "Parameter" task in prefect 1, it's worth noting that this is no longer necessary in Prefect 2 as you can simply include the parameter definition in you flow/task functions and pass data as you normally would in python (for reference https://discourse.prefect.io/t/guide-to-implementing-parameters-between-prefect-1-0-and-2-0/1321).
j

Jon

10/10/2022, 4:01 PM
thanks @Mason Menges! here's what was happening: i hadn't declared my class methods as tasks, so they didn't understand how to parse the Parameter object. i couldn't figure out how to declare class methods as tasks, but i did test wrapping the logic i had in the flow as a function + task decorator and it worked. is it possible for class methods to be tasks?
m

Mason Menges

10/10/2022, 4:07 PM
Generally speaking yes, this covers the basics I think https://discourse.prefect.io/t/can-i-define-my-tasks-as-classes-rather-than-functions/54
j

Jon

10/10/2022, 4:12 PM
i dont think that's what im asking about. I want to do something like this: 
class Example():
    @task
    def method_a():
        ...

    @task
    def method_b():
        ...

with Flow() as Flow:
    with Example() as e:
        e.method_a() # a task in the flow
        e.method_b() # a task in the flow
that doc is to convert a function to a class. here i want to convert class methods to tasks
the only way i can get my flow to work is if i do: 
class Example():
    def method_a():
        ...
    def method_b():
        ...

@task
def run_example():
    with Example() as e:
        e.method_a() # not a task? lost in the ether
        e.method_b() # also not a task, lost in the ether


with Flow() as Flow:
    run_example() # task with a bunch of logic i've lost observability on
here's a thread on what im talking about https://linen.prefect.io/t/2435019/that-makes-sense-and-because-of-how-coupled-prefect-is-to-da
z

Zach Schuster

11/02/2022, 12:33 AM
@Jon Was trying to do this same thing with no luck. Did you end up going with a functional approach?
j

Jon

11/02/2022, 12:58 PM
@Zach Schuster yes, ended up mostly with the functional approach. we had a lot of class methods that really were just lone functions wrapped in a class namespace. i broke those out into tasks. but we also have some truly class-based stuff, like playwright or selenium, where we wrapped the entire scrape operation under one task. in our case, a headless browser can't be serialized so even if we wanted to force our scrapers to a functional approach, we couldn't. which tbh is sad, bc i want to leverage the task decorator for stuff like retries
z

Zach Schuster

11/02/2022, 1:20 PM
Yeah absolutely. Thanks for the information
✅ 1
4 Views
#best-practices-coordination-planeJoin Slack
Powered by