• Madison Schott

    Madison Schott

    6 months ago
    Hi all, how do I upgrade to the latest version of dbt within my Prefect flow? I am running it on a Docker container
    Madison Schott
    Kevin Kho
    11 replies
    Copy to Clipboard
  • Myles Steinhauser

    Myles Steinhauser

    6 months ago
    I’m trying to follow the Scheduling a Flow-of-Flows example in the docs, but I keep running into the following error:
    Error during execution of task: ValueError('Received both `flow_id` and `flow_name`. Only one flow identifier can be passed.')
    More details in thread.
    Myles Steinhauser
    Kevin Kho
    33 replies
    Copy to Clipboard
  • m

    Milton

    6 months ago
    Hi there, we are running the prefect agents in kubernetes as per https://docs.prefect.io/orchestration/agents/kubernetes.html#running-in-cluster. We notice if we don’t specify the agent name with
    --name
    , it will assume the default name
    Kubernetes
    . This is okay when you only run one replica, but when you increase the replica number to 2, both will take the default name and in the Prefect UI, the two agents are treated as the same agent. So what is the recommended way to deploy multiple agents for HA purposes in Kubernetes?
    m
    Kevin Kho
    8 replies
    Copy to Clipboard
  • Wei Mei

    Wei Mei

    6 months ago
    Hilo, hopefuly simply question. When I write my code password=Secret(“PREFECT_PW”).get(), how do I test this when i run it locally?
    Wei Mei
    Kevin Kho
    3 replies
    Copy to Clipboard
  • Henry

    Henry

    6 months ago
    Hello. We upgrade from 0.15.14 to 1.1.0 and it seems like our flows are still running but nothign shows up on the dashboard unless we pick a project
    Henry
    Kevin Kho
    +1
    14 replies
    Copy to Clipboard
  • Patrick Tan

    Patrick Tan

    6 months ago
    Hi, regarding prefect logger, is there any way to store them to external storage like my own database, in additions to Prefect Cloud? Eg [2022-03-25 15:13:44-0400] INFO - prefect.TaskRunner | Task 'get_config['textlist_id']': Finished task run for task with final state: 'Success' [2022-03-25 15:13:44-0400] INFO - prefect.TaskRunner | Task 'query_from_opensearch': Starting task run... [2022-03-25 15:13:44-0400] INFO - prefect.TaskRunner | Task 'query_from_opensearch': Finished task run for task with final state: 'Success' [2022-03-25 15:13:44-0400] INFO - prefect.TaskRunner | Task 'from_s3': Starting task run... [2022-03-25 15:13:45-0400] INFO - prefect | Read from S3: 3 [2022-03-25 15:13:45-0400] INFO - prefect.TaskRunner | Task 'from_s3': Finished task run for task with final state: 'Success' [2022-03-25 15:13:45-0400] INFO - prefect.TaskRunner | Task 'validate': Starting task run...
    Patrick Tan
    Kevin Kho
    4 replies
    Copy to Clipboard
  • Danny Vilela

    Danny Vilela

    6 months ago
    Hey all! Simple question: if I raise an error within a task (i.e.,
    ValueError
    ) is it possible to recover that traceback/error from a state handler? Right now I can say “this task failed and will be retrying in X minutes”, but not “this task failed for reason Y and will retry in X minutes”. Or can we pass keyword arguments to a state handler with this signature:
    # What I have now.
    def notify_on_retry(task: Task, old_state: State, new_state: Retrying) -> State: ...
    
    # Maybe what I want?
    def notify_on_retry(task: Task, old_state: State, new_state: Retrying, message: str) -> State: ...
    
    # Alternative?
    def notify_on_retry(task: Task, old_state: State, new_state: Retrying, **kwargs) -> State: ...
    Danny Vilela
    Kevin Kho
    6 replies
    Copy to Clipboard
  • Harry Baker

    Harry Baker

    6 months ago
    is it possible to create a helper function/task that implements both create_flow_run and wait_for_flow_run? I tried:
    @task()
    def flow_run_helper(flow_name, project_name):
        cfr = create_flow_run(flow_name=flow_name, project_name=project_name)
        wfr = wait_for_flow_run(cfr, stream_logs=True, raise_final_state=True)
        return wfr
    but its yelling at me about "ValueError: Could not infer an active Flow context while creating edge". my app does a lot of chaining of flows, so i wanted to streamline this
    Harry Baker
    Anna Geller
    +1
    9 replies
    Copy to Clipboard
  • l

    Lee Briggs

    6 months ago
    still wrapping my head around some concepts here, so bear with me. I've got a a working prototype up and running which uses a flask webapp to dispatch flows to prefect cloud. Currently, I'm seeding the data (ie the tasks and flows to run) with a "bootstrap" script that needs to run before the webapp is launched, but I'd like it really to be a "batteries included" experience. How are people generally solving this? I could have a "setup" page in my flask app, but I don't want to get super involved for now
    l
    Kevin Kho
    6 replies
    Copy to Clipboard
  • Dominic Pham

    Dominic Pham

    6 months ago
    Hi all, when I create a connection to a db using prefect.task.sql_server and I create a global temp table, will the temp table persist even when the flow is done? Or will I have to explicitly close the connection as a final task in the flow?
    Dominic Pham
    Kevin Kho
    +1
    7 replies
    Copy to Clipboard