• David Yang

    David Yang

    5 months ago
    Hi, I tried to create an VM from Prefect Agent in Azure Marketplace. The API key generated in Prefect cloud is 40 characters. But the Prefect Agent template requires the key length should be 1-22 characters. Can Prefect team help me to solve this issue?
    David Yang
    1 replies
    Copy to Clipboard
  • Greg Kennedy

    Greg Kennedy

    5 months ago
    Hi all. Some of my flows started failing overnight and after investigation I have found that the EC2 instance that the flows run on has run out of disk space. After a bit of digging I realised that the prefect flow results are filling up the C: drive. What is the best practice for persisting these results? I don't think we really even need them, except maybe in the event of a failure where we might rerun failed jobs.
    Greg Kennedy
    1 replies
    Copy to Clipboard
  • c

    Carlos Cueto

    5 months ago
    Hi. I'm new to Prefect and having some trouble understanding the concept of tasks and task runs. I'm trying to return data from a task, but attempting to print the results of the task only prints out
    <Task: fetch_data>
    . This is what I'm trying to do:
    @task
    def fetch_data():
    return {"data": "random data"}
    with Flow('Get-Data') as flow:
    flow.run_config = LocalRun()
    data = fetch_data()
    <http://logger.info|logger.info>(data)
    flow.run()
    c
    Kevin Kho
    4 replies
    Copy to Clipboard
  • b

    Bihag Kashikar

    5 months ago
    Hi Guys, I was reading the documentation for running prefect agent on gcp vertex ai and noticed the links on the page here are broken -> screenshot of links
    b
    Anna Geller
    2 replies
    Copy to Clipboard
  • l

    Leanna Morinishi

    5 months ago
    Hi! Our team wants to use Parameters with default values to record DRIs for Flows (regardless of who created it or updated it last), so that we can query for it. It seems like I have to use a Parameter in a task for it to show up in Prefect UI? Is there a nicer/better way for me to have a Parameter show up in the UI or be graphQL queryable than to write a dummy task that just prints the Parameter?
    l
    Anna Geller
    +1
    3 replies
    Copy to Clipboard
  • Jacob Blanco

    Jacob Blanco

    5 months ago
    How can I revoke user API Keys?
    Jacob Blanco
    Anna Geller
    15 replies
    Copy to Clipboard
  • Trung Đô Trần

    Trung Đô Trần

    5 months ago
    👋 Hello, team!
    Trung Đô Trần
    Anna Geller
    2 replies
    Copy to Clipboard
  • Olivér Atanaszov

    Olivér Atanaszov

    5 months ago
    Hi, I have a scheduled flow run that had a failed run during the night (probably due to transient network issues). Although Lazarus is enabled for this flow, I did not see any logs reflecting that, see https://docs.prefect.io/orchestration/concepts/services.html#lazarus. Do I miss something?
    Olivér Atanaszov
    Sylvain Hazard
    3 replies
    Copy to Clipboard
  • s

    Stephen Lloyd

    5 months ago
    I have a flow that when I run locally, works fine. Here’s my flow
    with Flow(PREFECT_FLOW_NAME, storage=STORAGE, run_config=RUN_CONFIG) as flow:
        creds = get_credentials()
        conn = get_connection(creds)
        tables = get_tables()
        save_data = load_to_s3.map(tables, conn=unmapped(conn))
        conn.set_upstream(creds)
        save_data.set_upstream(tables)
        save_data.set_upstream(conn)
    It’s failing on
    get_connection
    and the relevant code is:
    @task
    def get_credentials():
        return PrefectSecret(PREFECT_FLOW_NAME).run()
    
    @task
    def get_connection(creds):
        return connectors.get_redshift_connector(creds)
    
    # from another file...
    import redshift_connector
    
    
    def get_redshift_connector(creds: dict) -> object:
        conn = redshift_connector.connect(
            host=creds['host'],
            database=creds['database'],
            user=creds['user'],
            password=creds['password']
        )
        return conn
    When I move to running in ECS, it fails with the following trace…
    Unexpected error: TypeError("cannot pickle 'Struct' object")
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/site-packages/prefect/engine/runner.py", line 48, in inner
        new_state = method(self, state, *args, **kwargs)
      File "/usr/local/lib/python3.8/site-packages/prefect/engine/task_runner.py", line 930, in get_task_run_state
        result = self.result.write(value, **formatting_kwargs)
      File "/usr/local/lib/python3.8/site-packages/prefect/engine/results/s3_result.py", line 71, in write
        binary_data = new.serializer.serialize(new.value)
      File "/usr/local/lib/python3.8/site-packages/prefect/engine/serializers.py", line 73, in serialize
        return cloudpickle.dumps(value)
      File "/usr/local/lib/python3.8/site-packages/cloudpickle/cloudpickle_fast.py", line 73, in dumps
        cp.dump(obj)
      File "/usr/local/lib/python3.8/site-packages/cloudpickle/cloudpickle_fast.py", line 602, in dump
        return Pickler.dump(self, obj)
    TypeError: cannot pickle 'Struct' object
    s
    Anna Geller
    13 replies
    Copy to Clipboard
  • Malthe Karbo

    Malthe Karbo

    5 months ago
    Hi (prefect) - running flows without cloud beta (orion) works fine, but if I login - all works as expected until end of flow, I then receive an error consistenly:
    14:17:55.129 | ERROR   | Flow run 'arrogant-yak' - Crash detected! Execution was interrupted by an unexpected exception.
    followed by
    prefect.exceptions.Abort: This run has already terminated.
    . This happens regardless of task success/failure. Again, running without cloud this works fine
    Malthe Karbo
    Anna Geller
    +1
    13 replies
    Copy to Clipboard