• r

    Richard Hughes

    2 years ago
    I have a flow that appears to be running and then it fails with this message, however, I don't think it is failing.
    No heartbeat detected from the remote task; marking the run as failed.
    r
    Dylan
    16 replies
    Copy to Clipboard
  • m

    Marwan Sarieddine

    2 years ago
    Hi folks, I think I am facing a prefect bug when I enable
    PREFECT_FLOWS_CHECKPOINTING
    and I use a
    pd.DataFrame
    as a parameter - I am using prefect
    v0.13.3
    export PREFECT_FLOWS_CHECKPOINTING=true
    
    
    In [1]: import pandas as pd
    In [2]: from prefect import Parameter, task, Flow
    In [3]: @task
       ...: def simple(df):
       ...:     return df
       ...: 
    In [4]: with Flow("test") as flow:
       ...:     df = Parameter(name="df")
       ...:     simple(df)
       ...: flow.run(df=pd.DataFrame())
    [2020-08-20 14:20:38] ERROR - prefect.TaskRunner | Unexpected error: TypeError('Object of type DataFrame is not JSON serializable')
    Traceback (most recent call last):
      File "~/.pyenv/versions/3.7.7/envs/infima/lib/python3.7/site-packages/prefect/engine/runner.py", line 48, in inner
        new_state = method(self, state, *args, **kwargs)
      File "~/.pyenv/versions/3.7.7/envs/infima/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 838, in get_task_run_state
        result = self.result.write(value, **formatting_kwargs,)
      File "~/.pyenv/versions/3.7.7/envs/infima/lib/python3.7/site-packages/prefect/engine/results/prefect_result.py", line 60, in write
        new.location = self.serializer.serialize(new.value).decode("utf-8")
      File "~/.pyenv/versions/3.7.7/envs/infima/lib/python3.7/site-packages/prefect/engine/serializers.py", line 100, in serialize
        return json.dumps(value).encode()
      File "~/.pyenv/versions/3.7.7/lib/python3.7/json/__init__.py", line 231, in dumps
        return _default_encoder.encode(obj)
      File "~/.pyenv/versions/3.7.7/lib/python3.7/json/encoder.py", line 199, in encode
        chunks = self.iterencode(o, _one_shot=True)
      File "~/.pyenv/versions/3.7.7/lib/python3.7/json/encoder.py", line 257, in iterencode
        return _iterencode(o, 0)
      File "~/.pyenv/versions/3.7.7/lib/python3.7/json/encoder.py", line 179, in default
        raise TypeError(f'Object of type {o.__class__.__name__} '
    TypeError: Object of type DataFrame is not JSON serializable
    Out[4]: <Failed: "Some reference tasks failed.">
    when I disable
    PREFECT_FLOWS_CHECKPOINTING
    flow runs successfully
    export PREFECT_FLOWS_CHECKPOINTING=false
    
    In [1]: import pandas as pd
    In [2]: from prefect import Parameter, task, Flow
    In [3]: @task
       ...: def simple(df):
       ...:     return df
       ...: 
    In [4]: with Flow("test") as flow:
       ...:     df = Parameter(name="df")
       ...:     simple(df)
       ...: flow.run(df=pd.DataFrame())
    Out[4]: <Success: "All reference tasks succeeded.">
    m
    Dylan
    13 replies
    Copy to Clipboard
  • William Smith

    William Smith

    2 years ago
    Hi all, are there any examples where a task can be paused awaiting some sort of manual approval?
    William Smith
    Dylan
    65 replies
    Copy to Clipboard
  • William Smith

    William Smith

    2 years ago
    Using prefect cloud, have a very simply flow running. Sometimes my flows just don't run and are in a scheduled state, the agent is running it is waiting for flow runs. It seems to work fine the first couple times, any ideas?
    William Smith
    Dylan
    10 replies
    Copy to Clipboard
  • b

    Brian Mesick

    2 years ago
    Follow ups from yesterday’s question, since I still can’t seem to make it work. • Is there anything special needed to make a
    PrefectResult
    work for a task, presuming I’m using Docker storage for the flow? • Does adding a
    PrefectResult
    to a task somehow transparently change the labels of a Flow?
    b
    1 replies
    Copy to Clipboard
  • Adam

    Adam

    2 years ago
    Hi everyone, hope your Thursday is going well. I have a flow that stuck on in a “Scheduled” state on Prefect Cloud. There are no other flows scheduled or running currently and the agent is alive and logging “Waiting for flow runs”. How can I debug why this flow isn’t being submitted? The flow run id is 80230641-deec-466f-b12c-cea1ef8704ae
    Adam
    Dylan
    6 replies
    Copy to Clipboard
  • Ruben Silva

    Ruben Silva

    2 years ago
    Hello 😃 I'm wondering if someone can explain to me how does prefect core execute mapping with the DaskExecuter. In the Docs, it's said that Prefect automatically creates a copy of the task for each element of its input data. However, looking at the code I'm not understanding where that is being done, and when those new tasks are being submitted to Dask.
    Ruben Silva
    Dylan
    2 replies
    Copy to Clipboard
  • Jimmy Le

    Jimmy Le

    2 years ago
    Hey folks! I'm working on a personal project and I'm hoping to use Prefect to automate some of the workflows 🙂
    Jimmy Le
    Jeremiah
    2 replies
    Copy to Clipboard
  • e

    Eric

    2 years ago
    Hi all, when I registered a flow, the flow-run will be created automatically depends on the given schedule with python code. I wonder are there any ways to mark a flow-run "skipped" when the task found the connection to RDB is temporary dropped(in our use case, we need "skipped" instead of "failed"). That is, I want to change the flow-run mark in Task layer. Is that possible? I've found a way is "client.set_flow_run_state(flow_run_id="ce7ce40b-a21b-4273-b39f-6ebea071a76e", state=skip_state)", but the flow-run is created by scheduler, I could not find the flow-run id automatically. Thanks!!
    e
    nicholas
    12 replies
    Copy to Clipboard
  • h

    Howard Cornwell

    2 years ago
    Hi all. Is it possible to migrate from 0.11.5 to 0.13.3? I get the following error, so I assume it’s not possible?
    graphql_1   |                                                                                  
    graphql_1   | Running Alembic migrations...                                                    
    graphql_1   | INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
    graphql_1   | INFO  [alembic.runtime.migration] Will assume transactional DDL.
    graphql_1   |                                                                                  
    graphql_1   | Could not upgrade the database!
    graphql_1   | Error: Can't locate revision identified by '7e65dadba625'
    h
    nicholas
    +1
    7 replies
    Copy to Clipboard