• p

    Philip MacMenamin

    5 months ago
    Quietening down logs - if I'm mapping out to a lot of job, have a lot of params etc, it can be difficult to follow logs. Is there a way I can set logging such that I only see something if it's broken. If I
    export PREFECT__LOGGING__LEVEL=INFO
    I see:
    python3 tmp/shell_task.py 
    [2022-04-19 21:09:30+0100] INFO - prefect.FlowRunner | Beginning Flow run for 'My Flow'
    [2022-04-19 21:09:30+0100] INFO - prefect.TaskRunner | Task 'MyShellTask': Starting task run...
    [2022-04-19 21:09:30+0100] INFO - prefect.MyShellTask | lsto echo
    [2022-04-19 21:09:30+0100] INFO - prefect.TaskRunner | Task 'MyShellTask': Finished task run for task with final state: 'Success'
    [2022-04-19 21:09:30+0100] INFO - prefect.TaskRunner | Task 'problem': Starting task run...
    [2022-04-19 21:09:30+0100] INFO - prefect.TaskRunner | FAIL signal raised: FAIL('Oh no!')
    [2022-04-19 21:09:30+0100] INFO - prefect.TaskRunner | Task 'problem': Finished task run for task with final state: 'Failed'
    [2022-04-19 21:09:30+0100] INFO - prefect.FlowRunner | Flow run FAILED: some reference tasks failed.
    If I
    export PREFECT__LOGGING__LEVEL=ERROR
    I see nothing. Ideally I'd like to only see messages about broken stuff. Ideas?
    p
    Nate
    +1
    24 replies
    Copy to Clipboard
  • j

    Jason

    5 months ago
    What's a good starting area for the local dev story? We're deploying to a production ECS cluster with a decent Github CI/CD so far. Ideally a simple
    Makefile
    would spin up a few Docker containers that one could test flows on. How can I abstract between a local and ECS agent in my flows to allow something like a
    env
    var to swap between?
    j
    Kevin Kho
    +1
    4 replies
    Copy to Clipboard
  • j

    Jai P

    5 months ago
    are there any best practices around testing flows/tasks in prefect 2.0? I see this page but something we're noticing that testing can be particularly slow on flows, (sometimes taking ~1s to start up a each test) and it appears we always need to wrap tasks inside of a flow to test them
    j
    Michael Adkins
    +3
    24 replies
    Copy to Clipboard
  • Josh

    Josh

    5 months ago
    Any idea what might be causing errors with artifact generation? I’m getting these errors in my task runs:
    Error during execution of task: ClientError([{'path': ['create_task_run_artifact'], 'message': 'Task run <task_run_id> not found', 'extensions': {'code': 'INTERNAL_SERVER_ERROR'}}])
    Josh
    Kevin Kho
    15 replies
    Copy to Clipboard
  • a

    Ahmed Ezzat

    5 months ago
    Cloud anyone from the devs take a look at this?https://github.com/PrefectHQ/prefect/discussions/5440
    a
    1 replies
    Copy to Clipboard
  • Chris Reuter

    Chris Reuter

    5 months ago
    Hey all! Come hang with @Kevin Kho and myself on PrefectLive 📺 today at 3p Eastern live on Twitch. He'll be showing off Modin, PySpark, Pandas and Fugue. We'll also probably have an invigorating ⌨️ mechanical keyboard debate and would love for you to join us! For event updates and reminders make sure to join the #events channel.
  • j

    Jason

    5 months ago
    I'm getting some weird behavior regarding modules. I can correctly import my flow and the Config class from
    projects
    from both Python and Jupyter Notebook:
    from projects.example.flows.hello_world
    from projects import Config
    But when I run the same import from Prefect, I get the following error:
    pipenv run prefect run -m projects.example.flows.hello_world                    
    No module named 'projects'
    j
    Anna Geller
    +1
    5 replies
    Copy to Clipboard
  • Constantino Schillebeeckx

    Constantino Schillebeeckx

    5 months ago
    I have a flow that runs 121 mapped tasks - two of those tasks failed however the overall flow was nor marked as failed
    Constantino Schillebeeckx
    Kevin Kho
    19 replies
    Copy to Clipboard