• p

    Prasanth Kothuri

    9 months ago
    can I enforce order of the tasks in the flow ? today I noticed that some tasks were scheduled before even though they are right at the end in my code
    p
    Anna Geller
    2 replies
    Copy to Clipboard
  • j

    Jawaad Mahmood

    9 months ago
    Hello! I'm stuck on something that should be simple, but can't find answer in channel history or docs! I am using prefect.storage.Docker for flow storage inside Docker image. I would like to push this image up to Dockerhub. When I try that using the code below, I get "InterruptedError: unauthorized: authentication required" error. I am able to successfully login to Dockerhub using Docker SDK, but not sure how to pass this authenticated client to prefect.storage.Docker. And I am not sure how I authenticate within prefect.storage.Docker. As suggested by the docs, I am running the Docker daemon locally, and logged into docker in the terminal both of which are configured to push up to the repository. How can I fix? Thanks!
    from prefect import task, Flow
    from prefect.executors import LocalExecutor
    from prefect.run_configs import DockerRun
    from prefect.storage import Docker
    import docker
    
    with Flow("some_flow") as flow:
        do_something
    
        docker_client = docker.DockerClient()
        docker_client.login(username=<env_user>,password=<env_pass>)
    
        flow.storage = Docker(
            registry_url='<http://registry.hub.docker.com/repository/docker/<user>/<repo>|registry.hub.docker.com/repository/docker/<user>/<repo>>'
            ,image_name='<some_flow>'
            ,files={
                <origin path>:<dest path>
            }
            ,python_dependencies = ['pandas','numpy','prefect']
            ,env_vars={
                "PYTHONPATH": "$PYTHONPATH:assets/:root/:data/:image"
            }
            ,base_image='python:3.7.3'
        )
    
        flow.run_config = DockerRun(labels=['my-label']
                                    )
        flow.executor = LocalExecutor()
        flow.register(project_name="some_project")
    j
    Anna Geller
    5 replies
    Copy to Clipboard
  • Kyrylo Zaitsev

    Kyrylo Zaitsev

    9 months ago
    Hi. I'd like to embed a markdown artifact in my pipeline, but I'm facing the issue with the artifact size limit:
    requests.exceptions.HTTPError: 413 Client Error: Payload Too Large for url: <http://0.0.0.0:4200/>
    This markdown I obtained by converting an HTML page in .md format. I deployed prefect using docker-compose, is there any way to increase GraphQL payload size limit?
    Kyrylo Zaitsev
    Anna Geller
    2 replies
    Copy to Clipboard
  • Suresh R

    Suresh R

    9 months ago
    Hi, I'd like to know how we can manage authentication and role based access in prefect server deployment.
    Suresh R
    1 replies
    Copy to Clipboard
  • p

    Prasanth Kothuri

    9 months ago
    We run flows every 30 minutes which means we accumulate lot of flow runs , can I specify retention on flow runs ? if not where is this data stored so that I can setup a script to do periodic cleanups
    p
    Anna Geller
    4 replies
    Copy to Clipboard
  • Raúl Mansilla

    Raúl Mansilla

    9 months ago
    Hello all, I´m facing an issue that I´m not able to fix easily…RuntimeError: Unable to find any timezone configuration
    Raúl Mansilla
    Anna Geller
    +1
    17 replies
    Copy to Clipboard
  • Alexis Lucido

    Alexis Lucido

    9 months ago
    Hello everyone. I am still exploring Prefect's functionnalities and would like to use State Handlers to send alerts in case of failure. I was wondering whether we could pass some more arguments such as email receivers, title and body to a State Handler signature? I would like to use some yml file already configured rather than configuring Prefect Secrets. Thanks a lot, and happy holidays!
    Alexis Lucido
    Kevin Kho
    2 replies
    Copy to Clipboard
  • dev

    dev

    9 months ago
    Hello everyone. I am using prefect server and local agent to setup. The prefect agent keeps throwing 
    ERROR - agent | Failed to query for ready flow runs
    , even though I am able to run the flows.
    dev
    Anna Geller
    5 replies
    Copy to Clipboard
  • Alexis Lucido

    Alexis Lucido

    8 months ago
    Hi all, and all the best for the year to come. My Prefect processes running locally generated too many logs, filling up the memory of my on-premise virtual machine. Our Ops team increased the size of the VM memory and I have disabled checkpointing, however many late runs are registered (several thousands). I want to delete these late runs, but I guess I need to launch our agent (a local-type one) that tries to catch up with all the late runs I am trying to delete through the UI. Thus, the process of deleting the late runs is very slow, i.e. it deletes a few of them per minute. I add that we persist the backend database to keep an audit trace, or at least the best one we could have. Is it possible to delete the late runs without lauching the agent / trying to catch up with the old ones? Thank you very much in advance. Alexis
    Alexis Lucido
    Kevin Kho
    14 replies
    Copy to Clipboard
  • y

    Yash

    8 months ago
    Hi everyone, is there any way to get flow run name inside flow while executing or any way to pass flow run name as a parameter for scheduled flow?
    y
    Kevin Kho
    +1
    8 replies
    Copy to Clipboard