• Mathijs Miermans

    Mathijs Miermans

    8 months ago
    Can the ECS Agent execute flows from a custom Docker image that we build? Our goal is to have our production and local development environments be as similar as possible. Locally we run flows in a custom Docker image with all of our Prefect code, that's based on the Prefect image. Currently we use S3 storage for production, and while we haven't run into any significant issues with that, we're prefer to make the environments more similar. I couldn't find the answer in the docs: • LocalStorage docs say it's only compatible with the Local Agent, so not with our ECS Agent. • Docker storage seems the closest to what we'd want, but I understand it would build a separate Docker image for each flow, instead of using a single Docker image.
    Mathijs Miermans
    Kevin Kho
    21 replies
    Copy to Clipboard
  • Suresh R

    Suresh R

    8 months ago
    Hi! We want to use state handler to create an OpsGenie alarm when the flow fails and resolve the same alarm when the flow succeeds after restart. is there a way to store the alert id somewhere so that it can be used for closing the alert later.
    Suresh R
    Anna Geller
    2 replies
    Copy to Clipboard
  • k

    Kamil Gorszczyk

    8 months ago
    Hello everyone! I'm currently upgrading to the newest prefect version (from 0.15.3 - server 2021.04.06) and while re-registering my flows I'm getting a GraphQL Error: validation failed - expected one of the values edge_pkey for type „edge_constraint“, but found „“edge_flow_id_task_ids_key“. It only appears when I use a Parameter and use the returned value in a task. Registering a simple flow or even one of my old flows with the parameters disabled works just fine.
    k
    Anna Geller
    4 replies
    Copy to Clipboard
  • s

    shijas km

    8 months ago
    hi I have a question, I am new to prefect , I have configured prefect cloud to run the flow, the flow is created in an aws ec2 and I run the job and scheduled it , now its running on cloud ui my question is the actual execution happens in my prefect cloud or in my aws ec2 ? is there any way i can run my prefect flow with docker in prefect cloud ?
    s
    Anna Geller
    4 replies
    Copy to Clipboard
  • g

    Guilhelm PANAGET

    8 months ago
    Hi everyone, I'm running a Prefect server and when we run some tasks generating a lot of logs (like 600 lines in a second), the UI stops to show them at some moment and there are no logs for the following tasks : they are not delayed, they just never appear (the prefect cli doesn't show as well). Is it a known issue ? Has someone run into the same kind of problem here ? Thank you !
    g
    Anna Geller
    2 replies
    Copy to Clipboard
  • k

    Kirk Quinbar

    8 months ago
    I have created an Azure VM with Docker and Prefect Agent using the marketplace template "Prefect Agents". My flows and dependencies are stored in an Azure Container Registry. I used Cloud Prefect for my UI and got everything running as expected. I wanted to verify what happens with the authentication from the VM to the Container Registry if the VM ever has to restart. Once i restarted the VM, I noticed in the Cloud Prefect UI that the agent stopped polling. I am assuming that the VM restart, did not restart the Prefect Agent on the VM. Has anyone experienced this specifically when creating a VM with the marketplace template? Maybe this is a bug in the marketplace template that doesnt account for VM restarts. Under normal circumstances i would think there would almost never be a restart, but i know Azure isnt perfect, plus if there are updates that need to get done to the VM.
    k
    Anna Geller
    5 replies
    Copy to Clipboard
  • Philipp Eisen

    Philipp Eisen

    8 months ago
    Hey! What would you say is the best way of running prefect in production with minimal flow / task startup time?
    Philipp Eisen
    Anna Geller
    3 replies
    Copy to Clipboard
  • a

    Alexander Kloumann

    8 months ago
    Just arrived! First question here. I am trying to figure out Prefect logging using the documentation here:https://docs.prefect.io/core/concepts/logging.html#logging-from-tasks But when I add this to one of my tasks and run it I see no log file:
    logger = prefect.context.get("logger")
    <http://logger.info|logger.info>("An info message.")
    I tried running it from the interface on Prefect Cloud but that doesn't do anything either. What am I missing here?
    a
    Kevin Kho
    +1
    9 replies
    Copy to Clipboard
  • Stephen Herron

    Stephen Herron

    8 months ago
    quick question - i set-up my ecs agent with what I think is a custom image - say prefect with pandas. I then try to run a flow from cloud, simple hello-world with
    import pandas
    This doesn’t seem to work unless I specifically supply the custom image (and the task execution arn). I would have expected if run_config:image is null it would use the one from the task_definition? Does it default to something other than the container def?
    Stephen Herron
    Kevin Kho
    7 replies
    Copy to Clipboard