b

    BK Lau

    1 year ago
    Q: if i successfully run a Flow/Task locally and now wanted to run the same flow/task in another remote cluster and I "register" the Flow/task with the Prefect server, its my understanding that only the Flow/Task DAG metadata got serialized to the Prefect Server/Database. My question is how does the full complement of the Flow/Task code and its third-party dependency libraries got deployed or loaded in the remote cluster? Am I missing something here? Who is the deploying the code to the remote cluster??
    nicholas

    nicholas

    1 year ago
    Hi @BK Lau - you’re correct that the Server never received your flow’s code, only the metadata. the Agent you deploy to poll the API is responsible for retrieving and deploying your flow from your configured storage method to your configured execution environment.
    b

    BK Lau

    1 year ago
    @nicholas so that Flow/Task must be commited to a storage like
    github
    and made available to the Agent? Is there an Agent on every compute/worker node or just one Agent?
    nicholas

    nicholas

    1 year ago
    Just 1 agent @BK Lau - you don’t need to use storage so long as the code lives on the same filesystem as the agent.
    (By not using storage your flow defaults to LocalStorage)
    b

    BK Lau

    1 year ago
    @nicholas Does the metadata passed to Prefect Server includes the location, the Agent should get the Flow code from??
    nicholas

    nicholas

    1 year ago
    Yes, that’s correct