@Marvin, when deploying a flow via code, how can I tell it that the code lives in a local folder, not in a docker image or remote location?

To deploy a flow from a local folder: 1. Code Example:

python
from prefect import flow

@flow(log_prints=True)
def my_workflow():
    print("I can call any Python code here, including Prefect tasks or flows")

if __name__ == "__main__":
    my_workflow()  # Call it like a normal Python function
    # Optionally serve it as a long-lived process
    # my_workflow.serve("my-deployment-name")

(Code example source: https://raw.githubusercontent.com/zzstoatzz/prefect-code-examples/main/examples/flows/basic.py) 2. Usage Documentation: In the Prefect deployment documentation, you can set up and deploy your flows while specifying code storage locations. Learn more from Deployment Guide and Storage Guide. If no storage is explicitly configured, Prefect uses LocalFileSystem storage by default. For more details, check out Prefect's Concepts of Storage.

@Marvin my code already exists on the server and doesn't need to be pulled from a repo or remote location, how can I run the 'deploy' command and tell it to use this local code?

To deploy a flow using local code already on the server, you can follow these steps: 1. Ensure the flow is defined in your Python script. 2. Use LocalFileSystem for flow storage: By default, Prefect will use

LocalFileSystem

storage if no other storage is explicitly configured. Here's a simple code example of how to create and deploy a flow that assumes your code is already on the server:

python
from prefect import flow, deployment
from prefect.filesystems import LocalFileSystem

local_file_system = LocalFileSystem(basepath="/path/to/your/flow/code")

@flow
def my_local_flow():
    print("Executing flow from local file system")

if __name__ == "__main__":
    deployment.deploy(
        flow=my_local_flow,
        name="local-deployment",
        storage=local_file_system
    )

### Steps: 1. Replace

"/path/to/your/flow/code"

with the actual path where your flow code resides. 2. Run this script on your server to deploy the flow using the local file system. ### Documentation: You can read more about this setup in the Prefect documentation: - Flow code storage options - Quickstart By following these instructions, you can ensure the Prefect deployment uses the local code stored on your server.

@Marvin can I use the 'prefect.deploy' function to deploy a local flow?

Yes, you can use the

prefect.deploy

function to deploy a local flow. Below is an example extracted from the documentation which demonstrates deploying both a local flow and a flow from a remote repository:

python
from prefect import deploy, flow

@flow(log_prints=True)
def local_flow():
    print("I'm a flow!")

if __name__ == "__main__":
    deploy(
        local_flow.to_deployment(name="example-deploy-local-flow"),
        # Example of a remote flow deployment for comparison
        # flow.from_source(
        #     source="<https://github.com/org/repo.git>",
        #     entrypoint="flows.py:my_flow",
        # ).to_deployment(
        #     name="example-deploy-remote-flow",
        # ),
        work_pool_name="my-work-pool",
        image="my-registry/my-image:dev",
        push=False,  # Indicates that the code should not be pushed to any repository or remote storage
    )

### Key Points: 1. Local Flow: The

local_flow.to_deployment(name="example-deploy-local-flow")

line is where the local flow is specified for deployment. 2. work_pool_name: Indicates the work pool with which the deployment should be associated. 3. image: The Docker image to be used (if required). 4. push=False: Ensures that the code is not pushed to a remote repository or storage, keeping it local. ### Documentation: Learn more from Prefect's Deployment Guide. By following this example, you can ensure that your deployment uses the code that is already on the server without needing to pull it from a remote location.

@Marvin, great, that's getting closer. How can I call this without using a Docker image?

To deploy a local flow without using a Docker image, you can leverage Prefect's local storage capabilities and ensure the flow is deployed directly. Here's an example:

python
from prefect import flow, deploy
from prefect.filesystems import LocalFileSystem

# Define the flow
@flow(log_prints=True)
def local_flow():
    print("Executing flow from local file system")

# Define the storage location
local_file_system = LocalFileSystem(basepath="/path/to/your/flow/code")

if __name__ == "__main__":
    # Deploy the flow using the LocalFileSystem
    deploy(
        flow=local_flow,
        name="local-deployment",
        storage=local_file_system,
        work_pool_name="my-work-pool",
    )

### Steps: 1. Replace

"/path/to/your/flow/code"

with the actual path where your flow code resides. 2. Run this script on your server to deploy the flow using the local file system without Docker. ### Example Source: Here is a simpler flow example without deployment; to illustrate the basic setup: Basic Flow Example. This approach ensures your Prefect deployment uses the code stored locally on your server without involving Docker.

hey @skaaks https://docs-3.prefect.io/3.0rc/resources/upgrade-agents-to-workers#deploying-from-a-local-file

@Nate, thanks for the heads up! For context, I'm trying to replicate the following prefect.yaml using a script to perform the deployment:

# build section allows you to manage and build docker images
build:

# push section allows you to manage if and how this project is uploaded to remote locations
push:

# pull section allows you to provide instructions for cloning this project in remote locations
pull:

# the deployments section allows you to provide configuration for deploying flows
deployments:
- name: nba_api_pipeline
  version:
  tags: []
  description:
  entrypoint: flows/nba_api/nba_api_pipeline.py:nba_api_pipeline
  parameters: {
    process_id
  }
  work_pool:
    name: default_work_pool
    work_queue_name: nba_api_pipeline
    job_variables: {}
  schedules: []

real quick before I go on about

prefect.yaml

- have you explored

serve

and know that you don't want to use it? it sort of circumvents this problem

Interesting. I haven't explored it a great deal, only when testing. Would that mean that the script I'm running the 'serve' from would act as a service? And by doing so, would those flows being served be able to be scheduled via the API, similar to how a standard deployment is?

yes to both https://docs-3.prefect.io/3.0rc/resources/daemonize-processes#setting-up-a-systemd-service-for-serve

Ok great, I'll look into that option a bit more. Thank you!

on work pools and workers, if you end up going that route you'll need to make sure that your worker has the ability to access your code somehow, whether that's: • cloning the repo it lives in to the runtime env • changing directory to the place on disk where it lives (if the worker is on the same machine) and this stuff would happen in the

pull

step of your deployment. so totally possible, but with

serve

you only have to run the file where it lives anyways, so there's no ambiguity / decoupling to worry about (which to be clear, lots of times work pools / workers are necessary and nice, but not always)

Ok, I've set up a deployment to be served via a script file, and am getting an error about it not being able to find the flow code for the deployment. Here's a sample of the serve code that's running in the script:

from flows.nba_api import nba_api_status_log_pipeline

nba_api_status_log_pipeline.nba_api_status_log_pipeline.serve(
            name="nba_api_status_log_pipeline",
            cron="*/5 * * * *"
        )

hrm thats odd. does it make a difference if you import the flow object

nba_api_status_log_pipeline

directly instead of the module it lives in?

So I'm understanding correctly, you're saying because the log says 'Worker 'Prefect_Server_Worker' submitting flow run...', that means it was a worker that submitted the flow, not the 'serve' app?

interesting! thanks for the detail there might be something in the handling here that we can improve to avoid that conflict as far as where the server thinks the code lives but glad you got it working!

Once last quick note on this, I've not been able to run my flows locally using this 'serve' method (although they've been working for me in other environments), and I'm guessing it has to do with the file path of the flow code location. On my local box, the code lives in the 'C:/Code/CEP Database/flows' folder (with a 'space' character in the path), and the error message I'm seeing when attempting to run one of those deployments is:

17:41:10.506 | INFO    | prefect.flow_runs.runner - Runner 'nba_ngss_launcher_service' submitting flow run '27b8b291-887d-470a-95f6-c0b5b81ce7bf'
17:41:10.547 | INFO    | prefect.flow_runs.runner - Opening process...
17:41:10.569 | INFO    | prefect.flow_runs.runner - Completed submission of flow run '27b8b291-887d-470a-95f6-c0b5b81ce7bf'
'c:\Code\CEP' is not recognized as an internal or external command,
operable program or batch file.
17:41:10.596 | ERROR   | prefect.flow_runs.runner - Process for flow run 'nice-gazelle' exited with status code: 1
17:41:10.633 | INFO    | prefect.flow_runs.runner - Reported flow run '27b8b291-887d-470a-95f6-c0b5b81ce7bf' as crashed: Flow run process exited with non-zero status code 1.

To me, it looks as though it's having an issue with the space in the full flow file path and not using the full path to locate the file

hmm yeah it might have to do with the space - windows paths have been edge cases in the past. i can try to reproduce this later