<@ULVA73B9P> my local prefect runs are not showing...
# ask-marvin
s
@Marvin my local prefect runs are not showing changes from local code, they are doing git clone this in my deploy.py how can i change the source to take from local file deployment = await flow.from_source( source=GitRepository( url="https://github.com/calmtales/workflows.git", credentials=github_credentials, ), entrypoint="generate_image_dataset.py:generate_image_dataset_pipeline", ).deploy( name="generate-image-dataset", _# work_pool_name="dataset-pool",_ work_pool_name="local-pool", tags=["image-generation", "gemini"],
m
thought for 8.5 seconds
Ah, for local development, you're encountering one of those delightfully frustrating moments where your code keeps pulling from GitHub instead of using your local changes. Let me help you fix that. To use local files instead of Git, you'll want to modify your deployment code to use
path
instead of
from_source
. Here's how to change your code:
Copy code
python
deployment = await flow.deploy(
    name="generate-image-dataset",
    work_pool_name="local-pool",
    tags=["image-generation", "gemini"],
)
That's it - much simpler! When you don't specify a source, Prefect automatically uses your local files. A few important notes: 1. Make sure your
generate_image_dataset.py
file is in your current working directory, or use a relative/absolute path to specify its location 2. The local work pool will automatically find and use your local Python environment 3. Any changes you make to your local code will be reflected immediately in subsequent flow runs If you later want to switch back to deploying from GitHub, you can simply revert to your original
from_source
code. For more complex local development scenarios, you might also want to look at the
prefect.deployments.runner
module which provides additional options for local development workflows. Let me know if you need any clarification or run into other issues!
s
@Marvin I also have a prefect.yaml can you tell me how i should defien both prefect.yaml and deploy.py to ensure correct local source deployment = await deploy( name="image-customization", _# work_pool_name="running-hub-worker",_ work_pool_name="local-pool", tags=["image-customization", "gcp"],) - name: image-customization version: tags: [] concurrency_limit: description: image customization entrypoint: image_customization.py:image_customization parameters: {} work_pool: name: local-pool work_queue_name: job_variables: {} schedules: []| 2. In what way do both files work when i run or deploy something
m
thought for 11.2 seconds
Let me explain how
prefect.yaml
and
deploy.py
work together and how to configure them properly for local development. Here's how to structure both files: 1. deploy.py:
Copy code
python
from prefect import flow

# Import your flow directly
from image_customization import image_customization

deployment = await image_customization.deploy(
    name="image-customization",
    work_pool_name="local-pool",
    tags=["image-customization", "gcp"],
)
2. prefect.yaml:
Copy code
yaml
name: image-customization
entrypoint: image_customization.py:image_customization  # relative path from project root
work_pool:
  name: local-pool
tags: ["image-customization", "gcp"]
Now, here's how these files work together: 1. Deployment Methods: - `prefect deploy`: Uses the
prefect.yaml
configuration - Running `deploy.py`: Uses the Python deployment configuration - Both methods create the same deployment, just through different interfaces 2. When You Run Things: - `prefect deployment run image-customization/image-customization`: Runs the deployment - Local work pool will: - Use your local Python environment - Use files from your local filesystem - Respect any changes you make to your local code immediately 3. Best Practices: - Keep both files in sync (same name, tags, work pool) - Store them in your project root directory - Make sure your entrypoint file (
image_customization.py
) is relative to where you run the deployment from 4. Directory Structure Example:
Copy code
your_project/
├── prefect.yaml
├── deploy.py
├── image_customization.py
└── other_project_files/
You can deploy using either:
Copy code
bash
# Using prefect.yaml
prefect deploy

# Using deploy.py
python deploy.py
Both methods will create a deployment that runs your local code. The main difference is that
deploy.py
gives you more programmatic control, while
prefect.yaml
provides a declarative configuration that's easier to version control and share. Remember: When using the local work pool, Prefect will always use your local files, so any changes you make will be picked up in the next run without needing to redeploy. Let me know if you need any clarification on how these pieces fit together!