https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> I have the following setup: 1 contain...
# ask-marvin
g
@Marvin I have the following setup: 1 container that runs my prefect server 1 container that runs my lightweight "backend" API which simply uses the prefect python client to run remote deployments and parse results 1 container that is running my prefect flows (in a worker pool) question: where do I need to put these env variables:
Copy code
# PREFECT
PREFECT_LOCAL_API_URL="<http://127.0.0.1:4200/api>"
PREFECT_IBM_API_URL="https://<ibm_stuff>/api"
PREFECT_RESULTS_PERSIST_BY_DEFAULT=true
PREFECT_LOGGING_LEVEL=DEBUG
PREFECT_DEPLOYMENT_NAME="lead-companion-workflows"
Where do they make sense each?
m
✅ thought for 64.0 seconds
Based on your setup and the research, let me break down where each environment variable should be set: 1. Prefect Server Container:
Copy code
PREFECT_LOGGING_LEVEL=DEBUG  # Useful for server-side debugging
2. Backend API Container (using Prefect client):
Copy code
PREFECT_LOCAL_API_URL="<http://127.0.0.1:4200/api>"  # Points to your Prefect server
PREFECT_IBM_API_URL="https://<ibm_stuff>/api"      # If needed for IBM-specific functionality
PREFECT_LOGGING_LEVEL=DEBUG                        # If you want client-side debug logs
3. Worker Container:
Copy code
PREFECT_LOCAL_API_URL="<http://127.0.0.1:4200/api>"  # Points to your Prefect server
PREFECT_RESULTS_PERSIST_BY_DEFAULT=true            # Affects flow run results persistence
PREFECT_LOGGING_LEVEL=DEBUG                        # For worker-side debugging
Important notes: 1. 
PREFECT_LOCAL_API_URL
needs to be adjusted to use the proper container networking hostname instead of 
127.0.0.1
. For example, if using Docker Compose, you would use the service name of your Prefect server container. 2. 
PREFECT_DEPLOYMENT_NAME
doesn't appear to be a standard Prefect environment variable. If this is for your application logic, put it where your application needs it (likely in the Backend API container). 3. For container networking: - Replace 
127.0.0.1
with the appropriate container network hostname - Example: If your Prefect server container is named 
prefect-server
in Docker Compose, other containers might use: 
PREFECT_LOCAL_API_URL="<http://prefect-server:4200/api>"
4. Security considerations: - If you're using authentication, you might need to add 
PREFECT_API_KEY
in both the Backend API and Worker containers - Consider using Docker secrets or Kubernetes secrets for sensitive values Would you like me to clarify any part of this setup or provide more specific information about any of these environment variables?
g
Copy code
# Prefect configuration file
# See <https://docs.prefect.io/concepts/settings/> for all available settings

# Global Prefect settings
prefect-version: 3.0.0+

# API Configuration
api:
  url: "<http://127.0.0.1:4200/api>"

# Results Configuration
results:
  persist_by_default: true

# Default deployment settings
deployments:
  - name: lead-companion-workflows
    # You can add deployment-specific configurations here

# Logging configuration
logging:
  level: INFO

# Flow run configuration
flows:
  # Default retry settings for flows
  default_retries: 1

# Task configuration
tasks:
  # Default retry settings for tasks
  default_retries: 1
@Marvin Is this a valid prefect.yaml config file?
m
✅ thought for 44.8 seconds
The configuration file you've shown isn't quite correct for Prefect 3.x. The format you're showing seems to be mixing concepts from different versions or systems. Let me explain the correct approach: 1. In Prefect 3.x, 
prefect.yaml
is primarily used for deployment configuration, not general Prefect settings. 2. For general Prefect settings, you should use either: - Environment variables (as discussed in the previous question) - The Prefect profiles system (configured via 
prefect profile
) Here's how you should handle each of these settings: For General Settings: Use environment variables or create a profile:
Copy code
bash
prefect profile create local
prefect config set PREFECT_API_URL="<http://127.0.0.1:4200/api>"
prefect config set PREFECT_RESULTS_PERSIST_BY_DEFAULT=true
prefect config set PREFECT_LOGGING_LEVEL="INFO"
For Deployment Configuration: A valid 
prefect.yaml
in Prefect 3.x looks more like this:
Copy code
yaml
name: lead-companion-workflows
version: 0.1.0

deployments:
  - name: default
    entrypoint: path/to/your/flow.py:flow_function
    schedule: null
    parameters: {}
    work_pool:
      name: your-work-pool
      work_queue_name: default

build: null
push: null
pull:
  - prefect.deployments.steps.git_clone
Key differences: 1. The file structure is focused on deployment configuration 2. There's no global 
prefect-version
field 3. Settings like logging level, API URL, and results persistence are handled through environment variables or profiles, not in this file 4. The deployment section is more specific about entrypoints and work pools Would you like me to: 1. Help you set up a proper 
prefect.yaml
for your deployments? 2. Show you how to configure those general settings using the profile system? 3. Explain more about any specific setting you're trying to configure?
Open in Slack
PreviousNext
Powered by