Romain Lefebvre
11/19/2025, 2:31 PMasync with get_client() as client:...
But my prefect self hosted server isn't located at localhost. How can I provide the prefect client, the API url and auth string ?
I've tried doing
os.environ["PREFECT_PROFILES_PATH"] = "/dev/null"
# Set the Prefect API URL
os.environ["PREFECT_API_URL"] = PREFECT_API_URL
# Set the Basic Auth string (username:password format)
# Prefect will use this for Basic Authentication
os.environ["PREFECT_API_AUTH_STRING"] = f"{PREFECT_USERNAME}:{PREFECT_PASSWORD}"
But in the end, it seems like it's still trying to contact localhost as defined in my prefect profile :
🚀 you are connected to:
<http://127.0.0.1:4200>
PREFECT_PROFILE='ephemeral'
PREFECT_API_AUTH_STRING='********' (from profile)
PREFECT_API_URL='<http://127.0.0.1:4200/api>' (from profile)
PREFECT_RESULTS_PERSIST_BY_DEFAULT='true' (from profile)
PREFECT_SERVER_ALLOW_EPHEMERAL_MODE='true' (from profile)Marvin
11/19/2025, 2:31 PMMarvin
11/19/2025, 2:37 PMimport os
# Set these BEFORE importing or calling anything that uses Prefect settings
os.environ["PREFECT_API_URL"] = "<http://your-host:4200/api>"
os.environ["PREFECT_API_AUTH_STRING"] = "username:password" # or use PREFECT_API_KEY
from prefect import get_client
async with get_client() as client:
...
Tip: The URL should include /api for Prefect Server.
Option 2: Override settings just for this block
- This guarantees a fresh override even if settings were already cached.
from prefect.settings import temporary_settings, PREFECT_API_URL, PREFECT_API_AUTH_STRING
from prefect import get_client
with temporary_settings({
PREFECT_API_URL: "<http://your-host:4200/api>",
PREFECT_API_AUTH_STRING: "username:password", # or use PREFECT_API_KEY
}):
async with get_client() as client:
...
Option 3: Bypass get_client() and instantiate the client explicitly
- This gives you full control without relying on profiles or env vars.
from prefect.client.orchestration import PrefectClient
async with PrefectClient(
api="<http://your-host:4200/api>",
auth_string="username:password", # or api_key="..."
) as client:
...
Verify what Prefect will use
- From your shell, you can confirm the sources:
prefect config view --show-sources
- If you prefer to persist the change in your profile (instead of env vars), set it:
prefect config set PREFECT_API_URL="<http://your-host:4200/api>"
prefect config set PREFECT_API_AUTH_STRING="username:password" # or PREFECT_API_KEY="..."
Notes
- In Prefect 3.x, use PREFECT_API_URL, PREFECT_API_AUTH_STRING (Basic), or PREFECT_API_KEY (Bearer). There’s no PREFECT_API_ROOT_URL.
- If both Basic and API key are set, Basic (PREFECT_API_AUTH_STRING) will be used.
- get_client() does not accept a base URL/auth parameter; it only takes httpx_settings. Use temporary_settings or instantiate PrefectClient directly to override.
If you share a small code snippet showing where you set the env vars relative to your Prefect imports/usage, I can point out exactly why it’s still reading from the profile.Romain Lefebvre
11/19/2025, 3:13 PMflow_run = await client.read_flow_run(UUID(flow_run_id))
state = flow_run.state
result = await state.result() -> raised hereMarvin
11/19/2025, 3:13 PMMarvin
11/19/2025, 3:19 PMstate.result(), your Python process downloads the payload directly from the configured result storage (S3 in your case) using s3fs/fsspec. If your current environment can’t reach that bucket or doesn’t have credentials/deps, you’ll see “All connection attempts failed” even though the storage key is correct.
What to check
- Use the async method in async code
- Prefer await state.aresult(...) in async code. state.result() is the sync helper.
- Ensure required deps are installed where you call result()
- Install s3fs (and boto3): pip install s3fs boto3 (or include them in your project env)
- Without s3fs/boto3, S3 reads will fail.
- Ensure AWS credentials are available in the environment running state.result()
- The worker that wrote the result may have had creds, but your current process may not.
- Provide credentials via env vars (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, optional AWS_SESSION_TOKEN, AWS_REGION) or store them in the block (if appropriate).
- If the bucket is private, anonymous access will fail with a similar message.
- If using an S3-compatible service (MinIO/Ceph), set endpoint/config in the block
- Your RemoteFileSystem / S3 block should include endpoint_url and addressing style if needed, for example:
RemoteFileSystem(
basepath="<s3://my-bucket/prefix/>",
settings={
"key": "...", "secret": "...",
"client_kwargs": {
"endpoint_url": "<https://minio.example.com>",
"region_name": "us-east-1"
},
# For some S3-compatible services you may need:
# "config_kwargs": {"s3": {"addressing_style": "path"}}
}
)
- Network/VPC routing
- If your bucket is reachable only from a VPC/private network, a local laptop (or another host) may fail to connect.
Quick diagnostics
- Confirm the result metadata and referenced storage block
flow_run = await client.read_flow_run(UUID(flow_run_id))
state = flow_run.state
# Inspect where Prefect thinks the result lives
meta = state.data.result_metadata # contains storage_key and storage_document_id
print(meta.storage_key, meta.storage_document_id)
# Optional: fetch the storage block doc to verify basepath/settings
block_doc = await client.read_block_document(meta.storage_document_id)
print(block_doc.block_type_slug, block_doc.name)
print(block_doc.data) # shows basepath/settings that will be used for S3
- Test S3 access directly from this environment
import s3fs
fs = s3fs.S3FileSystem() # picks up your AWS_* env vars or ~/.aws/credentials
# Does the object exist?
print(fs.exists("<s3://your-bucket/path/from/meta.storage_key>"))
# Try to read a few bytes
with fs.open("<s3://your-bucket/path/from/meta.storage_key>", "rb") as f:
print(f.read(16))
If that fails with a similar error, it’s a credentials/connectivity issue outside Prefect.
- Retrieve result with retries (default is 10 attempts)
# Async
value = await state.aresult(raise_on_failure=True, retry_result_failure=True)
# Sync
value = state.result(raise_on_failure=True, retry_result_failure=True)
Common fixes
- Install deps in the reading environment: pip install s3fs boto3
- Provide AWS credentials where you call state.result()
- If using S3-compatible storage, ensure your block’s settings include the correct endpoint and addressing style.
- If you don’t want to put credentials in the block, ensure the environment running the read has them via env/instance profile.
If you can share:
- The block type/slug used for result storage
- Whether this environment has AWS_* env vars or an IAM role
- Whether you’re using AWS S3 or an S3-compatible service
…I can suggest the exact configuration to resolve the connection error.Romain Lefebvre
11/19/2025, 3:25 PMMarvin
11/19/2025, 3:25 PMMarvin
11/19/2025, 3:27 PM/block_documents/{id}?include_secrets=true.
- That internal call uses the active Prefect settings context. It does not reuse your explicitly created client instance.
- In your case, that internal call is picking up the profile default (localhost), hence the GET to <http://127.0.0.1:4200/api/>....
So yes, seeing localhost there means the settings used during state.result() are not the same as the ones you configured for your earlier client call.
Fixes
Pick one of these patterns so the settings are correct when state.aresult() runs:
1) Use temporary_settings around the call
from uuid import UUID
from prefect import get_client
from prefect.settings import temporary_settings, PREFECT_API_URL, PREFECT_API_AUTH_STRING
with temporary_settings({
PREFECT_API_URL: "<http://your-host:4200/api>",
PREFECT_API_AUTH_STRING: "username:password", # or use PREFECT_API_KEY
}):
async with get_client() as client:
flow_run = await client.read_flow_run(UUID(flow_run_id))
state = flow_run.state
result = await state.aresult() # async version
2) Set environment variables before Prefect settings are used
import os
os.environ["PREFECT_API_URL"] = "<http://your-host:4200/api>"
os.environ["PREFECT_API_AUTH_STRING"] = "username:password" # or PREFECT_API_KEY
from uuid import UUID
from prefect import get_client
async with get_client() as client:
flow_run = await client.read_flow_run(UUID(flow_run_id))
state = flow_run.state
result = await state.aresult()
Notes
- Prefer await state.aresult() in async code; state.result() is the sync helper.
- If you instantiated PrefectClient(api=...) directly without setting PREFECT_API_URL, state.aresult() will still open its own client using settings (and hit the profile’s URL). Use one of the two patterns above so the settings context is correct.
- Ensure s3fs/boto3 and AWS creds are available in the environment running this code so the S3 read can succeed once the API call is fixed.
Quick sanity check
- Run:
prefect config view --show-sources
and confirm PREFECT_API_URL is coming from env/temporary settings (not profile) when you execute the code that calls state.aresult().
If you’re still seeing localhost after wrapping with temporary_settings, share the snippet around the aresult() call and how you’re currently constructing the client, and I’ll pinpoint the mismatch.