https://prefect.io
Join Slack
Hi, I’m trying to create a deployment with a minio object store as storage layer. However it seems t...
# ask-community
s
Hi, I’m trying to create a deployment with a minio object store as storage layer. However it seems that the flow code is not registered. I’m trying to debug this but I can’t seem to adjust the log level of Orion. I’ve tried setting PREFECT_LOGGING_SERVER_LEVEL to DEBUG but I still only see INFO logging from my server. I’m running my own server so no cloud here.
✅ 1
It also seems that the cli argument —log-level DEBUG doesn’t solve this either.
Copy code
___ ___ ___ ___ ___ ___ _____    ___  ___ ___ ___  _  _
| _ \ _ \ __| __| __/ __|_   _|  / _ \| _ \_ _/ _ \| \| |
|  _/   / _|| _|| _| (__  | |   | (_) |   /| | (_) | .` |
|_| |_|_\___|_| |___\___| |_|    \___/|_|_\___\___/|_|\_|

Configure Prefect to communicate with the server with:

    prefect config set PREFECT_API_URL=<http://0.0.0.0:4201/api>

View the API reference documentation at <http://0.0.0.0:4201/docs>

Check out the dashboard at <http://0.0.0.0:4201>



INFO:     Started server process [37876]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on <http://0.0.0.0:4201> (Press CTRL+C to quit)
INFO:     127.0.0.1:58582 - "GET /runs HTTP/1.1" 200 OK
INFO:     127.0.0.1:58582 - "GET /ui-settings HTTP/1.1" 200 OK
I think I can’t upload my flow code to the storage layer as the S3Bucket end point is defined within the k8s cluster. Does that make sense?
a
to enable debug logs you can do: prefect config set PREFECT_DEBUG_MODE=1 or prefect config set PREFECT_LOGGING_LEVEL=DEBUG
and for minio you would need to use RemoteFileSystem block - check https://docs.prefect.io/concepts/filesystems/?h=remote+file#remotefilesystem-examples
I can understand why this could be hard to configure on a K8s cluster with a locally running server since networking can be a bit tedious to set up, you can always start with the free tier of Prefect Cloud which would make this setup a bit easier
s
Not S3Bucket from prefect_aws?
a
for minio you would need RemoteFileSystem block, not the S3 block
s
a
for K8s deployment, you can also check out this setup with flow code directly baked into the Docker image, this way you don't even need minio or S3, this might be a good way for you (fewer moving parts) https://github.com/anna-geller/prefect-dataplatform/blob/main/deploy_locally_k8s.py
s
Sure? I see minio explicitly mentioned there?
Will check out that post.
There are indeed quite some moving parts:)
Looks nice. I now do something similar I think. Will check out the details. Thanks!
🙌 1
I got this working so all good here!
12 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by