Channels
#prefect-community
Title
s
Sander
10/29/2022, 8:22 PMHi, I’m trying to create a deployment with a minio object store as storage layer. However it seems that the flow code is not registered. I’m trying to debug this but I can’t seem to adjust the log level of Orion. I’ve tried setting PREFECT_LOGGING_SERVER_LEVEL to DEBUG but I still only see INFO logging from my server. I’m running my own server so no cloud here.
✅ 1
It also seems that the cli argument —log-level DEBUG doesn’t solve this either.
Copy code
___ ___ ___ ___ ___ ___ _____ ___ ___ ___ ___ _ _
| _ \ _ \ __| __| __/ __|_ _| / _ \| _ \_ _/ _ \| \| |
| _/ / _|| _|| _| (__ | | | (_) | /| | (_) | .` |
|_| |_|_\___|_| |___\___| |_| \___/|_|_\___\___/|_|\_|
Configure Prefect to communicate with the server with:
prefect config set PREFECT_API_URL=<http://0.0.0.0:4201/api>
View the API reference documentation at <http://0.0.0.0:4201/docs>
Check out the dashboard at <http://0.0.0.0:4201>
INFO: Started server process [37876]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on <http://0.0.0.0:4201> (Press CTRL+C to quit)
INFO: 127.0.0.1:58582 - "GET /runs HTTP/1.1" 200 OK
INFO: 127.0.0.1:58582 - "GET /ui-settings HTTP/1.1" 200 OKI think I can’t upload my flow code to the storage layer as the S3Bucket end point is defined within the k8s cluster. Does that make sense?
a
Anna Geller
10/29/2022, 9:32 PM
to enable debug logs you can do: prefect config set PREFECT_DEBUG_MODE=1 or prefect config set PREFECT_LOGGING_LEVEL=DEBUG
and for minio you would need to use RemoteFileSystem block - check https://docs.prefect.io/concepts/filesystems/?h=remote+file#remotefilesystem-examples
I can understand why this could be hard to configure on a K8s cluster with a locally running server since networking can be a bit tedious to set up, you can always start with the free tier of Prefect Cloud which would make this setup a bit easier
s
Sander
10/29/2022, 9:35 PMNot S3Bucket from prefect_aws?
a
Anna Geller
10/29/2022, 9:35 PM
for minio you would need RemoteFileSystem block, not the S3 block
s
Sander
10/29/2022, 9:36 PMa
Anna Geller
10/29/2022, 9:36 PM
for K8s deployment, you can also check out this setup with flow code directly baked into the Docker image, this way you don't even need minio or S3, this might be a good way for you (fewer moving parts) https://github.com/anna-geller/prefect-dataplatform/blob/main/deploy_locally_k8s.py
s
Sander
10/29/2022, 9:37 PMSure? I see minio explicitly mentioned there?
Will check out that post.
There are indeed quite some moving parts:)
Looks nice. I now do something similar I think. Will check out the details. Thanks!
🙌 1
I got this working so all good here!
3 Views