https://prefect.io
Join Slack
:wave: Hi all! Is there a good place to ask about logging configuration options using Prefect Cloud?...
# ask-community
j
👋 Hi all! Is there a good place to ask about logging configuration options using Prefect Cloud? 😄
k
Hi @Joe Goldbeck, feel free to ask here
🙌 1
j
ty! Scenario: considering using Prefect for project including HIPAA data, which we expect may occasionally get logged. Thus, we do not want to ship task logs outside of our environment. Is there anyway to set up Prefect Cloud where we could still view our task logs via a Prefect UI, but without them leaving the environment? We didn’t see anything in Prefect 1 that looked promising. Does Prefect 2 perhaps expand our options here?
k
I think you’d need to use Prefect Server considering the restrictions because the Prefect Cloud UI queries from our database so you’d need to send those logs for it to get rendered in the UI
🙏 1
I don’t think the story changes so much in Prefect 2.0, except for you can have your own dedicated “Prefect Cloud” instance in the future where you own the infrastructure. Not super clear on the form of this yet and if it solves the exact use case because it’s in active development
👍 1
What some users do here though is that they ship the logs somewhere like DataDog on their own infrastructure, and then you can turn off sending logs to Prefect Cloud
👍 2
But you wont be able to see it in our UI in that case.
👍 1
j
That makes sense! That’s the direction we’re considering. It feels a lot less streamlined of an experience (working with HIPAA data seems to result in that…), but workable. Thank you so much for taking the time to give it a think and share your thoughts!!
👍 1
5 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by