https://prefect.io logo
Title
r

Raghav Sharma

11/16/2021, 11:44 AM
Hi All! I hope you're having a good day. I am new to Prefect and need your help in understanding Prefect w.r.t databases. I need to add a database and run queries to get data. Can someone please guide what steps should i follow ?
a

Anna Geller

11/16/2021, 11:53 AM
Welcome to the community @Raghav Sharma! You could have a look at the task library: there are many tasks that can help you work with various databases e.g. MySQL, Postgres, SQL Server, BigQuery, Snowflake, and more: ā€¢ https://docs.prefect.io/core/task_library/overview.html#task-library-in-action ā€¢ all those tasks are described in the API reference documentation e.g. https://docs.prefect.io/api/latest/tasks/postgres.html But you can also decorate any Python function that interacts with a database with @task and this will turn it into a Prefect task. More on that: ā€¢ https://docs.prefect.io/core/concepts/flows.html#functional-api ā€¢ https://docs.prefect.io/core/concepts/tasks.html#overview
šŸ™‚ 1
r

Raghav Sharma

11/16/2021, 11:57 AM
Thanks a ton @Anna Geller! Will check these :)
šŸ™Œ 1
@Anna Geller, is there a way to connect to a database via the Prefect UI ?
a

Anna Geller

11/18/2021, 3:04 PM
@Raghav Sharma What exactly do you have in mind? You can set a database secret in the Prefect UI and use this secret to connect to a database from a task. In general, Prefect follows engineering best practices and therefore workflow is defined as code, not through drag and drop. So you can have a task defined in code and you can use the UI to observe the states of your flows and tasks. Have you seen our getting started guide? It will walk you through all the steps from installation to building and running your first flows: https://docs.prefect.io/core/getting_started/quick-start.html
šŸ‘ 1
r

Raghav Sharma

11/19/2021, 7:21 AM
Hi @Anna Geller, my requirement is to connect to a database by passing required arguments such as username, password, host and db name(say) via Prefect UI. I am not sure if this is possible via UI. Currently i am connecting via defining a task and flow in my code.
Also, need to know if a particular response from a task that contains meta-data from a database can be displayed on the UI or not. Please guide. Thanks!
a

Anna Geller

11/19/2021, 12:11 PM
@Raghav Sharma great to hear you got it working with database interaction from a task. Regarding displaying information from a task in the UI, we do have an Artifact API that allows you to publish an arbitrary markdown report into the UI. To learn more about how this works, check the following resources: ā€¢ this PR will add docs about this feature: https://github.com/PrefectHQ/prefect/pull/5158/files?short_path=8f250fa#diff-8f250fabfa7040afe2d36b421f8adfcbcc1314398538fbe69c884e0e1ff66ed2 ā€¢ this blog post: https://medium.com/the-prefect-blog/introducing-the-artifacts-api-b9e5972db043
r

Raghav Sharma

11/19/2021, 12:23 PM
Ok! Cool. Thanks for these references. Will check these out and let you know if in case i need anything.
šŸ‘ 1