Hello! I have two small questions related to blocks:
• Even if the
prefect_dbt
and
prefect_airbyte
collections exists, I was wondering if it makes sense to have something like a dedicated block for Airbyte (which allow us to store the
server_url
but also invoke a
trigger_sync
function) and the same question with a DBT block (and we store all the information that we can find in the
profiles.yml
, for exemple)?
• What is the best way to log from a function of a
Block
subclass? Can we use the
get_run_logger()
or is preferable to use some sort of callbacks that are going to be invoked inside a
@task
or something?
Thanks!
✅ 1
a
Anna Geller
09/10/2022, 12:38 AM
Great questions! you can definitely add a custom Block for airbyte if needed. Generally speaking, logging from an application component such as a block is not needed, even often discouraged, but you can in the end do it as you want in your custom blocks and collections - check out the prefect.utilities module, it has one for logging as well
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.