🧵 on Logging

Example where a task fails after a bunch of tasks succeeded. The flow is also marked as failed in this case.

Is the cache just for within the flow?

For the dbt caching / rerun we do something like this :

@flow
def myflow(force: bool = False):
    mytask.with_options(refresh_cache=force)

Lab 102 Instructions

• Use a flow that grabs weather data from open-meteo

• Add retries

• Start a Prefect server instance

• Run your flow

• Inspect in the UI

• Stretch: add caching

@Emil Christensen to the zoom pls

Share your code for 102 here (🧵):

We’ll get back together at 9pm ET. Feel free to take a break prior. 🙂

Runtime context: https://docs.prefect.io/latest/concepts/runtime-context/

Creating new block types docs: https://docs.prefect.io/2.11.5/concepts/blocks/#creating-new-block-types

You’ll see webhooks tomorrow. 🚀

103 Lab Instructions: • Use a previously made flow • Create a Cloud profile (or use an existing one) • Authenticate for the CLI • Make an email notification automation for flow run completion • Run the flow a few times from the CLI • See the event feed in the UI • Stretch: create a Slack notification automation

Getting this error when trying to auth with the api key? Any ideas?

Unable to authenticate with Prefect Cloud. Your key is not in our expected format.

We’ll get back together in 3 minutes at 9:50pm ET.

Automation action is failing for me with this

"reason": "The notification block was invalid: ValidationError(model='PrefectCloudEmail', errors=[{'loc': ('emails',), 'msg': 'value is not a valid list', 'type': 'type_error.list'}])"

Recording from day 1: https://us06web.zoom.us/rec/share/XZ3md8QzM1kFDMdCCrPVR5l7FO-lLq_Neg4k3mV9s2QjA9o0AmDDE3wTBAVAdpbc.jXdRf7bGeRHTVRSh Passcode: Wb4Xf^4t

See you at at 7pm ET (a little under an hour and a half from now).

Kicking off momentarily in the same Zoom room as yesterday. 🙂

104 Lab Instructions 📚 : • Use a flow from the 103 lab • Make a deployment via interactive CLI commands • Start a worker • Run a deployment from the UI • See the flow run logs in the UI • Stretch: Create a deployment that has default parameters & override the default parameters at runtime in the UI

Can we get the slides for today?

Day 2 slides:

Side note, if you get stuck, you can take a look at the Tutorial in the docs. 🙂 (just swap out the script for one that you used in 103)

I got a flow deployed (thumbs)!

error coming from

prefect deploy

running in terminal

Traceback (most recent call last):
  File "/Users/zachs/Documents/repos/prefect_cert_course/venv/lib/python3.9/site-packages/prefect/cli/_utilities.py", line 41, in wrapper
    return fn(*args, **kwargs)
  File "/Users/zachs/Documents/repos/prefect_cert_course/venv/lib/python3.9/site-packages/prefect/utilities/asyncutils.py", line 255, in coroutine_wrapper
    return call()
  File "/Users/zachs/Documents/repos/prefect_cert_course/venv/lib/python3.9/site-packages/prefect/_internal/concurrency/calls.py", line 382, in __call__
    return self.result()
  File "/Users/zachs/Documents/repos/prefect_cert_course/venv/lib/python3.9/site-packages/prefect/_internal/concurrency/calls.py", line 282, in result
    return self.future.result(timeout=timeout)
  File "/Users/zachs/Documents/repos/prefect_cert_course/venv/lib/python3.9/site-packages/prefect/_internal/concurrency/calls.py", line 168, in result
    return self.__get_result()
  File "/usr/local/Cellar/python@3.9/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
  File "/Users/zachs/Documents/repos/prefect_cert_course/venv/lib/python3.9/site-packages/prefect/_internal/concurrency/calls.py", line 345, in _run_async
    result = await coro
  File "/Users/zachs/Documents/repos/prefect_cert_course/venv/lib/python3.9/site-packages/prefect/cli/deploy.py", line 249, in deploy
    await _run_single_deploy(
  File "/Users/zachs/Documents/repos/prefect_cert_course/venv/lib/python3.9/site-packages/prefect/client/utilities.py", line 51, in with_injected_client
    return await fn(*args, **kwargs)
  File "/Users/zachs/Documents/repos/prefect_cert_course/venv/lib/python3.9/site-packages/prefect/cli/deploy.py", line 312, in _run_single_deploy
    deploy_config["entrypoint"] = await prompt_entrypoint(app.console)
  File "/Users/zachs/Documents/repos/prefect_cert_course/venv/lib/python3.9/site-packages/prefect/cli/_prompts.py", line 598, in prompt_entrypoint
    discovered_flows = await _search_for_flow_functions()
  File "/Users/zachs/Documents/repos/prefect_cert_course/venv/lib/python3.9/site-packages/prefect/deployments/base.py", line 555, in _search_for_flow_functions
    return [fn for file_fns in await asyncio.gather(*coros) for fn in file_fns]
  File "/Users/zachs/Documents/repos/prefect_cert_course/venv/lib/python3.9/site-packages/prefect/deployments/base.py", line 471, in _find_flow_functions_in_file
    async with OPEN_FILE_SEMAPHORE:
  File "/usr/local/Cellar/python@3.9/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/locks.py", line 14, in __aenter__
    await self.acquire()
  File "/usr/local/Cellar/python@3.9/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-5251' coro=<_find_flow_functions_in_file() running at /Users/zachs/Documents/repos/prefect_cert_course/venv/lib/python3.9/site-packages/prefect/deployments/base.py:471> cb=[_gather.<locals>._done_callback() at /usr/local/Cellar/python@3.9/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/tasks.py:767]> got Future <Future pending> attached to a different loop
An exception occurred.

Confirming that if you don’t specify a git repo or another storage location Prefect does use an absolute path @James Constable.

Blog post on Prefect workflow patterns Bianca referred to: https://www.prefect.io/blog/workflow-design-patterns

Blog post on event-driven flows with webhooks: https://www.prefect.io/blog/event-driven-flows-with-prefect

GH Issue for multiple prefect.yaml files.

@Elizabeth Bing - 🧵 for error