oops — she/her

We are doing intros and popcorning to the next person. Feel free to put a hand up in Zoom if you haven’t gone yet. 🙋

Name changing on Zoom: 1. Sign in to the Zoom web portal. 2. In the navigation menu, click Profile. 3. By your account profile name, click Edit. 4. For your account profile name, enter your First Name and Last Name. 5. For your Display Name, enter a name (e.g. your first name, full name, or nickname) that will be shown to others on Zoom. 6. Click Save.

i love my marvin

GitHub repo: https://github.com/discdiver/pacc-2023 Slides:

Lab 101 Instructions: Use Open-Meteo API • Write flow code that fetches other weather metrics • Make at least 3 tasks in the flow • Example: windspeed for the last hour: _weather.json()[“hourly”][“windspeed_10m”][0]_ Docs: open-meteo.com/en/docs

Share your code from lab 101 here: 🧵

You all are rockin it! We’ll get back together at 12:15pm ET (11 minutes from now)

Note the GitHub repo and slides above. 🙂

Hey everybody 👋 I haven't written much Python that calls APIs yet, but I remember the Prefect docs have great examples - I'm modifying this code from here: https://docs.prefect.io/2.10.13/tutorial/first-steps/#run-a-basic-flow-with-tasks Hope this helps!

import requests
from prefect import flow, task

@task
def call_api(url):
    response = requests.get(url)
    print(response.status_code)
    return response.json()

@flow
def api_flow(url):
    fact_json = call_api(url)
    return fact_json

print(api_flow("<https://catfact.ninja/fact>"))

How was that (emoji)?

Hi all, I was interested in seeing how can I come over the limitation for task in task , thus I created flows

from prefect import flow,task
import httpx

# API to hit - <https://api.open-meteo.com/v1/forecast?latitude=52.52&longitude=13.41&hourly=weathercode,windspeed_10m,soil_temperature_0cm>

@task
def get_windspeed_10m(lat,lon):
    r = httpx.get(f'<https://api.open-meteo.com/v1/forecast?latitude={lat}&longitude={lon}&hourly=windspeed_10m&forecast_days=1>')
    wind_speed = r.json()['hourly']['windspeed_10m'][0]
    return wind_speed

@task
def get_windspeed_80m(lat,lon):
    r = httpx.get(f'<https://api.open-meteo.com/v1/forecast?latitude={lat}&longitude={lon}&hourly=windspeed_80m&forecast_days=1>')
    wind_speed = r.json()['hourly']['windspeed_80m'][0]
    return wind_speed

@flow
def get_weather(lat,lon):
    wind_speed_80 = get_windspeed_80m(lat,lon)
    wind_speed = get_windspeed_10m(lat,lon)
    return {
        "wind_speed_80m" :wind_speed_80,
        "wind_speed_10m" :wind_speed
    }


@task
def get_soil_temperature_0cm(lat,lon):
    r = httpx.get(f'<https://api.open-meteo.com/v1/forecast?latitude={lat}&longitude={lon}&hourly=soil_temperature_0cm&forecast_days=1>')
    soil_temp = r.json()['hourly']['soil_temperature_0cm'][0]
    return soil_temp

@flow
def get_all(lat,lon):
    obj = get_weather(lat,lon)
    soil_temp = get_soil_temperature_0cm(lat,lon)
    print({
        "wind_10m" : obj['wind_speed_10m'],
        "wind_80m" :obj['wind_speed_80m'],
        "soil_temp" :soil_temp,
    })

if __name__  == '__main__' :
    get_all(52,17)

Will you share these slides with us at the end of the course?

The slides are all pinned above. 🙂

thanks

Lab 102 Instructions: • Use a flow that grabs weather data from open-meteo • Add retries • Add a subflow that transforms or aggregates the data • Start a Prefect server instance • Run your flow • Inspect in the UI • Stretch: add caching

I’m noticing some errors I don’t totally understand when I have

prefect server start

running 🧵

If you have to jump out of Zoom to enable screen sharing, just message us in Slack and we’ll get you back into your Breakout Room. 🙂

How does this work?

@task(cache_key_fn=task_input_hash)

i read the docs here https://docs.prefect.io/2.10.13/api-ref/prefect/tasks/#prefect.tasks.task_input_hash and I’m still a lil confused

hi, we have some questions, I don’t know the number of the room though

Would someone be able to come into our room? I think we're #3

I get this error: await self.lifespan(scope, receive, send) File "C:\Users\10353302\AppData\Local\anaconda3\lib\site-packages\starlette\routing.py", line 677, in lifespan async with self.lifespan_context(app) as maybe_state: File "C:\Users\10353302\AppData\Local\anaconda3\lib\contextlib.py", line 199, in aenter return await anext(self.gen) File "C:\Users\10353302\AppData\Local\anaconda3\lib\site-packages\prefect\server\api\server.py", line 520, in lifespan await stop_services() File "C:\Users\10353302\AppData\Local\anaconda3\lib\site-packages\prefect\server\api\server.py", line 502, in stop_services if app.state.services: File "C:\Users\10353302\AppData\Local\anaconda3\lib\site-packages\starlette\datastructures.py", line 705, in getattr raise AttributeError(message.format(self.class.name, key)) AttributeError: 'State' object has no attribute 'services' (base) C:\Users\10353302\training>prefect version Version: 2.10.13 API version: 0.8.4 Python version: 3.10.9 Git commit: 179edeac Built: Thu, Jun 8, 2023 4:10 PM OS/Arch: win32/AMD64 Profile: local Server type: ephemeral Server: Database: sqlite SQLite version: 3.40.1

We’ll chat about caching for a sec when we come back.

https://docs.prefect.io/2.10.13/concepts/states/#create-flow-run-state-change-hooks Some webhooks to explore if you want to add more conditional workflows!

Let’s come back together at 1:10pm ET!

Apologies I need to drop off for 30 mins but will be back

something I’ve noticed - it so happens that I have separate

.py

files between the two labs we’ve done, but without thinking I’ve named the top-level flows in each file

pipeline()

. In the Prefect UI I think they seem to be treated as one flow, but the flow runs will have different contents / tasks depending on which file I run. Seems like this could be helpful or really confusing - any thoughts about how this might work in practice?

Santhosh wrote a script that successfully cached. It ran on both of our machines, so it looks like the script we were using at first was the issue

Not entirely sure what the cause was yet

my group's code for those who wanted it