https://prefect.io logo
m

Michael Booth

08/10/2021, 1:23 AM
Hi 🙂 - trying to run the intro tutorial i.e. https://docs.prefect.io/core/tutorial/01-etl-before-prefect.html and I've having troubles with the OpenSky API by the look of it (presumably all APIs have bad days?) Should I just wait til it gets "better" to run this part of the tutorial or is something else going on? Cheers, Michael
Copy code
(prefect) mjboothaus@Michaels-Air tutorial % python 01_etl.py 
...TRUNCATED - See thread
As a follow-up -- I ran the 2nd part of the tutorial
02_etl_flow.py
and I guess this illustrates the more helpful messages/info provided by the prefect library when tasks fail
Copy code
(prefect) mjboothaus@Michaels-Air tutorial % python 02_etl_flow.py
...TRUNCATED... See thread
SOLVED - seems API was down - now OK
k

Kevin Kho

08/10/2021, 2:17 AM
Hey @Michael Booth, I just went through the steps on an environment with python 3.8.10 and can not replicate. Do you still have the same issue? Could you also please move the traceback and code to this thread so we don’t crowd the main channel when you get a chance?
m

Michael Booth

08/10/2021, 2:53 AM
Hi @Kevin Kho - thanks for your reply. I just re-ran and it is fine now -- seems there was a temporary problem with the API. Very happy to move the traceback & code to this thread -- although I don't use Slack alot - could you detail please (i will remove from original channel message for now)
k

Kevin Kho

08/10/2021, 3:06 AM
You just edit the message (on the top right of the message). Copy the code blocks into the thread as new messages. In the original post you can just say something like "code in thread"
m

Michael Booth

08/10/2021, 3:10 AM
Output moved from channel to thread below: 01_etl.py
Copy code
(prefect) mjboothaus@Michaels-Air tutorial % python 01_etl.py 
...TRUNCATED...
  File "/Users/mjboothaus/code/github/prefecthq/prefect/examples/tutorial/aircraftlib/opensky.py", line 63, in _api_request_json
    response.raise_for_status()
  File "/opt/homebrew/Caskroom/mambaforge/base/envs/prefect/lib/python3.8/site-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 502 Server Error: Bad Gateway for url: <https://opensky-network.org/api/states/all?lamin=37.15294209056667&lamax=40.75094679823332&lomin=-79.76136868215879&lomax=-75.13474242904122>
02_etl_flow.py
Copy code
(prefect) mjboothaus@Michaels-Air tutorial % python 02_etl_flow.py
...TRUNCATED...
requests.exceptions.HTTPError: 502 Server Error: Bad Gateway for url: <https://opensky-network.org/api/states/all?lamin=37.15294209056667&lamax=40.75094679823332&lomin=-79.76136868215879&lomax=-75.13474242904122>
[2021-08-10 11:24:07+1000] INFO - prefect.TaskRunner | Task 'extract_live_data': Finished task run for task with final state: 'Failed'
[2021-08-10 11:24:07+1000] INFO - prefect.TaskRunner | Task 'transform': Starting task run...
[2021-08-10 11:24:07+1000] INFO - prefect.TaskRunner | Task 'transform': Finished task run for task with final state: 'TriggerFailed'
[2021-08-10 11:24:07+1000] INFO - prefect.TaskRunner | Task 'load_live_data': Starting task run...
[2021-08-10 11:24:07+1000] INFO - prefect.TaskRunner | Task 'load_live_data': Finished task run for task with final state: 'TriggerFailed'
[2021-08-10 11:24:07+1000] INFO - prefect.FlowRunner | Flow run FAILED: some reference tasks failed.
Thanks again Kevin -- if you haven't see it before take a look at Datasette.io (no affiliation) -- good for viewing/querying SQLite databases like that created in the tutorial
k

Kevin Kho

08/10/2021, 3:14 AM
No worries and thanks for moving it! I think the intention is to actually move away from that tutorial so that people dont need to learn a dataset before Prefect
m

Michael Booth

08/10/2021, 3:20 AM
Yes -- it is a "non-trivial" example - although that has some up sides - but it probably does distract from Prefect intro a little. Cheers!
3 Views