m

    Michael Booth

    1 year ago
    Hi 🙂 - trying to run the intro tutorial i.e. https://docs.prefect.io/core/tutorial/01-etl-before-prefect.html and I've having troubles with the OpenSky API by the look of it (presumably all APIs have bad days?) Should I just wait til it gets "better" to run this part of the tutorial or is something else going on? Cheers, Michael
    (prefect) mjboothaus@Michaels-Air tutorial % python 01_etl.py 
    ...TRUNCATED - See thread
    As a follow-up -- I ran the 2nd part of the tutorial
    02_etl_flow.py
    and I guess this illustrates the more helpful messages/info provided by the prefect library when tasks fail
    (prefect) mjboothaus@Michaels-Air tutorial % python 02_etl_flow.py
    ...TRUNCATED... See thread
    SOLVED - seems API was down - now OK
    Kevin Kho

    Kevin Kho

    1 year ago
    Hey @Michael Booth, I just went through the steps on an environment with python 3.8.10 and can not replicate. Do you still have the same issue? Could you also please move the traceback and code to this thread so we don’t crowd the main channel when you get a chance?
    m

    Michael Booth

    1 year ago
    Hi @Kevin Kho - thanks for your reply. I just re-ran and it is fine now -- seems there was a temporary problem with the API. Very happy to move the traceback & code to this thread -- although I don't use Slack alot - could you detail please (i will remove from original channel message for now)
    Kevin Kho

    Kevin Kho

    1 year ago
    You just edit the message (on the top right of the message). Copy the code blocks into the thread as new messages. In the original post you can just say something like "code in thread"
    m

    Michael Booth

    1 year ago
    Output moved from channel to thread below: 01_etl.py
    (prefect) mjboothaus@Michaels-Air tutorial % python 01_etl.py 
    ...TRUNCATED...
      File "/Users/mjboothaus/code/github/prefecthq/prefect/examples/tutorial/aircraftlib/opensky.py", line 63, in _api_request_json
        response.raise_for_status()
      File "/opt/homebrew/Caskroom/mambaforge/base/envs/prefect/lib/python3.8/site-packages/requests/models.py", line 953, in raise_for_status
        raise HTTPError(http_error_msg, response=self)
    requests.exceptions.HTTPError: 502 Server Error: Bad Gateway for url: <https://opensky-network.org/api/states/all?lamin=37.15294209056667&lamax=40.75094679823332&lomin=-79.76136868215879&lomax=-75.13474242904122>
    02_etl_flow.py
    (prefect) mjboothaus@Michaels-Air tutorial % python 02_etl_flow.py
    ...TRUNCATED...
    requests.exceptions.HTTPError: 502 Server Error: Bad Gateway for url: <https://opensky-network.org/api/states/all?lamin=37.15294209056667&lamax=40.75094679823332&lomin=-79.76136868215879&lomax=-75.13474242904122>
    [2021-08-10 11:24:07+1000] INFO - prefect.TaskRunner | Task 'extract_live_data': Finished task run for task with final state: 'Failed'
    [2021-08-10 11:24:07+1000] INFO - prefect.TaskRunner | Task 'transform': Starting task run...
    [2021-08-10 11:24:07+1000] INFO - prefect.TaskRunner | Task 'transform': Finished task run for task with final state: 'TriggerFailed'
    [2021-08-10 11:24:07+1000] INFO - prefect.TaskRunner | Task 'load_live_data': Starting task run...
    [2021-08-10 11:24:07+1000] INFO - prefect.TaskRunner | Task 'load_live_data': Finished task run for task with final state: 'TriggerFailed'
    [2021-08-10 11:24:07+1000] INFO - prefect.FlowRunner | Flow run FAILED: some reference tasks failed.
    Thanks again Kevin -- if you haven't see it before take a look at Datasette.io (no affiliation) -- good for viewing/querying SQLite databases like that created in the tutorial
    Kevin Kho

    Kevin Kho

    1 year ago
    No worries and thanks for moving it! I think the intention is to actually move away from that tutorial so that people dont need to learn a dataset before Prefect
    m

    Michael Booth

    1 year ago
    Yes -- it is a "non-trivial" example - although that has some up sides - but it probably does distract from Prefect intro a little. Cheers!