r

    rectalogic

    2 months ago
    so flows work locally but
    __file__
    is not defined when prefect cloud runs the flow. Any idea what the issue is?
    @Phillip Kelly
    so it sounds like
    __file__
    is not useable with GitHub storage? https://github.com/PrefectHQ/prefect/issues/3529 What about other local assets that the flow needs to load that are in the github repo (e.g. local .sql files). Will they even be accessible to the flow, is prefect not checking out the repo when it runs the flow?
    what about relative imports from other modules in my github repo? does the agent not checkout the entire github repository when running my flow?
    unbelievable, I guess it's not supported at all. I really don't understand the reasoning behind most of the decisions prefect makes.. https://github.com/PrefectHQ/prefect/issues/4574
    Kevin Kho

    Kevin Kho

    2 months ago
    Yes the Github Storage only gets the Flow file. We know this not user friendly so it is being re-designed for Prefect 2.0. For Prefect 1.0, you need to package the dependencies in a container.
    Stuff like SQL files can be achieved with Git Storage here
    r

    rectalogic

    2 months ago
    is there any way to make the Git Storage use
    GITHUB_ACCESS_TOKEN
    PAT env var? or does it have to use a prefect secret or ssh privkey?
    Kevin Kho

    Kevin Kho

    2 months ago
    I think no. You need to use a Secret or the ssh key
    r

    rectalogic

    2 months ago
    ok, we switched to Git storage and now
    __file__
    is correct e.g.
    /tmp/tmpxy6wy7bw/cureatr/flows/...
    - but we get
    FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpxy6wy7bw/cureatr/flows/patient_funnel/patient_funnel.sql'
    when trying to open a file relative to
    __file__
    - that path seems correct. Do we have to do something specific to make Git storage clone the entire repo?
    Kevin Kho

    Kevin Kho

    2 months ago
    On that, Prefect will clone the repo, load in the SQL, and then delete the files so it has to be read in outside of tasks when the script is ran
    r

    rectalogic

    2 months ago
    ugh, ok. You guys sure don't make anything easy...
    Kevin Kho

    Kevin Kho

    2 months ago
    Yeah we know Git storage can’t be the same form for sure for 2.0