https://prefect.io logo
#prefect-community
Title
# prefect-community
e

Edvard Kristiansen

03/01/2022, 3:05 PM
Referencing Local Files in Code I am attempting to reference some key files that are stored locally on the VM where the flows were registered from, but prefect cant seem to find the files in the local file system. I have tried to change the storage to local with no luck. Sorry if I have missed something very basic here! It runs fine on my local machine with similar file references(have adapted the code to fit with the VMs file locations before registering the flow).
Copy code
@task
def run():
    script....

key_location = "~/prefect/Prefect/keys/key.json"

with Flow("pipeline", executor=LocalDaskExecutor(), run_config=LocalRun(labels=["local"]), storage=Local()) as flow:
    run()
I am getting error: FileNotFoundError: [Errno 2] No such file or directory: Any ideas?
k

Kevin Kho

03/01/2022, 3:10 PM
Hi @Edvard Kristiansen, maybe you can try specifying the working directory like the last example here? Does the place with the agent also have the files? The files are not stored along with your Flow
e

Edvard Kristiansen

03/01/2022, 3:25 PM
Hmm that has some very odd behaviour. Checking the working_dir in the UI it doubles up the path by just adding on the working dir I specified in the file. I assume this feature might be for getting deeper into the files rather than specifying exactly the path? I have all the files stored in the same place in the VM where I register the flow from.
k

Kevin Kho

03/01/2022, 3:28 PM
A bit confused what you are running into. This feature is so if you have custom modules you want to import in a folder, you can run the Flow in that folder to resolve the imports. Could you show me a screenshot or code of what you are running into?
e

Edvard Kristiansen

03/01/2022, 3:35 PM
Ah looks like working_dir="." did what I needed it to. It was storing the flow in the .prefect folder when this wasnt specified.
I have a github repo synced onto the actual VM itself which mirrors my local development.
k

Kevin Kho

03/01/2022, 3:37 PM
I think the Local storage will still store it in the
.prefect
folder by default. It shouldn’t depend on the RunConfig. But you can specify the path and use
stored_as_script=True
so that it doesn’t store it in the
.prefect
folder
e

Edvard Kristiansen

03/02/2022, 9:33 AM
That seemed like I had to change to using Github as the storage method? It seems like its now running fine when I set the working directory like above.
k

Kevin Kho

03/02/2022, 1:59 PM
Github storage is script based by default and it would work, but you don’t need it. You can also do
flow.storage = Local(path="….",stored_as_script=True)
to point to the Python file on local and it should be the same. I do highly recommend Github storage though
4 Views