hi folks, how to set the docker storage configurat...
# ask-communityl
Laura Vaida
02/26/2021, 4:36 PMhi folks, how to set the docker storage configuration right? i have this example:
Copy code
Docker(
env_vars={
# append modules directory to PYTHONPATH
"PYTHONPATH": "C://Users/laura.vaida.000/anaconda3/envs/prefect/Lib/site-packages"
},
stored_as_script=True
)Laura Vaida
02/26/2021, 4:37 PMerror message is
/opt/prefect/healthcheck.py151 UserWarning: Flow uses module which is not importable. Refer to documentation on how to import custom modules https://docs.prefect.io/api/latest/storage.html#docker
flows = cloudpickle_deserialization_check(flow_file_paths)
Traceback (most recent call last):
File "/opt/prefect/healthcheck.py", line 151, in <module>
flows = cloudpickle_deserialization_check(flow_file_paths)
File "/opt/prefect/healthcheck.py", line 44, in cloudpickle_deserialization_check
flows.append(cloudpickle.loads(flow_bytes))
ModuleNotFoundError: No module named 'pandas'
j
J. Martins
02/26/2021, 4:40 PMlooks like you are missing ‘pandas’. You could simply provide Docker( …python_dependencies=[‘pandas’], ….)
J. Martins
02/26/2021, 4:41 PMyou can include any pip installable packages you need in that list. Hope this helps
l
Laura Vaida
02/26/2021, 4:46 PMthanks tried that
Laura Vaida
02/26/2021, 4:47 PMValueError: A
path.pyLaura Vaida
02/26/2021, 4:47 PMseems to be working but this error now appears
Laura Vaida
02/26/2021, 4:49 PM• `path (str, optional)`: a direct path to the location of the flow file in the Docker image if
stored_as_script=TrueLaura Vaida
02/26/2021, 4:49 PMguess that arises because of my setting stored_as_Script = true, what path is meant by that?
j
J. Martins
02/26/2021, 4:50 PMi guess so, if you skip setting stored_as_script, it should work
J. Martins
02/26/2021, 4:51 PMotherwise you need to provide the path where the script is located
z
Zanie
02/26/2021, 4:53 PM
If you use
stored_as_scriptl
Laura Vaida
02/26/2021, 5:00 PMif i use gcp should this be a alocal path?
j
J. Martins
02/26/2021, 5:04 PMI guess you mean GCS?
l
Laura Vaida
02/26/2021, 5:04 PMi use docker storage on google cloud platform
j
J. Martins
02/26/2021, 5:05 PMif you use GSC and stored_as_script=True you will need to provide local_script_path=
l
Laura Vaida
02/26/2021, 5:07 PMno i dont think
Laura Vaida
02/26/2021, 5:08 PMTraceback (most recent call last):
File "C:/Users/laura.vaida.000/Prefect/Billwerk/Billwerk_Contracts.py", line 175, in <module>
flow.storage= Docker(registry_url="gcr.io/keller-steering-enabling/flows", image_name="billwerk-contracts",python_dependencies=["pandas"],
File "C:\Users\laura.vaida.000\anaconda3\envs\prefect\lib\site-packages\prefect\storage\docker.py", line 222, in init
super().__init__(stored_as_script=stored_as_script, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'local_script_path'
Laura Vaida
02/26/2021, 5:08 PMLaura Vaida
02/26/2021, 5:08 PMi use container registry
j
J. Martins
02/26/2021, 5:11 PMIf you want to store the flows in the container as a script check Michaels answer
J. Martins
02/26/2021, 5:12 PMotherwise you can skip setting stored_as_script
l
Laura Vaida
02/26/2021, 5:12 PMah okey didn't get that at the first moment
Laura Vaida
02/26/2021, 5:12 PMthanks! 🙂
2 Views