Adam
02/11/2021, 12:42 PMJosh Greenhalgh
02/11/2021, 12:52 PMJim Crist-Harif
02/11/2021, 3:51 PMI have many of the above aspects done but it’s all becoming quite a big mess.I'd be interested in hearing more about what issues you've run into. Could you comment more on this?
Alex Papanicolaou
02/11/2021, 5:15 PMale
02/11/2021, 8:38 PMBilly McMonagle
02/11/2021, 9:22 PMZanie
Jim Crist-Harif
02/11/2021, 9:33 PMthis thread is making me rethink our use of docker storage. I'd like to know the advantages of choosing docker vs github/s3 storage... it sounds like it's possible that github/s3/etc might result in much faster build times, since images would not need to rebuilt on every code change.We definitely want to write more docs on this, but I just pushed a few updates to our storage docs that you might find useful: https://docs.prefect.io/orchestration/flow_config/storage.html. See in particular: https://docs.prefect.io/orchestration/flow_config/storage.html#choosing-a-storage-class
Billy McMonagle
02/11/2021, 9:46 PMJim Crist-Harif
02/11/2021, 9:47 PMModule
storage + a custom docker image would be cleaner.Billy McMonagle
02/11/2021, 9:47 PMZanie
DockerRun
feels cleaner. I think DockerStorage
is a good way for a beginner to create an environment for their flow, especially since you can specify your python requirements and such using documented kwargs. It starts to feel heavy at the project-level though.Alex Papanicolaou
02/11/2021, 10:30 PMZanie
FlowBuilder
at one point while exploring this as wellMichael Hadorn
02/12/2021, 8:39 AMAdam
02/12/2021, 12:35 PMZanie
Matthew Alhonte
02/12/2021, 7:32 PMAdam
03/04/2021, 1:55 PMZanie