https://prefect.io logo
Join Slack
Powered by
Prod does not really have much in it. Ideally I'd ...
# ask-community
r
Prod does not really have much in it. Ideally I'd just like to use the command line to tell prefect server to load up the flow in the existing docker container without the flow present outside of it.
j
You can do this with the new 
prefect build
and 
prefect register
CLIs. Build will build your flow(s) (building any images, pushing source to S3, etc...) and write the output to a JSON file. This file can later be used with 
prefect register
with the 
--json
flag to register the flows in a project (or multiple projects) without requiring access to the flow source itself.
r
I wish I saw this. Interestingly I am using docker storage and the normal build process. I now am running my container in once off to 'self-register' I worked out how to load the cloud pickled flow and register it again with the container I have made. That works perfectly outside the container but I want to have the container do it itself. Now I am trying to use the client api and to do a register of that cloud pickled flow
and getting: GRAPHQL_VALIDATION_FAILED: Cannot query field "user" on type "Query".
inside the container when I try to talk to the prefect API.
I guess if I can't use my code to do what you said from inside the container the command line stuff will suffer the same fate. If I could run it in the container I think I could also use the command line thing above and have a self registering container with the same technique
Anyway, I'll go back to trying your advice.
Ideally I'd like everything inside the docker image but that said I could maybe copy stuff out with an echo at registration time.
wow, that's new new new 🙂
giving it a try now
My docker file has quite a lot of tasks so I want it in a separate common file. Is there a way to allow a flow to import a local module in: with prefect.context({"loading_flow": True, "local_script_path": path}): exec(contents, namespace) It seems the flow is loaded into a string and execed with a prefect context but I am not sure where.
ah got it, moving on, never mind me
my only issue is I have to:
Copy code
import sys, os
sys.path.append(os.path.abspath(os.getcwd()))
in my flow to get the code that is used to build the lists of files and requirements in my custom docker storage as the prefect.context wrapped python 'exec' does not have globals passed. I do appreciate the loading_flow so I know I am loading and not running.
j
in my flow to get the code that is used to build the lists of files and requirements in my custom docker storage as the prefect.context wrapped python 'exec' does not have globals passed.
Ah, nice catch, can fix.
I'm having a hard time following the above thread, are there other issues you still have that are outstanding?
r
Thanks for adding the local directory. I switched over to the cli way. Now you fixed adding the local directory I have nothing outstanding. All working working (pre your fix). I'll go back and take those lines out. Next thing for me is to make a prefect task to register the flow. I can use an exec to run the command but I think I'll try and use the library.
Open in Slack
PreviousNext
Powered by