I'm currently working on using the prefect API thr...
# ask-community
j
I'm currently working on using the prefect API through an iPython environment (Jupyter Notebook atm). Because my environment is heavily containerized, I'm looking at using the Webhook Storage, since that'll let me reuse a bunch of existing infrastructure. Am I correct in my reading of the source that if I want to store the flow as a script (i.e. not have it pickled), I would need to write a file out temporarily to local storage, if the flow is some python functions defined in a cell?
f
Hi Joseph, you might want to use an S3 storage to save your flow code. You can get an example here.
👍 1
j
Unfortunately I don't have any S3 storage since I'm running on a physical host in my datacenter, and not a cloud environment
f
Can't you instantiate a MinIO container ?
k
Hi @Joe! Maybe you can try
stored_as_script=True
in your Webhook storage. Docs here
j
I probably could, but I'd like to avoid adding even more different stuff to the overall system. using the existing stuff is a lot better.
Yeah, there's a setting for it, but, here's an excerpt from webhook:
Copy code
for flow_name, flow in self._flows.items():
            <http://self.logger.info|self.logger.info>("Uploading flow '{}'".format(flow_name))

            data = flow_to_bytes_pickle(flow)
            if self.stored_as_script:

                # these checks are here in build() instead of the constructor
                # so that serialization and deserialization of flows doesnot fail
                if not self.flow_script_path:
                    msg = "flow_script_path must be provided if stored_as_script=True"
                    self.logger.critical(msg)
                    raise RuntimeError(msg)

                if not os.path.isfile(self.flow_script_path):
                    msg = "file '{}' passed to flow_script_path does not exist".format(
                        self.flow_script_path
                    )
                    self.logger.critical(msg)
                    raise RuntimeError(msg)

                with open(self.flow_script_path, "r") as f:
                    data = f.read().encode("utf-8")

            req_function = self._method_to_function[self.build_request_http_method]
which looks like I need a preexisting local file to use to do that.
k
I see what you mean. Let me look into it a bit more.
Yeah I’m honestly not seeing other way. I will double check with the team though.
Confirmed that you need to create a file.
j
Thanks, now to figure out if there's cell magic for this.