Hi, Should flows always be in a separate standalon...
# prefect-cloud
s
Hi, Should flows always be in a separate standalone file or can they be dynamically created within a function or a class and registered as a Deployment. I'm trying something like this just want some inputs on whether it would work or not or what is the right way to dynamically create flows. I have a process which tries to convert data extracted from different loaders and store it in a db. These loaders and destination db are dynamically configured by the users so Im trying something like this. Any help , guidance or advice is appreciated.
Copy code
class PrefectExecutor: 
   ## appropriate init method 
 
   def setup(self):
        """
        Create deployment flows for all the loaders
        """
        for loader in self.loaders:
            print(loader.config.id)
            self._build_and_deploy_loader(loader)

    def _build_and_deploy_loader(self, loader):
        try:
            from prefect import flow
            from prefect.deployments import Deployment
            from prefect.filesystems import LocalFileSystem
        except ImportError:
            print(
                """
                Prefect is not found. Install prefect with "pip install prefect==2.10.21"
                """
            )

        @flow
        def process_job(parameters: dict):
            print(parameters)
            stack = Stack(etl=loader, destination_db = self.destination_db)
            stack.etl.run(**parameters)

        deployment = Deployment.build_from_flow(
            process_job, name=str(loader.config.id), storage=LocalFileSystem.load("local")
        )
        deployment.apply()
n
hey @Sam Joel - have you checked out the `.serve()` method at all?
if you're using process infra and local filesystem (like your example above),
.serve()
is probably more or less almost exactly what you're doing, except you wouldn't need an agent, the served process would effectively do the agents job and then you could just trigger the deployment
s
@Nate Yeah I checked out the serve but underneath isn't the serve method also creating a deployment ? and the main doubt I had is that the flow uses in memory variables from its parent class and when I create a deployment it tries to store the code to a filesystem how does the remote flow execution work in that scenario. Since it might not have access to the in memory variables . Can you please shed some light on this ?
n
> isn't the serve method also creating a deployment yes! > the flow uses in memory variables from its parent class I'm not quite sure what you mean by that, but I can't think of a scenario where you can't get the objects your flow needs by importing and / or accepting them via flow arguments - do you have a different situation?