Has anyone gotten flows executing in a Dask Gatewa...
# prefect-community
t
Has anyone gotten flows executing in a Dask Gateway remote environment? based on the docs it looks like i'll be writing my own, but wanted to double check there isn't a way to do it currently without writing a new environment
j
@Thomas La Piana I haven't done it yet, but (1) I don't think it would be difficult and (2) you should be able to follow the pattern I used for
DaskCloudProviderEnvironment
in this PR #2360
(FWIW, we've been testing
DaskCloudProviderEnvironment
in a Flow scheduled to run every 15 minutes for the last week and it's been very reliable.)
t
thanks!
exactly what i was looking for 😄
👍 1
@Joe Schmid thanks a ton! rewriting that to work for DaskGateway was a piece of cake thanks to all of the heavy lifting you'd done already. got it up and running
j
@Thomas La Piana, glad to hear it. Also keep an eye on this issue: https://github.com/PrefectHQ/prefect/issues/2508 @Jim Crist-Harif had a great idea around decoupling Environments from Dask which I think makes a ton of sense. It's probably a bit longer-term, i.e. no need to stop your work on an Environment that uses Dask Gateway, but I suspect it'll be a better approach in the long run.
t
huge thanks for that link 🙂