https://prefect.io logo
Title
j

Jeff Brainerd

03/11/2020, 11:08 PM
Hi Prefecters, I’m struggling with a peculiarity of Django, but I’m wondering if the problem is somewhat more general. The basic issue is that I need to initialize django before importing any ORM objects that many of my prefect tasks use. When Dask is running a task, I’m able to use an initialization hook to call that django function before the flow is deserialized with cloudpickle. The problem comes when building the prefect docker image -- the healthcheck fails during cloudpickle load. I’ve done some code spelunking and I can’t see an obvious way to override or provide a hook to the healthcheck script short of just hacking it (which I’m doing just to get it to pass). So I’m interested if there’s an existing workaround, or if this problem has come up before and if anyone else would benefit from some kind of mechanism for overriding, disabling or providing some kind of pre-run hook for the healthcheck script. Thanks all 🙏
c

Chris White

03/12/2020, 1:10 AM
Hey @Jeff Brainerd - this is super interesting. First off, if you’d like to open an issue for a
ignore_healthchecks: bool
flag on
Docker
storage we’d be happy to get that in before the next release. Out of curiosity, is it too clunky to import the ORM objects within the task definition (as opposed to the top of the file)?
j

Jeremiah

03/12/2020, 1:24 AM
@Marvin archive “How can I skip healthchecks with Docker storage?”
j

Jeremiah

03/12/2020, 1:25 AM
@Jeff Brainerd I’ve got a PR ready that will add the keyword argument Chris just described
j

Jeff Brainerd

03/12/2020, 1:50 PM
Thanks @Jeremiah and @Chris White for the fast response! That’s a great help.
👍 1
And to your question above, I’ve played around with adding the call to django as you suggest but it is rather onerous, as it has to be added in many source files (most of which are used by other non-prefect functions). It’s possible there’s an elegant way to call it but I haven’t found it.
👍 1