I'm seeing some lag time when creating a Docker st...
# prefect-communitya
Adam Roderick
05/27/2020, 12:47 PMI'm seeing some lag time when creating a Docker storage object.
Copy code
INFO - prefect.Docker | Building the flow's Docker storage...n
nicholas
05/27/2020, 1:02 PM
Hi @Adam Roderick - if you start your agent with the
--verbosenicholas
05/27/2020, 1:03 PM
As for what could be causing the delay, my first guess is that Docker could be hitting some resource ceilings, I'm curious what that output will be though
nicholas
05/27/2020, 1:04 PM
It could also just be a very large image
nicholas
05/27/2020, 1:05 PM
Sorry, just realized I read that wrong, you're seeing that delay when building the storage at the registration step, right?
a
Adam Roderick
05/27/2020, 1:09 PM☝️ yes exactly that.
flow.storage = Docker(...Adam Roderick
05/27/2020, 1:10 PMthen
flow.register(...n
nicholas
05/27/2020, 1:15 PM
Ah ok, I don't think there's a good way to get insight into what's going on there; the code for that step is pretty straightforward. Which OS are you running?
a
Adam Roderick
05/27/2020, 1:16 PMWSL on Windows
Adam Roderick
05/27/2020, 1:16 PMIt does have a small amount of lag normally. But nothing like this
Adam Roderick
05/27/2020, 1:17 PMIt's Ubuntu 20.04 LTS
Adam Roderick
05/27/2020, 1:17 PMCould the code be downloading a large docker base image every time, rather than using the locally cached image?
Adam Roderick
05/27/2020, 1:17 PMPython:3.7 is the base image in my Dockerfile
n
nicholas
05/27/2020, 1:18 PM
That could be the issue but I think there's normally some pulling output if that's the case, and afaik the Python image isn't anything craz. For reference here's the build step for the Client.
nicholas
05/27/2020, 1:19 PM
Which version of Docker are you running?
a
Adam Roderick
05/27/2020, 1:20 PMDocker version 19.03.8, build afacb8b7f0
Adam Roderick
05/27/2020, 1:34 PMIt looks like WSL and docker combination is slow
n
nicholas
05/27/2020, 1:35 PM
Yeah I'm just reading some of the issues on GitHub about that, sorry for the delay
nicholas
05/27/2020, 1:35 PM
Unfortunately I don't think there's much Prefect can do to speed that up (or get insight into it) 😕
a
Adam Roderick
05/27/2020, 1:35 PMWhen I run a timeit on 'from prefect.environments.storage import Docker` on windows, it takes just under 1 second
Adam Roderick
05/27/2020, 1:35 PMWhen I run the same on WSL, it takes 10-13 seconds
Adam Roderick
05/27/2020, 1:36 PMI know that's not a prefect issue--thanks for looking into it
n
nicholas
05/27/2020, 1:36 PM
Wow, that's stark
a
Adam Roderick
05/27/2020, 1:36 PMIf you have any ideas now or in the future, I'm all ears
Adam Roderick
05/27/2020, 1:36 PMYeah--very stark. The other imports from Prefect have no difference
n
nicholas
05/27/2020, 1:38 PM
Oh one option is to use VSCode and its Remote WSL plugin, which'll store code in the WSL filesystem
nicholas
05/27/2020, 1:39 PM
Since it seems to be a networking issue
a
Adam Roderick
05/27/2020, 1:42 PMThat's the setup I am using
Adam Roderick
05/27/2020, 1:42 PMIt's a great option for development. But exhibits the same behavior
n
nicholas
05/27/2020, 1:43 PM
Oh no 😧
nicholas
05/27/2020, 1:47 PM
Well it sounds like the slowness was definitely on Docker's radar at some point, seems like networking was something they were looking to solve with WSL 2 😕 wish I could be of more help here, I'll keep you posted if I come across anything from the team.
a
Adam Roderick
05/27/2020, 1:53 PMThanks, I'll do the same