Hey when pinning my helm deployed prefect to 0 14 0 I notice Prefect Community #prefect-server

Hey, when pinning my helm deployed prefect to 0.14...

Joël Luijmes

12/24/2020, 5:13 PM

Hey, when pinning my helm deployed prefect to 0.14.0, I noticed the tags 0.14.0 and core-0.14.0. As I’m running it myself I probably need the core- version. However, when I used that version for the Kubernetes Agent, it fails to start (see thread for logs).

👀 1

Joël Luijmes

12/24/2020, 5:13 PM

Copy code

[2020-12-24 17:08:11,786] DEBUG - agent | Verbose logs enabled                        
[2020-12-24 17:08:11,787] DEBUG - agent | Environment variables: []                   
[2020-12-24 17:08:11,787] DEBUG - agent | Max polls: None                             
[2020-12-24 17:08:11,787] DEBUG - agent | Agent address: <http://0.0.0.0:8080>          
[2020-12-24 17:08:11,787] DEBUG - agent | Log to Cloud: True                          
[2020-12-24 17:08:11,788] DEBUG - agent | Prefect backend: server                     
Traceback (most recent call last):         
  File "/usr/local/bin/prefect", line 8, in <module>                                                                                                                        
    sys.exit(cli())                                                                   
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 829, in __call__                                                                                        
    return self.main(*args, **kwargs)      
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 782, in main                                                                                            
    rv = self.invoke(ctx)                                                             
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke                                                                                         
    return _process_result(sub_ctx.command.invoke(sub_ctx))                           
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke                                                                                         
    return _process_result(sub_ctx.command.invoke(sub_ctx))                           
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke                                                                                         
    return _process_result(sub_ctx.command.invoke(sub_ctx))                           
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke                                                                                         
    return ctx.invoke(self.callback, **ctx.params)                                    
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 610, in invoke                                                                                          
    return callback(*args, **kwargs)                                                  
  File "/usr/local/lib/python3.8/site-packages/prefect/cli/agent.py", line 282, in start                                                                                    
    start_agent(KubernetesAgent, image_pull_secrets=image_pull_secrets, **kwargs)                                                                                           
  File "/usr/local/lib/python3.8/site-packages/prefect/cli/agent.py", line 108, in start_agent                                                                              
    agent = agent_cls(labels=labels, env_vars=env_vars, **kwargs)                     
  File "/usr/local/lib/python3.8/site-packages/prefect/agent/kubernetes/agent.py", line 120, in __init__                                                                    
    from kubernetes import client, config                                             
ModuleNotFoundError: No module named 'kubernetes'

Joël Luijmes

12/24/2020, 5:15 PM

I noticed on Docker Hub, that for most images the latest tag aligns with core-0.14.0. But with prefecthq/prefect it does not, then it aligns with 0.14.0

Chris White

12/24/2020, 5:38 PM

Hi @Joël Luijmes, the

core-0.14.0

tag only installs the base Prefect package with no additional extras; the

0.14.0

tag does install “orchestration extras” which are the standard additional packages required for Cloud deployments (e.g., kubernetes, boto3, etc.) We typically recommend using the 0.14.0 tag for agents and the

core-0.14.0

tag for the base image of your flows because it’s slimmer. I’ll open an issue to update our documentation to clarify this distinction!

Chris White

12/24/2020, 5:39 PM

@Marvin open “Expand Docker tag documentation to explain extra package installations”

Marvin

12/24/2020, 5:39 PM

https://github.com/PrefectHQ/prefect/issues/3894

Joël Luijmes

12/25/2020, 9:52 AM

So if I pin my version, I should use version 0.14.0? And if I were to manually build the flow images use the core-0.14.0?

Zanie

12/28/2020, 3:21 PM

I think that sounds right @Joël Luijmes — this will be a little easier once we start publishing the helm chart with pinned versions.

core-

just excludes the

all_extras

dependencies in the pip install, but the K8s agent needs the kubernetes extra so it can’t use

core-

. When writing your own flows that need Prefect but not all the extras, it makes sense to start with the

core-

base for Docker storage

👍 1

2 Views

Open in Slack

Previous Next