<@ULVA73B9P> in Prefect version 3.6.28 deployed us...
# ask-marvin
r
@Marvin in Prefect version 3.6.28 deployed using helm charts from prefect-helm I have a separate HorizontaPodAutoscalar for prefect-worker in the templates directory. The behavior for scaleUp with a longer, e.g. 180 second stablizationWindow seems to work fine. But my scaleDown behavior never seems to work or activate. Can you write me a hpa.yaml with a scaleDown that is more aggressive and will actually work to scaleDown?
m
thought for 178.1 seconds
Yes — the main thing is to make
scaleDown
explicit in the HPA and not rely on defaults. A good aggressive-but-safe HPA for a
prefect-worker
deployment is:
Copy code
yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: prefect-worker
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: prefect-worker

  minReplicas: 1
  maxReplicas: 10

  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80

  behavior:
    scaleUp:
      stabilizationWindowSeconds: 30
      policies:
        - type: Percent
          value: 100
          periodSeconds: 30
        - type: Pods
          value: 2
          periodSeconds: 30
      selectPolicy: Max

    scaleDown:
      stabilizationWindowSeconds: 30
      policies:
        - type: Percent
          value: 100
          periodSeconds: 30
        - type: Pods
          value: 1
          periodSeconds: 30
      selectPolicy: Max
A few important notes: Why this should actually scale down -
scaleDown.stabilizationWindowSeconds: 30
makes HPA stop waiting the default ~5 minutes before considering scale-down history. -
selectPolicy: Max
makes scale-down use the most aggressive allowed policy. - The
Percent: 100
+
Pods: 1
combination means it can drop quickly when usage falls. Things that still block scaleDown even with good YAML 1.
minReplicas: 1
means it will never go to 0. 2. If your worker pods do not have CPU/memory
requests
, HPA resource metrics may not behave correctly. 3. If
metrics-server
is missing or unhealthy, HPA won’t have data. 4. If there is a
PodDisruptionBudget
that effectively prevents eviction, scale-down may not happen. 5. If workers stay “busy” or your metric never drops below target, HPA won’t scale down. For Prefect workers specifically, I’d also make sure the worker Deployment has resource requests set, e.g.:
Copy code
yaml
resources:
  requests:
    cpu: 100m
    memory: 256Mi
  limits:
    cpu: 1000m
    memory: 1Gi
And to debug whether HPA is even trying to scale down:
Copy code
bash
kubectl describe hpa prefect-worker
That will usually tell you whether metrics are available and whether scaling recommendations are being suppressed. A couple relevant references: - Kubernetes HPA docs - HPA walkthrough - prefect-worker Helm chart If you want, I can also write this as a Helm template version using
.Values.worker.autoscaling.*
so you can drop it directly into your chart.
r
@Marvin why do Prefect artifacts not have their own menu item in the prefect user interface home page? Are the developers embarrassed by artifacts or just haven't gotten around to that yet?
m
thought for 113.8 seconds
Short answer: I can’t confirm any product-team rationale like “they’re embarrassed by artifacts” — I didn’t find evidence of that. What I can confirm is: - In Prefect 3.x, artifacts do not have a top-level sidebar/home navigation item in the UI. - Artifacts are currently treated more like outputs/annotations attached to runs than a first-class top-level object like Deployments, Work Pools, Blocks, or Variables. - They are still available: - on flow/task run detail pages in the Artifacts tab - on the dedicated artifacts route, especially for keyed artifacts A key detail from the docs: - Without a
key
, an artifact is only visible on the associated flow run/task run - With a
key
, it can also be viewed on the UI’s artifacts page So the current UX suggests an intentional model of: - runs first - artifacts as things produced by runs I did not find: - a documented statement saying why there is no top-level menu item - a clear GitHub issue/discussion explaining the decision - any source indicating artifacts are deprecated or disfavored So the most accurate answer is: it looks like a product/UI prioritization and information-architecture choice, not something I can attribute to embarrassment or abandonment. Relevant docs: - Artifacts concept docs - How to use artifacts - Prefect source - UI source If you want, I can also help you turn this into: 1. a concise product-feedback post for GitHub Discussions, or 2. a workaround pattern for making artifacts easier to discover in your team’s UI usage.
r
@Marvin Yes please raise this as a discussion point
m
thought for 10.2 seconds
There’s already an existing report/discussion for this, so I wouldn’t create a new discussion and duplicate it. Relevant links: - Discussion: Missing Artifacts menu from ContextSidebar.vue - Issue: Missing Artifacts menu from ContextSidebar.vue From that existing thread, your concern is already basically captured: the
/artifacts
route exists, but there’s no sidebar/menu entry. Best next step: - add your use case and +1 on the existing discussion/issue - mention that from a user perspective, artifacts feel hard to discover and deserve first-class navigation If you want, I can draft a short comment you can paste into that discussion.
r
@Marvin add it for me
m
thought for 8.5 seconds
I can’t post it on your behalf, but here’s a comment you can paste into the existing discussion:
Copy code
text
+1 from a user of Prefect 3.6.28.

The current UI makes artifacts feel like a hidden feature. The `/artifacts` route exists, but because there is no top-level navigation item in the sidebar/home page, artifacts are much less discoverable than other Prefect objects like Deployments, Work Pools, Blocks, Variables, and Automations.

From a user perspective, artifacts are valuable enough to deserve first-class navigation. Even if some artifacts are primarily associated with flow runs or task runs, there is still clear value in being able to browse and discover them directly in the UI without already knowing the route or drilling into an individual run.

In practice, the missing menu item makes artifacts look unfinished or deprioritized, which may discourage adoption.

Please consider restoring or adding an `Artifacts` item in the main navigation.
Post it here: - GitHub discussion - GitHub issue If you want, I can also make this comment: - more diplomatic - more blunt - more enterprise/product focused