< Marvin> Where is prefect worker logs at Prefect Community #ask-marvin

Join Slack

<@ULVA73B9P> Where is prefect worker logs at

# ask-marvin

Klashe

10/25/2024, 4:12 AM

@Marvin Where is prefect worker logs at

Marvin

10/25/2024, 4:12 AM

To assist you accurately, could you please confirm which version of Prefect you are using? Prefect 2.x or 3.x? The method to access worker logs may vary between these versions.

Klashe

10/25/2024, 4:13 AM

Prefect 3.x . I'm looking for the linux logfiles of the work pool infrastructure

Nate

10/25/2024, 4:14 AM

hey @Klashe - you have to tag marvin each time if you want it to see it but to answer the question, its just logged to stdout wherever you say

prefect worker start

unless you otherwise configure / redirect to some log file

Klashe

10/25/2024, 4:15 AM

Thank you!

👍 1

Klashe

10/25/2024, 4:19 AM

@Nate, Does the work pool automatically pull deployment images from my local docker unauthenticate registry? Or do I have to configure the pool to do it? I see there's a configuration for authenticated docker registries, but unsure of unauthenticated.

Nate

10/25/2024, 4:47 AM

its mostly up to the docker client, so if you say your deployment's image is

some-local-image

and it can find that image locally, it will pull that if you have

image

zzstoatzz/some-dockerhub-repo:latest

then the docker client knows to pull from dockerhub, but you'd need some

docker login

so that process can pull from the repo for example, when people deploy a worker as an ECS service, it needs to have the right IAM permissions to pull from ECR - does that make sense? you can setup a docker work pool to use some registry credentials, but some big orgs for example dont trust us to store those credentials so they put them where the

prefect worker start

process can use them client-side

Klashe

10/25/2024, 4:50 AM

Makes sense. I actually have a local docker registry (no login) at 192.168.0.1:5000 and I name my images 192.168.0.15000/prefect etllatest . Will it pull those automatically?

Nate

10/25/2024, 4:52 AM

it should yeah, but i half remember some edge case with local registries - if thats not working can you try using a tag other than

latest

🙌 1

Nate

10/25/2024, 4:52 AM

if that doesnt work feel free to DM details and I can look tomorrow

Klashe

10/25/2024, 4:56 AM

I will try. Thank you so much for your assistance tonight!

good luck 1

👍 1

Klashe

10/26/2024, 5:33 AM

@Nate, I figured it out. I didn't have in the "work_pool" block a "job_variable" with "image" in my prefect.yaml. So it kept pulling the latest prefect image from dockerhub rather than my custom dockerfile . I fixed it with this:

Copy code

work_pool:
    name: test_pool
    work_queue_name: default
    job_variables: 
      image: "{{ build-image.image }}"      <-------

Thanks again for all your help.

Nate

10/26/2024, 6:36 PM

aha! makes sense

Nate

10/26/2024, 6:36 PM

glad you figured it out catjam

Open in Slack

Previous Next