https://prefect.io logo
Channels
Powered by
r

Ray Shi

01/13/2022, 4:33 AM
Hello, glad to join the Prefect community. I have a question on the flow that i registered on the UI, but it always shows this error and pointing to a directory different than my teammates. Just feel like on the UI it is using a different of the version of the getApplicant task. But it can be executed locally with no error. see screenshots below. What would be the cause for this? Thanks
k

Kevin Kho

01/13/2022, 4:49 AM
Hi @Ray Shi, did you re-register the flow after changing it?
r

Ray Shi

01/13/2022, 5:29 AM
Yes i did. It showed up as a new version on the UI.but still has same error.
k

Kevin Kho

01/13/2022, 5:40 AM
What storage are you using?
If it’s script based like Github, I think you didn’t update the version in Git?
r

Ray Shi

01/13/2022, 5:45 AM
um, i tripple checked that the tasks are up to date with our production codebase. And also checked diff with others in the company. pretty sure they are the same code. It is weird that after i register it to UI, it will error out.
k

Kevin Kho

01/13/2022, 5:48 AM
i think the only thing to help here is add more logging statements and see if they populate in your Flow?
Are you using Docker/Kubernetes for the Flow?
r

Ray Shi

01/13/2022, 5:59 AM
Yes we do use Docker images. But I was told that it would need update if there are new packages used. What are some of the checks I need to do with using Docker image?
k

Kevin Kho

01/13/2022, 6:02 AM
Do you use the 
DockerRun
and 
docker agent
with Prefect? Or do you just run 
local agent
inside a Docker container?
a

Anna Geller

01/13/2022, 10:00 AM
Can you share your storage and run config definition?
r

Ray Shi

01/13/2022, 4:58 PM
Copy code
STORAGE = S3(bucket="aven-etl-workflows")
RUN_CONFIG = ECSRun(
    task_definition_arn="prefect-ecs-ml-tasks:2",
    labels=["prod", "ecs"],
    run_task_kwargs=dict(
        cluster="prefect-ecs-tasks",
        launchType="FARGATE",
        networkConfiguration={
            'awsvpcConfiguration': {
                'subnets': [
                    'subnet-0a989b19518b3206e', 'subnet-0f893d51e54868283', 'subnet-0859b2ff39561515b',
                ],
                'securityGroups': [
                    'sg-0090032547625a7c2',
                ],
                'assignPublicIp': 'DISABLED'
            }
        },
    ),
)
k

Kevin Kho

01/13/2022, 5:03 PM
This looks pretty fine and should work as long you re-register. This just uses the default prefect image to run right? You don’t have any configuration on the agent side?
a

Anna Geller

01/13/2022, 5:07 PM
@Ray Shi it looks like some packaging issue. Are you sure the getApplicants is baked into the Docker image that you are using? What is your process for packaging dependencies and building the image?
Powered by