Hello again! Is it possible to define the storage ...
# prefect-server
w
Hello again! Is it possible to define the storage within the flow itself? For example:
k
Hey @William Clark it is not. Could you tell me more about what you are trying to achieve? If you get the chance, could you move the code snippet into the thread to now crowd the main channel?
I see you want to provide a parameterized task definition to ECSRun?
w
Sorry about that Kevin! Yes exactly!
I'm thinking about creating a global variable that will will store the parameterized path and then pass that variable to the run_config. Do you think that will work?
k
I am not seeing a way this can work because the RunConfig is serialized on the backend as well. It’s not a run-time thing that can be changed. I think Orion will give more flexibility around these thogh
a
btw, @William Clark we’ve recently added many examples to ECSRun in this docstring. Perhaps one of those options is what you’re looking for?
the reason I’m mentioning that is that I see that you’re manually loading task definition from S3 - there’s no need to do that if you provide a path to a YAML task definition that you stored on S3, e.g.:
Copy code
flow.run_config = ECSRun(
    task_definition="<s3://bucket/flow_task_definition.yaml>",
    image="<http://XXXX.dkr.ecr.us-east-1.amazonaws.com/image_name:latest|XXXX.dkr.ecr.us-east-1.amazonaws.com/image_name:latest>",
    cpu="2 vcpu",
)
this is how an example task definition YAML can look like:
Copy code
family: prefectFlow
requiresCompatibilities:
    - FARGATE
networkMode: awsvpc
cpu: 1024
memory: 2048
taskRoleArn: arn:aws:iam::XXX:role/prefectTaskRole
executionRoleArn: arn:aws:iam::XXX:role/prefectECSAgentTaskExecutionRole
containerDefinitions:
- name: flow
    image: "<http://XXX.dkr.ecr.us-east-1.amazonaws.com/image_name:latest|XXX.dkr.ecr.us-east-1.amazonaws.com/image_name:latest>"
    essential: true
    environment:
    - name: AWS_RETRY_MODE
        value: "adaptive"
    - name: AWS_MAX_ATTEMPTS
        value: "10"
    logConfiguration:
    logDriver: awslogs
    options:
        awslogs-group: "/ecs/prefectEcsAgent"
        awslogs-region: "us-east-1"
        awslogs-stream-prefix: "ecs"
        awslogs-create-group: "true"