Jeff Rose
01/16/2025, 5:12 PMNate
01/16/2025, 6:21 PMNate
01/16/2025, 6:23 PMcreate_work_pool
but again, the simplest way to create a work pool is the CLI via prefect work-pool create ...
Jeff Rose
01/16/2025, 6:32 PMNate
01/16/2025, 6:35 PMprefect deploy
Jeff Rose
01/16/2025, 6:36 PMNate
01/16/2025, 6:37 PMjob_variables
dict that is the same thing as the job_variables
section of a yaml deployment definitionJeff Rose
01/16/2025, 6:37 PMDeployment.build_from_flow
Jeff Rose
01/16/2025, 6:37 PMJeff Rose
01/16/2025, 6:38 PMinfrastructure=Block.load(f"ecs-task/{service}-{env}"),
Jeff Rose
01/16/2025, 6:38 PMJeff Rose
01/16/2025, 6:39 PMstorage=Block.load("s3/flow-storage"),
also have this param if that mattersJeff Rose
01/16/2025, 6:39 PMNate
01/16/2025, 6:42 PMDeployment.build_from_flow
with some_flow.from_source(source=..., entrypoint=...).deploy(work_pool_name=.., etc)
• convert your infra blocks to work pools (EcsTask block -> ECSWorker work pool, see this auto converter thing called publish_as_work_pool
)
that is to say, work pools replace "infra blocks", from_source
replaces "storage blocks"
might be helpfulJeff Rose
01/16/2025, 6:45 PMJeff Rose
01/16/2025, 6:45 PMJeff Rose
01/16/2025, 7:09 PMJeff Rose
01/16/2025, 7:10 PMNate
01/16/2025, 7:15 PMpip install 'prefect<3'
then you're definitely goodJeff Rose
01/16/2025, 7:46 PMJeff Rose
01/21/2025, 7:43 PMJeff Rose
01/21/2025, 7:47 PMprefect worker start --pool "rn-etl-services-work-pool-${env}"