Michal Mucha
06/02/2021, 3:54 PMJohn Ramirez
06/02/2021, 4:51 PMjcozar
06/02/2021, 5:54 PMrun_task_kwargs
in the ECSRun configuration definition.
• The arguments cpu
and memory
from the ECSRun configuration definition seems is not working. Even if I use valid values (cpu=256, memory=512) it creates the TaskDefinition using 1vcpu and 2GB RAM (I think it is the default configuration). I tried run_task_kwargs
with the keyword overrides
but it does not work neither. Do you know what I am doing wrong?
Thank you very much!!Jason Prado
06/02/2021, 7:42 PMcreateFlowRun
?
2. Is there anything like this around already? Would love to not reinvent the wheel and deploy a new service.Zach Schumacher
06/02/2021, 8:06 PMChris Leber
06/02/2021, 8:52 PMJustin Chavez
06/02/2021, 9:46 PMfrom typing import Dict, List
from prefect import task, Flow
@task
def generate_user_status_list() -> List[Dict]:
return [
{
"user": "User 1",
"status": "active"
},
{
"user": "User 2",
"status": "inactive"
}
]
@task(log_stdout=True)
def print_user_status(user: str, status: str):
print(f"Status of {user} is {status}")
with Flow('check user status') as flow:
user_status_list = generate_user_status_list
print_user_status.map(user_status_list) # <<<< Fails here as it doesn't recognize the parameters in the dictionaries
flow.run()
It's the second to last line that I am trying to figure out. I want each member of the generated list in the first task to be the parameters to the second. It's like I am trying to do the parameterized flow run in the StartFlowRun section, except I want to run a task instead of a flow with the parameters: https://docs.prefect.io/core/idioms/flow-to-flow.html#running-a-parametrized-flow
I can get around this by creating separate tasks that create each parameter as a list of str, but was hoping for a simple all in one task that generates the params for some of my more complex downstream tasks.Michael Law
06/03/2021, 7:32 AMStéphan Taljaard
06/03/2021, 8:24 AMRanu Goldan
06/03/2021, 1:22 PMagent:k8s-prod
to 100
But I’ve seen the maximum of concurrent run is only 22-24, the other runs is scheduled
I have 1 kubernetes agent deployment on my k8s cluster
How to make the flow run concurrency more than 24? Anything I miss?
Thanks in advanceJulio Venegas
06/03/2021, 1:45 PMZach Schumacher
06/03/2021, 3:40 PMbral
06/03/2021, 5:46 PMCharles Liu
06/03/2021, 6:53 PMJason Prado
06/03/2021, 7:33 PMPostgresFetch
.
db_host = PrefectSecret('db_host')
fetch = PostgresFetch(
host=db_host,
...,
)
I can’t do this because UserWarning: A Task was passed as an argument to PostgresFetch, you likely want to first initialize PostgresFetch with any static (non-Task)...
. Does this mean I need to either put the DB host in my code or get it from another method like an environment variable? Why is host
part of the static args to PostgresFetch
if I’m likely to extract it from a Secret?Pedro Machado
06/03/2021, 7:49 PM--show-flow-logs
. It looks like the flow is not communicating with Prefect Cloud.
The flow is running an R script. I am using the rocker/r-ver:4.1.0
image as a base and installing python 3.7 via apt-get
. I suspect the issue is related to this. Maybe I am not installing python correctly. See my docker file in the thread. Thanks!Hugo Kitano
06/03/2021, 10:17 PMECSRun
functionality could work, but it is designed for running flows, and this would necessitate flows in flows which could be a little tricky. What would be the best way to reconfigure a Fargate task as a Prefect task?DK
06/04/2021, 1:49 AMWai Kiat Tan
06/04/2021, 6:30 AMash
06/04/2021, 9:04 AMPrabin Mehta
06/04/2021, 9:05 AMrunning flows through rest API.
I want to run a flow passing config with the request call. Can anyone help me on this?Dali
06/04/2021, 10:37 AMMarwan Sarieddine
06/04/2021, 2:12 PMHugo Kitano
06/04/2021, 5:18 PMFina Silva-Santisteban
06/04/2021, 5:34 PMHamza Ahmed
06/04/2021, 5:45 PMimagePullSecrets
value in the prefecthq/prefect-server
helm chart?
Tried multiple ways, but none seem to work (examples in comments)Julio Venegas
06/04/2021, 11:23 PMMichael Law
06/04/2021, 11:58 PMOussama Louati
06/05/2021, 9:49 AMColin
06/05/2021, 3:45 PMColin
06/05/2021, 3:45 PMZach Schumacher
06/05/2021, 9:29 PM