Shuchita Tripathi
04/04/2022, 2:55 PMBernd Grolig
04/04/2022, 3:02 PMDonnchadh McAuliffe
04/04/2022, 3:52 PMDockerfile
for the agent:
FROM prefecthq/prefect:2.0b2-python3.9
RUN prefect cloud login --key cloud_api_key
ENTRYPOINT prefect agent start 'queue_id'
However, the second line requires me to specify the workspace - is there some flag I can add (like --workspace {workspace}
) ? There probably is a much better way to set up an agent, any other docs would be appreciated! Thank youjoshua mclellan
04/04/2022, 5:13 PMMadison Schott
04/04/2022, 5:21 PMAtul Anand
04/04/2022, 7:09 PMChris Reuter
04/04/2022, 7:36 PMEthan Veres
04/04/2022, 8:56 PMShiyu Gan
04/05/2022, 2:24 AMEddie Atkinson
04/05/2022, 3:39 AMShiyu Gan
04/05/2022, 4:10 AMJonathan Mathews
04/05/2022, 8:43 AMMuhammad Daniyal
04/05/2022, 10:36 AMdef fun1(): <some code here>
def fun2(): <some code here>
def fun3(): <some code here>
def fun4(): <some code here>
def fun5(): <some code here>
def fun6(): <some code here>
def execute_flow():
@task
def t1():
fun1()
@task
def t2():
fun2()
@task
def t3():
fun3()
@task
def t4():
fun4()
@task
def t5():
fun5()
@task
def t6():
fun6()
with Flow('my flow') as f:
a = t1()
b = t2()
t3(a)
t4(b)
c = t5()
d = t6(c)
output = f.run()
result = output.result[d]._result.value
return result
the expected behaviour of flow would be t1 -> t2 -> t3 -> t4 -> t5 -> t6
but it is not working this way
instead this is what happenning
t1 -> t5{fails but shows 'success'} -> t6{fails since result from t5 are not appropiate} -> t2 -> t3{shows success message without even going through half way of function} -> t4{same result as t3}
except t5 and t6, every task is time takingHawkar Mahmod
04/05/2022, 10:54 AMUnexpected error: TypeError("can't pickle sqlalchemy.cprocessors.UnicodeResultProcessor objects")
Christian
04/05/2022, 10:59 AMJonathan Mathews
04/05/2022, 11:00 AMprefect auth login --key <replaced with my key>
I get the following error:
Unauthorized. Invalid Prefect Cloud API key.
The cloud UI says my key has been createdPatrick Koch
04/05/2022, 2:35 PM└── 14:27:40 | INFO | Entered state <Failed>: Failed to load and execute Flow's environment: ModuleNotFoundError("No module named '/home/prefect-flow-example'")
Flow run failed!
The file "prefect-flow-example.py" is available at my local host, I've set the proper ENV variables for the connection to the Prefect-Server.
Do I need to define some Storage Options before? An Azure subscription would be available.
I'm curious about what you would suggest as best practice 🙂
Thank you a lot in advance!
PatrickMatthew Roeschke
04/05/2022, 3:10 PMprefect get flows
cli command?Matt Brown
04/05/2022, 3:25 PMKen Nguyen
04/05/2022, 3:43 PMschema
within the profiles.yml), and have the dbt task within that flow output to a schema of the user’s choice.Jonathan Mathews
04/05/2022, 3:50 PMDbtShellTask
because dbt wouldn’t have been installed). It’s coming back with an error where I need to specify a execution_role_arn
in the ECSRun
which I’ve now done, but can I use the previously defined role, or do I need to define something else? Thanks!Rhys Mansal
04/05/2022, 6:24 PMFina Silva-Santisteban
04/05/2022, 8:27 PMJonathan Mathews
04/05/2022, 8:36 PMRhys Mansal
04/05/2022, 9:29 PMAn error occurred (ClientException) when calling the RegisterTaskDefinition operation: Invalid setting for container 'flow'. At least one of 'memory' or 'memoryReservation' must be specified.
I have set memory (and cpu) in the ECSRun task_definition (both quoted and unquoted) and still get this error. Does anyone know what I might try?Matt Alhonte
04/05/2022, 11:17 PMShiyu Gan
04/06/2022, 1:29 AMShiyu Gan
04/06/2022, 5:54 AMShiyu Gan
04/06/2022, 6:32 AM*prefect.core.flow.Flow.run_agent*
do? It's unclear from documentation and src code, which seems to say it runs an Agent from inside a Flow, which might be confusing since I was under the impression that Agent runs Flows, not the other way around.Shiyu Gan
04/06/2022, 6:46 AMservices
enumerated on this page still update to date and complete?