Dipen Chawla
08/24/2021, 5:36 PMArun Dass
08/24/2021, 10:11 PMRyan Smith
08/25/2021, 7:00 AMBruno Murino
08/25/2021, 9:01 AM502 Server Error: Bad Gateway for url: <http://prefect.landbay.internal:8081/api/graphql/graphql>
however I’m not sure there’s an issue with how I’ve configured the URLs as the flow runs 99% of time without raising this…..
For reference, these are the only 2 URLS I set:
PREFECT__SERVER__ENDPOINT=<https://prefect.prodenv.landbay.co.uk/api/graphql>
PREFECT__API__HOST=<https://prefect.prodenv.landbay.co.uk/api/graphql>
Pierre Monico
08/25/2021, 9:04 AMDockerRun
run config. However, since I deployed to the server the runs are not very reliable. I keep getting random errors in a totally non-deterministic way (cf. image).
The flows all run fine on my machine and were also running fine when executed in a Docker container I was managing myself (on cloud infrastructure). Some of the errors include:
• HTTPConnectionPool(host='host.docker.internal', port=4200): Read timed out. (read timeout=15)
• sqlalchemy.exc.NoSuchTableError
(I am writing to Postgres tables - the table does exist and sometimes the run even succeeds)
• TypeError: 'NoneType' object is not subscriptable
• No heartbeat detected from the remote task; retrying the run.This will be retry 1 of 3.
I believe most of the errors are related to some time out issue / the state of the task can’t be monitored but I am confused as to why this happens. My VM has sufficient resources (I checked the monitoring) but I am thinking it might be worth scaling it up?
I know the question might be a bit broad but would be happy to know if anyone experienced something similar and / or knows the reasons / a fix.Michael Hadorn
08/25/2021, 12:42 PMExpected type UUID!
see here: https://github.com/PrefectHQ/prefect/issues/4687
Can you fix this also in the server edition?Tom Forbes
08/25/2021, 1:59 PMTask 'xy[z]': Finished task run…
messages?Lawrence Finn
08/25/2021, 3:36 PMGaylord Cherencey
08/25/2021, 9:41 PMCOMING SOON - Task Affinity Run each of a flow's tasks in a completely different environment, including new dependencies or platforms.
How far this feature is and will this mean that we could define a Docker image per task for example?Rachel
08/25/2021, 10:16 PMsubprocess.CalledProcessError: Command '['docker-compose', 'pull']' returned non-zero exit status 15.
and later
subprocess.CalledProcessError: Command '['docker-compose', 'down']' returned non-zero exit status 15.
YD
08/25/2021, 10:51 PM@task(timeout=60)
def task_1():
x = func1(...)
return x == 0
@task(timeout=60)
def task_2(r):
y = func2(...)
return y == 0
@task(timeout=60)
def task_3():
z = func3(...)
return z == 0
def main():
with Flow("my-flow", executor=LocalDaskExecutor()) as flow:
r = task_1()
r = task_2(r)
r2 = task_3()
if task_1
times out and fail, task_2
will not run.
What can I change so that I can still use the LocalDaskExecutor
but allow task_2
to run even if task_1
failed ?Tom Tom
08/25/2021, 11:46 PMDockerRun(image="prefecthq/prefect")
with prefect agent docker start
i´m getting this Error:
Failed to load and execute Flow's environment: ModuleNotFoundError("No module named 'C'")
I know that i can´t import other dependencies but i only import following modules from prefect in the script:
from prefect import Flow, task
from prefect.run_configs import LocalRun,DockerRun
Or is it possible to run DockerRun()
in an existing Container?Sumit Kumar Rai
08/26/2021, 3:14 AMPierre Monico
08/26/2021, 8:40 AMModule
storage with stored_as_script=True
, why do all the flows and their dependencies need to be resolved?
e.g. all my flows import db
which is an instance of sqlalchemy.create_engine()
that I setup with an environment variable. If I try to register e.g. from a CI pipeline, I have to set all those variables just to be able to register a script that will be stored as a file anyway?
(NB: can’t use the CLI prefect register
because it is not able to resolve any of the dependencies of my flows (they are in a folder), spitting ModuleNotFoundError
)Rachel
08/26/2021, 4:05 PMHenry
08/27/2021, 2:10 AMCould not upgrade the database!
Error: Can't load plugin: sqlalchemy.dialects:postgres
Ben Collier
08/27/2021, 10:31 AMMarcus Hughes
08/27/2021, 3:42 PMWilson Bilkovich
08/27/2021, 6:37 PMERROR:asyncio:Fatal error on SSL transport
Serdar Tumgoren
08/27/2021, 6:52 PMMay
08/27/2021, 8:19 PMSomething went wrong while trying to fetch failed flow information. Please try refreshing your page. If this error persists, return to this page after a few minutes.
latest prefect-server deployment via helm - We get this error after 2 weeks of flows runs - now we cantt see our flows 😞Michael
08/30/2021, 8:14 AMCasey Green
08/30/2021, 8:38 PMmutation { archive_flow(...) { ... } }
doesn't appear to be valid anymore, I only see delete
mutations.Stanislav Molleker
08/31/2021, 3:50 PMMichail Melonas
09/01/2021, 2:25 PMsecret = PrefectSecret("SOME_SECRET")
flow.storage = S3(bucket="some-bucket", secrets=["AWS_CREDENTIALS"])
flow.run_config = KubernetesRun(
image="<http://registry.gitlab.com/<some_image>|registry.gitlab.com/<some_image>>",
labels=["prefect-14"],
env={
"PREFECT__LOGGING__LEVEL": "DEBUG",
"SECRET": secret
},
image_pull_secrets=["image-puller"],
)
and then running:
flow.register("ProjectName")
I get:
TypeError: Object of type PrefectSecret is not JSON serializable
Any thoughts on this?Jeevan Acharya
09/01/2021, 2:56 PMAlexander van Eck
09/02/2021, 9:55 AMvar: str = Parameter('var')
mypy complains with
Incompatible types in assignment (expression has type "Parameter", variable has type "str")
Even though after calling this will be a string 🤔
Any thoughts on how to achieve this?ash
09/02/2021, 10:05 AM1.) I have flow which gets data from an api and i want to publish it to slack directly, how should i do it without writing it anywhere.
2.) If i want to save my flow/task results on a machine from where I am registering my flows despite flow running on pods on k8 ,how should i do it.
Anton Chekmarev
09/02/2021, 12:06 PMAlfie
09/03/2021, 6:25 AM