Kyle McChesney
10/14/2022, 4:37 PMdef flow_updater(flow, current_state, next_state):
if isinstance(next_state, Failed):
error_information = {k.name: v.message for k, v in next_state.result.items() if type(v) is Failed}
else:
error_information = None
report(error_information)
When I run this flow locally using prefect run -p path/to/flow.py
with a simulated error, I get the expected error information, however when I run it on our deployment (not cloud, backend in AWS). I get error_information equal to {}
(i.e. no task results have a result of type Failed
). Is there some other type that gets used when running connected to server? (I am using type(v) is Failed
instead of isinstance
cause I don want to report TriggerFailed
, etc)Pedro Henrique
10/14/2022, 4:39 PMPedro Henrique
10/14/2022, 4:41 PMPedro Henrique
10/14/2022, 4:41 PMJosh
10/14/2022, 4:48 PMJosh Paulin
10/14/2022, 5:16 PMtask_input_hash
as the cache_key_fn
?
On step 3 I’m seeing the flow fail that it can’t find the cache file…Nace Plesko
10/14/2022, 5:30 PMdatabase
and a few files in it and all those work fine, but now that I created a new one, the run on the Prefect is saying that it can't find it. I'm probably just missing something obvious, but because of my lack of knowledge of both Python and Prefect I'd really appreciate help on this. Thank you in advance!Josh Paulin
10/14/2022, 5:54 PMNone
. Wondering if this is intentional. Minimal example in the thread.
Also found out I can’t downgrade from 2.6 to 2.5 because of what looks like some database migrations? Running prefect orion database reset -y
on a database with 2.5 after upgrading to 2.6 and it fails.redsquare
10/14/2022, 7:54 PMTaylor Babin
10/14/2022, 8:32 PMOmar Sultan
10/15/2022, 8:41 AMError: Got unexpected extra argument (prefect.orion.api.server:create_app)
Sander
10/15/2022, 10:16 AMKostiantyn Liepieshov
10/15/2022, 10:34 AMAndrei Tulbure
10/15/2022, 3:11 PMFrancesco Bartoli
10/15/2022, 4:29 PMcurl -X 'POST' \
'<http://prefect-server:4200/api/block_types/install_system_block_types>' \
-H 'accept: application/json' \
-H 'x-prefect-api-version: 2.6.1' \
-d ''
I’m getting few of them available from that page, the Remote File System one doesn’t come up though. Is this expected with the latest version?Sander
10/15/2022, 8:02 PMMarcos
10/16/2022, 12:51 AM.map
calls, if one task fails in the first map, then all dependent tasks stay in NotReady state. Example flow in the commentsChristopher
10/16/2022, 2:59 PMFady Khallaf
10/16/2022, 3:31 PMapiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: prefect
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 3Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
namespace: prefect
name: orion
spec:
selector:
matchLabels:
app: orion
replicas: 1
template:
metadata:
labels:
app: orion
spec:
containers:
- name: api
image: prefecthq/prefect:2.6.0-python3.10
command: ["prefect", "orion", "start", "--host", "0.0.0.0", "--log-level", "WARNING"]
ports:
- containerPort: 4200
env:
- name: PREFECT_API_URL
value: '<https://mydomain/api>'
volumeMounts:
- mountPath: ~/.prefect
name: prefect
restartPolicy: Always
volumes:
- name: prefect
persistentVolumeClaim:
claimName: prefect
Omar Sultan
10/16/2022, 4:09 PMchicago-joe
10/16/2022, 5:26 PMchicago-joe
10/16/2022, 10:27 PMFile "/usr/local/lib/python3.9/site-packages/prefect/utilities/dispatch.py", line 186, in lookup_type
raise KeyError(
KeyError: "No class found for dispatch key 'ecs-task' in registry for type 'Block'."
Santhosh Solomon (Fluffy)
10/17/2022, 3:25 AMYD
10/17/2022, 4:14 AMFinished task run for task with final state: 'TimedOut'
after 3 second the task is running
even though the task is set to @task(timeout=600, **params)
and it was running fine till friday.
it also runs fine when using flow.run()
I am using https://cloud.prefect.io/
any suggestions?
I deleted the flow an reregister it, but it did not helpDarren Fleetwood
10/17/2022, 4:58 AMMark
10/17/2022, 7:54 AMSULEMAN KHAN
10/17/2022, 8:43 AMcache
when running it with celery using this command.
celery -A app.tasks.celery_app worker --max-tasks-per-child=1 --loglevel=ERROR
Everything works fine when I use this command
celery -A app.tasks.celery_app worker --loglevel=ERROR
Jared Robbins
10/17/2022, 1:00 PMvk
10/17/2022, 1:02 PMflow.storage = Module('samples.flow_orion')
Now i’m kind of forced to specify exact full filename, even though I have file in python path.
That’s very inconvenient for cases when developers have no control over the final image layout.
That’s how I create deployments now:
entrypoint = 'samples.flow_orion:entrypoint' # this is just module, available on path
flow = prefect.utilities.importtools.import_object(entrypoint)
# here is path/entrypoint i need to specify path. could I use module name instead?
deployment = Deployment.build_from_flow(
name=flow.name,
flow=flow,
work_queue_name="kubernetes",
skip_upload=True,
infrastructure=KubernetesJob(
image=image,
namespace='prefect2',
job=k8s_template_orion(flow)
),
path='/app/lib/python3.9/site-packages/samples/', # TODO
entrypoint='flow_orion.py:entrypoint'
)
deployment.apply()
Did i miss how to use module name instead of path?Nic
10/17/2022, 1:42 PM