Nathaniel Russell
10/13/2022, 7:26 PMMichał Augoff
10/13/2022, 9:29 PMProjects
abstraction from Prefect 1? I found it useful to be able to group flows and check system dashboard per project. Or would you recommend to just use tags and custom views instead? (which is only possible with flow runs, not with flows or deployments)Santhosh Solomon (Fluffy)
10/14/2022, 3:44 AMZac Hooper
10/14/2022, 4:49 AMMalavika S Menon
10/14/2022, 8:41 AMwonsun
10/14/2022, 9:11 AMSimon Macklin
10/14/2022, 10:01 AMRomain
10/14/2022, 11:40 AMapply_map
. Is is correct to assume that I can easily change my apply_map uses, by subflow runned in parallel?John Mizerany
10/14/2022, 1:44 PMSuccessfully registered 0 blocks
and the block is not created in the Cloud UI. I have been following the instructions given in the collections documentation and the prefect-dbt githubIlya Galperin
10/14/2022, 3:03 PM<https://app.prefect.cloud/account/ACCOUNTID/workspace/WORKSPACEID/flow-runs/flow-run/FLOWRUNID>
while others use:
<https://app.prefect.cloud/account/ACCOUNTID/workspace/WORKSPACEID/flow-run/FLOWRUNID>
Where the first example has flow-runs/flow-run
and the second just uses a flow-run
format.
Some of our users get a 404 error on all links of the first type, and others get 404 on all links of the second type. Does anyone know what might be happening or how to fix this?Sander
10/14/2022, 4:19 PMKyle McChesney
10/14/2022, 4:37 PMdef flow_updater(flow, current_state, next_state):
if isinstance(next_state, Failed):
error_information = {k.name: v.message for k, v in next_state.result.items() if type(v) is Failed}
else:
error_information = None
report(error_information)
When I run this flow locally using prefect run -p path/to/flow.py
with a simulated error, I get the expected error information, however when I run it on our deployment (not cloud, backend in AWS). I get error_information equal to {}
(i.e. no task results have a result of type Failed
). Is there some other type that gets used when running connected to server? (I am using type(v) is Failed
instead of isinstance
cause I don want to report TriggerFailed
, etc)Pedro Henrique
10/14/2022, 4:39 PMPedro Henrique
10/14/2022, 4:41 PMPedro Henrique
10/14/2022, 4:41 PMJosh
10/14/2022, 4:48 PMJosh Paulin
10/14/2022, 5:16 PMtask_input_hash
as the cache_key_fn
?
On step 3 I’m seeing the flow fail that it can’t find the cache file…Nace Plesko
10/14/2022, 5:30 PMdatabase
and a few files in it and all those work fine, but now that I created a new one, the run on the Prefect is saying that it can't find it. I'm probably just missing something obvious, but because of my lack of knowledge of both Python and Prefect I'd really appreciate help on this. Thank you in advance!Josh Paulin
10/14/2022, 5:54 PMNone
. Wondering if this is intentional. Minimal example in the thread.
Also found out I can’t downgrade from 2.6 to 2.5 because of what looks like some database migrations? Running prefect orion database reset -y
on a database with 2.5 after upgrading to 2.6 and it fails.redsquare
10/14/2022, 7:54 PMTaylor Babin
10/14/2022, 8:32 PMOmar Sultan
10/15/2022, 8:41 AMError: Got unexpected extra argument (prefect.orion.api.server:create_app)
Sander
10/15/2022, 10:16 AMKostiantyn Liepieshov
10/15/2022, 10:34 AMAndrei Tulbure
10/15/2022, 3:11 PMFrancesco Bartoli
10/15/2022, 4:29 PMcurl -X 'POST' \
'<http://prefect-server:4200/api/block_types/install_system_block_types>' \
-H 'accept: application/json' \
-H 'x-prefect-api-version: 2.6.1' \
-d ''
I’m getting few of them available from that page, the Remote File System one doesn’t come up though. Is this expected with the latest version?Sander
10/15/2022, 8:02 PMMarcos
10/16/2022, 12:51 AM.map
calls, if one task fails in the first map, then all dependent tasks stay in NotReady state. Example flow in the commentsChristopher
10/16/2022, 2:59 PMFady Khallaf
10/16/2022, 3:31 PMapiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: prefect
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 3Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
namespace: prefect
name: orion
spec:
selector:
matchLabels:
app: orion
replicas: 1
template:
metadata:
labels:
app: orion
spec:
containers:
- name: api
image: prefecthq/prefect:2.6.0-python3.10
command: ["prefect", "orion", "start", "--host", "0.0.0.0", "--log-level", "WARNING"]
ports:
- containerPort: 4200
env:
- name: PREFECT_API_URL
value: '<https://mydomain/api>'
volumeMounts:
- mountPath: ~/.prefect
name: prefect
restartPolicy: Always
volumes:
- name: prefect
persistentVolumeClaim:
claimName: prefect
Fady Khallaf
10/16/2022, 3:31 PMapiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: prefect
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 3Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
namespace: prefect
name: orion
spec:
selector:
matchLabels:
app: orion
replicas: 1
template:
metadata:
labels:
app: orion
spec:
containers:
- name: api
image: prefecthq/prefect:2.6.0-python3.10
command: ["prefect", "orion", "start", "--host", "0.0.0.0", "--log-level", "WARNING"]
ports:
- containerPort: 4200
env:
- name: PREFECT_API_URL
value: '<https://mydomain/api>'
volumeMounts:
- mountPath: ~/.prefect
name: prefect
restartPolicy: Always
volumes:
- name: prefect
persistentVolumeClaim:
claimName: prefect
Q
10/17/2022, 12:17 AM~/.prefect
, but ~
does not get expanded by K8s (tilde pointing to homedir is a shell concept), so your volume gets mounted into a folder called /~
.
df -h
# Filesystem Size Used Available Use% Mounted on
# /dev/storageclass/pvcname
# 2.9G 6.0M 2.9G 0% /~/.prefect
Try changing mountPath
to /root/.prefect
or setting `PREFECT_HOME`(ref).Fady Khallaf
10/18/2022, 1:39 PM