Vadym Dytyniak
07/25/2022, 12:31 PMEvent: 'FailedScheduling' on pod 'prefect-job-79c82cd5-w6nfd'
Message: 0/7 nodes are available: 2 node(s) had taint {<http://dask.corp.com/component|dask.corp.com/component>: scheduler}, that the pod didn't tolerate, 5 node(s) didn't match Pod's node affinity/selector
Do you have ideas how to disable them?Steph Clacksman
07/25/2022, 1:20 PMTom Klein
07/25/2022, 1:28 PMSubmitted
and nothing seems to be happening with itDavid
07/25/2022, 1:47 PMDavid
07/25/2022, 1:47 PMRajvir Jhawar
07/25/2022, 2:01 PMPrass
07/25/2022, 2:13 PMmlops
section, I've added prefect - pip install productivity-stack[mlops]
Chu
07/25/2022, 2:54 PMAndrew Pruchinski
07/25/2022, 3:34 PMConnection aborted
issue. Even if there are some failures, we do want the flow to continue and the task triggers are set accordingly. Thank youLeon Kozlowski
07/25/2022, 4:13 PMMikkel Duif
07/25/2022, 4:48 PMprefect orion database reset -y
?Mikkel Duif
07/25/2022, 4:48 PMDeployments
in Orion UI locally. If i inspect the network, I see it will successfully return my deployments, but they are never rendered in the UI.Jenia Varavva
07/25/2022, 5:16 PMKubernetes Agent
, KubernetesRun
and DaskExecutor
in the face of various components outages. E.g. what will Prefect be able to recover on its own, and which case would require a manual intervention, if during a flow execution one of the following dies: prefect server (I suppose it’s ok as it’s stateless), kubernetes agent, KubernetesRun
job executing a flow, dask scheduler, dask workers.Gunther Wallach
07/25/2022, 5:47 PMValueError: Flow could not be deserialized successfully. Error was: ValueError('A task with the slug "Parameter" already exists in this flow.')
This shows up after the step:
[2022-07-25 17:27:11-0700] INFO - prefect.Docker | Pushing image to the registry..
Parameters are currently in their own params.py file, and are referenced in the main flow file with:
import <main_directory>.tasks.params as params
Mikkel Duif
07/25/2022, 6:01 PMprefect deployment create
command? we use it to compare created deployments with existing deployments to clean out unused deployments.Tyler Wanner
07/25/2022, 6:07 PMDivya
07/25/2022, 6:11 PMprefect orion start.
It stops running quickly as below:
Please let me know how do I troubleshoot this. Is it because of the windows machine that I am using?alex
07/25/2022, 7:32 PMTom Klein
07/25/2022, 8:51 PMChu
07/25/2022, 9:08 PMMichael Reynolds
07/25/2022, 9:40 PM@flow
. I will attach a micro example of what i am trying to accomplish... would someone mind helping me understanding the error message:
FAILED my-prefect-project/tests/flows.py::test_pipeline - TypeError: __init__() missing 2 required positional arguments: 'host' and 'port'
li liang
07/26/2022, 6:38 AMVersion: 2.0b12
API version: 0.7.0
Python version: 3.9.1
Git commit: 18d44c8c
Built: Fri, Jul 22, 2022 4:23 PM
OS/Arch: win32/AMD64
Profile: default
Server type: <client error>
Mike Kovetsky
07/26/2022, 8:43 AMprefect storage create
in cli, but my 2.0b13 doesn’t have no such command returning the error Error: No such command ‘storage’.
Should I create the storage using this doc ? There is no example for GCP, i found this one in the community. But there is no
from prefect.blocks.storage import GoogleCloudStorageBlock
import in my venv 😞haris khan
07/26/2022, 9:30 AMTyndyll
07/26/2022, 10:20 AMdata = returns_list()
with case(data, len(data) > 1):
do_stuff_with_data(data)
with case(data, len(data) == 0):
do_stuff_with_no_data()
Lucien Fregosi
07/26/2022, 12:00 PMBhupesh Kemar Singh
07/26/2022, 12:28 PMBhupesh Kemar Singh
07/26/2022, 12:33 PMDominik Wagner
07/26/2022, 1:28 PM@flow
def my_flow():
if <somehow_get_state_of_this_flow?> == "Retrying":
do_something()
Divya
07/26/2022, 1:41 PMDivya
07/26/2022, 1:41 PMStéphan Taljaard
07/26/2022, 1:44 PMDivya
07/26/2022, 2:31 PMTaylor Curran
07/26/2022, 2:39 PMDivya
07/26/2022, 3:22 PMSebastian
07/27/2022, 9:38 AMTaylor Curran
07/27/2022, 1:16 PMOur CTO Chris White did a great job of answering this question live the other day.of that recording.Here's a link to the timestamp▾
In summary
• Lots of people were unfamiliar with GraphQL and asking for RestAPI
• For Prefect Server specifically, because we chose to use GraphQL, that’s the reason why Prefect Server is so heavy - we needed the query schema off of the database (which is what Hasura does)
• For our business logic, we had all of our mutations defined in Python code
• Those are two separate GraphQL schemas so we had to stitch them together
• All of this resulted in a very large, heavy container that was Prefect Server
• REST is very lightweight and allows us to automatically pull in Prefect flows into the UI even if they ran before the UI was launched
• Finally, GraphQL is very hard to manage when exposing publicly to a high volume of usage
Sebastian
07/27/2022, 1:26 PM