Numline108/29/2022, 6:20 PM
Numline108/29/2022, 6:20 PM
Maggie Dart-Padover08/29/2022, 7:25 PM
OSError: could not get source code
does remove it from my environment, but then if I try to run the task definition again I still get the error. I'm not totally sure if this is a bug or expected behavior. Is my workflow here just wrong for prefect?
Sam Garvis08/29/2022, 7:27 PM
I get this error
prefect deployment build name.name -n name_dev -t dev_wq_k8s -sb gcs/dev --work-queue=dev_wq_k8s -ib kubernetes-job/dev-k8s-job
Even though in the Prefect 2.0 UI for creating a k8s job block, it says
Job is missing required attributes at the following paths: /apiVersion, /kind, /metadata, /spec (type=value_error)
ash08/29/2022, 8:41 PM
Richard Freeman08/29/2022, 10:30 PM
Nace Plesko08/29/2022, 11:04 PM
is not logging all the outputs from the script that it's running. I am looking at https://docs-v1.prefect.io/api/latest/tasks/shell.html#shelltask and seems like just setting
should do the job, but I'm still seeing just
. I also tried setting
Command failed with exit code 1
, but still the same behavior. Has anyone ran into this issue in the past?
Walter Cavinaw08/30/2022, 1:01 AM
Yousef Hosny08/30/2022, 1:47 AM
doesn't work from a jupyter notebook cell, but does work from
script file. Also, is there anyway to deploy a flow to prefect cloud using jupyter notebook ?
Tommy Nam08/30/2022, 2:22 AM
Anat Tal Gagnon08/30/2022, 4:17 AM
Faheem Khan08/30/2022, 7:33 AM
Anat Tal Gagnon08/30/2022, 8:25 AM
Adrien Besnard08/30/2022, 9:12 AM
collections exists, I was wondering if it makes sense to have something like a dedicated block for Airbyte (which allow us to store the
but also invoke a
function) and the same question with a DBT block (and we store all the information that we can find in the
, for exemple)? • What is the best way to log from a function of a
subclass? Can we use the
or is preferable to use some sort of callbacks that are going to be invoked inside a
or something? Thanks!
Aditya Sharma08/30/2022, 10:47 AM
Parwez Noori08/30/2022, 11:20 AM
Tarek08/30/2022, 12:59 PM
Josh Paulin08/30/2022, 1:24 PM
Anat Tal Gagnon08/30/2022, 1:39 PM
Tim Enders08/30/2022, 1:56 PM
Pickle error on clients in Prefect 2.0?
_pickle.PicklingError: Pickling client objects is explicitly not supported. Clients have non-trivial state that is local and unpickleable.
Marcelo Ortega08/30/2022, 2:35 PM
karteek08/30/2022, 3:53 PM
Matt Melgard08/30/2022, 3:54 PM
This is the CLI command I’m using to build the deployment:
RuntimeError: File system created with scheme 'gcs' from base path '<gcs://mattmelgard-test-bucket-01>' could not be created. You are likely missing a Python module required to use the given storage protocol.
and I have a block defined for the bucket with the name
prefect deployment build tutorial-flow/flow/tutorial-flow.py:my_flow -n tutorial-flow-k8s -t test -i kubernetes-job -sb gcs/tutorial-bucket
but I’m not sure what could be missing here, I have pretty much all the python packages installed via
pip install 'prefect[all_extras]'
with Flow( "Subscriptions API Import", result=GCSResult(bucket="platformsh-vendor-prefect"), # executor=LocalExecutor(), executor=LocalDaskExecutor(scheduler="threads", num_workers=5), state_handlers=[flow_failure],
Florent VanDeMoortele08/30/2022, 4:46 PM
to create an image with my flow, context, scheduler and all the required virtual environnement. Then I register flow to Prefect with
as environment. I'm using
to run it. To deploy on 2.x, I simply try to use
but it doesn't work without storage. So I try to install main branch of Prefect because of a merged PR ( https://discourse.prefect.io/t/how-to-build-deployments-with-flow-code-and-dependencies-being-baked-into-a-docker-image/1341 ) to use
but it's not compatible with
. I also try to push directly my Docker image with
but push doesn't already work. So, I'm a bit confused because I can't find explicit documentation on Discourse and I read different responses because of the different releases on 2.x. Is
the best practice to deploy a completely self contained flow? In this case, do you have any idea of the release roadmap to fix this (and to update documentation) ? If not, what can I should use? Thank you !
Dylan McReynolds08/30/2022, 5:25 PM
, is there a way to schedule a subflow to start at a particular time in the future? Or some other way to accomplish a long-delayed subflow/task?
Shaoyi Zhang08/30/2022, 5:51 PM
to Cloud UI? When using Kubernetes agent, those logs only show up in job pods logs and are not available in Cloud UI.
Renan Matias08/30/2022, 5:52 PM
and it worked normally (I used Insomnia). Now, I need to make this same request through Prefect Cloud. I was already able to create the deployment and I was able to run the flow normally through the Prefect UI with the agent active. Can you help me?
Ashley Felber08/30/2022, 6:23 PM
Marcos08/30/2022, 6:27 PM
@flow async def test(): connected = await connect_db.submit() task1 = await db_stuff.submit(wait_for=[connected]) await disconnect_db.submit(wait_for=[task1])
never is executed but I’d like it to do it