Chris L.
08/15/2022, 8:32 AMworker_client
context manager works as expected in Prefect 1.0 (see https://discourse.prefect.io/t/how-to-use-dask-without-mapping-in-prefect-1-0-using-das[…]-client-to-call-client-submit-inside-a-prefect-task/470/13) but this behavior doesn't seem to migrate over to 2.0Hawkar Mahmod
08/15/2022, 8:53 AMDekel R
08/15/2022, 1:51 PMTask 'upload_data_to_bq_task': Exception encountered during task execution!
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/prefect/engine/task_runner.py", line 880, in get_task_run_state
value = prefect.utilities.executors.run_task_with_timeout(
File "/usr/local/lib/python3.8/dist-packages/prefect/utilities/executors.py", line 468, in run_task_with_timeout
return task.run(*args, **kwargs) # type: ignore
File "/Users/dekelr/PycharmProjects/similarity-filter-layer/prefect_tasks/upload_data_to_bq.py", line 24, in upload_data_to_bq_task
SystemError: unknown opcode
The code fails when running this specific row - (batch size is either an int or None)
if batch_size is None:
Everything works fine when I change this row to “if not batch_size:”
After some troubleshooting - I found this thread - https://github.com/PrefectHQ/prefect/issues/3635
Running the same flow with the original row of -
if batch_size is None:
Still doesn’t work when registering with python 3.8 (same python version as in the container)
Can you please explain this really odd behavior?
Thanksdammy arinde
08/15/2022, 2:05 PMOscar Björhn
08/15/2022, 2:22 PMSam Garvis
08/15/2022, 3:15 PMHamza Naanani
08/15/2022, 3:24 PMJames Brady
08/15/2022, 4:58 PMLana Dann
08/15/2022, 5:57 PMPedro Machado
08/15/2022, 8:04 PMJai P
08/15/2022, 8:55 PMdatamongus
08/15/2022, 11:25 PMprefect-gcp
seems to only refer to Prefect 2.0James Brady
08/16/2022, 5:24 AMemo loic
08/16/2022, 9:20 AMAndreas Nigg
08/16/2022, 10:18 AMOscar Björhn
08/16/2022, 10:38 AMMarcin Grzybowski
08/16/2022, 11:26 AMParwez Noori
08/16/2022, 1:45 PMAndreas Nigg
08/16/2022, 1:50 PMTim Enders
08/16/2022, 1:59 PMsqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
microft
08/16/2022, 2:44 PMTim Enders
08/16/2022, 4:40 PMNone
defined on a mapped task? I now get this error (TypeError: object of type 'NoneType' has no len()
) when trying to map across the subsequent function signature.
@task
def get_items_list(
client, entity, page,
response_attr=None, path=None, params=None
):
Sam Garvis
08/16/2022, 5:45 PMSecret.load("slackbot-oauth").get()
, I get AttributeError: 'coroutine' object has no attribute 'get'
whether I run this in an ipynb or with a python file through the terminal.
Is it expected to run this command with await and async?
Or is this not supposed to happen?Blake Hamm
08/16/2022, 6:16 PMAlexander Belikov
08/16/2022, 6:19 PMmultiprocessing.Process
as
processes = []
for chunk in url_chunks:
kwargs["mp3_urls"] = chunk
processes.append(mp.Process(foo=foo, kwargs=kwargs))
for p in processes:
p.start()
for p in processes:
p.join()
and it worked. The main reason for using multiprocessing.Process
is to have a large model in memory shared between processes.
In fact it works when I run this exact flow with prefect via a python script calling the flow as a fucntion, it works as expected. However, when run it as prefect deployment run ...
or via a scheduled run, the processes in p.start()
don't run. The flow result is marked as success to my surprise.
Any ideas?chicago-joe
08/16/2022, 7:24 PMVlad Tudor
08/16/2022, 7:42 PMKubernetesAgent
locally and to bind it to minikube for testing purposes. For now, I see the Agent in the Server UI, but the tasks are not executed. How do I configure this? Thank you for you patience, I am new to Prefect.
😄 minikube v1.26.1 on Ubuntu 20.04
✨ Using the docker driver based on existing profile
👍 Starting control plane node minikube in cluster minikube
🚜 Pulling base image ...
🏃 Updating the running docker "minikube" container ...
🐳 Preparing Kubernetes v1.24.3 on Docker 20.10.17 ...
▪ Generating certificates and keys ...
▪ Booting up control plane ...
▪ Configuring RBAC rules ...
🔎 Verifying Kubernetes components...
▪ Using image <http://gcr.io/k8s-minikube/storage-provisioner:v5|gcr.io/k8s-minikube/storage-provisioner:v5>
🌟 Enabled addons: storage-provisioner, default-storageclass
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
Tim Enders
08/16/2022, 8:54 PMsqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30.00 (Background on this error at: <https://sqlalche.me/e/14/3o7r>)
Hammad Haqqani
08/16/2022, 10:03 PMprefect agent ecs start --cluster arn:aws:ecs:us-west-2:00000000:cluster/prefect-dev --label aws-ecs
Usage: prefect agent [OPTIONS] COMMAND [ARGS]...
Try 'prefect agent --help' for help.
╭─ Error ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ No such command 'ecs'. │
╰───────────────────────────────────────────
Ben Muller
08/16/2022, 10:38 PM