Zac Hooper
09/08/2022, 4:31 AMAndreas Nord
09/08/2022, 9:06 AMBrad
09/08/2022, 9:08 AMDavid Peláez
09/08/2022, 9:58 AMVadym Dytyniak
09/08/2022, 10:47 AMBradley Collins
09/08/2022, 12:30 PMAnirudh MV
09/08/2022, 12:38 PMNo such command 'backend'.
Please help me fix this. Could this be because my prefect client and prefect server might be running different versions?Eli Treuherz
09/08/2022, 2:04 PMRaghuram M
09/08/2022, 2:11 PMFelipe Fernandez
09/08/2022, 2:54 PMDat Tran
09/08/2022, 3:07 PMIgor Morgunov
09/08/2022, 3:28 PMMapped Child X
? I know I can get it from logs, but would be cool if I could see it in this screenSlackbot
09/08/2022, 3:35 PMJeffery Newburn
09/08/2022, 4:56 PMVenkat Ramakrishnan
09/08/2022, 5:07 PMRoger Webb
09/08/2022, 5:19 PMMark Li
09/08/2022, 5:59 PMError: No such command 'orion'.
I’m assuming this failure is coming from when it’s calling ‘prefect orion start’
Does anyone know what could be causing prefect to not recognize the orion command?Venkat Ramakrishnan
09/08/2022, 6:02 PMAndrew Pruchinski
09/08/2022, 6:17 PM%
and pandas.read_sql. There seems to be an issue during compilation during a prefect flow run where it's complaining about the wildcard. We can run the queries outside of a prefect run no problem. When executing in prefect, we are getting formatting errors. Errors listed in threadsMarc Lipoff
09/08/2022, 6:30 PMIan Andres Etnyre Mercader
09/08/2022, 7:59 PMprefect 2.3.1
, prefect-dask: 0.2.0
and got several errors running code similar to this pseudo code :
@task
def dask_context(item):
name="myblock-name"
config = await JSON.load(name)
@flow(task_runner=DaskTaskRunner())
def do_stuff():
with tags("only_4"):
result_list = dask_context.map(data_list)
I got this Exception multiple times after it running for a while (sometimes minutes, sometimes hours):
Encountered exception during execution:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/httpcore/backends/asyncio.py", line 33, in read
return await self._stream.receive(max_bytes=max_bytes)
File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 1265, in receive
await self._protocol.read_event.wait()
File "/usr/local/lib/python3.9/asyncio/locks.py", line 226, in wait
await fut
asyncio.exceptions.CancelledError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/httpcore/_exceptions.py", line 8, in map_exceptions
yield
File "/usr/local/lib/python3.9/site-packages/httpcore/backends/asyncio.py", line 35, in read
return b""
File "/usr/local/lib/python3.9/site-packages/anyio/_core/_tasks.py", line 118, in __exit__
raise TimeoutError
TimeoutError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions
yield
File "/usr/local/lib/python3.9/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
resp = await self._pool.handle_async_request(req)
File "/usr/local/lib/python3.9/site-packages/httpcore/_async/connection_pool.py", line 253, in handle_async_request
raise exc
File "/usr/local/lib/python3.9/site-packages/httpcore/_async/connection_pool.py", line 237, in handle_async_request
response = await connection.handle_async_request(request)
File "/usr/local/lib/python3.9/site-packages/httpcore/_async/connection.py", line 90, in handle_async_request
return await self._connection.handle_async_request(request)
File "/usr/local/lib/python3.9/site-packages/httpcore/_async/http11.py", line 105, in handle_async_request
raise exc
File "/usr/local/lib/python3.9/site-packages/httpcore/_async/http11.py", line 84, in handle_async_request
) = await self._receive_response_headers(**kwargs)
File "/usr/local/lib/python3.9/site-packages/httpcore/_async/http11.py", line 148, in _receive_response_headers
event = await self._receive_event(timeout=timeout)
File "/usr/local/lib/python3.9/site-packages/httpcore/_async/http11.py", line 177, in _receive_event
data = await self._network_stream.read(
File "/usr/local/lib/python3.9/site-packages/httpcore/backends/asyncio.py", line 35, in read
return b""
File "/usr/local/lib/python3.9/contextlib.py", line 137, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.9/site-packages/httpcore/_exceptions.py", line 12, in map_exceptions
raise to_exc(exc)
httpcore.ReadTimeout
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/prefect/engine.py", line 1185, in orchestrate_task_run
result = await task.fn(*args, **kwargs)
File "/tmp/tmpq25d58myprefect/lib/dask_context.py", line 14, in _dask_task
with await get_context(flow_run.parameters[block_name]):
File "/tmp/tmpq25d58myprefect/./lib/config_context.py", line 40, in get_context
config = await JSON.load(name)
File "/usr/local/lib/python3.9/site-packages/prefect/blocks/core.py", line 606, in load
block_document = await client.read_block_document_by_name(
File "/usr/local/lib/python3.9/site-packages/prefect/client.py", line 1268, in read_block_document_by_name
response = await self._client.get(
File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1751, in get
return await self.request(
File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1527, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
File "/usr/local/lib/python3.9/site-packages/prefect/client.py", line 257, in send
await super().send(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1614, in send
response = await self._send_handling_auth(
File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1642, in _send_handling_auth
response = await self._send_handling_redirects(
File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1679, in _send_handling_redirects
response = await self._send_single_request(request)
File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1716, in _send_single_request
response = await transport.handle_async_request(request)
File "/usr/local/lib/python3.9/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
resp = await self._pool.handle_async_request(req)
File "/usr/local/lib/python3.9/contextlib.py", line 137, in __exit__
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.9/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ReadTimeout
I started running this code with a tag concurrency limit of 20 then switched to 4. The more threads the exception was raised faster.
Currently running on a server of 44 cores and 256 GB.Bradley Hurley
09/08/2022, 8:35 PMBrad Clark
09/08/2022, 9:33 PMprefect cloud login
i get
Unable to authenticate with Prefect Cloud. Please ensure your credentials are correct.
Felix Tremblay
09/08/2022, 9:33 PMAaron Goebel
09/08/2022, 10:57 PMYSF
09/09/2022, 1:06 AMflow = Flow("My imperative flow!")
# define some new tasks
name = Parameter("name")
second_add = add.copy()
# add our tasks to the flow
flow.add_task(add)
flow.add_task(second_add)
flow.add_task(say_hello)
# create non-data dependencies so that `say_hello` waits for `second_add` to finish.
say_hello.set_upstream(second_add, flow=flow)
# create data bindings
add.bind(x=1, y=2, flow=flow)
second_add.bind(x=add, y=100, flow=flow)
say_hello.bind(person=name, flow=flow)
If I wanted to give the add
task a mapped input?Yousef Hosny
09/09/2022, 1:26 AMYaron Levi
09/09/2022, 4:08 AMfrom weekly_bonus_calc import weekly_bonus_calc
from prefect.deployments import Deployment
from prefect.filesystems import GitHub
storage = GitHub(
repository="<https://github.com/yuvital/yuvital-prefect-github.git>"
)
deployment = Deployment.build_from_flow(
flow=weekly_bonus_calc,
name="weekly-bonus-calc",
work_queue_name="work-queue",
storage=storage,
schedule={'cron': '*/5 * * * *', 'timezone': "Asia/Jerusalem"}
)
deployment.apply()
Also, I’ve created a GitHub block via the UI (attaching a screenshot).
Can you point out the correct way to setup this.
Another thing I don’t understand is where are the Git credentials should go since this is obviously a private repo.
Thank you for any help on the issue 🙏ash
09/09/2022, 6:17 AMBal Raj
09/09/2022, 6:27 AM