https://prefect.io
Join Slack
Hi all, I'm working on a project using document processing pipelines for a US government client. We ...
a

Ari Hershowitz

over 3 years ago
Hi all, I'm working on a project using document processing pipelines for a US government client. We need to deploy to Redhat servers, which (probably) means using podman instead of docker. My initial attempts to install Prefect with podman didn't work. Is there a good guide on Centos/Redhat Os?
a
a
z
  • 3
  • 8
  • 216
When setting up an automation in Prefect 2 cloud, is there a way to test to make sure it works? Or w...
j

John Mizerany

over 2 years ago
When setting up an automation in Prefect 2 cloud, is there a way to test to make sure it works? Or would you just have to wait for the event to happen to trigger it
j
m
  • 2
  • 4
  • 213
<@ULVA73B9P> How can I setup prefect 3.x using docker-compose? Note that I want to run flows using D...
e

Eren Bilaloglu

10 months ago
@Marvin How can I setup prefect 3.x using docker-compose? Note that I want to run flows using Docker work pool
e
m
  • 2
  • 7
  • 212
I am experiencing a dependency issue with "pendulum" in prefect that doesnt seem to make sense; I am...
g

Grayson Wagstaff

over 1 year ago
I am experiencing a dependency issue with "pendulum" in prefect that doesnt seem to make sense; I am attempting to add "msgraph-sdk" package to my internal package ("prefect-trellis") but getting this error from poetry: 
Because no versions of msgraph-sdk match >1.4.0,<2.0.0
 and msgraph-sdk (1.4.0) depends on microsoft-kiota-serialization-form (>=0.1.0), msgraph-sdk (>=1.4.0,<2.0.0) requires microsoft-kiota-serialization-form (>=0.1.0).
And because no versions of microsoft-kiota-serialization-form match >0.1.0
 and microsoft-kiota-serialization-form (0.1.0) depends on pendulum (>=3.0.0), msgraph-sdk (>=1.4.0,<2.0.0) requires pendulum (>=3.0.0).
And because prefect (2.19.6) depends on pendulum (<3.0)
 and no versions of prefect match >2.19.6,<3.0.0, msgraph-sdk (>=1.4.0,<2.0.0) is incompatible with prefect (>=2.19.6,<3.0.0).
So, because prefect-trellis depends on both prefect (^2.19.6) and msgraph-sdk (^1.4.0), version solving failed.
reading this, it seems like Prefect has a dependency on pendulum < 3.0 while graph-sdk has dependency on pendulum >= 3.0; however checking prefect's requirement, I only see pendulum referenced in the client which requires pendulum >= 3.0.0, <4 https://github.com/PrefectHQ/prefect/blob/2c25cd0df52959d9c499cc0487ec52d7afe04dc8/requirements-client.txt#L21. Is this dependency on pendulum < 3.0 intended for the main prefect package?
g
n
  • 2
  • 5
  • 212
I keep getting these "object type 'coroutine' has no attribute _______" messages whenever I try to f...
j

Jacob Bedard

over 2 years ago
I keep getting these "object type 'coroutine' has no attribute _______" messages whenever I try to follow the python API documentation. Is there a way to work around this? Spyder has the best variable explorer of the Python IDEs, which makes it ideal for data science, but I'm getting different behavior on the same script, in the same environment, between pycharm and spyder. I'd really love to find a workaround for spyder, because as someone who inspects data coming back from new systems and APIs all the time, I like being able to inspect the structure of those with Spyder's great little variable explorer. It's the closest thing we have to RStudio in the python world (RIP yhat rodeo)
j
t
r
  • 3
  • 9
  • 212
Hello community, I tried to connect Prefecta with my SQL server database, but I got this error. who ...
r

RAISS Zineb

over 3 years ago
Hello community, I tried to connect Prefecta with my SQL server database, but I got this error. who has already had this error, or has an idea about this error?
r
k
a
  • 3
  • 22
  • 212
Hey all, I’m encountering this error when I try to deploy to prefect cloud: ``` &gt; Running build_d...
d

Daniel

about 1 year ago
Hey all, I’m encountering this error when I try to deploy to prefect cloud: 
> Running build_docker_image step...
Step 1/10 : FROM --platform=linux/amd64 python:3.10-slim
 ---> 38ced5e623fb
Step 2/10 : WORKDIR /opt/prefect/amy-ml-server
Traceback (most recent call last):
  File "/Users/dansimons/amygda/menv/lib/python3.10/site-packages/prefect/deployments/steps/core.py", line 154, in run_steps
    step_output = await run_step(step, upstream_outputs)
  File "/Users/dansimons/amygda/menv/lib/python3.10/site-packages/prefect/deployments/steps/core.py", line 125, in run_step
    result = await from_async.call_soon_in_new_thread(
  File "/Users/dansimons/amygda/menv/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 327, in aresult
    return await asyncio.wrap_future(self.future)
  File "/Users/dansimons/amygda/menv/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 352, in _run_sync
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/dansimons/amygda/menv/lib/python3.10/site-packages/prefect_docker/deployments/steps.py", line 200, in build_docker_image
    raise BuildError(event["error"])
prefect.utilities.dockerutils.BuildError: NotFound: content digest sha256:9acd79fa542a002076583004f9dd5867a308eeff29a35fb033fdbf34cf30b58e: not found

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/dansimons/amygda/menv/lib/python3.10/site-packages/prefect/cli/_utilities.py", line 42, in wrapper
    return fn(*args, **kwargs)
  File "/Users/dansimons/amygda/menv/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 304, in coroutine_wrapper
    return call()
  File "/Users/dansimons/amygda/menv/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 432, in __call__
    return self.result()
  File "/Users/dansimons/amygda/menv/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 318, in result
    return self.future.result(timeout=timeout)
  File "/Users/dansimons/amygda/menv/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 179, in result
    return self.__get_result()
  File "/usr/local/Cellar/python@3.10/3.10.13_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/Users/dansimons/amygda/menv/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 389, in _run_async
    result = await coro
  File "/Users/dansimons/amygda/menv/lib/python3.10/site-packages/prefect/cli/deploy.py", line 428, in deploy
    await _run_single_deploy(
  File "/Users/dansimons/amygda/menv/lib/python3.10/site-packages/prefect/client/utilities.py", line 100, in with_injected_client
    return await fn(*args, **kwargs)
  File "/Users/dansimons/amygda/menv/lib/python3.10/site-packages/prefect/cli/deploy.py", line 657, in _run_single_deploy
    await run_steps(build_steps, step_outputs, print_function=app.console.print)
  File "/Users/dansimons/amygda/menv/lib/python3.10/site-packages/prefect/deployments/steps/core.py", line 182, in run_steps
    raise StepExecutionError(f"Encountered error while running {fqn}") from exc
prefect.deployments.steps.core.StepExecutionError: Encountered error while running prefect_docker.deployments.steps.build_docker_image
An exception occurred.
It used to work with the exact same prefect yaml & dockerfile, not sure why it’s stopped working. Any help would be much appreciated. Thanks!
d
n
  • 2
  • 15
  • 211
``` build: - prefect_docker.deployments.steps.build_docker_image: id: build_image...
g

Gemma

over 2 years ago
build:
      - prefect_docker.deployments.steps.build_docker_image:
          id: build_image
          requires: prefect-docker>=0.3.0
          image_name: <http://724930167716.dkr.ecr.eu-west-2.amazonaws.com/prefect-training-2023|724930167716.dkr.ecr.eu-west-2.amazonaws.com/prefect-training-2023>
          tag: 0.0.1
          dockerfile: Dockerfile
          push: true
          platform: linux/amd64
g
t
j
  • 3
  • 3
  • 210
Please help.
d

Denis

about 1 year ago
Please help.
d
n
  • 2
  • 19
  • 209
Hello :wave: Is there a way to retry `CRASHED` tasks or to avoid setting them as `CRASHED`? <This i...
j

Jonathan Langlois

over 2 years ago
Hello 👋 Is there a way to retry 
CRASHED
tasks or to avoid setting them as 
CRASHED
? This issue was resolved and we can have Dask to retry 
RUNNING
tasks now. But since the prefect 
2.8.7
version, prefect detects that the Dask worker will be shut down and sets the task as 
CRASHED
. When Dask resend the failing task, we end up with 
Task run '...' already finished
.
j
s
  • 2
  • 2
  • 208
Previous222324Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by