Zachary Piazza
07/18/2023, 3:58 PM10:52:38.256 | INFO | Task run 'print_hello-99' - Crash detected! Execution was interrupted by an unexpected exception: Exception: PrefectHTTPStatusError("Client error '404 Not Found' for url '<http://ephemeral-prefect/api/task_runs/7fa61d80-fdb2-490f-b0f6-e4d8709103b5/set_state>'\nResponse: {'exception_message': 'Task run with id 7fa61d80-fdb2-490f-b0f6-e4d8709103b5 not found'}\nFor more information check: <https://httpstatuses.com/404>")
import dask_kubernetes
from dask_kubernetes import make_pod_spec
from kubernetes.client import V1Pod
from prefect import flow, get_run_logger, task
from distributed import Client
from prefect_dask.task_runners import DaskTaskRunner
client = Client("localhost:9000")
@task
def print_hello(name):
return f"Hello {name}!"
@flow(
task_runner=DaskTaskRunner(address="localhost:9000"),
)
def hello_world():
greetings = []
with open('names.txt') as f:
names = f.read().splitlines()
for name in names:
print_hello.submit(name)
return greetings
if __name__ == "__main__":
greetings = hello_world()
print(greetings)
Emil Christensen
07/18/2023, 4:11 PMZachary Piazza
07/18/2023, 4:13 PMprefect kubernetes manifest server
command to generate the YAML spec and then port forwarded the service port (4200) to my local machine and set the URL using prefect config set PREFECT_API_URL="<http://127.0.0.1:4200/api>"
Emil Christensen
07/18/2023, 4:20 PMAll connection attempts failed
? Could you share your full stacktrace? The Dask workers also need to be able to access the server. I’d highly recommend starting out with Prefect cloud rather than self-hosting if possible.Zachary Piazza
07/18/2023, 4:22 PM11:15:34.213 | INFO | Task run 'print_hello-99' - Crash detected! Request to <http://127.0.0.1:4200/api/task_runs/cdcffbd8-aace-4e4d-b80a-aba5473b355b/set_state> failed: ConnectError: All connection attempts failed.
Emil Christensen
07/18/2023, 4:24 PMZachary Piazza
07/18/2023, 4:25 PMEmil Christensen
07/18/2023, 4:25 PMso whatever I said for PREFECT_API_URL on the prefect cli gets forwarded to the workers?Yep, that’s right… any Prefect settings get propagated.
Zachary Piazza
07/18/2023, 4:28 PM