to run mapped tasks in parallel inside a single GKE k8s container with what, if I understand the jargon correctly, is known as vertical scaling. Are there any resources that show the optimal way to do that?
--run-task-kwargs`` . However when I add the yaml path to the command, I start to receive this error:
ERROR - agent | Failed to infer default networkConfiguration, please explicitly configure using
Error: No such option: --run-task-kwargs s3://*****/options.yaml
but is there a way to delete multiple flow runs at once ? Thanks !
I was setting the local server as option and started as usual:
No special options. Images are pulled and all the services started. However some errors (regarding graphql mainly) start to apper on logs.
prefect backend server prefect server start
(I have the full trace until stopping them). I also checked (while waiting the spin up - just displaying the failing one)
graphql_1 | Error applying Hasura metadata from /prefect-server/services/hasura/migrations/metadata.yaml graphql_1 | graphql_1 | Could not upgrade the database! graphql_1 | Error: Failed to apply Hasura metadata! Error: 503 Server Error: Service Unavailable for url: <http://hasura:3000/v1/query> apollo_1 | Checking GraphQL service at <http://graphql:4201/health> ... tmp_graphql_1 exited with code 1
which is continously restarting. I have tryied: • previuos Prefect version (0.15.0) with the same effect. • the same env on my local machine with success (so I suspect from the remote configuration, but I could not figure what is missing) Thanks!
> docker ps 7036b92f538d prefecthq/server:core-0.15.13 "tini -g -- bash -c …" About a minute ago Restarting (1) 1 second ago tmp_graphql_
Env: • Local Agent + Local Storage (Running in docker [python:3.10] ... yeah yeah lol) • External Postgres 12 • Prefect Server in Kubernetes (Official HELM) • Django Integrated Code: The Source is located in
Failed to load and execute Flow's environment: ModuleNotFoundError("No module named '/home/magnox/'")
We have added
to the path and Python path on execution. Notes: • Sometimes a single container will work fine (using the same exact image and everything)(Cant seem to figure out why) • I have tried with and without Dask • I tried to symlink the /code -> /home/magnox to no avail