Chris Gunderson
10/27/2022, 3:20 PMJehan Abduljabbar
10/27/2022, 4:09 PMPaco Ibañez
10/27/2022, 5:09 PMdammy arinde
10/27/2022, 5:28 PMJon
10/27/2022, 6:49 PMTypeError: not all arguments converted during string formatting
is there a way to get the parameter's value without giving function with a task decorator?Carlo
10/27/2022, 7:15 PMAndreas Nigg
10/27/2022, 7:54 PMJens Freund
10/27/2022, 11:01 PMFile "C:\Users\Testbenutzer\anaconda3\envs\prefect_test_project_v3\lib\site-packages\paramiko\sftp_client.py", line 810, in get
with open(localpath, "wb") as fl:
PermissionError: [Errno 13] Permission denied: '.'
23:16:29.384 | INFO | prefect.infrastructure.process - Process 'upbeat-crab' exited cleanly.
To solve the PermissionError, I tried two approaches, which unfortunately both failed:
1. Try to change the PREFECT_LOCAL_STORAGE_PATH
2. Running Powershell, from which I start Prefect, as admin
My problem seems to be practically the same as that reported by @Mike He in the discourse forum (https://discourse.prefect.io/t/agent-debugging-agent-execution-error-with-ftp-storage-in-windows/1587) and here in Slack (https://prefect-community.slack.com/archives/C03D12VV4NN/p1663319368100279). (The only difference I see is that in my case, the last error of the traceback comes from the paramiko library instead of fsspec.) I saw that he already developed a workaround, which however includes changing the Prefect source code which I have not tried yet.
I would write details, e.g. the full traceback, in the thread. If anyone had an idea or a hint how this problem could be solved, preferably without manually changing the Prefect source code, you would help me again a lot. Thank you very much in advance!Andrei Tulbure
10/28/2022, 8:25 AMNic
10/28/2022, 9:21 AMdef cache_within_flow_run(context, parameters):
return f"{context.flow_run_id}-{task_input_hash(context, parameters)}"
How would i pass the flow_run_id into the task this function and further into the task decorator?
@task(cache_key_fn=cache_within_flow_run)
def cached_task():
print('running an expensive operation')
return 42
Camilo Fernandez
10/28/2022, 9:49 AMDeepanshu Aggarwal
10/28/2022, 11:48 AMCarlo
10/28/2022, 1:02 PM23:02:20.896 | INFO | prefect.engine - Engine execution of flow run '682567a2-e507-4aa1-b5c7-25bb2cc2573f' aborted by orchestrator: This run has already terminated.
Nic
10/28/2022, 1:05 PMJon
10/28/2022, 2:11 PMPaco Ibañez
10/28/2022, 2:23 PMvholmer
10/28/2022, 2:44 PMawait my_task.submit(_flow_name_=FLOW_NAME, _env_=env, _run_name_=run_name, _wait_for_=[preprocess], _parameters_=parameters)
However after upgrading to 2.6.5 my_task isn't even being run. My workaround is to just run this:
await my_task(_flow_name_=FLOW_NAME, _env_=env, _run_name_=run_name, _wait_for_=[preprocess], _parameters_=parameters)
But it seems really weird that submit should have stopped working. Has anyone else ran into this issue?Arun Giridharan
10/28/2022, 3:36 PMstartFlowRun
method. Initially when I was calling the first flow without using the wait
parameter, the flows worked fine, but after I added the parameter, the parent flow failed with TypeError: Object of type SUCCESS is not JSON serializable
. I was wondering if this is a known issue. Prefect version is 1.4.0Mansour Zayer
10/28/2022, 4:01 PMorder by column_name nulls last
? I have this line order_by: {start_time: desc}
in my query.
More context in commentSlackbot
10/28/2022, 7:34 PMLuca Schneider
10/28/2022, 8:06 PMKirill Popov
10/28/2022, 8:22 PM19:41:45.583 | INFO | prefect.agent - Submitting flow run 'd03564b9-ad4e-4310-8662-184616d6406f'
19:41:47.416 | INFO | prefect.agent - Completed submission of flow run 'd03564b9-ad4e-4310-8662-184616d6406f'
19:41:48.471 | INFO | prefect.infrastructure.kubernetes-job - Job 'expert-lyrebird-xjq42': Pod has status 'Pending'.
19:41:49.155 | INFO | prefect.infrastructure.kubernetes-job - Job 'expert-lyrebird-xjq42': Pod has status 'Running'.
19:41:52.098 | INFO | Flow run 'expert-lyrebird' - Created task run 'My Example Task-c06c9343-0' for task 'My Example Task'
19:41:52.098 | INFO | Flow run 'expert-lyrebird' - Executing 'My Example Task-c06c9343-0' immediately...
19:41:52.345 | INFO | Task run 'My Example Task-c06c9343-0' - Finished in state Completed()
19:41:52.443 | INFO | Flow run 'expert-lyrebird' - Finished in state Completed()
When running the agent locally but in docker container, the logs look similar.
Now I want the agent to run in a kubernetes deployment to automatically manage its uptime. Only 1 replica of agent pod is used.
However I only receive part of the log in the agent cli output -- it looks like no info is coming back from the worker pods back to the agent pod...
19:59:58.687 | INFO | prefect.agent - Submitting flow run 'a510b13d-b0b5-4067-8610-518e6a2b45a8'
19:59:59.268 | INFO | prefect.agent - Completed submission of flow run 'a510b13d-b0b5-4067-8610-518e6a2b45a8'
19:59:59.305 | INFO | prefect.infrastructure.kubernetes-job - Job 'secret-iguana-zlm4f': Pod has status 'Pending'.
20:00:01.349 | INFO | prefect.infrastructure.kubernetes-job - Job 'secret-iguana-zlm4f': Pod has status 'Running'.
What is going on ? how can I make the agent running on K8S produce a complete log ?
let me know if you have any ideas -- I am fairly new to k8s so might be something basic, but looks unexpected to meYD
10/28/2022, 11:31 PMnohup prefect agent start -q … > ~/tmp/prefect_agent.log &
or is there a different way to keep the agent running?
is there best practice around this in the docs? I did not find it so far
ThanksAhmed Ezzat
10/29/2022, 12:23 AMSander
10/29/2022, 8:22 PMTyson Chavarie
10/29/2022, 10:08 PMTyson Chavarie
10/29/2022, 10:23 PMTyson Chavarie
10/29/2022, 10:27 PMFailed to load and execute flow run: ClientError([{'path': ['secret_value'], 'message': 'Unable to complete operation', 'extensions': {'code': 'API_ERROR'}}])
we are hard down...pk13055
10/30/2022, 10:29 AMorion
+ agent
) on a remote server. However, I get an error message in the orion logs after changing PREFECT_ORION_API_HOST
from 0.0.0.0
to SERVER_IP
. The particular error is:
ERROR: [Errno 99] error while attempting to bind on address ('SERVER_IP', 4200): cannot assign requested address
This does not make sense, since I can specifically access the same SERVER_IP
through the browser and the particular port has been allowed through the firewall as well.
(PS - orion appears to connect for a brief second, and then immediately throws this error and attempts to restart).
Additionally, attached below is the relevant section of my `docker-compose.yml`:
version: "3.9"
services:
db:
image: timescale/timescaledb:latest-pg14
volumes:
- $PWD/data/db:/var/lib/postgresql/data
- $PWD/config/db:/docker-entrypoint-initdb.d/
healthcheck:
test: [ "CMD-SHELL", "pg_isready" ]
interval: 10s
timeout: 5s
retries: 5
environment:
- POSTGRES_DB=$POSTGRES_DB
- POSTGRES_USER=$POSTGRES_USER
- POSTGRES_PASSWORD=$POSTGRES_PASSWORD
networks:
- db_network
orion:
image: prefecthq/prefect:2.6.3-python3.10
restart: always
volumes:
- $PWD/prefect:/root/.prefect
entrypoint: [ "prefect", "orion", "start" ]
environment:
- PREFECT_ORION_API_HOST=$SERVER_IP
- PREFECT_ORION_DATABASE_CONNECTION_URL=$PREFECT_DB_URL
ports:
- 4200:4200
depends_on:
- db
networks:
- prefect_network
- db_network
agent:
build: ./prefect
restart: always
entrypoint: [ "prefect", "agent", "start", "-q", "main_queue" ]
environment:
- PREFECT_API_URL=<http://orion:4200/api>
networks:
- prefect_network
- db_network
cli:
build: ./prefect
entrypoint: "bash"
working_dir: "/root/flows"
volumes:
- "$PWD/prefect/flows:/root/flows"
environment:
- PREFECT_API_URL=<http://orion:4200/api>
networks:
- prefect_network
- db_network
networks:
db_network:
prefect_network:
Rabea Yousof
10/30/2022, 10:44 AM