Tomas Moreno
02/16/2023, 9:50 PMPREFECT__CLOUD__REQUEST_TIMEOUT=60
but don't think it's quite working. does anyone have any experience with situations like this? full stack trace in threadTraceback (most recent call last):
File "/home/runner/work/dwh/dwh/deploy/register_flows.py", line 626, in <module>
main()
File "/home/runner/work/dwh/dwh/deploy/register_flows.py", line 620, in main
create_proj_and_register_flows(flows, args)
File "/home/runner/work/dwh/dwh/deploy/register_flows.py", line 283, in create_proj_and_register_flows
register_flow(flow, flow_file, args)
File "/home/runner/work/dwh/dwh/deploy/register_flows.py", line 373, in register_flow
flow.register(
File "/home/runner/work/dwh/dwh/.venv/lib/python3.10/site-packages/prefect/core/flow.py", line 1727, in register
registered_flow = client.register(
File "/home/runner/work/dwh/dwh/.venv/lib/python3.10/site-packages/prefect/client/client.py", line 1176, in register
res = self.graphql(
File "/home/runner/work/dwh/dwh/.venv/lib/python3.10/site-packages/prefect/client/client.py", line 570, in graphql
raise ClientError(result["errors"])
prefect.exceptions.ClientError: [{'path': ['create_flow_from_compressed_string'], 'message': 'Operation timed out', 'extensions': {'code': 'API_ERROR'}}]
Alex Dancho
02/17/2023, 12:25 AMBianca Hoch
02/17/2023, 2:47 PMTomas Moreno
02/17/2023, 2:53 PMBianca Hoch
02/17/2023, 2:54 PMTomas Moreno
02/17/2023, 2:56 PM[2021-01-14 15:00:00.000] DEBUG --- Serialized flow: {"__version__": "0.15.13", "edges": [], "environment": null, "name": "dbt_debug_canary", "parameters": [], "reference_tasks": [], "run_config": {"__version__": "0.15.13", "cpu": null, "env": {"ENVIRONMENT": "user", "PREFECT_PROJECT": "tmorenovasquez", "PREFECT__CLOUD__HEARTBEAT_MODE": "thread", "PREFECT__LOGGING__EXTRA_LOGGERS": "['flows']", "PREFECT__LOGGING__FORMAT": "%(levelname)s - %(name)s | %(message)s", "PREFECT__LOGGING__LEVEL": "DEBUG", "REDSHIFT_DBNAME": "tmorenovasquez", "REDSHIFT_HOST": "<http://non-prd-data-warehouse.cxqs6ydij0jm.us-west-2.redshift.amazonaws.com|non-prd-data-warehouse.cxqs6ydij0jm.us-west-2.redshift.amazonaws.com>", "REDSHIFT_PORT": "5439", "REDSHIFT_PW_NAME": "PREFECT-non-prd-data-warehouse-tmorenovasquez", "REDSHIFT_USERNAME": "tmorenovasquez"}, "execution_role_arn": null, "image": null, "labels": ["ecs"], "memory": null, "run_task_kwargs": {"cluster": "prd-prefect-tasks", "networkConfiguration": {"awsvpcConfiguration": {"assignPublicIp": "ENABLED", "securityGroups": ["sg-01206324ae14d8ee5", "sg-0d1550363c8c9ded6"], "subnets": ["subnet-0175d939b3fb111ac", "subnet-049e95fa3d88ac4d6", "subnet-074b1f2db66fcf8e9", "subnet-0dc0ac04035637572"]}}}, "task_definition": {"containerDefinitions": [{"image": "XXX", "logConfiguration": {"logDriver": "awslogs", "options": {"awslogs-group": "/ecs/prefect-tasks", "awslogs-region": "us-west-2", "awslogs-stream-prefix": "tmorenovasquez-dbt_debug_canary"}}, "name": "flow"}], "cpu": 512.0, "executionRoleArn": "arn:aws:iam::792470144447:role/prefect-ecs-execution-role", "family": "tmorenovasquez-dbt_debug_canary", "memory": 1024.0, "networkMode": "awsvpc", "taskRoleArn": "XXX"}, "task_definition_arn": null, "task_definition_path": null, "task_role_arn": null, "type": "ECSRun"}, "schedule": {"__version__": "0.15.13", "adjustments": [], "clocks": [{"__version__": "0.15.13", "cron": "00 10 * * *", "day_or": true, "end_date": null, "labels": null, "parameter_defaults": {}, "start_date": {"dt": "2021-01-01T00:00:00", "tz": "Etc/UTC"}, "type": "CronClock"}], "filters": [], "not_filters": [], "or_filters": []}, "storage": {"__version__": "0.15.13", "access_token_secret": "GITHUB_ACCESS_TOKEN", "base_url": null, "flows": {}, "path": "flows/flow_dbt_debug_canary.py", "ref": "express_partners_hotpatch", "repo": "DispatchHealth/dwh", "secrets": [], "type": "GitHub"}, "tasks": [{"__version__": "0.15.13", "auto_generated": false, "cache_for": null, "cache_key": null, "cache_validator": {"fn": "prefect.engine.cache_validators.never_use", "kwargs": {}}, "inputs": {"command": {"required": true, "type": "<class 'str'>"}, "dbt_kwargs": {"required": false, "type": "<class 'dict'>"}, "env": {"required": false, "type": "<class 'dict'>"}, "execute": {"required": false, "type": "<class 'bool'>"}, "helper_script": {"required": false, "type": "<class 'str'>"}, "options": {"required": false, "type": "<class 'str'>"}}, "max_retries": 3, "name": "DbtTask", "outputs": "typing.Optional[typing.List[str]]", "retry_delay": 600, "skip_on_upstream_skip": true, "slug": "DbtTask-1", "tags": [], "timeout": null, "trigger": {"fn": "prefect.triggers.all_successful", "kwargs": {}}, "type": "flows.utils.dbt.DbtTask"}], "type": "flows.utils.dhflow.DHFlow"}
[2021-01-14 15:00:00.000] DEBUG --- Hashed flow: b17c314c684e792a6a86e09e634adf722cc423a208fde6e79b9e1fa0b9b7df20
[2021-01-14 15:00:00.000] DEBUG --- Hashed register_flows.py adca5ee46c2931827cec7a963253763c0abfba38de9096ab6f7ade2d845586cf
[2021-01-14 15:00:00.000] DEBUG --- Full idempotency_key b17c314c684e792a6a86e09e634adf722cc423a208fde6e79b9e1fa0b9b7df20adca5ee46c2931827cec7a963253763c0abfba38de9096ab6f7ade2d845586cf
env
section and an export
statement just to see if it was any wonkiness with that and both time dout
- name: Register Flows feature branch
if: env.IS_PR == 'true' && env.SHOULD_RUN == 'true'
env:
PREFECT_PROJECT: ${{ github.event.pull_request.user.login }}
ENVIRONMENT: user
BRANCH: ${{ github.head_ref }}
run: |
source $VENV
export PREFECT__CLOUD__REQUEST_TIMEOUT=60
python deploy/register_flows.py -l DEBUG -b $BRANCH -e $ENVIRONMENT -t $BRANCH --no-schedule
alex
02/17/2023, 4:52 PMBianca Hoch
02/17/2023, 5:13 PMTomas Moreno
02/20/2023, 6:16 PM