Luis Henrique
09/19/2022, 10:07 PMMichael Ng
09/19/2022, 10:17 PMKarim Soliman
09/19/2022, 11:45 PM16 September 2022,01:33:41 prefect.CloudTaskRunner INFO Task 'create_ingestion_params': Starting task run...
16 September 2022,01:33:41 prefect.CloudTaskRunner INFO Task 'create_ingestion_params': Finished task run for task with final state: 'Success'
16 September 2022,01:33:41 prefect.CloudTaskRunner INFO Task 'create_flow_run': Starting task run...
16 September 2022,01:33:41 prefect.CloudTaskRunner INFO Task 'create_flow_run': Finished task run for task with final state: 'Mapped'
16 September 2022,01:33:42 prefect.CloudTaskRunner INFO Task 'create_flow_run[0]': Starting task run...
16 September 2022,01:33:42 prefect.create_flow_run[0] INFO Creating flow run 'fierce-flounder-Tome Ingestion' for flow 'Tome Ingestion'...
16 September 2022,01:33:42 prefect.create_flow_run[0] INFO Created flow run 'fierce-flounder-Tome Ingestion': <https://cloud.prefect.io/tome/flow-run/8d5af013-baa8-4830-be75-7d212a4f3e74>
16 September 2022,01:33:42 prefect.CloudTaskRunner INFO Task 'create_flow_run[0]': Finished task run for task with final state: 'Success'
16 September 2022,01:33:43 prefect.CloudTaskRunner INFO Task 'create_flow_run[1]': Starting task run...
16 September 2022,01:33:43 prefect.create_flow_run[1] INFO Creating flow run 'fierce-flounder-Tome Ingestion' for flow 'Tome Ingestion'...
16 September 2022,01:33:43 prefect.create_flow_run[1] INFO Created flow run 'fierce-flounder-Tome Ingestion': <https://cloud.prefect.io/tome/flow-run/7729a5c3-5fb3-474f-bfbc-7d2fbce841d8>
16 September 2022,01:33:44 prefect.CloudTaskRunner INFO Task 'create_flow_run[1]': Finished task run for task with final state: 'Success'
log from my local python execution
[2022-09-19 23:44:08+0000] INFO - prefect.CloudTaskRunner | Task 'create_ingestion_params': Starting task run...
INFO [task_runner.py:241] Task 'create_ingestion_params': Starting task run...
[2022-09-19 23:44:14+0000] INFO - prefect.CloudTaskRunner | Task 'create_ingestion_params': Finished task run for task with final state: 'Success'
INFO [task_runner.py:335] Task 'create_ingestion_params': Finished task run for task with final state: 'Success'
[2022-09-19 23:44:14+0000] INFO - prefect.CloudTaskRunner | Task 'create_flow_run': Starting task run...
INFO [task_runner.py:241] Task 'create_flow_run': Starting task run...
[2022-09-19 23:44:14+0000] INFO - prefect.CloudTaskRunner | Task 'create_flow_run': Finished task run for task with final state: 'Mapped'
INFO [task_runner.py:335] Task 'create_flow_run': Finished task run for task with final state: 'Mapped'
[2022-09-19 23:44:15+0000] INFO - prefect.CloudTaskRunner | Task 'get_task_run_result': Starting task run...
INFO [task_runner.py:241] Task 'get_task_run_result': Starting task run...
Stephen Lloyd
09/20/2022, 3:17 AMStephen Lloyd
09/20/2022, 3:18 AMChing
09/20/2022, 7:27 AMJP
09/20/2022, 8:26 AM10
but are plenty, when it is left to Unlimited?
When there are no visible scheduled jobs, my agent polling that queue would not retrieve any jobs either 😕 As soon as I change it to Unlimited
, it comes to life and starts running all the late jobs like nothing had happened.
I’m using Prefect Cloud v2Carlos Paiva
09/20/2022, 8:29 AMAndrei Tulbure
09/20/2022, 8:42 AMAndreas Nord
09/20/2022, 8:55 AMAndreas Nord
09/20/2022, 9:07 AM08:57:40.650 | INFO | Task run 'task_name-6b80d8b6-24' - Finished in state Completed()
08:57:59.061 | INFO | prefect.infrastructure.docker-container - Docker container 'adaptable-quoll' has status 'exited'
And the Prefect Cloud log like this:
Created task run 'task_name-6b80d8b6-24' for task 'task_name'
10:57:39 AM
Executing 'task_name-6b80d8b6-24' immediately..
The flow run is stuck at 'running' in the Prefect Cloud UI.
This is only the case for one of my flows, but the final tasks of the flow are the same tasks as for my other flows.José Duarte
09/20/2022, 12:10 PMVadym Dytyniak
09/20/2022, 12:41 PMpydantic.error_wrappers.ValidationError: 9 validation errors for Deployment
schedule -> interval
field required (type=value_error.missing)
schedule -> calendar_id
extra fields not permitted (type=value_error.extra)
schedule -> run_times
extra fields not permitted (type=value_error.extra)
schedule -> cron
field required (type=value_error.missing)
schedule -> calendar_id
extra fields not permitted (type=value_error.extra)
schedule -> run_times
extra fields not permitted (type=value_error.extra)
schedule -> rrule
field required (type=value_error.missing)
schedule -> calendar_id
extra fields not permitted (type=value_error.extra)
schedule -> run_times
extra fields not permitted (type=value_error.extra)
Robin Weiß
09/20/2022, 1:56 PMprefect deployment run flow/deployment paramA=1 paramB=2
Am I missing something super obvious here? 😅
Thanks!Evan Curtin
09/20/2022, 1:56 PMResult
equivalent in prefect 2.0? 🙂Vadym Dytyniak
09/20/2022, 2:37 PMRDATE:20220920T200000Z,20220921T133000Z
UI says: 'Every year' schedule:Vadym Dytyniak
09/20/2022, 2:41 PMSam Garvis
09/20/2022, 2:43 PMAlexandru Anghel
09/20/2022, 3:21 PMTypeError: cannot pickle '_thread.RLock' object
I can see the application being registered inside the spark cluster.
Thanks!badasstronaut
09/20/2022, 4:06 PMconda
package for prefect-aws
. I can just use pip
to keep moving forward, but I was curious about the inconsistency. Other prefect packages are available.Chris Gunderson
09/20/2022, 5:00 PMMadison Schott
09/20/2022, 7:11 PMMadison Schott
09/20/2022, 7:50 PMwith Flow("data_pipeline_test_new_account", storage=STORAGE, run_config=RUN_CONFIG) as data_pipeline_test_new_account:
Chris Gunderson
09/20/2022, 10:11 PMRUN /bin/sh -c "prefect cloud login -k $(cat ./api_key.txt) --workspace $(cat ./workspace.txt)"
Faheem Khan
09/21/2022, 12:24 AMinternal server error
. Other errors that appears from agent logs are, RunTime error: The connection pool was closed while 9 HTTP requests/responses were still in flight.
Ryan Sattler
09/21/2022, 1:03 AMDan Horne
09/21/2022, 4:55 AMYaron Levi
09/21/2022, 5:03 AMVaikath Job
09/21/2022, 5:25 AMNic
09/21/2022, 6:42 AMNic
09/21/2022, 6:42 AMMason Menges
09/21/2022, 3:23 PM