itay livni
01/08/2021, 2:04 AMFailed to load and execute Flow's environment: SyntaxError("invalid or missing encoding declaration for '/home/.../MOC/etl_moc.py'")
-After the flow began and executed some tasks successfully. The flow was run locally. Is this a bug or something not configured correctly on my end? ThanksJoël Luijmes
01/08/2021, 12:32 PMDan Corbiani
01/08/2021, 2:31 PMJeff Williams
01/08/2021, 2:39 PMMarwan Sarieddine
01/08/2021, 2:58 PMVipul
01/08/2021, 3:27 PMMatt
01/08/2021, 8:28 PMmatta
01/09/2021, 2:46 AMFailed to retrieve task state with error: ValidationError({'_schema': 'Invalid data type: None'},)
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/prefect/engine/cloud/task_runner.py", line 193, in initialize_run
map_index=map_index,
File "/usr/local/lib/python3.6/site-packages/prefect/client/client.py", line 1387, in get_task_run_info
state = prefect.engine.state.State.deserialize(task_run_info.serialized_state)
File "/usr/local/lib/python3.6/site-packages/prefect/engine/state.py", line 362, in deserialize
state = StateSchema().load(json_blob)
File "/usr/local/lib/python3.6/site-packages/marshmallow_oneofschema/one_of_schema.py", line 144, in load
raise exc
marshmallow.exceptions.ValidationError: {'_schema': 'Invalid data type: None'}
Using 0.13.19 right now
Looks like some kind of serialization problem?Dr. Jones
01/09/2021, 11:26 AM[2021-01-09 11:20:53,118] ERROR - My_Agent | 400 Client Error: Bad Request for url: <http://apollo:4200/>
The following error messages were provided by the GraphQL server:
GRAPHQL_VALIDATION_FAILED: Cannot query field "run_config" on type "flow_run".
The GraphQL query was:
query {
flow_run(where: { id: { _in: ["84b340e4-9f74-425b-9abd-27247fdc3759"] }, _or: [{ state: { _eq: "Scheduled" } }, { state: { _eq: "Running" }, task_runs: { state_start_time: { _lte: "2021-01-09T11:20:50.061124+00:00" } } }] }) {
id
version
state
serialized_state
parameters
scheduled_start_time
run_config
flow {
core_version
name
storage
id
environment
version
}
task_runs(where: { state_start_time: { _lte: "2021-01-09T11:20:50.061124+00:00" } }) {
serialized_state
version
task_id
id
}
}
}
The passed variables were:
null
Prash S
01/09/2021, 6:41 PMjeff n
01/10/2021, 12:02 AMJosh Pitts
01/10/2021, 2:17 AMJosh Pitts
01/10/2021, 2:17 AM@task(target="func_task_target.txt", checkpoint=True, result=LocalResult(dir="~/.prefect"))
def func_task():
return 99
Josh Pitts
01/10/2021, 2:17 AMJosh Pitts
01/10/2021, 2:18 AMJosh Pitts
01/10/2021, 2:30 AMPREFECT__FLOWS__CHECKPOINTING=true
, what is actually happening regarding target
, checkpoint=True
, and result=LocalTarget()
? or are those just ignored?Josh Pitts
01/10/2021, 3:12 AMDanny Vilela
01/10/2021, 11:35 PMPedro Machado
01/11/2021, 4:04 AMDbtShellTask
, it doesn't show output why it's running or when it succeeds. The only time I see the detailed output is when there is a failure.
I tried passing these args and it still doesn't show streamed output in the logs, not even after the task completes.
return_all=True,
log_stdout=True,
log_stderr=True,
I also set the PREFECT__LOGGING__LEVEL=DEBUG
everywhere I could think of: prefect server, agent, docker storage, run config. I see DEBUG level messages on Prefect Server, but nothing from the task.
I am running 0.14.2
Any ideas?Maurits de Ruiter
01/11/2021, 9:54 AMJarek Piotrowski
01/11/2021, 1:27 PMVincent
01/11/2021, 2:24 PMdistributed.utils_perf - WARNING - full garbage collections took 10% CPU time recently (threshold: 10%)
...
distributed.core - INFO - Event loop was unresponsive in Scheduler for 7.07s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
...
distributed.comm.tcp - INFO - Connection closed before handshake completed
...
distributed.scheduler - INFO - Close client connection: Client-worker-3c6d8642-53b5-11eb-800e-32b98c347770
When I scale the job down to 250 nodes and 3 threads per worker, I still get 100% utilization, but it is slightly more stable. where the only warning messages is
distributed.utils_perf - WARNING - full garbage collections took 11% CPU time recently (threshold: 10%)
Thanks for any advice!Equipe AI HOC
01/11/2021, 3:06 PMBrett Naul
01/11/2021, 5:08 PMRiley Hun
01/11/2021, 6:43 PMBilly McMonagle
01/11/2021, 7:18 PMprefect agent <AGENT TYPE> start --env KEY=VALUE --env KEY2=VALUE2
Instead, is it possible to do this by setting an environment variable on the agent, similar to how labels are set?
- name: PREFECT__CLOUD__AGENT__LABELS
value: "[]"
Peter Roelants
01/11/2021, 7:20 PMflow.register
the creation and registration need to happen in the same call. Is there an example somewhere on how to decouple these steps?
For example how to create and store a Docker build artefact that encapsulate a flow, and running/registering the flow stored in the Docker artefact at a later time without access to the original flow file.Joël Luijmes
01/11/2021, 7:33 PMJeremy Phelps
01/11/2021, 10:16 PMgoogle.cloud
Python library does Prefect depend on?Jeremy Phelps
01/11/2021, 11:10 PMFailed to load and execute Flow's environment: DefaultCredentialsError('Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see <https://cloud.google.com/docs/authentication/getting-started>')