Andreas
06/14/2022, 4:41 PMJehan Abduljabbar
06/14/2022, 5:26 PMRio McMahon
06/14/2022, 5:42 PMsklearn
or similar into it) then run prefect register --project "Project Name" -l label -p flow.py
from within the pipeline. Realistically I could probably use the container that I am running the flow from within but am curious if there is an easier way. Thanks.Tony Yun
06/14/2022, 6:26 PMTask 'RunNamespacedJob - DBT Run': Finished task run for task with final state: 'Queued'
Paco Ibañez
06/14/2022, 8:24 PMaaron
06/14/2022, 9:21 PMstored_as_script=True
). I’m attempting to use the ExecuteNotebook task and I’m getting the following only with debug logs on the Docker agent:
[2022-06-14 20:54:12+0000] INFO - prefect.CloudFlowRunner | Beginning Flow run for 'Test Docker Notebook Flow'
[2022-06-14 20:54:13+0000] INFO - prefect.CloudTaskRunner | Task 'notebook0': Starting task run...
Executing: 0%| | 0/2 [00:00<?, ?cell/s]Operation not permitted (src/thread.cpp:309)
This is erroring out in the container when trying to access the notebook but not making it back to the Prefect API to change task state (the task run just hangs until it times out with a No heartbeat detected from the remote task
error). Any ideas?Ahmed Ezzat
06/14/2022, 10:51 PM2.0b6
Jovan Sakovic
06/14/2022, 11:12 PMFaheem Khan
06/15/2022, 12:02 AMWalter Cavinaw
06/15/2022, 2:35 AMwonsun
06/15/2022, 7:03 AMconfig.toml
file be in the path where prefect is installed? There is only backend.toml
file in the path where my prefect is installed, are those two the same?
Also i can't read backend.toml
. I got the error message like below:
TomlDecodeError: Found invalid character in key name: ':'. Try quoting the key name. (line 1 column 2 char 1)
What's the backend.toml file?Jeff Kehler
06/15/2022, 7:38 AMapollo_url
under [server.ui]
in the .prefect/config.toml
file does not appear to be working. How do I change the host that the UI will call to access the API?Andreas
06/15/2022, 10:17 AMDaskTaskRunner
tasks can stay in a pending state forever if they have custom dependencies (using _`wait_for`)_ on tasks that have failed. The failed state of the upstream tasks do not cause the downstream tasks to fail.. This isn't the case when using the default ConcurrentTaskRunner
where the failed state correctly propagates downwards and causes the dependent tasks to fail by defaultSheila
06/15/2022, 10:31 AMMatt Delacour
06/15/2022, 10:45 AM[2022-06-15 110852+0200] ERROR - prefect.S3 | Error uploading Flow to S3 bucket prefect-flows-proto: An error occurred (SignatureDoesNotMatch) when calling the PutObject operation: The request signature we calculated does not match the signature you provided. Check your key and signing method.Error the error + snippet of code
frojo
06/15/2022, 10:52 AMwith Flow('slack-test',state_handlers=[post_to_slack]) as flow_slack:
ini_message="Some Init Message"
#
post_slack_init(ini_message)
#
count()
We get the initial flow message start (post_to_slack
notifies us in start/end/failure), but we don't get
the "ini_message" until the flow ends. Even more if count
task (wich also carries a state handler) fails we get the fail of the task in first place and then the post_slack_init
(just a post request no state handler on this task) message. Any Clues? Thanks in advance!Toby Rahloff
06/15/2022, 10:59 AMApostolos Papafragkakis
06/15/2022, 12:20 PMGintautas Jankus
06/15/2022, 1:05 PMAn error occurred (AccessDeniedException) when calling the RegisterTaskDefinition operation: User: is not authorized to perform: ecs:RegisterTaskDefinition on resource: * because no identity-based policy allows the ecs:RegisterTaskDefinition action
I have tried also to pass flow_a task_definition_arn to parent_flow, then it runs flow, but gets this error:`in create_flow_run ValueError: Received both flow_id
and flow_name
. Only one flow identifier can be passed.` Maybe someone can share the best practices how I could create flow that would run in order these two already registered flows ? Big thanks in advance ! 🙂Adam
06/15/2022, 1:54 PMflow.register
and getting an error from the requests
module, presumably when it’s trying to register the flow to prefect cloud. It’s getting a webpage in the response, with a lot of obfuscated js. Any ideas why this is happening? We’re using prefect 0.14 on python 3.10.4Matteo Fiorillo
06/15/2022, 2:33 PMMia
06/15/2022, 4:09 PMJacob Bedard
06/15/2022, 5:07 PMOmar Sultan
06/15/2022, 5:57 PMError during execution of task: ConnectTimeout(MaxRetryError("HTTPConnectionPool(host='prefect-apollo.prefect', port=4200): Max retries exceeded with url: /graphql (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f32b0d48910>, 'Connection to prefect-apollo.prefect timed out. (connect timeout=60)'))"))
This seems to affect all running tasks at the time, so for example if I have 2 or 3 tasks running, they all fail at the exact same time and give this errorR
06/15/2022, 7:00 PMJoshua Allen
06/15/2022, 7:18 PMJoshua Allen
06/15/2022, 7:20 PMAlex de Geofroy
06/15/2022, 7:21 PMPedro Machado
06/15/2022, 7:57 PMenv
argument of a class derived from ShellTask
.
bash = LoggedShellTask(stream_output=True)
# more stuff here ...
with Flow(
FLOW_NAME, storage=docker_storage, run_config=run_config, result=PrefectResult()
) as flow:
run_results = bash(
helper_script=get_helper_script(repo_path),
command=get_command(ml_script_path, ml_script_arguments, conda_env=conda_env),
env=dict(
SNOWFLAKE_USER=snowflake_user,
SNOWFLAKE_PASS=snowflake_pass,
SNOWFLAKE_DATABASE=snowflake_database,
SNOWFLAKE_OUTPUT_SCHEMA=snowflake_output_schema,
),
log_file_path=get_log_file_path(output_dir),
)
The issue is Prefect automatically creates a List
and Dict
task that have PrefectResults
by default and this exposes the secrets in the UI.
A couple of ideas that come to mind:
1. Assign a specific results objects to each task (not at the flow level)
2. Create a wrapper task that receives all the secrets and returns a dict that is passed to the ShellTask
3. Create a ShellTask
that accepts each secret as a parameter
Any suggestions? One feature I'd like to preserve is the ability to restart the flow and I'd rather not persist the secrets anywhere.
Thanks!ibrahem
06/15/2022, 8:42 PM