Aiden Price
08/31/2021, 3:17 AMWilson Bilkovich
08/31/2021, 3:36 AMLocal
storage when I'm building my own docker image that contains the flow. Do I understand that correctly?Bastian Röhrig
08/31/2021, 6:55 AMfrom prefect import Flow, task
from prefect.executors import LocalDaskExecutor
from prefect.engine.signals import SUCCESS
@task(timeout=60)
def succeed():
raise SUCCESS()
with Flow("timeout-test") as flow:
succeed()
flow.executor = LocalDaskExecutor()
flow.register(project_name="tutorial")
Is this the intended behaviour? Can anyone reproduce it?Tomoyuki NAKAMURA
08/31/2021, 9:08 AMSeonghwan Hong
08/31/2021, 10:06 AMSeonghwan Hong
08/31/2021, 10:06 AMSimon Gasse
08/31/2021, 12:59 PMmap
in downstream tasks? Let's say a mapped task skips for some inputs. In this case, I would still like the downstream task that runs on the return values of the mapped task to continue with the results of all successful task runs. An example is in the thread 🙂Wilson Bilkovich
08/31/2021, 1:19 PM/bin/sh
as an entrypoint. Is there any way to override that?Paulo Maia
08/31/2021, 2:45 PMNicholas Chammas
08/31/2021, 3:56 PMDidier Marin
08/31/2021, 4:06 PMMark McDonald
08/31/2021, 5:07 PMMark McDonald
08/31/2021, 7:48 PMTony Yun
08/31/2021, 8:35 PMstorage = Docker(
dockerfile="./Dockerfile"
)
But the register always report the module not exists.
ModuleNotFoundError: No module named 'criteo'
Wilson Bilkovich
08/31/2021, 8:41 PMWilson Bilkovich
08/31/2021, 11:36 PMHamza Ahmed
09/01/2021, 12:37 AMA Lazarus process attempted to reschedule this run 3 times without success. Marking as failed.
The only other errors are in the prefect towel logs [see comments]
Any clue as to what could be wrong?Aric Huang
09/01/2021, 12:41 AMProcessPoolExecutor
and am having trouble getting print statements and stack traces from subprocesses to show up in the Prefect Cloud UI. When running the flow locally the logs show up in stdout/stderr, and having the task defined with @task(log_stdout)
makes print statements in subprocesses show up with a └── <time> | INFO
prefix (which I thought means it's getting picked up by the Prefect logger). However the subprocess print statements don't show up at all in the Prefect Cloud UI logs. Stack traces from subprocesses that crash also print locally but don't show up in the UI, which only shows a stack trace for the main process with concurrent.futures.process.BrokenProcessPool
. Is there a way to have a flow capture all subprocess output so it's visible in the UI?Jason Kim
09/01/2021, 2:21 AMCA Lee
09/01/2021, 4:27 AMhaf
09/01/2021, 8:42 AMhaf
09/01/2021, 8:45 AMMichael
09/01/2021, 9:34 AMKonstantin
09/01/2021, 11:08 AMwith Flow(...)
block. If you're trying to run this task outside of a Flow context, you need to call `GetItem(...).run(...)`")Konstantin
09/01/2021, 12:08 PMHey there! how to add a token file to the storage in the prefect. for example, the error is this:
Task 'request_web [0]': Calling task.run () method ...
15:05:12
INFO
CloudTaskRunner
[Errno 2] No such file or directory: './Tokens/access_token.xml'
Evan Brown
09/01/2021, 2:23 PMDan Stoner
09/01/2021, 2:28 PMdelete project
.
$ prefect delete --help
Usage: prefect delete [OPTIONS] COMMAND [ARGS]...
Delete commands that refer to mutations of Prefect API metadata.
Usage:
$ prefect delete [OBJECT]
Arguments:
project Delete projects
...
marios
09/01/2021, 2:41 PMAn Hoang
09/01/2021, 3:28 PMParameter
as the default for another? Here's what I did
#task
@task(target= {output_folder_path}/result)
def example_task(output_folder_path)
....
#in flow
step1_folder_path = Parameter("step1_folder_path")
output_folder_path = Parameter("output_folder_path", default = step1_folder_path)
When I do this, I got a warning
<ipython-input-226-3b472991f1ef>:8: UserWarning: A Task was passed as an argument to Parameter, you likely want to first initialize Parameter with any static (non-Task) arguments, then call the initialized task with any dynamic (Task) arguments instead. For example:
my_task = Parameter(...) # static (non-Task) args go here
res = my_task(...) # dynamic (Task) args go here
see <https://docs.prefect.io/core/concepts/flows.html#apis> for more info.
output_folder_path = Parameter("output_folder_path", default = step1_folder_path)
When I execute the flow, it runs but the target
doesn't work as intended, the task still runs every time even though the target files are there. When I provide a hardcoded string to the default of output_folder_path
, everything works fineSamuel Hinton
09/01/2021, 4:01 PM