John Marx
08/30/2021, 11:43 PMJohn Marx
08/30/2021, 11:43 PMYD
08/31/2021, 12:52 AMprefect agent local start
it still tries to find the python3.7
I pip uninstalled prefect and install it several times, but it does not help.
how can i clean install it so it will run using the python3.6 ?Pinakpani Mukherjee
08/31/2021, 1:44 AMAiden Price
08/31/2021, 3:17 AMWilson Bilkovich
08/31/2021, 3:36 AMLocal
storage when I'm building my own docker image that contains the flow. Do I understand that correctly?Bastian Röhrig
08/31/2021, 6:55 AMfrom prefect import Flow, task
from prefect.executors import LocalDaskExecutor
from prefect.engine.signals import SUCCESS
@task(timeout=60)
def succeed():
raise SUCCESS()
with Flow("timeout-test") as flow:
succeed()
flow.executor = LocalDaskExecutor()
flow.register(project_name="tutorial")
Is this the intended behaviour? Can anyone reproduce it?Tomoyuki NAKAMURA
08/31/2021, 9:08 AMSeonghwan Hong
08/31/2021, 10:06 AMSeonghwan Hong
08/31/2021, 10:06 AMSimon Gasse
08/31/2021, 12:59 PMmap
in downstream tasks? Let's say a mapped task skips for some inputs. In this case, I would still like the downstream task that runs on the return values of the mapped task to continue with the results of all successful task runs. An example is in the thread 🙂Wilson Bilkovich
08/31/2021, 1:19 PM/bin/sh
as an entrypoint. Is there any way to override that?Paulo Maia
08/31/2021, 2:45 PMNicholas Chammas
08/31/2021, 3:56 PMDidier Marin
08/31/2021, 4:06 PMMark McDonald
08/31/2021, 5:07 PMMark McDonald
08/31/2021, 7:48 PMTony Yun
08/31/2021, 8:35 PMstorage = Docker(
dockerfile="./Dockerfile"
)
But the register always report the module not exists.
ModuleNotFoundError: No module named 'criteo'
Wilson Bilkovich
08/31/2021, 8:41 PMWilson Bilkovich
08/31/2021, 11:36 PMHamza Ahmed
09/01/2021, 12:37 AMA Lazarus process attempted to reschedule this run 3 times without success. Marking as failed.
The only other errors are in the prefect towel logs [see comments]
Any clue as to what could be wrong?Aric Huang
09/01/2021, 12:41 AMProcessPoolExecutor
and am having trouble getting print statements and stack traces from subprocesses to show up in the Prefect Cloud UI. When running the flow locally the logs show up in stdout/stderr, and having the task defined with @task(log_stdout)
makes print statements in subprocesses show up with a └── <time> | INFO
prefix (which I thought means it's getting picked up by the Prefect logger). However the subprocess print statements don't show up at all in the Prefect Cloud UI logs. Stack traces from subprocesses that crash also print locally but don't show up in the UI, which only shows a stack trace for the main process with concurrent.futures.process.BrokenProcessPool
. Is there a way to have a flow capture all subprocess output so it's visible in the UI?Jason Kim
09/01/2021, 2:21 AMCA Lee
09/01/2021, 4:27 AMhaf
09/01/2021, 8:42 AMhaf
09/01/2021, 8:45 AMMichael
09/01/2021, 9:34 AMKonstantin
09/01/2021, 11:08 AMwith Flow(...)
block. If you're trying to run this task outside of a Flow context, you need to call `GetItem(...).run(...)`")Konstantin
09/01/2021, 12:08 PMHey there! how to add a token file to the storage in the prefect. for example, the error is this:
Task 'request_web [0]': Calling task.run () method ...
15:05:12
INFO
CloudTaskRunner
[Errno 2] No such file or directory: './Tokens/access_token.xml'
Evan Brown
09/01/2021, 2:23 PMEvan Brown
09/01/2021, 2:23 PMKevin Kho
09/01/2021, 2:24 PMEvan Brown
09/01/2021, 2:36 PMKevin Kho
09/01/2021, 2:41 PMlog_stdout=True
to ShellTask
? It inherits from Task so it can take all of the kwargs.Evan Brown
09/01/2021, 2:55 PMKevin Kho
09/01/2021, 2:57 PMstream_output=True
like this:
from os import path
from prefect import Flow
from prefect.tasks.shell import ShellTask
shelltask = ShellTask(
name="shell_task",
log_stderr=True,
return_all=True,
stream_output=True
)
with Flow("shelltest") as flow:
shelltask(command="echo 'test'")
flow.run()
Evan Brown
09/01/2021, 3:15 PMKevin Kho
09/01/2021, 3:15 PMEvan Brown
09/01/2021, 3:15 PMKevin Kho
09/01/2021, 3:16 PMEvan Brown
09/01/2021, 3:18 PM