Omar Sultan
10/16/2022, 4:09 PMchicago-joe
10/16/2022, 5:26 PMchicago-joe
10/16/2022, 10:27 PMFile "/usr/local/lib/python3.9/site-packages/prefect/utilities/dispatch.py", line 186, in lookup_type
raise KeyError(
KeyError: "No class found for dispatch key 'ecs-task' in registry for type 'Block'."
Santhosh Solomon (Fluffy)
10/17/2022, 3:25 AMYD
10/17/2022, 4:14 AMFinished task run for task with final state: 'TimedOut'
after 3 second the task is running
even though the task is set to @task(timeout=600, **params)
and it was running fine till friday.
it also runs fine when using flow.run()
I am using https://cloud.prefect.io/
any suggestions?
I deleted the flow an reregister it, but it did not helpDarren Fleetwood
10/17/2022, 4:58 AMMark
10/17/2022, 7:54 AMSULEMAN KHAN
10/17/2022, 8:43 AMcache
when running it with celery using this command.
celery -A app.tasks.celery_app worker --max-tasks-per-child=1 --loglevel=ERROR
Everything works fine when I use this command
celery -A app.tasks.celery_app worker --loglevel=ERROR
Jared Robbins
10/17/2022, 1:00 PMvk
10/17/2022, 1:02 PMflow.storage = Module('samples.flow_orion')
Now i’m kind of forced to specify exact full filename, even though I have file in python path.
That’s very inconvenient for cases when developers have no control over the final image layout.
That’s how I create deployments now:
entrypoint = 'samples.flow_orion:entrypoint' # this is just module, available on path
flow = prefect.utilities.importtools.import_object(entrypoint)
# here is path/entrypoint i need to specify path. could I use module name instead?
deployment = Deployment.build_from_flow(
name=flow.name,
flow=flow,
work_queue_name="kubernetes",
skip_upload=True,
infrastructure=KubernetesJob(
image=image,
namespace='prefect2',
job=k8s_template_orion(flow)
),
path='/app/lib/python3.9/site-packages/samples/', # TODO
entrypoint='flow_orion.py:entrypoint'
)
deployment.apply()
Did i miss how to use module name instead of path?Nic
10/17/2022, 1:42 PMMark Li
10/17/2022, 1:43 PMJustin Trautmann
10/17/2022, 2:35 PMprefect concurrency-limit inspect
command but is in fact in status completed. this effectively lowers the concurrency-limit for all subsequent tasks as these pseudo-active tasks keep occupying a slot for the tag. are we at all supposed to adapt the concurrency limit while tagged tasks are being executed or is this considered bad practice?
Please find in the thread a self-contained example flow that can be used to reproduce the behavior. Any advice i much appreciated. Thank you.redsquare
10/17/2022, 2:57 PM/usr/local/bin/python: No module named prefect.engine.__main__; 'prefect.engine' is a package and cannot be directly executed
Evan Curtin
10/17/2022, 3:17 PMKelvin DeCosta
10/17/2022, 5:03 PMECSTask
infrastructure.
I'm programmatically creating a Deployment
for a simple flow and it seems to have registered on our Prefect Cloud.
However, the deployment run fails without log messages in Prefect Cloud.
The agent logs botocore.exceptions.ParamValidationError
, which makes me think this might be out of my reach.
I'm currently in what seems to be the exact same situation as @Anthony Desmier (link to thread) is in.
Not really sure where to go next.
Any help is appreciated a lot, thank you for your time!Ben Ayers-Glassey
10/17/2022, 5:15 PMprefect deployment apply
command has a typo in its output:
Successfully loaded 'sampleflow_dev'
Successfully uploaded 4 files to <gcs://dev-prefect-work/flow_storage/>
Deployment 'sampleflow/sampleflow_dev' successfully created with id 'c3b605e4-ca96-45eb-b51a-b048ca6d2e08'.
View Deployment in UI: <https://app.prefect.cloud/account/a9c0f124-ca06-4646-a501-57a405ebf3c7/workspace/43f1b8a7-ed9c-46d2-a88a-55eef95b8ef7/deployment/c3b605e4-ca96-45eb-b51a-b048ca6d2e08>
To execute flow runs from this deployment, start an agent that pulls work from the 'kubernetes' work queue:
$ prefect agent start -q 'kubernetes'
...that link it shows is incorrect:
https://app.prefect.cloud/account/a9c0f124-ca06-4646-a501-57a405ebf3c7/workspace/43f1[…]55eef95b8ef7/deployment/c3b605e4-ca96-45eb-b51a-b048ca6d2e08
...it should be:
https://app.prefect.cloud/account/a9c0f124-ca06-4646-a501-57a405ebf3c7/workspace/43f1[…]/deployments/deployment/c3b605e4-ca96-45eb-b51a-b048ca6d2e08
...in other words, it's missing a "/deployments"Kevin Wang
10/17/2022, 5:32 PMSean Davis
10/17/2022, 5:56 PMasyncio.Semaphore
or asyncio.Queue
and the like? Thanks for any suggestions or pointers.Xavier Lesperance
10/17/2022, 6:34 PMJarvis Stubblefield
10/17/2022, 8:52 PM10 failed, 4486 passed, 39 skipped, 3 xfailed, 11 errors in 1173.85s (0:19:33)
I wasn’t expecting to have failures… I’ve added my code and am now running the 20 minutes of tests to ensure it’s working as expected… I’m assuming so long as I don’t get to 11 failed
in the summary I should be okay right?Cameron Chafetz
10/17/2022, 10:21 PMkent
10/18/2022, 1:11 AMfrom prefect import flow,get_run_logger
@flow
def my_favorite_function():
logger = get_run_logger()
<http://logger.info|logger.info>("aaaaaaaaaaaaaaaaaaaaaaaaaa")
print("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")
return 42
prefect deployment build flows/tutorial.py:my_favorite_function -n etl --cron "*/1 * * * *" -q etl --apply
prefect agent start -q 'etl
Is there a problem?John Ramey
10/18/2022, 2:03 AMTask.map
can do something similar to generate tasks, but I’d prefer subflows for my project. Is this possible? If so, any docs or resources y’all can point me to? Thanks!Shane Hu
10/18/2022, 2:13 AM[tool.poetry.dependencies]
python = "^3.10"
prefect = "^2.6.1"
playwright = "^1.27.1"
import prefect
from playwright.sync_api import Page, sync_playwright
@prefect.task
def go_to_a_web(page: Page):
page.goto("<https://formulae.brew.sh/cask/slack>")
@prefect.flow
def main_flow():
with sync_playwright() as pw:
browser = pw.chromium.launch(headless=False)
page = browser.new_page()
go_to_a_web(page)
main_flow()
I got
greenlet.error: cannot switch to a different thread
any help would be much appreciatedShrikkanth
10/18/2022, 5:27 AMdef post_to_slack_on_success(task, old_state, new_state):
if new_state.is_successful():
flow_run = FlowRunView.from_flow_run_id(prefect.context.get("flow_run_id"))
task_run = flow_run.get_task_run(task_run_id=prefect.context.get("task_run_id"))
msg = task_run.get_result()
SlackTask(message=msg).run()
flow.state_handlers.append(slack)
Second flow function try:
def post_to_slack_on_success(task, old_state, new_state):
if new_state.is_successful():
print(new_state)
print(old_state)
print("Result : ", new_state.result, new_state.message)
Screenshot of the alert:Shruti Hande
10/18/2022, 5:58 AMAndreas Nigg
10/18/2022, 6:04 AMAngelika Tarnawa
10/18/2022, 8:30 AMprefect.exceptions.ClientError: [{'path': ['get_or_create_task_run_info'], 'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'locations': [{'line': 2, 'column': 101}], 'path': None}}}]
I have tried to re-register flow and I'm sure that flow run and registered flow are the sameSlackbot
10/18/2022, 10:21 AMSlackbot
10/18/2022, 10:21 AM