Pranit
09/01/2022, 12:12 PMOscar Björhn
09/01/2022, 12:15 PMSlackbot
09/01/2022, 12:32 PMDavid Hlavaty
09/01/2022, 12:33 PMpython:3-9slim
and I then install all my dependencies including Prefect from a lock file. I prefer this over using the official Prefect images as it means there is a single source of truth for which version of Prefect I am using.
I had my image "misconfigured" by setting working directory to /opt/prefect/flows
. This then resulted in a cryptic message from shutil
when running my flows as it tried to copy files from the flow source directory ( /opt/prefect/flows
by default) to a working directory (/opt/prefect/flows
) which are the same.
Would be good if Prefect detected this and did not attempt to copy the files. Or at least checked if the working directory is the same and failed with helpful error message.Dennis Hinnenkamp
09/01/2022, 1:53 PMVenkat Ramakrishnan
09/01/2022, 1:59 PMVlad Tudor
09/01/2022, 2:09 PMdocker-compose
that runs on a remote VM.
However, I cannot configure the correct URL for the graphql
(when opening the UI, I get the error Couldn't connect to Prefect Server at <http://localhost:4200/graphql>
)
I tried to configure it from the config.toml
file with the URL of my machine:
[server]
[server.ui]
graphql_url = "http://<<MACHINE_PUBLIC_IP>>:4200/graphql"
but it still tries the access localhost.
Any help will be thoroughly appreciated. It's been 12 hours..Pranit
09/01/2022, 2:09 PMYoussef Ben Farhat
09/01/2022, 2:42 PMAlexander Kloumann
09/01/2022, 2:53 PMClint M
09/01/2022, 2:53 PMSam Garvis
09/01/2022, 3:35 PMBlake Stefansen
09/01/2022, 4:41 PMAdam Brusselback
09/01/2022, 5:17 PMVenkat Ramakrishnan
09/01/2022, 5:31 PMdeployment = Deployment.build_from_flow(flow=print_pipeline_hour,\
name="Hourly Pipeline Deployment", version="1", tags=["Iris"],\
schedule={'rrule': 'FREQ=HOURLY;UNTIL=20220912T040000Z', 'timezone': "Asia/Kolkata"},
work_queue_name="hour-work-queue")
Henning Holgersen
09/01/2022, 5:52 PMKrishnan Chandra
09/01/2022, 6:34 PMDockerContainer
and KubernetesJob
create new containers/jobs per flow run, but what I’d ideally like to do is run a fleet of long-running workers that process from the queue. What’s the best infrastructure to choose in Prefect 2.0 to achieve this?Kevin Grismore
09/01/2022, 6:36 PMParwez Noori
09/01/2022, 6:43 PMkiran
09/01/2022, 9:56 PMConcurrent
or Sequential
? In one section, the docs say the default is Concurrent
but then in another section, they say “Make sure you use .submit()
to run your task with a task runner. Calling the task directly, without .submit()
, from within a flow will run the task sequentially instead of using a specified task runner.” which seems to imply that the default is actually Sequential
Blake Hamm
09/01/2022, 11:23 PMKubernetesJob
blocks can interact with (the new) DockerContainer
block. Is there a way to deploy with --ib
as a DockerContainer
as well as use the KubernetesJob
? Or is there any roadmap to connect the two together (like how the DockerContainer
block can access a DockerRegistry
block?
I imagine this isn't feasible, but generally, I really like the ability to pass in a manifest file to a KubernetesJob
block. Specifically, I'm using EKS on AWS Fargate and really like the ability to define the resources. On the other hand the new DockerContainer
block seems really handy to manage environments for specific flows. Right now I have one image on ECR with all the dependencies and it's much heavier than it needs to be.
From a CI/CD perspective it would be great to have an action creating DockerContainer
blocks based on the individual flows and another action creating KubernetesJob
blocks based on different manifest files. In an ideal world, I would love a way to deploy using a DockerContainer
block as the image inside the KubernetesJob
block.
A current (hacky) solution could be to loop through the flow-level docker files, register them to ECR based on their flow name and create all the necessary KubernetesJob
blocks for each individual flow. I could use some kind of "resource" tag to pick the necessary manifest file. This would create a distinct KubernetesJob
block for every flow even though I might only be using 3 distinct manifest files (just lot's of unique containers for each flow). This would also require an --sb (unlike the new standalone DockerContainer
block.Venkat Ramakrishnan
09/02/2022, 4:27 AMAndreas Nord
09/02/2022, 10:04 AMADD flows /opt/prefect/flows
Which seems to be consistent with the deployment
storage: null
path: /opt/prefect/flows
entrypoint: flows\healthcheck.py:healthcheck
It seems that the deployment worked, but I can't find this path (/opt/prefect/flows) locally. I'm on WindowsKlemen Strojan
09/02/2022, 12:04 PMRoss Teach
09/02/2022, 1:21 PMprefect.exceptions.PrefectHTTPStatusError: Server error '500 Internal Server Error' for url '<https://api.prefect.cloud/api/accounts/043b2649-9d07-4c5e-8225-521ba2275e68/workspaces/689b139b-a725-4c2b-b167-86a705b8789d/task_runs/>'
Response: {'exception_message': 'Internal Server Error'}
For more information check: <https://httpstatuses.com/500>
José Duarte
09/02/2022, 1:47 PMSeth Goodman
09/02/2022, 2:25 PMkwmiebach
09/02/2022, 2:46 PMsqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 140193261803264 and this is thread id 140192735962880.
I can also paste the part of the code where sqlite is called.
I am also trying to guess the reason behind the error. There is an sqlite object created in a function. Within the function I define another function which uses this object. But both python functions are not prefect flows or tasks. They live within a bigger prefect flow. So here comes my first question: Does prefect 2 create a different thread for each nested function inside a flow or a task. Otherwise I cannot explain why the 2 parts of the code would run in different threads.Sam Garvis
09/02/2022, 2:48 PMprefect deployment build flowname.flowname -n flowname_dev --work-queue=dev-wq-1 -ib kubernetes-job/dev-k8s-job -sb gcs/dev --override image_pull_policy=Always
When I run this, the yaml created has this. Why does it still included the manifest.json? I thought that was deprecated
### DO NOT EDIT BELOW THIS LINE
###
flow_name: Colorado Watcher
manifest_path: colorado_watcher-manifest.json
Josh Paulin
09/02/2022, 4:55 PMJosh Paulin
09/02/2022, 4:55 PMEncountered exception during execution:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/prefect/engine.py", line 587, in orchestrate_flow_run
result = await run_sync(flow_call)
File "/usr/local/lib/python3.9/site-packages/prefect/utilities/asyncutils.py", line 56, in run_sync_in_worker_thread
return await anyio.to_thread.run_sync(call, cancellable=True)
File "/usr/local/lib/python3.9/site-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "activity_processor/flow/parent.py", line 43, in api_activity_processor
process_hourly_api(normalized_process_time, dry_run, validate_results)
File "/usr/local/lib/python3.9/site-packages/prefect/flows.py", line 384, in __call__
return enter_flow_run_engine_from_flow_call(
File "/usr/local/lib/python3.9/site-packages/prefect/engine.py", line 160, in enter_flow_run_engine_from_flow_call
return run_async_from_worker_thread(begin_run)
File "/usr/local/lib/python3.9/site-packages/prefect/utilities/asyncutils.py", line 136, in run_async_from_worker_thread
return anyio.from_thread.run(call)
File "/usr/local/lib/python3.9/site-packages/anyio/from_thread.py", line 49, in run
return asynclib.run_async_from_thread(func, *args)
File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 970, in run_async_from_thread
return f.result()
File "/usr/local/lib/python3.9/concurrent/futures/_base.py", line 446, in result
return self.__get_result()
File "/usr/local/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
raise self._exception
File "/usr/local/lib/python3.9/site-packages/prefect/client.py", line 104, in with_injected_client
return await fn(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/prefect/engine.py", line 435, in create_and_begin_subflow_run
flow_run.state.data._cache_data(await _retrieve_result(flow_run.state))
File "/usr/local/lib/python3.9/site-packages/prefect/results.py", line 38, in _retrieve_result
serialized_result = await _retrieve_serialized_result(state.data)
File "/usr/local/lib/python3.9/site-packages/prefect/client.py", line 104, in with_injected_client
return await fn(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/prefect/results.py", line 34, in _retrieve_serialized_result
return await filesystem.read_path(result.key)
File "/usr/local/lib/python3.9/site-packages/prefect/filesystems.py", line 149, in read_path
raise ValueError(f"Path {path} does not exist.")
ValueError: Path /root/.prefect/storage/4762715b441f4b3c8011b92dc1d5361f does not exist.
Michael Adkins
09/02/2022, 4:57 PMcreate_and_begin_subflow_run
we can see that it attempts to do flow_run.state.data._cache_data(await _retrieve_result(flow_run.state))
— this call is rehydrating the result on the stateJosh Paulin
09/02/2022, 6:26 PMMichael Adkins
09/02/2022, 8:55 PMJosh Paulin
09/02/2022, 9:02 PMMichael Adkins
09/02/2022, 9:10 PMJosh Paulin
09/02/2022, 9:51 PMMichael Adkins
09/02/2022, 9:54 PMJosh Paulin
09/02/2022, 9:56 PMMichael Adkins
09/02/2022, 9:57 PMJosh Paulin
09/02/2022, 10:15 PMMichael Adkins
09/02/2022, 10:59 PM