Nathan Low
04/06/2023, 12:46 PMDownloading flow code from storage at 'EzeCastle'
08:11:02 AM
DEBUG
Importing flow code from 'ezecastle_daily_p_and_l_loader.py:hourly_loader'
08:11:02 AM
ERROR
Flow could not be retrieved from deployment.
Traceback (most recent call last):
File "<frozen importlib._bootstrap_external>", line 846, in exec_module
File "<frozen importlib._bootstrap_external>", line 982, in get_code
File "<frozen importlib._bootstrap_external>", line 1039, in get_data
FileNotFoundError: [Errno 2] No such file or directory: 'ezecastle_daily_p_and_l_loader.py'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\PrefectAgent\.venv\lib\site-packages\prefect\engine.py", line 268, in retrieve_flow_then_begin_flow_run
flow = await load_flow_from_flow_run(flow_run, client=client)
File "C:\PrefectAgent\.venv\lib\site-packages\prefect\client\utilities.py", line 47, in with_injected_client
return await fn(*args, **kwargs)
File "C:\PrefectAgent\.venv\lib\site-packages\prefect\deployments.py", line 187, in load_flow_from_flow_run
flow = await run_sync_in_worker_thread(load_flow_from_entrypoint, str(import_path))
File "C:\PrefectAgent\.venv\lib\site-packages\prefect\utilities\asyncutils.py", line 91, in run_sync_in_worker_thread
return await <http://anyio.to|anyio.to>_thread.run_sync(
File "C:\PrefectAgent\.venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "C:\PrefectAgent\.venv\lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "C:\PrefectAgent\.venv\lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run
result = context.run(func, *args)
File "C:\PrefectAgent\.venv\lib\site-packages\prefect\flows.py", line 786, in load_flow_from_entrypoint
flow = import_object(entrypoint)
File "C:\PrefectAgent\.venv\lib\site-packages\prefect\utilities\importtools.py", line 195, in import_object
module = load_script_as_module(script_path)
File "C:\PrefectAgent\.venv\lib\site-packages\prefect\utilities\importtools.py", line 158, in load_script_as_module
raise ScriptError(user_exc=exc, path=path) from exc
prefect.exceptions.ScriptError: Script at 'ezecastle_daily_p_and_l_loader.py' encountered an exception: FileNotFoundError(2, 'No such file or directory')
Nelson Griffiths
04/06/2023, 1:30 PMNathan Low
04/06/2023, 1:51 PMFilip Panovski
04/06/2023, 3:24 PMpath
kwarg when creating your deployments? I had some issues with that some time ago as well. Since my repo is (still) small, I ended up just placing all my flow code on the root level and omitted the path
kwarg.Nelson Griffiths
04/06/2023, 4:22 PMpath
. I actually have a need for it unless I want to set up a separate bucket for each flow, which I don't currently want to. @Nathan Low I fixed this by simply changing my path
arg and deploying again. No clue why this worked. Seems like a bug but also have no idea how to reproduce it.Jeff Hale
04/07/2023, 3:59 PMprefect version
?Nathan Low
04/07/2023, 4:04 PMJeff Hale
04/07/2023, 4:32 PMprefect version
result?Nelson Griffiths
04/07/2023, 4:35 PM2.10.2
release. Was chatting in another thread about this with @Chris White and he got things moving and a bug fix in.Jeff Hale
04/07/2023, 4:41 PMNathan Low
04/07/2023, 8:55 PMJeff Hale
04/10/2023, 3:12 PMice_liquidity_loader.py
directly under the ICE
directory?Nathan Low
04/10/2023, 3:14 PMChris White
04/10/2023, 3:16 PMNathan Low
04/10/2023, 3:27 PM# import ice_liquidity_loader
from prefect.deployments import Deployment
from prefect.filesystems import SMB
import ice_liquidity_loader
from prefect.orion.schemas.schedules import CronSchedule
# Schedule Info
# <https://crontab.guru/#0,30_11-12_*_*_1-5>
# "At minute 0 and 30 past every hour from 11 through 12 on every day-of-week from Monday through Friday."
storage = SMB.load("local-code") # load a pre-defined block
schedule = CronSchedule(cron="0,30 11-12 * * 1-5", timezone="America/New_York")
__path = "ICE"
file_name = "liquidity"
__tags = [f"{__path} Loaders"]
__work_queue_name = "QUEUE"
__first_file_date = "2019-05-02"
deployment = Deployment.build_from_flow(
flow=ice_liquidity_loader.ice_liquidity_loader_date_range,
name="Load All Time",
# tags=["test"],
parameters={"start_date": __first_file_date},
version="1",
work_queue_name=__work_queue_name,
storage=storage,
skip_upload=False,
path=__path,
tags=__tags,
)
Chris White
04/10/2023, 3:31 PMprefect project init --recipe local
prefect deploy ice_liquidity_loader.py:ice_liquidity_loader_date_range -n "Load All Time" -q "QUEUE" -p "local-pool" -version "1"
The only thing to note is that this will require a worker (which is a typed version of an agent), so in this case you'd need to start a local worker:
prefect worker start -p local-pool -t process
Nathan Low
04/10/2023, 3:32 PMChris White
04/10/2023, 3:50 PMNathan Low
04/10/2023, 3:56 PMChris White
04/10/2023, 4:00 PMNelson Griffiths
04/10/2023, 4:37 PMChris White
04/10/2023, 4:43 PMNathan Low
04/10/2023, 4:54 PMChris White
04/10/2023, 5:18 PMNathan Low
04/10/2023, 5:22 PMChris White
04/10/2023, 5:29 PMNathan Low
04/10/2023, 5:35 PMChris White
04/10/2023, 5:52 PMNathan Low
04/10/2023, 6:35 PMChris White
04/10/2023, 10:03 PMNathan Low
04/10/2023, 10:05 PMChris White
04/10/2023, 10:10 PMNathan Low
04/10/2023, 10:12 PMChris White
04/10/2023, 10:19 PMNathan Low
05/17/2023, 3:24 PM