Matt Camp
05/17/2021, 1:35 AM# debug mode
debug = true
# base configuration directory (typically you won't change this!)
home_dir = "~/.prefect/"
backend = "server"
[server]
host = "<http://apollo.prefect.campground.lan>"
port = "4200"
endpoint = "${server.host}:${server.port}"
my apollo server is at <http://apollo.prefect.campground.lan:4200/>
my graphl server is at <http://graphql.prefect.campground.lan:4200/graphql>
my ui is at <http://prefect.campground.lan>
Any glaring problems with what I've done?
TIAZanie
05/17/2021, 2:57 PMPREFECT__CLOUD__AGENT__LEVEL=DEBUG prefect agent local start
(or however you start your agents) and share a bit of the logs?Matt Camp
05/17/2021, 8:39 PMmyagent | [2021-05-17 20:24:20,734] DEBUG - CampNetAgent | Completed flow run submission (id: 073c2eef-abee-4fb3-8292-7b36d841829a)
myagent | [2021-05-17 20:24:20,735] DEBUG - CampNetAgent | Completed flow run submission (id: c7396f57-60e0-42e6-b129-d21b49253e34)
myagent | [2021-05-17 20:24:20,743] DEBUG - CampNetAgent | Completed flow run submission (id: 672206dd-76e5-472a-a3b8-0a064a01bf05)
myagent | [2021-05-17 20:24:21,030] DEBUG - CampNetAgent | Querying for flow runs
myagent | [2021-05-17 20:24:21,055] DEBUG - CampNetAgent | No flow runs found
myagent | [2021-05-17 20:24:21,056] DEBUG - CampNetAgent | Next query for flow runs in 1.0 seconds
myagent | No module named 'weather'
myagent | Traceback (most recent call last):
myagent | File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
myagent | return _run_code(code, main_globals, None,
myagent | File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
myagent | exec(code, run_globals)
myagent | File "/usr/local/lib/python3.9/site-packages/prefect/__main__.py", line 4, in <module>
myagent | cli()
myagent | File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1134, in __call__
myagent | return self.main(*args, **kwargs)
myagent | File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1059, in main
myagent | rv = self.invoke(ctx)
myagent | File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1665, in invoke
myagent | return _process_result(sub_ctx.command.invoke(sub_ctx))
myagent | File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1665, in invoke
myagent | return _process_result(sub_ctx.command.invoke(sub_ctx))
myagent | File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1401, in invoke
myagent | return ctx.invoke(self.callback, **ctx.params)
myagent | File "/usr/local/lib/python3.9/site-packages/click/core.py", line 767, in invoke
myagent | return __callback(*args, **kwargs)
myagent | File "/usr/local/lib/python3.9/site-packages/prefect/cli/execute.py", line 90, in flow_run
myagent | raise exc
myagent | File "/usr/local/lib/python3.9/site-packages/prefect/cli/execute.py", line 67, in flow_run
myagent | flow = storage.get_flow(flow_data.name)
myagent | File "/usr/local/lib/python3.9/site-packages/prefect/storage/local.py", line 102, in get_flow
myagent | return extract_flow_from_module(
myagent | File "/usr/local/lib/python3.9/site-packages/prefect/utilities/storage.py", line 124, in extract_flow_from_module
myagent | module = importlib.import_module(mod_name)
myagent | File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
myagent | return _bootstrap._gcd_import(name[level:], package, level)
myagent | File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
myagent | File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
myagent | File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
myagent | File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
myagent | File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
myagent | File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
myagent | File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
myagent | ModuleNotFoundError: No module named 'weather'
myagent | No module named 'weather'
Zanie
05/17/2021, 8:41 PMLocal
storage here? If you're using local storage then your agent needs access to the files. You could use remote storage instead like S3
or GitHub
storage so that you don't need the flow source on the agent.Matt Camp
05/17/2021, 8:43 PM~/.prefect
accessible as a volume to everything or is it a different folder?Zanie
05/17/2021, 8:45 PMLocal
storage defaults to ~/.prefect/flows
but you can make it whatever you'd likeMatt Camp
05/17/2021, 8:46 PM$PWD/prefect/prefect
that I am mapping to the Graphql container and my Agent at
/root/.prefect
and I'm still having issues.
my Agent says
____ __ _ _ _
| _ \ _ __ ___ / _| ___ ___| |_ / \ __ _ ___ _ __ | |_
| |_) | '__/ _ \ |_ / _ \/ __| __| / _ \ / _` |/ _ \ '_ \| __|
| __/| | | __/ _| __/ (__| |_ / ___ \ (_| | __/ | | | |_
|_| |_| \___|_| \___|\___|\__| /_/ \_\__, |\___|_| |_|\__|
|___/
[2021-05-18 01:43:38,319] INFO - agent | Starting LocalAgent with labels ['0f43a1ca6a0f']
[2021-05-18 01:43:38,319] INFO - agent | Agent documentation can be found at <https://docs.prefect.io/orchestration/>
[2021-05-18 01:43:38,319] INFO - agent | Agent connecting to the Prefect API at <http://apollo.prefect.campground.lan:4200>
[2021-05-18 01:43:38,325] INFO - agent | Waiting for flow runs...
[2021-05-18 01:43:38,325] DEBUG - agent | Running agent heartbeat...
[2021-05-18 01:43:38,328] DEBUG - agent | Max Workers: 12
[2021-05-18 01:43:38,329] DEBUG - agent | Sleeping heartbeat for 60.0 seconds
[2021-05-18 01:43:38,329] DEBUG - agent | Querying for flow runs
[2021-05-18 01:43:38,346] DEBUG - agent | No flow runs found
[2021-05-18 01:43:38,347] DEBUG - agent | Next query for flow runs in 0.5 seconds
[2021-05-18 01:43:38,847] DEBUG - agent | Querying for flow runs
[2021-05-18 01:43:38,866] DEBUG - agent | No flow runs found
[2021-05-18 01:43:38,866] DEBUG - agent | Next query for flow runs in 1.0 seconds
[2021-05-18 01:43:39,866] DEBUG - agent | Querying for flow runs
[2021-05-18 01:43:39,885] DEBUG - agent | No flow runs found
[2021-05-18 01:43:39,885] DEBUG - agent | Next query for flow runs in 2.0 seconds
[2021-05-18 01:43:41,885] DEBUG - agent | Querying for flow runs
[2021-05-18 01:43:41,951] DEBUG - agent | No flow runs found
[2021-05-18 01:43:41,952] DEBUG - agent | Next query for flow runs in 4.0 seconds
Zanie
05/18/2021, 3:32 AMStarting LocalAgent with labels ['0f43a1ca6a0f']
does your flow run have a matching label? You may want to start the agent with` --no-hostname-label`Matt Camp
05/19/2021, 2:50 AMZanie
05/19/2021, 4:07 AMprefect diagnostics
command should show all config changesMatt Camp
05/20/2021, 1:53 AM