https://prefect.io logo
Join Slack
Powered by
Greetings, I am having an issue registering a Flow...
# prefect-community
s
Greetings, I am having an issue registering a Flow with a simple ShellTask (Prefect/Cloud 1.0). The RunConfig for the flow is pointing to a custom image in ECR that contains the binary that we are attempting to run via the shell task (using the ECSRun type). The path to the binary in the shell command does not exist locally. When I attempt to register the flow, it fails with a “no such file or directory” error. Does the register cli command attempt to verify the shell task locally? How can I get around this when I only want it to reference the location in the container image?
k
It should need it. Can I see how you call the ShellTask?
s
I create the task:
Copy code
dcm_task = ShellTask(
    return_all=True,
    stream_output=True
)
and run it in the flow:
Copy code
output = dcm_task.run(command='./service/dcm-service')
where the path to the binary inside the ECR image is the command
I mainly want to get back the log data from the binary service and output it.
k
Can I see your traceback?
Ah wait. don’t call run in the flow
Just do:
Copy code
output = dcm_task.run(command='./service/dcm-service')
s
When I use the register cli command this is the error:
Copy code
Collecting flows...
[2022-07-07 15:12:55-0500] INFO - prefect.ShellTask | /var/folders/7d/41lxpm4s6039384cprdz33rm0000gn/T/prefect-va5kr_fc: line 1: /service/dcm-service: No such file or directory
[2022-07-07 15:12:55-0500] ERROR - prefect.ShellTask | Command failed with exit code 127
Traceback (most recent call last):
  File "/Users/scotthenley/Projects/prefect-flows/.direnv/python-3.9.12/bin/prefect", line 8, in <module>
    sys.exit(cli())
  File "/Users/scotthenley/Projects/prefect-flows/.direnv/python-3.9.12/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/scotthenley/Projects/prefect-flows/.direnv/python-3.9.12/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/scotthenley/Projects/prefect-flows/.direnv/python-3.9.12/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/scotthenley/Projects/prefect-flows/.direnv/python-3.9.12/lib/python3.9/site-packages/click/core.py", line 1635, in invoke
    rv = super().invoke(ctx)
  File "/Users/scotthenley/Projects/prefect-flows/.direnv/python-3.9.12/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/scotthenley/Projects/prefect-flows/.direnv/python-3.9.12/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/scotthenley/Projects/prefect-flows/.direnv/python-3.9.12/lib/python3.9/site-packages/prefect/cli/build_register.py", line 64, in inner
    return func(*args, **kwargs)
  File "/Users/scotthenley/Projects/prefect-flows/.direnv/python-3.9.12/lib/python3.9/site-packages/prefect/cli/build_register.py", line 810, in register
    register_internal(
  File "/Users/scotthenley/Projects/prefect-flows/.direnv/python-3.9.12/lib/python3.9/site-packages/prefect/cli/build_register.py", line 536, in register_internal
    source_to_flows = collect_flows(
  File "/Users/scotthenley/Projects/prefect-flows/.direnv/python-3.9.12/lib/python3.9/site-packages/prefect/cli/build_register.py", line 255, in collect_flows
    flows = load_flows_from_script(s.location)
  File "/Users/scotthenley/Projects/prefect-flows/.direnv/python-3.9.12/lib/python3.9/site-packages/prefect/cli/build_register.py", line 133, in load_flows_from_script
    namespace = runpy.run_path(abs_path, run_name="<flow>")
  File "/Users/scotthenley/.pyenv/versions/3.9.12/lib/python3.9/runpy.py", line 268, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/Users/scotthenley/.pyenv/versions/3.9.12/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/Users/scotthenley/.pyenv/versions/3.9.12/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/scotthenley/Projects/prefect-flows/prefect/test/dcm_api_service.py", line 36, in <module>
    output = dcm_task.run(command='/service/dcm-service')
  File "/Users/scotthenley/Projects/prefect-flows/.direnv/python-3.9.12/lib/python3.9/site-packages/prefect/utilities/tasks.py", line 456, in method
    return run_method(self, *args, **kwargs)
  File "/Users/scotthenley/Projects/prefect-flows/.direnv/python-3.9.12/lib/python3.9/site-packages/prefect/tasks/shell.py", line 164, in run
    raise prefect.engine.signals.FAIL(
prefect.engine.signals.FAIL: Command failed with exit code 127
k
You shouldn’t be calling .run in the Flow. The Flow block will do that for you
s
ahh, so how do I reference the task in the flow block?
k
Copy code
shell = ShellTask(..)
with Flow(..) as flow:
    shell(...)
s
can insert the command in the flow block like this:
Copy code
output = dcm_task(command='/service/dcm-service')
or should command be part of the ShellTask initialization?
k
It can be inserted in both places
s
awesome thanks, that did the trick
it makes sense.. run() tries to run it immediately at registration
👀 1
k
Yes exactly!
3 Views
Open in Slack
PreviousNext
Powered by