https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> I have my prefect.yaml file with all ...
# ask-marvin
m
@Marvin I have my prefect.yaml file with all deployments specified. I want to prefect deploy all flows at same time, how to do it?
m
To deploy all flows specified in your 
prefect.yaml
file simultaneously, you can use the 
prefect deploy
command. This command is designed to deploy all the flows defined in the 
prefect.yaml
file, which allows for customization in terms of push and pull steps. Here's a basic example of how you might structure your 
prefect.yaml
file to define multiple deployments:
Copy code
yaml
flows:
  - name: flow_one
    entrypoint: path/to/flow_one.py:flow_function_name
  - name: flow_two
    entrypoint: path/to/flow_two.py:flow_function_name
Once your 
prefect.yaml
is configured, you can run:
Copy code
bash
prefect deploy
This command will deploy all the flows defined in the file. You can refer to the Prefect deployment documentation for more details.
m
@Marvin I mean I want to deploy all flows in this yaml in one command.
m
To deploy all flows specified in your 
prefect.yaml
file using a single command, you can use the 
prefect deploy
command without specifying individual flow names. This command automatically reads the 
prefect.yaml
file and deploys all the flows defined within it. Simply navigate to the directory containing your 
prefect.yaml
file and execute:
Copy code
bash
prefect deploy
This will trigger the deployment process for all flows specified in the YAML file. If you need more information, you can check the Prefect deployment documentation.
m
After I run prefect deploy, it asks me to select individual flows and deploy @Marvin
m
To deploy all flows specified in your 
prefect.yaml
file without being prompted to select each one individually, you can use the 
--all
flag with the 
prefect deploy
command. This flag tells Prefect to deploy all deployments defined in the YAML file in one go. Here's the command you should run:
Copy code
bash
prefect deploy --all
This will automatically deploy every flow listed in your 
prefect.yaml
file. For more details, you can refer to the Prefect deployment documentation.
n
you can also use 
prefect --no-prompt deploy --all
🙌 1
to avoid it asking you questions
m
@Nate Follow up question as I ran into a very strange issue with deploying one my flows. I'm able to run this flow locally no problems, and the line of code it's complaining 
server = mssql_auth.value["server"]
, I have it in all my flows and have no issues deploying the rest. Any thoughts?
Copy code
Traceback (most recent call last):
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "C:\Users\maung\Desktop\DevOps\AppScripts\PMS\Dentitek\dentitek_carriers_flow.py", line 7, in <module>
    from dentitek_carriermapping_subflow import dentitek_carriermapping_subflow
  File "C:\Users\maung\Desktop\DevOps\AppScripts\PMS\Dentitek\dentitek_carriermapping_subflow.py", line 35, in <module>
    server = mssql_auth.value["server"]
             ^^^^^^^^^^^^^^^^
AttributeError: 'coroutine' object has no attribute 'value'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\maung\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\flows.py", line 1742, in load_flow_from_entrypoint
    flow = import_object(entrypoint)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maung\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\utilities\importtools.py", line 205, in import_object
    module = load_script_as_module(script_path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maung\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\utilities\importtools.py", line 168, in load_script_as_module
    raise ScriptError(user_exc=exc, path=path) from exc
prefect.exceptions.ScriptError: Script at 'AppScripts\\PMS\\Dentitek\\dentitek_carriers_flow.py' encountered an exception: AttributeError("'coroutine' object has no attribute 'value'")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\maung\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\cli\_utilities.py", line 42, in wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maung\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\cli\_types.py", line 153, in sync_fn
    return asyncio.run(async_fn(*args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maung\AppData\Local\Programs\Python\Python312\Lib\asyncio\runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "C:\Users\maung\AppData\Local\Programs\Python\Python312\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maung\AppData\Local\Programs\Python\Python312\Lib\asyncio\base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "C:\Users\maung\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\cli\deploy.py", line 451, in deploy
    await _run_single_deploy(
  File "C:\Users\maung\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\client\utilities.py", line 103, in with_injected_client
    return await fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maung\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\cli\deploy.py", line 501, in _run_single_deploy
    flow = load_flow_from_entrypoint(deploy_config["entrypoint"])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maung\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\flows.py", line 1751, in load_flow_from_entrypoint
    flow = safe_load_flow_from_entrypoint(entrypoint)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maung\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\flows.py", line 2005, in safe_load_flow_from_entrypoint
    namespace = safe_load_namespace(source_code, filepath=path)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maung\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\utilities\importtools.py", line 446, in safe_load_namespace
    module = importlib.import_module(module_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maung\AppData\Local\Programs\Python\Python312\Lib\importlib\__init__.py", line 90, in import_module
  File "C:\Users\maung\AppData\Local\Programs\Python\Python312\Lib\importlib\__init__.py", line 90, in import_module
  File "C:\Users\maung\AppData\Local\Programs\Python\Python312\Lib\importlib\__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "C:\Users\maung\Desktop\DevOps\AppScripts\PMS\Dentitek\dentitek_carriermapping_subflow.py", line 35, in <module>
    server = mssql_auth.value["server"]
             ^^^^^^^^^^^^^^^^
AttributeError: 'coroutine' object has no attribute 'value'
An exception occurred.
sys:1: RuntimeWarning: coroutine 'sync_compatible.<locals>.coroutine_wrapper.<locals>.ctx_call' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
n
Copy code
server = mssql_auth.value["server"]
             ^^^^^^^^^^^^^^^^
AttributeError: 'coroutine' object has no attribute 'value'
so you're expecting 
mssql_auth
to be a JSON block, or secret or something, but its a 
coroutine
that you need to 
await
before accessing the 
value
attribute
can you show the code where you load this block?
m
Yeah so I have this in every flow:
Copy code
# Configure SQL Server 
sql_driver = config['APP']['PREFECTBLOCKS']['SQLSERVER']['DRIVER']
database = config['APP']['PREFECTBLOCKS']['SQLSERVER']['DBNAME']
mssql_block = config['APP']['PREFECTBLOCKS']['SQLSERVER'][env]
mssql_auth = JSON.load(mssql_block)
server = mssql_auth.value["server"]
port = mssql_auth.value["port"]
n
can you try this?
Copy code
mssql_auth = JSON.load(mssql_block, _sync=True)
m
Intersting, okay
So in my case here, why should I set it to _sync=True? And why does it not cause issue in other flows? All of them are async
n
All of them are async
hmm its surprising that you're not encountering this in all of your flows then. are you calling .load within each of these async flows?
m
Yes I am in same exact way
n
what version of prefect are you using?
m
I'm using prefect version 3.1.1
n
hmm ok. in newest prefect we have an explicit async method 
aload
that you would await like this
Copy code
mssql_auth = JSON.aload(mssql_block)
but in your case I'd say you should either use
Copy code
mssql_auth = await JSON.load(mssql_block)
which infers 
load
should be async based on the context (a pattern we're moving away from, outlined in the linked issue) or
Copy code
mssql_auth = JSON.load(mssql_block, _sync=True)
which will force the 
load
method to behave sync
5 Views
Open in Slack
PreviousNext
Powered by