https://prefect.io logo
Join Slack
Powered by
Hello, I'm trying to <automatically provision a wo...
# prefect-cloud
h
Hello, I'm trying to automatically provision a work pool on Azure (ACI) but getting a 
FileNotFoundError: [WinError 2] The system cannot find the file specified
(I'm obviously doing this on a Windows machine). Full stack trace in the thread. Appreciate any guidance!
Copy code
C:\Users\Henry bruce>prefect config view
20:05:28.752 | DEBUG   | prefect.profiles - Using profile 'ephemeral'
PREFECT_PROFILE='ephemeral'
PREFECT_API_KEY='********' (from profile)
PREFECT_API_URL='<https://api.prefect.cloud/api/accounts/XXX/workspaces/XXX>' (from profile)
PREFECT_LOGGING_LEVEL='DEBUG' (from profile)
PREFECT_SERVER_ALLOW_EPHEMERAL_MODE='True' (from profile)

C:\Users\Henry bruce>prefect work-pool create --type azure-container-instance:push --provision-infra una-workpool-qa-rg
20:05:38.213 | DEBUG   | prefect.profiles - Using profile 'ephemeral'
20:05:38.563 | DEBUG   | prefect.client - Connecting to API at <https://api.prefect.cloud/api/accounts/XXX/workspaces/XXX/>
20:05:41.308 | DEBUG   | prefect.client - Connecting to API at <https://api.prefect.cloud/api/accounts/XXX/workspaces/XXX/>
Traceback (most recent call last):
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\cli\_utilities.py", line 42, in wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\cli\_types.py", line 153, in sync_fn
    return asyncio.run(async_fn(*args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\asyncio\runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\asyncio\base_events.py", line 664, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\cli\work_pool.py", line 181, in create
    template_contents = await provisioner.provision(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\client\utilities.py", line 103, in with_injected_client
    return await fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\infrastructure\provisioners\container_instance.py", line 882, in provision
    await self._verify_az_ready()
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\infrastructure\provisioners\container_instance.py", line 199, in _verify_az_ready
    await self.azure_cli.run_command("az --version", ignore_if_exists=True)
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\site-packages\prefect\infrastructure\provisioners\container_instance.py", line 88, in run_command
    result = await run_process(shlex.split(command), check=False)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\site-packages\anyio\_core\_subprocesses.py", line 58, in run_process
    async with await open_process(
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\site-packages\anyio\_core\_subprocesses.py", line 131, in open_process
    return await get_async_backend().open_process(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\site-packages\anyio\_backends\_asyncio.py", line 2270, in open_process
    process = await asyncio.create_subprocess_exec(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\asyncio\subprocess.py", line 222, in create_subprocess_exec
    transport, protocol = await loop.subprocess_exec(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\asyncio\base_events.py", line 1708, in subprocess_exec
    transport = await self._make_subprocess_transport(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\asyncio\windows_events.py", line 399, in _make_subprocess_transport
    transp = _WindowsSubprocessTransport(self, protocol, args, shell,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\asyncio\base_subprocess.py", line 36, in __init__
    self._start(args=args, shell=shell, stdin=stdin, stdout=stdout,
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\asyncio\windows_events.py", line 873, in _start
    self._proc = windows_utils.Popen(
                 ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\asyncio\windows_utils.py", line 153, in __init__
    super().__init__(args, stdin=stdin_rfd, stdout=stdout_wfd,
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Users\Henry bruce\AppData\Local\Programs\Python\Python312\Lib\subprocess.py", line 1538, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [WinError 2] The system cannot find the file specified
An exception occurred.

C:\Users\Henry bruce>az --version
azure-cli                         2.63.0

core                              2.63.0
telemetry                          1.1.0

Dependencies:
msal                              1.30.0
azure-mgmt-resource               23.1.1

Python location 'C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe'
Extensions directory 'C:\Users\Henry bruce\.azure\cliextensions'

Python (Windows) 3.11.8 (tags/v3.11.8:db85d51, Feb  6 2024, 22:03:32) [MSC v.1937 64 bit (AMD64)]

Legal docs and information: <http://aka.ms/AzureCliLegal|aka.ms/AzureCliLegal>


Your CLI is up-to-date.

C:\Users\Henry bruce>
Account & workspace IDs redacted. Azure CLI up to date & logged in.
@Marvin any ideas?
m
Of course. Could you please provide more details about what you need help with? It will help me assist you more effectively.
h
Details on this thread @Marvin
m
Apologies for the inconvenience. It seems there was an error in my attempt to search both in GitHub issues and the Prefect documentation concurrently. Let's try searching each source individually to gather information about threads in Prefect. I'll start with searching the Prefect documentation. It seems the documentation mentions threads in various contexts, such as DBT configurations and Prefect workers for GCP and Docker. Here are some relevant excerpts: 1. DBT Configurations: - Threads: The number of threads representing the max number of paths through the graph DBT may work on at once. - Source 2. Vertex Worker (GCP): - Reference to threads in the context of vertex workers, but without specific details in the excerpt. - Source 3. Docker Worker: - Similar reference to threads for the Docker worker, though the specific details are also not included in the excerpt. - Source For more detailed information, please refer to the linked documentation. Would you like me to search for GitHub issues related to threads as well?
h
No, that's not the issue @Marvin. I'm trying to automatically provision a work pool on Azure (ACI) but getting a 
FileNotFoundError: [WinError 2] The system cannot find the file specified
error.
m
It appears that there was an issue when trying to search across multiple sources simultaneously. Let's attempt them individually. First, I'll search GitHub issues for information related to your error. The search results did not provide specific guidance on resolving the 
FileNotFoundError: [WinError 2] The system cannot find the file specified
error related to automatic infrastructure provisioning on Azure ACI. However, here are some potential causes and solutions based on common issues when working with Azure and Prefect: 1. Azure CLI Installation: Ensure that the Azure CLI is installed and properly set up on your machine. You can install it from the Azure CLI Installation Guide. 2. Path Configuration: Make sure that the Azure CLI executable is included in your system's PATH environment variable. This can typically be done by modifying the environment variables on your operating system. 3. Command Execution: Ensure that the commands you are trying to run via Prefect are executable through the command line. Test running the commands manually in your terminal to verify. 4. Prefect ACI Worker Setup: Follow the detailed setup instructions for configuring the ACI work pool and verify each step. Here is the link to the setup guide for Prefect Azure ACI Worker. Could you please verify these points and let me know if that resolves the issue? If not, please provide more details about the context in which you're encountering this error.
h
For future reference, it doesn't look like the use of anyio / asyncio in the Prefect CLI is setup to cater for Windows execution. Use something else (MacOS worked perfectly, of course). Submitted issue #15048.
24 Views
Open in Slack
PreviousNext
Powered by