asyncio subprocess error: OSError: [WinError 6] The handle is invalid

[natbprice]

I believe this issue is similar to #518 and #1448. Maybe it can be fixed with the same patch method as in #1458 and #1479.

I wasn't able to create a good minimal example, but here is how it can be reproduced (reticulate 1.32.0.9001):

reticulate::source_python("reprex.py")

import asyncio
import logging
from azure.identity.aio import AzureCliCredential

logging.basicConfig(level=logging.DEBUG)

async def run_command():
    credential = AzureCliCredential()
    token = await credential.get_token("https://gbif-open-data-us-east-1.blob.core.windows.net/")
    return token
  
def get_token():
    token = asyncio.run(run_command())
    print(token)

get_token()

Debug error

DEBUG:asyncio:Using proactor: IocpProactor
WARNING:azure.identity.aio._internal.decorators:AzureCliCredential.get_token failed: Failed to execute 'cmd'
Traceback (most recent call last):
  File ""C:\WorkingDir\.venv\lib\site-packages\azure\identity\aio\_credentials\azure_cli.py", line 135, in _run_command
    proc = await asyncio.create_subprocess_exec(
  File "C:\PROGRA~1\Python310\lib\asyncio\subprocess.py", line 218, in create_subprocess_exec
    transport, protocol = await loop.subprocess_exec(
  File "C:\PROGRA~1\Python310\lib\asyncio\base_events.py", line 1681, in subprocess_exec
    transport = await self._make_subprocess_transport(
  File "C:\PROGRA~1\Python310\lib\asyncio\windows_events.py", line 399, in _make_subprocess_transport
    transp = _WindowsSubprocessTransport(self, protocol, args, shell,
  File "C:\PROGRA~1\Python310\lib\asyncio\base_subprocess.py", line 36, in __init__
    self._start(args=args, shell=shell, stdin=stdin, stdout=stdout,
  File "C:\PROGRA~1\Python310\lib\asyncio\windows_events.py", line 901, in _start
    self._proc = windows_utils.Popen(
  File "C:\PROGRA~1\Python310\lib\asyncio\windows_utils.py", line 153, in __init__
    super().__init__(args, stdin=stdin_rfd, stdout=stdout_wfd,
  File "C:\PROGRA~1\Python310\lib\subprocess.py", line 837, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File "C:\PROGRA~1\Python310\lib\subprocess.py", line 1272, in _get_handles
    p2cread = _winapi.GetStdHandle(_winapi.STD_INPUT_HANDLE)
OSError: [WinError 6] The handle is invalid

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File ""C:\WorkingDir\.venv\lib\site-packages\azure\identity\aio\_internal\decorators.py", line 21, in wrapper
    token = await fn(*args, **kwargs)
  File ""C:\WorkingDir\.venv\lib\site-packages\azure\identity\aio\_credentials\azure_cli.py", line 102, in get_token
    output = await _run_command(command, self._process_timeout)
  File ""C:\WorkingDir\.venv\lib\site-packages\azure\identity\aio\_credentials\azure_cli.py", line 148, in _run_command
    raise error from ex
azure.identity._exceptions.CredentialUnavailableError: Failed to execute 'cmd'
Error in py_run_file_impl(file, local, convert) : 
  azure.identity._exceptions.CredentialUnavailableError: Failed to execute 'cmd'

I attempted to create an example with less dependencies, but it does not reproduce the error so I must be missing something. See https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/identity/azure-identity/azure/identity/aio/_credentials/azure_cli.py

import asyncio
import sys
import os
import logging

logging.basicConfig(level=logging.DEBUG)

COMMAND_LINE = "az account get-access-token --output json --resource {}"

async def run_command():
    resource = "https://gbif-open-data-us-east-1.blob.core.windows.net/"
    command = COMMAND_LINE.format(resource)
    if sys.platform.startswith("win"):
        args = ("cmd", "/c " + command)
    else:
        args = ("/bin/sh", "-c", command)

    working_directory = "C:\WINDOWS"

    try:
        proc = await asyncio.create_subprocess_exec(
            *args,
            stdout=asyncio.subprocess.PIPE,
            stderr=asyncio.subprocess.PIPE,
            stdin=asyncio.subprocess.DEVNULL,
            cwd=working_directory,
            env=dict(os.environ, AZURE_CORE_NO_COLOR="true"),
        )
        stdout_b, stderr_b = await asyncio.wait_for(proc.communicate(), 10)
        output = stdout_b.decode()
        stderr = stderr_b.decode()
    except OSError as ex:
        raise ex
        
    if proc.returncode == 0:
        return output
  
def get_token():
    token = asyncio.run(run_command())
    print(token)

get_token()

[t-kalinowski]

Thank you for the bug report! I could reproduce locally and have a tentative fix.

Can you please try to install and confirm it works?

pak::pak("rstudio/reticulate#1485")

[natbprice]

It works! Thanks for your fast fix.

[flyaflya]

I am thrilled this is fixed. Thanks! Any chance at getting this patch to CRAN? This is causing issues for my new Windows users.

[t-kalinowski]

Should be on CRAN in ~1 week - need to wait at least 30 days since the last CRAN submission (2023-09-11).

[flyaflya]

Cool. Thanks for the update. Much appreciated.

[bioinfotec]

Thank you for the bug report! I could reproduce locally and have a tentative fix.

Can you please try to install and confirm it works?

pak::pak("#1485")

This method downgrade reticulate version to 1.32.0. However, a new problem which is about "In system2(command = python, args = shQuote(script), stdout = TRUE, :" arised.

By the way, this new problem above seems to be only in low version. And when I upgrade it to the lasted version, it disappeared. But the origin problem "OSError: [WinError 6] The handle is invalid" appears again.

[t-kalinowski]

@bioinfotec This issue is quite old (the current reticulate release is 1.40). Are you experiencing issues with the current release? If so, can you please open a new issue?