Skip to content

putils

Platform specific helpers to manage locating native dynamic libraries.

This module hosts features similar to https://github.com/rdotnet/dynamic-interop-dll/blob/main/DynamicInterop/PlatformUtility.cs

Functions:

  • augment_path_env –

    Build a new list of directory paths, prepending prior to an existing env var with paths.

  • build_new_path_env –

    Propose an update to an existing environment variable, based on the path(s) specified in another environment variable. This function is effectively meant to be useful on Windows only.

  • find_full_path –

    Find the full path of a library in under the python.

  • library_short_filename –

    Based on the library name, return the platform-specific expected library short file name.

  • update_path_windows –

    If called on Windows, append an environment variable, based on the path(s) specified in another environment variable. This function is effectively meant to be useful on Windows only.

augment_path_env 

augment_path_env(
    added_paths: Union[str, List[str]],
    subfolder: Optional[str] = None,
    to_env: str = "PATH",
    prepend: bool = False,
) -> str

Build a new list of directory paths, prepending prior to an existing env var with paths.

New paths are prepended only if they do already exist.

Parameters:

  • added_paths (Union[str, List[str]]) –

    paths prepended

  • subfolder (str, default: None ) –

    Optional subfolder name to append to each in path prepended. Useful for 64/32 bits variations. Defaults to None.

  • to_env (str, default: 'PATH' ) –

    Environment variable with existing Paths to start with. Defaults to 'PATH'.

Returns:

  • str ( str ) –

    Content (set of paths), typically for a updating/setting an environment variable

Source code in src/refcount/putils.py 
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
def augment_path_env(
    added_paths: Union[str, List[str]],
    subfolder: Optional[str] = None,
    to_env: str = "PATH",
    prepend: bool = False,
) -> str:
    """Build a new list of directory paths, prepending prior to an existing env var with paths.

    New paths are prepended only if they do already exist.

    Args:
        added_paths (Union[str,List[str]]): paths prepended
        subfolder (str, optional): Optional subfolder name to append to each in path prepended. Useful for 64/32 bits variations. Defaults to None.
        to_env (str, optional): Environment variable with existing Paths to start with. Defaults to 'PATH'.

    Returns:
        str: Content (set of paths), typically for a updating/setting an environment variable
    """
    path_sep = os.pathsep
    if isinstance(added_paths, str):
        added_paths = [added_paths]
    prior_path_env = os.environ.get(to_env)
    prior_paths = prior_path_env.split(path_sep) if prior_path_env is not None else []

    def _my_path_join(x: str, subfolder: str):  # avoid trailing path separator  # noqa: ANN202
        if subfolder is not None and subfolder != "":
            return os.path.join(x, subfolder)
        return x

    if subfolder is not None:
        added_paths = [_my_path_join(x, subfolder) for x in added_paths]
    added_paths = [x for x in added_paths if os.path.exists(x)]
    new_paths = (added_paths + prior_paths) if prepend else (prior_paths + added_paths)
    # TODO: check for duplicate folders, perhaps.
    return path_sep.join(new_paths)

build_new_path_env 

build_new_path_env(
    from_env: str = "LIBRARY_PATH",
    to_env: str = "PATH",
    platform: Optional[str] = None,
) -> str

Propose an update to an existing environment variable, based on the path(s) specified in another environment variable. This function is effectively meant to be useful on Windows only.

Parameters:

  • from_env (str, default: 'LIBRARY_PATH' ) –

    name of the source environment variable specifying the location(s) of custom libraries to load. Defaults to 'LIBRARY_PATH'.

  • to_env (str, default: 'PATH' ) –

    environment variable to update, most likely the Windows PATH env var. Defaults to 'PATH'.

Returns:

  • str ( str ) –

    the proposed updated content for the 'to_env' environment variable.

Source code in src/refcount/putils.py 
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
def build_new_path_env(
    from_env: str = "LIBRARY_PATH",
    to_env: str = "PATH",
    platform: Optional[str] = None,
) -> str:
    """Propose an update to an existing environment variable, based on the path(s) specified in another environment variable. This function is effectively meant to be useful on Windows only.

    Args:
        from_env (str, optional): name of the source environment variable specifying the location(s) of custom libraries to load. Defaults to 'LIBRARY_PATH'.
        to_env (str, optional): environment variable to update, most likely the Windows PATH env var. Defaults to 'PATH'.

    Returns:
        str: the proposed updated content for the 'to_env' environment variable.
    """
    platform = sys.platform if platform is None else platform
    path_sep = os.pathsep
    shared_lib_paths = os.environ.get(from_env)
    if shared_lib_paths is not None:
        # We could consider a call to a logger info here
        subfolder = _win_architecture()
        shared_lib_paths_vec = shared_lib_paths.split(path_sep)
        return augment_path_env(shared_lib_paths_vec, subfolder, to_env=to_env)
    print(  # noqa: T201
        f"WARNING: a function was called to look for environment variable '{from_env}' to update the environment variable '{to_env}', but was not found. This may be fine, but if the package fails to load because a native library is not found, this is a likely cause.",
    )
    prior_path_env = os.environ.get(to_env)
    if prior_path_env is not None:
        return prior_path_env
    return ""

find_full_path 

find_full_path(
    name: str, prefix: Optional[str] = None
) -> Union[str, None]

Find the full path of a library in under the python.

installation directory, or as devised by ctypes.find_library

Parameters:

  • name (str) –

    Library name, e.g. 'R' for the R programming language.

Returns:

  • Union[str, None] –

    Union[str, None]: First suitable library full file name.

Examples:

>>> from refcount.putils import *
>>> find_full_path("gfortran")
'/home/xxxyyy/anaconda3/envs/wqml/lib/libgfortran.so'
>>> find_full_path("R")
'libR.so'
Source code in src/refcount/putils.py 
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
def find_full_path(name: str, prefix: Optional[str] = None) -> Union[str, None]:
    """Find the full path of a library in under the python.

        installation directory, or as devised by ctypes.find_library

    Args:
        name (str): Library name, e.g. 'R' for the R programming language.

    Returns:
        Union[str, None]: First suitable library full file name.

    Examples:
        >>> from refcount.putils import *
        >>> find_full_path("gfortran")
        '/home/xxxyyy/anaconda3/envs/wqml/lib/libgfortran.so'
        >>> find_full_path("R")
        'libR.so'
    """
    full_libpath = None
    if prefix is None:
        prefix = sys.prefix
    if name is None:
        return None
    lib_short_fname = library_short_filename(name)
    prefixed_lib_pat = os.path.join(prefix, "lib*", lib_short_fname)
    prefixed_libs = glob(prefixed_lib_pat)
    if prefixed_libs:
        full_libpath = prefixed_libs[0]
    if not full_libpath:
        full_libpath = ctypes_find_library(name)
    return full_libpath

library_short_filename 

library_short_filename(
    library_name: Optional[str],
    platform: Optional[str] = None,
) -> str

Based on the library name, return the platform-specific expected library short file name.

Parameters:

  • library_name (str) –

    name of the library, for instance 'R', which results out of this function as 'libR.so' on Linux and 'R.dll' on Windows

Raises:

Returns:

  • str ( str ) –

    expected short file name for the library, for this platform

Source code in src/refcount/putils.py 
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
def library_short_filename(library_name: Optional[str], platform: Optional[str] = None) -> str:
    """Based on the library name, return the platform-specific expected library short file name.

    Args:
        library_name (str): name of the library, for instance 'R', which results out of this
            function  as 'libR.so' on Linux and 'R.dll' on Windows

    Raises:
        ValueError: invalid argument

    Returns:
        str: expected short file name for the library, for this platform
    """
    if platform is None:
        platform = sys.platform
    if library_name is None:
        raise ValueError("library_name cannot be None")
    if platform == "win32":
        return f"{library_name}.dll"
    if platform == "linux":
        return f"lib{library_name}.so"
    if platform == "darwin":
        return f"lib{library_name}.dylib"
    raise NotImplementedError(f"Platform '{platform}' is not (yet) supported")

update_path_windows 

update_path_windows(
    from_env: str = "LIBRARY_PATH", to_env: str = "PATH"
) -> None

If called on Windows, append an environment variable, based on the path(s) specified in another environment variable. This function is effectively meant to be useful on Windows only.

Parameters:

  • from_env (str, default: 'LIBRARY_PATH' ) –

    name of the source environment variable specifying the location(s) of custom libraries to load. Defaults to 'LIBRARY_PATH'.

  • to_env (str, default: 'PATH' ) –

    environment variable to update, most likely the Windows PATH env var. Defaults to 'PATH'.

Returns:

  • None –

    None

Source code in src/refcount/putils.py 
231
232
233
234
235
236
237
238
239
240
241
242
def update_path_windows(from_env: str = "LIBRARY_PATH", to_env: str = "PATH") -> None:
    """If called on Windows, append an environment variable, based on the path(s) specified in another environment variable. This function is effectively meant to be useful on Windows only.

    Args:
        from_env (str, optional): name of the source environment variable specifying the location(s) of custom libraries to load. Defaults to 'LIBRARY_PATH'.
        to_env (str, optional): environment variable to update, most likely the Windows PATH env var. Defaults to 'PATH'.

    Returns:
        None
    """
    if sys.platform == "win32":
        os.environ[to_env] = build_new_path_env(from_env, to_env, sys.platform)