asv.util

Various low-level utilities.

Attributes

WIN
TIMEOUT_RETCODE
terminal_width
long_path_open
_global_locks
_find_unsafe
ON_PYPY

Exceptions

UserError	Common base class for all non-exit exceptions.
ParallelFailure	Custom exception to work around a multiprocessing bug
ProcessError	Raised when run() is called with check=True and the process

Classes

DebugLogBuffer
ParsedPipDeclaration

Functions

human_list(input_list)	Formats a list of strings in a human-friendly way.
human_file_size(size[, err])	Returns a human-friendly string representing a file size
human_value(value, unit[, err])	Formats a value in a given unit in a human friendly way.
parse_human_time(string[, base_period])	Parse a human-specified time period to an integer number of seconds.
which(filename[, paths])	Emulates the UNIX which command in Python.
has_command(filename)	Returns True if the commandline utility exists.
check_call(args[, valid_return_codes, timeout, dots, ...])	Runs the given command in a subprocess, raising ProcessError if it
check_output(args[, valid_return_codes, timeout, ...])	Runs the given command in a subprocess, raising ProcessError if it
_killpg_safe(pgid, signo)	Same as os.killpg, but deal with OSX/BSD
is_main_thread()	Return True if the current thread is the main thread.
write_json(path, data[, api_version, compact])	Writes JSON to the given path, including indentation and sorting.
load_json(path[, api_version, js_comments])	Loads JSON from the given path.
update_json(cls, path, api_version[, compact])	Perform JSON file format updates.
iter_chunks(s, n)	Iterator that returns elements from s in chunks of size n.
pick_n(items, n)	Pick n items, attempting to get equal index spacing.
get_multiprocessing(parallel)	If parallel indicates that we want to do multiprocessing,
iter_subclasses(cls)	Returns all subclasses of a class.
hash_equal(a, b)	Returns True if a and b represent the same commit hash.
get_cpu_info()	Gets a human-friendly description of this machine's CPU.
get_memsize()	Returns the amount of physical memory in this machine.
format_text_table(rows[, num_headers, ...])	Format rows in as a reStructuredText table, in the vein of:
_datetime_to_timestamp(dt, divisor)
datetime_to_timestamp(dt)	Convert a Python datetime object to a UNIX timestamp.
datetime_to_js_timestamp(dt)	Convert a Python datetime object to a JavaScript timestamp.
js_timestamp_to_datetime(ts)	Convert a JavaScript timestamp to a Python datetime object.
is_nan(x)	Returns True if x is a NaN value.
is_na(value)	Return True if value is None or NaN
mean_na(values)	Take a mean, with the understanding that None and NaN stand for
geom_mean_na(values)	Compute geometric mean, with the understanding that None and NaN
sanitize_filename(filename)	Replace characters to make a string safe to use in file names.
namedtuple_with_doc(name, slots, doc)
recvall(sock, size)	Receive data of given size from a socket connection
interpolate_command(command, variables)	Parse a command with interpolated variables to a sequence of commands.
truncate_float_list(item[, digits])	Truncate floating-point numbers (in a possibly nested list)
_init_global_locks(lock_dict)	Initialize global locks in a new multiprocessing process
new_multiprocessing_lock(name)	Create a new global multiprocessing lock
get_multiprocessing_lock(name)	Get an existing global multiprocessing lock
get_multiprocessing_pool([parallel])	Create a multiprocessing.Pool, managing global locks properly
git_default_branch()
search_channels(cli_path, pkg, version)
construct_pip_call(pip_caller, parsed_declaration)
get_matching_environment(environments[, result])
replace_python_version(arg, new_version)

Module Contents

asv.util.WIN

asv.util.TIMEOUT_RETCODE = -256

asv.util.terminal_width

exception asv.util.UserError

Common base class for all non-exit exceptions.

exception asv.util.ParallelFailure

Custom exception to work around a multiprocessing bug https://bugs.python.org/issue9400

__reduce__()

__str__()

Return str(self).

reraise()

asv.util.human_list(input_list)

Formats a list of strings in a human-friendly way.

asv.util.human_file_size(size, err=None)

Returns a human-friendly string representing a file size that is 2-4 characters long.

For example, depending on the number of bytes given, can be one of:

256b
64k
1.1G

Parameters

sizeint

The size of the file (in bytes)

Returns

sizestr

A human-friendly representation of the size of the file

asv.util.human_value(value, unit, err=None)

Formats a value in a given unit in a human friendly way.

Parameters

valueanything

The value to format

unitstr

The unit the value is in. Currently understands seconds and bytes.

errfloat, optional

Std. error in the value

asv.util.parse_human_time(string, base_period='d')

Parse a human-specified time period to an integer number of seconds. The following format is accepted: <number><suffix>

Raises a ValueError on parse error.

asv.util.which(filename, paths=None)

Emulates the UNIX which command in Python.

Raises an OSError if no result is found.

asv.util.has_command(filename)

Returns True if the commandline utility exists.

exception asv.util.ProcessError(args, retcode, stdout, stderr)

Raised when run() is called with check=True and the process returns a non-zero exit status.

Attributes:

cmd, returncode, stdout, stderr, output

args

retcode

stdout

Alias for output attribute, to match stderr

stderr

__str__()

Return str(self).

asv.util.check_call(args, valid_return_codes=(0,), timeout=600, dots=True, display_error=True, shell=False, env=None, cwd=None)

Runs the given command in a subprocess, raising ProcessError if it fails.

See check_output for parameters.

class asv.util.DebugLogBuffer(log)

buf = []

first = True

linebreak_re

log

lock

__call__(c)

_process(c)

asv.util.check_output(args, valid_return_codes=(0,), timeout=600, dots=True, display_error=True, shell=False, return_stderr=False, env=None, cwd=None, redirect_stderr=False, return_popen=False)

Runs the given command in a subprocess, raising ProcessError if it fails. Returns stdout as a string on success.

Parameters

valid_return_codeslist, optional

A list of return codes to ignore. Defaults to only ignoring zero. Setting to None ignores all return codes.

timeoutnumber, optional

Kill the process if it does not produce any output in timeout seconds. If None, there is no timeout. Default: 10 min

dotsbool, optional

If True (default) write a dot to the console to show progress as the subprocess outputs content. May also be a callback function to call (with no arguments) to indicate progress.

display_errorbool, optional

If True (default) display the stdout and stderr of the subprocess when the subprocess returns an error code.

shellbool, optional

If True, run the command through the shell. Default is False.

return_stderrbool, optional

If True, return both the (stdout, stderr, errcode) as a tuple.

envdict, optional

Specify environment variables for the subprocess.

cwdstr, optional

Specify the current working directory to use when running the process.

redirect_stderrbool, optional

Whether to redirect stderr to stdout. In this case the returned stderr (when return_stderr == True) is an empty string.

return_popenbool, optional

Whether to return immediately after subprocess.Popen.

Returns

stdout, stderr, retcode : when return_stderr == True stdout : otherwise

asv.util._killpg_safe(pgid, signo)

Same as os.killpg, but deal with OSX/BSD

asv.util.is_main_thread()

Return True if the current thread is the main thread.

asv.util.write_json(path, data, api_version=None, compact=False)

Writes JSON to the given path, including indentation and sorting.

Parameters

pathstr

File name to write

dataobject

Data to serialize as JSON

api_versionint, optional

API version number

compactbool, optional

Whether to produce compact, non-human readable JSON. Disables sorting and indentation.

asv.util.load_json(path, api_version=None, js_comments=False)

Loads JSON from the given path.

Parameters

pathstr

File name

api_versionstr or None

API version identifier

js_commentsbool, optional

Whether to allow nonstandard javascript-style comments in the file. Note that this slows down the loading significantly.

asv.util.update_json(cls, path, api_version, compact=False)

Perform JSON file format updates.

Parameters

clsobject

Object containing methods update_to_X which updates the given JSON tree from version X-1 to X.

pathstr

Path to JSON file

api_versionint

The current API version

asv.util.iter_chunks(s, n)

Iterator that returns elements from s in chunks of size n.

asv.util.pick_n(items, n)

Pick n items, attempting to get equal index spacing.

asv.util.get_multiprocessing(parallel)

If parallel indicates that we want to do multiprocessing, imports the multiprocessing module and sets the parallel value accordingly.

asv.util.iter_subclasses(cls)

Returns all subclasses of a class.

asv.util.hash_equal(a, b)

Returns True if a and b represent the same commit hash.

asv.util.get_cpu_info()

Gets a human-friendly description of this machine’s CPU.

Returns ‘’ if it can’t be obtained.

asv.util.get_memsize()

Returns the amount of physical memory in this machine.

Returns ‘’ if it can’t be obtained.

asv.util.format_text_table(rows, num_headers=0, top_header_span_start=0, top_header_text=None)

Format rows in as a reStructuredText table, in the vein of:

========== ========== ==========
--         top header text, span start 1
---------- ---------------------
 row0col0     r0c1      r0c2
========== ========== ==========
 row1col0     r1c1      r1c2
 row2col0     r2c1      r2c2
========== ========== ==========

asv.util._datetime_to_timestamp(dt, divisor)

asv.util.datetime_to_timestamp(dt)

Convert a Python datetime object to a UNIX timestamp.

asv.util.datetime_to_js_timestamp(dt)

Convert a Python datetime object to a JavaScript timestamp.

asv.util.js_timestamp_to_datetime(ts)

Convert a JavaScript timestamp to a Python datetime object.

asv.util.is_nan(x)

Returns True if x is a NaN value.

asv.util.is_na(value)

Return True if value is None or NaN

asv.util.mean_na(values)

Take a mean, with the understanding that None and NaN stand for missing data.

asv.util.geom_mean_na(values)

Compute geometric mean, with the understanding that None and NaN stand for missing data.

asv.util.long_path_open

asv.util.sanitize_filename(filename)

Replace characters to make a string safe to use in file names.

This is not a 1-to-1 mapping.

The implementation needs to match www/asv.js:escape_graph_parameter

asv.util.namedtuple_with_doc(name, slots, doc)

asv.util.recvall(sock, size)

Receive data of given size from a socket connection

asv.util.interpolate_command(command, variables)

Parse a command with interpolated variables to a sequence of commands.

The command is parsed as in posix-style shell (by shlex) and split to parts. Additional constructs recognized:

• ENVVAR=value <command>: parsed as declaring an environment variable named ‘ENVVAR’.

• return-code=value <command>: parsed as declaring valid return codes.

• in-dir=value <command>: parsed as declaring working directory for command.

Parameters

commandstr

Command to execute, posix shell style.

variablesdict

Interpolation variables.

Returns

commandlist of str

Command arguments.

envdict

Environment variables declared in the command.

return_codes{set, int, None}

Valid return codes.

cwd{str, None}

Current working directory for the command, if any.

asv.util.truncate_float_list(item, digits=5)

Truncate floating-point numbers (in a possibly nested list) to given significant digits, for a shorter base-10 representation.

asv.util._global_locks

asv.util._init_global_locks(lock_dict)

Initialize global locks in a new multiprocessing process

asv.util.new_multiprocessing_lock(name)

Create a new global multiprocessing lock

asv.util.get_multiprocessing_lock(name)

Get an existing global multiprocessing lock

asv.util.get_multiprocessing_pool(parallel=None)

Create a multiprocessing.Pool, managing global locks properly

asv.util._find_unsafe

asv.util.git_default_branch()

asv.util.search_channels(cli_path, pkg, version)

class asv.util.ParsedPipDeclaration(declaration)

pkgname = None

specification = None

flags = []

is_editable = False

path = None

_parse_declaration(declaration)

asv.util.construct_pip_call(pip_caller, parsed_declaration: ParsedPipDeclaration)

asv.util.ON_PYPY = True

asv.util.get_matching_environment(environments, result=None)

asv.util.replace_python_version(arg, new_version)