mljet.utils#

Submodules#

mljet.utils.conn module#

Helper functions for connecting across the network.

mljet.utils.conn.find_free_port()[source]#

Find a free port.

Return type:

int

Returns:

Number of free port to use.

mljet.utils.conn.is_port_in_use(port)[source]#

Check if port is in use.

Parameters:

port (int) – port to check

Return type:

bool

Returns:

True if port is in use, False otherwise

mljet.utils.logging_ module#

Logging module.

class mljet.utils.logging_.RichEmojiFilteredHandler(*args, enable_emoji=False, **kwargs)[source]#

Bases: RichHandler

Extended rich handler with emoji filter support.

Initializes the instance - basically setting the formatter to None and the filter list to empty.

HIGHLIGHTER_CLASS#

alias of ReprHighlighter

KEYWORDS = ['GET', 'POST', 'HEAD', 'PUT', 'DELETE', 'OPTIONS', 'TRACE', 'PATCH']#
acquire()#

Acquire the I/O thread lock.

addFilter(filter)#

Add the specified filter to this handler.

close()#

Tidy up any resources used by the handler.

This version removes the handler from an internal map of handlers, _handlers, which is used for handler lookup by name. Subclasses should ensure that this gets called from overridden close() methods.

createLock()#

Acquire a thread lock for serializing access to the underlying I/O.

emit(record)#

Invoked by logging.

Return type:

None

filter(record)#

Determine if a record is loggable by consulting all the filters.

The default is to allow the record to be logged; any filter can veto this and the record is then dropped. Returns a zero value if a record is to be dropped, else non-zero.

Changed in version 3.2: Allow filters to be just callables.

flush()#

Ensure all logging output has been flushed.

This version does nothing and is intended to be implemented by subclasses.

format(record)[source]#

Extends RichHandler. Format to filter out emoji.

Return type:

str

get_level_text(record)#

Get the level name from the record.

Parameters:

record (LogRecord) – LogRecord instance.

Returns:

A tuple of the style and level name.

Return type:

Text

get_name()#
handle(record)#

Conditionally emit the specified logging record.

Emission depends on filters which may have been added to the handler. Wrap the actual emission of the record with acquisition/release of the I/O thread lock. Returns whether the filter passed the record for emission.

handleError(record)#

Handle errors which occur during an emit() call.

This method should be called from handlers when an exception is encountered during an emit() call. If raiseExceptions is false, exceptions get silently ignored. This is what is mostly wanted for a logging system - most users will not care about errors in the logging system, they are more interested in application errors. You could, however, replace this with a custom handler if you wish. The record which was being processed is passed in to this method.

property name#
release()#

Release the I/O thread lock.

removeFilter(filter)#

Remove the specified filter from this handler.

render(*, record, traceback, message_renderable)#

Render log for display.

Parameters:

  • record (LogRecord) – logging Record.

  • traceback (Optional[Traceback]) – Traceback instance or None for no Traceback.

  • message_renderable (ConsoleRenderable) – Renderable (typically Text) containing log message contents.

Returns:

Renderable to display log.

Return type:

ConsoleRenderable

render_message(record, message)#

Render message text in to Text.

Parameters:

  • record (LogRecord) – logging Record.

  • message (str) – String containing log message.

Returns:

Renderable to display log message.

Return type:

ConsoleRenderable

setFormatter(fmt)#

Set the formatter for this handler.

setLevel(level)#

Set the logging level of this handler. level must be an int or a str.

set_name(name)#
mljet.utils.logging_.init(verbose=False, enable_emoji=False, rich=True)[source]#
Init logging.
Args:
verbose (bool):

Verbose level (DEBUG) or not (INFO).

enable_emoji (bool):

Enable emoji in logs or not.

rich (bool):

Enable rich handler & traceback or not.

Raises:

AnyError: If anything bad happens.

mljet.utils.names_generator module#

Docker image name generator.

mljet.utils.names_generator.get_random_name()[source]#
Return type:

str

mljet.utils.requirements module#

Module for scanning Python files for requirements.

mljet.utils.requirements.extract_modules(node, ignore_mods=None)[source]#

Extract the modules from an import node

Parameters:

  • node (Union[Import, ImportFrom]) – The import node

  • ignore_mods (Optional[List[str]]) – List of modules to ignore

Return type:

dict

mljet.utils.requirements.freeze()[source]#

Get a dictionary of installed packages and their versions

Return type:

Dict[str, str]

Returns:

A dictionary of installed packages and their versions

mljet.utils.requirements.get_pkgs_distributions()[source]#

Get a dictionary of installed packages and their module names

Return type:

dict

Returns:

A dictionary of installed packages and their module names

mljet.utils.requirements.get_source_from_notebook(path)[source]#

Extract the source code from a Jupyter notebook

Parameters:

path (Union[str, Path]) – Path to the notebook

Return type:

str

Returns:

The source code as a string

Raises:

RuntimeError – If the notebook is not valid JSON

mljet.utils.requirements.make_requirements_txt(path, out_path='requirements.txt', strict=True, extensions=None, ignore_mods=None)[source]#

Make a requirements.txt file from a directory of files.

Parameters:

  • path (Union[str, Path]) – Path to the directory

  • out_path (Union[str, Path]) – Path to the output file

  • extensions (Optional[List[str]]) – List of file extensions to scan. Defaults to [‘py’, ‘ipynb’]

  • strict (Optional[bool]) – Set only the exact version of the packages

  • ignore_mods (Optional[List[str]]) – List of modules to ignore

Return type:

Dict[str, str]

Returns:

A dict of requirements and their versions

Raises:

ValueError – If the path is not correct

mljet.utils.requirements.merge(*requirements_lists)[source]#

Merge requirements lists.

Parameters:

requirements_lists (List[str]) – list of requirements

Return type:

List[str]

Returns:

Merged requirements

mljet.utils.requirements.merge_requirements_txt(*files, ignore_prefixes=None)[source]#

Merge requirements.txt files.

Parameters:
Return type:

List[str]

Returns:

Final requirements.txt file content

mljet.utils.requirements.scan_requirements(path, extensions=None, ignore_mods=None, ignore_names=None)[source]#

Scan a directory of file for requirements.

Parameters:

  • path (Union[str, Path]) – Path to the directory

  • extensions (Optional[List[str]]) – List of file extensions to scan. Defaults to [‘py’, ‘ipynb’]

  • ignore_mods (Optional[List[str]]) – List of modules to ignore

  • ignore_names (Optional[List[str]]) – List of file/dirs names to ignore

Return type:

Dict[str, str]

Returns:

A dict of requirements and their versions

Raises:

ValueError – If the path is not correct

mljet.utils.requirements.validate(req)[source]#

mljet.utils.types module#

Module contains type’s aliases.

final class mljet.utils.types.Estimator(*args, **kwargs)[source]#

Bases: Protocol

Protocol for estimators.

fit(*args, **kwargs)[source]#
Return type:

Any

predict(*args, **kwargs)[source]#
Return type:

Any

final class mljet.utils.types.Serializer(*args, **kwargs)[source]#

Bases: Protocol

Protocol for serializers.

static dump(obj, file, *args, **kwargs)[source]#
Return type:

None

static load(file, *args, **kwargs)[source]#
Return type:

Any

mljet.utils.utils module#

Utils module.

mljet.utils.utils.drop_unnecessary_kwargs(func, kwargs)[source]#

Drop unnecessary kwargs.

Return type:

dict

mljet.utils.utils.is_package_installed(name)[source]#

Check if package is installed and valid.

Return type:

bool

mljet.utils.utils.parse_cls_name(obj)[source]#

Parse class name.

Return type:

str

Module contents#

Helper functions for the mljet package.