text:
<!DOCTYPE html>

  <meta charset="utf-8">

  <meta name="viewport" content="width=device-width, initial-scale=1.0">

  <title>mlens.parallel package — mlens 0.2.0 documentation</title>

    <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />

    <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Lato" type="text/css" />

    <link rel="stylesheet" href="../_static/css/pygments.css" type="text/css" />

    <link rel="stylesheet" href="../_static/css/mlens-theme.css" type="text/css" />

    <link rel="stylesheet" href="../_static/css/sphinx-glr.css" type="text/css" />

        <link rel="index" title="Index"
              href="../genindex.html"/>
        <link rel="search" title="Search" href="../search.html"/>
    <link rel="top" title="mlens 0.2.0 documentation" href="../index.html"/>

                0.2.0

  <form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>

Installation

Install

Bleeding edge

Dependencies

Test build

High-level API

Getting started

Preliminaries

Ensemble guide

Building an ensemble

Multi-layer ensembles

Model selection guide

The scoring function

A simple evaluation

Preprocessing

Model Selection across preprocessing pipelines

Visualization guide

Advanced features tutorial

Propagating input features

Probabilistic ensemble learning

Advanced Subsemble techniques

General multi-layer ensemble learning

Passing file paths as data input

Ensemble model selection

Ready-made ensemble classes

Super Learner

References

Notes

Subsemble

References

Notes

Blend Ensemble

Sequential Ensemble

Mechanics

Learner Mechanics

The Learner API

Basics

Partitioning

Preprocessing

Parallel estimation

Layer Mechanics

The Layer API

Basics

Parallel Mechanics

Sequential Mechanics

Details

Memory consumption

Memory mapping

ML-Ensemble memory profiling

Memory performance benchmark

Gotcha's

Performance benchmarks

MNIST

Benchmark

The Friedman Regression Problem 1

Benchmark

References

Scale benchmarks

Single process vs multi-process

Ensemble comparison

Hacking ML-Ensemble

Troubleshooting

Bad interaction with third-party packages

Array copying during fitting

Documentation

API

Front-end API

Ensemble estimators

Model Selection

Preprocessing

Visualization

Low-level API

Parallel estimation API

Ensemble Base classes

Indexers

Global configurations

Additional Information

License

Change log

Package index

mlens

mlens package

Subpackages

mlens.ensemble package

mlens.estimators package

mlens.index package

mlens.metrics package

mlens.model_selection package

mlens.parallel package

mlens.preprocessing package

mlens.testing package

mlens.utils package

mlens.visualization package

Submodules

mlens.config module

Module contents

          mlens

Docs »

mlens.parallel package

             View page source

mlens.parallel package¶

Submodules¶

mlens.parallel.backend module¶

ML-Ensemble

<col class="field-name" />
<col class="field-body" />

author:	Sebastian Flennerhag

copyright:	2017

license:	MIT

Parallel processing backend classes. Manages memory-mapping of data, estimation
caching and job scheduling.

class mlens.parallel.backend.BaseProcessor(backend=None, n_jobs=None, verbose=None)[source]¶

Bases: object

Parallel processing base class.

Base class for parallel processing engines.

<col class="field-name" />
<col class="field-body" />

Parameters:

backend (str, optional) – Type of backend. One of 'threading', 'multiprocessing',
'sequential'.

n_jobs (int, optional) – Degree of concurrency.

verbose (bool, int, optional) – Level of verbosity of the
Parallel instance.

backend¶

caller¶

clear()[source]¶

Destroy cache and reset instance job parameters.

initialize(job, X, y, path, warm_start=False, return_preds=False, **kwargs)[source]¶

Initialize processing engine.

Set up the job parameters before an estimation call. Calling
clear()
undoes initialization.

<col class="field-name" />
<col class="field-body" />

Parameters:

job (str) – type of job to complete with each task. One of 'fit',
'predict' and 'transform'.

X (array-like of shape [n_samples, n_features]) – Input data

y (array-like of shape [n_samples,], optional.) – targets. Required for fit, should not be passed to predict or
transform jobs.

path (str or dict, optional) – Custom estimation cache. Pass a string to force use of persistent
cache on disk. Pass a dict for in-memory cache (requires
backend != 'multiprocessing'.

return_preds (bool or list, optional) – whether to return prediction ouput. If True, final prediction
is returned. Alternatively, pass a list of task names for which
output should be returned.

warm_start (bool, optional) – whether to re-use previous input data initialization. Useful if
repeated jobs are made on the same input arrays.

**kwargs (optional) – optional keyword arguments to pass onto the task's call method.

Returns:
out – An output parameter dictionary to pass to pass to an estimation
method. Either None (no output), or
{'final':True} for only final prediction, or
{'final': False, 'return_names': return_preds} if a list of
task-specific output was passed.

Return type:
dict

job¶

n_jobs¶

verbose¶

class mlens.parallel.backend.Job(job, stack, split)[source]¶

Bases: object

Container class for holding and managing job data.

Job is intended as a on-the-fly job handler that keeps track
of input data, predictions, and manages estimation caches.

Changed in version 0.2.0.

See also

ParallelProcessing, ParallelEvaluation

<col class="field-name" />
<col class="field-body" />

Parameters:

job (str) – Type of job to run. One of 'fit', 'transform', 'predict'.

stack (bool) – Whether to stack outputs when calls to
update() are made. This will make
the predict_out array become predict_in.

split (bool) – Whether to create a new sub-cache when the
args property is called.

args(**kwargs)[source]¶

Produce args dict

New in version 0.2.0.

Returns the arguments dictionary passed to a task of a parallel
processing manager. Output dictionary has the following form:

out = {'auxiliary':
           {'X': self.predict_in, 'P': self.predict_out},
       'main':
           {'X': self.predict_in, 'P': self.predict_out},
       'dir':
           self.subdir(),
       'job':
            self.job
        }

<col class="field-name" />
<col class="field-body" />

Parameters:	**kwargs (optional) – Optional keyword arguments to pass to the task.

Returns:	args – Arguments dictionary

Return type:	dict

clear()[source]¶

Clear output data for new task

dir¶

job¶

kwargs¶

predict_in¶

predict_out¶

rebase()[source]¶

Rebase output labels to input indexing.

Some indexers that only generate predictions for subsets of the
training data require the targets to be rebased. Since indexers
operate in a strictly sequential manner, rebase simply drop the first
n observations in the target vector until number of observations
remaining coincide.

See also

BlendIndex

shuffle(random_state)[source]¶

Shuffle inputs.

Permutes the indexing of predict_in and y arrays.

<col class="field-name" />
<col class="field-body" />

Parameters:	random_state (int, obj) – Random seed number or generator to use.

split¶

stack¶

subdir()[source]¶

Return a cache subdirectory

If split is en force, a new sub-cache will be created in the
main cache. Otherwise the same sub-cache as used in previous call
will be returned.

New in version 0.2.0.

<col class="field-name" />
<col class="field-body" />

Returns:	cache – Either a string pointing to a cache persisted to disk, or an
in-memory cache in the form of a list.

Return type:	str, list

tmp¶

update()[source]¶

Updated output array and shift to input if stacked.

If stacking is en force, the output array will replace the input
array, and used as input for subsequent jobs. Sparse matrices are
force-converted to csr format.

y¶

class mlens.parallel.backend.ParallelEvaluation(*args, **kwargs)[source]¶

Bases: mlens.parallel.backend.BaseProcessor

Parallel cross-validation engine.

Minimal parallel processing engine. Similar to ParallelProcessing,
but offers less features, only fits the callers indexer, and excepts
no task output.

process(caller, case, X, y, path=None, **kwargs)[source]¶

Process caller.

<col class="field-name" />
<col class="field-body" />

Parameters:

caller (iterable) – Iterable for evaluation job.s Expected caller is a
Evaluator instance.

case (str) – evaluation case to run on the evaluator. One of
'preprocess' and 'evaluate'.

X (array-like of shape [n_samples, n_features]) – Input data

y (array-like of shape [n_samples,], optional.) – targets. Required for fit, should not be passed to predict or
transform jobs.

path (str or dict, optional) – Custom estimation cache. Pass a string to force use of persistent
cache on disk. Pass a dict for in-memory cache (requires
backend != 'multiprocessing'.

class mlens.parallel.backend.ParallelProcessing(*args, **kwargs)[source]¶

Bases: mlens.parallel.backend.BaseProcessor

Parallel processing engine.

Engine for running computational graph.

ParallelProcessing is a manager for executing a sequence of tasks
in a given caller, where each task is run sequentially, but assumed to be
parallelized internally. The main responsibility of
ParallelProcessing is to handle memory-mapping, estimation
cache updates, input and output array updates and output collection.

<col class="field-name" />
<col class="field-body" />

Parameters:

caller (obj) – the caller of the job. Either a Layer or a meta layer class
such as Sequential.

*args (optional) – Optional arguments to BaseProcessor

**kwargs (optional) – Optional keyword arguments to
BaseProcessor.

get_preds(dtype=None, order='C')[source]¶

Return prediction matrix.

<col class="field-name" />
<col class="field-body" />

Parameters:

dtype (numpy dtype object, optional) – data type to return

order (str (default = 'C')) – data order. See numpy.asarray for details.

Returns:
P – Prediction array

Return type:
array-like

map(caller, job, X, y=None, path=None, return_preds=False, wart_start=False, split=False, **kwargs)[source]¶

Parallel task mapping.

Run independent tasks in caller in parallel.

Warning

By default, the :~mlens.parallel.backend.ParallelProcessing.map` runs
on a shallow cache, where all tasks share the same cache. As such, the
user must ensure that each task has a unique name, or cache retrieval
will be corrupted. To commit a seperate sub-cache to each task, set
split=True.

<col class="field-name" />
<col class="field-body" />

Parameters:

caller (iterable) – Iterable that generates accepted task instances. Caller should be
a child of the BaseBackend class,
and tasks need to implement an appropriate call method.

job (str) – type of job to complete with each task. One of 'fit',
'predict' and 'transform'.

X (array-like of shape [n_samples, n_features]) – Input data

y (array-like of shape [n_samples,], optional.) – targets. Required for fit, should not be passed to predict or
transform jobs.

path (str or dict, optional) – Custom estimation cache. Pass a string to force use of persistent
cache on disk. Pass a dict for in-memory cache (requires
backend != 'multiprocessing'.

return_preds (bool or list, optional) – whether to return prediction ouput. If True, final prediction
is returned. Alternatively, pass a list of task names for which
output should be returned.

warm_start (bool, optional) – whether to re-use previous input data initialization. Useful if
repeated jobs are made on the same input arrays.

split (bool, default = False) – whether to commit a separate sub-cache to each task.

**kwargs (optional) – optional keyword arguments to pass onto each task.

Returns:
out – Prediction array(s).

Return type:
array-like, list, optional

process(caller, out, **kwargs)[source]¶

Process job.

Main method for processing a caller. Requires the instance to be
setup by a prior call to
initialize().

See also

map(),
stack()

<col class="field-name" />
<col class="field-body" />

Parameters:

caller (iterable) – Iterable that generates accepted task instances. Caller should be
a child of the BaseBackend class,
and tasks need to implement an appropriate call method.

out (dict) – A dictionary with output parameters. Pass an empty dict for no
output. See
initialize() for more
details.

Returns:
out – Prediction array(s).

Return type:
array-like, list, optional

stack(caller, job, X, y=None, path=None, return_preds=False, wart_start=False, split=True, **kwargs)[source]¶

Stacked parallel task mapping.

Run stacked tasks in caller in parallel.

This method runs a stack of tasks as a stack, where the output of
each task is the input to the next.

Warning

By default, the stack()
method runs on a deep cache, where each tasks has a separate cache.
As such, the user must ensure that tasks don't depend on data cached
by previous tasks. To run all tasks in a single sub-cache, set
split=False.

<col class="field-name" />
<col class="field-body" />

Parameters:

caller (iterable) – Iterable that generates accepted task instances. Caller should be
a child of the BaseBackend class,
and tasks need to implement an appropriate call method.

job (str) – type of job to complete with each task. One of 'fit',
'predict' and 'transform'.

X (array-like of shape [n_samples, n_features]) – Input data

y (array-like of shape [n_samples,], optional.) – targets. Required for fit, should not be passed to predict or
transform jobs.

path (str or dict, optional) – Custom estimation cache. Pass a string to force use of persistent
cache on disk. Pass a dict for in-memory cache (requires
backend != 'multiprocessing'.

return_preds (bool or list, optional) – whether to return prediction output. If True, final prediction
is returned. Alternatively, pass a list of task names for which
output should be returned.

warm_start (bool, optional) – whether to re-use previous input data initialization. Useful if
repeated jobs are made on the same input arrays.

split (bool, default = True) – whether to commit a separate sub-cache to each task.

**kwargs (optional) – optional keyword arguments to pass onto each task.

Returns:
out – Prediction array(s).

Return type:
array-like, list, optional

mlens.parallel.backend.dump_array(array, name, path)[source]¶

Dump array for memmapping.

<col class="field-name" />
<col class="field-body" />

Parameters:

array (array-like) – Array to be persisted

name (str) – Name of file

path (str) – Path to cache.

Returns:
f – memory-mapped array.

Return type:
array-like

mlens.parallel.base module¶

ML-Ensemble

<col class="field-name" />
<col class="field-body" />

author:	Sebastian Flennerhag

copyright:	2017

license:	MIT

Base classes for parallel estimation

Schedulers for global setups:

0:

Base setups - independent of other features:

IndexMixin._setup_0_index

1:

Global setups - reserved for aggregating classes:

Layer._setup_1_global

2:

Dependents on 0:

ProbaMixin.__setup_2_multiplier

3:

Dependents on 0, 2:

OutputMixin.__setup_3__output_columns

Note that schedulers are experimental and may change without a deprecation
cycle.

class mlens.parallel.base.BaseBackend(backend=None, n_jobs=-1, dtype=None, raise_on_exception=True)[source]¶

Bases: object

Base class for parallel backend

Implements default backend settings.

class mlens.parallel.base.BaseEstimator(*args, **kwargs)[source]¶

Bases: mlens.parallel.base.ParamMixin, mlens.externals.sklearn.base.BaseEstimator, mlens.parallel.base.BaseParallel

Base Parallel Estimator class

Modified Scikit-learn class to handle backend params that we want to
protect from changes.

get_params(deep=True)[source]¶

class mlens.parallel.base.BaseParallel(name, *args, **kwargs)[source]¶

Bases: mlens.parallel.base.BaseBackend

Base class for parallel objects

<col class="field-name" />
<col class="field-body" />

Parameters:

name (str) – name of instance. Should be unique.

backend (str or object (default = 'threading')) – backend infrastructure to use during call to
mlens.externals.joblib.Parallel. See Joblib for further
documentation. To set global backend, see mlens.config.set_backend.

raise_on_exception (bool (default = True)) – whether to issue warnings on soft exceptions or raise error.
Examples include lack of layers, bad inputs, and failed fit of an
estimator in a layer. If set to False, warnings are issued instead
but estimation continues unless exception is fatal. Note that this
can result in unexpected behavior unless the exception is anticipated.

verbose (int or bool (default = False)) – level of verbosity.

n_jobs (int (default = -1)) – Degree of concurrency in estimation. Set to -1 to maximize
paralellization, while 1 runs on a single process (or thread
equivalent). Cannot be overriden in the add method.

dtype (obj (default = np.float32)) – data type to use, must be compatible with a numpy array dtype.

setup(X, y, job, skip=None, **kwargs)[source]¶

Setup instance for estimation

class mlens.parallel.base.BaseStacker(stack=None, verbose=False, *args, **kwargs)[source]¶

Bases: mlens.parallel.base.BaseEstimator

Base class for instanes that stack job estimators

get_params(deep=True)[source]¶

Get parameters for this estimator.

<col class="field-name" />
<col class="field-body" />

Parameters:	deep (boolean, optional) – whether to return nested parameters.

pop(idx)[source]¶

Pop a previous push with index idx

push(*stack)[source]¶

Push onto stack

replace(idx, item)[source]¶

Replace a current member of the stack with a new instance

verbose¶

Verbosity

class mlens.parallel.base.IndexMixin[source]¶

Bases: object

Indexer mixin

Mixin for handling indexers.

To use this mixin the instance inheriting it must set the
indexer or indexers attribute in __init__ (not both).

class mlens.parallel.base.OutputMixin[source]¶

Bases: mlens.parallel.base.IndexMixin

Output Mixin

Mixin class for interfacing with ParallelProcessing when outputs are
desired.

Note

To use this mixin the instance inheriting it must set the
feature_span attribute and __no_output__ flag in __init__.

set_output_columns(X, y, job, n_left_concats=0)[source]¶

Set output columns for prediction array

shape(job)[source]¶

Prediction array shape

size(attr)[source]¶

Get size of dim 0

class mlens.parallel.base.ParamMixin[source]¶

Bases: mlens.externals.sklearn.base.BaseEstimator, object

Parameter Mixin

Mixin for protecting static parameters from changes after fitting.

To use this mixin the instance inheriting it must set
__static__=list() and _static_fit_params_=dict()  in __init__.

class mlens.parallel.base.ProbaMixin[source]¶

Bases: object

"Probability Mixin

Mixin for probability features on objects
interfacing with ParallelProcessing

Note

To use this mixin the instance inheriting it must set the proba
and the _classes(=None)``attribute in ``__init__.

classes_¶

Prediction classes during proba

mlens.parallel.handles module¶

ML-Ensemble

<col class="field-name" />
<col class="field-body" />

author:	Sebastian Flennerhag

copyright:	2017

license:	MIT

Handles for mlens.parallel.

class mlens.parallel.handles.Group(indexer=None, learners=None, transformers=None, name=None, **kwargs)[source]¶

Bases: mlens.parallel.base.BaseEstimator

A handle for learners and transformers that share a common indexer.

Lightweight class for pairing a set of independent learners with
a set of transformers that all share the same cross-validation strategy.
A Group instance is an acceptable caller to
ParallelProcessing.

New in version 0.2.0.

Note

All instances will share the same indexer. If instances have a
different indexer, that indexer will be replaced.

See also

To run a Group instance, see run().
To handle several groups, use the Layer
class.

<col class="field-name" />
<col class="field-body" />

Parameters:

indexer (inst, optional) – A :module:`mlens.index` indexer to build learner and transformers on.
If not passed, the first indexer of the learners will be enforced
on all instances.

learners (list, inst, optional) – Learner instance(s) attached to
indexer. Note that Group overrides previous
indexer parameter settings.

transformers (list, inst, optional) – Transformer instance(s) attached to
indexer. Note that Group overrides previous
indexer parameter settings.

name (str, optional) – name of group

**kwargs (optional) – Optional keyword arguments to the
BaseParallel backend.

get_params(deep=True)[source]¶

class mlens.parallel.handles.Pipeline(pipeline, name=None, return_y=False)[source]¶

Bases: mlens.externals.sklearn.base.BaseEstimator

Transformer pipeline

Pipeline class for wrapping a preprocessing pipeline of transformers.

<col class="field-name" />
<col class="field-body" />

Parameters:

pipeline (list, instance) –
A Transformer instance or a list of
Transformer
instances. Accepted input formats:

option_1 = transformer_1
option_2 = [transformer_1, transformer_2]
option_3 = [("tr-1", transformer_1), ("tr-2", transformer_2)]
option_4 = [transformer_1, ("tr-2", transformer_2)]

name (str, optional) – name of pipeline.

return_y (bool, default = False) – If True, both X and y will be returned in a
transform() call.

fit(X, y=None)[source]¶

Fit pipeline.

Note that the Pipeline accepts both X and y arguments, and
can return both X and y, depending on the transformers. The
pipeline itself does no checks on the input.

<col class="field-name" />
<col class="field-body" />

Parameters:

X (array-like of shape [n_samples, n_features]) – Input data

y (array-like of shape [n_samples, ]) – Targets

Returns:
self – Fitted pipeline

Return type:
instance

fit_transform(X, y=None)[source]¶

Fit and transform pipeline.

Note that the Pipeline accepts both X and y arguments, and
can return both X and y, depending on the transformers. The
pipeline itself does no checks on the input.

<col class="field-name" />
<col class="field-body" />

Parameters:

X (array-like of shape [n_samples, n_features]) – Input data

y (array-like of shape [n_samples, ]) – Targets

Returns:

X_processed (array-like of shape [n_samples, n_preprocessed_features]) – Preprocessed input data

y (array-like of shape [n_samples, ], optional) – Preprocessed targets

get_params(deep=True)[source]¶

transform(X, y=None)[source]¶

Transform pipeline.

Note that the Pipeline accepts both X and y arguments, and
can return both X and y, depending on the transformers.
Pipeline itself does not checks the input.

<col class="field-name" />
<col class="field-body" />

Parameters:

X (array-like of shape [n_samples, n_features]) – Input data

y (array-like of shape [n_samples, ]) – Targets

Returns:

X_processed (array-like of shape [n_samples, n_preprocessed_features]) – Preprocessed input data

y (array-like of shape [n_samples, ], optional) – Original or preprocessed targets, depending on the transformers.

mlens.parallel.handles.make_group(indexer, estimators, preprocessing, learner_kwargs=None, transformer_kwargs=None, name=None)[source]¶

Creating a Group from a set learners and transformers

Utility function for creating mapping a set of estimators and
preprocessing pipelines to a Group of
Learner and
Transformer instances.

<col class="field-name" />
<col class="field-body" />

Parameters:

indexer (instance or None, default = None) – Indexer instance to use. See :module:`mlens.index` for details.

estimators (dict of lists or list of estimators.) –
If preprocessing is None or list, estimators should
be a list. The list can either contain estimator instances,
named tuples of estimator instances, or a combination of both.

option_1 = [estimator_1, estimator_2]
option_2 = [("est-1", estimator_1), ("est-2", estimator_2)]
option_3 = [estimator_1, ("est-2", estimator_2)]

If different preprocessing pipelines are desired, a dictionary
that maps estimators to preprocessing pipelines must be passed.
The names of the estimator dictionary must correspond to the
names of the estimator dictionary.

preprocessing_cases = {"case-1": [trans_1, trans_2].
                       "case-2": [alt_trans_1, alt_trans_2]}

estimators = {"case-1": [est_a, est_b].
              "case-2": [est_c, est_d]}

The lists for each dictionary entry can be any of option_1,
option_2 and option_3.

preprocessing : dict of lists or list, optional (default = None)

preprocessing pipelines for given layer. If
the same preprocessing applies to all estimators, preprocessing
should be a list of transformer instances. The list can contain the
instances directly, named tuples of transformers,
or a combination of both.

option_1 = [transformer_1, transformer_2]
option_2 = [("trans-1", transformer_1),
            ("trans-2", transformer_2)]
option_3 = [transformer_1, ("trans-2", transformer_2)]

If different preprocessing pipelines are desired, a dictionary
that maps preprocessing pipelines must be passed. The names of the
preprocessing dictionary must correspond to the names of the
estimator dictionary.

preprocessing_cases = {"case-1": [trans_1, trans_2].
                       "case-2": [alt_trans_1, alt_trans_2]}

estimators = {"case-1": [est_a, est_b].
              "case-2": [est_c, est_d]}

The lists for each dictionary entry can be any of option_1,
option_2 and option_3.

transformer_kwargs : dict, optional

Keyword arguments to pass to the
Transformer instances.

learner_kwargs : dict, optional

Keyword arguments to pass to the
Learner instances.

name : str, optional

Name of group. Should be unique.

mlens.parallel.layer module¶

ML-Ensemble

<col class="field-name" />
<col class="field-body" />

author:	Sebastian Flennerhag

license:	MIT

copyright:	2017

Layer module.

class mlens.parallel.layer.Layer(name=None, propagate_features=None, shuffle=False, random_state=None, verbose=False, stack=None, **kwargs)[source]¶

Bases: mlens.parallel.base.OutputMixin, mlens.parallel.base.IndexMixin, mlens.parallel.base.BaseStacker

Layer of preprocessing pipes and estimators.

Layer is an internal class that holds a layer and its associated data
including an estimation procedure. It behaves as an estimator from an
Scikit-learn API point of view.

<col class="field-name" />
<col class="field-body" />

Parameters:

propagate_features (list, range, optional) – Features to propagate from the input array to the output array.
Carries input features to the output of the layer, useful for
propagating original data through several stacked layers. Propagated
features are stored in the left-most columns.

verbose (int or bool (default = False)) –
level of verbosity.

verbose = 0 silent (same as verbose = False)

verbose = 1 messages at start and finish (same as verbose = True)

verbose = 2 messages for preprocessing and estimators

verbose = 3 messages for completed job

If verbose >= 10 prints to sys.stderr, else sys.stdout.

shuffle (bool (default = False)) – Whether to shuffle data before fitting layer.

random_state (obj, int, optional) – Random seed number to use for shuffling inputs

**kwargs (optional) – optional arguments to BaseParallel.

collect(path)[source]¶

Collect cache estimators

data¶

Cross validated scores

indexers¶

Check indexer

learners¶

Generator for learners in layer

raw_data¶

Cross validated scores

set_output_columns(X, y, job, n_left_concats=0)[source]¶

Compatibility method for setting learner output columns

transformers¶

Generator for learners in layer

mlens.parallel.learner module¶

ML-Ensemble

<col class="field-name" />
<col class="field-body" />

author:	Sebastian Flennerhag

copyright:	2017

license:	MIT

Learner classes

class mlens.parallel.learner.Cache(obj, verbose)[source]¶

Bases: object

Cache wrapper for IndexedEstimator

class mlens.parallel.learner.EvalLearner(estimator, preprocess, name, attr, scorer, error_score=None, verbose=False, **kwargs)[source]¶

Bases: mlens.parallel.learner.Learner

EvalLearner is a derived class from Learner used for cross-validated
scoring of an estimator.

<col class="field-name" />
<col class="field-body" />

Parameters:

estimator (obj) – estimator to construct learner from

preprocess (str) – preprocess cache refernce

indexer (obj, None) – indexer to use for generating fits.
Set to None to fit only on all data.

name (str) – name of learner. If preprocess is not None,
the name will be prepended to preprocess__name.

attr (str (default='predict')) – predict attribute, typically one of 'predict' and 'predict_proba'

scorer (func) – function to use for scoring predictions during cross-validated
fitting.

error_score (int, float, None (default = None)) – score to set if cross-validation fails. Set to None to raise error.

verbose (bool, int (default = False)) – whether to report completed fits.

raise_on_exception (bool (default=True)) – whether to warn on non-fatal exceptions or raise an error.

gen_fit(X, y, P=None, refit=True)[source]¶

Generator for fitting learner on given data

class mlens.parallel.learner.EvalSubLearner(job, parent, estimator, in_index, out_index, in_array, targets, index)[source]¶

Bases: mlens.parallel.learner.SubLearner

sub-routine for cross-validated evaluation.

data¶

Score data

fit(path)[source]¶

Evaluate sub-learner

class mlens.parallel.learner.EvalTransformer(estimator, indexer=None, name=None, **kwargs)[source]¶

Bases: mlens.parallel.learner.Transformer

Evaluator version of the Transformer.

Derived class from Transformer adapted to cross-validated grid-search.
See Transformer for more details.

class mlens.parallel.learner.IndexedEstimator(estimator, name, index, in_index, out_index, data)[source]¶

Bases: object

Indexed Estimator

Lightweight wrapper around estimator dumps during fitting.

data¶

estimator¶

Deep copy of estimator

in_index¶

index¶

name¶

out_index¶

class mlens.parallel.learner.Learner(estimator, indexer=None, name=None, preprocess=None, attr=None, scorer=None, proba=False, **kwargs)[source]¶

Bases: mlens.parallel.base.ProbaMixin, mlens.parallel.learner._BaseEstimator

Wrapper for base learners.

<col class="field-name" />
<col class="field-body" />

Parameters:

estimator (obj) – estimator to construct learner from

preprocess (str, obj) – preprocess transformer. Pass either the string
cache reference or the transformer instance. If the latter,
the preprocess will refer to the transformer name.

name (str) – name of learner. If preprocess is not None,
the name will be prepended to preprocess__name.

attr (str (default='predict')) – predict attribute, typically one of 'predict' and 'predict_proba'

scorer (func) – function to use for scoring predictions during cross-validated
fitting.

output_columns (dict, optional) – mapping of prediction feature columns from learner to columns in
output array. Normally, this map is {0: x}, but if the indexer
creates partitions, each partition needs to be mapped:
{0: x, 1: x + 1}. Note that if output_columns are not given at
initialization, the set_output_columns method must be called before
running estimations.

verbose (bool, int (default = False)) – whether to report completed fits.

**kwargs (bool (default=True)) – Optional ParallelProcessing arguments. See BaseParallel.

scorer¶

Copy of scorer

class mlens.parallel.learner.SubLearner(job, parent, estimator, in_index, out_index, in_array, targets, out_array, index)[source]¶

Bases: object

Estimation task

Wrapper around a sub_learner job.

data¶

fit data

fit(path)[source]¶

Fit sub-learner

predict(path)[source]¶

Predict with sublearner

transform(path)[source]¶

Predict with sublearner

class mlens.parallel.learner.SubTransformer(job, parent, estimator, in_index, in_array, targets, index, out_index=None, out_array=None)[source]¶

Bases: object

Sub-routine for fitting a pipeline

data¶

fit data

fit(path)[source]¶

Fit transformers

predict(path=None)[source]¶

Dump transformers for prediction

transform(path=None)[source]¶

Dump transformers for prediction

class mlens.parallel.learner.Transformer(estimator, indexer=None, name=None, **kwargs)[source]¶

Bases: mlens.parallel.learner._BaseEstimator

Preprocessing handler.

Wrapper for transformation pipeline.

<col class="field-name" />
<col class="field-body" />

Parameters:

indexer (obj, None) – indexer to use for generating fits.
Set to None to fit only on all data.

estimator (obj) – transformation pipeline to construct learner from

name (str) – name of learner. If preprocess is not None,
the name will be prepended to preprocess__name.

output_columns (dict, optional) – If transformer is to be used to output data, need to
set output_columns. Normally, this map is
{0: x}, but if the indexer
creates partitions, each partition needs to be mapped:
{0: x, 1: x + 1}.

verbose (bool, int (default = False)) – whether to report completed fits.

raise_on_exception (bool (default=True)) – whether to warn on non-fatal exceptions or raise an error.

mlens.parallel.wrapper module¶

ML-Ensemble

<col class="field-name" />
<col class="field-body" />

author:	Sebastian Flennerhag

copyright:	2017

license:	MIT

Estimator wrappers around base classes.

class mlens.parallel.wrapper.EstimatorMixin[source]¶

Bases: object

Estimator mixin

Mixin class to build an estimator from a mlens.parallel backend
class. The backend class should be set as the _backend attribute
of the estimator during a fit call via a _build method. E.g:

Foo(EstimatorMixin, Learner):

    def __init__(self, ...):

        self._backend = None

    def _build(self):
        self._backend = Learner(...)

It is recommended to combine EstimatorMixin with
parallel.base.ParamMixin.

fit(X, y, proba=False, refit=True)[source]¶

Fit

Fit estimator.

<col class="field-name" />
<col class="field-body" />

Parameters:

X (array of size [n_samples, n_features]) – input data

y (array of size [n_features,]) – targets

proba (bool, optional) – whether to fit for later predict_proba calls. Will register number
of classes to expect in later predict and transform calls.

refit (bool (default = True)) – Whether to refit already fitted sub-learners.

Returns:
self – fitted estimator.

Return type:
instance

fit_transform(X, y, proba=False, refit=True)[source]¶

Fit

Fit estimator and return cross-validated predictions.

<col class="field-name" />
<col class="field-body" />

Parameters:

X (array of size [n_samples, n_features]) – input data

y (array of size [n_features,]) – targets

proba (bool, optional) – whether to fit for later predict_proba calls. Will register number
of classes to expect in later predict and transform calls.

refit (bool (default = True)) – Whether to refit already fitted sub-learners.

Returns:
P – prediction generated by cross-validation.

Return type:
array of size [n_samples, n_prediction_features]

predict(X, proba=False)[source]¶

Predict

Predict using full-fold estimator (fitted on all data).

<col class="field-name" />
<col class="field-body" />

Parameters:

X (array of size [n_samples, n_features]) – input data

proba (bool, optional) – whether to predict class probabilities

Returns:
P – prediction with full-fold estimator.

Return type:
array of size [n_samples, n_prediction_features]

transform(X, proba=False)[source]¶

Transform

Use cross-validated estimators to generate predictions.

<col class="field-name" />
<col class="field-body" />

Parameters:

X (array of size [n_samples, n_features]) – input data

proba (bool, optional) – whether to predict class probabilities

Returns:
P – prediction generated by cross-validation.

Return type:
array of size [n_samples, n_prediction_features]

mlens.parallel.wrapper.check_inputs(X, y, check_level)[source]¶

Wrapper to check inputs

mlens.parallel.wrapper.get_backend(instance)[source]¶

Check whether backend exists and return

mlens.parallel.wrapper.reset_flags(resets)[source]¶

Reset proba on backend

mlens.parallel.wrapper.run(caller, job, X, y=None, map=True, **kwargs)[source]¶

Utility for running a ParallelProcessing job on a set of callers.

Run is a utility mapping for setting up a ParallelProcessing job and
executing across a set of callers. By default run executes:

out = mgr.map(caller, job, X, y, **kwargs)

run() handles temporary parameter changes, for instance running
a learner with proba=True that has proba=False as default.
Similarly, instances destined to not produce output can be forced to
yield predictions by passing return_preds=True as a keyword argument.

Note

To run a learner with a preprocessing dependency, the
instances need to be wrapped in a Group

run(Group(learner, transformer), 'predict', X, y)

<col class="field-name" />
<col class="field-body" />

Parameters:

caller (instance, list) – A runnable instance, or a list of instances.

job (str) – type of job to run. One of 'fit', 'transform', 'predict'.

X (array-like) – input

y (array-like, optional) – targets

map (bool (default=True)) – whether to run a ParallelProcessing.map() job. If False,
will instead run a ParallelProcessing.stack() job.

**kwargs (optional) – Keyword arguments. run() searches for
proba and return_preds to temporarily update callers to run
desired job and return desired output. Other kwargs are passed
to either map or stack.

mlens.parallel.wrapper.set_flags(backend, flags)[source]¶

Set proba on backend

mlens.parallel.wrapper.set_output(kwargs, job, map)[source]¶

Set the __no_output__ flag

mlens.parallel.wrapper.set_predict(kwargs)[source]¶

Set attr argument and proba

Module contents¶

ML-ENSEMBLE

<col class="field-name" />
<col class="field-body" />

author:	Sebastian Flennerhag

copyright:	2017

license:	MIT

Computational graph module for memory-neutral parallel processing of
deep general-purpose ensembles.

class mlens.parallel.ParallelProcessing(*args, **kwargs)[source]¶

Bases: mlens.parallel.backend.BaseProcessor

Parallel processing engine.

Engine for running computational graph.

ParallelProcessing is a manager for executing a sequence of tasks
in a given caller, where each task is run sequentially, but assumed to be
parallelized internally. The main responsibility of
ParallelProcessing is to handle memory-mapping, estimation
cache updates, input and output array updates and output collection.

<col class="field-name" />
<col class="field-body" />

Parameters:

caller (obj) – the caller of the job. Either a Layer or a meta layer class
such as Sequential.

*args (optional) – Optional arguments to BaseProcessor

**kwargs (optional) – Optional keyword arguments to
BaseProcessor.

get_preds(dtype=None, order='C')[source]¶

Return prediction matrix.

<col class="field-name" />
<col class="field-body" />

Parameters:

dtype (numpy dtype object, optional) – data type to return

order (str (default = 'C')) – data order. See numpy.asarray for details.

Returns:
P – Prediction array

Return type:
array-like

map(caller, job, X, y=None, path=None, return_preds=False, wart_start=False, split=False, **kwargs)[source]¶

Parallel task mapping.

Run independent tasks in caller in parallel.

Warning

By default, the :~mlens.parallel.backend.ParallelProcessing.map` runs
on a shallow cache, where all tasks share the same cache. As such, the
user must ensure that each task has a unique name, or cache retrieval
will be corrupted. To commit a seperate sub-cache to each task, set
split=True.

<col class="field-name" />
<col class="field-body" />

Parameters:

caller (iterable) – Iterable that generates accepted task instances. Caller should be
a child of the BaseBackend class,
and tasks need to implement an appropriate call method.

job (str) – type of job to complete with each task. One of 'fit',
'predict' and 'transform'.

X (array-like of shape [n_samples, n_features]) – Input data

y (array-like of shape [n_samples,], optional.) – targets. Required for fit, should not be passed to predict or
transform jobs.

path (str or dict, optional) – Custom estimation cache. Pass a string to force use of persistent
cache on disk. Pass a dict for in-memory cache (requires
backend != 'multiprocessing'.

return_preds (bool or list, optional) – whether to return prediction ouput. If True, final prediction
is returned. Alternatively, pass a list of task names for which
output should be returned.

warm_start (bool, optional) – whether to re-use previous input data initialization. Useful if
repeated jobs are made on the same input arrays.

split (bool, default = False) – whether to commit a separate sub-cache to each task.

**kwargs (optional) – optional keyword arguments to pass onto each task.

Returns:
out – Prediction array(s).

Return type:
array-like, list, optional

process(caller, out, **kwargs)[source]¶

Process job.

Main method for processing a caller. Requires the instance to be
setup by a prior call to
initialize().

See also

map(),
stack()

<col class="field-name" />
<col class="field-body" />

Parameters:

caller (iterable) – Iterable that generates accepted task instances. Caller should be
a child of the BaseBackend class,
and tasks need to implement an appropriate call method.

out (dict) – A dictionary with output parameters. Pass an empty dict for no
output. See
initialize() for more
details.

Returns:
out – Prediction array(s).

Return type:
array-like, list, optional

stack(caller, job, X, y=None, path=None, return_preds=False, wart_start=False, split=True, **kwargs)[source]¶

Stacked parallel task mapping.

Run stacked tasks in caller in parallel.

This method runs a stack of tasks as a stack, where the output of
each task is the input to the next.

Warning

By default, the stack()
method runs on a deep cache, where each tasks has a separate cache.
As such, the user must ensure that tasks don't depend on data cached
by previous tasks. To run all tasks in a single sub-cache, set
split=False.

<col class="field-name" />
<col class="field-body" />

Parameters:

caller (iterable) – Iterable that generates accepted task instances. Caller should be
a child of the BaseBackend class,
and tasks need to implement an appropriate call method.

job (str) – type of job to complete with each task. One of 'fit',
'predict' and 'transform'.

X (array-like of shape [n_samples, n_features]) – Input data

y (array-like of shape [n_samples,], optional.) – targets. Required for fit, should not be passed to predict or
transform jobs.

path (str or dict, optional) – Custom estimation cache. Pass a string to force use of persistent
cache on disk. Pass a dict for in-memory cache (requires
backend != 'multiprocessing'.

return_preds (bool or list, optional) – whether to return prediction output. If True, final prediction
is returned. Alternatively, pass a list of task names for which
output should be returned.

warm_start (bool, optional) – whether to re-use previous input data initialization. Useful if
repeated jobs are made on the same input arrays.

split (bool, default = True) – whether to commit a separate sub-cache to each task.

**kwargs (optional) – optional keyword arguments to pass onto each task.

Returns:
out – Prediction array(s).

Return type:
array-like, list, optional

class mlens.parallel.ParallelEvaluation(*args, **kwargs)[source]¶

Bases: mlens.parallel.backend.BaseProcessor

Parallel cross-validation engine.

Minimal parallel processing engine. Similar to ParallelProcessing,
but offers less features, only fits the callers indexer, and excepts
no task output.

process(caller, case, X, y, path=None, **kwargs)[source]¶

Process caller.

<col class="field-name" />
<col class="field-body" />

Parameters:

caller (iterable) – Iterable for evaluation job.s Expected caller is a
Evaluator instance.

case (str) – evaluation case to run on the evaluator. One of
'preprocess' and 'evaluate'.

X (array-like of shape [n_samples, n_features]) – Input data

y (array-like of shape [n_samples,], optional.) – targets. Required for fit, should not be passed to predict or
transform jobs.

path (str or dict, optional) – Custom estimation cache. Pass a string to force use of persistent
cache on disk. Pass a dict for in-memory cache (requires
backend != 'multiprocessing'.

class mlens.parallel.Layer(name=None, propagate_features=None, shuffle=False, random_state=None, verbose=False, stack=None, **kwargs)[source]¶

Bases: mlens.parallel.base.OutputMixin, mlens.parallel.base.IndexMixin, mlens.parallel.base.BaseStacker

Layer of preprocessing pipes and estimators.

Layer is an internal class that holds a layer and its associated data
including an estimation procedure. It behaves as an estimator from an
Scikit-learn API point of view.

<col class="field-name" />
<col class="field-body" />

Parameters:

propagate_features (list, range, optional) – Features to propagate from the input array to the output array.
Carries input features to the output of the layer, useful for
propagating original data through several stacked layers. Propagated
features are stored in the left-most columns.

verbose (int or bool (default = False)) –
level of verbosity.

verbose = 0 silent (same as verbose = False)

verbose = 1 messages at start and finish (same as verbose = True)

verbose = 2 messages for preprocessing and estimators

verbose = 3 messages for completed job

If verbose >= 10 prints to sys.stderr, else sys.stdout.

shuffle (bool (default = False)) – Whether to shuffle data before fitting layer.

random_state (obj, int, optional) – Random seed number to use for shuffling inputs

**kwargs (optional) – optional arguments to BaseParallel.

collect(path)[source]¶

Collect cache estimators

data¶

Cross validated scores

indexers¶

Check indexer

learners¶

Generator for learners in layer

raw_data¶

Cross validated scores

set_output_columns(X, y, job, n_left_concats=0)[source]¶

Compatibility method for setting learner output columns

transformers¶

Generator for learners in layer

class mlens.parallel.Group(indexer=None, learners=None, transformers=None, name=None, **kwargs)[source]¶

Bases: mlens.parallel.base.BaseEstimator

A handle for learners and transformers that share a common indexer.

Lightweight class for pairing a set of independent learners with
a set of transformers that all share the same cross-validation strategy.
A Group instance is an acceptable caller to
ParallelProcessing.

New in version 0.2.0.

Note

All instances will share the same indexer. If instances have a
different indexer, that indexer will be replaced.

See also

To run a Group instance, see run().
To handle several groups, use the Layer
class.

<col class="field-name" />
<col class="field-body" />

Parameters:

indexer (inst, optional) – A :module:`mlens.index` indexer to build learner and transformers on.
If not passed, the first indexer of the learners will be enforced
on all instances.

learners (list, inst, optional) – Learner instance(s) attached to
indexer. Note that Group overrides previous
indexer parameter settings.

transformers (list, inst, optional) – Transformer instance(s) attached to
indexer. Note that Group overrides previous
indexer parameter settings.

name (str, optional) – name of group

**kwargs (optional) – Optional keyword arguments to the
BaseParallel backend.

get_params(deep=True)[source]¶

class mlens.parallel.Pipeline(pipeline, name=None, return_y=False)[source]¶

Bases: mlens.externals.sklearn.base.BaseEstimator

Transformer pipeline

Pipeline class for wrapping a preprocessing pipeline of transformers.

<col class="field-name" />
<col class="field-body" />

Parameters:

pipeline (list, instance) –
A Transformer instance or a list of
Transformer
instances. Accepted input formats:

option_1 = transformer_1
option_2 = [transformer_1, transformer_2]
option_3 = [("tr-1", transformer_1), ("tr-2", transformer_2)]
option_4 = [transformer_1, ("tr-2", transformer_2)]

name (str, optional) – name of pipeline.

return_y (bool, default = False) – If True, both X and y will be returned in a
transform() call.

fit(X, y=None)[source]¶

Fit pipeline.

Note that the Pipeline accepts both X and y arguments, and
can return both X and y, depending on the transformers. The
pipeline itself does no checks on the input.

<col class="field-name" />
<col class="field-body" />

Parameters:

X (array-like of shape [n_samples, n_features]) – Input data

y (array-like of shape [n_samples, ]) – Targets

Returns:
self – Fitted pipeline

Return type:
instance

fit_transform(X, y=None)[source]¶

Fit and transform pipeline.

Note that the Pipeline accepts both X and y arguments, and
can return both X and y, depending on the transformers. The
pipeline itself does no checks on the input.

<col class="field-name" />
<col class="field-body" />

Parameters:

X (array-like of shape [n_samples, n_features]) – Input data

y (array-like of shape [n_samples, ]) – Targets

Returns:

X_processed (array-like of shape [n_samples, n_preprocessed_features]) – Preprocessed input data

y (array-like of shape [n_samples, ], optional) – Preprocessed targets

get_params(deep=True)[source]¶

transform(X, y=None)[source]¶

Transform pipeline.

Note that the Pipeline accepts both X and y arguments, and
can return both X and y, depending on the transformers.
Pipeline itself does not checks the input.

<col class="field-name" />
<col class="field-body" />

Parameters:

X (array-like of shape [n_samples, n_features]) – Input data

y (array-like of shape [n_samples, ]) – Targets

Returns:

X_processed (array-like of shape [n_samples, n_preprocessed_features]) – Preprocessed input data

y (array-like of shape [n_samples, ], optional) – Original or preprocessed targets, depending on the transformers.

class mlens.parallel.Learner(estimator, indexer=None, name=None, preprocess=None, attr=None, scorer=None, proba=False, **kwargs)[source]¶

Bases: mlens.parallel.base.ProbaMixin, mlens.parallel.learner._BaseEstimator

Wrapper for base learners.

<col class="field-name" />
<col class="field-body" />

Parameters:

estimator (obj) – estimator to construct learner from

preprocess (str, obj) – preprocess transformer. Pass either the string
cache reference or the transformer instance. If the latter,
the preprocess will refer to the transformer name.

name (str) – name of learner. If preprocess is not None,
the name will be prepended to preprocess__name.

attr (str (default='predict')) – predict attribute, typically one of 'predict' and 'predict_proba'

scorer (func) – function to use for scoring predictions during cross-validated
fitting.

output_columns (dict, optional) – mapping of prediction feature columns from learner to columns in
output array. Normally, this map is {0: x}, but if the indexer
creates partitions, each partition needs to be mapped:
{0: x, 1: x + 1}. Note that if output_columns are not given at
initialization, the set_output_columns method must be called before
running estimations.

verbose (bool, int (default = False)) – whether to report completed fits.

**kwargs (bool (default=True)) – Optional ParallelProcessing arguments. See BaseParallel.

scorer¶

Copy of scorer

class mlens.parallel.Transformer(estimator, indexer=None, name=None, **kwargs)[source]¶

Bases: mlens.parallel.learner._BaseEstimator

Preprocessing handler.

Wrapper for transformation pipeline.

<col class="field-name" />
<col class="field-body" />

Parameters:

indexer (obj, None) – indexer to use for generating fits.
Set to None to fit only on all data.

estimator (obj) – transformation pipeline to construct learner from

name (str) – name of learner. If preprocess is not None,
the name will be prepended to preprocess__name.

output_columns (dict, optional) – If transformer is to be used to output data, need to
set output_columns. Normally, this map is
{0: x}, but if the indexer
creates partitions, each partition needs to be mapped:
{0: x, 1: x + 1}.

verbose (bool, int (default = False)) – whether to report completed fits.

raise_on_exception (bool (default=True)) – whether to warn on non-fatal exceptions or raise an error.

class mlens.parallel.EvalLearner(estimator, preprocess, name, attr, scorer, error_score=None, verbose=False, **kwargs)[source]¶

Bases: mlens.parallel.learner.Learner

EvalLearner is a derived class from Learner used for cross-validated
scoring of an estimator.

<col class="field-name" />
<col class="field-body" />

Parameters:

estimator (obj) – estimator to construct learner from

preprocess (str) – preprocess cache refernce

indexer (obj, None) – indexer to use for generating fits.
Set to None to fit only on all data.

name (str) – name of learner. If preprocess is not None,
the name will be prepended to preprocess__name.

attr (str (default='predict')) – predict attribute, typically one of 'predict' and 'predict_proba'

scorer (func) – function to use for scoring predictions during cross-validated
fitting.

error_score (int, float, None (default = None)) – score to set if cross-validation fails. Set to None to raise error.

verbose (bool, int (default = False)) – whether to report completed fits.

raise_on_exception (bool (default=True)) – whether to warn on non-fatal exceptions or raise an error.

gen_fit(X, y, P=None, refit=True)[source]¶

Generator for fitting learner on given data

class mlens.parallel.EvalTransformer(estimator, indexer=None, name=None, **kwargs)[source]¶

Bases: mlens.parallel.learner.Transformer

Evaluator version of the Transformer.

Derived class from Transformer adapted to cross-validated grid-search.
See Transformer for more details.

mlens.parallel.make_group(indexer, estimators, preprocessing, learner_kwargs=None, transformer_kwargs=None, name=None)[source]¶

Creating a Group from a set learners and transformers

Utility function for creating mapping a set of estimators and
preprocessing pipelines to a Group of
Learner and
Transformer instances.

<col class="field-name" />
<col class="field-body" />

Parameters:

indexer (instance or None, default = None) – Indexer instance to use. See :module:`mlens.index` for details.

estimators (dict of lists or list of estimators.) –
If preprocessing is None or list, estimators should
be a list. The list can either contain estimator instances,
named tuples of estimator instances, or a combination of both.

option_1 = [estimator_1, estimator_2]
option_2 = [("est-1", estimator_1), ("est-2", estimator_2)]
option_3 = [estimator_1, ("est-2", estimator_2)]

If different preprocessing pipelines are desired, a dictionary
that maps estimators to preprocessing pipelines must be passed.
The names of the estimator dictionary must correspond to the
names of the estimator dictionary.

preprocessing_cases = {"case-1": [trans_1, trans_2].
                       "case-2": [alt_trans_1, alt_trans_2]}

estimators = {"case-1": [est_a, est_b].
              "case-2": [est_c, est_d]}

The lists for each dictionary entry can be any of option_1,
option_2 and option_3.

preprocessing : dict of lists or list, optional (default = None)

preprocessing pipelines for given layer. If
the same preprocessing applies to all estimators, preprocessing
should be a list of transformer instances. The list can contain the
instances directly, named tuples of transformers,
or a combination of both.

option_1 = [transformer_1, transformer_2]
option_2 = [("trans-1", transformer_1),
            ("trans-2", transformer_2)]
option_3 = [transformer_1, ("trans-2", transformer_2)]

If different preprocessing pipelines are desired, a dictionary
that maps preprocessing pipelines must be passed. The names of the
preprocessing dictionary must correspond to the names of the
estimator dictionary.

preprocessing_cases = {"case-1": [trans_1, trans_2].
                       "case-2": [alt_trans_1, alt_trans_2]}

estimators = {"case-1": [est_a, est_b].
              "case-2": [est_c, est_d]}

The lists for each dictionary entry can be any of option_1,
option_2 and option_3.

transformer_kwargs : dict, optional

Keyword arguments to pass to the
Transformer instances.

learner_kwargs : dict, optional

Keyword arguments to pass to the
Learner instances.

name : str, optional

Name of group. Should be unique.

mlens.parallel.run(caller, job, X, y=None, map=True, **kwargs)[source]¶

Utility for running a ParallelProcessing job on a set of callers.

Run is a utility mapping for setting up a ParallelProcessing job and
executing across a set of callers. By default run executes:

out = mgr.map(caller, job, X, y, **kwargs)

run() handles temporary parameter changes, for instance running
a learner with proba=True that has proba=False as default.
Similarly, instances destined to not produce output can be forced to
yield predictions by passing return_preds=True as a keyword argument.

Note

To run a learner with a preprocessing dependency, the
instances need to be wrapped in a Group

run(Group(learner, transformer), 'predict', X, y)

<col class="field-name" />
<col class="field-body" />

Parameters:

caller (instance, list) – A runnable instance, or a list of instances.

job (str) – type of job to run. One of 'fit', 'transform', 'predict'.

X (array-like) – input

y (array-like, optional) – targets

map (bool (default=True)) – whether to run a ParallelProcessing.map() job. If False,
will instead run a ParallelProcessing.stack() job.

**kwargs (optional) – Keyword arguments. run() searches for
proba and return_preds to temporarily update callers to run
desired job and return desired output. Other kwargs are passed
to either map or stack.

mlens.parallel.get_backend(instance)[source]¶

Check whether backend exists and return

        © Copyright 2017, Sebastian Flennerhag.

  Built with Sphinx using a theme provided by Read the Docs.


text:
class EvtId;

class EvtBCVFF : public EvtSemiLeptonicFF {

public:

  EvtBCVFF(int idV, int fit);
  void getvectorff( EvtId parent, EvtId daught,
                       double t, double mass, double *a1f,
                       double *a2f, double *vf, double *a0f );

  void getscalarff(EvtId, EvtId, double, double, double*,
		   double*);

  void gettensorff(EvtId, EvtId, double, double, double*,
		   double*, double*, double*);

  void getbaryonff(EvtId, EvtId, double, double, double*,
		   double*, double*, double*);

  void getdiracff(EvtId, EvtId, double, double, double*, double*,
                  double*, double*, double*, double*);

  void getraritaff(EvtId, EvtId, double, double, double*, double*,
		   double*, double*, double*, double*, double*, double*);

private:

  int idVector, whichfit;

};

#endif

redpajama_set_name: RedPajamaGithub


text:
<?xml version="1.0" encoding="utf-8"?>
<labels>
	时间跟踪
	时间跟踪
	没有在跟踪时间
	需支付
	日期
	日期
	跟踪的时间
	可支付的时间
	评论
	时间跟踪
	时间预览
	时间跟踪
	总跟踪时间
	总需支付时间
	点击编辑此记录
	开始跟踪时间
	停止跟踪时间
	现在的任务
	显示项目任务
	已花费时间
	预计工作量
	点击跳到此任务
	点击跳到此项目
	退出时停止时间跟踪
</labels>


text:
<?xml version="1.0" encoding="utf-8"?>
<PreferenceScreen
    xmlns:android="http://schemas.android.com/apk/res/android">
    <PreferenceCategory android:title="Parental Settings">
        <EditTextPreference
            android:key="whitelist_entry"
            android:title="To whitelist"
            android:summary="Add what to whitelist"
            />

        <ListPreference
            android:key="whitelisted_stuff"
            android:title="Whitelisted"
            android:dialogTitle="Whitelisted web pages"
            android:summary="Webpages your child can see"
            />

        <EditTextPreference
            android:key="entertainment_limit"
            android:title="Entertainment limit"
            android:summary="How long can child use entertainment apps"
            />
    </PreferenceCategory>
</PreferenceScreen>


text:
"""Component to integrate ambilight for TVs exposing the Joint Space API."""
from __future__ import annotations

from typing import Any

from haphilipsjs import PhilipsTV
from haphilipsjs.typing import AmbilightCurrentConfiguration

from homeassistant import config_entries
from homeassistant.components.light import (
    ATTR_BRIGHTNESS,
    ATTR_EFFECT,
    ATTR_HS_COLOR,
    COLOR_MODE_HS,
    COLOR_MODE_ONOFF,
    SUPPORT_BRIGHTNESS,
    SUPPORT_COLOR,
    SUPPORT_EFFECT,
    LightEntity,
)
from homeassistant.core import HomeAssistant, callback
from homeassistant.helpers.update_coordinator import CoordinatorEntity
from homeassistant.util.color import color_hsv_to_RGB, color_RGB_to_hsv

from . import PhilipsTVDataUpdateCoordinator
from .const import CONF_SYSTEM, DOMAIN

EFFECT_PARTITION = ": "
EFFECT_MODE = "Mode"
EFFECT_EXPERT = "Expert"
EFFECT_AUTO = "Auto"
EFFECT_EXPERT_STYLES = {"FOLLOW_AUDIO", "FOLLOW_COLOR", "Lounge light"}

async def async_setup_entry(
    hass: HomeAssistant,
    config_entry: config_entries.ConfigEntry,
    async_add_entities,
):
    """Set up the configuration entry."""
    coordinator = hass.data[DOMAIN][config_entry.entry_id]
    async_add_entities(
        [
            PhilipsTVLightEntity(
                coordinator, config_entry.data[CONF_SYSTEM], config_entry.unique_id
            )
        ]
    )

def _get_settings(style: AmbilightCurrentConfiguration):
    """Extract the color settings data from a style."""
    if style["styleName"] in ("FOLLOW_COLOR", "Lounge light"):
        return style["colorSettings"]
    if style["styleName"] == "FOLLOW_AUDIO":
        return style["audioSettings"]
    return None

def _parse_effect(effect: str):
    style, _, algorithm = effect.partition(EFFECT_PARTITION)
    if style == EFFECT_MODE:
        return EFFECT_MODE, algorithm, None
    algorithm, _, expert = algorithm.partition(EFFECT_PARTITION)
    if expert:
        return EFFECT_EXPERT, style, algorithm
    return EFFECT_AUTO, style, algorithm

def _get_effect(mode: str, style: str, algorithm: str | None):
    if mode == EFFECT_MODE:
        return f"{EFFECT_MODE}{EFFECT_PARTITION}{style}"
    if mode == EFFECT_EXPERT:
        return f"{style}{EFFECT_PARTITION}{algorithm}{EFFECT_PARTITION}{EFFECT_EXPERT}"
    return f"{style}{EFFECT_PARTITION}{algorithm}"

def _is_on(mode, style, powerstate):
    if mode in (EFFECT_AUTO, EFFECT_EXPERT):
        if style in ("FOLLOW_VIDEO", "FOLLOW_AUDIO"):
            return powerstate in ("On", None)
        if style == "OFF":
            return False
        return True

    if mode == EFFECT_MODE:
        if style == "internal":
            return powerstate in ("On", None)
        return True

    return False

def _is_valid(mode, style):
    if mode == EFFECT_EXPERT:
        return style in EFFECT_EXPERT_STYLES
    return True

def _get_cache_keys(device: PhilipsTV):
    """Return a cache keys to avoid always updating."""
    return (
        device.on,
        device.powerstate,
        device.ambilight_current_configuration,
        device.ambilight_mode,
    )

def _average_pixels(data):
    """Calculate an average color over all ambilight pixels."""
    color_c = 0
    color_r = 0.0
    color_g = 0.0
    color_b = 0.0
    for layer in data.values():
        for side in layer.values():
            for pixel in side.values():
                color_c += 1
                color_r += pixel["r"]
                color_g += pixel["g"]
                color_b += pixel["b"]

    if color_c:
        color_r /= color_c
        color_g /= color_c
        color_b /= color_c
        return color_r, color_g, color_b
    return 0.0, 0.0, 0.0

class PhilipsTVLightEntity(CoordinatorEntity, LightEntity):
    """Representation of a Philips TV exposing the JointSpace API."""

    def __init__(
        self,
        coordinator: PhilipsTVDataUpdateCoordinator,
        system: dict[str, Any],
        unique_id: str,
    ) -> None:
        """Initialize light."""
        self._tv = coordinator.api
        self._hs = None
        self._brightness = None
        self._system = system
        self._coordinator = coordinator
        self._cache_keys = None
        super().__init__(coordinator)

        self._attr_supported_color_modes = [COLOR_MODE_HS, COLOR_MODE_ONOFF]
        self._attr_supported_features = (
            SUPPORT_EFFECT | SUPPORT_COLOR | SUPPORT_BRIGHTNESS
        )
        self._attr_name = self._system["name"]
        self._attr_unique_id = unique_id
        self._attr_icon = "mdi:television-ambient-light"
        self._attr_device_info = {
            "name": self._system["name"],
            "identifiers": {
                (DOMAIN, self._attr_unique_id),
            },
            "model": self._system.get("model"),
            "manufacturer": "Philips",
            "sw_version": self._system.get("softwareversion"),
        }

        self._update_from_coordinator()

    def _calculate_effect_list(self):
        """Calculate an effect list based on current status."""
        effects = []
        effects.extend(
            _get_effect(EFFECT_AUTO, style, setting)
            for style, data in self._tv.ambilight_styles.items()
            if _is_valid(EFFECT_AUTO, style)
            and _is_on(EFFECT_AUTO, style, self._tv.powerstate)
            for setting in data.get("menuSettings", [])
        )

        effects.extend(
            _get_effect(EFFECT_EXPERT, style, algorithm)
            for style, data in self._tv.ambilight_styles.items()
            if _is_valid(EFFECT_EXPERT, style)
            and _is_on(EFFECT_EXPERT, style, self._tv.powerstate)
            for algorithm in data.get("algorithms", [])
        )

        effects.extend(
            _get_effect(EFFECT_MODE, style, None)
            for style in self._tv.ambilight_modes
            if _is_valid(EFFECT_MODE, style)
            and _is_on(EFFECT_MODE, style, self._tv.powerstate)
        )

        return sorted(effects)

    def _calculate_effect(self):
        """Return the current effect."""
        current = self._tv.ambilight_current_configuration
        if current and self._tv.ambilight_mode != "manual":
            if current["isExpert"]:
                settings = _get_settings(current)
                if settings:
                    return _get_effect(
                        EFFECT_EXPERT, current["styleName"], settings["algorithm"]
                    )
                return _get_effect(EFFECT_EXPERT, current["styleName"], None)

            return _get_effect(
                EFFECT_AUTO, current["styleName"], current.get("menuSetting", None)
            )

        return _get_effect(EFFECT_MODE, self._tv.ambilight_mode, None)

    @property
    def color_mode(self):
        """Return the current color mode."""
        current = self._tv.ambilight_current_configuration
        if current and current["isExpert"]:
            return COLOR_MODE_HS

        if self._tv.ambilight_mode in ["manual", "expert"]:
            return COLOR_MODE_HS

        return COLOR_MODE_ONOFF

    @property
    def is_on(self):
        """Return if the light is turned on."""
        if self._tv.on:
            mode, style, _ = _parse_effect(self.effect)
            return _is_on(mode, style, self._tv.powerstate)

        return False

    def _update_from_coordinator(self):
        current = self._tv.ambilight_current_configuration
        color = None

        if (cache_keys := _get_cache_keys(self._tv)) != self._cache_keys:
            self._cache_keys = cache_keys
            self._attr_effect_list = self._calculate_effect_list()
            self._attr_effect = self._calculate_effect()

        if current and current["isExpert"]:
            if settings := _get_settings(current):
                color = settings["color"]

        mode, _, _ = _parse_effect(self._attr_effect)

        if mode == EFFECT_EXPERT and color:
            self._attr_hs_color = (
                color["hue"] * 360.0 / 255.0,
                color["saturation"] * 100.0 / 255.0,
            )
            self._attr_brightness = color["brightness"]
        elif mode == EFFECT_MODE and self._tv.ambilight_cached:
            hsv_h, hsv_s, hsv_v = color_RGB_to_hsv(
                *_average_pixels(self._tv.ambilight_cached)
            )
            self._attr_hs_color = hsv_h, hsv_s
            self._attr_brightness = hsv_v * 255.0 / 100.0
        else:
            self._attr_hs_color = None
            self._attr_brightness = None

    @callback
    def _handle_coordinator_update(self) -> None:
        """Handle updated data from the coordinator."""
        self._update_from_coordinator()
        super()._handle_coordinator_update()

    async def _set_ambilight_cached(self, algorithm, hs_color, brightness):
        """Set ambilight via the manual or expert mode."""
        rgb = color_hsv_to_RGB(hs_color[0], hs_color[1], brightness * 100 / 255)

        data = {
            "r": rgb[0],
            "g": rgb[1],
            "b": rgb[2],
        }

        if not await self._tv.setAmbilightCached(data):
            raise Exception("Failed to set ambilight color")

        if algorithm != self._tv.ambilight_mode:
            if not await self._tv.setAmbilightMode(algorithm):
                raise Exception("Failed to set ambilight mode")

    async def _set_ambilight_expert_config(
        self, style, algorithm, hs_color, brightness
    ):
        """Set ambilight via current configuration."""
        config: AmbilightCurrentConfiguration = {
            "styleName": style,
            "isExpert": True,
        }

        setting = {
            "algorithm": algorithm,
            "color": {
                "hue": round(hs_color[0] * 255.0 / 360.0),
                "saturation": round(hs_color[1] * 255.0 / 100.0),
                "brightness": round(brightness),
            },
            "colorDelta": {
                "hue": 0,
                "saturation": 0,
                "brightness": 0,
            },
        }

        if style in ("FOLLOW_COLOR", "Lounge light"):
            config["colorSettings"] = setting
            config["speed"] = 2

        elif style == "FOLLOW_AUDIO":
            config["audioSettings"] = setting
            config["tuning"] = 0

        if not await self._tv.setAmbilightCurrentConfiguration(config):
            raise Exception("Failed to set ambilight mode")

    async def _set_ambilight_config(self, style, algorithm):
        """Set ambilight via current configuration."""
        config: AmbilightCurrentConfiguration = {
            "styleName": style,
            "isExpert": False,
            "menuSetting": algorithm,
        }

        if await self._tv.setAmbilightCurrentConfiguration(config) is False:
            raise Exception("Failed to set ambilight mode")

    async def async_turn_on(self, **kwargs) -> None:
        """Turn the bulb on."""
        brightness = kwargs.get(ATTR_BRIGHTNESS, self.brightness)
        hs_color = kwargs.get(ATTR_HS_COLOR, self.hs_color)
        effect = kwargs.get(ATTR_EFFECT, self.effect)

        if not self._tv.on:
            raise Exception("TV is not available")

        mode, style, setting = _parse_effect(effect)

        if not _is_on(mode, style, self._tv.powerstate):
            mode = EFFECT_MODE
            setting = None
            if self._tv.powerstate in ("On", None):
                style = "internal"
            else:
                style = "manual"

        if brightness is None:
            brightness = 255

        if hs_color is None:
            hs_color = [0, 0]

        if mode == EFFECT_MODE:
            await self._set_ambilight_cached(style, hs_color, brightness)
        elif mode == EFFECT_AUTO:
            await self._set_ambilight_config(style, setting)
        elif mode == EFFECT_EXPERT:
            await self._set_ambilight_expert_config(
                style, setting, hs_color, brightness
            )

        self._update_from_coordinator()
        self.async_write_ha_state()

    async def async_turn_off(self, **kwargs) -> None:
        """Turn of ambilight."""

        if not self._tv.on:
            raise Exception("TV is not available")

        if await self._tv.setAmbilightMode("internal") is False:
            raise Exception("Failed to set ambilight mode")

        await self._set_ambilight_config("OFF", "")

        self._update_from_coordinator()
        self.async_write_ha_state()


text:
static mk_context_t*
__mk_memory_map_get_context(mk_type_ref self)
{ return ((struct mk_memory_map_s*)self)->context; }

//|++++++++++++++++++++++++++++++++++++|//
static mk_error_t
__mk_memory_map_init_object(struct mk_memory_map_s* self, mk_vm_offset_t offset, mk_vm_address_t address, mk_vm_size_t length, bool require_full, mk_memory_object_t* memory_object)
{
#pragma unused (self)
#pragma unused (offset)
#pragma unused (address)
#pragma unused (length)
#pragma unused (require_full)
#pragma unused (memory_object)
    fprintf(stderr, "No default implementation of __mk_memory_map_init_object.");
    __builtin_trap();
}

//|++++++++++++++++++++++++++++++++++++|//
static void
__mk_memory_map_free_object(struct mk_memory_map_s* self, mk_memory_object_t* memory_object)
{
#pragma unused (self)
#pragma unused (memory_object)
    fprintf(stderr, "No default implementation of __mk_memory_map_free_object.");
    __builtin_trap();
}

//|++++++++++++++++++++++++++++++++++++|//
static bool
__mk_memory_map_has_mapping(struct mk_memory_map_s* self, mk_vm_offset_t offset, mk_vm_address_t address, mk_vm_size_t length, mk_error_t* error)
{
    mk_error_t err;
    mk_memory_object_t memory_object;

    if ((err = mk_memory_map_init_object(self, offset, address, length, false, &memory_object))) {
        MK_ERROR_OUT = err;
        return false;
    }

    bool retValue = mk_memory_object_length(&memory_object) > length;

    mk_memory_map_free_object(self, &memory_object);
    return retValue;
}

//|++++++++++++++++++++++++++++++++++++|//
static vm_size_t
__mk_memory_map_copy_bytes(struct mk_memory_map_s* self, mk_vm_offset_t offset, mk_vm_address_t address, void* buffer, mk_vm_size_t length, bool require_full, mk_error_t* error)
{
    mk_error_t err;
    mk_memory_object_t memory_object;

    if ((err = mk_memory_map_init_object(self, offset, address, length, require_full, &memory_object))) {
        MK_ERROR_OUT = err;
        return false;
    }

    vm_size_t mappingLength = mk_memory_object_length(&memory_object);
    memcpy(buffer, (void*)mk_memory_object_address(&memory_object), MIN(length, (mk_vm_size_t)mappingLength));

    mk_memory_map_free_object(self, &memory_object);
    return (vm_size_t)MIN(length, (mk_vm_size_t)mappingLength);
}

//|++++++++++++++++++++++++++++++++++++|//
static uint8_t
__mk_memory_map_read_byte(struct mk_memory_map_s* self, mk_vm_offset_t offset, mk_vm_address_t address, mk_data_model_ref data_model, mk_error_t* error)
{
#pragma unused (data_model)
    uint8_t retValue;
    vm_size_t readSize;

    readSize = mk_memory_map_copy_bytes(self, offset, address, &retValue, sizeof(uint8_t), true, error);
    if (readSize < sizeof(uint8_t))
        return 0;

    return retValue;
}

//|++++++++++++++++++++++++++++++++++++|//
static uint16_t
__mk_memory_map_read_word(struct mk_memory_map_s* self, mk_vm_offset_t offset, mk_vm_address_t address, mk_data_model_ref data_model, mk_error_t* error)
{
    uint16_t retValue;
    vm_size_t readSize;

    readSize = mk_memory_map_copy_bytes(self, offset, address, &retValue, sizeof(uint16_t), true, error);
    if (readSize < sizeof(uint16_t))
        return 0;

    if (data_model.data_model)
        return mk_data_model_get_byte_order(data_model)->swap16( retValue );
    else
        return retValue;
}

//|++++++++++++++++++++++++++++++++++++|//
static uint32_t
__mk_memory_map_read_dword(struct mk_memory_map_s* self, mk_vm_offset_t offset, mk_vm_address_t address, mk_data_model_ref data_model, mk_error_t* error)
{
    uint32_t retValue;
    vm_size_t readSize;

    readSize = mk_memory_map_copy_bytes(self, offset, address, &retValue, sizeof(uint32_t), true, error);
    if (readSize < sizeof(uint32_t))
        return 0;

    if (data_model.data_model)
        return mk_data_model_get_byte_order(data_model)->swap32( retValue );
    else
        return retValue;
}

//|++++++++++++++++++++++++++++++++++++|//
static uint64_t
__mk_memory_map_read_qword(struct mk_memory_map_s* self, mk_vm_offset_t offset, mk_vm_address_t address, mk_data_model_ref data_model, mk_error_t* error)
{
    uint64_t retValue;
    vm_size_t readSize;

    readSize = mk_memory_map_copy_bytes(self, offset, address, &retValue, sizeof(uint64_t), true, error);
    if (readSize < sizeof(uint64_t))
        return 0;

    if (data_model.data_model)
        return mk_data_model_get_byte_order(data_model)->swap64( retValue );
    else
        return retValue;
}

const struct _mk_memory_map_vtable _mk_memory_map_class = {
    .base.super                 = &_mk_type_class,
    .base.name                  = "memory_map",
    .base.get_context           = &__mk_memory_map_get_context,
    .init_object                = &__mk_memory_map_init_object,
    .free_object                = &__mk_memory_map_free_object,
    .has_mapping                = &__mk_memory_map_has_mapping,
    .copy_bytes                 = &__mk_memory_map_copy_bytes,
    .read_byte                  = &__mk_memory_map_read_byte,
    .read_word                  = &__mk_memory_map_read_word,
    .read_dword                 = &__mk_memory_map_read_dword,
    .read_qword                 = &__mk_memory_map_read_qword
};

intptr_t mk_memory_map_type = (intptr_t)&_mk_memory_map_class;

//----------------------------------------------------------------------------//
#pragma mark -  Static Methods
//----------------------------------------------------------------------------//

//|++++++++++++++++++++++++++++++++++++|//
mk_memory_map_ref mk_memory_map_for_object(mk_memory_object_ref mobj)
{ return (mk_memory_map_ref)(struct mk_memory_map_s*)mobj.memory_object->mapping; }

//----------------------------------------------------------------------------//
#pragma mark -  Instance Methods
//----------------------------------------------------------------------------//

//|++++++++++++++++++++++++++++++++++++|//
mk_error_t mk_memory_map_init_object(mk_memory_map_ref map, mk_vm_offset_t offset, mk_vm_address_t address, mk_vm_size_t length, bool require_full, mk_memory_object_t* memory_object)
{ MK_TYPE_INVOKE(map, memory_map, init_object)(map, offset, address, length, require_full, memory_object); }

//|++++++++++++++++++++++++++++++++++++|//
void mk_memory_map_free_object(mk_memory_map_ref map, mk_memory_object_t* memory_object)
{ MK_TYPE_INVOKE(map, memory_map, free_object)(map, memory_object); }

//|++++++++++++++++++++++++++++++++++++|//
bool mk_memory_map_has_mapping(mk_memory_map_ref map, mk_vm_offset_t offset, mk_vm_address_t address, mk_vm_size_t length, mk_error_t* error)
{ MK_TYPE_INVOKE(map, memory_map, has_mapping)(map, offset, address, length, error); }

//|++++++++++++++++++++++++++++++++++++|//
vm_size_t mk_memory_map_copy_bytes(mk_memory_map_ref map, mk_vm_offset_t offset, mk_vm_address_t address, void* buffer, mk_vm_size_t length, bool require_full, mk_error_t* error)
{ MK_TYPE_INVOKE(map, memory_map, copy_bytes)(map, offset, address, buffer, length, require_full, error); }

//|++++++++++++++++++++++++++++++++++++|//
uint8_t mk_memory_map_read_byte(mk_memory_map_ref map, mk_vm_offset_t offset, mk_vm_address_t address, mk_data_model_ref data_model, mk_error_t* error)
{ MK_TYPE_INVOKE(map, memory_map, read_byte)(map, offset, address, data_model, error); }

//|++++++++++++++++++++++++++++++++++++|//
uint16_t mk_memory_map_read_word(mk_memory_map_ref map, mk_vm_offset_t offset, mk_vm_address_t address, mk_data_model_ref data_model, mk_error_t* error)
{ MK_TYPE_INVOKE(map, memory_map, read_word)(map, offset, address, data_model, error); }

//|++++++++++++++++++++++++++++++++++++|//
uint32_t mk_memory_map_read_dword(mk_memory_map_ref map, mk_vm_offset_t offset, mk_vm_address_t address, mk_data_model_ref data_model, mk_error_t* error)
{ MK_TYPE_INVOKE(map, memory_map, read_dword)(map, offset, address, data_model, error); }

//|++++++++++++++++++++++++++++++++++++|//
uint64_t mk_memory_map_read_qword(mk_memory_map_ref map, mk_vm_offset_t offset, mk_vm_address_t address, mk_data_model_ref data_model, mk_error_t* error)
{ MK_TYPE_INVOKE(map, memory_map, read_qword)(map, offset, address, data_model, error); }