part 1.5 - Move XRE_GetBinaryPath code in a dedicated class.
part 2 - Load the XUL library with the standalone glue and get rid of the startup shell script.
part 1 - Move DLL blocklist in XRE, and inline NS_SetDllDirectory and environment sanitization, which now needs to be called manually.
1) If all the shared libraries installed by an --enable-libxul build have sufficiently stable public ABIs, we could give them SONAMEs, install them in /usr/lib, install firefox-bin etc. as just plain 'firefox' etc., and throw the wrapper scripts away in an installed configuration.  (We would still need a wrapper script for running out of the build tree, but that's not the case we care about for performance, and that script could be much simpler.)  The SONAMEs we assigned would presumably involve the Gecko version -- it seems to me that we do make pretty strong binary compatibility guarantees within one Gecko version.
That would have the advantage of not needing the shell script even when running out of the build directory, and not having to worry about expanded binary compatibility constraints, but the disadvantage of maybe not working universally.
#3 is worth looking into (I think there's already a bug, but we can use this one). I think that would drop support for older versions of ld.so, so we should figure out exactly when it was added and which Linux versions we'd be breaking.
Although I'm surprised that, if we fix the arithmetic in bug 550310, we'd still be seing a lot of overhead on Linux systems: typically execing a single process is very lightweight. Do we have numbers measuring the actual cost?
> figure out exactly when it was added and which Linux versions we'd be breaking.
According to glibc's git, version 2.0.96, released in 1998.
That seems like it should just get fixed.
> is very lightweight. Do we have numbers measuring the actual cost?
So that's less than two milliseconds of overhead even with the heavyweight shell.
It seems $ORIGIN is what you want to use.
these because their handling is pretty inconvenient.
We currently have a stable method of doing binary linking (using XPCOM glue dynamic linking). This allows binary extensions to link against libxpcom.so and be compatible with multiple major versions if they are written to do so. Using a versioning scheme which changes the soname at each major release would make this difficult/impossible, and would increase the likelihood of loading two versions of XPCOM in the same process, which is obviously bad.
Isn't the firefox-bin on mozilla release builds using the standalone glue under the hood ? As such, isn't loading libxpcom.so from the gredir ? Then, the nspr and nss libs are loaded by being listed in dependentlibs.list. Then what exactly is LD_LIBRARY_PATH supposed to add ?
No, mozilla release builds don't use the standalone glue/xulrunner stub (they aren't firefox+xulrunner, just an all-in-one build).
I have a patch in bug 557027 that fixes the Fennec build to use -rpath=$ORIGIN and it works quite nicely on my Ubuntu system. We should just do the same thing for the Firefox build and ditch the wrapper script.
FWIW I just did some experiments with a program that lets you change the rpath after the fact (it's called 'patchelf' and it's a little buggy, but not too much for experiments) and it appears that we would need -rpath=$ORIGIN:$ORIGIN/.. for every .so in components/, as well as -rpath=$ORIGIN for the main executable.  I'm not sure.  The difference between DT_RPATH and DT_RUNPATH (which I have not been able to find clear documentation on) may be coming into play here.
Nonetheless, I'm all for this approach.  I would like to know what the implications for a firefox-atop-xulrunner build are, though, since that seems to be how the distros do it.
> versions of XPCOM in the same process, which is obviously bad.
If it is possible for binary extensions to link against libxpcom.so and be compatible with multiple major versions, then the hypothetical SONAME doesn't need bumping as often as every major release, ne?  I was just being conservative.
> are the -g and --debugger options.
As I understand it, the -g/--debugger mode of the script is only necessary because the script exists in the first place -- if there is no script, I can just do "gdb --args ./dist/bin/firefox file://whatever" and it'll work.  Yes/no?
I am completely unfamiliar with dsp wrappers.  Is there something we could be doing internal to the executable that would obviate the need for them?
Do you think it makes sense to merge that into early app startup?  Wouldn't be difficult.
> on) may be coming into play here.
You shouldn't need any rpath on components.
> to be how the distros do it.
Much less of a problem, since the standalone glue doesn't need rpath at all.
> just do "gdb --args ./dist/bin/firefox file://whatever" and it'll work.
> doing internal to the executable that would obviate the need for them?
dsp wrappers redirect /dev/dsp to the corresponding audio daemon (esd, artsd, pulseaudio, etc.) or in the case of aoss, redirects it on alsa output. I'm not entirely sure this is necessary nowadays for firefox itself, but I guess this is still the case for the main contender: the flash plugin. One of the problems firefox has, and which got worse with the <video> support is that it has several ways it can interact with audio hardware: through libcanberra or libesd from nsISound, through alsa from libsoundfish, and through whatever a plugin might use.
I'll throw together a candidate patch and we can see what it does.
This should be most of what we need. I guess we'll need a similar fix for Solaris. I know rpath=$ORIGIN is supported there, but I don't know the specifics. I have no idea what BSDs support in this regard.
It's overkill to a) have everything copied over to the dist directory and b) have all binaries and library include that rpath.
IMHO, it would be better to have that rpath added to the firefox binary and probably libxul.so, too. But only these two.
Oops, I intended to do this only |ifdef PROGRAM| so that only binaries would get it. Guess I forgot to add that to the patch.
IIRC how rpath works, |ifdef PROGRAM| won't be enough, because libxul.so needs to have an rpath too ; except if you link all the libraries to the firefox binary.
We're shipping Fennec on Maemo with this exact configuration, and it works fine.
Confirmed, I just checked, and it works with rpath on the program only. I must have been confused with something else.
Note, however, that the change in js/src would make the js shell also have an rpath, which may not be entirely expected, especially on standalone builds of the js engine.
I don't think rpath=$ORIGIN is any worse than anything else. (And the JS shell is more of a testing tool than anything end-users would really use.) We keep config/rules.mk and js/src/config/rules.mk in sync just to preserve our sanity.
I don't know if it's *supposed* to be this way, but I've had to use run-mozilla.sh to invoke the js shell in the past, because it wanted some library from dist/lib that it wasn't getting.
Just a small note on the binary name.
One of the reasons I'm still providing a firefox-bin executable is that historically, on unix, because of the startup script, some desktop environments would pick firefox-bin as the name of the application they need to restart (in desktop session management, most of the time). I never checked if that was still a concern, but if that is, a symlink could be helpful.
> desktop session management, most of the time).
I've been sort of thinking of tackling a GTK+3 update project.  Innn my copious free time.
Can we run dist/bin/firefox-bin with RPATH on Linux?
It doesn't work on Solaris because RPATH=$ORIGIN doesn't follow symbolic links.
I agree with Mike Hommey, we should at least leave firefox-bin as a symbolic link.
Adding a firefox-bin symlink should be possible. It might be a little tricky because during packaging we explicitly dereference symlinks, but I can probably work something out.
The dynamic loader on Linux (at least) also understands $LIB, which expands to lib or lib64 according to whether it's used in a32- or 64-bi executable.  But, as I write this, I realize that is of more use to LD_LIBRARY_PATH setting (for a script that may call 32- or 64-bit binaries along the way) than an RPATH setting (which is obviously in a fixed-binary setting - that of the executable containing it).
My understanding of the difference is based on how Solaris used to document it.  One of them, RPATH, was used before LD_LIBRARY_PATH (and hence could not be overridden by it) and the other, RUNPATH, was used after.  At one point (last century) I'm sure I had Solaris executables with *only* RPATH set, that was non-overridable.  Can't reproduce it now, though.  Still have the RPATH-only executables, but LD_LIBRARY_PATH overrides it.
I just remembered an issue we hit a while ago with the wrapper script gone: other gecko application with the wrapper script broke the application without.
For example, you have both thunderbird and firefox. You run thunderbird with the wrapper script and firefox without. This means thunderbird sets MOZILLA_FIVE_HOME. When firefox is not already opened and the user opens an url link from thunderbird with firefox, firefox starts with MOZILLA_FIVE_HOME set and tries to use it.
I don't know if that still applies, but I thought it was worth mentioning.
Ahhh - another example of global names for app-specific values causing problems (LD_LIBRARY_PATH being the Prime Example).
If Firefox is built to start without a script why would it look for MOZILLA_FIVE_HOME at all?  And if it wants to provide a variable for debug/test then it should use something specific - such as MOZILLA_FIREFOX_HOME.
(and Thunderbird could use MOZILLA_THUNDERBIRD_HOME - you can easily set two variables to have the same value, but you can't set one variable to two different values at the same time).
This is a proof of concept of replacing the firefox binary with one using the standalone glue to load libxul.so. This method has two advantages: we don't need to rely on rpath, and it allows to more easily hook things such as bug 632404.
This patch works on Mac and Linux, though on Mac tests fail because of a script doing "find . -maxdepth 4 -type f -name firefox-bin", which fails because firefox-bin, while kept for compatibility, is a symbolic link.
This is related to the DLL blacklisting. I wonder if we could initialize it in XRE_main instead of wmain.
Last but not least, I do think the GetBinaryPath function, that is mostly a copy of XRE_GetBinaryPath from toolkit/xre/nsAppRunner.cpp should be put as a helper function with a proper API in the XPCOM standalone glue.
Same one, but against the build-system branch, and with the leaks fixed.
I do understand that we want to get rid of the glue and the xpcom stub, but the standalone glue currently is the most convenient way to do this. I think we can still remove the glue later, by creating a better API for this dynamic loading type of embedding, which really doesn't need to be very elaborated, but can be a separate bug.
On linux the binary will still be firefox-bin, but on mac it will be firefox?
I really don't like copying the binary-path code like that. Can we move it into a .h-inl file which is included from the couple different places it is needed, or maybe move it into the glue code?
> On linux the binary will still be firefox-bin, but on mac it will be firefox?
It's firefox everywhere, with a compatibility symbolic link kept on linux and mac.
> needed, or maybe move it into the glue code?
Note that we'll need to revert bug 627591 first, but this will be replaced with bug 632404 landing at the same time.
of using its own copy of GetBinaryPath.
I want ehsan to look at this also.
Under the new setup, how is wmain executed?  (I need to know who calls it, and which processes does it get run in).
Also, what is the purpose of this patch?  Is it just refactoring stuff?
wmain is still the first function executed when the program starts (modulo static initializers). That doesn't change. AIUI, all processes run it.
> static initializers). That doesn't change. AIUI, all processes run it.
this means both the main process and plugin-container run it.
wmain is the app entry point on Windows: it takes unicode arguments (WCHAR) and converts them to UTF8 so that we can be fully unicode-aware on Windows.
> and converts them to UTF8 so that we can be fully unicode-aware on Windows.
Yeah, I was just checking to make sure that we're not changing that by passing something else besides -ENTRY:wmainCRTStarup to the linker with these patches.
We don't want NS_SetDllDirectory or SanitizeEnvironmentVariables to be called as plugin process startup <http://mxr.mozilla.org/mozilla-central/source/ipc/app/MozillaRuntimeMain.cpp#52>.  But we _do_ want SanitizeEnvironmentVariables to be called the first time NS_SetDllDirectory is called.  Which makes this patch wrong.
> first time NS_SetDllDirectory is called.  Which makes this patch wrong.
Mmmmm... How exactly does the following not do what you want?
As I said to glandium on IRC, I think we should move SanitizeEnvironmentVariables calls to the callers of NS_SetDllDirectory.
This merges part 0.5 and part 1, because that was going to be messy.
This implements what we discussed on IRC.
New Try push for the new patch queue.
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/mh@glandium.org-72cc4fe65a0d/ is the link to the build in case anyone testing needs it.
It took a little while, but I made sure to test all the bugs Ehsan mentioned in the private comment, with the exception of the last one which I couldn't figure out how to check. I found no regressions. Hopefully that's sufficient.
Just to be complete, I used the try builds in comment #65 on the OSs in the bugs were reported on.
I tested the remaining one with success (that is, the try build does the right thing).
Tentatively try someone else. Mike or Ted, would you have some time to review this? I'd like to land it this week-end.
Not qualified to do this one, sorry. Kyle maybe? Taras?
r+ for the C++ bits. This is mostly a trickier platform-independent equivalent of what I did with windows linker switches in bug 627591.
I love this so much. Does this need to be fixed separately for other apps? I know I already fixed this for Fennec on Linux a while back, but presumably we ought to fix Thunderbird and maybe XULRunner.
On Mac, does this survive as a symlink when we do all the silly rsync'ing to the app bundle?
So awful, but nice hack.
> we ought to fix Thunderbird and maybe XULRunner.
I was planning to file a followup bug to have a generic app launcher, actually. There's no reason why we need to continue to duplicate that initialization code in mobile/app, xulrunner/app, suite/app, mail/app, and other places.
That wouldn't be safe until we fix other applications, I believe.
It looks like so, as it is a symlink in the .dmg archive.
> It looks like so, as it is a symlink in the .dmg archive.
I think the best short term option would be to copy firefox to firefox-bin (it's only a few dozens kilobytes).
> That wouldn't be safe until we fix other applications, I believe.
Okay, make sure you remember this bit in the followup.
> > It looks like so, as it is a symlink in the .dmg archive.
> (it's only a few dozens kilobytes).
Okay, I can deal with that.
So, what went wrong is that while most stuff work, a few of our leak reporting is happening with atexit(). Except that at the time the process exits, the function that was registered is nowhere, since libxul is unloaded. This is why, e.g. we don't unload gconf in the system pref service.
Interestingly, this only affected linux debug builds.
The chromium code comes with an AtExitManager which would allow to avoid this problem.
> Interestingly, this only affected linux debug builds.
And this is as such because on top of the atexit problem, the malloc/free/... wrappers from nsTraceMalloc.c are not used.
After this fix, I have to export NSDISTMODE=copy to compile firefox, otherwise it won't work.
This is unlikely to be the same thing as comment #31. Please file a followup bug with the error you get when you build without NSDISTMODE=copy.
This looks like it broke running with -g, unfortunately.  Was that an expected outcome?
Yes, you either run it through run-mozilla.sh, or, like any other program, just run with gdb --args. We need to update the wiki.
Given that this is a startup performance improvement, would it make any sense to allow _debug_ builds to continue using the old setup?  It's not like we're worried about startup perf there.
We could stick code at the very beginning of main (or possibly XRE_main, not sure) that scans argv for "-g" and "-d" and if it finds them, splices them out and execs the debugger.
> out and execs the debugger.
That'd be main(), but I'm not a big fan of keeping that legacy option. Maybe we should bring this up on dev-platform?


Due to the impact of the human way of life, which has largely turned its back on natural conditions and has subjected Nature to its own needs, earthbound rhythms and cycles have been thrown into disorder.
Nature itself has more than abundantly proved how electromagnetic fields in the vicinity of power lines can exert a great effect on the immediate environment.
It has been known for a long time, and not only in naturopathic circles, that chronic illnesses, lack of energy, are related to or even directly retraceable to radiation exposure and electromagnetic pollution; even modern medicine has begun increasingly to confirm this.


text:
A graphical heuristic for reduction and partitioning of large datasets for scalable supervised training
Sumedh Yadav ORCID: orcid.org/0000-0001-5630-356X1 &
Mathis Bode2
Journal of Big Data volume 6, Article number: 96 (2019) Cite this article
A scalable graphical method is presented for selecting and partitioning datasets for the training phase of a classification task. For the heuristic, a clustering algorithm is required to get its computation cost in a reasonable proportion to the task itself. This step is succeeded by construction of an information graph of the underlying classification patterns using approximate nearest neighbor methods. The presented method consists of two approaches, one for reducing a given training set, and another for partitioning the selected/reduced set. The heuristic targets large datasets, since the primary goal is a significant reduction in training computation run-time without compromising prediction accuracy. Test results show that both approaches significantly speed-up the training task when compared against that of state-of-the-art shrinking heuristics available in LIBSVM. Furthermore, the approaches closely follow or even outperform in prediction accuracy. A network design is also presented for a partitioning based distributed training formulation. Added speed-up in training run-time is observed when compared to that of serial implementation of the approaches.
Two decades earlier, some of the most seminal works in machine learning were done on training set selection [1, 2] under the banner of relevance reasoning. However, the better part of recent works have been exclusively towards feature selection [3, 4]. With increased processing power, run time of training is feasible even for datasets erstwhile considered large. Additionally, dimensionality (d) dominates dataset size (n) in the algorithmic complexities of learning algorithms. In the training phase, less data points mean fewer generalization guarantees, however, as we are moving in the era of big data, even the fastest classification algorithms are taking un-feasible time to train models. When data sources are abundant, it is befitting to separate data based on relevance to the learning task. This has led to a renewed interest in the once famous problem statement of relevance reasoning [5, 6]. Reasoning on relevance to get improved scalability of classification algorithms is currently explored on graphical/network data [7] and learned models [8].
One research area where training set selection has been given attention to is support vector machines (SVM). Generally, these selection methods can be divided into two types, primarily based on whether n is reduced or not. The first type of methods aims to modify the SVM formulation so that it can be applied to large datasets. Many approaches have worked successfully in the past, including sequential minimal optimization (SMO) [9, 10], which is an algorithm of solving the optimization problem associated with SVM in an efficient manner and genetic programming [11]. The first type of methods, however, do not benefit from reducing the size of training data because they only deal with data handling. Reduction of data size is the quintessential advantage of the second kind of methods [12,13,14,15,16,17,18,19,20]. These methods focus on segregating the data points on relevance to the classification task. Computation time is reduced by actively reducing n. During the mid-2000s, handling data was the central theme of research instead of reducing it. Despite the apparent advantage of reduction of data, researchers made efforts for methods of the first type [9,10,11, 21] in comparison to the latter type. For instance, formulations implemented in LIBSVM [21], which is a widely used benchmark library for SVM methods, are of the first type of methods.
The few existing works of the second type are limited in one respect or another. For instance, speed-ups upto 15 times (for 0.5% sampling) are possible on clustering based SVM (CB-SVM) works by Yang et al. [20] Salvador, Awad [12, 13], however, these are limited to linear kernels only. A geometric approach of minimum ball enclosing by Cervantes et al. [14] requires two stages of SVM training, first on the cluster centers, and later on the de-clustered data. A similar method by Li et al. [15] suffers from a random selection of data. They have reported 92.0% prediction accuracy for a separable dataset for which LIBSVM reproduces 99.9%.
The presented approach also falls under the second type. As will be shown, the presented selection scheme is very deterministic in prediction accuracy. A model trained with the presented heuristic results in close or better prediction accuracy than that of full data training. One recurring problem with the second type of methods is inefficient space searching scalability [15,16,17,18,19]. It becomes worse for high dimensional data, where the heuristic takes more time than training itself. For instance, the training phase with the method proposed by Chau et al. [22] is 8 times slower compared to the original training of LIBSVM for a dataset with only nine features. On the contrary, a speed-up of about 10 times is demonstrated here for a dataset with a number of features more than 25,000. Inefficient space search is addressed by the use of state-of-the-art approximate nearest neighbor (ANN) methods [23] which are highly scalable.
In LIBSVM, the SMO decomposition method of Fan et al. [10] is available for the classification task. Training set selection compares in principle to working set selection (WSS) in the context of SMO or similar decomposition methods. Furthermore, a shrinking technique is available for these formulations to remove bounded components during iterations, effectively reducing the optimization problem [10]. As will be shown, data selection in the presented heuristic only depends on the underlying classification patterns, giving it an essential advantage of generic applicability to the majority of classification algorithms, including the SMO formulations of LIBSVM. For these reasons, the state-of-the-art shrinking heuristic of LIBSVM is compared to the presented heuristic.
The presented heuristic augments clustering based approaches [12, 13, 20] by constructing an (approximate) information graph out of the clustered data. This graph acts as a proxy for reducing the training set. A novel edge weight scheme captures the underlying classification patterns in the graph. The graph is then pruned via filtering on the edge weights to select a relevant dataset that can be used for the training task. Furthermore, a graph coarsening approach is presented to break the selected/reduced set into further partitions that are independently available for training, leading to an approximate learning scheme. Both methods lead to a reduction in the number of training data points, which reduces complexity of the training algorithm, giving performance advantages.
Most of the existing methods of both types are limited to the SVM class of algorithms [9,10,11,12,13,14,15,16,17,18,19,20,21]. Generic applicability on a majority of classification algorithms is another advantage of the presented heuristic. It gives an opportunity to use the heuristic as a pre-processing tool, separate from the classification algorithm. Since the data points are selected based on their relevance to the classification task, the resulting reduced training set is much more balanced in size across the target classes. In other words, the formulation addresses the problem statement of class imbalance, which is a topic of current research in big data [24].
The remainder of this paper is organized as follows, the 'Methodology' section describes formulations of the proposed heuristic in detail. The heuristic is evaluated with a number of tests and datasets in the 'Results' section. Finally, the 'Conclusions and future work' section summarizes concluding remarks, and ideas for future work in the heuristic formulation.
The heuristic procedure organically divides into the following steps:
Step 1: Clustering step
Step 2: Graph knitting scheme
Step 3: Graph shedding scheme
Step 4: Graph clubbing scheme
Pre-processor for the testing phase
The training phase proceeds after the first four steps, whereas the testing phase follows the last step of the heuristic formulation. The clustering step is used to get a computationally feasible resolution of the underlying data. A weighted graph is constructed next in the graph knitting scheme, using a three parts algorithm and an edge weight scheme, which completely captures the classification patterns. Significant nodes of this graph with respect to the classification task are determined in the graph shedding step. The reduced dataset obtained after the graph shedding step is executed several times faster compared to the original dataset when used for the training phase. Finally, the graph clubbing step divides these nodes into partitions that can be trained independently using a directional aspect of the graph coarsening objective achieved via another three parts algorithm. The graph clubbing step results in a partitioned dataset which can be used in two execution modes for the training phase, first in serial mode, and second in distributed mode. Due to an approximate scheme (explained later in the section 'Graph clubbing scheme'), the serial execution is faster than that of the training on the reduced dataset. Distributed execution can be used to leverage worker threads/processes to get multiple times speed-up. Figure 1 summarizes the steps of the heuristic along with the resulting modifications on the dataset.
Step 1 to 4 of the heuristic/procedure
Because of the multiple data partitions which translate into as many classifiers, there is a need to determine which classifier to choose for testing a data point. This is achieved by the pre-processor for the testing phase. Lastly, a network application is designed which distributes the obtained partitions in a load-balancing and a communication-free manner.
From a computational point of view, the run-time profile of the first four steps for a typical run case (\(n \approx 100\hbox {k}, d = 2, n_c \approx 350\)) is,
Clustering step takes \(\approx 750\,\hbox {ms}\) or > 98%
Graph knitting scheme takes \(\approx 10\,\hbox {ms}\) or 1%
Graph shedding and graph clubbing schemes take \(\approx 2\,\hbox {ms}\) (\(< 1\%\))
On the other hand, the pre-processor for the testing phase takes \(\approx \) 5% of the testing phase time. The clustering step is predominant over subsequent steps with a run-time complexity in n, whereas that of all the other steps is in the order of number of clusters (\(n_{c}\)). The input parameter to the clustering step, \(n_{c}\), controls the granularity of data representation. Typically, the ratio of n to \(n_{c}\), which is also known as nominal VC dimension, is 10 to 300, explaining the run-time dominance of the clustering step. The graph knitting scheme becomes the most computation-intensive among the subsequent steps, because it involves heavy space searching. It is to be noted that the run-time percentage profile can vary a lot depending on the nominal VC dimension or the input parameter \(n_{c}.\).
Clustering step
The clustering step is used to lower the resolution of the underlying data. In principle, the presented heuristic does not require this step. However, it is not computationally feasible to execute the subsequent steps with a run-time complexity in n instead of \(n_{c}\). Additionally, it will also affect the generic applicability of the heuristic, which is discussed later in the 'Graph shedding scheme' step.
The step consists of a standard K-means++ [25] clustering algorithm, and a metric to store classification patterns of the original data. K-means++ provides improved initial seeding of clusters over the traditional K-means method. This choice leads to running the clustering algorithm for a nominal number of iterations, typically 5. In the current implementation, every cluster center maintains the target class through the weighted average calculation over all its data points. However, advanced metrics can be constructed to unearth more characteristics of patterns from the clustered data.
Although this step serves only for coarsening the data representation, it dominates the computation cost of the first four steps. Two state-of-the-art K-means++ implementations were tested, K-MeansRex [26] and the scalable mlpack package [27]. For a test run with data points in the range of 1 to 100k (\(d = 2, n_c = 100\)), K-means++ from mlpack was 1.82 times faster on average in execution than K-MeansRex's implementation. Therefore, K-means++ from mlpack is chosen as the standard clustering algorithm in this work.
Given the vast research literature available for clustering methods, there are more advanced implementations available. One of such improvements would be the scalable K-means++ by Bahmani et al. [28], which is shown to be considerably faster than native K-means++. Another practical option is to exploit K-means implementations from a proven distributed computing platform [29].
Graph knitting scheme
From this step onwards, the presented heuristic digresses from most of the existing geometric approaches of the second type, primarily because of the choice of graph to represent the classification dataset, and the use of seminal works in neighbor searching methods. First, the choice of a weighted graph opened the possibility of using well researched work on graph coarsening, which is the foundation of the graph clubbing step. Second, most of the existing approaches could not benefit from seminal works in the neighbor searching methods, which have contributed profoundly to the success of computer vision. The fast library for approximate nearest neighbors (FLANN) search engine [23] is used in this work.
An information graph is constructed once a reasonable representation of the underlying data is obtained. Two major challenges include determination of neighbors, and capturing the classification patterns in edge weights. First, neighbors are determined such that the whole hypothesis is covered while passively enforcing regularity and planarity in the graph. The neighbors are determined in two steps, followed by a step that controls skewness of the graph. Second, a two-fold pattern capturing edge weight scheme is presented in Eq. 2.
The first challenge of neighbor determination is addressed in two stages, superficial neighbor search (SNS), and exclusive neighbor search (ENS), presented in Part I, and II, respectively. In the algorithm, the number of neighbors is nominally controlled by an input parameter nn, the number of desired neighbors. For every \(i{\text {th}}\) node, the variable \(no\_tot\_neigh[i]\) in Line 1 of Part I is used to track the size of the neighbor list.
However, the neighbor list for the \(i{\text {th}}\) node can be terminated before adding nn neighbors by updating the variable \(neigh\_list\_finish[i]\) to TRUE in Line 4 of Part I. By limiting the number of neighbors in this way, construction of the graph can be controlled.
Part I is used to look for nn nearest neighbors, regardless of the target class. The algorithm takes an empty graph \(G(V, \phi )\) as input, which is formed with the set of the cluster centers V after the clustering step. This set is the search space passed to the FLANN space indexing utility in Line 8 and 9. The input constant R is explained later in Eq. 1. Objective of the part is to fill the graph G with \(E_I\) set of edges. This is similar to the construction of a K-nearest neighbor graph (K-NNG). However, an input parameter, MAX_SAME_CLASS_NEIGH, is used to limit the number of same class neighbors in Line 13, so that the remainder of edges for the \(i{\text {th}}\) node, \(nn - no\_tot\_neigh[i]\), are constructed for nodes with the opposite target class. Nodes for which all nn neighbors are found will vary depending on characteristics of the data. They will be excluded from the next parts. An additional computation of reach in Line 15 is maintained for every node. The metric presented in Eq. 1 is similar to the Hausdorff distance [19]. The distance utility of FLANN, in Line 10, is used to compute the summation in Eq. 1, whereas a scaling constant (R) controls the reach according to
$$\begin{aligned} r_i = R \times \sum _{j\epsilon N_i} |x_i - x_j|, \end{aligned}$$
where R is the scaling constant, \(r_{i}\) is reach of the \(i{\text {th}}\) node, \(x_{i}\) is the position of the \(i{\text {th}}\) node, \(x_{j}\) is the position of the \(j{\text{th}}\) node, and \(N_{i}\) denotes the set of same class neighbors for the \(i{\text {th}}\) node.
In order to capture the classification patterns completely, it is necessary to make edges along the hypothesis of the classification data. So, Part II extends neighbor searching exclusively for nodes of the opposite target class. The \(i{\text {th}}\) node is considered only if there is a remainder requirement of neighbors, \(nn_{r}[i]\), where \(nn_{r}[i] = nn - no\_tot\_neigh[i]\). Nodes of class 2 form the search space in Line 3, in which neighbors are searched for class 1 nodes. This step along with the vice-versa case forms one iteration of the ENS procedure. For each iteration, a search space of the opposite target class is constructed in Line 3.
Inclusion of a neighbor after space searching in Line 5 is stringent compared to SNS. An input parameter, NEIGH_LIMIT and the computed reach are used to limit the availability of node j as prospect neighbor in Line 9. For every \(j{\text {th}}\) node added as a neighbor, the variable \(node\_neigh[j]\), initiated in Line 6, is incremented. Reach is used to further update the boolean \(neigh\_list\_finish[i]\) for node i, even if \(nn - no\_tot\_neigh[i] > 0\) in Line 13. Such an \(i{\text {th}}\) node tends to be an internal node of a target class. In other words, reach only encourages the convex hull nodes of one class to choose neighbors with nodes of the opposite class, aiding in planar construction of the graph.
The \(node\_neigh\) counter is required in Part II to avoid a node that might habitually come up as a prospect neighbor despite not being very representative of a target class. It otherwise leads to a skewed graph with respect to node degree. \(node\_neigh\) in Line 3 of Part III is used to reduce the search space of every target class, controlling the skewness of the constructed graph.
The second challenge is to capture the classification patterns in edge weights, for which a two-fold edge weight scheme is designed. First, each node measures its internal pattern as the absolute difference from one of the two target classes. Second, every pair of nodes in an edge measures their external pattern by the relative difference of their target classes. Individual contributions are added via the power scheme in Eq. 2 to weigh the edge according to
$$\begin{aligned} w_{i,j} = C_I^{1-|\text {tc}(i)|} + C_I^{1-|\text {tc(j)}|} + C_E^{|\text {tc}(i)-\text {tc}(j)|}, \end{aligned}$$
where \(w_{i,j}\) is the weight of the \((i, j){\text {th}}\) edge, \(\text {tc}(i)\) denotes target class of the \(i{\text {th}}\) node, \(C_{I}\) and \(C_{E}\) are constants for internal and external classification patterns, respectively, and quantities in || are absolute values.
The use of a state-of-the-art implementation like FLANN for neighbor searching cannot be overemphasized. For instance, in a typical run case (\(n_{c} > 300\) and \(nn > 3\)), the approximate neighbor searching method of FLANN was \(\approx \) 1 000 times faster when compared to the exact algorithm for nearest neighbor searching, which involves computing distances with all the remaining nodes for every node, and then sorting them to determine the nearest neighbors. The run-time advantage is clear when comparing the complexity of exact graph construction, \(O(n^3\text {log}(n))\), to that of approximate methods offered by FLANN [23].
Graph shedding scheme
Once a weighted graph is obtained, an edge cut based filtering presented in Algorithm 2 separates the training dataset into relevant and non-relevant. For every node, the neighbor list is iterated to check for a significant edge in Line 5, and when found, that node is added to \(relevant\_node\_list\) in Line 11. This leads to a training set selection that the second type of approach aims to achieve. Note that the result of this step depends on the characteristics of the graph, such as how well connected the graph is, and how well the underlying classification patterns have been captured. Algorithm 1 with the edge weight scheme addresses these issues.
The role of nominal VC dimension drips down to the pruned graph as well. Since it controls the granularity of the underlying data, it also controls the granularity of data selection. That gives the heuristic an essential advantage in terms of limiting data shrinkage while selecting the relevant data points. Because the selection is done via clusters, and the ratio of n to \(n_{c}\) is typically \(> 10\), many data points that are not very close to the hypothesis boundary of the classification patterns are also selected. That gives an extra buffer of data points upon which another selection method of both types applies. The majority of classification algorithms, for example neural methods, gaussian processes et cetera, can use the heuristic. Until this step, the presented heuristic's aim matches with the existing approaches of the first and second type. For comparison purposes, the heuristic until this step (including) is referred to as GSH for the remainder of this work. Edge cut for GSH is referred to as GS edge cut.
Graph clubbing scheme
This step extends the problem statement of training set selection to further breaking the reduced training set into few partitions or critical chunks, each of which can be trained independently, by virtue of Part I to III of Algorithm 3. The main aim is to design an approximate formulation that is theoretically faster, even for serial execution. The algorithm divides the training set into few partitions such that the number of computations are reduced significantly in the training phase. Consider that the order of complexity of most of the classification algorithms is higher than linear, that is \(a > 1\) if \(O(n^{a})\) is the complexity. The graph clubbing scheme doesn't change the order, however it results in significant reduction in total computations. For example, for a classification algorithm with \(O(n^{2})\) complexity, if \(C \times n^{2}\), where C is a constant, is the original number of computations; then after data reduction by the graph clubbing scheme, there are four equal sized partitions/critical chunks. Now the number of computations is \(C \times (4 \times (\frac{n}{4})^2 ) = C \times \frac{n^2}{4}\) or a quarter of the original number.
Independence during the training phase of each obtained partition is mainly accomplished because of the directional aspect of the algorithm, which is achieved via the edge weight scheme. The directional aspect is responsible for two objectives, namely obtaining equally-sized partitions, and ensuring orthogonality of the hypothesis boundary with neighboring partitions' boundary. Two ways in which the edge weight scheme is leveraged for the directional aspect is in the priority aspect of the partial weighted matching (PWM) algorithm, Part I, and the reassessment aspect of the coarsening formulation, Part II. The graph clubbing algorithm, Part III, ties Part I, and II in an iterative scheme. Each of such obtained partitions can now be trained independently, giving further leverage for a nominal number of worker processes.
The partial weighted matching (PWM), Part I, is designed for the weighted graph obtained from the graph knitting step. The obtained matching is partial because edges weighing less than EDGE_CUT (input parameter) are filtered out in Line 4. So only cluster points closer to the hypothesis boundary are chosen for training. Sorting in Line 6, before ordered matching in Line 9, adds the weighted aspect to the matching. It enables the heaviest edges to be picked earliest for contraction, subtly addressing both main objectives of the directional aspect. Since the heaviest edges cover the classification patterns, prioritized selection of them results in uniform size of partitions. Prioritized selection also means that the most significant patterns are given preference, which conversely means the least significant patterns are avoided. So the hypothesis boundary, along which the least significant patterns reside, is orthogonal to the contracted edges, where the most significant patterns reside. Higher prediction accuracies are obtained because of the preference of contraction of the heaviest edges.
The coarsening formulation of Part II applies the directional aspect, as edge contraction occurs in this part. It is to be noted that the coarsening formulation is different in aim to otherwise researched formulations. Most of the popular formulations are intended for reducing communication cost or preserving the global structure while getting a low-cost representation of data [30]. Furthermore, unlike Kernighan-Lin and other matching based coarsening objectives, the presented optimization objective is deterministic in execution.
In the part, reassessment of target class and edge list for newly contracted nodes in Lines 6 and 10, respectively, augments the standard coarsening step in Line 3. By using different values of \(C_I\) and \(C_E\) for initial versus reassessment edge weights, the precedence of the kind of edges is established in the matching scheme. Original heavy edges are prioritized over re-assessed edges of newly contracted edges, achieving the orthogonality property. Transition of coarsening from original edges to newly contracted nodes can be captured by the virtue of drastic decrease in graph cost metric, presented in Eq. 3,
$$\begin{aligned} \text {cost}(G) = \sum _{e\epsilon E'} w_e, \end{aligned}$$
where \(\text {cost}(G)\) is cost of the graph G, e is an edge, \(E^{'}\) denotes the subset of E (set of all edges) such that \(\{e \mid e \epsilon E \text{ and } > {{\text {EDGE\_CUT}}}\}\), and \(w_{e}\) is the weight of edge e.
One technical choice is to use MIN(i, j) in Line 4 for identifying the new node that is a result of the contraction of edge between nodes i and j.
Part I and II are executed in the iterations of Part III in Lines 5 and 6, respectively. In Line 7, kink detection in graph cost is used as a termination condition of the iterative algorithm. However, a variable for the number of coarsening iterations, i.e. MAX_NUM_OF_COARSENING_ITER in Line 4, is used in the majority of tests. This step concludes the formulation of the heuristic. It is referred to as GCH for the remainder of this work. Similarly, edge cut is referred to as GC edge cut.
The implementation of few optimizations improved the run-time. First, the starting nodes in Line 2, Part I, are the ones that are identified relevant in GSH. After each iteration, half of the nodes that belong to contracted edges are reduced in the update of relevant nodes list (\( relevant\_nodes\_list\) in Algorithm 2). As a result, the complexity of the matching algorithm reduces with the coarsening iterations. Second, the neighbor list of a node is sorted. As a result, edge contraction computation is linear (in complexity) to nn. It is computationally canonical to the sorted union of two lists. Lastly, usual numerical optimizations, such as masking to avoid dynamic memory allocations, and indexing (at the expense of memory) for O(1) searching are used.
An event-driven, multi-process algorithm is designed to distribute the partitions obtained after GCH in a communication-free manner. In the network application, processes assume a (single) master or (multiple) worker role. Part I and II of Algorithm 4, respectively, describe master's and worker's side of the event-handling design.
For the master process which executes Part I, one partition from the list of partitions, \(parts\_list[]\), is communicated to the requesting worker process (the one which issues the DATA_REQUEST event in Line 3 of Part II) by trigger of the DATA event in Line 6. The worker process, upon receiving the event DATA, proceeds to training with the received data partition in Line 5 of Part II. After collecting acknowledgements of the completion of training of all data partitions in Line 9, the master process terminates every worker by issuing TERM_TRAIN event. The design implements a round-robin scheme, which balances load of the network queries. That is accomplished by having a queue data structure for recording DATA_REQUEST events in Line 3. It is to be noted that conflict of two simultaneous entries is resolved by time stamps, making the queue fair with respect to a worker's request.
Instead of directly using TCP channels for message communication, the distributed messaging API ZeroMQ [31] is used. It provides essential safety and liveness properties on network channels. However, apart from message guarantees such as the liveness and safety property of 'once only message delivery', ZeroMQ is rudimentary compared to higher level message passing libraries. This gives an opportunity to design and optimize various aspects of the architecture. One such aspect is the messaging protocol. A couple of messaging protocols are designed as shown in Fig. 2. Protocol 2 implements a single float/double entry (in character array or CA) messaging scheme, whereas protocol 1 first requires marshaling all entries of a data point before messaging it.
Messaging protocols. Protocol 2, i.e. single entry versus Protocol 1, i.e. marshaling protocol is presented for data of the \(i{\text {th}}\) point
Another aspect that was tested is connection time of the network. Connection time is measured on the master process, and included the following steps:
Start of TCP channels (wrapped in the API)
Initialize a hash table
Receive connection request from all worker processes
Send connection confirmation to all worker processes
The connection procedure requires step 2 for maintaining worker processes' information, giving an opportunity to optimize the step as required. A light-weight hash table (\(\approx 0.6\) MB) and hash key is designed which generates unique keys. The design helps to reduce the overhead of starting and running the multi-process application.
Unlike the application of GSH, which results in a single training set, multiple data partitions are obtained after GCH, and the training set is the union of these partitions. It means that there would be as many classifiers as the number of partitions. Hence, it is needed to determine which classifier to choose for predicting a point from the testing dataset, \(V_{T}\). Algorithm 5, nearest hypothesis search (NHS), is used for this task. A search space is formed consisting of nodes of the coarsened graph in Line 1. ANN search for the nearest hypothesis follows in Line 5.
Once the nearest hypothesis is determined, prediction of the target class for testing data points follows. This added step before the testing phase only takes about 5–7% of the run-time of the testing phase for the SVM class of algorithms, as will be shown in the 'Results' section.
Results are presented in two major parts, first with tests on parameter space of the heuristic, and second for gauging performance of the heuristic. All the tests were conducted on a variety of datasets.
Parameter space of heuristic
In this set of tests, the focus is on working details, and exemplifying steps of the heuristic. Datasets similar to that shown in Fig. 3, with parameters summarized in Table 1, are extensively used.
Near linearly-separable dataset
Table 1 Dataset along with steps parameters
Node reach and ENS
Tests in this section present heuristic tools that capture original classification data into the weighted graph. These tools are designed to handle real datasets, which vary diversely in characteristics. A mix of real and synthetic datasets that mimic varying characteristics is considered.
A timeline of the ENS procedure with \(nn = 4\) is presented in Fig. 4. It is based on the dataset of Table 1. However, class 2 data points were intentionally translated to create separation, which is very typical for real data. It is evident that the connectivity of the graph increases with more iterations. Skewness control of the constructed graph, explained in Part III of Algorithm 1, was carried out at end of the iterations of the ENS procedure, resulting in reduction of available nodes for search as shown in Table 2.
Exclusive neighbor search. Number of ENS iterations are 0, 2, and 4 in a–c, respectively. Zoomed view of dataset in Fig. 2
Table 2 Search space reduction
A second way to control connectivity is fine tuning of the reach equation, presented in Eq. 1. In the next test, the scaling constant R is varied, and results are presented in Fig. 5. Three cases are shown in succession, under-reach (a), ideal-reach (b), and over-reach (c). Even in the overreach case, inner nodes are not able to make opposite class neighbors, enforcing planar construction of the graph.
Reach controlled graph knitting. Scaling constant R is 0.5, 1.0, and 1.25 in a–c, respectively. Zoomed view of dataset in Fig. 2
The ENS procedure was next run on a real dataset, called cuff-less blood pressure estimation dataset, from the UCI machine learning repository [32, 33]. It is a three attribute, 12 000 instances, real, multivariate dataset. Results are presented in Fig. 6, with subfigure (a) showing the dataset, (b) showing cluster centers, whereas (c) and (d) showing the constructed graph without and with ENS procedure, respectively. The latter graph is connected because necessary edges are present between opposite class nodes, covering the entire hypothesis. Class imbalance is analysed next, and results are presented in Table 3. Column 2 shows the number of data points for each class originally, and Column 3 shows the number of data points after ENS, as presented in the graph of Fig. 6d. Imbalance of target classes, quantified by standard deviation (SD), is significantly reduced after ENS.
Graph knitting procedure. Scaled UCI dataset, clustered data, constructed graph with no and 1 ENS iteration in a–d, respectively. Dashed edges in d are formed after ENS procedure
Table 3 Class imbalance
The aim of this test is to exemplify the twofold edge weight scheme, shown in Eq. 2. It shows the role of edge weights in the outcome of GSH, summarized in Fig. 7. As classification patterns in the underlying data gets more confused in succession in Subfigures (a) and (c), more edges are weighted significantly, resulting in the selection of a bigger training set by GSH.
Reduction of training set. Classification data is more confused in (c) compared to (a). b, d show cluster nodes that were selected for (a) and (c) cases, respectively
This test exemplifies the formulation of the graph clubbing scheme. Although the edge weight scheme is important in the application of GSH, it is primarily designed to play a crucial role in the priority/directional aspect of the coarsening objective. Constants for initial edge weights were such that \(C_E\) was kept significantly higher than \(C_I\), as shown in Table 4. Two cases of re-assessments were considered, namely case I and case II.
Table 4 Pattern measure constants of Step 2
Since initial constants were kept significantly higher than their re-assessment counterparts, original nodes were contracted first in both cases, as can be seen in Fig. 8a, b. Transition to contraction of re-assessed type nodes is reflected in the graph cost characteristic in Fig. 9 for case I. After the \(2{\text {nd}}\) iteration, the slope magnitude decreases significantly because of contraction of all original nodes that have significantly higher edge weights dictated by heavier initial constants. Edge contraction continues to aggressively club the graph for case I, including the re-assessed nodes. This results in fewer partitions compared to case II in Fig. 8b. Clubbing is almost shut off in case II because of trivial re-assessment constants. Note that in general, the more coarsening iterations, the fewer partitions.
Partitioning of training set. Number of coarsening iterations is 2 and 10 for (a) and (b), respectively. In each subfigure, the first plot is for the re-assessment case I, whereas the overlaying plot with shifted origin is for case II
Graph cost function with coarsening iterations
Several tests were conducted to evaluate performance of the presented methods, including GSH, serial GCH, and distributed GCH. In addition, the network architecture for distributed GCH is evaluated.
Dataset I was a synthetically constructed two-dimensional, near-linearly separable dataset with parameters shown in Table 5. Dataset II was a similar dataset, but in place of a (nearly) straight-separating hyperplane, a spherical separating hyperplane of radius 0.2 was employed. Otherwise, it uses the same parameters as Dataset I. A dataset from the UCI machine learning repository, called skin segmentation dataset [33, 34], is the next dataset, called Dataset III for the remainder. This classification dataset has four numeric features, and \(\approx 245\hbox {k}\) observation instances. The nominal VC dimension is 245. Table 6 summarizes other relevant parameters.
Table 5 Dataset I and II along with steps parameters
Table 6 Dataset III along with steps parameters
An unstructured text dataset from the zenodo public repository is referred to as Dataset IV. This e-commerce dataset consists of the title and description of products for four categories, out of which two categories, electronics and clothing are considered, and feature extraction methods including term frequency-inverse document frequency (tf-idf) and word2vec [35] were applied. Parameters of the dataset and steps, along with feature engineering details, are summarized in Table 7. Lastly, an image dataset consisting of two categories, apparels and speakers, from the zenodo public repository is referred to as Dataset V, on which a VGG16 [36] neural network is applied to get features vectors. Other parameters for this dataset are summarized by Table 8.
Table 7 Dataset IV, and test 1 and 2 parameters
Table 8 Dataset V, and test 3 parameters
Note that the computation setup was kept consistent. Programming language of all codes for which computation time was measured is C/C++. That includes every step of the presented heuristic, and the classification algorithms. Lastly, time stamping was carried out on an otherwise idle system.
The main aim of the following set of tests is to compare the training phase using GSH against the state-of-the-art shrinking heuristics of LIBSVM. Classification algorithms used in these tests were all variants of LIBSVM's SMO implementation [10], which is a method of the second type.
Two testing variables are evaluated. First, computation run-time of GSH along with the classification algorithm on reduced training data is compared against that of the same classification algorithm but augmented with the shrinking heuristic. Second, the prediction accuracy of the learned models obtained from both the above setups is compared.
Results are presented in the following way. Each of Figs. 10, 11, 12, 13, 14, 15, 16, 17 and 18 is for a SVM classifier, that includes C-SVM and nu-SVM with linear, polynomial, and rbf kernels. Subfigure (a) is used to present computation run-time results, and (b) is used for prediction accuracy comparison. Training dataset size was geometrically varied from 1k to 200k, and testing dataset size was kept three times that of the training. Accordingly, the parameter \(n_{c}\) was varied geometrically in the range given in Table 5. Note that training time is included in 'LIBSVM shrinking heuristic' and 'GSH' plots in Subfigure (a), but is not explicitly written for convenience. So the 'GSH' plots include the run-time of Step 1 to 3 of the heuristic along with the training phase. A third line-plot denoted 'SVM training (post GSH)' is presented in Subfigure (a) to separate computation run-time of GSH from subsequent training.
C-SVM with linear kernel
C-SVM with polynomial kernel
nu-SVM with linear kernel
nu-SVM with polynomial kernel
Difference in prediction accuracies from LIBSVM for the approaches
Compound performance gains from applying GSH after SRS
nu-SVM with rbf kernel
C-SVM with linear kernel was the first classification algorithm tested, as presented in Fig. 10. The GSH scales even for a low number of points (\(\approx 10\hbox {k}\)), as shown in Subfigure (a), however, scaling becomes more evident with more training data points. Furthermore, it is visible that GSH took a small fraction of total time, as indicated by the separation between the second and third line plot. It highlights the scalability issue of the SVM formulation. In Subfigure (b), the prediction accuracy of reduced training closely follows that of full training, clearly visible after \(\approx 20\hbox {k}\) testing data points.
C-SVM with polynomial kernel was the second classifier tested, as presented in Fig. 11. Observations here follow that of the earlier case. Although, one noticeable difference is a better run-time profile compared to the previous case. This is because of the higher run-time complexity of the polynomial kernel classifier compared to that of the linear kernel presented in Fig. 10.
For the second SMO implementation, nu-SVM, Figs. 12, and 13 summarize linear and polynomial kernel cases, respectively. Note that the training phase with nu-SVM in Figs. 12a and 13a is significantly slower compared to C-SVM in Figs 10a and 11a. Furthermore, it is observed that reduced training can result in improved prediction accuracy, as can be seen in Fig. 13b. For the polynomial kernel, GSH performed better than native training by about 6%.
The next set of two tests emphasize the consistent observation of better prediction accuracy observed for GSH. Given that sampling is a part of GSH, two sampling methods, simple random sampling (SRS) and reservoir sampling (RS) (in the context of big data tool Apache Hadoop [37]), as commonly used in the field of big data, are considered.
In the first test, the sampling size for the two methods is kept the same as the reduced training set size obtained after GSH. Findings are reported in terms of the difference in prediction accuracy from LIBSVM, and presented in Fig. 14. It can be seen that GSH exhibits near \(0\%\) accuracy difference compared to other approaches. This consistent observation is attributed to the edge weight scheme (applied in the graph knitting step or Step 2) which captures all the classification patterns. In a sampling method of any type, some data points which are important with respect to the classification task are bound to be lost because of the involved stochastics.
In the second test, GSH was applied on the sampled data from the SRS method, for which the sampling fraction was varied from 0.2 to 0.8 for a set of two datasizes, 30 146 and 14 142. The cases are referred to as 1 and 2, respectively, in the computation time results presented in Fig. 15, which are averaged over 10 runs. Table 9 summarizes the prediction accuracy discrepancies of the sampling method from LIBSVM in Column 2, and that of GSH from the sampling method in Column 3. With only compromising \({1/3}{\text {rd}}\) to 1/2 confidence in the prediction accuracies, as evident by comparing Column 4 and 5, compound performance gains shown in Fig. 15 can be leveraged.
Table 9 Results on prediction accuracy
Dataset I was used in all the tests until now, and near-perfect accuracy plots support that it is an ideal dataset. A more realistic case, Dataset II, was considered for Figs. 16, 17 and 18.
Again, C-SVM with linear kernel was the first classification algorithm tested, as presented in Fig. 16. However, prediction accuracies are in a lower range than before, as shown in Subfigure (b). Nevertheless, both setups are practically identical in prediction accuracy. Similar run-time improvements are observed in Subfigure (a). For nu-SVM with linear kernel, presented in Fig. 17, scaling observations are similar to the corresponding Fig. 12, which used Dataset I. Note that the prediction accuracies are only about 50%, which is the expected value for random selection between two target classes. However, note that the main argument here is not the absolute prediction accuracy, rather the closeness of it for both setups. To improve the prediction accuracy, results were obtained for a radial basis function (rbf) kernel, which is widely considered a robust kernel type in SVM classifiers, and are presented in Fig. 18. This resulted in slower learning, as observed in Subfigure (a), but near perfect prediction accuracy, as shown in Subfigure (b).
The next set of tests was conducted on Dataset III, for evaluating the performance of the heuristic on real data, and barplots in Figure 19 are used to present the findings. Computation run-time was scaled to accommodate different classification algorithms, namely C-SVM and nu-SVM with linear, polynomial, and rbf kernels. For all six cases, reported prediction accuracy is low, as shown in Table 10, but close for both setups. Scaling improvements are consistent to the previous reportings for Dataset I and II.
Fraction of LIBSVM training time. a–c is for C-SVM, whereas d–f is for nu-SVM with linear, polynomial, and rbf kernels, respectively
Table 10 Prediction accuracy (%)
The next set of two tests was conducted on Dataset IV, for evaluating the performance of the heuristic on high-dimensional real data where various feature extraction methods including term frequency-inverse document frequency (tf-idf) and word2vec [35], classification algorithms including CNN from Tensorflow [38] and LIBSVM, dataset size, and dimensions were varied, as summarized in Table 7.
In the first test, use of tf-idf to compute feature vectors resulted in dimensions more than 10k, and the classification algorithm was CNN from Tensorflow [39]. Dataset size was the variable, and results are presented in Table 11, from which scaling advantages of GSH without change in prediction accuracy are apparent. In the second test, a state-of-the-art feature extraction method, word2vec, was applied to generate the vectors for a set of two dimensions, 500 and 1 000, which were then used to train a LIBSVM classifier with linear kernel. It is apparent from the results presented in Table 12 that a speed-up of \(\approx 10\) can be achieved with \(0.77\%\) improvement in average prediction accuracy.
Table 11 Test parameters and results
Similar to the previous two tests, the last test was conducted on Dataset V, where the feature extraction method of VGG16 [36] and LIBSVM classifier with linear kernel was used, along with other parameters summarized in Table 8. Results are presented in the last four rows of the table, which shows that a speed-up of 4.7 times can be achieved over more than \(25\hbox {k}\) features without compromising any prediction accuracy.
Overall, scalability improvements are observed consistently over a number of classifiers for Dataset I to V. Furthermore, the prediction accuracy either closely follows original training or even outperforms it in a few instances.
Serial GCH
The next set of tests are aimed to compare run-time of serial GCH to that of GSH. More precisely, serial execution of GCH, which represents approximate learning, is fared against (reduced training data) GSH learning. In GCH, the graph clubbing step was an added cost over GSH, so the 'GCH' plots include the run-time of Step 1 to 4 of the heuristic along with serial execution of the training phase for the obtained partitions (after Step 4). Results are presented for Dataset I.
The number of iterations was set 10, and used as the termination condition for the graph clubbing algorithm. The pre-processor was used, and the collected prediction accuracies were weight-averaged over all reported partitions. Edge weight constants given in Table 13 were used throughout.
Table 13 Pattern measure constants (of Step 2) for GCH
Only GSH and GCH are compared along with respective line-plots of subsequent training for all the classifiers, as presented in Figs. 22 and 23. However, for the first classifier, C-SVM with linear kernel, shown in Fig. 20, computation run-time of LIBSVM with shrinking heuristic is also plotted. An accuracy plot, Subfigure (b), and a plot for the number of partitions in Fig. 21 are also presented.
Partitions after GCH
In Fig. 20a, a clear scalability hierarchy is observed, with serial GCH > GSH > LIBSVM shrinking heuristic. In the accuracy plot, shown in Subfigure (b), approximate learning closely follows native learning for more than 8k training data points. However, for fewer points, the accuracy of the model trained after GCH, which is \(\approx \) 95%, is not as good as native training at \(99\%\). 2 to 5 partitions were obtained as shown in Fig. 21. For polynomial kernel classifiers, presented in Figs. 22 and 23, scaling advantages with serial GCH compared to GSH are observed even for fewer data points (\(\approx 5\hbox {k}\)).
Distributed GCH
The main aim here was to demonstrate run-time advantages with worker processes for the communication-free training scheme discussed earlier. The plots here differ from 'serial GCH' plots only by the mode of execution of the training phase, that is, obtained partitions are distributed to worker processes as per the network design or Algorithm IV. The scheme only aims for coarse grain parallelization, such that individual data chunks/partitions can be distributed to worker processes.
Results are presented in Fig. 24 for C-SVM with a polynomial kernel classifier on Dataset I. The distribution invariantly depends on the number of obtained partitions. Therefore, scaling results in Subfigure (a) cannot be interpreted without the plot for the number of partitions, which is shown in Subfigure (b).
Scalability with workers (a), and obtained partitions (b)
Significant improvements are observed with more workers. However, sub-optimal distribution with number of workers more than 2 can be seen in Fig. 25. Mean number of data points per chunk/partition is plotted as a function of the number of total data points. Using the number of partitions/chunks from Fig. 24b, it is straightforward to interpret the monotonic increase of the plot. In a hypothetical case where the partitions are all equally-sized, SD would be zero. However, as the number of total data points increases, error bars for SD are observed to be commensurate to the mean number of data points. Unequal size of partitions is the apparent reason for the sub-optimal distribution. Overall, Table 14 summarizes the performance gains for all the approaches, namely GSH, serial GCH and distributed GCH, relative to the shrinking heuristic available in LIBSVM for C-SVM with a polynomial kernel on Dataset I.
Mean size of partitions with SD errorbars
Table 14 Scaling improvements of the heuristic approaches
The main motivation for this set of tests was to evaluate various implementation aspects of the network that implemented distributed GCH. The first aspect is the connection time of the distributed application. Fig. 26 shows connection time measurements with the number of workers.
Performance of the designed connection procedure
Apparently, the connection time is very small (\(\approx 120\,\hbox {ms}\)) when compared to the run-time of the training phase. The connection procedure was replicated on another distributed messaging API, MPICH3.3 [40], and the connection time with ZeroMQ was observed to be 1.53 times faster than with MPICH3.3.
The next aspect of the network is the messaging protocols. The primary motivation behind the protocol design is to utilize the shape of the \(i{\text {th}}\) data point, that has an n-dimensional feature vector (\(x_{i}\)), and a 1-dimensional target class value (tc\((i))\). Two of such protocols as previously discussed were fared in run-time. A couple of opposing factors are at play between these protocols. Data marshaling is an extra step in protocol 1 (before actual messaging) when compared to protocol 2. On the other hand, message count/traffic increases (multiple times governed by the dimensionality of data) in protocol 2 compared to protocol 1. Overall, protocol 1 performed better as can be been in Fig. 27. Although increased marshaling may probably result in even better scaling, a larger memory footprint on the channel is un-advisable as it can result in an overflow scenario.
Evaluation of messaging protocols. Protocol 1 is used in this work
Messaging efficiency was tested in the context of prediction/testing phase of the distributed implementation. Reported in Fig. 28, it is very efficient at \(\approx \) 1% of the prediction time. It is found to be even more efficient at \(\approx \) 0.5% for the training phase. Pre-processing run-time is at \(\approx \) 7%, as can be seen in Fig. 28.
Messaging and pre-processing. Run-time of message communications and pre-processing measured in percentage of the testing phase
Note that external libraries are used in the formulation, including a clustering method, an approximate nearest neighbor implementation, two messaging APIs, and many classification algorithms. The heuristic execution is not very sensitive to the parameters of these external implementations. For instance, only 3–5 iterations of clustering were needed, input parameters for approximate nearest neighbor and classification algorithms were kept at default. However, the heuristic is sensitive to the parameters which are part of the heuristic steps.
Conclusions and future work
In this work, an algorithm with an edge weight scheme is designed to construct a weighted graph that effectively captures the classification patterns of the training dataset. Another graph coarsening algorithm with a directional aspect is formulated that divides the reduced dataset into partitions that can be trained independently using a novel communication-free network.
GSH provides an evident serial scalability advantage, and generic applicability holds for every classifier. Then, approximate learning is the reason behind serial GCH's added run-time advantage over GSH. Finally, distribution of chunks/partitions (after GCH) of the training set to worker processes results in further run-time improvements in distributed GCH. Overall, for all the approaches, namely GSH, serial, and distributed GCH, scaling benefits against full data training with classification algorithms from LIBSVM and Tensorflow are accompanied by no compromise in prediction accuracy.
Unequal coarsening and control over the number of partitions still needs to be further investigated in the current implementation of GCH in two possible areas,
First is the directional aspect of the objective. That is, if the partitioning was better directed, better orthogonality between the contour of the underlying classification patterns and of partition boundaries shall be obtained in the approximate procedure. The two-fold pattern measure scheme should be designed to get the desired control over the direction of coarsening.
Modifying the objective function of the current graph coarsening scheme with parts of traditional coarsening objectives. Since they directly add size difference as a cost, it will address the unequal coarsening problem in the presented objective. Furthermore, the number of partitions is directly controlled in these objectives.
Dataset I and II used during the current study are available from the corresponding author on a reasonable request. Furthermore, two real datasets used are available in the UCI machine learning repository, https://archive.ics.uci.edu/ml/datasets/skin+segmentation. https://archive.ics.uci.edu/ml/datasets/Cuff-Less+Blood+Pressure+Estimation. Dataset IV and V are available in the zenodo public repository, https://doi.org/10.5281/zenodo.3355823. https://doi.org/10.5281/zenodo.3356917, respectively. Other resources with respect to the manuscript are shared at https://sumedhyadav.github.io/graph_heu/. Tests were performed with following computing specfications: Operating system(s): Ubuntu 16.04 LTS. Programming language: C/C++, python (for visualization). Visualization tools: matplotlib and networkx modules in python. External Libraries: FLANN (1.8.4), mlpack (3.0.4), LIBSVM (3.23), Tensorflow (1.12.0), Keras (2.2.4). Version requirements: g++ 5.4.0, Python 2.7.12, matplotlib 1.5.1, networkx 2.2, numpy 1.15.4.
WSS:
working set selection
SMO:
sequential minimal optimization
VC dimension:
Vapnik Chervonenkis dimension
approximate nearest neighbors
K-NNG:
K-nearest neighbors graph
superficial neighbor search
ENS:
exclusive neighbor search
GSH:
graph shedding heuristic
GCH:
graph clubbing heuristic
NHS:
nearest hypothesis search
SRS:
reservoir sampling
number of data points
number of features
\(n_{c}\) :
number of initial clusters (input parameter)
\(w_{i,j}\) :
weight of \((i, j)\) edge
\(\text {tc}(i)\) :
target class of the \(i{th}\) node
\(C_{I}\) :
constant for internal classification pattern
\(C_{E}\) :
constant for external classification pattern
reach constant (input parameter)
\(r_{i}\) :
reach of the \(i{th}\) node
\(x_{i}\) :
position of the \(i{th}\) node
\(x_{j}\) :
position of the \(j{th}\) node
\(N_{i}\) :
set of neighbors of \(i{th}\) node
\(\text {cost}(G)\) :
cost of graph \(G\)
an edge
\(E^{'}\) :
subset of E (set of all edges) such that \(\{e \mid e \, \epsilon \, E \text{ and } > \text{ GC } \text{ edge } \text{ cut }\}\)
\(w_{e}\) :
weight of an edge e
Levy AY, Fikes RE, Sagiv Y. Speeding up inferences using relevance reasoning: a formalism and algorithms. Artif Intell. 1997;97:83–136. https://doi.org/10.1016/S0004-3702(97)00049-0.
Blum AL, Langley P. Selection of relevant features and examples in machine learning. Artif Intell. 1997;97:245–71. https://doi.org/10.1016/S0004-3702(97)00063-5.
Weng J, Young DS. Some dimension reduction strategies for the analysis of survey data. J Big Data. 2017;4(1):43. https://doi.org/10.1186/s40537-017-0103-6.
Guyon I, Gunn S, Nikravesh M, Zadeh LA. Feature extraction: foundations and applications. In: Studies in fuzziness and soft computing, vol 207. Berlin Heidelberg: Springer, Springer-Verlag; 2008. https://doi.org/10.1007/978-3-540-35488-8.
Fayed H. A data reduction approach using hyperspherical sectors for support vector machine. In: DSIT '18:Proceedings of the 2018 international conference on data science and information technology, Singapore, Singapore. ACM, New York, NY, USA; 2018. https://doi.org/10.1145/3239283.3239317.
Coleman C, Mussmann S, Mirzasoleiman B, Bailis P, Liang P, Leskovec J, Zadaria M. Select via vroxy: efficient data selection for training deep networks; 2019. https://openreview.net/forum?id=ryzHXnR5Y7. Accessed on 1 Feb 2019.
Loukas A, Vandergheynst P. Spectrally approximating large graphs with smaller graphs; 2018. CoRR arXiv:1802.07510.
Weinberg AI, Last M. Selecting a representative decision tree from an ensemble of decision-tree models for fast big data classification. J Big Data. 2019;6(1):23. https://doi.org/10.1186/s40537-019-0186-3.
Chen PH, Fan RE, Lin CJ. A study on smo-type decomposition methods for support vector machines. Trans Neural Netw. 2006;17:893–908. https://doi.org/10.1109/TNN.2006.875973.
Fan RE, Chen PH, Lin CJ. Working set selection using second order information for training support vector machines. J Mach Learn Res. 2005;6:1889–918.
Nalepa J, Kawulok M. A memetic algorithm to select training data for support vector machines. In: GECCO '14:Proceedings of the 2014 annual conference on genetic and evolutionary computation, Vancouver, BC, Canada. ACM, New York, NY, USA; 2014. https://doi.org/10.1145/2576768.2598370.
Salvador S, Chan P. Determining the number of clusters/segments in hierarchical clustering/segmentation algorithms. In: ICTAI '04:Proceedings of the 16th IEEE international conference on tools with artificial intelligence. IEEE Computer Society, Washington, DC, USA; 2004. https://doi.org/10.1109/ICTAI.2004.50.
Awad M, Khan L, Bastani F, Yen IL. An effective support vector machines (SVMs) performance using hierarchical clustering. In:16th IEEE international conference on tools with artificial intelligence, p. 663-7; 2004. https://doi.org/10.1109/ICTAI.2004.26.
Cervantes J, Li X, Yu W, Li K. Support vector machine classification for large data sets via minimum enclosing ball clustering. Neurocomputing. 2008;71:611–9. https://doi.org/10.1016/j.neucom.2007.07.028.
Li X, Cervantes J, Yu W. Two-stage svm classification for large data sets via randomly reducing and recovering training data. In: IEEE international conference on systems, man and cybernetics, October 2007; 2007. https://doi.org/10.1109/ICSMC.2007.4413814.
Wang J, Neskovic P, Cooper LN. A minimum sphere covering approach to pattern classification. In: ICPR'06:18th international conference on pattern recognition. 2006, 3:433-6; 2006. https://doi.org/10.1109/ICPR.2006.102.
Mavroforakis ME, Theodoridis S. A geometric approach to support vector machine (SVM) classification. Trans Neural Netw. 2006;17:671–82. https://doi.org/10.1109/TNN.2006.873281.
Fung G, Mangasarian OL. Data selection for support vector machine classifiers. In: KDD '00:Proceedings of the Sixth ACM SIGKDD international conference on knowledge discovery and data mining. Boston, Massachusetts, USA. ACM, New York, NY, USA; 2000. https://doi.org/10.1145/347090.347105.
Wang J, Neskovic P, Cooper LN. Training data selection for support vector machines. In: ICNC'05:Proceedings of the first international conference on advances in natural computation—Volume Part I, Changsha, China. Springer-Verlag, Berlin, Heidelberg, Germany; 2005. https://doi.org/10.1007/11539087_71.
Yu H, Yang J, Han J. Classifying large data sets using SVMs with hierarchical clusters. In: KDD '03:Proceedings of the ninth ACM SIGKDD international conference on knowledge discovery and data mining, Washington, DC, USA. ACM, New York, NY, USA; 2003. https://doi.org/10.1145/956750.956786.
Chang CC, Lin CJ. LIBSVM: a Library for support vector machines. ACM Trans Intell Syst Technol. 2011;2:27. https://doi.org/10.1145/1961189.1961199.
Chau LA, Li X, Yu W. Convex and concave hulls for classification with support vector machine. Neurocomputing. 2013;122:198–209. https://doi.org/10.1016/j.neucom.2013.05.040.
Muja M, Lowe DG. Scalable nearest neighbor algorithms for high dimensional data. IEEE Trans Pattern Anal Mach Intell. 2014;36:2227–40. https://doi.org/10.1109/TPAMI.2014.2321376.
Leevy JL, Khoshgoftaar TM, Bauder RA, Seliya N. A survey on addressing high-class imbalance in big data. J Big Data. 2018;5(1):42. https://doi.org/10.1186/s40537-018-0151-6.
Arthur D, Vassilvitskii S. K-means++: The advantages of careful seeding. In: SODA '07:Proceedings of the eighteenth annual ACM-SIAM symposium on discrete algorithms. New Orleans, Louisiana. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA; 2007.
Hughes M. KmeansRex : Fast, vectorized C++ implementation of K-Means using the Eigen matrix template library; 2018. https://github.com/michaelchughes/KMeansRex. Accessed on 1 Dec 2018.
Curtin RR, Edel M, Lozhnikov M, Mentekidis Y, Ghaisas S, Zhang S. mlpack 3: a fast, flexible machine learning library. J Open Source Softw. 2018;3:726. https://doi.org/10.21105/joss.00726.
Bahmani B, Moseley B, Vattani A, Kumar R, Vassilvitskii S. Scalable K-means++. Proc VLDB Endow. 2012;5:622–33. https://doi.org/10.14778/2180912.2180915.
Zaharia M, Xin RS, Wendell P, Das T, Armbrust M, Dave A, Meng X, Rosen J, Venkataraman S, Franklin MJ, Ghodsi A, Gonzalez J, Shenker S, Stoica I. Apache Spark: a unified engine for big data processing. Commun ACM. 2016;59:56–65. https://doi.org/10.1145/2934664.
Dhillon IS, Guan Y, Kulis B. Weighted graph cuts without eigenvectors a multilevel approach. IEEE Trans Pattern Anal Mach Intell. 2007;29:1944–57. https://doi.org/10.1109/TPAMI.2007.1115.
Akgul F. ZeroMQ. Birmingham: Packt Publishing; 2013.
Kachuee M, Kiani M, Mohammadzade H, Shabany M. Cuff-less high-accuracy calibration-free blood pressure estimation using pulse transit time dataset; 2015. https://archive.ics.uci.edu/ml/datasets/Cuff-Less+Blood+Pressure+Estimation. Accessed 1 Dec 2018.
Dua D, Graff C. UCI machine learning repository; 2019. http://archive.ics.uci.edu/ml. Accessed 1 Dec 2018.
Bhatt R, Dhall A. Skin segmentation dataset; 2012. http://archive.ics.uci.edu/ml/datasets/skin+segmentation. Accessed 1 Dec 2018.
Řehůřek R, Sojka P. Software Framework for Topic Modelling with Large Corpora. Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks 45-50; 2010. http://is.muni.cz/publication/884893/en. Accessed 28 July 2019.
Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition; 2014. CoRR arXiv:1409.1556.
Shvachko K, Kuang H, Radia S, Chansler R. The Hadoop Distributed File System, Proceedings of the 2010 IEEE 26th symposium on mass storage systems and technologies (MSST) MSST '10:1-10; 2010. https://doi.org/10.1109/MSST.2010.5496972.
Abadi M et al.. TensorFlow: Large-scale machine learning on heterogeneous systems; 2015. https://www.tensorflow.org/. Accessed 27 July 2019.
François C et al. Keras; 2015. https://keras.io. Accessed 27 July 2019.
MPICH3.3. A high performance and widely portable implementation of the message passing interface (MPI) standard; 2018. https://www.mpich.org/. Accessed 1 Feb 2019.
SY acknowledges Gautam Kumar's contributions in the project.
Gstech Technology Pvt. Ltd., 415, 2nd Floor, 16th Cross Road, 17th Main Road, HSR Layout Sector 4, Bengaluru, 560102, India
Sumedh Yadav
Institute for Combustion Technology, RWTH Aachen University, Templergraben 64, 52056, Aachen, Germany
Mathis Bode
Search for Sumedh Yadav in:
Search for Mathis Bode in:
This work is joint research of SY and MB. SY did all implementation, and run all test cases. All authors read and approved the final manuscript.
Correspondence to Sumedh Yadav.
Yadav, S., Bode, M. A graphical heuristic for reduction and partitioning of large datasets for scalable supervised training. J Big Data 6, 96 (2019) doi:10.1186/s40537-019-0259-3
Training set selection/reduction
Distributed machine learning
Training set partition
Network architecture design
Data variety
Class imbalance

redpajama_set_name: RedPajamaCommonCrawl


Minimum price for the offer must be greater than 60% of the Product.
What do you get when you combine Gio Ponti and Bill Curry? This lamp! But seriously it looks like a collaboration between the two. We have not had success in tracking down the maker of this fabulous lamp even though there is a sticker on the steel blue glass ball globe. Maybe you can make out what it says. No matter. This white painted wood conical base with its glass ball globe is simply stunning. Use it in your entryway. Use it on a sofa table, desk, or bookshelf. No matter where you use it, it will look fantastic and its ready to go. Just make it yours.


joe1927 Math instructor, Math and English Tutor, Researcher, Proof Reader, Researcher, C, C++, MATLAB, Minitab, MS Office, some auto repair (brakes, tune up) Legal Advice for a civil appeal I need to get my transcript made for an appeal that was approved but the court reporter refuses to do it and won't answer my attorney's calls or letter. I think he's dragging his feet also. I was told by an attorney that once I get my transcript made I'll be in a much better position, but the reporter and hence the court won't make it even though it was approved informa pauperis.


TiaSummer . paxican55. WilliamAndEverly. katycaKiss.
Verra4play1ZanderCuteGuyRainLexAgataFery3 .SUBMESSYBloomBoomLaurieFlemineTWOPASSIONSEX .MysteriousHotBoySundayMorningXpaulaDELIGHT69KatyHot95 .UrCaprizVerra4playDonaDearJhonnyfitness .EroticLaylaRoxyHoneyyYorgosHereLilsweetmess .MorenitalokaRainLexBonnieUrDreamBraxxtonBanx .DamiansdSammyNJulieMDxChrisFoxBrownyxxAngel .MarselNaughtyJuzzieDamiansdANDY8 .MorenitalokaJuzzieMagicDanteEmmaCerise .MoanBabyXBrownyxxAngelAllOverXRoxyHoneyy .CathyLikesPornBrownyxxAngelEmmaCeriselariisa .
ARTSofDOMINATIONAlishiaIvyEbonyxCandyMagicDante .WilliamAndEverlyAuroraClarkNatashaQuinnRoxyHoneyy .LustySonyaZarinaBlondieBrais1FETISHSLUTSLAV .AlishiaIvypaxican55LatinBeautyToULilsweetmess .UrPinoyFantasyTRANSCUMexploderDamiansdRainLex .AlishiaIvyNasttyRubyyGaryDrake1VirraGo .BonnieUrDreamGiaBarbaralikeCathyLikesPornGaryDrake1 .AllOverXNasttyRubyyWilliamAndEverlySundayMorningX .SBBoy1VirraGoSexyFabiHottieMom40 .ChrisFoxMAAHYRAMagicDanteEmmaCerise .English212ARTSofDOMINATIONSunnyRoxanalariisa .


Need lite to moderate garbage service at your home or business? The 100 gallon poly-cart with weekly service could be exactly what you need. We would be happy to give you a price quote by clicking here or by calling Garbage Gators at 806-622-7777.
The next size up from our 1.5yd dumpster is our Garbage Gator 3yd Dumpster. We provide a weekly service for this can. Rates can be quoted by clicking here or by calling Garbage Gators at 806-622-7777.
Maybe your remodeling your house, or whatever your large scale trash disposal need maybe we offer the Garbage Gator 30yd Roll-Off Garbage Container. Rates vary depending on mileage outside of the Amarillo/Canyon area and can be quoted by clicking here or by calling Garbage Gators at 806-622-7777.
PDF of our complete line of Garbage Gator Dumpsters.


New Zealand Probiotic Whey improves overall gut health and boosts the immune system. It also improves overall protein and nutrient bio-availability. Whey protein is an excellent protein choice for individuals of all ages who value the role of a healthy diet in helping to maintain and improve their health. Whey protein isolate, the purest form available, is unsurpassed as a source of the essential amino acids required in the daily diet. Essential amino acids are the building blocks for healthy muscles, skin, nails and other body tissue. Whey protein is a naturally complete protein, meaning that it contains all of the essential amino acids required in the daily diet. Protein helps to stabilize blood glucose levels by slowing the absorption of glucose into the bloodstream. This in turn reduces hunger by lowering insulin levels and making it easier for the body to burn fat.
The advantage of Quantum Nutrition Schinoussa Probiotic whey isolate is a greater absorption of daily nutrients, increased production of B-vitamins and more protein absorption. Protein with probiotics increases the known benefits of lean muscle tissue, fat loss, immune function, digestion & anti-aging.
Ingredients: Ingredients: Undenatured Whey Protein Isolate, Lactobacillus acidophilus, lactobacillus plantarum, Lactobacillus caseium, Lactobacillus bulgaricus, Lactobacillus rhamnosus, Lactobacillus brevis, Bifidobacterium bifidus, bifidobacterium longum, inulin, protease, natural berry, stevia.


text:
NZ Geodetic Marks
National Geodetic Office
This dataset provides information about the position, position accuracy, mark name, mark type, condition and unique four letter code for geodetic marks in terms of a New Zealand's official geodetic datum.
The dataset only contains marks that are within the New Zealand mainland and offshore islands. These positions have been generated using geodetic observations such as precise differential GPS or electronic distance and theodolite angles measurements. The positions are either 2D or 3D depending of the availability of this measurement data.
The source data is from Toitū Te Whenua Land Information New Zealand's (LINZ) Landonline system where it is used by Land Surveyors. This dataset is updated daily to reflect changes made in the Landonline.
Geodetic marks with a coordinate order of 5 or less have been positioned in terms of NZGD2000. Lower order marks (order 6 and greater) are derived from cadastral surveys, lower accuracy measurement techniques or inaccurate historical datum transformations, and may be significantly less accurate.
The accuracy of NZGD2000 coordinates is described by a series of 'orders' classifications. Positions in terms of NZGD2000 are described by three-dimensional coordinates (latitude, longitude, ellipsoidal height). The accuracy of a survey mark is indicated by its Order. Orders are classifications based on the quality of the coordinate in relation to the datum and in relation to other surrounding marks. For more information see
Note that the accuracy applies at the time the mark was last surveyed. Refer to the web geodetic database for historical information about mark coordinates.
Note also that the existence of a mark in this dataset does not imply that there is currently a physical mark in the ground - the dataset includes destroyed or lost historical marks. The geodetic database provides more information on the mark status, valid at last time it was visited by LINZ or a maintenance contractor.
Geodetic
id, geodetic_code, current_mark_name, description, mark_type, beacon_type, mark_condition, order, land_district, latitude, longitude, ellipsoidal_height,
608 - Browse all revisions
Imported on Jan. 27, 2023 from Postgres in NZGD2000.


When you're itching to get your great idea off the ground, it might be tempting to take money from wherever you can, but in the longer term this strategy could actually limit growth and cost you money.
Having a huge debt hanging over you, in the form of a bank loan or other funding that you have to pay back, has obvious issues for your cash flow. So what other options are there, and how will they affect you in the long run?
Firstly, while it might seem like a straightforward and comparatively low-risk strategy, trying to fund your company with the savings you have available can cause problems. At some point, the likelihood is that you'll hit a financial hurdle and will need to look for outside help as your business starts to get off the ground. If you've already used up your personal finance or equity options to get to where you are, you no longer have these as a last resort.
Desperation is never a good look, not least when you're looking for a bit of leverage with banks or other investors. While you might think you're limiting the potential damage to what you currently have in the bank, it's often a better idea to get a more reliable source of funding lined up early so that you aren't backed into a corner just as you need a break.
What's more, you do need to take into account your "burn rate" for cash in the business. While you might be able to launch your software or online empire from a back room with few outgoings, the same will not be true of a company based on manufacturing, which will need significant investment early on. No startup should be spending a fortune on frills, but trying to cut costs where they reduce quality may cost you more in the long run.
One of the better ways to deal with the issue is to look for investment that is based on equity share, rather than loans. This means that your funders are intrinsically involved in the success of the company and will do better out of the situation if you turn a decent profit, rather than looking to claw back a loan, even when this will hurts the business overall.
That said, giving away too much equity is potentially costly and dangerous in the long run. You'll need to make sure that you maintain executive control, and that you are not underselling part of your company if it's going to take off further down the line!
Including early buyback options may be a good idea, but more importantly, looking for a partner that offers far more than money means that you are not just giving away a chunk of your company, you are actively investing in the guidance, networks and strategic wisdom that can help your company to grow. A whole lot more exciting than negotiating repayment terms on a bank loan, and with far less risk to a startup's success.


5 trophies in 2 years. Has any manager won more in the last two season?
This entry was posted in Arsenal and tagged Arsenal, She Wore, She Wore A Yellow Ribbon, Unai Emery on May 25, 2018 by keenosafc.
A manager at any level can be a good at one club and then move to another and because of the new clubs board and players bomb out. To everybody who is raving about Emery I would advise, wait and see how he he performs at Arsenal. I hope to God it all works out for him, the players and the fans. In the meantime let's all have a bit of patience before committing ourselves.