text:
Q: Java's Fork/Join vs ExecutorService - when to use which? I just finished reading this post: What's the advantage of a Java-5 ThreadPoolExecutor over a Java-7 ForkJoinPool? and felt that the answer is not straight enough.
Can you explain in simple language and examples, what are the trade-offs between Java 7's Fork-Join framework and the older solutions?
I also read the Google's #1 hit on the topic Java Tip: When to use ForkJoinPool vs ExecutorService from javaworld.com but the article doesn't answer the title question when, it talks about api differences mostly ...

A: Brian Goetz describes the situation best: https://www.ibm.com/developerworks/library/j-jtp11137/index.html

Using conventional thread pools to implement fork-join is also challenging because fork-join tasks spend much of their lives waiting for other tasks. This behavior is a recipe for thread starvation deadlock, unless the parameters are carefully chosen to bound the number of tasks created or the pool itself is unbounded. Conventional thread pools are designed for tasks that are independent of each other and are also designed with potentially blocking, coarse-grained tasks in mind — fork-join solutions produce neither.

I recommend reading the whole post, as it has a good example of why you'd want to use a fork-join pool. It was written before ForkJoinPool became official, so the coInvoke() method he refers to became invokeAll().

A: Fork-join allows you to easily execute divide and conquer jobs, which have to be implemented manually if you want to execute it in ExecutorService. In practice ExecutorService is usually used to process many independent requests (aka transaction) concurrently, and fork-join when you want to accelerate one coherent job.

A: Fork-Join framework is an extension to Executor framework to particularly address 'waiting' issues in recursive multi-threaded programs. In fact, the new Fork-Join framework classes all extend from the existing classes of the Executor framework.
There are 2 characteristics central to Fork-Join framework

*

*Work Stealing (An idle thread steals work from a thread having tasks
queued up more than it can process currently)

*Ability to recursively decompose the tasks and collect the results.
(Apparently, this requirement must have popped up along with the
conception of the notion of parallel processing... but lacked a solid
implementation framework in Java till Java 7)

If the parallel processing needs are strictly recursive, there is no choice but to go for Fork-Join, otherwise either of executor or Fork-Join framework should do, though Fork-Join can be said to better utilize the resources because of the idle threads 'stealing' some tasks from busier threads.

A: Fork Join is an implementation of ExecuterService. The main difference is that this implementation creates DEQUE worker pool. Where task is inserted from oneside but withdrawn from any side. It means if you have created new ForkJoinPool() it will look for the available CPU and create that many worker thread. It then distribute the load evenly across each thread. But if one thread is working slowly and others are fast, they will pick the task from the slow thread. from the backside. The below steps will illustrate the stealing better.
Stage 1 (initially):
W1 -> 5,4,3,2,1
W2 -> 10,9,8,7,6
Stage 2:
W1 -> 5,4
W2 -> 10,9,8,7,
Stage 3:
W1 -> 10,5,4
W2 -> 9,8,7,
Whereas Executor service creates asked number of thread, and apply a blocking queue to store all the remaining waiting task. If you have used cachedExecuterService, it will create single thread for each job and there will be no waiting queue.

A: Fork-join is particularly good for recursive problems, where a task involves running subtasks and then processing their results.  (This is typically called "divide and conquer" ... but that doesn't reveal the essential characteristics.)
If you try to solve a recursive problem like this using conventional threading (e.g. via an ExecutorService) you end up with threads tied up waiting for other threads to deliver results to them.
On the other hand, if the problem doesn't have those characteristics, there is no real benefit from using fork-join.

References:

*

*Java Tutorials: Fork/Join.

*Java Tip: When to use ForkJoinPool vs ExecutorService:

A: Java 8 provides one more API in Executors
static ExecutorService  newWorkStealingPool()

Creates a work-stealing thread pool using all available processors as its target parallelism level.

With addition of this API,Executors provides different types of ExecutorService options.
Depending on your requirement, you can choose one of them or you can look out for ThreadPoolExecutor which provides better control on Bounded Task Queue Size, RejectedExecutionHandler mechanisms.

*

*static ExecutorService    newFixedThreadPool(int nThreads)

Creates a thread pool that reuses a fixed number of threads operating off a shared unbounded queue.

*static ScheduledExecutorService  newScheduledThreadPool(int corePoolSize)

Creates a thread pool that can schedule commands to run after a given delay, or to execute periodically.

*static ExecutorService   newCachedThreadPool(ThreadFactory threadFactory)

Creates a thread pool that creates new threads as needed, but will reuse previously constructed threads when they are available, and uses the provided ThreadFactory to create new threads when needed.

*static ExecutorService   newWorkStealingPool(int parallelism)

Creates a thread pool that maintains enough threads to support the given parallelism level, and may use multiple queues to reduce contention.

Each of these APIs are targeted to fulfil respective business needs of your application.  Which one to use will depend on your use case requirement.
e.g.

*

*If you want to process all submitted tasks in order of arrival, just use newFixedThreadPool(1)

*If you want to optimize performance of big computation of recursive tasks, use ForkJoinPool or newWorkStealingPool

*If you want to execute some tasks periodically or at certain time in future, use newScheduledThreadPool
Have a look at one more nice article by PeterLawrey on ExecutorService use cases.
Related SE question:
java Fork/Join pool, ExecutorService and CountDownLatch


text:
Q: Cron service fails to start I have an Ubuntu 14.04 and I am trying to start cron service without success. I always get this after running service cron start:
Job for cron.service failed because the control process exited with error code.
systemctl status cron.service says that the cron service is loaded but failed:
cron.service
   Loaded: loaded (/etc/init.d/cron; bad; vendor preset: enabled)
   Active: failed (Result: exit-code) since Fri 2018-06-15 09:06:19 AWST; 6min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 22707 ExecStart=/etc/init.d/cron start (code=exited, status=1/FAILURE)

 cron[22707]: Rather than invoking init scripts through /etc/init.d, use the service(8)
 cron[22707]: utility, e.g. service cron start
 cron[22707]: initctl: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused
 cron[22707]: Since the script you are attempting to invoke has been converted to an
 cron[22707]: Upstart job, you may also use the start(8) utility, e.g. start cron
 cron[22707]: start: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused
 systemd[1]: cron.service: Control process exited, code=exited status=1
 systemd[1]: Failed to start cron.service.
 systemd[1]: cron.service: Unit entered failed state.
 systemd[1]: cron.service: Failed with result 'exit-code'.

Any hint on how to make cron service to successfully start?

A: If cron service is not active, failed or trying to restart, then killing it shouldnt be a problem. Follow the steps:
Kill it using:

pkill -f crond

Start it using :

systemctl start crond

Check crond status:

systemctl status cron.service

A: I encountered similar issue, and this is what I was seeing
 systemctl status crond.service
● crond.service - Command Scheduler
   Loaded: loaded (/usr/lib/systemd/system/crond.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Thu 2019-02-07 13:34:38 EST; 2min 37s ago
  Process: 31470 ExecStart=/usr/sbin/crond -n $CRONDARGS (code=exited, status=1/FAILURE)
 Main PID: 31470 (code=exited, status=1/FAILURE)

What I did to resolve this by checking my cron log. I am running in rhel, and my log is at /var/log/cron, my issue was due to this
...(CRON) DEATH (can't lock /var/run/crond.pid, other pid may be 31364): Resource temporarily unavailable
so I just did
pkill -f crond

then
systemctl start crond.service

then it's back to normal
systemctl status crond.service
● crond.service - Command Scheduler
   Loaded: loaded (/usr/lib/systemd/system/crond.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2019-02-07 13:37:19 EST; 1s ago
 Main PID: 31490 (crond)
   CGroup: /system.slice/crond.service
           └─31490 /usr/sbin/crond -n

redpajama_set_name: RedPajamaStackExchange


text:
Q: how to match all return stmt from cxxMethodDecl node I want to find all the methods which params is reference type,
to add some codes before all return stmt.
Here is my code:
Matcher.addMatcher(cxxMethodDecl().bind("r"), &HandlerForReturn);

        const CXXMethodDecl *re = Result.Nodes.getNodeAs<CXXMethodDecl>("r");
        if(sourceManager->isWrittenInMainFile(re->getBeginLoc())) {
            if (re->getNameAsString() == "ChkMemCanUse") {
                for (auto i = 0; i < re->getNumParams(); i++) {
                    auto paramDecl = re->getParamDecl(i);
                    if (paramDecl->getType().getTypePtr()->isReferenceType()) {
                        //TODO
                    }
                }
            }
        }

I'm new to clang. My question is How can I find all return stmt from CXXMethodDecl?
or any other solutions?

A: Clang AST matchers are not really designed to match and more importantly bind a variable number of nodes.
So, I suggest to maintain your current code finding interesting methods and gathering all of the return statements on your own. It's actually pretty easy with clang::RecursiveASTVisitor template.
Here is how it can be done:
class ReturnCollector : public clang::RecursiveASTVisitor<ReturnCollector> {
public:
  static constexpr auto AVERAGE_NUMBER_OF_RETURNS = 5;
  using Returns = llvm::SmallVector<clang::ReturnStmt *,
                                    AVERAGE_NUMBER_OF_RETURNS>;

  static Returns collect(clang::CXXMethodDecl *MD) {
    ReturnCollector ActualCollector;
    ActualCollector.TraverseDecl(MD);
    return ActualCollector.Visited;
  }

  bool VisitReturnStmt(clang::ReturnStmt *RS) {
    Visited.push_back(RS);
    return true;
  }

private:
  ReturnCollector() = default;

  Returns Visited;
};

It can be used like this:
/// clang::CXXMethodDecl *MD
auto ReturnStmts = ReturnCollector::collect(MD);

llvm::errs() << "Returns of the '" << MD->getName() << "' method:\n";
for (auto *Return : ReturnStmts) {
  Return->dump();
}
llvm::errs() << "\n";

This code applied to the following snippet:
class A {
  int foo(int x) {
    if (x > 10) {
      if (x < 100) {
        return 20;
      }
      return x + x / 2;
    }
    return 10;
  }

  int bar() {
    return 42;
  }
};

produces this output:
Returns of the 'foo' method:
ReturnStmt 0x3e6e6b0
`-IntegerLiteral 0x3e6e690 'int' 20
ReturnStmt 0x3e6e7c0
`-BinaryOperator 0x3e6e7a0 'int' '+'
  |-ImplicitCastExpr 0x3e6e788 'int' <LValueToRValue>
  | `-DeclRefExpr 0x3e6e6f0 'int' lvalue ParmVar 0x3e6e308 'x' 'int'
  `-BinaryOperator 0x3e6e768 'int' '/'
    |-ImplicitCastExpr 0x3e6e750 'int' <LValueToRValue>
    | `-DeclRefExpr 0x3e6e710 'int' lvalue ParmVar 0x3e6e308 'x' 'int'
    `-IntegerLiteral 0x3e6e730 'int' 2
ReturnStmt 0x3e6e828
`-IntegerLiteral 0x3e6e808 'int' 10

Returns of the 'bar' method:
ReturnStmt 0x3e6e878
`-IntegerLiteral 0x3e6e858 'int' 42

I hope this will help you solving your problem!


text:
Q: Uploading a file to django model with a form - is_valid() fails I'm trying to upload a file to a django model with a form, however it fails when it reaches form.is_valid().
I'm adding the form to a Twitter Bootstrap modal, which I guess could be causing issues?
When I print form.errors, I get a message saying that the form is required. I'm obviously adding a text file to the form before submission, so I'm not sure what the issue could be.
views.py - Successfully gets past request.method == 'POST'. Fails at form.is_valid().
if request.method == 'POST':
    form = ConfigurationForm(request.POST, request.FILES)
    print("ERROR " + str(form.errors))
    if form.is_valid():
         print("Success")
    elif not form.is_valid():
        print("Failed")

forms.py:
from django import forms
from .models import NetworkApplications

class ConfigurationForm(forms.Form):
    config_file = forms.FileField(label="Select a file")

    class Meta:
        model = NetworkApplications

In my template:
<form  method="post" enctype="multipart/form-data">{% csrf_token %}
    {{form.config_file}}
</form>

request.FILES seems to be empty.
form.errors returns this:

config_file

This field is
required.

Footer:

    <button type="button" class="btn btn-default" data-dismiss="modal" onclick="destroyModal('newApplicationModal');">Close</button>
    <button type="button" id="newNetworkModalSave" class="btn btn-primary" onclick="addNewApplication();" data-dismiss="modal" >Save</button>


text:
Q: Clear function name that indicates it will return a Promise (JavaScript) I wanted to know if there was a naming convention that I should follow for functions that return a promise that clearly explained that .then() should be added if you expect to have the promise work.
I modified an old function addFilterForLanguage(lang) to return a promise for downloading that language's filter.  In the past, it wasn't an asynchronous task, and filter would be called before it had arrived.  I want a better name to clearly indicate that .then() is necessary.  We often have new interns (one of which wrote the original code) who might not know about promises, and often assume that it would/will "just work."
Is there a convention for this yet, or is addFilterForLanguage_ButDontForgetToCall_Then_AndPassInAFunctionIfYouWantItToRunToMakeItSyncronous(lang) the best I can do?
(I thought addFilterForLanguage(lang).then(myFunction) was clear enough, but you never know.)

A: The Bluebird Promise library adds Async as a suffix to a function name when you use Promise.promisifyAll on a Node-style callback function.
For instance, transforming fs.exists with promisifyAll will turn it into fs.existsAsync.
We've used that convention in any code where Promises need to interact with Node-style callbacks.

A: As far as I know, this is something that is either implemented as a coding guideline within teams or it's not. In the past, I have worked on teams, were we decided to add an Async suffix to anything that returned a promise, so in you case, addFilterForLanguageAsync.


text:
Q: Lapack's dgeqrf in c++ My project is about finding Q in QR decomposition for very large matrices (e.g. 500*500) in C++. I've started to use Lapack package recently and its especial function "dgeqrf". I started by a simple matrix as below in Code:blocks:
 #include <iostream>
 #include <lapacke.h>

using namespace std;

int main()
{
    double a[6][2] = {{0,2},{2,-1},{2,-1},{0,1.5},{2,-1},{2,-1}};
    int m=6;
    int n=2;
    int info = 0;
    int lda = m;
    int lwork = n;
    double *work;
    double *tau;
    dgeqrf_(&m, &n, a, &lda, tau, work, &lwork, &info);

}

After running the code, I saw this error in "dgeqrf" line:
error: cannot convert 'double (*)[2]' to 'double*' for argument '3' to 'void dgeqrf_(int*, int*, double*, int*, double*, double*, int*, int*)'

Who can help me about this error?Do I have mistake in parameter definition?
Also, after running, how I can work with the Q matrix? Can I define a new matix double q[][]=dgeqrf(....) and use it in my project?
I'm sorry if my question was very basic but I couldn't find the solution.

A: double a[12] = {0, 2, 2,  0, 2, 2,  // row1
                2,-1,-1,1.5,-1,-1}; // row2


text:
Q: Can not create object in Django views I'd like to create objects in a views, but can not figure out why I get 500 error instead.
Here is the relevant code:
lvt = LastVisitedTopic.objects.create(user=uid, topic=t.id, lastvisited=lv)

And the model is:
class LastVisitedTopic(models.Model):
    user = models.ForeignKey(User)
    topic = models.ForeignKey(Topic)
    lastvisited = models.DateTimeField(auto_now=True)
    class Meta:
    managed = True
    app_label = 'myforum'

The parameters passed to the create method seem to be ok:
print 'uid, t.id, lv \n', uid, t.id, lv

Yields:
    uid, t.id, lv
1 202798 2014-10-19 03:10:00+00:00

I'm stock on this for a while so really appreciate your clues.
UPDATE:
here is the whole views, in case it can help:
def notify_ajax(request):
    #if request.method == 'GET':
        args = {}
        alerts = []
        mt_alerts = []
        uid = request.user.id
        numalerts = 0

        if Topic.objects.filter(Q(post__creator_id=uid) | Q(creator_id=uid)).exists():
            mypartopics = Topic.objects.filter(Q(post__creator_id=uid) | Q(creator_id=uid)).distinct().order_by("-created")
            for t in mypartopics:
                print t.title
                #Check whether there is record of the topic in LastVisitedTopic
                if not LastVisitedTopic.objects.filter(topic_id=t.id, user_id=uid).exists():
                    print 'LastVisitedTopic not exists so to be created...'
                    #Calculate the record
                    #If topic has posts by this user
                    if Post.objects.filter(topic_id=t.id, creator_id=uid).exists():
                        print 'this topic has posts...\n'
                    #Last post time by this user is assumed as the last time he visited the topic
                        last_post = Post.objects.filter(topic_id=t.id, creator_id=uid).latest('created')
                        print 'latest post found'
                        lv = last_post.created
                        print 'lv to be added from post', lv
                        print 'uid, t.id, lv \n', uid, t.id, lv
                        lvt = LastVisitedTopic.objects.create(user=uid, topic=t.id, lastvisited=lv)
                        print 'last visit record created'
                    #Else last visit is assumed as the time when the topic is created
                    else:
                        lv = t.created
                        print 'lv added from topic'

                        lvt = LastVisitedTopic.objects.create(user=uid,topic=t.id, lastvisited='%s')

                        print 'last visit record created equal to topic creation time'

                else:
                    print 'record for this topic exists',
                    track = LastVisitedTopic.objects.get(topic_id=t.id, user_id=uid)
                    print 'last visited at', track.lastvisited

                #Check whether the topic lastposted field is not empty, if so, fill it
                if not t.lastposted:
                    if Post.objects.filter(topic_id=t.id).exists():
                        topic_lp= Post.objects.filter(topic_id=t.id).latest('created')
                        t.lastposted = topic_lp.created
                        t.save()
                        print 'new t.lastposted saved'
                    else:
                        t.lastposted = t.created
                        t.save()
                else:
                    print 't.lastposted for this topic exists', t.lastposted

                #Now compare last visit by the user with last topic post
                try:
                    print '\n\n\nnow trying...'
                    track = LastVisitedTopic.objects.get(topic_id=t.id, user_id=uid)
                    print 'track', track.id
                    last_visit = track.lastvisited
                    print 'last visit', last_visit
                    print 't.lastposted', t.lastposted

                    if t.lastposted > last_visit :
                        print 'topic posted after the last visit'
                        #last_posts = Post.objects.filter(topic_id=t.id).latest()
                        #if last_post.created >= last_visit:
                        alerts.append({'title':t.title, 'slug':t.slug})
                    else:
                        print 'lastposted before last visit'
                except:
                    print 't excpeton'
                    pass

            print 'alerts', alerts
            print 'alerst length', len(alerts)
            args['alerts'] = alerts
            return render(request, '_alerts.html', args)

A: Like the error says, you need an actual User, not an ID. You could just pass request.user directly. You'll probably get the same error for topic; again, you should just pass the object itself.
LastVisitedTopic.objects.create(user=request.user, topic=t, lastvisited=lv)


text:
Q: How to save a .html file to S3 in a Sagemaker processing container I am working in Sagemaker using python trying to profile a dataframe that is saved in a S3 bucket with pandas profiling. The data is very large so instead of spinning up a large EC2 instance, I am using a SKLearn processor.
Everything runs fine but when the job finishes it does not save the pandas profile (a .html file) in a S3 bucket or back in the instance Sagemaker is running in.
When I try to export the .html file that is created from the pandas profile, I keep getting errors saying that the file cannot be found.
Does anyone know of a way to export the .html file out of the temporary 24xl instance that the SKLearn processor is running in to S3? Below is the exact code I am using

import os
import sys
import subprocess
def install(package):
    subprocess.check_call([sys.executable, "-q", "-m", "pip", "install", package])
install('awswrangler')
install('tqdm')
install('pandas')
install('botocore==1.19.4')
install('ruamel.yaml')
install('pandas-profiling==2.13.0')
import awswrangler as wr
import pandas as pd
import numpy as np
import datetime as dt
from dateutil.relativedelta import relativedelta
from string import Template
import gc
import boto3

from pandas_profiling import ProfileReport

client = boto3.client('s3')
session = boto3.Session(region_name="eu-west-2")

%%writefile casetableprofile.py

import os
import sys
import subprocess
def install(package):
    subprocess.check_call([sys.executable, "-q", "-m", "pip", "install", package])
install('awswrangler')
install('tqdm')
install('pandas')
install('botocore')
install('ruamel.yaml')
install('pandas-profiling')
import awswrangler as wr
import pandas as pd
import numpy as np
import datetime as dt
from dateutil.relativedelta import relativedelta
from string import Template
import gc
import boto3

from pandas_profiling import ProfileReport

client = boto3.client('s3')
session = boto3.Session(region_name="eu-west-2")

def run_profile():

    query = """
    SELECT  * FROM "healthcloud-refined"."case"
    ;
    """
    tableforprofile = wr.athena.read_sql_query(query,
                                            database="healthcloud-refined",
                                            boto3_session=session,
                                            ctas_approach=False,
                                            workgroup='DataScientists')
    print("read in the table queried above")

    print("got rid of missing and added a new index")

    profile_tblforprofile = ProfileReport(tableforprofile,
                                  title="Pandas Profiling Report",
                                  minimal=True)

    print("Generated carerequest profile")

    return profile_tblforprofile

if __name__ == '__main__':

    profile_tblforprofile = run_profile()

    print("Generated outputs")

    output_path_tblforprofile = ('profile_case.html')
    print(output_path_tblforprofile)

    profile_tblforprofile.to_file(output_path_tblforprofile)

    #Below is the only part where I am getting errors
import boto3
import os
s3 = boto3.resource('s3')
s3.meta.client.upload_file('/opt/ml/processing/output/profile_case.html', 'intl-euro-uk-datascientist-prod','Mark/healthclouddataprofiles/{}'.format(output_path_tblforprofile))

import sagemaker
from sagemaker.processing import ProcessingInput, ProcessingOutput

session = boto3.Session(region_name="eu-west-2")

bucket = 'intl-euro-uk-datascientist-prod'

prefix = 'Mark'

sm_session = sagemaker.Session(boto_session=session, default_bucket=bucket)
sm_session.upload_data(path='./casetableprofile.py',
                                bucket=bucket,
                                key_prefix=f'{prefix}/source')

import boto3
#import sagemaker
from sagemaker import get_execution_role
from sagemaker.sklearn.processing import SKLearnProcessor

region = boto3.session.Session().region_name

S3_ROOT_PATH = "s3://{}/{}".format(bucket, prefix)

role = get_execution_role()
sklearn_processor = SKLearnProcessor(framework_version='0.20.0',
                                     role=role,
                                     sagemaker_session=sm_session,
                                     instance_type='ml.m5.24xlarge',
                                     instance_count=1)

sklearn_processor.run(code='s3://{}/{}/source/casetableprofile.py'.format(bucket, prefix),
                      inputs=[],
                      outputs=[ProcessingOutput(output_name='output',
                                                source='/opt/ml/processing/output',
                                                destination='s3://intl-euro-uk-datascientist-prod/Mark/')])

Error message I get:
"FileNotFoundError: [Errno 2] No such file or directory: '/opt/ml/processing/output/profile_case.html'"
Thank you so much in advance!!!

A: received the answer via Amazon re:post:
Hi,
Firstly, you should not (usually) need to directly interact with S3 from your processing script: The fact that you've configured your ProcessingOutput means that any files your script saves in /opt/ml/processing/output should automatically get uploaded to your s3://... destination URL. Of course there might be particular special cases where you want to directly access S3 from your script, but in general the processing job inputs and outputs should do it for you, to keep your code nice and simple.
I'm no Pandas Profiler expert, but I think the error might be coming from here:
output_path_tblforprofile = ('profile_case.html')
print(output_path_tblforprofile)

profile_tblforprofile.to_file(output_path_tblforprofile)

Doesn't this just save the report to profile_case.html in your current working directory? That's not the /opt/ml/processing/output directory: It's usually the folder where the script is downloaded to the container I believe. The FileNotFound error is telling you that the HTML file is not getting created in the folder you expect, I think.
So I would suggest to make your output path explicit e.g. /opt/ml/processing/output/profile_case.html, and also remove the boto3/s3 section at the end - hope that helps!


text:
Q: If I zero initialize a struct at the start of a loop using {0}, will it be zeroed out every iteration? Let's assume that I have a for loop, and a very large struct as a stack variable:
for (int x=0 ; x <10; x++)
{
    MY_STRUCT structVar = {0};
    …code using structVar…
}

Will every compiler actually zero out the struct at the start of every loop? Or do I need to use memset to zero it out?
This is a very large struct and I want to allocate it on the stack, and I need to make sure every member of it is zeroed out at the start of every iteration. So do I need to use memset?
I can manually inspect the executable that I compile, but I need to make sure if there is any standard for this, or it just depends on the compiler.
Note that this code does compile. I am using Visual Studio.

A:
Will every compiler actually zero out the struct at the start of every loop?

Any compiler that conforms to the C Standard will do this. From this Draft C11 Standard (bold emphasis mine):

6.8 Statements and blocks
…
3    A block allows a set of declarations and statements to be grouped into
one syntactic unit. The initializers of objects that have automatic
storage duration, and the variable length array declarators of
ordinary identifiers with block scope, are evaluated and the values
are stored in the objects (including storing an indeterminate value in
objects without an initializer) each time the declaration is reached
in the order of execution, as if it were a statement, and within each
declaration in the order that declarators appear.

In the case of a for or while loop, a declaration/initializer inside the loop's scope block is reached repeatedly on each and every iteration of the loop.

A: See 6.2.4, paragraph 6:

If an initialization is specified for the object, it is performed each
time the declaration or compound literal is reached in the execution
of the block; otherwise, the value becomes indeterminate each time the
declaration is reached

A:
Will every compiler actually zero out the struct at the start of every loop?

Yes, or it will produce machine code with equivalent functionality ("observable behavior") as if you had performed a zero-out.
As long as you initialize one single member in the struct, then the rest of them will get set to zero/null ("as if they had static storage duration"). Similarly, any padding bytes added to the struct by the compiler will get set to zero. This is guaranteed by the C standard ISO:9899:2018 6.7.9 §10, §19 and §21.
Generally, the place where the zero-out actually occurs in the resulting executable depends on how the data is used. If you for example zero the struct at the beginning of the loop body, then write to various members and print it all in the end of the loop body, the compiler don't have many other choices but to zero-out everything at each lap of the loop. Example:
for (int x=0 ; x <10; x++)
{
    MY_STRUCT structVar = {0};
    ...
    structVar.foo = a;
    structVar.bar = b;
    printf("%d %d\n", structVar.foo, structVar.bar);
}

On the other hand, the compiler might in this case be smart enough to realize that the struct is just a pointless middle man and replace this all with the equivalent printf("%d %d\n", a, b);, meaning that the struct would be removed entirely from the machine code.
Overall, discussing optimizations like this can't be done without a specific use-case, compiler and target system.

Or do I need to use memset to zero it out?

No. MY_STRUCT structVar = {0}; is functionally 100% equivalent of memset(structVar, 0, sizeof structVar);.

This is a very large struct and I want to allocate it on the stack

That's a different matter than initialization. It is indeed unwise to allocate large objects on the stack. In that case consider replacing it with for example this:
MY_STRUCT* structVar = malloc(sizeof *structVar);
for (int x=0 ; x <10; x++)
{
  memset(structVar, 0, sizeof *structVar);
  ...
}
free(structVar);

A: For the variables (of automatic storage class) defined within the body of a loop, the variables are (notionally) recreated for each iteration of the loop and have indeterminate values for each iteration if not initialized.
Regarding use of the {0} initializer for an object with automatic storage duration, note the following:

*

*As per 6.7.9/19 and 6.7.9/21,  elements of the object that have no explicit initializer will be initialized implicitly the same as object of static storage duration. As per 6.7.9/10, those elements will be initialized to value zero or a null pointer as appropriate, and any padding will be initialized to zero bits.

Using memset(ptr, 0, size) sets size bytes from address ptr onwards to 0, but note the following:

*

*For some unusual execution environments, a pointer object with all bytes zero might not represent a null pointer value (so might not compare equal to 0).

*For some unusual execution environments, a floating point object with all bytes zero might not represent a valid floating point value or might not compare equal to 0.0.
In summary, using the {0} initializer is the most portable way to set all elements of the object to compare equal to 0, and to set all padding bits or bytes to 0. Using memset instead is generally OK except for some weird execution environments.


text:
Q: This XML file does not appear to have any style information associated with it. The document tree is shown below I am using SPRING MVC. From the client to controller I did a PUT with the below xml.
<http-request method="PUT" url="/serviceproviders/{sp_id}/accessNumbers/{new_an_id}/languages" mime="application/xml">
        <content>
        <![CDATA[
         <additionalLanguages xmlns="http://www.XXX.com/pcs/model/schemas">
         <additionalLangIds>1003</additionalLangIds>
         </additionalLanguages>
        ]]>
        </content>

additionalLanguages is java class and declared like below
 @XmlRootElement(name = "additionalLanguages")
 @XmlAccessorType(XmlAccessType.NONE)
 public  class AdditionalLanguages implements Serializable{
     @XmlElement(name="additionalLangIds")
     private String sLanguageIDs = "";
     //getter & setter for sLanguageIDs
 }

Controller is returning view of
    "org.springframework.web.servlet.view.RedirectView".
And this view is inherently calling GET for the same url "/serviceproviders/{sp_id}/accessNumbers/{new_an_id}/languages" which is returning instance of AdditionalLanguages class.
Finally I am expecting to receive a response in browser "1003" which is nothing but
<HTML>
<BODY>
<additionalLanguages xmlns="http://www.XXX.com/pcs/model/schemas">
<additionalLangIds>1003</additionalLangIds>
</additionalLanguages>
</BODY>
</HTML>

But what i got is the below "warning and XML content" in the browser instead of "1003" value
This XML file does not appear to have any style information associated with it. The document tree is shown below.
<additionalLanguages xmlns="http://www.radisys.com/pcs/model/schemas">
<additionalLangIds>1003</additionalLangIds>
</additionalLanguages>

This output is not mapped with client expectation and This is causing the below error message at client side.
Execute action [repitition=0]: PUT -http://{SERVER}/PcsRESTApi/serviceproviders/1/accessNumbers/2020/languages
---HTTP RESPONSE---
HTTP Status: 500 - Internal Server Error
Response Headers:
  Date=Sat, 07 Jul 2012 23:19:45 GMT
  Server=Apache/2.2.16 (Unix) mod_jk/1.2.28
  errorCode=unknown
  errorMessage=Could not find acceptable representation
  Accept-Charset=big5, big5-hkscs, compound_text, euc-jp, euc-kr, gb18030, gb2312, gbk, ibm-thai, ibm00858, ibm01140, ibm01141, ibm01142, ibm01143, ibm01144, ibm01145, ibm01146, ibm01147, ibm01148, ibm01149, ibm037, ibm1026, ibm1047, ibm273, ibm277, ibm278, ibm280, ibm284, ibm285, ibm297, ibm420, ibm424, ibm437, ibm500, ibm775, ibm850, ibm852, ibm855, ibm857, ibm860, ibm861, ibm862, ibm863, ibm864, ibm865, ibm866, ibm868, ibm869, ibm870, ibm871, ibm918, iso-2022-cn, iso-2022-jp, iso-2022-jp-2, iso-2022-kr, iso-8859-1, iso-8859-13, iso-8859-15, iso-8859-2, iso-8859-3, iso-8859-4, iso-8859-5, iso-8859-6, iso-8859-7, iso-8859-8, iso-8859-9, jis_x0201, jis_x0212-1990, koi8-r, koi8-u, shift_jis, tis-620, us-ascii, utf-16, utf-16be, utf-16le, utf-32, utf-32be, utf-32le, utf-8, windows-1250, windows-1251, windows-1252, windows-1253, windows-1254, windows-1255, windows-1256, windows-1257, windows-1258, windows-31j, x-big5-solaris, x-euc-jp-linux, x-euc-tw, x-eucjp-open, x-ibm1006, x-ibm1025, x-ibm1046, x-ibm1097, x-ibm1098, x-ibm1112, x-ibm1122, x-ibm1123, x-ibm1124, x-ibm1381, x-ibm1383, x-ibm33722, x-ibm737, x-ibm834, x-ibm856, x-ibm874, x-ibm875, x-ibm921, x-ibm922, x-ibm930, x-ibm933, x-ibm935, x-ibm937, x-ibm939, x-ibm942, x-ibm942c, x-ibm943, x-ibm943c, x-ibm948, x-ibm949, x-ibm949c, x-ibm950, x-ibm964, x-ibm970, x-iscii91, x-iso-2022-cn-cns, x-iso-2022-cn-gb, x-iso-8859-11, x-jis0208, x-jisautodetect, x-johab, x-macarabic, x-maccentraleurope, x-maccroatian, x-maccyrillic, x-macdingbat, x-macgreek, x-machebrew, x-maciceland, x-macroman, x-macromania, x-macsymbol, x-macthai, x-macturkish, x-macukraine, x-ms950-hkscs, x-mswin-936, x-pck, x-utf-16le-bom, x-utf-32be-bom, x-utf-32le-bom, x-windows-50220, x-windows-50221, x-windows-874, x-windows-949, x-windows-950, x-windows-iso2022jp
  Content-Length=5243
  Connection=close
  Content-Type=application/xml
Response Body:
unknown: Could not find acceptable representation
org.springframework.web.HttpMediaTypeNotAcceptableException Could not find acceptable representation
    at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter$ServletHandlerMethodInvoker.writeWithMessageConverters(AnnotationMethodHandlerAdapter.java:954)
    at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter$ServletHandlerMethodInvoker.handleResponseBody(AnnotationMethodHandlerAdapter.java:898)
    at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter$ServletHandlerMethodInvoker.getModelAndView(AnnotationMethodHandlerAdapter.java:847)
    at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:423)
    at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:409)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:774)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)

My question is why the output is not a value? and why it is XML?
What i am missing here.
Kindly help me. I am struggling for long time. You are my only hope.

Resolution
Only add @ResponseBody when you are returning a value(object) directly from the controller. If you are performing a redirect url; don't add this annotations.

A: The output from the controller looks perfectly valid - from your description it looks like your "PUT" request worked cleanly, returned the RedirectView, and in the redirected method, it returned "AdditionalLanguages" which I am assuming you have annotated with @ResponseBody, which is the reason why Spring used a MessageConverter(in this this case Jaxb) to convert your AdditonalLanguages response to XML.
So in short, the output is a value because likely you have mapped the response from the redirected view to @ResponseBody.

A: rfc include='reference.I-D.ietf-anima-grasp'?>
<?rfc include='reference.I-D.greevenbosch-appsawg-cbor-cddl'?>
<?rfc include='reference.I-D.ietf-anima-bootstrapping-keyinfra'?>
<?rfc include='reference.I-D.ietf-anima-autonomic-control-plane'?>
<?rfc include='reference.RFC.3633'?>
</references> <references title="Informative References">
<?rfc include='reference.RFC.2629'?>
<?rfc include='reference.RFC.7575'?>
<?rfc include='reference.RFC.7576'?>
<?rfc include='reference.I-D.ietf-anima-reference-model'?>
</references>
</back>
</rfc


text:
Q: Pandas: compute Series or DataFrame permutations I have a dataframe of rows, indexed by name, and a single column containing a point tuple:
import pandas as pd
d = {'coords': {'a': (1, 2), 'b': (3, 4), 'c': (5, 6), 'd': (7, 8)}}
df = pd.dataframe(d)

What I'd like to do is retrieve data in the same way as if I ran itertools.permutations on the point data tuples, with a tuple-length of 2:
from itertools import permutations
list(permutations([(1, 2), (3, 4), (5, 6), (7, 8)], 2))

[((1, 2), (3, 4)),
 ((1, 2), (5, 6)),
 ((1, 2), (7, 8)),
 ((3, 4), (1, 2)),
 ((3, 4), (5, 6)),
 ((3, 4), (7, 8)),
 ((5, 6), (1, 2)),
 ((5, 6), (3, 4)),
 ((5, 6), (7, 8)),
 ((7, 8), (1, 2)),
 ((7, 8), (3, 4)),
 ((7, 8), (5, 6))]

The aim here is easy retrieval of point coordinates for any combination of two places (a, b --> (1, 2), (3, 4) etc.), but I have no idea how to calculate this, or whether I could use a MultiIndex to do it. An index-based solution would be ideal, because I'd also like to store data (e.g. a computed route) for each location pair.

A: Using your df as the starting point:
Index = list(permutations(df.index, 2))
new_df = pd.DataFrame({
         'route' : [[df.loc[Ind[0], 'coords'], df.loc[Ind[1], 'coords']] for Ind in Index]
                      }, index = Index)

Not sure if this is what you want, but this gives me this:
In [21]: new_df
Out[21]:
                   route
(a, b)  [(1, 2), (3, 4)]
(a, c)  [(1, 2), (5, 6)]
(a, d)  [(1, 2), (7, 8)]
(b, a)  [(3, 4), (1, 2)]
(b, c)  [(3, 4), (5, 6)]
(b, d)  [(3, 4), (7, 8)]
(c, a)  [(5, 6), (1, 2)]
(c, b)  [(5, 6), (3, 4)]
(c, d)  [(5, 6), (7, 8)]
(d, a)  [(7, 8), (1, 2)]
(d, b)  [(7, 8), (3, 4)]
(d, c)  [(7, 8), (5, 6)]


text:
Q: queueing in linux-htb I am trying to understand the queuing mechanism of linux-htb QDisc and QDisc of linux tc in general.
What I could gather:
During TX, the packet is queued into the queue inside the linux tc. This queue by default follows a pfifo_fast QDisc with a txqueuelen of 1000. The packet scheduler dequeues the packet from this queue and puts it onto the TX driver queue (ring buffer).
When we use linux-htb, the txqueuelen is inherited only for the default queue. [Link].
My question:
Consider the tree (rates are specified in kbits/sec in parenthesis ()):
      1:   root qdisc (class htb)
     (100)
     / | \
    /  |  \
   /   |   \
 1:1  1:2  1:3    parent qdiscs (class htb)
(30)  (10)  (60)

*

*Are there internal queues maintained for each of the parent htb classes (1:1, 1:2 and 1:3)? If yes what is their queue length? If not, how many queues are actually maintained and for what purpose? What is their queue length?

*What exactly is meant by Queueing Discipline (QDisc)? Is it the property of the data structure used (queue)? or it is a property of the packet scheduler? Or maybe both combined?

*While reading the source code of htb QDisc [Link], I came accross something called a direct queue. What is a direct_queue?
Provide link to relevant sources if possible.

A: Disclaimer: Those are a lot of questions and I haven't used HTB in like a decade? So I can't answer with confidence. But since you got zero replies so far, maybe this is still of some help.

Are there internal queues maintained for each of the parent htb classes (1:1, 1:2 and 1:3)?

They are leaf classes, and each is represented by a qdisc queue struct, so I assume that counts as internal queues. Not sure about the queue length - sorry.
Code from sch_htb.c:
struct htb_class_leaf {
    int     deficit[TC_HTB_MAXDEPTH];
    struct Qdisc    *q;
} leaf;

the Qdisc structure is defined in include/net/sch_generic.h

What exactly is meant by Queueing Discipline (QDisc)?

This depends on context but basically, it's a kernel API in which packets are enqueued and dequeued; so the QDisc is given some control about the order in which (or time when)  packets that come in should go out again (or dropped altogether, even). That is how QDiscs like HTB, SFQ or PRIO then shape traffic in various ways, like prioritize or impose bandwidth limits.
Comment from sch_api.c:
   Generally, queueing discipline ("qdisc") is a black box,
   which is able to enqueue packets and to dequeue them (when
   device is ready to send something) in order and at times
   determined by algorithm hidden in it.

And HTB is just one of several such algorithms.

What is a direct_queue?

Not part of the API but handled internally... so you could consider it part of the HTB algorithm.
If you deliberately classify packets to X:0 or if the default class doesn't exist, HTB decided to put them in a separate queue, and on dequeue it will attempt to send those packets out first.
Comment from sch_htb.c
 * [...] If we end up with classid MAJOR:0 we enqueue the skb into special
 * internal fifo (direct). These packets then go directly thru. If we still
 * have no valid leaf we try to use MAJOR:default leaf. It still unsuccessful
 * then finish and return direct queue.

...and this is where it dequeues direct packets first.
This is usually the result of a misconfiguration (sending packets to nonexistent class), but the HTB devs decided in that case, all traffic should be passed, instead of all traffic should be dropped (too destructive).

A: I am going to answer my own question since I have done some source code reading and research work myself. If I had not done some research work myself, the answers by frostschutz and sourcejedi would be of a great help too – they seem to be correct as far as my knowledge goes (although not into as much detail, but they give you a starting point to do the rest of the research yourself).
Some theory:
There are two kinds of queuing disciplines: classful and classless.
Classful disciplines are (as the answer by sourcejedi says) flexible. They allow you to attach children  classful qdiscs to them and can share bandwidth with other classes, when possible. Leaf classes have a classless qdisc (elementary/fundamental qdisc) attached to them (also called an elementary qdisc). The queues managed by these elementary qdiscs is where the packets get queued and dequeued. The packets are dequeued and enqueued from these classes by an algorithm corresponding to the class. Examples of classful qdiscs are: HTB and CBQ.
Classless qdiscs are the fundamental or the elementary qdiscs, which are rigid in the sense that they cannot have children qdiscs attached to them, nor can they share bandwidth. In naive terms, they are on their own. These qdiscs own a queue from which they queue and dequeue packets according to the algorithm corresponding the qdisc. Examples of classless qdisc: pfifo, bfifo, pfifo_fast (default used by Linux tc), tbf, sfq and a few more.
In the example tree in the question, each of the leaf htb classes 1:1, 1:2 and 1:3 has an elementary qdisc attached to it, which is by default pfifo (not pfifo_fast). The elementary qdisc attached to the leaf can be changed using the tc userspace utility in the following way:
tc qdisc add dev eth0 parent 1:11 handle 30: pfifo limit 5
tc qdisc add dev eth0 parent 1:12 handle 40: sfq perturb 10

More details about this can be found
in the HTB Linux queuing discipline manual.
Therefore the tree actually looks like this:
       1: root qdisc (class htb)
     (100)
     / | \
    /  |  \
   /   |   \
 1:1  1:2  1:3    parent qdiscs (class htb)
(30)  (10)  (60)
 ||    ||    || -----> pfifo qdiscs (queue length: txqueuelen (default, can be changed by tc utitlity))

Notice the parameter txqueuelen is an interface-specific parameter. That means the parameter is a property of the interface and can be changed using iproute2 or ifconfig. By default, its value is 1000. Here is an example of how to change it to 200 it via iproute2:
ip link set eth0 txqueuelen 200

When a leaf node is created (in context of HTB qdisc), pfifo qdisc is attached to the leaf class by default.
This pfifo is initialized with a queue limit of txqueuelen of the interface.
This can be found in the function htb_change_class()
in sch_htb.c, line 1395:
/* create leaf qdisc early because it uses kmalloc(GFP_KERNEL)
 * so that can't be used inside of sch_tree_lock
 * -- thanks to Karlis Peisenieks
 */
new_q = qdisc_create_dflt(sch->dev_queue, &pfifo_qdisc_ops,
                          classid, NULL);

For the default queue length of a pfifo queue, refer to sch_fifo.c, line 61:
u32 limit = qdisc_dev(sch)->tx_queue_len;

The kernel interacts directly with the root qdisc (maybe classful or classless) when it wants to queue or dequeue a packet. If the root qdisc is classful and has children, then it first classifies the packet (decides which child to send the packet to).
Kernel source where it is done: sch_htb.c, line 209:
static struct htb_class *htb_classify(struct sk_buff *skb, struct Qdisc *sch,
                                      int *qerr)

On reading the comments,
one can easily infer that this function returns one of the following:

*

*NULL, if the packet should be dropped

*-1, if the packet should be queued into direct_queue

*a leaf node
(which contains an elementary qdisc, where the packets actually end up)
This function traverses through all the interior nodes (Classes) of the tree
until it returns a leaf node, where the packet should be queued.
While dequeuing, each of the classes follow the algo associated with their qdisc to decide which of the children to dequeue from, and children do the same thing, until a packet is dequeued from an elementary qdisc attached to a leaf class. This also ensures that the rate of a child class is no more than its parent. (Since the parent will decide whether the packet will pass through or not). I have not gone through the source of dequeuing in htb, so I can't provide a source for that.
Direct queue: It is a special internal fifo queue maintained by the HTB qdisc from which the packets are dequeued at hardware speed. Its queue length is txqueuelen. A packet ends up in a direct queue if HTB is unable to classify it into one of the children qdiscs, and the default is not specified.
So the answers to my own questions:

*

*Yes, since they are leaf nodes, by default they are pfifo queues with queue length txqueuelen of the interface which is 1000 by default and can be changed.

*A queuing discipline is like the algo together with the queue combined in a single package! If you ask me, queuing discipline is property of both the type of queue and the packet scheduler (here packet scheduler means the algo which enqueues and dequeues the packet). For example, a queue might be of type pfifo or bfifo. The algo used for enqueuing and dequeuing is same, but the queue lengths are measured in bytes for the byte fifo (bfifo). Packets are dropped in a bfifo, when the byte limit is reached. The default byte limit is calculated as mtu*txqueuelen. When a packet is enqueued, for example, the packet length is added to the current queue length. Similarly, when the packet is dequeued, the packet length is subtracted from the queue length.

*Answered above.
Some sources one might consult:

*

*Linux Network Traffic Control — Implementation Overview (PDF)

*Journey to the Center of the Linux Kernel: Traffic Control, Shaping and QoS

*Demystification of TC - Criteo R&D Blog

*Queueing in the Linux Network Stack - Linux Journal

A: Some quick searches based on vague memories:
HTB is flexible. By default I think you get a FIFO for each leaf class, maybe it uses the default configuration for FIFO. However you can also e.g. use PFIFO or FQ_CODEL for a leaf class. See e.g. "optionally attach queuing disciplines to the leaf classes" here: http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm
I think you will see them if you run tc qdisc show.
"This 'direct' queue is only used if one of your filter explicitly targets the '0' classid of htb qdisc" https://lists.bufferbloat.net/pipermail/cerowrt-devel/2013-June/006507.html . The direct queue is not shaped. Apparently the length of the direct queue can now be controlled, although I do not know how (or whether) tc supports this. https://patchwork.ozlabs.org/patch/225546/
I would not say there is a separate "packet scheduler". A packet scheduler is a QDisc (although, man tc-tbf describes itself as "never schedules traffic"; what it means is that it never re-orders it).


text:
Q: Mongo: Import results of an aggregate query? Sorry for the basic question, I'm new to mongo and learning my way around.
I have run an aggregate query in Mongo:
> var result = db.urls.aggregate({$group : {_id : "$pagePath"} });
> result
{ "_id" : "/section1/page1" }
{ "_id" : "/section1/page2" }
...
Type "it" for more

I would now like to save the results of this aggregate query into a new collection. This is what I've tried:
> db.agg1.insert(result);
WriteResult({ "nInserted" : 1 })

But this seems to have inserted all the rows as just one row:
> db.agg1.count()
1
> db.agg1.findOne();
{ "_id" : "/section1/page1" }
{ "_id" : "/section1/page2" }
...
Type "it" for more

How can I insert these as separate rows?
I've tried inserting the _id directly, without success:
> db.agg1.insert(result._id);
2014-12-17T15:23:26.679+0000 no object passed to insert! at src/mongo/shell/collection.js:196

A: Use the $out pipeline operator for that:
db.urls.aggregate([
    {$group : {_id : "$pagePath"} },
    {$out: "agg1"}
]);

Note that $out was added in MongoDB 2.6.


text:
Q: How to replicate tables from multiple RDS databases into a single database? Here's the use case:

*

*We are going to have multiple Postgres DBs, each serving a different service. For instance, reviewDB serving review-service and mediaDB serving media-service. These DBs will be hosted in AWS RDS. They may share the same RDS instance, or some of them on dedicated RDS instances depending on the load.

*Meanwhile, I want to be able to sync all the tables from all those DBs into a separate database for analytics/report purposes(e.g. so that we can join tables from different DBs). Replication lag would be ok if it is less than 30 minutes.

Would this type of multi-db-to-single-db replication easy to set up on AWS? Like how easily AWS RDS allow us to set up H/A pair and read-replicas.


text:
Q: How to deal with a non-significant threshold value in ordinal regression in SPSS? I am running an ordinal regression in SPSS. I have independent variables on nominal and ordinal scales. My dependent variable is ordinal, indicating a level of action, coded 0, 1, 2. Unfortunately, I currently don't know how to deal with the non-significance of a threshold level for my dependent variable indicated by SPSS in the parameter estimates table.
The SPSS output of the parameter estimates looks like this:
Threshold       Level of action = 0     Estimate =  .6 Sig. = .143
Threshold Level of action = 1   Estimate = 2.2  Sig. = .000
Independent variables …
I read that "non-significance of a threshold suggests that the cutting point is not truly different and therefore some levels of the dependent variable need to be combined" (Garson, G. D. (2012). Ordinal Regression. Asheboro, NC: Statistical Associates Publishers).
I have not come across such a statement before. Also some ordinal regressions I had a look at did not mention an issue regarding a non-significant threshold and interpreted the output nonetheless.
So now I am wondering how severe that issue is. If I stick with ordinal regression, how would this non-significant threshold impact my results (given all other assumption are met)? Would they still be valid? Or does this mean that I definitely have to combine the outcome categories 0 and 1, which basically would mean I need to run binary logistic regression instead of ordinal regression?


text:
Q: Limiting regex match Say I have this HTML fragment

                Regular price:

                11,100.00 USD

                Web price:

                9,100.00 USD

How would I get the first amount (11,100.00) using regex?
I tried
/Regular price.+(\d[^\s]+)\sUSD/is

but it's not working, it's returning '00'. Obviously I'm a newbie at regex and I'm hoping to get by without picking up a book.
And I HAVE to use regex to parse this HTML because that particular site was designed by someone who doesn't know about classes or IDs.

A: You do not have to use regex. You can always search the DOM to get to the element you want, however, in this particular case (you're not parsing HTML, you're just searching for a certain string), your only issue is that your regex is greedy. Adding a ? will modify it to not be greedy (stop as soon as it finds a match, instead of getting the longest possible match):
/Regular price.+?(\d[^\s]+)\sUSD/is

A: Use .+? instead of .+. with the question mark you start the lazy operator.

A: TXR solution: ( http://www.nongnu.org/txr )
Script in data.txr:
@(skip)

@(skip)
                Regular price:

                @price USD

Run:
$ txr data.txr data.html
price="11,100.00"


text:
Q: How to restrict external access to ports exposed by Docker containers, while still allowing links to work? I have a couple services running on a machine in docker containers that need to be available to a specific IP address, and need to be available to containers with links (as defined in a docker-compose file). I've tried these commands, but this seems to also drop packets from containers on the same docker network:
iptables -I DOCKER-USER ! -s 8.8.8.8/32 -p tcp -m tcp --dport 1234 -j DROP

iptables -I DOCKER-USER ! -s 8.8.8.8/32 -p tcp -m tcp --dport 12345 -j DROP

How can I set up iptables to drop packets going to ports 1234 and 12345 from all ip addresses, except 8.8.8.8 and any container with links (as defined in a docker-compose file)?
docker-compose seems to create a bridge interface, and a lot of virtual interfaces with random names (one for each container?).
Here is the docker-compose.yml file I'm using:
version: '3'
services:
    django:
        build: .
        image: django
        depends_on:
            - mysql-db
            - sel
        ports:
            - "80:54321"
        links:
            - mysql-db
            - sel
            - rabbit
            - redis
        volumes:
            - vol1:/media/DataRaid1/vol1
            - /etc/secret_key.txt:/etc/secret_key.txt
            - django-media:/var/www/media
        environment:
            - DJANGO_SETTINGS_MODULE=a_project.settings_production
            - COMMIT
        command: httpd-foreground"
    mysql-db:
        image: mysql:5.7.18
        environment:
            - MYSQL_ROOT_PASSWORD=APASSWORD
        volumes:
            - mysql-data:/var/lib/mysql
            - ./custom-mysql.cnf:/etc/mysql/conf.d/custom-mysql.cnf
    sel:
        image: "selenium/standalone-chrome"
        volumes: #Fixes bug https://github.com/elgalu/docker-selenium/issues/20
            - /dev/shm:/dev/shm
    # Redis
    redis:
        image: redis:4.0.9
        hostname: redis
        ports:
            - "1234:1234"

    # RabbitMQ
    rabbit:
        hostname: rabbit
        image: rabbitmq
        environment:
            - RABBITMQ_DEFAULT_USER=AUSER
            - RABBITMQ_DEFAULT_PASS=APASSWORD
        ports:
            - "12345:12345"
    # Celery worker
    worker:
        build: .
        command: bash -c "sleep 10 && celery worker -A a_project.celeryconf -Q default -n default@%h"
        volumes:
          - .:/var/www
        links:
          - mysql-db
          - rabbit
          - redis
        depends_on:
          - rabbit
        environment:
            - DJANGO_SETTINGS_MODULE=a_project.settings_production
            - C_FORCE_ROOT=true
volumes:
    vol1:
    mysql-data:
    django-media:

A: *

*Use Network mode for each container:
network_mode: "host"

*Use ports with binding to 127.0.0.1 + some proxy container:
ports:
  - "127.0.0.1:12345:12345"

*Do not use ports at all + some proxy container

*Add iptables rule to appropriate docker chain, not sure which one right now
...


text:
Q: Controlling for individual differences in a repeated-measures design? I have been analysing a study where around 500 participants each were given a survey, so there were repeated measurements of each participant. Participants were male and female uni students who were presented with 4 character descriptions differing on gender (male/female) and health behavior (healthy/unhealthy), which they had to judge on, for example, weakness. Analyses I have conducted are paired-sample t-tests and two-way repeated measures ANOVAs. Now my supervisors want me to add control variables into the analysis, namely; participant gender, participant SES, participant sexual orientation, etc. by using an ANCOVA. However, I keep finding that all these individual differences in the participants are already controlled for by design, as repeated-measures ANOVA is an analysis within subjects thus individual differences don't play a role. This would mean that significance I have found now is valid and not confounded by the individual characteristics of the participants. Is this correct?
However, then I am confused on how to see whether the effect of the vignette on the dependent variable (e.g., weakness) differs according to participant gender and so on. Do I have to use a mixed ANOVA for this?
For clarification: Each participant read 4 vignettes with character descriptions, these descriptions depicted 2 x 2 manipulated actor variables: health behavior (healthy or unhealthy) x gender (male or female). So these were the independent variables, two categorical within-subject factors. Dependent variables were measured per vignette, for example, the weakness dependent variable was measured by averaging the scores on 4 Likert items (e.g., "To what extent does this person come across as insecure?"). Also, participant characteristics were measured (age, gender, health behavior, education, etc.).
Chunck of data:

PpNr = Participant number, Weak_C1X = Average weakness judgment of healthy male, Weak_C2X = Average weakness judgment of unhealthy male, Weak_C3X = Average weakness judgment of healthy female, Weak_C4X = Average weakness judgment of unhealthy female, SES = Socio-economic status of participant, Age = Age of participant, Gender = Gender of participant

A: It's not totally clear from your description what is your dependent variable that you analyze, but let's assume it's some continuous variable and you've measured it in three different situations. Let's say that's before the study starts, during the study when participants have received their first intervention and during the study when participants have received their second intervention, where there's two interventions and for each participants we randomly do A or B first followed by the other one.
One plausible analysis here is to fit a mixed effects model to only the on-intervention outcomes adjusted for the pre-intervention outcome such as (using R notation):
lme4::lmer( outcome ~ 1 + intervention + timepoint + pre_intervention_outcome + timepoint:pre_intervention_outcome + (1|Subject) )

One thing that happens here is that there's a random-subject effect on the intercept due to + (1|Subject) that says there's between-subject variation beyond what is explained by having the pre_intervention_outcome + timepoint:pre_intervention_outcome terms of the model. However, there usually will be remaining residual variation, which maybe could be explained by other explanatory variables like e.g. gender. One could adjust for that just as for the pre-intervention outcome. If one does not adjust for that, the extra unexplained variability is being absorbed into between subject or within subject variation.
With more timepoints, it becomes relevant to discuss how timepoints could be correlated. E.g. if one wants to leave that flexible something like a MMRM with an unstructured covariance matrix might be an option.


text:
Q: How to show '->' in front of the directory information in Linux when do ll -a? Actually as shown bellow I wondered
How to show '->' in front of the directory information in Linux when do ll -a?
Is it something related to repository/git or it is done by another person intentionally?

A: The printed line with the -> is a symbolic link. Not really a directory (or a standard file).
The name of the link is latest (size of 5 characters). The target of this link is 9.0.2 (in the same directory).
So it's done intentionally by another person or a install program.
Read man ln for futher information. ln is the command for create links.
There is no relation with Git or whatelse like it.


text:
Q: How to implement soft deletion in DataGridView I have a DataGridView bound to a object data source. The object has a property "IsDeleted". When the user presses the delete key, or clicks the delete button, or deletes a row in some other way, i want to set the "IsDeleted" flag instead of removing the row. (I then want the datagridview to update).
What is the single point of contact i need to achieve this behaviour?
I don't want to try to handle all the user paths seperately.

A: You can handle the event UserDeletingRow to Cancel it manually and perform your own deletion like this:
private void dataGridView1_UserDeletingRow(object sender, DataGridViewRowCancelEventArgs e){
   e.Cancel = true;//Cancel the actual deletion of row
   //You can just hide the row instead
   e.Row.Visible = false;
   //Then set the IsDeleted of the underlying data bound item to true
   ((YourObject)e.Row.DataBoundItem).IsDeleted = true;
}

You just said that your object has a property called IsDeleted, so I suppose it's called YourObject, you have to cast the DataBoundItem to that type so that you can access the IsDeleted property and set it to true. That's all.


text:
Q: Utilizando um map para armazenar 1 ID para N valores? EDIT por ausência de detalhes, repostei a questão aqui. A pergunta foi sinalizada e um moderador irá deletá-la assim que possível, obrigado.
Eu tenho os seguintes IDs e valores, respectivamente:
1 - 18;
1 - 19;
1 - 20;
3 - 21;
3 - 22;
8 - 23;
8 -24;
8 - 25;
11 - 26;
3 - 27;
3 - 28;

Como eu posso fazer para que, quando eu passe o valor do ID via parâmetro, ele me retorne os valores deste ID? POr exemplo, se eu passasse o ID 3, ele me retornaria os valores 21, 22, 27 & 28.
Eu deveria usar um map <integer, List<Integer>>, correto? Mas como eu posso fazer isso? Como também eu faria para adicionar estes valores dentro do meu List?

A: Voce pode utilizar HashMap.
 Faz o seguinte:
  HashMap<Integer,ArrayList<Integer>> map=new HashMap<Integer,ArrayList<Integer>>();
      //  chave = 3  ,  {1,22,27,28}
   map.put(3, new ArrayList<Integer>(Arrays.asList(1,22,27,28)));
   System.out.println( "Valores : " + map.get(3));
   // Resultado = "Valores : [1, 22, 27, 28]"

Onde 3 é a chave, basta trocar a chave e iniciar um novo ArrayList de Integer

A: Segue a lógica para gerenciar o map, para buscar o valor basta usar o get passando o id. Qualquer dúvida sobre o código é só perguntar.
Map<integer, List<Integer>> map  = new HashMap<integer, List<Integer>>();

public void adicionar(Integer id, Integer elemento){
  if(map.containsKey(id)){
     map.get(id).add(elemento);
  }else{
     List<Integer>> lista = new ArrayList();
     lista.add(elemento);
     map.put(id,lista);
  }
}


text:
Q: How can I generate MD5 hash for an Element in XML How can I generate MD5 hash for Place Element in XML. I want to calculate all the attributes and values in Place Element.

<?xml version="1.0" encoding="iso-8859-9"?>
<?hash 73533cd0fa219185648ff9554b065bd3?>
<Place Code="1111" Date="2016-06-18 11:28:58">
  <User Name="NAME" Number="1"/>
</Place>

//Place Element
        Element results = doc.createElement("Place");
        doc.appendChild(results);

        results.setAttribute("Code", "1111");
        results.setAttribute("Date", sdt);

        while((rs!=null) && (rs.next())) {

            Element row = doc.createElement("User");
            results.appendChild(row);

            // Attributes
            Attr xnumber = doc.createAttribute("Number");
            xnumber.setValue(rs.getString("number_db"));
            row.setAttributeNode(xnumber);

            Attr xname = doc.createAttribute("Name");
            xname.setValue(rs.getString("name_db"));
            row.setAttributeNode(xname);
    }

EDIT: MD5 code..
    String result = null;
    char hexDigits[] = {'0', '1', '2', '3', '4', '5', '6', '7',
            '8', '9', 'a', 'b', 'c', 'd', 'e', 'f'};

    MessageDigest md = MessageDigest.getInstance("MD5");
    md.update(source);
    //The result should be one 128 integer
    byte temp[] = md.digest();
    char str[] = new char[16 * 2];
    int k = 0;
    for (int i = 0; i < 16; i++) {
        byte byte0 = temp[i];
        str[k++] = hexDigits[byte0 >>> 4 & 0xf];
        str[k++] = hexDigits[byte0 & 0xf];
    }
    result = new String(str);

I use source = rs.getBytes("name_db"); for attributes and values but hash code not changing. What must I do?


text:
Q: JavaFx передача данных между окнами Каким образом можно обмениваться данными между окнами не по средствам сеттеров?
Я вроде как слышал что для более сложных приложений используется другой метод.

A: Можно привязать значения к Property переменным, привязать их значения к полям и изменять их. (гуглить IntegerProperty с остальными и Bindings)
twitchViewersLabel.textProperty().bind(InterfacePropertyHelper.twitchViewersProperty
Можно использовать схему с логикой обсервабла, когда изменения в одном месте программы вызывают метод в другом месте.


text:
Q: Rcpp fails to compile c++ code after series of updates I'd like to keep my OS, R, and R packages up to date. I unknowingly treaded into deep water by upgrading to OSX 10.15.6 and upgrading to R 4.0.2. Currently, Rcpp is failing to compile c++ code, which I believe is causing the CRAN installation for certain packages to fail (e.g. glmmTMB), and is also causing installations from source to fail. I'll describe what I've done and hopefully someone can elucidate a solution.

In a rough order, here's what I've done:

*

*Installed OSX 10.15.6

I don't remember the date but it was fairly recent. I don't know how closely this is related to the problem, I was having issues installing other R packages (e.g. rstan) that needed Rcpp prior to this.

*Ran into an issue trying to run some old glmmTMB models

The exact error was identical to this issue. I followed the various solutions on that thread to no avail. Perhaps the most frustrating was when attempting to install from source, which then failed to compile some c++.
This wasn't the first time I had seen similar errors. Something similar happened when I tried installing rstan, so I asked for some help on their repo under a similar issue. Nothing helped.
This was on R 3.6, so I thought maybe it'd be worth updating R and glmmTMB. Before doing this I wanted to install Xcode, as opposed to just the Command Line Developer Tools, in hopes that the installation would ensure I'd have everything clang- and c++ compiler-related squared away.

*Installed Xcode to help set a baseline c++ environment

Relatively painless, am able to build and compile fairly simple c++ projects.

*Installed R 4.0.2

From the r-project site, I installed R-4.0.2.pkg without issues and with the default install settings. According to the R project Tools site, I was reassured in installing Xcode and continued on to install the GNU Fortran 8.2 installer.

*Followed @coatless's guide to installing Rcpp

Discovered this guide and followed each step, except for those related to xcode-select.

*Testing Rcpp and RcppArmadillo from the guide

One of the last steps is to test the installation with a simple helloworld.cpp test. This fails with this error (abridged for clarity):
> Rcpp::sourceCpp("~/Documents/BergenLab/nlp_cancer_metaphor/helloworld.cpp")
In file included from helloworld.cpp:1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppArmadillo/include/RcppArmadillo.h:31:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppArmadillo/include/RcppArmadilloForward.h:26:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include/RcppCommon.h:29:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include/Rcpp/r/headers.h:67:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include/Rcpp/platform/compiler.h:100:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cmath:317:9: error: no member named 'signbit' in the global namespace
using ::signbit;
      ~~^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make: *** [helloworld.o] Error 1
clang++ -mmacosx-version-min=10.13 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include   -I"/Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppArmadillo/include" -I"/Users/alex/Documents/BergenLab/nlp_cancer_metaphor" -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include   -fPIC  -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -c helloworld.cpp -o helloworld.o
Error in Rcpp::sourceCpp("~/Documents/BergenLab/nlp_cancer_metaphor/helloworld.cpp") :
  Error 1 occurred building shared library.

*Tried to debug Rcpp to no avail

Found some somewhat related reports, such as Packages cannot build from source due to math.h not found, Rcpp doesn't under macOS, math.h not found, Source Cpp file failed due to math.h not found, and Cannot compile R packages with c++ code after updating to macOS Catalina.
None of the solutions in these answers have helped at all. The only solution which gave me feedback was using installer to install macOS_SDK_headers_for_macOS_10.14.pkg which subsequently gave me installer: Error - the package path specified was invalid (not sure how what the implications of this are).

*Installing glmmTMB from source fails

One of the solutions from an above glmmTMB issue recommends installing from source. Here is the error on my attempt:
* installing *source* package 'glmmTMB' ...
** package 'glmmTMB' successfully unpacked and MD5 sums checked
** using staged installation
** libs
clang++ -mmacosx-version-min=10.13 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.0/Resources/library/TMB/include' -I'/Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include' -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include   -fPIC  -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -c glmmTMB.cpp -o glmmTMB.o
In file included from glmmTMB.cpp:1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/TMB/include/TMB.hpp:53:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/Core:96:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/complex:245:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cmath:317:9: error: no member named 'signbit' in the global namespace
using ::signbit;
      ~~^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
13 warnings and 20 errors generated.
make: *** [glmmTMB.o] Error 1
ERROR: compilation failed for package 'glmmTMB'
* removing '/Library/Frameworks/R.framework/Versions/4.0/Resources/library/glmmTMB'
* restoring previous '/Library/Frameworks/R.framework/Versions/4.0/Resources/library/glmmTMB'
Warning in install.packages :
  installation of package 'glmmTMB' had non-zero exit status

Overall, I'm still unable to run my glmmTMB models (and I presume unable to build rstan as well). Something with my c++ compiler must be off but I'm a c++ beginner and am unsure how to interpret the errors I'm getting. Something's wrong with the header files, but it seems perhaps like it's not due to math.h being missing (like most other posts). It should go without saying but any help would be greatly appreciated!

Edit 1
I've tried to narrow down the error and thus followed each step from @coatless's guide, exactly as listed there and according to the R installation documentation. My exact steps:

*

*Removed gfortran 8.2. Removed ~/.Renviron and ~/.R/Makevars. Unlinked both of these files from Rstudio with unlink().

*Ran xcode-select --install, which returned with "use "Software Update" to install updates." Accordingly, softwareupdate --list estimated that nothing needed updating. Regardless, I uninstalled Command Line Tools with sudo rm -rf /Library/Developer/CommandLineTools and installed the Command Line Tools with xcode-select —install. Up to this point everything is working as expected. Small c++ programs are being compiled without issue by both gcc and clang++.

*According to the R manual, from https://mac.r-project.org/libs-4/, I installed pcre2-10.34-darwin.17-x86_64.tar.gz,  xz-5.2.4-darwin.17-x86_64.tar.gz , and readline-5.2.14-darwin.17-x86_64.tar.gz to /usr/local. No big issues here, although I'm not really sure how to verify that these work as expected.

*Restarted an R session in RStudio.

*Installed Rcpp and RcppArmadillo with install.packages(c('Rcpp', 'RcppArmadillo'))

*Created a ~/helloworld.cpp with the code from Slide 15 mentioned in the comments.

*Running Rcpp::sourceCpp("~/helloworld.cpp") fails with the exact error as mentioned in the body of the post. Similarly evalCpp() fails with a similar error.

A: After a bit of digging, the issue turned out to be neither glmmTMB, Rcpp, or really anything high-level. From what I understand: When R uses Rcpp to compile c++ code, a certain set of arguments are passed to the system c++ compiler. In my case this compiler was clang++ since I had just installed Xcode. The c++ compiler relies on a set of c++ header files to be included in exactly the right order. Seemed like the order of mine were incorrect, thus causing a slew of errors regarding "missing template definitions."
I looked into changing the order of the includes but couldn't find any straightforward solution. I knew that Rcpp was calling a very specific array of arguments to clang++, so I looked into how that might elucidate an answer. I looked into the manuals for clang++ to figure out what -isysroot and -I meant and felt like they might play a part in reconfiguring how the system header files were being compiled. From one of the threads below I discovered the $(R RHOME)/etc/Makeconf Makefile and proceeded to change the directory of the OSX SDK from what I believe was the Command Line Tools path to the Xcode path. This seemed to solve the issue.
Some threads I used to find this solution: Cannot compile R packages with c++ code after updating to macOS Catalina and Catalina C++: Using <cmath> headers yield error: no member named 'signbit' in the global namespace.
What worked for me:

*

*Note the current header file search

From clang++ -Wp,-v -E -:
#include <...> search starts here:
 /usr/local/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.3/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)

*Note the OSX SDK path

From xcrun --show-sdk-path:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk

*Modify the R configuration

With vim $(R RHOME)/etc/Makeconf, changed the CPPFLAGS flag to:
CPPFLAGS = -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -I/usr/local/include

Notice how this SDK directory is what was found in step 2. This apparently cleans up the location and order of the SDK header file includes.

*Reinstall glmmTMB from source

I was still seeing a FreeADFunObject error when trying to fit a glmmTMB model, so I went back to this GH thread and followed the first suggestion: to reinstall from source. This wasn't working before since I needed a properly set up c++ configuration, which I fixed in the last two steps.
Hence, install.packages("glmmTMB", type="source") installed without issue and I can now build and fit glmmTMB models.

For posterity's sake:
> sessionInfo()
R version 4.0.2 (2020-06-22)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Catalina 10.15.6

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base


text:
Q: Add spaces using kotlin string functions/string format This might been asked here couple of times.. what i am trying to do adding space between every four char of a string(8888319024981442). my string length is exactly 16.   String.format is not helpful
avoiding the usage of split or creating multiple strings in memory.
is there any kotlin function/String.format that can be quickly used.

A: I don't think there is a very simple way to do this, but there is the traditional one:
val number = "8888319024981442"
val list = mutableListOf<String>()
for (i in 0..3) { list.add(number.substring(i*4, (i+1)*4))}
println(list.joinToString(" "))

EDIT
Or @IR42 simple answer
number.chunked(4).joinToString(separator = " ")

A: I don't think there's an answer which is both simple and elegant and avoids all temporary objects.
For the former, IR42's use of chunked() is probably best.
Here's a stab at the latter:
val number = "8888319024981442"
val result = buildString {
    for (i in 0 until number.length) {
        if (i % 4 == 0 && i > 0)
            append(' ')
        append(number[i])
    }
}
println(result) // '8888 3190 2498 1442'

This creates only a single StringBuilder, and then a single String from it — which is the minimum possible*.  It's a bit ugly and long-winded, but if avoiding all temporary objects is really important**, then that's probably about the best you can do.
(* Or at least, the minimum possible given the conditions.  For better performance, consider passing the StringBuilder itself on without creating a String from it.  Even better, use an existing StringBuilder in instead of creating one at all.  But of course all that needs changes to the surrounding code.)
(** While there are situations in which this is really important, in practice they're pretty unusual.  I'd recommend going with the simple version until you've done some profiling and proved that this is a bottleneck and that the complex version really does perform better in your case.  And even then, fold it into a utility function to keep the main code clear.)


text:
Q: WordPress plugin output formatting I'm currently working on my very first Wordpress plugin, and it is coming along nicely. I do have one question about the formatting of the generated output.
At the moment, the plugin outputs a table with the standings of several teams of a sports-club. So for every team it shows the opponents, their ranking, games played, games won and number of points scored in the games (in Dutch "doelsaldo", but I don't know how to translate that properly...).
I have not added any type of styling to the table or it's cells, figuring that the installed theme should take care of this as much as possible. I am doing the development on a localhost-installation, using the default theme, and it is nice-looking. However, when I installed it to a test-site using a different theme, it is obvious that it lacks a lot of styling.
I'm guessing I'm not the first one that runs into this problem, so I was wondering what is the advised approach for this? Should I abandon the table-tag altogether (though in my opinion, this kind of data is exactly what should be put in a table), or is it advisable to arrange everything with inline-css? Maybe I'm missing another option altogether. I'm just interested in the preferred approach for this since there are literally thousands (if not millions) of different themes out there...

A: Take a look at this question (and answer) - both mine, related to the formatting of the settings input area.  Disable <th> $title in Plugin Options Screen?
The default is to put settings in a table, with cells for the label, input, and despcription. If you want to have another format (not tables ,perhaps some styling, etc), then take a look at my answer. (Hint: it's related to replacing do_settings() in the form with your own field rendering.)


text:
Q: CodeIgniter Change $db['xxx']['database'] have 1 mysql db, and 5 sql dbs.
Can I have this in config/database.php
$db['one']['hostname'] = 'localhost';
$db['one']['username'] = 'myuser';
$db['one']['password'] = 'mypass';
$db['one']['database'] = 'dbONE';
$db['one']['dbdriver'] = 'mssql';
$db['one']['dbprefix'] = '';
$db['one']['pconnect'] = FALSE;
$db['one']['db_debug'] = TRUE;
$db['one']['cache_on'] = FALSE;
$db['one']['cachedir'] = 'application/cache/';
$db['one']['char_set'] = 'utf8';
$db['one']['dbcollat'] = 'utf8_general_ci';
$db['one']['swap_pre'] = '';
$db['one']['autoinit'] = TRUE;
$db['one']['stricton'] = FALSE;

$active_group = 'default';
$active_record = TRUE;
$db['default']['hostname'] = 'localhost7';
$db['default']['username'] = 'root';
$db['default']['password'] = '';
$db['default']['database'] = 'dbMySql';
$db['default']['dbdriver'] = 'mysql';
$db['default']['dbprefix'] = '';
$db['default']['pconnect'] = FALSE;
$db['default']['db_debug'] = TRUE;
$db['default']['cache_on'] = FALSE;
$db['default']['cachedir'] = 'application/cache/';
$db['default']['char_set'] = 'utf8';
$db['default']['dbcollat'] = 'utf8_general_ci';
$db['default']['swap_pre'] = '';
$db['default']['autoinit'] = TRUE;
$db['default']['stricton'] = FALSE;

And want to change the value of $db['one']['database'] FROM dbONE to dbTWO on fly
is it possible or I must to write ALL DB configuration on config/database.php
Thanks in advance
I try this:
Case "dbONE":
    $db['one']['database'] = 'dbONE';
    $this->db_one = $this->load->database('one',TRUE);
    $d=$this->invoice_model->invo('001');
    break;

  case "dbTWO":
    $db['one']['database'] = 'dbTWO';
    $this->db_one = $this->load->database('one',TRUE);
    $d=$this->invoice_model->invo('001');
    break;

And always return the value from the db defined at databases.php

A: Have you try:
$db['one']['database'] = 'dbTWO';
$this->load->database($db);


text:
Q: Transferring Image (Downloaded from Firebase) into collection View Cell on home Page Im trying to transfer an image between two view controllers. It is a profile image of the current logged in User, if there is one. The image is successful in downloading from Firebase. It downloads on a launch screen along with the username. If there is no user logged in, view controller changes to a login screen, if there is a logged in user it transfers to a home screen.
This is the launch screen. Both the username label and the image view are clear so user cannot see.
import UIKit
import Firebase

class LaunchScreen: UIViewController {

@IBOutlet weak var usernameLabel: UILabel!
@IBOutlet weak var userImage: UIImageView!

var ref : DatabaseReference!
var storage: StorageReference!

override func viewDidLoad() {
    super.viewDidLoad()
    ref = Database.database().reference()
    storage = Storage.storage().reference()

    checkUser()

    // Do any additional setup after loading the view.
}

func checkUser() {

    Auth.auth().addStateDidChangeListener { (auth, user) in

        if user != nil {
            let uid = Auth.auth().currentUser?.uid
            self.ref.database.reference().child("users/profile").child(uid!).observe(.value, with: { (snapshot: DataSnapshot) in

                if let dictionary = snapshot.value as? [String: AnyObject] {
                    let currentUser = dictionary["username"] as! String
                    let url = dictionary["photoURL"] as! String

                    let storageRef = self.storage.storage.reference(forURL: url)
                    storageRef.getData(maxSize: 1 * 1024 * 1024, completion: { (data, error) in
                        if let error = error {
                            print(error)
                        } else {
                            self.userImage.image = UIImage(data: data!)
                            self.reloadInputViews()
                        }
                    })

                    guard let image = self.userImage.image else {return}
                    self.usernameLabel.text = currentUser
                    let imageJPG = image.jpegData(compressionQuality: 1)
                    print(imageJPG?.count)

                    let defaults = UserDefaults.standard
                    defaults.set(self.usernameLabel.text, forKey: "user")
                    defaults.synchronize()

                }
            })
            _ = Timer.scheduledTimer(withTimeInterval: 3.0, repeats: false, block: { (timer) in
                let vc = self.storyboard?.instantiateViewController(withIdentifier: "homeVC")
                self.present(vc!, animated: true, completion: nil)

            })

            print("loggedIn")

        } else {
            _ = Timer.scheduledTimer(withTimeInterval: 3.0, repeats: false, block: { (timer) in
            let loginvc = self.storyboard?.instantiateViewController(withIdentifier: "loginVC")
            self.present( loginvc!,animated: true, completion: nil)

            })
            print("loggedOut")

        }

    }
}

}
This is my home page. where there is a side menu. the side menu pops open from the left hand side of the screen and on it are four collection view cells. the top one is the profile cell which will display the username and the user profile image. that image is where I want the downloaded Firebase userImage in launchScreenVC to be transferred too.
class home VC:  UIViewController {

  @IBOutlet weak var profileMenuButton: UIButton!

   override func viewDidLoad() {

    super.viewDidLoad()
    collectionView.delegate = self
    collectionView.dataSource = self
    collectionView.reloadData()
    }

  let blackView = UIView()
let collectionView: UICollectionView = {
    let layout = UICollectionViewFlowLayout()
    let cv = UICollectionView(frame: .zero , collectionViewLayout: layout )
    cv.backgroundColor = UIColor.init(displayP3Red: 50/255, green: 51/255, blue: 51/255, alpha: 1)
    return cv
}()

@IBAction func profileMenu() {

    let width = view.frame.width - 60

    collectionView.frame =  CGRect(x: 0, y: 0, width: width, height: view.frame.height)

    blackView.backgroundColor = UIColor(white: 0, alpha: 0.5)
    blackView.addGestureRecognizer(UITapGestureRecognizer(target: self, action: #selector(handleDismiss)))
    blackView.frame = view.frame
    blackView.alpha = 0
    view.addSubview(blackView)
    view.addSubview(collectionView)
    UIView.animate(withDuration: 1.5, delay: 0, options: .curveEaseOut, animations: {
        self.blackView.alpha = 1
        self.collectionView.frame = CGRect(x: 0 , y: 0, width: width, height: self.view.frame.height)
    }, completion: nil)
        handleSearchDismiss(searchView)

}

@objc func handleDismiss() {
    UIView.animate(withDuration: 0.5){

        if let window = UIApplication.shared.keyWindow {
            self.collectionView.frame = CGRect(x: 0 - (window.frame.width), y: 0, width: window.frame.width, height: self.collectionView.frame.height)
        }

        self.blackView.alpha = 0

    }
}
}

extension homeVC: UICollectionViewDataSource, UICollectionViewDelegate, UICollectionViewDelegateFlowLayout, {

func collectionView(_ collectionView: UICollectionView, cellForItemAt indexPath: IndexPath) -> UICollectionViewCell {

    if indexPath.item == 0 {
        let cell1 = collectionView.dequeueReusableCell(withReuseIdentifier: "ProfileCellID", for: indexPath) as! profileCell
        let defaults = UserDefaults.standard
        cell1.usernameLabel.text = (defaults.object(forKey: "user") as! String)
        cell1.usernameLabel.textColor = UIColor.lightGray
        cell1.usernameLabel.font = UIFont.boldSystemFont(ofSize: 25)
//            print(imageData)
//            cell1.profileImage.image = (defaults.object(forKey: "userImage") as! Data)

        return cell1
    } else if indexPath.item == 1{
        let cell2 = collectionView.dequeueReusableCell(withReuseIdentifier: "Settings", for: indexPath) as! settingsCell

        return cell2
    } else if indexPath.item == 2 {
        let cell3 = collectionView.dequeueReusableCell(withReuseIdentifier: "Favourites", for: indexPath) as! favouritesCell

        return cell3
    } else {
        let cell4 = collectionView.dequeueReusableCell(withReuseIdentifier: "Logout", for: indexPath) as! logoutCell

        return cell4
    }

 }
}

this is the profile cell class
class profileCell: UICollectionViewCell {

override init(frame: CGRect) {
    super.init(frame: frame)
    setUpViews()
}

let profileImage: UIImageView = {
    let imageView = UIImageView()
    return imageView
}()

let usernameLabel: UILabel = {
  let label = UILabel()
    return label
}()

func setUpViews() {

    addSubview(profileImage)
    addSubview(usernameLabel)
    profileImage.frame = CGRect(x: (center.x)-60,  y: 30, width: 120, height: 120)
    profileImage.layer.cornerRadius = (profileImage.frame.width/2)
    usernameLabel.frame = CGRect(x: (center.x)-60, y: 125, width: 120, height: 120)
    usernameLabel.textAlignment = NSTextAlignment.center

}

required init?(coder aDecoder: NSCoder) {
    fatalError("init(coder:) has not been implemented")
}

}

Any Help or tips would be greatly appreciated! Thanks Guys!


text:
Q: Cannot have plain array after removal ones in multi dimensional array How do we have plain array after ones in multi dimensional array are removed ?
@p=();
my @l=([9,7],[3,4]);

push(@p,['FOO',[@l]]);
push(@p,['BAR',[[8,0]]]);
splice(@$_,0,1) for @p;
print "\n",@$_ for @p;

ARRAY(0x5624cf01e340)
ARRAY(0x5624cf028d78)

How do we have it simple/plain 2D array with number of 1x2 size on inner/2nd dimensional one. In other words, how linearize it to such?

A: Have to de-reference elements which are array-references, and copy data. For a single level:
@ary = map { ref eq "ARRAY" ? @$_ : $_ } @ary;

The ref builtin by default takes $_.  Also consider using reftype from Scalar::Util, instead.  Then also look into is_arrayref from Ref::Util.  In simple and straighforward cases the builtin is fine.
In case there are deeper levels use the above on each level. For the question's quest, to retain one level of nesting from the two-level deep input data:
for my $elem (@p) {
    @$elem = map { ref eq "ARRAY" ? @$_ : $_ } @$elem;
}

Since $elem aliases the current element of @p this code changes the array in place.
If there are further levels, of possibly varying depth, can handle that either recursively or iteratively.  Crude implementations:
Recursive:
sub flatten_recurse {
    return map { ref eq 'ARRAY' ? flatten_recurse(@$_) : $_ } @_
}

This could be expensive on large data structures (but worry only if that is indeed the case).
Iterative:
sub flatten_iter {
    my @deque = @_;
    my @flat;
    while (@deque) {
        my $front = shift @deque;
        if (ref($front) eq 'ARRAY') {
            unshift @deque, @$front;
        } else {
            push @flat, $front;
        }
    }
    return \@flat;
}

Here one can also use pop and push, in varying combinations. The above keeps the order, as much as that makes sense when reducing a complex structure to a flattened one.
If there are yet other complexities, like other types of references (themselves containing arrayrefs to be unpacked), then this need be adjusted accordingly.  One would also need to add checks, in particular for circular references.  See for instance this SO page and this perlmonks page.


text:
Q: XSLT How to access the value of a matched node in a template I want to be able to access the 'value' of the node I am matching on in my template.
 <factfind>
   <myelement>This is a value I want to retreive</myelement>
   ....
   ....
 </factfind>

Basically, I want to use the following template (something like it)
 <xsl:template name="get-my-element" match="myelement">

   <xsl:value-of select="$this"/>
 </xsl:template>

I know I could match a template on <factfind> but the problem is I already have a template that is matching that element and handling it in its own way. If I make two templates that handle the one node, only the first one is called.
So you might know the context a bit better, here is how I have it setup atm.
 <xsl:template match="loggedin">

      <xsl:apply-templates select="entities"/>

      <xsl:apply-templates select="menuitem"/>

      <xsl:apply-templates select="factfind" />

 </xsl:template>

The menuitem template would like to make a call-template to the get-my-element template to access the information inside of it, without effecting the behaviour of my factfind template.

A: Simple, the dot expression gives you the context item:
<xsl:value-of select="."/>


text:
Q: How do I determine the smallest dimension of an irreducible $\mathbb{F}_p[G]$-module with a prescribed trivial fixed point space? This is a crosspost from MSE since I haven't found an answer there yet.
I am not very familiar with modular representation theory or Brauer theory yet, however lately I have needed to use $\mathbb{F}_p[G]$-modules for constructing counterexamples.  Right now I am just making them in Magma, but I am looking to understand what I'm doing a little better.
Say that I am given a group $G$ which I would like to act on an elementary abelian $p$-group $E$ so that a prescribed proper subgroup $H\leqslant G$ acts fixed point freely on $E$, with $|H|$ and $p$ coprime.  So, I construct an absolutely irreducible $\mathbb{F}_p[G]$-module with a trivial fixed point space under restriction to $H$.  Often I have other subgroups which I would explicitly like to not act fixed point freely on $E$ (usually faithfully or trivially) in which case I also insist the module have a nontrivial fixed point space under restriction to those subgroups.
I am trying to figure out how to determine the smallest dimension of such a module - in other words, I want to know how big $E$ has to be.  Obviously this is going to depend on the structure of $G$, and in particular the structure of $H$.  Which group properties determine this, precisely?  Does anyone know where I could read about this specifically?
There is some existing theory.  For example, one can prove that if $F$ is a Frobenius group with cyclic complement $C$, and $F$ acts faithfully on an elementary abelian $p$-group $E$, then $|C|$ divides the minimum dimension of $E$ over $\mathbb{F}_p$.  Are there more general results?