text:
Q: Eclipse import from required project not working i have a project in my workspace that i build with maven following these instructions: http://sweble.org/repos/
Now i want to make another project that uses some classes from that project. I added the sweble project as a required project to my new project. (Properties->Java Build Path->Projects->Add).
The classes from the sweble project are now suggested via autocomplete and if i use them it automatically adds imports to my class, but after saving that class those imports are marked with the error "The import org.sweble cannot be resolved".
I cannot resolve this error. I find it very strange that autocomplete works but the actual import doesn't.
The src folder of the sweble project is called src/main/java and it contains the package "org.sweble.wikitext.dumpreader" the import that gets suggested is called: "import org.sweble.wikitext.dumpreader.DumpReader;"
Does anybody know how to resolve this error or what might be misconfigured here?

A: Are you sure you have to pull the git repo? maybe you can just add Sweble like any other mavenized project to your project's pom.xml? see http://mvnrepository.com/artifact/org.sweble/sweble/1.1.0
<dependency>
    <groupId>org.sweble</groupId>
    <artifactId>sweble</artifactId>
    <version>1.1.0</version>
</dependency>


text:
Q: Problem with python looping when I have no loop in my code I have a main program that calls some modules. For some reason when I run the code it loops over parts of the main code when there is no loop in the code.
import os
import datetime
import multiprocessing as mp
import shutil

#make temp folder for data files
date = str(datetime.datetime.now())
date = date[0:19]
date = date.replace(':', '-')
temp_folder_name = 'temp_data_files_' + date
os.mkdir(temp_folder_name)

#make folder for sim files
date = str(datetime.datetime.now())
date = date[0:19]
date = date.replace(':', '-')
save_folder_name = 'Sim_files_' + date
os.mkdir(save_folder_name)

#make data files and save in temp folder
import make
make.data('model_1',temp_folder_name) #model name and folder for results

#run file on multiple cores
import distributed
corecount = mp.cpu_count() # edit this value to the number of cores you want to use on your computer
if __name__ == '__main__':
    distributed.simulate(temp_folder_name, corecount ,save_folder_name)

The program should make two folders. It then uses 'make' to make some files and put them in the temp folder. It then should use 'distributed' to do some things with the files and save them in the 'sim_files' folder. But for some reason it makes several folders in each instance (with slightly different time stamps).
The distributed function includes some links but I don't think these should have an effect on the main program.
The if __name__ == ... line is to do with multiprocessing a guard against infinitely looping

A: I have found a solution to this. It is to do with the way multiprocessing works, the child processes import the main program like modules. This lead to the main program being run for each instance of multiprocessing.
The solution is to move the if __name__ == '__main__': line to the start of the main. This ensures that it only runs when it is being called as itself, rather than when it is imported like a module :)

redpajama_set_name: RedPajamaStackExchange


text:
Q: Stack Exchange Format for Corporate Networks I think the Stack Exchange format is great and would like to suggest to my employer that they implement an internal SE for collaboration on our processes. Obviously the network would need to be private to just our company due to proprietary and confidential information.
Does SE license their format that we could purchase a corporate license for?
If the answer is no, then the follow up question; is SE format protected by intellectual property rights that would prevent us from implementing our own similar system?


text:
Q: Warning: Value was defined but never used; consider replacing with a boolean test I have a function that is getting the following warning:

Value intVal was defined but never used; consider replacing with a boolean test.

func textField(textField: UITextField, shouldChangeCharactersInRange range: NSRange, replacementString string: String) -> Bool
{
    text = (timerTxtFld.text! as NSString).stringByReplacingCharactersInRange(range, withString: string)
    if let intVal = Int(text) {
        timerDoneBtn.alpha = 1
        timerDoneBtn.enabled = true
    } else {
        timerDoneBtn.enabled = false
    }
    return true
}

Could anyone help me work out what I need to do to get rid of the error?

A: Just remove the let and do the comparison on the result of Int directly. You're creating intVal for no reason, and it's complaining that that's an unused variable.
func textField(textField: UITextField, shouldChangeCharactersInRange range: NSRange, replacementString string: String) -> Bool
{
    text = (timerTxtFld.text! as NSString).stringByReplacingCharactersInRange(range, withString: string)
    if Int(text) != nil
    {
        timerDoneBtn.alpha = 1
        timerDoneBtn.enabled = true
    }
    else
    {
        timerDoneBtn.enabled = false
    }
    return true
}

A: It's not an error, it's a warning. The compiler is telling you that you created the const intVal but never used it.
Just change your if statement to
if Int(text) != nil
{

}


text:
Q: Java preferences API throughput I'm using Java Preferences API to store window position and size of Swing application. At this moment, I'm listening to window resize/reposition events and storing the position and size every time they change. However, that means that if user slowly resizes window which is 200px wide to 400px wide, I'll probably write new window size about 200 times during pretty short time.
Preferences API uses whichever datastore is available on the host system (windows registry for Windows etc.) - but the question is, what are limitations or best practices for properties API? Is it OK, or would it be smart to write only when user has finished resizing? Anyone had experiences with Properties API on different platforms?

A: Just re-read and realised you're talking about the app itself not a dialog, but the concept probably still applies: record the dimensions only on successful application shutdown.  I think this would be perfectly fine for most users / situations.

Why not record the new size only on "OK" or "Cancel" button event? I.e. don't store it dynamically.
If it's modal, you cannot lose anything by doing this, it's just the final value you want. Who cares if it was 307 px for a millisecond?


text:
Q: Why am I receiving a NaN on parsing an number in Javascript I can't find why I'm receiving a NaN for printing a number out with javascript.
The following code I use is repeated elsewhere on the website and works fine.
The URL in question is: http://all-american-gold10.mybigcommerce.com/us-silver-eagles/
My code is as follows:
    function setCheckPrice() {
  $('.NormalPrice').each(function(key, val) {
  var price = $(val).html().replace('$', '').replace(',', '');
  var newPrice = parseInt(price) * .03;
  var setPrice = Math.floor(price - newPrice);

        var credit = ' - credit card price';
        var check = '
$' + setPrice +' - check price
';

        $(val).append(credit);

        $(val).prepend(check);

        })

};

setCheckPrice();

My code takes a number found by a specific class name, does some math and spits in back out with some extra html to go along with it. I've tried to switch the ways I've outputted the code with append, prepend, before & after, but none seem to work? Any reason why this is acting so strange?

A: The price variable has additional html in it. You should consider having separate elements with a class that contains just the value that you need, and nothing else. As a quick fix, this change:
var price = $(val).val().replace('$', '').replace(',', '').replace(" ","");

get's me a value of 490 for price on that page, which is what you're looking for.


text:
Q: Hide an option on dropdown menu using javascript <select class="bookly-select-mobile bookly-js-select-employee">
    <option value="">Any</option>
    <option value="1">Brandon Davis</option>
    <option value="2">Ryan Floyd</option>
</select>

is there a javascript that can hide the "Any" option?

A: All you have to do is set display: none in CSS. One way to do this (assuming anything with a blank value should be hidden) is to use option[value=''] as your selector.
Of course, if you want to do this dynamically (in JS), you'll have to have some way to target the element.

A: I would suggest you do it with hidden attribute. Documentation here: https://www.w3schools.com/tags/att_global_hidden.asp

<select class="bookly-select-mobile bookly-js-select-employee">
    <option hidden value="">Any</option>
    <option value="1">Brandon Davis</option>
    <option value="2">Ryan Floyd</option>
</select>


text:
Q: Failed to load resource: net::ERR_CONNECTION_REFUSED & TypeError: Failed to fetch Have tried everything to connect my React frontend to my express backend (MongoDB). When I run nodemon server and go to URL "http://localhost:5000/api/projects/" I get full json list of data. When I fetch the same link in the frontend I get the errors: Failed to load resource:

net::ERR_CONNECTION_REFUSED** & **TypeError: Failed to fetch.

I've seen this issue linked often but have tried the suggested solutions including checking CORS is enabled and the API link is correct. Neither have worked.
  fetch("https://localhost:5000/api/projects/", {
    method: "GET",
  })
    .then((res) => console.log(res.data))
    .catch((e) => {
      console.error(e)
    })

A: I would suggest to use axios : https://www.npmjs.com/package/axios
Then, to fetch from your API :
//declare a state
const [projects, setProjects] = useState([])

axios.get('https://localhost:5000/api/projects/')
        .then(response => setProjects(response));


text:
Q: How to avoid copy and paste in a single T-SQL statement There is some new feature of SQL, I think that it was intruduced in 2005 that I never used. I can't remember what it was called, but I recall reading about it. It was an acronym similar to CT_ or something....
It was used to avoid reproducing the same code within a single SQL, for readability and (I assume) for performance reasons--because the common code would only be evaluated once.
What was it called? I am having a bit of trouble trying to find it and thought someone could easily tell me.

A: You're talking about a Common Table Expression or CTE. It's especially useful for recursive queries.


text:
Q: Can we use dynamic values in static json file Am having a json file for application configuration like below.
[
  {
    "name": "environment",
    "value": "prod"
  },
  {
    "name": "deployment_date",
    "value": "2022-12-21"
  }
]

The variable deployment_date, I want it as dynamic to current UTC date. Can we use any programing language to achieve this? something like getUTCDate().toString() instead "2022-12-21"?

A: Using jq:
jq '(.[] | select(.name == "deployment_date")).value |= (now | todate)' file.json

Output
[
  {
    "name": "environment",
    "value": "prod"
  },
  {
    "name": "deployment_date",
    "value": "2022-12-21T12:46:11Z"
  }
]

jq '(.[] | select(.name == "deployment_date")).value |= (now | strflocaltime("%Y-%m-%d"))' file.json

Output
[
  {
    "name": "environment",
    "value": "prod"
  },
  {
    "name": "deployment_date",
    "value": "2022-12-21"
  }
]


text:
Q: Fluent Validation - Multiple error messages according to query result I am trying to write validation rule for multiple string properties, to check whether they are unique.
so i made validator to check it.
_service method returns list of error codes, that correspond with properties which are not unique.
Now how can I get response message or several in accordance with returned result codes?
RuleFor(x => x.User)
    .MustAsync(async (q, context, token) =>
    {
        var errors = await _service.CheckUserNameEmailExistsAsync(q.User.UserName, q.User.Email, token);
         return !errors.Any();
     }).WithMessage(...);

A: You can go this way:
Create a local variable of your type, and put the result there. And in "WithMessage" refer to it and display the message you need.
example:
            SomeType errors;

            RuleFor(x => x.User)
                .MustAsync(async (q, context, token) =>
                {
                    errors = await _service.CheckUserNameEmailExistsAsync(q.User.UserName, q.User.Email, token);
                    return !errors.Any();
                }).WithMessage((_) =>
                {
                    if (errors....)
                        return "some message";
                    else if (errors...)
                        return "other some message";
                    else
                        return "another some message";
                });


text:
Q: Haxe lua.Table: String should be Int My problem is that haxe doesn't want to compile a following snippet to lua target. It complains about:
Main.hx:7: characters 11-17 : String should be Int
import lua.Table;

class Main {
    static public function my_test(): Table<String, Int>
    {
        var t: Table<String, Int> = Table.create();
        t["test"] = 1; # Heres the problem
        return t;
    }

    static public function main(): Void
    {
        var x = my_test();
        trace(x);
    }
}

If I change the problematic line to:
t[1] = 1; It strangely doesn't complain which seems not logical to me as it's a wrong type in my understanding.
If I put $type(t) somewhere into the snippet above, it correctly identifies it as lua.Table<String, Int>.
I looked into source of std/lua/Table.hx and based on some code there I used untyped keyword in my snippet: t[untyped "test"] = 1; only then it generates desired lua code that executes successfully:
Main.my_test = function()
  local t = ({});
  t.test = 1;
  do return t end;
end

Although I expected something like:
Main.my_test = function()
  local t = {};
  t["test"] = 1;
  do return t end;
end

So, why I have to use this keyword?
$ haxe --version
4.2.1+bf9ff69

# I compile with
$ haxe -D lua-vanilla --main Main --lua Main.lua

A: This seems to be a limitation of Haxe and how the external library is defined as an extern instead of wrapped in an abstract. In std/lua/Table.hx, Table<A, B> is defined as implementing ArrayAccess<B>. This tells Haxe that the type supports indexing into it with integers and that the element type is B. There is no way for it to indicate that the allowed indexer type is A. ArrayAccess<T>'s documentation states:

This interface should be used for externs only. Haxe does not support custom array access on classes. However, array access can be implemented for abstract types.
https://haxe.org/manual/types-abstract-array-access.html

Instead of defining lua.Table as a simple extern in this way, you might be able to define a wrapper around it as an abstract. That would let you specify an array setter/getter appropriately. An example of doing this is below.:
extern class LuaGTable<A, B> implements ArrayAccess<B> {
}

abstract MyLuaTable<K, V>(LuaGTable<K, V>) {
  inline public function new() {
    this = untyped __lua__("({})");
  }
  @:arrayAccess
  public inline function set(k: K, v) {
    this[untyped k] = v;
  }
  @:arrayAccess
  public inline function get(k: K) {
        return this[untyped k];
  }
}

The use of implements ArrayAccess<B> is required to permit use of the indexing operator in the expression this[untyped k] at all. What Haxe thinks we're doing is supplying an integer value to the indexing expression. The abstract itself provides the K/V-typed indexer.
Why this isn't the approach taken by the Haxe standard library is something I do not know. You could probably find or file a bug at their GitHub repository to start a discussion on this. I don't think making Table<A, B> hard to use was intentional.


text:
Q: Pepper robot android emulator is not receiving push notifications? I need to create a fcm push notifications in pepper robot android emulator.
While intializing the fcm notification, i am getting error like Google play services not installed.
It's a standalone android emulator, So i need to make sure whether the pepper robot emulator supports fcm push notifications?

A: Pepper emulator doesn't support Google Play Services and it won't be.
To test FCM push you need to use real Pepper or normal Android Studio Emulator.
All Pepper apps may be run on the normal android device or emulators.
RobotService not gain focus and Peppers' features don't works, but other "clean android" features works normally.


text:
Q: performance/load testing tools to test rest api(headless rest endpoint) without GUI Please help me in finding performance /load testing tool to test a non GUI endpoint. Also tools that allow tests to be exported in csv or xml format and are compatible with source control like GitHub and or tfs.
Thanks in advance

A: If you want to run simple, automated tests and test e.g. an API I would recommend using Locust - http://locust.io - if you like Python, or Gatling - http://gatling.io - if you're OK with scripting in Scala. These two are modern, well documented tools with a nice DX (developer experience). Jmeter is a bit old-fashioned, not so easy to get started with or to use and has a ton of features you're not likely to need.
Other options are:

*

*wrk - https://github.com/wg/wrk - very simple and high-performing tool, scriptable in Lua

*the Grinder - http://grinder.sourceforge.net/ - scriptable in Java or Python (Jython)

And if you want to use an online (SaaS) tool there are of course us (loadimpact.com), Loader.io and Blazemeter.com to choose from.

A: Check this Gatling tutorial, it has all the basics needed to quickly start performance testing with Gatling: Performance testing with Gatling

A: TFS provides web performance load tests out of the box.
https://www.visualstudio.com/en-us/get-started/load-testing.aspx

A: I would go for SoapUI or Apache JMeter, both satisfy your requirements.
Reference material:

*

*Using SoapUI for load testing

*Testing SOAP/REST Web Services Using JMeter


text:
Q: How to set wix property in custom action of vbs? I need to check if C:\Windows\System32\vcruntime*.dll exist before any other custom actions written by QT c++.
        <CustomAction Id="prechecksystem" Script="vbscript" Execute="deferred" Return="check">
            <![CDATA[
                Dim fso, sys32
                Set fso = CreateObject("Scripting.FileSystemObject")
                '0:WindowsFolder, 1:SystemFolder, 2:TemporaryFolder'
                Set sys32 = fso.GetSpecialFolder(1)

                If fso.FileExists(sys32 & "\vcruntime140.dll") Then
                  Session.Property("HSRSUPPORTVCRUNTIME") = "true"
                End If
            ]]>
        </CustomAction>

        <Property Id="HSRSUPPORTVCRUNTIME" Value="false" />

        <InstallUISequence>
            <Custom Action="prechecksystem" After="CostFinalize"></Custom>
            <Show Dialog="VcruntimeNotSupportWarning" Sequence="501"> <![CDATA[ ( HSRSUPPORTVCRUNTIME="false" ) ]]> </Show>
            ...
        <InstallUISequence/>

But msiexec says that variable HSRSUPPORTVCRUNTIME was not defined?

A: You don't need a custom action to do this and definitely should not use VBScript. Use FileSearch element.


text:
Q: How to flatten JSON result from aggregation output in above query I am trying this mongo aggregation I got the output but my required output is not getting anyone please suggest my problem
 const monthsEnum = {
            "_id": "year",
            "1": "January",
            "2": "February",
            "3": "March",
            "4": "April",
            "5": "May",
            "6": "June",
            "7": "July",
            "8": "August",
            "9": "September",
            "10": "October",
            "11": "November",
            "12": "December"
        };

        Light.aggregate([
            { "$match": {
                "CREATE_DATE": {
                    "$lte": new Date(),
                    "$gte": new Date(new Date().setDate(new Date().getDate()-120))
                }
            } },
            { "$group": {
                "_id": {
                    "month": { "$month": "$CREATE_DATE" },
                    "year": { "$year": "$CREATE_DATE" }
                },
                "avgofozone": { "$avg": "$OZONE" }
            } },
            { "$group": {
                "_id": "$year",
                "avgs": {
                    "$push": {
                        "k": { "$substr": ["$month", 0, -1 ] },
                        "v": "$avgofozone"
                    }
                }
            } },
            { "$replaceRoot": {
                "newRoot": {
                    "$mergeObjects": [
                        { "$arrayToObject": "$avgs" },
                        "$$ROOT"
                     ]
                }
            } },
            { "$project": { "avgs": 0 } }
        ], (err, data) => {
            console.log("naresh:" +JSON.stringify(data));
            const polute = Object.keys(data).reduce((p, c) => ({...p, monthsEnum[c]: data[c]}), {});
            res.json(polute);
        })
output:
[
    {
        "zone_type": "avgofozone",
        "year": 2018,
        "February": 21.07777777777778,
        "March": 17.8,
        "January": 17.8
    }
]
MY expected output is:
zone_type     year    February           March    January
avgofozone    2018    21.07777777777778  17.8      17.8


text:
Q: Laravel Image Default My user uploads a profile picture which is stored in storage/profile_picture/user1.png. I use Filesystem and Storage classes to do so.
To retrieve the image I use {!! Html::image(route('profile.thumbnail', $user->profilepic_filename), "Your Picture Here", ['class'=>'img-responsive']) !!}
In my Controller I have
public function thumbnail($filename)
{
    $user = User::where('profilepicture_filename', '=', $filename)->firstOrFail();
    $file = Storage::disk('local_profile')->get($user->profilepicture_filename);

    //$file = URL::asset('/images/default_profilepicture.png'); //doesn't work

    return (new Response($file, 200))->header('Content-Type', $mime);

}

}
I want to get a default image if the profile picture is not found or not uploaded. How can I do so?
Thanks,
K

A: you could just do in you view:
@if(!file_exist($file->name))

@else
    name}}">
@endif

or in your controller:
    if(!$file)
    {
       $file = '.../default/blah.png';
    }

A: For something like this I would just override the accessor (aka getter) on your User model.
http://laravel.com/docs/master/eloquent-mutators#accessors-and-mutators
Any database column, such as profilepicture_filename can be manipulated after it's retrieved using a get___Attribute method, where ___ is the column name in Camel Case
class User
{
    /**
     * @return string
     */
    public function getProfilepictureFilenameAttribute()
    {
        if (! $this->attributes['profilepicture_filename'])) {
            return '/images/default_profilepicture.png';
        }

        return $this->attributes['profilepicture_filename'];
    }
}

Now you simply have to do
profilepicture_filename) }}">

And it will display either their picture or the default if they don't have one. You no longer need the thumbnail route.


text:
Q: Precise conversion of 32-bit unsigned integer into a float in range (-1;1) According to articles like this, half of the floating-point numbers are in the interval [-1,1]. Could you suggest how to make use of this fact so to replace the naive conversion of a 32-bit unsigned integer into a floating-point number (while keeping the uniform distribution)?
Naive code:
uint32_t i = /* randomly generated */;
float f = (float)i / (1ui32<<31) - 1.0f;

The problem here is that first the number i is converted into float losing up to 8 lower bits of precision. Only then the number is scaled to [0;2) interval, and then to [-1;1) interval.
Please, suggest the solution in C or C++ for x86_64 CPU or CUDA if you know it.
Update: the solution with a double is good for x86_64, but is too slow in CUDA. Sorry I didn't expect such a response. Any ideas how to achieve this without using double-precision floating-point?

A: You can do the calculation using double instead so you don't lose any precision on the uint32_t value, then assign the result to a float.
float f = (double)i / (1ui32<<31) - 1.0;

A: In case you drop the uniform distribution constraint its doable on 32bit integer arithmetics alone:
//---------------------------------------------------------------------------
float i32_to_f32(int   x)
    {
    int exp;
    union _f32          // semi result
        {
        float f;        // 32bit floating point
        DWORD u;        // 32 bit uint
        } y;
    // edge cases
    if (x== 0x00000000) return  0.0f;
    if (x< -0x1FFFFFFF) return -1.0f;
    if (x> +0x1FFFFFFF) return +1.0f;
    // conversion
    y.u=0;                              // reset bits
    if (x<0){ y.u|=0x80000000; x=-x; }  // sign (31 bits left)
    exp=((x>>23)&63)-64;                // upper 6 bits -> exponent -1,...,-64 (not 7bits to avoid denormalized numbers)
    y.u|=(exp+127)<<23;                 // exponent bias and bit position
    y.u|=x&0x007FFFFF;                  // mantissa
    return y.f;
    }
//---------------------------------------------------------------------------
int f32_to_i32(float x)
    {
    int exp,man,i;
    union _f32          // semi result
        {
        float f;        // 32bit floating point
        DWORD u;        // 32 bit uint
        } y;
    // edge cases
    if (x== 0.0f) return  0x00000000;
    if (x<=-1.0f) return -0x1FFFFFFF;
    if (x>=+1.0f) return +0x1FFFFFFF;
    // conversion
    y.f=x;
    exp=(y.u>>23)&255; exp-=127;        // exponent bias and bit position
    if (exp<-64) return 0.0f;
    man=y.u&0x007FFFFF;                 // mantissa
    i =(exp<<23)&0x1F800000;
    i|= man;
    if (y.u>=0x80000000) i=-i;          // sign
    return i;
    }
//---------------------------------------------------------------------------

I chose to use only 29 bits + sign = ~ 30 bits of integer to avoid denormalized numbers havoc which I am too lazy to encode (it would get you 30 or even 31 bits but much slower and complicated).
But the distribution is not linear nor uniform at all:

in Red is the float in range <-1,+1> and Blue is integer in range <-1FFFFFFF,+1FFFFFFF>.
On the other hand there is no rounding at all in both conversions ...
PS. I think there might be a way to somewhat linearize the result by using a precomputed LUT for the 6 bit exponent (64 values).

A: The thing to realize is while (float)i does lose 8-bit of precision (so it has 24 bits of precision), the result only has 24 bits of precision as well. So this precision loss is not necessarily a bad thing (this is actually more complicated, because if i is smaller, it will lose less than 8-bits. But things will work out well).
So we just need to fix the range, so the originally non-negative value gets mapped to INT_MIN..INT_MAX.
This expression works: (float)(int)(value^0x80000000)/0x80000000.
Here's how it works:

*

*The (int)(value^0x80000000) part flips the sign bit, so 0x0 gets mapped to INT_MIN, and 0xffffffff gets mapped to INT_MAX.

*Then there is conversion to float. This is where some rounding happens, and we lose precision (but it is not a problem).

*Then just divide by 0x80000000 to get into the range [-1..1]. As this division just adjusts the exponent part, this division doesn't lose any precision.

So, there is only one rounding, the other operations doesn't lose precision. These chain of operations should have the same effect, as calculating the result in infinite precision, then doing the rounding to float (this theoretical rounding has the same effect as the rounding at the 2. step)
But, to be absolutely sure, I've verified with brute force checking all the 32-bit values that this expression results in the same value as (float)((double)value/0x80000000-1.0).

A: I suggest (if yout want to avoid division and use an accurately float-representable start value of 1.0*2^-32):
float e = i * ldexp(1.0,-32) - 1.0;

A:
Any ideas how to achieve this without using double-precision floating-point?

Without assuming too much about the insides of float:
Shift u until the most significant bit is set, halving the float conversion value.
"keeping the uniform distribution"
50% of the uint32_t values will be in the [0.5 ... 1.0)
25%  of the uint32_t values will be in the [0.25 ... 0.5)
12.5% of the uint32_t values will be in the [0.125 ... 0.25)
6.25% of the uint32_t values will be in the [0.0625 ... 0.125)
...
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>

float ui32to0to1(uint32_t u) {
  if (u) {
    float band = 1.0f/(1llu<<32);
    while ((u & 0x80000000) == 0) {
      u <<= 1;
      band *= 0.5f;
    }
    return (float)u * band;
  }
  return 0.0f;
}

Some test code to show functional equivalence to double.
int test(uint32_t u) {
  volatile float f0 = (float) ((double)u / (1llu<<32));
  volatile float f1 = ui32to0to1(u);
  if (f0 != f1) {
    printf("%8lX %.7e %.7e\n", (unsigned long) u, f0, f1);
    return 1;
  }
  return 0;
}

int main(void) {
  for (int i=0; i<100000000; i++) {
    test(rand()*65535u ^ rand());
  }
  return 0;
}

Various optimizations are possible, especially with assuming properties of float.  Yet for an initial answer, I'll stick to a general approach.
For improved efficiency, the loop needs only to iterate from 32 down to FLT_MANT_DIG which is usually 24.
float ui32to0to1(uint32_t u) {
  float band = 1.0f/(1llu<<32);
  for (int i = 32; (i>FLT_MANT_DIG && ((u & 0x80000000) == 0)); i--) {
    u <<= 1;
    band *= 0.5f;
  }
  return (float)u * band;
}

This answers maps [0 to 232-1] to [0.0 to 1.0)
To map to [0 to 232-1] to (-1.0 to 1.0).  It can form -0.0.
if (u >= 0x80000000) {
  return ui32to0to1((u - 0x80000000)*2);
} else
  return -ui32to0to1((0x7FFFFFFF - u)*2);
}


text:
Q: RT parallel processing in Rails I'm developing a sort of personalized search engine in Ruby on Rails, and I'm currently trying to find best way of sorting results depending on user's record, in real time.
Example: items that are searched for can have tags (separate entities with ids), for example item has tags=[1,
5, 10, 23, 45].
User, on the other hand, may have flagged some tags as of special interest, so say user has tags=[5, 23].
Score used for sorting the results should take into account number of item's tags which are "stared" by the user. For example, item's score would be 50% based on item's attributes and 50% on rank depending on user's (number of tags stared).
One idea was to inject this into sorting function in information retrieval system. But in Sphinx, which I will probably use, it would be very awkward to implement (when user's vector is large). I don't know about Lucene/solr, but they don't seem to have advanced non-text search capabilities that I need anyway (distance, date, time, etc.)
Other option is to retrieve intermediary set from IR system and then process it on application level. However, I'm quite sure that processing 100-1000 records sequentially and then sorting them in Rails would be very slow.
On the other hand, it seems like task that can be easily processed in parallel - divide 1000 records into sets that processed by separate threads and then sorted.
I read about several map reduce implementations, both universal like hadoop and rails-specific like skynet etc., but they to be best suited for large batch jobs, not real time processing (unless I'm mistaken?).
Is there any in-memory, light MR implementation that I could use for this? Or maybe you have some other ideas how to handle it?
(sidenote: I believe this setup is similar to how google news works, from what I understand from "Google News personalization: scalable online collaborative filtering" paper. They match in real time a set of candidate stories with set of clusters to which user belongs to (pre-calculated earlier) to sort stories in personalized way)

A: Map/Reduce is great for this kind of thing but you can deal with it in SQL using an intermediate table.
Presumably, you already have tables like this:

users (id, ...)
items (id, ...)
tags (id, ...)
users_tags (user_id, tag_id)
items_tags (item_id, tag_id)

So, why don't you also maintain a table like this:
users_items_tags (user_id, item_id, tag_id)
where each row means "this user and this item share this tag".
Then, your search query is something like this:

  select item_id, count(tag_id) as score
    from users_items_tags
   where user_id = <USER_ID>
group by item_id
order by score desc

When a user adds a tag, users_items_tags is updated like so:

insert into users_items_tags (user_id, item_id, tag_id)
     select <USER_ID>, item_id, <TAG_ID>
       from items_tags
      where tag_id = <TAG_ID>

and likewise when adding a tag to an item. When a tag is removed, just delete on the tag and user/item.
This solution has a few problem cases. If a particular tag is common amongst items then a lot of writes will be performed when a user adds that tag, and vice versa. If a tag is common amongst both items and users then the table will grow very large. You'll have to consider these cases for your particular dataset.


text:
Q: Kraken.com API - Adding more pairs to Ticker array I'm trying to add multiple pairs to the array below but I can only retrieve the last pair. So how can I get both pairs / add more pairs?
$ticker = $kraken->QueryPublic('Ticker', array('pair' => 'XXBTZEUR', 'pair' => 'XXRPXXBT'));

print "<pre>";
print_r($ticker);
print "</pre>";

Result:

Notice: Undefined index: XXBTZEUR in /home/... on line 14

Array
(
    [error] => Array
        (
        )

    [result] => Array
        (
            [XXRPXXBT] => Array
                (
                    [a] => Array
                        (
                            [0] => 0.000028120
                            [1] => 10836
                            [2] => 10836.000
                        )

A: You are using this associated array:
array('pair' => 'XXBTZEUR', 'pair' => 'XXRPXXBT')

An associated array is a set of key-value pairs. The key identifies the value. Since your key of pair is duplicate here, the latter abrogates the former. You will probably need to do as many queries as the number of pairs you need to work with.


text:
Q: Cannot Update Firebase data ? Kotlin I cannot update firebase data using kotlin.
Here's my button click listener. This' s fragment. So I Include this code in on View Created
 fun decodeUserEmailUpdate (emailAddressText: EditText) = emailAddressText.replace(",", ".")
    fun encodeUserEmail(emailAddressText: EditText) = emailAddressText.replace(".", ",")

    view.updateProfile.setOnClickListener { view ->
                val encodedEmail = encodeUserEmail(emailAddressText)
                val decodeUserEmailUpdate = decodeUserEmailUpdate(emailAddressText)
                val reference = FirebaseDatabase.getInstance().getReference("User")
                val checkUser = reference.orderByChild("emailAddress").equals(decodeUserEmailUpdate)

                if(checkUser != null) {

                    "fullName" to nameText.text.toString()
                    "age" to ageText.text.toString()
                    "bankAccNo" to bankText.text.toString()
                    "emailAddress" to encodedEmail
                    "phoneNo" to phoneText.text.toString()
                    "passwordRegister" to passwordText.text.toString()
                }
                else{

                }
        }

A: The way you are trying to update the data is wrong.
what is happening here is:
   view.updateProfile.setOnClickListener { view ->
            val encodedEmail = encodeUserEmail(emailAddressText)
            val decodeUserEmailUpdate = decodeUserEmailUpdate(emailAddressText)

            /**
             *  You get the Reference of Firebase Database "User"
             */

            val reference = FirebaseDatabase.getInstance().getReference("User")

            val checkUser = reference.orderByChild("emailAddress").equals(decodeUserEmailUpdate)

            /**
             *  You check whether checkUser is null or not
             *  checkUser will always be not equal to null, so its useless.
             *  Also, the below code doesn't do anything.
             */
            if (checkUser != null) {

                "fullName" to nameText.text.toString()
                "age" to ageText.text.toString()
                "bankAccNo" to bankText.text.toString()
                "emailAddress" to encodedEmail
                "phoneNo" to phoneText.text.toString()
                "passwordRegister" to passwordText.text.toString()
            } else {

            }
        }

What you are missing out on is, using the reference to set the data.
For example: If your structure looks something like this
Root--> User --> Email(ex. "ss@ss.com")
Your code should look something like this:
        val encodedEmail = encodeUserEmail(emailAddressText)
        val decodeUserEmailUpdate = decodeUserEmailUpdate(emailAddressText)

        /**
         *   get the reference of path where updates need to be carried out.
         */
        val reference = FirebaseDatabase.getInstance()
            .getReference("User")
            .child(decodeUserEmailUpdate)

        /**
         *   I believe, this is not required
         */
        val checkUser = reference.orderByChild("emailAddress").equals(decodeUserEmailUpdate)

        /**
         * Create a map of data that needs to be updated
         */
        val data = mapOf<String, Any>(
            "fullName" to nameText.text.toString(),
            "age" to ageText.text.toString(),
            "bankAccNo" to bankText.text.toString(),
            "emailAddress" to encodedEmail,
            "phoneNo" to phoneText.text.toString(),
            "passwordRegister" to passwordText.text.toString()
        )

        try {
            /**
             * Use your reference to update the children of the reference you got before
             */
            reference.updateChildren(data).addOnCompleteListener {
                if (it.isSuccessful) Log.d("FirebaseDatabase", "Data Updated")
                if (it.exception != null) Log.e("FirebaseDatabase", "Update Error", it.exception)
            }
        }
        /**
         * Catch any exceptions that might occur
         */
        catch (e: Exception) {
            Log.e("FirebaseDatabaseUpdate", "ERROR", e)
        }

Feel free to ask if you have any further questions.


text:
Q: Qual o menor número de irrigadores (círculos) necessários para se cobrir a superfície de uma folhagem (retângulo)? (programação competitiva) Participei de um contest e, em um dos problemas, não obtive o score total da questão, um dos casos teste deu wrong answer, mas não consegui identificar exatamente o meu erro.
O enunciado do problema:

N borrifadores estão instalados em faixas gramadas de L metros de comprimento e W metros de largura. Cada borrifador está instalado na linha central da faixa gramática. Para cada borrifador é fornecido sua posição como sendo a distância do lado mais a esquerda da linha central e também seu raio de operação.
Qual o menor número de borrifadores ligados para regar toda a faixa gramada?
Entrada
A entrada consiste de até 35 casos. A primeira linha para cada caso contém os inteiros N, L e W sendo 1≤N≤10000, 1≤L≤107, e 1≤W≤100. As próximas N linhas contém 2 inteiros dando a posição x (0≤x≤L) e o raio de operação r (1≤r≤1000) do borrifador.
Saída
Para cada caso de teste a saída deve ser o menor número de borrifadores necessários para regar toda a faixa gramada. Se for impossível regar toda a faixa a saída deve ser -1.
Exemplo de entrada
8 20 2
5 3
4 1
1 2
7 2
10 2
13 3
16 2
19 4
3 10 1
3 5
9 3
6 1
3 10 1
5 3
1 1
9 1

Exemplo de saída
6
2
-1

a ideia basicamente do código

*

*Percebe-se que a área útil do círculo para cobrir o comprimento a folhagem de comprimento l é o retângulo inscrito à circunferência com
os vértices sobre os lados da folhagem. Cada um desses retângulos
inscritos na circunferência será um intervalo variando de
x1=x-sqrt(r^2 - (w/2)^2) até x2=x+sqrt(r^2 - (w/2)^2)

*Inserir cada intervalo no vetor e ordenar em ordem crescente de x1

*iterar o vetor:

*i) se o primeiro valor encontrado for maior que 0, retorna -1

*ii) sempre que o x1 do intervalo seguinte no vetor estiver dentro da
área coberta (covered), a área coberta é atualizada ou não,
dependendo do alcance, incluímos, portanto, ou não esse intervalo na contagem

*iii)se a extensão L da folhagem estiver coberta, imprimimos a contagem de intervalos, se não imprimimos -1
O meu código:
#include <iostream>
#include <vector>
#include <algorithm>
#include <math.h>
#define fori(x, y) for(int i = x; i < y; ++i)
using namespace std;
int numcirc(vector < pair <double, double> > &v, int l){
    double cover = v[0].second;
    int cont = 1;
    if(v[0].first>0) return -1;
    else {
        fori(1, v.size()){
            if(v[i].first > cover) return -1;
            else{
                int t=i;
                double _cover = cover;
                while(v[t].first<=cover && t<v.size()){ //queremos incluir os intervalos de maior alcance
                    _cover=max(_cover, v[t].second);
                    t++;
                }
                if(_cover>cover){
                    cont++;
                    cover=_cover;
                }
                i=t-1;
            }
            if(cover>=l) return cont;
        }
    }
    return -1;
}
int main() {
    ios_base::sync_with_stdio(false);
    cin.tie(NULL);cout.tie(NULL);
    int n, l, w, x, r;
    double x1, x2;
    while(cin >> n >> l >> w){
        vector < pair< double, double > > v;
        fori(0,n){
            cin >> x >> r;
            if((double)r > (w/(double)2)){
                x1 = (double)(x - sqrt(abs(r*r - 0.25*w*w)));
                x2 = (double)(x + sqrt(abs(r*r - 0.25*w*w)));
                v.push_back(make_pair(x1, x2));
            }
            else{
                v.push_back(make_pair(10000001, 0.1)); //se r<=w/2, então não serve para cobrir a folhagem
            }
        }
        sort(v.begin(), v.end());
        cout << numcirc(v, l) << "\n";
    }
    return 0;
}

Não pude ter acesso aos casos testes. O que poderia estar ocasionando o wrong answer?

A: Como você mesmo reparou, esse problema pode ser reduzido de cobertura de área para cobertura de segmentos. A título de nivelar quem chegar nesta questão, vou explicar essa transformação. E a área, por incrível que pareça, não é relevante para a questão por um detalhe simples: o enunciado garantiu que todos os borrifadores estarão exatamente no centro.
Assim, o terreno que nos é fornecido é algo assim:

De modo geral, você só encontrará estes seguintes 3 tipos de círculos nessa linha central:

*

*o primeiro é um círculo que tem um diâmetro de irrigação menor do que W

*o círculo do meio é aquele que tem um diâmetro maior do que W

*o último círculo é aquele que tem um diâmetro de irrigação idêntico a W
Nesse tipo de caso, os pontos mais distantes da linha central do retângulo são aqueles que são mais difíceis de serem regados. Se esse ponto for regado, eu tenho por garantia que todo o segmento vertical associado a esse ponto fixo também é regado.
Então, assim sendo, nosso problema acaba sendo transformado em cobrir esses pontos extremos. Ou seja, cobrir um segmento inteiro.
Então, como faço para transformar o problema de áreas de círculos dentro do retângulo para segmentos?
Primeiro, vamos entender como cada tipo de círculo vai ser projetado no segmento inferior do retângulo (poderia usar simetricamente o superior, apenas escolhi uma projeção conveniente para fazer o desenho)?
O primeiro tipo de círculo não alcança a borda, logo ele não é útil em regar na borda inferior e sempre é desnecessário.
O segundo tipo vai projetar todo um intervalo, como ilustrado abaixo:

O raio do círculo foi ilustrado em vermelho, o segmento projetado do círculo foi indicado em ciano. O cálculo para saber onde começa e onde termina esse segmento já está na pergunta: [ x-sqrt(r^2 - (w/2)^2) , x+sqrt(r^2 - (w/2)^2) ], onde x é a posição do borrifador, r o seu raio e w a largura do campo.

 Como se chegou nesses valores? Se você fizer um segmento do centro do círculo até a parte inferior do retângulo de maneira perpendicular, perceberá que ele cairá exatamente no meio do segmento. Assim, como ele é perpendicular, se unir esse segmento, um dos raios e pegar o segmento da ponta do segmento perpendicular obterá um triângulo retângulo. A hipotenusa é conhecida, o raio do círculo, r; um dos catetos também é conhecido, que é o comprimento do segmento perpendicular do centro do círculo até o retângulo, que é metade da largura do campo, portanto w/2; sobrando então o outro cateto para se calcular o comprimento que é sqrt(r^2 - (w/2)^2). Então, se eu pegar o ponto do centro do círculo e andar para a esquerda, obtenho o ponto a esquerda do segmento como sendo x-sqrt(r^2 - (w/2)^2), e se andar para a direita x+sqrt(r^2 - (w/2)^2).

O terceiro tipo de círculo adicionará apenas um ponto no segmento inferior. Como estamos tratando de uma quantidade finita de círculos, a união de muitos pontos nunca chegará a fazer um segmento, logo esse tipo de borrifador também não contribuirá para fazer a irrigação, assim como o primeiro tipo de borrifador.
Dito isso, vamos começar a montar o algoritmo?
Primeiro, vamos precisar ler todos os pontos. Então, descartaremos os círculos que não contribuirão para fazer o intervalo; ou seja, ficaremos apenas com círculos cujos raios sejam estritamente maiores do que metade da largura. Então, transformamos os círculos filtrados em intervalos.
Pronto, agora temos apenas intervalos e desejamos saber qual o menor conjunto desses intervalos que cobrem todo um segmento. Sabe o que podemos usar para modelar interseções de intervalos? Um grafo de intervalos!
Sim, agora estou transformando o problema em em problema de grafos. Como montar esse grafo?
Um grafo de intervalo é uma maneira de representar, em grafo, segmentos e interseções. Cada segmento é um vértice e a existência de interseções entre dois intervalos é representado como uma aresta. Então, com o grafo montado, Precisamos sair de algum intervalo que contenha o 0 (origem) e, apenas navegando pelas arestas, chegar em algum intervalo que contenha o L (destino). Se não for possível, seja porque não tem nenhum intervalo que contenha o 0 para partida, seja porque não tem nenhum ponto de chegada que contenha o L ou porque o grafo é disconexo, então retorno -1.

Aqui na rede tem algumas perguntas interessantes que abordam o problema de grafos de intervalo. As que eu me lembro no momento:

*

*Escalonamento de máquinas - Teoria dos Grafos

*Como identificar um grafo inválido para problema de alocação de operadores por máquina?

*Algoritmo O(n log n) para identificar intervalos contíguos dada uma lista de intervalos
Inclusive, sugiro a leitura nessa ordem que coloquei, que não por acaso coincide com a ordem cronológica de publicação

O primeiro passo é descobrir qual o vértice de partida. Independente de qualquer coisa, o ponto de partida precisa englobar o ponto 0. Portanto, devo considerar como vértice candidato à partida todo intervalo cujo valor mais a esquerda é <=0. Similarmente com o vértice de chegada, valor mais a direita >=L.
Agora, como definir de maneiras mais assertivas esses pontos? Para o ponto de partida, aquele que cobrir até o máximo a direita terá cobertura total sobre qualquer outro ponto candidato de partida, portanto sobra apenas aquele cujo valor mais a direita é o maior. De maneira semelhante, o ponto de chegada é aquele cujo valor mais a esquerda é o menor.
Tem o caso especial de que o ponto de partida engloba a saída; nesse caso, existe ainda a possibilidade de que o meu detector de origem/destino encontre pontos distintos de origem e destino, mas é trivial verificar se apenas o borrifador de origem seria o suficiente para cobrir tudo. Para esse caso, basta retornar 1 e ir para a próxima entrada.
Definidos origem e destino, precisamos das arestas para navegar no grafo. A priori, um grafo de intervalo é um grafo não-direcionado. Porém, podemos fazer algumas otimizações e transformar nosso grafo de intervalo num grafo direcionado acíclico.
Eu vou adicionar uma aresta do intervalo [c1, f1] para o intervalo [c2, f2] se e somente se existir interseção entre esses intervalos e o segundo intervalo terminar depois do primeiro intervalo e o segundo intervalo não englobar totalmente o primeiro. Ou seja, se c2 <= f1 (precisa haver interseção) e f2 > f1 (o segundo intervalo necessariamente termina depois do primeiro intervalo) e c2 > c1 (o primeiro intervalo tem um pedaço anterior ao segundo intervalo). Obviamente que ordenar os intervalos pelo seu valor a esquerda de modo crescente nos permite fazer diversas otimizações na construção das arestas
Montando esse grafo, direcionado mesmo, basta rodar qualquer algoritmo de menor distância entre vértices (considere todas as arestas com peso 1) do ponto de origem até o ponto de destino. Se não for possível alcançar o destino, a saída precisa ser -1. Caso contrário, a saída é a distância encontrada mais um.

Pelo que entendi da sua abordagem, você tentou fazer algo mais guloso. Porém, a estratégia gulosa para funcionar precisa decidir com muita calma se deve inserir ou não o próximo intervalo. Pelo que pude olhar, a estratégia de escolher o próximo elemento que você usou foi realmente adequada (olho para frente todos os intervalos que tem alguma interseção com o que já foi coberto até o momento). Porém a escolha do ponto de partida é sempre fixa, não se tenta otimizá-lo para pegar como partida aquele que contém o 0 e vai até o mais longe possível. Aí, sim, você teria uma solução gulosa e sem backtracking.
Também percebi que você adiciona desnecessariamente na lista pontos que eu filtrei fora, fazendo eles serem um peso na ordenação que seria desnecessário (mas que no final das contas não seria significativo).


text:
Q: DBeaver and SQLite connection through SSH With DBeaver, is it possible to use an SQLite database through an SSH connection?
If yes, how do I set up the connection?

A: From a Dbeaver issue on github, it's not possible.

serge-rider commented on Nov 18, 2019
You cannot use SSH with SQLite.
  SQLite is embedded database, there is no such thing as SQLite server.
  So SSH doesn't make any sense for SQLite.

I try it too, and don't find a working way.

A: Have you tried using sshfs to mount the remote file system to your local file system over ssh?
sudo sshfs name@server:/path/to/folder /path/to/mount/point

I'm not sure how performant this would be, but theoretically it should hide from Dbeaver the fact that the database file is remotely located.

A: *

*Install FTP on your Linux server.

*Test FTP connection using Windows file explorer by typing this in the navigation bar : ftp://[ServerAdress] ex: ftp://203.0.113.10

*Create a new SQLite connection in Dbeaver. On path, click open and in file explorer type the same ftp://[ServerAdress].


text:
Q: How to bind VBA macro to specific cell? I'm working on an excel sheet where I have data organized in rows and need a way to run a VBA macro which will execute program based on data found in specific row.
What I'm struggling to achieve is to find a mechanism for users to simply click something button-like positioned after data to complete this task. Also users should be able to copy an existing row (and macro trigger with it) and execute macro related to that new row without having to do any code modifications.
What I have tried so far is excel button but it has no way of binding to certain cell. Also macro has to be assigned manually every time new row is added.
I also tried 'Worksheet_SelectionChange' event with filtering to certain column containing link-like text. This works fine in a way, but some users use arrow keys to navigate within sheet which accidentally triggers macros if macro cell gets selected. Also, mouse cursor doesn't change to pointing finger during hover over.
Last attempt was to try =HYPERLINK("#'Sheet1'!E3";"Run macro")-function and using it with 'Worksheet_FollowHyperlink'. However, in this case event does not trigger if hyperlink points to current sheet.
So any new ideas or maybe refining some of these to overcome problems I have described?

A: Here's an example of how you could do this:

*

*Add a button (MSForm, not ActiveX) in the cell. Make sure the top left corner of the button is inside the cell for the row you want to refer to.

*Assign the macro (let's say "MM" here for simplicity)

In the Macro:
Sub MM()
    Dim rowNumber As Long
    rowNumber = Shapes(Application.Caller).TopLeftCell.Row

    Debug.Print Range("A" & rowNumber).Value
End Sub


text:
Q: Как управлять приложением с графическим интерфейсом из другого приложения? Предположим, что в Linux или Windows запущено приложение с графическим интерфейсом. Когда я нажимаю кнопку, происходит какое-то событие. Я хочу, чтобы другое приложение могло нажимать кнопки, вводить текст в поля, считывать текстовые поля и т. п., так же, как это делает пользователь.
Есть ПО для автоматизации действий браузера Selenium.
Как написать такую программу как Selenuim, только для других приложений, на языках программирования С и С++?


text:
Q: how to filter out a null value from spark dataframe I created a dataframe in spark with the following schema:
root
 |-- user_id: long (nullable = false)
 |-- event_id: long (nullable = false)
 |-- invited: integer (nullable = false)
 |-- day_diff: long (nullable = true)
 |-- interested: integer (nullable = false)
 |-- event_owner: long (nullable = false)
 |-- friend_id: long (nullable = false)

And the data is shown below:
+----------+----------+-------+--------+----------+-----------+---------+
|   user_id|  event_id|invited|day_diff|interested|event_owner|friend_id|
+----------+----------+-------+--------+----------+-----------+---------+
|   4236494| 110357109|      0|      -1|         0|  937597069|     null|
|  78065188| 498404626|      0|       0|         0| 2904922087|     null|
| 282487230|2520855981|      0|      28|         0| 3749735525|     null|
| 335269852|1641491432|      0|       2|         0| 1490350911|     null|
| 437050836|1238456614|      0|       2|         0|  991277599|     null|
| 447244169|2095085551|      0|      -1|         0| 1579858878|     null|
| 516353916|1076364848|      0|       3|         1| 3597645735|     null|
| 528218683|1151525474|      0|       1|         0| 3433080956|     null|
| 531967718|3632072502|      0|       1|         0| 3863085861|     null|
| 627948360|2823119321|      0|       0|         0| 4092665803|     null|
| 811791433|3513954032|      0|       2|         0|  415464198|     null|
| 830686203|  99027353|      0|       0|         0| 3549822604|     null|
|1008893291|1115453150|      0|       2|         0| 2245155244|     null|
|1239364869|2824096896|      0|       2|         1| 2579294650|     null|
|1287950172|1076364848|      0|       0|         0| 3597645735|     null|
|1345896548|2658555390|      0|       1|         0| 2025118823|     null|
|1354205322|2564682277|      0|       3|         0| 2563033185|     null|
|1408344828|1255629030|      0|      -1|         1|  804901063|     null|
|1452633375|1334001859|      0|       4|         0| 1488588320|     null|
|1625052108|3297535757|      0|       3|         0| 1972598895|     null|
+----------+----------+-------+--------+----------+-----------+---------+

I want to filter out the rows have null values in the field of "friend_id".
scala> val aaa = test.filter("friend_id is null")

scala> aaa.count

I got :res52: Long = 0 which is obvious not right. What is the right way to get it?
One more question, I want to replace the values in the friend_id field. I want to replace null with 0 and 1 for any other value except null. The code I can figure out is:
val aaa = train_friend_join.select($"user_id", $"event_id", $"invited", $"day_diff", $"interested", $"event_owner", ($"friend_id" != null)?1:0)

This code also doesn't work. Can anyone tell me how can I fix it? Thanks

A: Let's say you have this data setup (so that results are reproducible):
// declaring data types
case class Company(cName: String, cId: String, details: String)
case class Employee(name: String, id: String, email: String, company: Company)

// setting up example data
val e1 = Employee("n1", null, "n1@c1.com", Company("c1", "1", "d1"))
val e2 = Employee("n2", "2", "n2@c1.com", Company("c1", "1", "d1"))
val e3 = Employee("n3", "3", "n3@c1.com", Company("c1", "1", "d1"))
val e4 = Employee("n4", "4", "n4@c2.com", Company("c2", "2", "d2"))
val e5 = Employee("n5", null, "n5@c2.com", Company("c2", "2", "d2"))
val e6 = Employee("n6", "6", "n6@c2.com", Company("c2", "2", "d2"))
val e7 = Employee("n7", "7", "n7@c3.com", Company("c3", "3", "d3"))
val e8 = Employee("n8", "8", "n8@c3.com", Company("c3", "3", "d3"))
val employees = Seq(e1, e2, e3, e4, e5, e6, e7, e8)
val df = sc.parallelize(employees).toDF

Data is:
+----+----+---------+---------+
|name|  id|    email|  company|
+----+----+---------+---------+
|  n1|null|n1@c1.com|[c1,1,d1]|
|  n2|   2|n2@c1.com|[c1,1,d1]|
|  n3|   3|n3@c1.com|[c1,1,d1]|
|  n4|   4|n4@c2.com|[c2,2,d2]|
|  n5|null|n5@c2.com|[c2,2,d2]|
|  n6|   6|n6@c2.com|[c2,2,d2]|
|  n7|   7|n7@c3.com|[c3,3,d3]|
|  n8|   8|n8@c3.com|[c3,3,d3]|
+----+----+---------+---------+

Now to filter employees with null ids, you will do --
df.filter("id is null").show

which will correctly show you following:
+----+----+---------+---------+
|name|  id|    email|  company|
+----+----+---------+---------+
|  n1|null|n1@c1.com|[c1,1,d1]|
|  n5|null|n5@c2.com|[c2,2,d2]|
+----+----+---------+---------+

Coming to the second part of your question, you can replace the null ids with 0 and other values with 1 with this --
df.withColumn("id", when($"id".isNull, 0).otherwise(1)).show

This results in:
+----+---+---------+---------+
|name| id|    email|  company|
+----+---+---------+---------+
|  n1|  0|n1@c1.com|[c1,1,d1]|
|  n2|  1|n2@c1.com|[c1,1,d1]|
|  n3|  1|n3@c1.com|[c1,1,d1]|
|  n4|  1|n4@c2.com|[c2,2,d2]|
|  n5|  0|n5@c2.com|[c2,2,d2]|
|  n6|  1|n6@c2.com|[c2,2,d2]|
|  n7|  1|n7@c3.com|[c3,3,d3]|
|  n8|  1|n8@c3.com|[c3,3,d3]|
+----+---+---------+---------+

A: Or like df.filter($"friend_id".isNotNull)

A: The following lines work well:
test.filter("friend_id is not null")

A: df.where(df.col("friend_id").isNull)

A: There are two ways to do it: creating filter condition 1) Manually 2) Dynamically.
Sample DataFrame:
val df = spark.createDataFrame(Seq(
  (0, "a1", "b1", "c1", "d1"),
  (1, "a2", "b2", "c2", "d2"),
  (2, "a3", "b3", null, "d3"),
  (3, "a4", null, "c4", "d4"),
  (4, null, "b5", "c5", "d5")
)).toDF("id", "col1", "col2", "col3", "col4")

+---+----+----+----+----+
| id|col1|col2|col3|col4|
+---+----+----+----+----+
|  0|  a1|  b1|  c1|  d1|
|  1|  a2|  b2|  c2|  d2|
|  2|  a3|  b3|null|  d3|
|  3|  a4|null|  c4|  d4|
|  4|null|  b5|  c5|  d5|
+---+----+----+----+----+

1) Creating filter condition manually i.e. using DataFrame where or filter function
df.filter(col("col1").isNotNull && col("col2").isNotNull).show

or
df.where("col1 is not null and col2 is not null").show

Result:
+---+----+----+----+----+
| id|col1|col2|col3|col4|
+---+----+----+----+----+
|  0|  a1|  b1|  c1|  d1|
|  1|  a2|  b2|  c2|  d2|
|  2|  a3|  b3|null|  d3|
+---+----+----+----+----+

2) Creating filter condition dynamically: This is useful when we don't want any column to have null value and there are large number of columns, which is mostly the case.
To create the filter condition manually in these cases will waste a lot of time. In below code we are including all columns dynamically using map and reduce function on DataFrame columns:
val filterCond = df.columns.map(x=>col(x).isNotNull).reduce(_ && _)

How filterCond looks:
filterCond: org.apache.spark.sql.Column = (((((id IS NOT NULL) AND (col1 IS NOT NULL)) AND (col2 IS NOT NULL)) AND (col3 IS NOT NULL)) AND (col4 IS NOT NULL))

Filtering:
val filteredDf = df.filter(filterCond)

Result:
+---+----+----+----+----+
| id|col1|col2|col3|col4|
+---+----+----+----+----+
|  0|  a1|  b1|  c1|  d1|
|  1|  a2|  b2|  c2|  d2|
+---+----+----+----+----+

A: From the hint from Michael Kopaniov, below works
df.where(df("id").isNotNull).show

A: Here is a solution for spark in Java. To select data rows containing nulls. When you have Dataset data, you do:
Dataset<Row> containingNulls =  data.where(data.col("COLUMN_NAME").isNull())

To filter out data without nulls you do:
Dataset<Row> withoutNulls = data.where(data.col("COLUMN_NAME").isNotNull())

Often dataframes contain columns of type String where instead of nulls we have empty strings like "". To filter out such data as well we do:
Dataset<Row> withoutNullsAndEmpty = data.where(data.col("COLUMN_NAME").isNotNull().and(data.col("COLUMN_NAME").notEqual("")))

A: for the first question, it is correct you are filtering out nulls and hence count is zero.
for the second replacing: use like below:
val options = Map("path" -> "...\\ex.csv", "header" -> "true")
val dfNull = spark.sqlContext.load("com.databricks.spark.csv", options)

scala> dfNull.show

+----------+----------+-------+--------+----------+-----------+---------+
|   user_id|  event_id|invited|day_diff|interested|event_owner|friend_id|
+----------+----------+-------+--------+----------+-----------+---------+
|   4236494| 110357109|      0|      -1|         0|  937597069|     null|
|  78065188| 498404626|      0|       0|         0| 2904922087|     null|
| 282487230|2520855981|      0|      28|         0| 3749735525|     null|
| 335269852|1641491432|      0|       2|         0| 1490350911|     null|
| 437050836|1238456614|      0|       2|         0|  991277599|     null|
| 447244169|2095085551|      0|      -1|         0| 1579858878|        a|
| 516353916|1076364848|      0|       3|         1| 3597645735|        b|
| 528218683|1151525474|      0|       1|         0| 3433080956|        c|
| 531967718|3632072502|      0|       1|         0| 3863085861|     null|
| 627948360|2823119321|      0|       0|         0| 4092665803|     null|
| 811791433|3513954032|      0|       2|         0|  415464198|     null|
| 830686203|  99027353|      0|       0|         0| 3549822604|     null|
|1008893291|1115453150|      0|       2|         0| 2245155244|     null|
|1239364869|2824096896|      0|       2|         1| 2579294650|        d|
|1287950172|1076364848|      0|       0|         0| 3597645735|     null|
|1345896548|2658555390|      0|       1|         0| 2025118823|     null|
|1354205322|2564682277|      0|       3|         0| 2563033185|     null|
|1408344828|1255629030|      0|      -1|         1|  804901063|     null|
|1452633375|1334001859|      0|       4|         0| 1488588320|     null|
|1625052108|3297535757|      0|       3|         0| 1972598895|     null|
+----------+----------+-------+--------+----------+-----------+---------+

dfNull.withColumn("friend_idTmp", when($"friend_id".isNull, "1").otherwise("0")).drop($"friend_id").withColumnRenamed("friend_idTmp", "friend_id").show

+----------+----------+-------+--------+----------+-----------+---------+
|   user_id|  event_id|invited|day_diff|interested|event_owner|friend_id|
+----------+----------+-------+--------+----------+-----------+---------+
|   4236494| 110357109|      0|      -1|         0|  937597069|        1|
|  78065188| 498404626|      0|       0|         0| 2904922087|        1|
| 282487230|2520855981|      0|      28|         0| 3749735525|        1|
| 335269852|1641491432|      0|       2|         0| 1490350911|        1|
| 437050836|1238456614|      0|       2|         0|  991277599|        1|
| 447244169|2095085551|      0|      -1|         0| 1579858878|        0|
| 516353916|1076364848|      0|       3|         1| 3597645735|        0|
| 528218683|1151525474|      0|       1|         0| 3433080956|        0|
| 531967718|3632072502|      0|       1|         0| 3863085861|        1|
| 627948360|2823119321|      0|       0|         0| 4092665803|        1|
| 811791433|3513954032|      0|       2|         0|  415464198|        1|
| 830686203|  99027353|      0|       0|         0| 3549822604|        1|
|1008893291|1115453150|      0|       2|         0| 2245155244|        1|
|1239364869|2824096896|      0|       2|         1| 2579294650|        0|
|1287950172|1076364848|      0|       0|         0| 3597645735|        1|
|1345896548|2658555390|      0|       1|         0| 2025118823|        1|
|1354205322|2564682277|      0|       3|         0| 2563033185|        1|
|1408344828|1255629030|      0|      -1|         1|  804901063|        1|
|1452633375|1334001859|      0|       4|         0| 1488588320|        1|
|1625052108|3297535757|      0|       3|         0| 1972598895|        1|
+----------+----------+-------+--------+----------+-----------+---------+

A: val df = Seq(
  ("1001", "1007"),
  ("1002", null),
  ("1003", "1005"),
  (null, "1006")
).toDF("user_id", "friend_id")

Data is:
+-------+---------+
|user_id|friend_id|
+-------+---------+
|   1001|     1007|
|   1002|     null|
|   1003|     1005|
|   null|     1006|
+-------+---------+

Drop rows containing any null or NaN values in the specified columns of the Seq:
df.na.drop(Seq("friend_id"))
  .show()

Output:
+-------+---------+
|user_id|friend_id|
+-------+---------+
|   1001|     1007|
|   1003|     1005|
|   null|     1006|
+-------+---------+

If do not specify columns, drop row as long as any column of a row contains null or NaN values:
df.na.drop()
  .show()

Output:
+-------+---------+
|user_id|friend_id|
+-------+---------+
|   1001|     1007|
|   1003|     1005|
+-------+---------+

A: A good solution for me was to drop the rows with any null values:
Dataset<Row> filtered = df.filter(row => !row.anyNull);
In case one is interested in the other case, just call row.anyNull.
(Spark 2.1.0 using Java API)

A: Another easy way to filter out null values from multiple columns in spark dataframe. Please pay attention there is AND between columns.
df.filter(" COALESCE(col1, col2, col3, col4, col5, col6) IS NOT NULL")

If you need to filter out rows that contain any null (OR connected) please use
df.na.drop()

A: I use the following code to solve my question. It works. But as we all know, I work around a country's mile to solve it. So, is there a short cut for that? Thanks
def filter_null(field : Any) : Int = field match {
    case null => 0
    case _    => 1
}

val test = train_event_join.join(
    user_friends_pair,
    train_event_join("user_id") === user_friends_pair("user_id") &&
    train_event_join("event_owner") === user_friends_pair("friend_id"),
    "left"
).select(
    train_event_join("user_id"),
    train_event_join("event_id"),
    train_event_join("invited"),
    train_event_join("day_diff"),
    train_event_join("interested"),
    train_event_join("event_owner"),
    user_friends_pair("friend_id")
).rdd.map{
    line => (
        line(0).toString.toLong,
        line(1).toString.toLong,
        line(2).toString.toLong,
        line(3).toString.toLong,
        line(4).toString.toLong,
        line(5).toString.toLong,
        filter_null(line(6))
        )
    }.toDF("user_id", "event_id", "invited", "day_diff", "interested", "event_owner", "creator_is_friend")


text:
Q: How to calculate roc auc score from positive unlabeled learning? I'm trying to adapt some code for positive unlabeled learning from this example, which runs with my data but I want to also calculate the ROC AUC score which I'm getting stuck on.
My data is divided into positive samples (data_P) and unlabeled samples (data_U), each with only 2 features/columns of data such as:
#3 example rows:
data_P

[[-1.471,  5.766],
       [-1.672,  5.121],
       [-1.371,  4.619]]

#3 example rows:
data_U

[[1.23,  6.26],
       [-5.72,  4.1213],
       [-3.1,  7.129]]

I run the positive-unlabeled learning as in the linked example:
known_labels_ratio = 0.5

NP = data_P.shape[0]
NU = data_U.shape[0]

T = 1000
K = NP
train_label = np.zeros(shape=(NP+K,))
train_label[:NP] = 1.0
n_oob = np.zeros(shape=(NU,))
f_oob = np.zeros(shape=(NU, 2))
for i in range(T):
    # Bootstrap resample
    bootstrap_sample = np.random.choice(np.arange(NU), replace=True, size=K)
    # Positive set + bootstrapped unlabeled set
    data_bootstrap = np.concatenate((data_P, data_U[bootstrap_sample, :]), axis=0)
    # Train model
      model = DecisionTreeClassifier(max_depth=None, max_features=None,
                                   criterion='gini', class_weight='balanced')
    model.fit(data_bootstrap, train_label)
    # Index for the out of the bag (oob) samples
    idx_oob = sorted(set(range(NU)) - set(np.unique(bootstrap_sample)))
    # Transductive learning of oob samples
    f_oob[idx_oob] += model.predict_proba(data_U[idx_oob])
    n_oob[idx_oob] += 1

predict_proba = f_oob[:, 1]/n_oob

This all runs fine but what I want is to run roc_auc_score() which I'm getting stuck on how to do without errors.
Currently I am trying:
y_pred = model.predict_proba(data_bootstrap)
roc_auc_score(train_label, y_pred)
ValueError: bad input shape (3, 2)

The problem seems to be that y_pred gives an output with 2 columns, looking like:
y_pred
array([[0.00554287, 0.9944571 ],
       [0.0732314 , 0.9267686 ],
       [0.16861796, 0.83138204]])

I'm not sure why y_pred ends up like this, is it giving the probability based on if the sample falls into 2 groups? Positive or other essentially? Could I just filter these to select per row the probability with the highest score? Or is there a way for me to change this or another way for me to calculate the AUCROC score?

A: y_pred must be a single number, giving the probability of the positive class p1; currently your y_pred consists of both probabilities [p0, p1] (with p0+p1=1.0 by definition).
Assuming that your positive class is class 1 (i.e. the second element of each array in y_pred), what you should do is:
y_pred_pos = [y_pred[i, 1] for i in range(len(y_pred))]
y_pred_pos # inspect
# [0.9944571, 0.9267686, 0.83138204]

roc_auc_score(train_label, y_pred_pos)

In case your y_pred is a Numpy array (and not a Python list), you could replace the list comprehension in the first command above with:
y_pred_pos  = y_pred[:,1]


text:
Q: script that create users and passwords from a .xls file I need to create users and passwords from an excel file for 20-25 persons.
lets say we have the list 1 i need a script to create them automatically , my teacher said pointers could be a way.
Will be used in a web interface based on a login page.
Any advice is welcome.


text:
Q: How do I load the app name, app image and package name into variables I can use in a separate activity? I have this example application which I downloaded, which displays a list of apps on the phone. I am trying to study and learn the code but wanted to add to it by allowing you to click on an app and then toast the app name and package location.
Here is the source
    public class AllAppsActivity extends ListActivity {

    private PackageManager packageManager = null;
    private List<ApplicationInfo> applist = null;
    private ApplicationAdapter listadaptor = null;

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        packageManager = getPackageManager();
        new LoadApplications().execute();
    }

    @Override
    protected void onListItemClick(ListView l, View v, int position, long id) {
        super.onListItemClick(l, v, position, id);

            ApplicationInfo app = applist.get(position);
            Intent intent = packageManager.getLaunchIntentForPackage(app.packageName);
            if (null != intent) {
                startActivity(intent);
            }

        Toast.makeText(AllAppsActivity.this, "You have selected ", Toast.LENGTH_SHORT).show();

    }

    private List<ApplicationInfo> checkForLaunchIntent(List<ApplicationInfo> list) {
        ArrayList<ApplicationInfo> applist = new ArrayList<ApplicationInfo>();
        for (ApplicationInfo info : list) {
             if (null != packageManager.getLaunchIntentForPackage(info.packageName)) {
                applist.add(info);
             }

        }

        return applist;
    }

    private class LoadApplications extends AsyncTask<Void, Void, Void> {
        private ProgressDialog progress = null;

        @Override
        protected Void doInBackground(Void... params) {
            applist = checkForLaunchIntent(packageManager.getInstalledApplications(PackageManager.GET_UNINSTALLED_PACKAGES));
            listadaptor = new ApplicationAdapter(AllAppsActivity.this, R.layout.snippet_list_row, applist);
            return null;
        }

        @Override
        protected void onPostExecute(Void result) {
            setListAdapter(listadaptor);
            progress.dismiss();
            super.onPostExecute(result);
        }

        @Override
        protected void onPreExecute() {
            progress = ProgressDialog.show(AllAppsActivity.this, null, "Loading application info...");
            super.onPreExecute();
        }

        @Override
        protected void onProgressUpdate(Void... values) {
            super.onProgressUpdate(values);
        }
    }
}

Here is ApplicationAdapter.java
public class ApplicationAdapter extends ArrayAdapter<ApplicationInfo> {
private List<ApplicationInfo> appsList = null;
private Context context;
private PackageManager packageManager;

public ApplicationAdapter(Context context, int textViewResourceId,
        List<ApplicationInfo> appsList) {
    super(context, textViewResourceId, appsList);
    this.context = context;
    this.appsList = appsList;
    packageManager = context.getPackageManager();
}

@Override
public int getCount() {
    return ((null != appsList) ? appsList.size() : 0);
}

@Override
public ApplicationInfo getItem(int position) {
    return ((null != appsList) ? appsList.get(position) : null);
}

@Override
public long getItemId(int position) {
    return position;
}

@Override
public View getView(int position, View convertView, ViewGroup parent) {
    View view = convertView;
    if (null == view) {
        LayoutInflater layoutInflater = (LayoutInflater) context.getSystemService(Context.LAYOUT_INFLATER_SERVICE);
        view = layoutInflater.inflate(R.layout.snippet_list_row, null);
    }

    ApplicationInfo data = appsList.get(position);
    if (null != data) {
        TextView appName = (TextView) view.findViewById(R.id.app_name);
        TextView packageName = (TextView) view.findViewById(R.id.app_paackage);
        ImageView iconview = (ImageView) view.findViewById(R.id.app_icon);

        appName.setText(data.loadLabel(packageManager));
        packageName.setText(data.packageName);
        iconview.setImageDrawable(data.loadIcon(packageManager));

    }
    return view;
}
};

Here is the xml for the listrow
    <?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="fill_parent"
    android:layout_height="wrap_content" >

    <ImageView
        android:id="@+id/app_icon"
        android:layout_width="50dp"
        android:layout_height="50dp"
        android:padding="3dp"
        android:contentDescription="@string/yolo"
        android:scaleType="centerCrop" />

    <LinearLayout
        android:layout_width="fill_parent"
        android:layout_height="fill_parent"
        android:gravity="center_vertical"
        android:orientation="vertical"
        android:paddingLeft="5dp" >

        <TextView
            android:id="@+id/app_name"
            android:layout_width="fill_parent"
            android:layout_height="wrap_content"
            android:gravity="center_vertical"
            android:textStyle="bold" />

        <TextView
            android:id="@+id/app_paackage"
            android:layout_width="fill_parent"
            android:layout_height="wrap_content"
            android:gravity="center_vertical" />
    </LinearLayout>

    </LinearLayout>

A: In your onListItemClicked you can call the getItem method in your adapter. From that you can get the fields you need and can pass them as arguments to your new activity. By the way are you sure you need a new activity? It's possible you could just swap out fragments.


text:
Q: Calculate standard deviation for groups of values using Python My data looks similar to this:
index name number difference
0     AAA  10     0
1     AAA  20     10
2     BBB  1      0
3     BBB  2      1
4     CCC  5      0
5     CCC  10     5
6     CCC  10.5   0.5

I need to calculate standard deviation for difference column based on groups of name.
I tried
data[['difference']].groupby(['name']).agg(['mean', 'std'])

and
data["std"]=(data['difference'].groupby('name').std())

but both gave KeyError for the variable that's passed to groupby(). I tried to resolve it with:
data.columns = data.columns.str.strip()

but the error persists.
Thanks in advance.

A: You can use groupby(['name']) on the full data frame first, and only apply the agg on the columns of interest:
data = pd.DataFrame({'name':['AAA','AAA','BBB','BBB','CCC','CCC','CCC'],
                    'number':[10,20,1,2,5,10,10.5],
                    'difference':[0,10,0,1,0,5,0.5]})
data.groupby(['name'])['difference'].agg(['mean', 'std'])


text:
Q: python script executed from php gives error On my debian i run xampp. I want to execute a python script using php shell_exec.
This is my php code:
shell_exec("/opt/lampp/htdocs/news/hello.py 2>1 &");

When i run it from the browser i get this error:
/usr/bin/python: /opt/lampp/lib/libz.so.1: no version information available (required by /usr/bin/python)

If i run it from the terminal window using this: php /opt/lampp/htdocs/page/index.php it works without any problems.
So any ideas how can i make it work from the browser?
Thanks

A: I'm guessing xampp comes with it's own libraries instead of using the system libraries, and that probably means it's setting LD_LIBRARY_PATH to it's local library directory.
This will also cause other programs started from php to use this libraries, which they may not be compatible with.
To veryfy that, try system("env");, that sould show you all the exported environment variables. If LD_LIBRARY_PATH is set, use:
shell_exec("LD_LIBRARY_PATH= /opt/lampp/htdocs/news/hello.py 2>1 &");

That unsets it before running the python script.


text:
Q: openssl command line to cipher RC4, not expected result, don't understand this is probably a stupid question, but I cannot figure it out.
Currently, I am using this website: http://www.fyneworks.com/encryption/rc4-encryption/ to cipher rc4 for a proof of concept. For instance, I am entering 'a' as a cleartext, 'a' as a password and I get '71' as a ciphertext (this is the ascii representation of the 'q').
I wanted to do the same from the command line, using openssl:
> echo a | openssl rc4 -nosalt  -out /tmp/uuu
enter rc4 encryption password:
Verifying - enter rc4 encryption password:

> cat /tmp/uuu | xxd
0000000: 5896                                     X.

So we are getting '5896' instead of '71' and this is what I don't understand. If someone could explain to me, I'd be grateful.
Thank you !

A: Thanks to a friend, we figured out what was wrong. He told me to print the key
echo -ne "a" |  openssl  rc4 -pass pass:a -e  -nopad    -nosalt -p
key=0CC175B9C0F1B6A831C399E269772661

We see that there is some padding added, with the 0x61 we entered at the end. It turns out openssl generates a key from the pass.
Instead, if we enter directly the key with the -K option:
echo -ne "a" |  openssl  rc4 -K 61 -e  -nopad    -nosalt -p
key=61000000000000000000000000000000

We see that there is a padding with '0's. ACtually, it doesn't want us to use a too small key (since for rc4 the key must be at least 40bits long). Now, let's try with a 128b key:
echo -ne "foobar" |  openssl  rc4 -K "6162636465666768696A6B6C6D6E6F70" -e  -nopad    -nosalt  | xxd
0000000: caaf 2cbf d334                           ..,..4

The result is the same as the one on the webpage :)

A: Work-in-progress
Here's an interesting pattern for you.  Using '0' as the encryption key we get some strong trends between the plaintext and the ciphertext.  See below.
What interests me about the difference between the two implementations is that fyne increases monotonically while OpenSSL is a bit of a stair-step.  I'll take another look at it later - I'm marking this a community wiki since I don't consider this an answer yet but I figured the analysis might help.
fyne:
0(0) = B8
0(1) = B9
0(2) = BA
0(3) = BB
0(4) = BC
0(5) = BD
0(6) = BE
0(7) = BF
0(8) = B0
0(9) = B1

OpenSSL:
0(0) = 72
0(1) = 73
0(2) = 70
0(3) = 71
0(4) = 76
0(5) = 77
0(6) = 74
0(7) = 75
0(8) = 7A
0(9) = 7B

Commands Used
cat -n N > /tmp/test #Where n is a number
openssl rc4 -e -nosalt -in /tmp/test -out /tmp/uuu
cat /tmp/uuu |xxd


text:
Q: POST method throws error 405 on npm http-server I am trying to make an Ajax request to run a php file on a local http server. I am getting an Error 405: method not allowed in my browser console.
I have tried the answers of some similar questions to no avail. I am using npm http-server to host this. I have tried enabling CORS on the http-server, which does not solve the problem.
I can narrow down my problem to the following code (using the answer given here).
/test.html:

        <title>Button</title>
        <meta charset="utf-8">

        <button type="button">Click Me</button>

/echo.php:
<?php
    header('Access-Control-Allow-Origin: *');
    header('Access-Control-Allow-Methods: GET, POST, OPTIONS');
    header('Access-Control-Allow-Headers: Content-Type, Accept');
    echo "HELLO WORLD";
?>

I start a server using http-server from npm. When I attempt to make POST requests (i.e., click the button), I get the following error in the server log:
[...] "POST /echo.php" Error (404): "Not found"

The echo.php file is definitely there -- I can make GET requests to the same url. I also see the following error in my browser console:
POST http://127.0.0.1:8080/echo.php 405 (Method Not Allowed)
    send @ jquery.min.js:4
    ajax @ jquery.min.js:4
    (anonymous function) @ test.html:15
    dispatch @ jquery.min.js:3
    q.handle @ jquery.min.js:3

When I make a GET request, no errors appear, and the code of echo.php is placed inside the
, as I would expect.
Is this a problem with my server configuration or just a code problem?
Update 1:
There isn't too much information I can get out of http-server. Here is the full log including the execution command:
$ http-server --cors -p 8080

Starting up http-server, serving ./
Available on:
  http://127.0.0.1:8080
Hit CTRL-C to stop the server
[Sat Sep 24 2016 16:47:14 GMT-0400 (EDT)] "GET /test.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.101 Safari/537.36"
[Sat Sep 24 2016 16:47:19 GMT-0400 (EDT)] "POST /echo.php" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.101 Safari/537.36"
[Sat Sep 24 2016 16:47:19 GMT-0400 (EDT)] "POST /echo.php" Error (404): "Not found"

Update 2
http-server does not support POST requests after all. See below for answer...

A: I figured it out! Kind of stupid, but hopefully this is helpful for other people who experience this issue. I was looking through the http-server repo and encountered the following GitHub issue.
npm http-server does not support POST requests. It is a read-only server, so has no code to facilitate this type of request.
I am now looking into JSON-server, which does support POST requests.


text:
Q: Why doesn't putting elinks in background and then resuming with `fg` work reliably? The elinks program takes over the terminal to let you browse the web. But its behavior in the context of job control is wonky.

*

*If I CTRL-Z suspend an elinks session, and then try to resume it with fg, it works.

*If I CTRL-Z suspend an elinks session, and then run some other program to completion (e.g. find), and then try to resume it with fg, it just prints elinks or elinks [the URL I invoked it with] to STDOUT and hangs. Then if I to CTRL-Z suspend again and fg a second time, elinks resumes for real, taking over the screen.
What explains this behavior? Is there a way to fix it?


text:
Q: Unknown function "asset" - Symfony 3 I migrated a project from Symfony 2.8 to 3.4.20 but there seems to be a problem with assetic not being detected at all.
I keep getting the message:

Did you forget to run "composer require symfony/asset"? Unknown function "asset".

I have installed symfony/asset but this didn't change anything.
Here is my config.yml
framework:
    templating:
        engines: ['twig']
    assets: ~

twig:
    debug:            "%kernel.debug%"
    strict_variables: "%kernel.debug%"

assetic:
    debug:          "%kernel.debug%"
    use_controller: "%kernel.debug%"
    bundles:
        - FOSUserBundle
        - MobiliteMainBundle
        - MobiliteUserBundle
    filters:
        cssrewrite: ~
    twig:
        functions:
            cssrewrite: ~

Here is the composer.json
{
  "name": "symfony/framework-standard-edition",
  "license": "MIT",
  "type": "project",
  "description": "FOS Mobilite",
  "autoload": {
    "psr-4": {
      "Mobilite\\MainBundle\\": "src/Mobilite/MainBundle",
      "Mobilite\\UserBundle\\": "src/Mobilite/UserBundle"
    },
    "classmap": [ "app/AppKernel.php", "app/AppCache.php" ]
  },
  "require": {
    "php": ">=7.2",
    "artscorestudio/datagrid-bundle": "^3.0",
    "doctrine/doctrine-bundle": "^1.6",
    "doctrine/doctrine-fixtures-bundle": ">=2.2.0",
    "doctrine/orm": "^2.5",
    "friendsofsymfony/user-bundle": "^2.1",
    "incenteev/composer-parameter-handler": "^2.0",
    "knplabs/knp-menu-bundle": ">=2.1.1",
    "knplabs/knp-snappy-bundle": ">=1.4",
    "paragonie/random_compat": "v2.0.17",
    "phpoffice/phpexcel": ">=1.8.1",
    "sensio/distribution-bundle": "^5.0.19",
    "sensio/framework-extra-bundle": "^5.0.0",
    "symfony/asset": "v3.4.20",
    "symfony/assetic-bundle": "^2.8",
    "symfony/monolog-bundle": ">=2.4",
    "symfony/swiftmailer-bundle": ">=2.3",
    "symfony/symfony": "3.4.*",
    "symfony/twig-bridge": "v3.4.20",
    "twig/extensions": ">=1.3.0",
    "twig/twig": "^1.0||^2.0",
    "white-october/pagerfanta-bundle": ">=v1.0.5"
  },
  "require-dev": {
    "sensio/generator-bundle": "^3.0",
    "symfony/phpunit-bridge": "^3.0"
  },
  "scripts": {
    "symfony-scripts": [
      "Incenteev\\ParameterHandler\\ScriptHandler::buildParameters",
      "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
      "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
      "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
      "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile",
      "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::prepareDeploymentTarget"
    ],
    "post-install-cmd": [
      "@symfony-scripts"
    ],
    "post-update-cmd": [
      "@symfony-scripts"
    ]
  },
  "config": {
    "platform": {
      "php": "7.2"
    },
    "sort-packages": true
  },
  "extra": {
    "symfony-app-dir": "app",
    "symfony-bin-dir": "bin",
    "symfony-var-dir": "var",
    "symfony-web-dir": "web",
    "symfony-tests-dir": "tests",
    "symfony-assets-install": "relative",
    "incenteev-parameters": {
      "file": "app/config/parameters.yml"
    },
    "branch-alias": {
      "dev-master": "3.4-dev"
    }
  }
}

And this is how I load my css
{% block stylesheets %}
    {% stylesheets
        '@MobiliteMainBundle/Resources/public/css/bootstrap.min.css'
        '@MobiliteMainBundle/Resources/public/css/bootstrap-theme.min.css'
        '@MobiliteMainBundle/Resources/public/css/jquery-ui-1.10.4.custom.min.css'
        '@MobiliteMainBundle/Resources/public/css/bootstrap-glyphicons.css'
        '@MobiliteMainBundle/Resources/public/css/global.css'
        '@MobiliteMainBundle/Resources/public/css/Desktop/select/select2.min.css'
        filter='cssrewrite' %}
        <link rel="stylesheet" href="{{ asset_url }}" />
    {% endstylesheets %}
{% endblock %}


text:
Q: Conditional flow of cells? I'd like to create interactive notebooks for class use, showing the step by step calculations needed for solving simple structural engineering problems, with the corresponding text paragraphs in cells explaining the purpose of every step performed.
However, these problems are usually not as "linear" as Mathematica notebooks are. For example, depending on the stress values, the set of formulas used might be completely different.
Imagine for example the problem is choosing a steel profile under bending: if we are in elastic regime, the procedure is different to the one if we were in plastic regime instead. You could argue "use two different notebooks then", but the point is that there still are a substantial number of common steps across the different cases.
So, what's the recommended way of doing this in Mathematica? Is there some sort of "conditional goto" from one cell to another?
I've read other questions suggesting to write functions for doing this. But, actually, the calculations will be the least important thing in my notebooks: their goal is the explanations describing the calculations procedure. Or even showing or not showing a graph, or a totally different type of graph depending on the values of previous cells.
So, I believe I'm not looking for writing functions, but for creating heavily non-linear and dynamically changing notebooks.
Thanks!!

A: This demo produces three tagged cells and defines a function next.  According to the criteria in next upon cell evaluation focus moves to the next stipulated cell.  It would be nice to put the next command as some kind of epilog function but I couldn't see how.  Also it would be nice to colour the background of the selected cell but that's quite tricky too, e.g. How to highlight cell.
Any ideas?
next := Which[
  DownValues[f] == {}, NotebookLocate[{InputNotebook[], "definef"}],
  Not[NumberQ[x]], NotebookLocate[{InputNotebook[], "setx"}],
  Not[NumberQ[y]], NotebookLocate[{InputNotebook[], "sety"}]]

SetAttributes[tagcell, HoldFirst]

tagcell[func_, tag_] := CellPrint[Cell[BoxData[ToBoxes[Defer[func]]],
   "Input", CellTags -> tag]]

tagcell[(y = x + f[3]; next), "sety"]
tagcell[(x = 1 + f[2]; next), "setx"]
tagcell[(f[x_] := x^2; next), "definef"]

next

y = x + f[3]; next

x = 1 + f[2]; next

f[x_] := x^2; next


text:
Q: Solving complicated equation involving integral of error functions I've been trying to solve the following equation for $\sigma$
$$\sigma^2 = \int_0^1 \left\{ \frac{1}{2} \operatorname{erfc} \left[ \frac{x + B}{\sqrt{2 (Rp - 1)} \, \sigma} \right] + \frac{1}{2} \operatorname{erfc} \left[ \frac{x - B}{\sqrt{2 (Rp - 1)} \, \sigma} \right] \right\}^{p - 1} \, x \, dx$$
with $B$, $p$ and $R$ fixed. In particular, I'm interested in the $B \to 0$ and $p \to \infty$ limits, but I'd like to take these limits only after solving the equation.
Any insight on how to do it? I've tried solving it numerically for some specific cases and seem to have found an asymptotic form, but I'm not convinced...
Thanks in advance :-)

A: Not a full answer (if such is possible at all), but note that $\text{erfc}(\cdot)$ is $O(1)$, so your expression becomes
$$
\sigma^2 = c^{p -1}\frac{1}{2}
$$
and so $\sigma^2$ either blows up to infinity or converges to 0.


text:
Q: Mimic elastic search index and queries on Java Lucene Hello I was using elastic search, but I would like to migrate to Java Lucene.
What I need is to recreate the index in the same way and to recreate the queries in the same way. I though I did it, but the scores I receive from the queries are not quite the same. The results are quite similar, but the score of the result itself is different. Maybe I am missing something. I cannot find enough info and default elastic settings are their equivalents in Lucene.
I have checked that my Lucene version is the same as on the elastic search that I used
I will show you my configurations in elastic and how I am reproducing it on Lucene.
This is my elastic search connector for creating the index:
"analysis": {
   "analyzer": {
            "default": {
               "tokenizer": "standard",
                "filter":  [ "lowercase", "asciifolding" ]
             }
    }
}

Those are my queries:
Query1)
 "query": {
      "dis_max": {
           "queries": [
                { "match": { "field1": "${term}" }}
           ],
      "tie_breaker": 1
     }
 }

Query 2)
 "query": {
      "dis_max": {
           "queries": [
                { "match": { "field1": "${term}" }},
                { "match": { "field2": "${term}" }},
                { "match": { "field2":  "${term}" }}
           ],
           "tie_breaker": 1
      }
 }

Below is Scala code:
Those are my field1, field2 and field3 options when creating the Lucene index:
 val CustomStringFieldType: FieldType = {
     val fieldType = new FieldType()
     fieldType.setStoreTermVectors(true)
     fieldType.setStoreTermVectorPositions(true)
     fieldType.setOmitNorms(false)
     fieldType.setTokenized(true)
     fieldType.setStored(false)
     fieldType.setIndexOptions(IndexOptions.DOCS_AND_FREQS_AND_POSITIONS)
     fieldType.freeze()
     fieldType
   }

I use doc.add(new Field("field1", field1Value, CustomStringFieldType)) to add the fields to a document.
Those are my queries. I am creating the tokens using Standard Analyzer and I am using BM25 similarity algorithm:
  def query1(tokens: Seq[String]): Query = {

     var booleanQueryBuilder = new BooleanQuery.Builder()

     for (token <- tokens) {
       booleanQueryBuilder = booleanQueryBuilder.add(
         new TermQuery(new Term("field1", token)),
         BooleanClause.Occur.SHOULD)
     }
     booleanQueryBuilder.build()
   }

 def query2(tokens: Seq[String]): Query = {

     val queries = new java.util.ArrayList[Query]()

     for (field <- ("field1", "field2", "field3") {

       var booleanQueryBuilder = new BooleanQuery.Builder()

       for (token <- tokens) {
         booleanQueryBuilder = booleanQueryBuilder.add(
           new TermQuery(new Term(field, token)),
           BooleanClause.Occur.SHOULD)
       }

       val booleanQuery: BooleanQuery = booleanQueryBuilder.build()
       queries.add(booleanQuery)
     }
     new DisjunctionMaxQuery(queries, broadQueryTieBreaker)
   }

A: The clear difference is your mappings settings.
While you're selected to have the following in Lucene
fieldType.setStoreTermVectors(true)
fieldType.setStoreTermVectorPositions(true)
fieldType.setOmitNorms(false)
fieldType.setTokenized(true)
fieldType.setStored(false)
fieldType.setIndexOptions(IndexOptions.DOCS_AND_FREQS_AND_POSITIONS)

By default, in Elasticsearch:
term_vector - disabled by default, so you need to have
PUT my_index
{
  "mappings": {
    "_doc": {
      "properties": {
        "text": {
          "type":        "text",
          "term_vector": "with_positions"
        }
      }
    }
  }
}

To store positions like you're doing in Lucene.
So, since you're migrating to Lucene you need to disable term vectors, like they were in your Elasticsearch


text:
Q: WebClient upload file I tried to upload a file using webclient like below
    val builder = MultipartBodyBuilder()
    builder.part("file", ByteArrayResource(multipartFile.bytes)).filename(multipartFile.name)

    webClient.post().uri(applicationProperties.uploadFile)
            .body(BodyInserters.fromMultipartData(builder.build()))
            .retrieve()
            .bodyToMono(UploadImageResponse::class.java)

but i have this error

{"timestamp":"2023-01-18T10:21:30.684+0000","status":415,"error":"Unsupported
Media Type","message":"Content type
'multipart/form-data;charset=UTF-8;boundary=jyVJSdCWsbbhz_2obRp_ZBwOtuzLG9yaLl'
not supported","path":"/api/ged/upload-file"}


text:
Q: How to add a work item tag within Visual Studio 2013 IDE? I'm using Visual Studio 2013 (Update 3) with git & Visual Studio Online as Source Control.
I can add tags to work items in the Visual Studio Online web experience but unable to do so in the Visual Studio 2013 IDE.
Is there any solution for it?

A: Adding and editing tags in Visual Studio 2013 has been enabled in Update 2. I'm currently running Visual Studio 2013 Update 4 and the functionality is available there.

Make sure you download and install the latest version of Visual Studio 2013 to get access to the latest goods.


text:
Q: Unbound value Mirage.load when executing make depend I'm trying to compile and build this simple Unikernel project https://github.com/MagnusS/mirage-stats-demo under Ubuntu 14.04 with latest versions of OCaml, OPAM and related packages, but when I execute the command make depend the following error appears:
make -C manifest
make[1]: Entering directory /home/osboxes/dev/mirage-stats-demo/manifest'
ocamlbuild manifest.native -pkgs unix,bytes,mirage
+ /usr/bin/ocamlc.opt -c -I /home/osboxes/.opam/system/lib/bytes -I
/home/osboxes/.opam/system/lib/cmdliner -I/home/osboxes/.opam/system/lib/fmt -I       /home/osboxes/.opam/system/lib/functoria -I /home/osboxes/.opam/system/lib/ipaddr -I /home/osboxes/.opam/system/lib/mirage -I /home/osboxes/.opam/system/lib/ocamlgraph -I /home/osboxes/.opam/system/lib/result -I /home/osboxes/.opam/system/lib/rresult -I /home/osboxes/.opam/system/lib/sexplib -I /usr/lib/ocaml -o manifest.cmo manifest.ml
File "manifest.ml", line 187, characters 12-23:
Error: Unbound value Mirage.load
Command exited with code 2.
Hint: Recursive traversal of subdirectories was not enabled for this build,
  as the working directory does not look like an ocamlbuild project (no
  '_tags' or 'myocamlbuild.ml' file). If you have modules in subdirectories,
  you should add the option "-r" or create an empty '_tags' file.

  To enable recursive traversal for some subdirectories only, you can use the
      following '_tags' file:
   true: -traverse
  <dir1> or <dir2>: traverse

Compilation unsuccessful after building 2 targets (0 cached) in 00:00:01.
make[1]: *** [all] Error 10
make[1]: Leaving directory `/home/osboxes/dev/mirage-stats-demo/manifest'
make: *** [manifest/manifest.native] Error 2