text:
Q: Postgres odd behavior with indices I've got a datavalue table with ~200M rows or so, with indices on both site_id and paramter_id. I need to execute queries like "return all sites with data" and "return all parameters with data". The site table has only 200 rows or so, and the parameter table has only 100 or so rows.
The site query is fast and uses the index:
EXPLAIN ANALYZE
select *
from site
where exists (
      select 1 from datavalue
      where datavalue.site_id = site.id limit 1
);

Seq Scan on site  (cost=0.00..64.47 rows=64 width=113) (actual time=0.046..1.106 rows=89 loops=1)
  Filter: (SubPlan 1)
  Rows Removed by Filter: 39
  SubPlan 1
    ->  Limit  (cost=0.44..0.47 rows=1 width=0) (actual time=0.008..0.008 rows=1 loops=128)
          ->  Index Only Scan using ix_datavalue_site_id on datavalue  (cost=0.44..8142.71 rows=248930 width=0) (actual time=0.008..0.008 rows=1 loops=128)
                Index Cond: (site_id = site.id)
                Heap Fetches: 0
Planning time: 0.361 ms
Execution time: 1.149 ms

The same query for parameters is rather slow and does NOT use the index:
EXPLAIN ANALYZE
select *
from parameter
where exists (
      select 1 from datavalue
      where datavalue.parameter_id = parameter.id limit 1
);

Seq Scan on parameter  (cost=0.00..20.50 rows=15 width=2648) (actual time=2895.972..21331.701 rows=15 loops=1)
  Filter: (SubPlan 1)
  Rows Removed by Filter: 6
  SubPlan 1
    ->  Limit  (cost=0.00..0.34 rows=1 width=0) (actual time=1015.790..1015.790 rows=1 loops=21)
          ->  Seq Scan on datavalue  (cost=0.00..502127.10 rows=1476987 width=0) (actual time=1015.786..1015.786 rows=1 loops=21)
                Filter: (parameter_id = parameter.id)
                Rows Removed by Filter: 7739355
Planning time: 0.123 ms
Execution time: 21331.736 ms

What the deuce is going on here? Alternatively, whats a good way to do this?
Some of the table description:
id BIGINT DEFAULT nextval('datavalue_id_seq'::regclass) NOT NULL,
value DOUBLE PRECISION NOT NULL,
site_id INTEGER NOT NULL,
parameter_id INTEGER NOT NULL,
deployment_id INTEGER,
instrument_id INTEGER,
invalid BOOLEAN,
Indexes:
    "datavalue_pkey" PRIMARY KEY, btree (id)
    "datavalue_datetime_utc_site_id_parameter_id_instrument_id_key" UNIQUE CONSTRAINT, btree (datetime_utc, site_id, parameter_id, instrument_id)
    "ix_datavalue_instrument_id" btree (instrument_id)
    "ix_datavalue_parameter_id" btree (parameter_id)
    "ix_datavalue_site_id" btree (site_id)
    "tmp_idx" btree (site_id, datetime_utc)
Foreign-key constraints:
    "datavalue_instrument_id_fkey" FOREIGN KEY (instrument_id) REFERENCES instrument(id) ON UPDATE CASCADE ON DELETE CASCADE
    "datavalue_parameter_id_fkey" FOREIGN KEY (parameter_id) REFERENCES parameter(id) ON UPDATE CASCADE ON DELETE CASCADE
    "datavalue_site_id_fkey" FOREIGN KEY (site_id) REFERENCES coastal.site(id) ON UPDATE CASCADE ON DELETE CASCADE
    "datavalue_statistic_type_id_fkey"

Edit: Here's the count distribution
select count(parameter_id), parameter_id from datavalue group by parameter_id

88169   14
2889171 8
15805   17
8570    12
4257262 21
3947049 15
1225902 2
4091090 3
103877  10
633764  11
994442  18
49232   20
14935   4
563638  13
2955919 7

A: Removing the limit 1 in  the sub query fixed the problem. More discussion on S.O.


text:
Q: Add date to url of blog page I'm building my first wagtail django site. My site has a blogging part and I'd like to add the publish date in the url. Currently, when adding a page, the URL becomes: example.com/blog/[slug] but I want it to be: example.com/blog/2015/11/19/[slug]
My blogpage:
class BlogPage(Page):
    main_image = models.ForeignKey(
        'wagtailimages.Image',
        null=True,
        blank=True,
        on_delete=models.SET_NULL,
        related_name='+'
    )
    date = models.DateField("Post date")
    intro = models.CharField(max_length=250)
    body = RichTextField(blank=True)

    search_fields = Page.search_fields + (
        index.SearchField('intro'),
        index.SearchField('body'),
    )

    content_panels = Page.content_panels + [
        FieldPanel('date'),
        ImageChooserPanel('main_image'),
        FieldPanel('intro'),
        FieldPanel('body'),
    ]

A: I'm not sure if you can integrate this with wagtail but here is an example of how you can achieve with django :

*

*Update your model to generate automatically the slug (E.g : based on title) in blog/models.py :

from django.db import models
from django.utils.text import slugify
from django.utils.timezone import now

class Post(models.Model):
    # your attributes
    date = models.DateTimeField(default=now())
    title = models.CharField(max_length=250)
    slug = models.SlugField()

    def save(self):
        """
        Generate and save slug based on title
        """
        super(Post, self).save()
        self.slug = slugify(self.title)
        super(Post, self).save()

*Add a new url in your blog app (blog/urls.py) :

url(r'^(?P<year>[0-9]{4})/(?P<month>[0-9]{2})/(?P<day>[0-9]{2})/(?P<slug>[\w-]+)$', views.post_details)

*Define the view to get the post in blog/views.py :

def post_details(request, year, month, day, slug):
    post = Post.objects.get(slug=slug, date__year=year, date__month=month, date__day=day)
    # do what you want with this post

A: Here is a more Wagtail specific way to implement this solution, it assumes you also have a BlogIndexPage that contains all the blog pages underneath. Although this would work even if your BlogPages exist under the HomePage, just move your route method accordingly.
Overview:

*

*We will be updating the sub-URLs on the BlogPageIndex (or whatever page that your blog pages are located within). We do this by adding the mixin RoutablePageMixin.

*We can then use @route as a decorator over a method on BlogPageIndex for any kind of custom views we want.

*Finally, we need to update our BlogPage model so its normal URL will follow our new URL mapping. We do this by overriding the set_url_path on our BlogPage.

*This will use the normal slug field from the BlogPage model and the field date_published to generate the new url_path.

See example /myapp/models.py
from django.shortcuts import get_object_or_404

from wagtail.contrib.routable_page.models import RoutablePageMixin, route
from wagtail.wagtailcore.models import Page

class BlogPage(Page):
    # using date_published instead of date to reduce ambiguity
    date_published = models.DateField(
        "Date post published", blank=True, null=True
    )
    # ...other fields:  main_image, intro, body
    # ...search_fields
    # ...content_panels

    # override the set_url_path method to use generate different URL
    # this is updated on save so each page will need to be re-published
    # old URL will still work
    def set_url_path(self, parent):
        # initially set the attribute self.url_path using the normal operation
        super().set_url_path(parent=parent)
        self.url_path = self.url_path.replace(
            self.slug, '{:%Y/%m/%d/}'.format(self.date_published) + self.slug
        )

class BlogIndexPage(RoutablePageMixin, Page):
    # note - must inherit RoutablePageMixin AND Page

    # ...other fields
    # ...search_fields
    # ...content_panels

    # route for sub-pages with a date specific URL for posts
    # this will NOT make a list of pages at blog/2018 just specific blogs only
    @route(r'^(?P<year>[0-9]{4})/(?P<month>[0-9]{2})/(?P<day>[0-9]{2})/(?P<slug>[\w-]+)/?$')
    def blog_page_by_date(self, request, year, month, day, slug, name='blog-by-date'):
        """Serve a single blog page at URL (eg. .../2018/01/23/my-title/)"""
        blog_page = get_object_or_404(
            BlogPage,
            date_published__year=year,
            date_published__month=month,
            date_published__day=day,
            slug=slug
        )
        return blog_page.serve(request)

    # Speficies that only BlogPage objects can live under this index page
    subpage_types = ['BlogPage']

Caveats:

*

*Each existing blog page will need to be re-saved to trigger the updates to the url_path.

*The original url_path will still work and not redirect, it is possible to add a redirect by overriding the serve method and checking if an old URL has been used, then redirect.

*This does not consider any SEO implications of having multiple URLs for pages.

redpajama_set_name: RedPajamaStackExchange


text:
Q: How to find $c$? from $T:\mathbb{R}^2\to \mathbb{R}^2$ How to find $c$? from $T:\mathbb{R}^2\to \mathbb{R}^2$ invertible given by $$T(x,y)=
\left(\begin{array}
15 & 14 \\
1 & 1 \end{array}\right) \left(\begin{array} xx\\ y  \end{array}\right)$$
and $$ T^{-1}(x,y)=\left(\begin{array} aax+by\\ cx+dy  \end{array}\right)  $$
I thought that I could do something like $T\cdot T^{-1}=I(x,y)$ but there is no Identity matrix from that composition, so how should I proceed? Also, I think is not the same $T\cdot T^{-1}$ and $T^{-1}\cdot T$ is it fine?
The other Idea by inverting $T$, I got  $T^{-1}(x,y)= \left(\begin{array}  z1 & -14 \\ -1 & 15 \end{array}\right)  \left(\begin{array} xx\\ y  \end{array}\right)= (x - 14 y, -x + 15 y)$ okay from here $c=-1$.


text:
Q: I want to block these 2 types of visitors to a site. How do I do this? I want to block these types of requests to my site. How is the best way to do it? I have listed what I have tried and my ideas so far.
I don't need any code just a pointer in the right direction.
I want to use the best method possible based on speed and I also don't want to block genuine people?
The site uses:
- PHP
- Symfony2
- Apache
- Ubuntu

I could easily do both of these in PHP, but thinking it may be better using Apache/Ubuntu etc.
(1) Block certain IP addresses. This would only be ones clearly doing something dodgy. E.g. a range of IP addresses from China if I have proven that all those IP addresses do is something dodgy.
I currently log the IP of every visitor. I later then manually look at this data and if I decide an IP (or range of) needs blocking I would then take manual action.
(2) Block anyone accessing my site by using the server IP address (e.g. 123.55.44.222) instead of the actual 'domain.com' domain name. I want to do this as it messes up the site (no doubt this could be resolved by changing my code, but nobody enters the IP so it's easier to do this).
With this I want a piece of code to detect if they are using the IP and just block them immediately (maybe return a 403 error)
Thanks.

A: *

*Block them at the firewall.

*Either fix your site to not be broken when accessed via IP, or change apache's configuration to not serve it as the default site.
That aside, obsessively reading your logs and trying to block "people doing bad things" is going to drive you [further] into madness, and is generally pointless. You're going to whittle away your remaining free time and sanity playing whackamole with botnets, proxies, and script kiddies.
Fix your code. Address the fact that people are doing these "bad things" by removing their ability to do so.

A: Spamhaus already maintains a blacklist of IP addresses and updates it every day. You can use this script to automatically fetch new list every day and update firewall to block the ip addresses.
If you wish to maintain your own blacklist, you can just replace the logic in the script that pertains to fetching from Spamhaus.
For second part of your question, you can do one of these

*

*Redirect a user to the domain name based url, if ip address was used. See this for an example

*Define a virtual host to match the ip address as the first virtual host. Here throw a HTTP 403 erro


text:
Q: A dimension-free upper bound for $\| X \|_{\infty}$ when $X \sim N(0,\Sigma)$ For a $p$-dimensional random vector $X \sim N(0,\Sigma)$, it is known that with high probability
$$
\| X \|_{\max} \le C \sqrt{\log p}
$$
where $C$ is some constant possibly depending on $\Sigma$. I would like to know if there is a simple dimension free bound that generalizes this result.
For context on a related problem, consider the $\| \cdot \|_2$ setting, then example Exercise 6.3.5. in Vershynin's High Dimensional Probability states that for a $m \times n$ constant matrix $B$, $Z \sim N(0,I)$, that
$$
P( \| BZ\|_{2} \ge C \| B \|_F + t) \le \exp \left ( -\frac{ct^2}{\| B \|^2_\text{op}}\right)
$$
where the subscript $F$ is the frobenius norm. Choosing $B = \sqrt{\Sigma}$ yields
$$
P( \| X \|_{2} \ge C \sqrt{\text{Tr}(\Sigma)} + t) \le \exp \left ( -\frac{ct^2}{\|\Sigma\|^2_{\text{op}}}\right),
$$
so that with high probability $\| X \|_2 \le C\sqrt{\text{Tr}(\Sigma)}$. In the worst case, $\Sigma = I$ and we get back the standard bound $\| X \|_2 \le C\sqrt{p}$, but for many problems of interest $\Sigma$ has a nice structure that makes the first bound more useful.
Is there a similar bound that generalizes the $\sqrt{\log p}$ bound? Ideally it would involve the trace, $\|X\|_{\max} \stackrel{?}{\le} C \sqrt{\log \text{Tr}(\Sigma)}$.
Note also that all results mentioned hold more generally for subGaussian random variables (with different universal constants), but for simplicity I am just looking at the Gaussian case.
An update: I have found a related dimension free result, Lemma 2.3 of this paper by Van Handel states that for $X_1,\dots, X_n$ be not necessarily independent subGaussian random variables with subGaussian norm $\sigma_i$, then
$$
E [\max_{i \le n} X_i] \le C \max_{i \le n} \sigma_{(i)} \sqrt{\log(i+1)}
$$
where $\sigma_{(1)} \ge \sigma_{(2)} \ge \dots \sigma_{(n)}$  are the ordered subGaussian norms. If the $X$'s are independent Gaussian there is a matching lower bound.


text:
Q: Copy last cell value from multiple created sheets to Checklist sheet file VBA I've extracted new worksheets based on a cell value (based on currency) in short I've filtered them and extracted (Since values can be added every now and then) with function of creating worksheets. now my problem is I need to copy a specific cell value from the created worksheets and paste it to a worksheet called ("Checklist"). I need to copy the last cell value from Column "U"
here's my code so far
 Sub SortCurrency()
    Dim currRange As Range, dataRng As Range, currCell As Range
    Call DeleteSheets
    Dim ws As Worksheet, ws2 As Worksheet

    Dim lastcol As Long, lastRowAC As Long, lastrowSrc As Long, lastRow As Long, lastrow2 As Long, LastRowU As Long

    Dim internalR As Long, internalUCheck As Long, internalS As Long, internalT As Long, internalU As Long, internalV As Long, internalW As Long, internalX As Long, internalY As Long, internalZ As Long, internalAE As Long
    Dim copyR As Range, copyS As Range, copyT As Range, copyU As Range, copyV As Range, copyW As Range, copyX As Range, copyY As Range, copyZ As Range, copyAE As Range
    Set copyR = Range("R" & Rows.Count).End(xlUp) 'find lastrow of column R
    Set copyS = Range("S" & Rows.Count).End(xlUp)
    Set copyT = Range("T" & Rows.Count).End(xlUp)
    Set copyU = Range("U" & Rows.Count).End(xlUp)
    Set copyV = Range("V" & Rows.Count).End(xlUp)
    Set copyW = Range("W" & Rows.Count).End(xlUp)
    Set copyX = Range("X" & Rows.Count).End(xlUp)
    Set copyY = Range("Y" & Rows.Count).End(xlUp)
    Set copyZ = Range("Z" & Rows.Count).End(xlUp)
    Set copyAE = Range("AE" & Rows.Count).End(xlUp)
    Set ws2 = Worksheets("Checklist")

    lastRowAC = ws2.Range("A2").End(xlDown).Row
    With Worksheets("JV501")

    .Select
        Set currRange = .Range("AB1", .Cells(.Rows.Count, "AB").End(xlUp))
        Set dataRng = Intersect(.UsedRange, currRange.EntireRow)
        lastcol = Range("A1").End(xlToRight).Column
        lastRow = Range("AB2").End(xlDown).Row
        Range("AB2:AB" & lastRow).Sort key1:=Range("AB2" & lastRow), _
        order1:=xlAscending, Header:=xlNo
Range("AF:XFD").Delete
        With .UsedRange
            .Resize(1, 1).Offset(0, lastcol - 1).Select
             With .Resize(1, 1).Offset(0, lastcol)
                With .Resize(currRange.Rows.Count)
                .Value = currRange.Value
                .RemoveDuplicates Array(1), Header:=xlYes
                Range("AB:AB").Copy Destination:=Worksheets("Checklist").Range("A1")
                For Each currCell In .SpecialCells(xlCellTypeConstants)
                    currRange.AutoFilter , field:=1, Criteria1:=currCell.Value

                    If Application.WorksheetFunction.Subtotal(103, currRange) - 1 > 0 Then
                        dataRng.SpecialCells(xlCellTypeVisible).Copy Destination:=GetOrCreateWorksheet(currCell.Value).Range("A1")

                        'subtotal of debit
                            lastrowSrc = Range("AC" & Rows.Count).End(xlUp).Row + 1
                            Range("AC" & lastrowSrc & ":AC" & lastrowSrc).Formula = "=SUBTOTAL(9,AC2:AC" & lastrowSrc - 1 & ")"

                            'copy ac to ad
                            Range("AC" & lastrowSrc & ":AC" & lastrowSrc).Cut Destination:=Range("AC" & lastrowSrc).Offset(0, 1)
                            Range("AC" & lastrowSrc).Offset(0, 1).Value = Range("AC" & lastrowSrc).Offset(0, 1).Value
  ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
                            internalR = Range("R" & Rows.Count).End(xlUp).Row + 1
                            copyR.Copy Destination:=Range("R" & internalR)

                            internalS = Range("S" & Rows.Count).End(xlUp).Row + 1
                            copyS.Copy Destination:=Range("S" & internalS)

                            internalT = Range("T" & Rows.Count).End(xlUp).Row + 1
                            copyT.Copy Destination:=Range("T" & internalT)

                            internalU = Range("U" & Rows.Count).End(xlUp).Row + 1
                            copyU.Copy Destination:=Range("U" & internalU)

                            internalV = Range("V" & Rows.Count).End(xlUp).Row + 1
                            copyV.Copy Destination:=Range("V" & internalV)

                            internalW = Range("W" & Rows.Count).End(xlUp).Row + 1
                            copyW.Copy Destination:=Range("W" & internalW)

                            internalX = Range("X" & Rows.Count).End(xlUp).Row + 1
                            copyX.Copy Destination:=Range("X" & internalX)

                            internalY = Range("Y" & Rows.Count).End(xlUp).Row + 1
                            copyY.Copy Destination:=Range("Y" & internalY)

                            internalZ = Range("Z" & Rows.Count).End(xlUp).Row + 1
                            copyZ.Copy Destination:=Range("Z" & internalZ)

                            internalAE = Range("AE" & Rows.Count).End(xlUp).Row + 1
                            copyAE.Copy Destination:=Range("AE" & internalAE)
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''

                        Range("J:Q").EntireColumn.Delete
                        Range("A:A").EntireColumn.Delete
                        Range("U:U").NumberFormat = "###,###,###.00"
                        Range("U:U").Font.Size = "10"

                        Columns("A:V").Select
                        Selection.EntireColumn.AutoFit
                        internalUCheck = Range("U" & Rows.Count).End(xlUp).Row
                         Range("U" & internalUCheck).Copy Destination:=Sheets("Checklist").Range("C2:C" & lastRowAC) '' this is the code I've got''
                    End If

                 Next currCell
                   Range("W:XFD").Delete
                 .ClearContents
                End With

             End With
         End With
         .AutoFilterMode = False
         Worksheets("Checklist").Activate
         Call countCurrency
         Call removeDups

        Worksheets("JV501").Activate
        Range("AF:XFD").Delete
        Range("A1").Activate

    End With
    End Sub

with this code for copying values in the loop
internalUCheck = Range("U" & Rows.Count).End(xlUp).Row
                             Range("U" & internalUCheck).Copy Destination:=Sheets("Checklist").Range("C2:C" & lastRowAC) '' this is the code I've got''

the thing is it only copies the last cell value from the last created sheet
example of created and extracted sheet (there are more than 10 created sheets and may add more in time depending on the user)
i.e Sheets("AED")
currency    |  debit   |  Credit
AED         |  25,000  |
            |          |  25,000

i.e Sheets("HKD")
currency    |  debit   |  Credit
 HKD        |  18,000  |
            |          |  18,000

i.e Sheets("TRY")
currency    |  debit   |  Credit
TRY         |  35,000  |
            |          |  35,000

from those sample sheets I need to copy the last cell value of Credit column (Col."U") to sheet ("Checklist"). more of consolidating values into one sheet
the output file is like this with my code
Currency    |  Count   | Credit
AED         |    6     |  35,000
USD         |    35    |  35,000
HKD         |    18    |  35,000
TRY         |    8     |  35,000

as you can see it doesn't loop with every created sheets, only getting the last cell value of column U in the last created sheet.
I've tried inserting inside loop but no good.
it should be like this: ("Checklist sheet")
    Currency    |  Count   | Credit
    AED         |    6     |  25,000
    USD         |    35    |  08,000
    HKD         |    18    |  18,000
    TRY         |    8     |  35,000

every help is appreciated! Thanks

A: I'd say your issue is you miss lastRowAC updating at every new sheet loop, like:
lastRowAC = lastRowAC + 1

before or after (it's up to your actual loop "twisting"):
Range("U" & internalUCheck).Copy Destination:=Sheets("Checklist").Range("C2:C" & lastRowAC)

I'd also add that your first lastRowAC setting, namely
lastRowAC = ws2.Range("A2").End(xlDown).Row

is prone to error since should you have no values below cell "A2" at your macro first run it'd return the last row index (some 1048576 from excel 2007  on)
it's safer to use:
lastRowAC = ws2.Cells(ws.Rows.Count, 1).End(xlUp).Row

and get last not empty row in column 1 of worksheet ws2, with the only warning that should column A contain no values or only one value in cell A1 it'd return 1 either way. In this case you may want to add a check and handle the return value properly


text:
Q: Very high error after full integer quantization of a regression network I have trained a fully connected neural network with one hidden layer of 64 nodes. I am testing with the Medical Cost dataset. With the original precision model, the mean absolute error is 0.22063259780406952. With a model quantized to float16 or integer quantization with float fallback, the difference between the original error and the low precision model's is never more than 0.1. However, if I do full integer quantization, the error shoots to unreasonable amounts. In this particular case, it jumps to nearly 60. I have no idea if this is a bug in TensorFlow, or if I'm using the APIs incorrectly or if this is a reasonable behavior after quantization. Any help is appreciated. The code showing the conversion and inference is shown below:

*

*Preprocessing

import math
import pathlib
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
import numpy as np
import pandas as pd
from sklearn import preprocessing as pr
from sklearn.metrics import mean_absolute_error

url = 'insurance.csv'
column_names = ["age", "sex", "bmi", "children", "smoker", "region", "charges"]

dataset = pd.read_csv(url, names=column_names, header=0, na_values='?')

dataset = dataset.dropna()  # Drop rows with missing values
dataset['sex'] = dataset['sex'].map({'female': 2, 'male': 1})
dataset['smoker'] = dataset['smoker'].map({'yes': 1, 'no': 0})

dataset = pd.get_dummies(dataset, prefix='', prefix_sep='', columns=['region'])

# this is a trick to convert a dataframe to 2d array, scale it and
# convert back to dataframe
scaled_np = pr.StandardScaler().fit_transform(dataset.values)
dataset = pd.DataFrame(scaled_np, index=dataset.index, columns=dataset.columns)

*

*Train and Test split

train_dataset = dataset.sample(frac=0.8, random_state=0)
test_dataset = dataset.drop(train_dataset.index)

train_features = train_dataset.copy()
test_features = test_dataset.copy()

train_labels = train_features.pop('charges')
test_labels = test_features.pop('charges')

*

*Original model training

def build_and_compile_model():
    model = keras.Sequential([
        layers.Dense(64,
                     activation='relu',
                     input_shape=(len(dataset.columns) - 1, )),
        layers.Dense(1)
    ])

    model.compile(loss='mean_absolute_error',
                  optimizer=tf.keras.optimizers.Adam(0.001))
    return model

dnn_model = build_and_compile_model()
dnn_model.summary()

dnn_model.fit(train_features,
              train_labels,
              validation_split=0.2,
              verbose=0,
              epochs=100)

print("Original error = {}".format(
    dnn_model.evaluate(test_features, test_labels, verbose=0)))

*

*Conversion to lower precision model

converter = tf.lite.TFLiteConverter.from_keras_model(dnn_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]

def representative_data_gen():
    for input_value in tf.data.Dataset.from_tensor_slices(
            train_features.astype('float32')).batch(1).take(100):
        yield [input_value]

converter.representative_dataset = representative_data_gen

# Full Integer Quantization
# Ensure that if any ops can't be quantized, the converter throws an error
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
# Set the input and output tensors to uint8 (APIs added in r2.3)
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8

tflite_model_quant = converter.convert()

dir_save = pathlib.Path(".")
file_save = dir_save / "model_16.tflite"
file_save.write_bytes(tflite_model_quant)

*

*Instantiate the TFLite model

interpreter = tf.lite.Interpreter(model_path=str(file_save))
interpreter.allocate_tensors()

*

*Evaluate the lower precision model

def evaluate_model(interpreter, test_images, test_labels):
    input_details = interpreter.get_input_details()[0]
    input_index = interpreter.get_input_details()[0]["index"]
    output_index = interpreter.get_output_details()[0]["index"]

    # Run predictions on every image in the "test" dataset.
    prediction_digits = []
    for test_image in test_images:
        if input_details['dtype'] == np.uint8:
            input_scale, input_zero_point = input_details['quantization']
            test_image = test_image / input_scale + input_zero_point

        test_image = np.expand_dims(test_image,
                                    axis=0).astype(input_details['dtype'])
        interpreter.set_tensor(input_index, test_image)

        # Run inference.
        interpreter.invoke()

        output = interpreter.get_tensor(output_index)
        prediction_digits.append(output[0])

    filtered_labels, correct_digits = map(
        list,
        zip(*[(x, y) for x, y in zip(test_labels, prediction_digits)
              if not math.isnan(y)]))
    return mean_absolute_error(filtered_labels, correct_digits)

print(evaluate_model(interpreter, test_features[:].values, test_labels))

A: When you are doing quantization (and machine learning in general), you need to be careful at what your data looks like. Will applying a certain level of quantization make sense with the data you have?
In the case of a regression problem like yours, with ground truth in the range [1121.8739;63770.42801], and some input data that is also in float, it is likely that training a model with that data, and then quantizing it in integer won't yield good results.
You trained the model to output values in the range [1121.8739;63770.42801], and after quantization in int8, it will be able to output only in the range [-127;128], without decimal points. Obviously, when you will compare the results of the quantized model with your ground-truth, the error will jump through the roof.
What can you do if you still want to apply quantization? You need to shift your data in the domain of the quantized set. In your case, convert your float32 data to int8 in a way they still make sense. You will see a big drop in performance in real use case. After all, with a regression problem, you shift from a domain of roughly 25 millions possible output values (assuming a mantissa of 23 bits and 8 bits of exponents, see Single Precision Floating Point and  How many floating-point numbers are in the interval [0,1]?), to a domain with 256 (2^8) possible outputs.
But a really really naive approach could be to apply the following transform :
def scale_down_data(data):
  max_value = data.max()
  min_value = data.min()
  # normalizing between -128 and 127
  scaled_down = 255*((data-min_value)/(max_value-min_value)) -128
  return scaled_down.astype(np.int8)

In practice, it would be better to look at the distribution of your data, and do a transform that gives you more range where the data is denser. You also don't want to limit the range of your regression to the bounds of your training set. And you need to do that analysis for every input or output that is not in the quantized domain.


text:
Q: If I do not have a specific target, is my travelling "targetless"? I'm writing on an english article in the domain of navigation and geographic routing. I have a use case for targetless navigation but I'm unsure if the term is the correct choise. I see targetless is an existing word in the english language, and it literally means without target.
In my specific use case, and in the oppisite to a traditional A-to-B navigation like you use in your car, I'm looking for a term or phrase that descibres the state that

*

*I'm in a explorative navigation mode

*I'm free to go where I want (as long as I can reach it somehow)

*Describes something scientists call A-to-N navigation [1]

Is targetless navigation the term I'm looking for?
[1] The correct scientific term is the single source shortest path (SSSP) problem class in case anyone wants to read up on that matter.

A: If something that normally takes a target does not have that target set yet, try untargeted.
Though this can also refer to something that is not yet targeted by anything as well.

A: If you are ok with something informal, you could use the neologism "destinationless"


text:
Q: How to compare data in a sqlite column to a const char in c++ I'm trying to compare all column data against a const char to see if the name in the column matches the player name that is connected to the game server. The comparison in the spaghetticode below does not work properly and results in new rows being made even if the player name already exists in the sqlite database. What is the correct way to do this comparison? Thanks in advance for your help.
Here's the snippet with the problematic section:
if(enable_sqlite_db) {
            sqlite3_stmt *stmt;
            sqlite3 *db;
            char *zErrMsg = 0;
            int  rc;
            const char *sql;
            bool name_match = false;
            const char* player_database_names;
            char *p_name = ci->name;
            char *p_ip = ci->ip;

            rc = sqlite3_open("playerinfo.db", &db);
            if( rc ){
                fprintf(stderr, "Can't open database: %s\n", sqlite3_errmsg(db));
                exit(0);
            }else{
                if(sql_console_msgs) fprintf(stdout, "Opened database successfully\n");
            }
            if( rc != SQLITE_OK ){
                fprintf(stderr, "SQL Database Error: %s\n", zErrMsg);
                sqlite3_free(zErrMsg);
            }else{
                defformatstring(sqlstrprep)("SELECT NAME FROM PLAYERINFO");
                rc = sqlite3_prepare_v2(db, sqlstrprep, -1, &stmt, NULL);

                while ((rc = sqlite3_step(stmt)) == SQLITE_ROW) {
                    int id = sqlite3_column_int(stmt, SQLITE_ROW);
                    int columns = sqlite3_column_count(stmt);
                    if(sql_console_msgs) out(ECHO_CONSOLE, "-- id: %d row: %d columns: %d", id, SQLITE_ROW, columns);
                    player_database_names = reinterpret_cast<const char*>(sqlite3_column_text(stmt, 0));
                }
                if(player_database_names == p_name) name_match = true;
                else if(player_database_names != p_name) name_match = false;
            }

Here is the full code:
  static int callback(void *NotUsed, int argc, char **argv, char **azColName){
        int i;
        for(i=0; i<argc; i++){
            printf("%s = %s\n", azColName[i], argv[i] ? argv[i] : "NULL");
        }
        printf("\n");
        return 0;
    }

    bool sql_console_msgs = true;
    void QServ::savestats(clientinfo *ci)
    {
        if(enable_sqlite_db) {
            sqlite3_stmt *stmt;
            sqlite3 *db;
            char *zErrMsg = 0;
            int  rc;
            const char *sql;
            bool name_match = false;
            const char* player_database_names;
            char *p_name = ci->name;
            char *p_ip = ci->ip;

            rc = sqlite3_open("playerinfo.db", &db);
            if( rc ){
                fprintf(stderr, "Can't open database: %s\n", sqlite3_errmsg(db));
                exit(0);
            }else{
                if(sql_console_msgs) fprintf(stdout, "Opened database successfully\n");
            }
            if( rc != SQLITE_OK ){
                fprintf(stderr, "SQL Database Error: %s\n", zErrMsg);
                sqlite3_free(zErrMsg);
            }else{
                defformatstring(sqlstrprep)("SELECT NAME FROM PLAYERINFO");
                //defformatstring(sqlstrprep)("SELECT group_concat(NAME) FROM PLAYERINFO");
                rc = sqlite3_prepare_v2(db, sqlstrprep, -1, &stmt, NULL);

                while ((rc = sqlite3_step(stmt)) == SQLITE_ROW) {
                    //sometimes returns incorrectly - keeps creating undesired new rows firstguy, secondguy, firstguy
                    int id = sqlite3_column_int(stmt, SQLITE_ROW);
                    int columns = sqlite3_column_count(stmt);
                    if(sql_console_msgs) out(ECHO_CONSOLE, "-- id: %d row: %d columns: %d", id, SQLITE_ROW, columns);
                    player_database_names = reinterpret_cast<const char*>(sqlite3_column_text(stmt, 0));
                    //if (std::to_string(player_database_names).find(p_name) != std::string::npos) name_match = true;
                    //else name_match = false;
                    if(sql_console_msgs) out(ECHO_CONSOLE, "-- player db names: %s", player_database_names);
                }
                //if(!strcmp(player_database_names, p_name)) name_match = true;
                //else if(strcmp(player_database_names, p_name)) name_match = false;
                if(player_database_names == p_name) name_match = true;
                else if(player_database_names != p_name) name_match = false;
            }

            sql = "CREATE TABLE IF NOT EXISTS PLAYERINFO("    \
            "NAME                       TEXT    NOT NULL,"    \
            "FRAGS                       INT    NOT NULL,"    \
            "DEATHS                      INT    NOT NULL,"    \
            "FLAGS                       INT    NOT NULL,"    \
            "PASSES                      INT    NOT NULL,"    \
            "IP                         TEXT    NOT NULL,"    \
            "ACCURACY          DECIMAL(4, 2)    NOT NULL,"    \
            "KPD               DECIMAL(4, 2)    NOT NULL);";
            rc = sqlite3_exec(db, sql, callback, 0, &zErrMsg);
            if( rc != SQLITE_OK ){
                fprintf(stderr, "SQLITE3 ERROR @ CREATE TABLE IF NOT EXISTS: %s\n", zErrMsg);
                sqlite3_free(zErrMsg);
            }else{
                if(sql_console_msgs) {
                    if(!name_match) fprintf(stdout, "-- No previous record found under that name\n");
                    else fprintf(stdout, "-- Found name already, updating record instead\n");
                }
            }

            char sqlINSERT[500];
            char sqlUPDATE[1000];
            int p_frags = ci->state.frags;
            int p_deaths = ci->state.deaths;
            int p_flags = ci->state.flags;
            int p_passes = ci->state.passes;
            int p_acc = (ci->state.damage*100)/max(ci->state.shotdamage, 1);
            int p_kpd = (ci->state.frags)/max(ci->state.deaths, 1);

            //name is different
            if(!name_match) {
                snprintf(sqlINSERT, 500, "INSERT INTO PLAYERINFO( NAME,FRAGS,DEATHS,FLAGS,PASSES,IP,ACCURACY,KPD ) VALUES (\"%s\", %d, %d, %d, %d, \"%s\", %d, %d)",p_name,p_frags,p_deaths,p_flags,p_passes,p_ip,p_acc,p_kpd);
                //sqlEscape(sqlINSERT);
                rc = sqlite3_exec(db, sqlINSERT, callback, 0, &zErrMsg);
            }
            //client name matches db record, update db if new info is > than db info
            else if(name_match)  {
                snprintf(sqlUPDATE, 10000,
                        "UPDATE PLAYERINFO SET FRAGS = %d+(SELECT FRAGS FROM PLAYERINFO) WHERE NAME = \"%s\";"     \
                        "UPDATE PLAYERINFO SET DEATHS = %d+(SELECT DEATHS FROM PLAYERINFO) WHERE NAME = \"%s\";"   \
                        "UPDATE PLAYERINFO SET FLAGS = %d+(SELECT FLAGS FROM PLAYERINFO) WHERE NAME = \"%s\";"     \
                        "UPDATE PLAYERINFO SET PASSES = %d+(SELECT PASSES FROM PLAYERINFO) WHERE NAME = \"%s\";"   \
                        "UPDATE PLAYERINFO SET ACCURACY = %d+(SELECT ACCURACY FROM PLAYERINFO) WHERE NAME = \"%s\";" \
                        "UPDATE PLAYERINFO SET KPD = %d+(SELECT KPD FROM PLAYERINFO) WHERE NAME = \"%s\";",
                        ci->state.frags, ci->name, ci->state.deaths, ci->name, ci->state.flags, ci->name, ci->state.passes, ci->name, p_acc, ci->name, p_kpd, ci->name);
                //sqlEscape(sqlUPDATE);
                rc = sqlite3_exec(db, sqlUPDATE, callback, 0, &zErrMsg);
            }
            if( rc != SQLITE_OK ){
                fprintf(stderr, "SQLITE3 ERROR @ INSERT & UPDATE: %s\n", zErrMsg);
                sqlite3_free(zErrMsg);
            }else{
                if(sql_console_msgs) fprintf(stdout, "Playerinfo modified\n");
            }
            sqlite3_close(db);
        }
    }

    void QServ::getstats(clientinfo *ci)
    {
        if(enable_sqlite_db) {
            sqlite3 *db;
            char *zErrMsg = 0;
            int rc;
            char *sql;
            const char* data = "Callback function called";

            rc = sqlite3_open("playerinfo.db", &db);
            if( rc ){
                fprintf(stderr, "Can't open database: %s\n", sqlite3_errmsg(db));
                exit(0);
            }

            if( rc != SQLITE_OK ){
                fprintf(stderr, "SQL Database Error: %s\n", zErrMsg);
                sqlite3_free(zErrMsg);
            }else{
                sqlite3_stmt *stmt;
                defformatstring(sqlstrprep)("SELECT NAME,FRAGS,ACCURACY,KPD FROM PLAYERINFO WHERE NAME == \"%s\";", ci->name);
                rc = sqlite3_prepare_v2(db, sqlstrprep, -1, &stmt, NULL);

                bool necho = false;
                while ((rc = sqlite3_step(stmt)) == SQLITE_ROW) {
                    const char* name = reinterpret_cast<const char*>(sqlite3_column_text(stmt, 0));
                    const char* allfrags = reinterpret_cast<const char*>(sqlite3_column_text(stmt, 1));
                    const char* avgacc = reinterpret_cast<const char*>(sqlite3_column_text(stmt, 2));
                    const char* avgkpd = reinterpret_cast<const char*>(sqlite3_column_text(stmt, 3));
                    if(!necho) {
                        if(avgacc == NULL) out(ECHO_SERV, "Name: \f0%s\f7, Total Frags: \f3%s\f7, Average KPD: \f6%s", name, allfrags, avgkpd);
                        else if(avgkpd == NULL) out(ECHO_SERV, "Name: \f0%s\f7, Total Frags: \f3%s\f7, Average Accuracy: \f2%s%%", name, allfrags, avgacc);
                        else out(ECHO_SERV, "Name: \f0%s\f7, Total Frags: \f3%s\f7, Average Accuracy: \f2%s%%\f7, Average KPD: \f6%s", name,allfrags,avgacc,avgkpd);
                        necho = true;
                    }
                }
            }
            sqlite3_close(db);
        }
    }

    void QServ::getnames(clientinfo *ci) {
        if(enable_sqlite_db) {
            sqlite3_stmt *stmt3;
            sqlite3 *db;
            int rc;
            rc = sqlite3_open("playerinfo.db", &db);
            defformatstring(sqlstrprep3)("SELECT group_concat(NAME, \", \") FROM PLAYERINFO WHERE IP == \"%s\";", ci->ip);
            rc = sqlite3_prepare_v2(db, sqlstrprep3, -1, &stmt3, NULL);
            while ((rc = sqlite3_step(stmt3)) == SQLITE_ROW) {
                std::string names(reinterpret_cast<const char*>(sqlite3_column_text(stmt3, 0)));
                defformatstring(nmsg)("Names from IP \f2%s\f7: %s", ci->ip, names.c_str());
                out(ECHO_SERV, nmsg);
            }
            sqlite3_close(db);
        }
    }

    void QServ::disconnectclient(clientinfo *ci)
    {
        if(enable_sqlite_db) savestats(ci);
    }

A: I'm always amused by the fact that nobody wants to provide a minimal working example. In the end, I created one myself to show how to do this thing:
#include <stdlib.h>
#include <stdio.h>
#include <stdbool.h>
#include <string.h>

#include "sqlite3.h"

int main(void)
{
    int rc;
    sqlite3 *db;
    rc = sqlite3_open("playerinfo.db", &db);
    if (rc != SQLITE_OK) {
        fprintf(stderr, "Can't open database: %s\n", sqlite3_errmsg(db));
        exit(EXIT_FAILURE);
    }
    else {
        fprintf(stdout, "Opened database successfully\n");
    }

    char *zErrMsg = NULL;
    rc = sqlite3_exec(db,
        "CREATE TABLE IF NOT EXISTS PlayerInfo (id INTEGER PRIMARY KEY, name TEXT);"
        "REPLACE INTO PlayerInfo(id, name) VALUES (1,'John'),(2,'Paul'),(3,'George'),(4,'Ringo'),(5,'Pete'),(6,'Stuart');",
        NULL, 0, &zErrMsg);
    if (rc != SQLITE_OK) {
        fprintf(stderr, "SQL Database Error: %s\n", zErrMsg);
        sqlite3_free(zErrMsg);
        exit(EXIT_FAILURE);
    }

    const char *search_name = "Ringo";
    bool name_match = false;

    sqlite3_stmt *stmt;
    char *zSql = "SELECT name FROM PlayerInfo";
    sqlite3_prepare_v2(db, zSql, -1, &stmt, NULL);
    while (sqlite3_step(stmt) == SQLITE_ROW) {
        const char* player_database_names = sqlite3_column_text(stmt, 0);

        if (strcmp(player_database_names, search_name) == 0) {
            name_match = true;
            break;
        }
    }
    sqlite3_finalize(stmt);

    printf("Search name: %s - Result: %s\n", search_name, name_match ? "true" : "false");

    sqlite3_close(db);
    return EXIT_SUCCESS;
}

The original code had a lot of very strange things. It clearly looks as if a bunch of stuff was pasted together without reason.
I never used SQLite before, but it's really very well done. So the idea is to

*

*Open the DB (sqlite3_open()) and check if successfull.

*Create and populate the DB if needed (sqlite3_exec() without a callback function).

*Query the DB rows with sqlite3_prepare_v2(), sqlite3_step(), and sqlite3_finalize().

*In the loop you search if the name is found.

*Close the DB (sqlite3_close()).

As reported in the comments, this is not a good idea. If you want to see if something exists, query if it exists:
    sqlite3_stmt *stmt;
    char *zSql = "SELECT EXISTS(SELECT 1 FROM PlayerInfo WHERE name=?)";
    sqlite3_prepare_v2(db, zSql, -1, &stmt, NULL);
    sqlite3_bind_text(stmt, 1, search_name, -1, SQLITE_STATIC);
    sqlite3_step(stmt);
    name_match = sqlite3_column_int(stmt, 0);
    sqlite3_finalize(stmt);

In this specific case, probably the parametrized query is an overkill, but who knows where that search_name comes from...
WARNING: no effort to have decent error handling.


text:
Q: StarCluster VolumeCreator AWS instance_type When attempting to create an EBS shared volume for all nodes in a StarCluster, the StarCluster createvolume command launches an AWS EC2 instance as the volume creator.  However, the default instance type that is used for this is t1.micro, which is no longer available in the AWS availability zone (us-east-1f in my case).  Is it possible to specify a different default instance type to use?  t2.small, for example?

A: I was able to find a work-around.  While t1.micro instances were not available in AWS zone us-east-1f, they are still available in us-east-1d.  So, I created the shared EBS volume there (and, to speed up data transfers to and from this volume, I changed my StarCluster config file to request that cluster nodes be launched in that same zone (us-east-1d)).
Still, if anyone knows an answer to the original question, I would prefer to be able to create EBS volumes and launch a cluster in the availability zone of my choice, and that would require being able to specify an instance type to use for the VolumeCreator node that is launched to create the EBS volume when the "$ starcluster createvolume" command is used.  IMO, this should be a config file setting or, at a minimum, an optional argument to pass to the createvolume command.


text:
Q: It is possible to center an SVG clip-path with respect to its container? I've been trying to set a svg over a iframe video something like img1!, I use the clip-path property css, but the problem is that the svg always is positionated over (0,0) Video's coordinates.
I tried to change the origin in the viewbox in the svg, also, I tried to change the size of the svg and the iframe to do responsive but not worked.

.hidden {
  position: absolute;
  overflow: hidden;
  width: 0;
  height: 0;
  pointer-events: none;
}

.x {
  background: red;
  height: 60vh;
  position: relative;
  width: 100%;
  display: flex;
  justify-content: center;
  svg {
    height: 60vh;
    path {
      height: auto;
      right: 0;
    }
  }
  iframe {
    position: absolute;
    top: 0;
    //left: 50%;
    //right: 50%;
    -webkit-clip-path: url(#clip);
    height: 60vh;
  }
}

  <svg version="1.1" id="Layer_1" class="hidden" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="50px" y="50px" viewBox="50 50 1534 1503" style="enable-background:new 50 50 1534 503;" xml:space="preserve">
        <defs>
            <clipPath id="clip">
                <path class="cls-1" d="M326,506c.33,3.33.83,6.66.94,10,.08,2.15.38,4,2,5.52,7.18,6.83,14.27,13.74,21.5,20.71l26.27-7.54V514.8c-7.95-7.6-17.82-13.73-25.13-23.68,11.3-4,21.23-7.94,31.41-11,11.14-3.37,21.78-7.26,29.72-18.19V445.58l-70.87-65.12c-1.24-9.74-2.51-20.11-3.9-30.46-.52-3.85-1.16-7.7-2.06-11.48a9.1,9.1,0,0,0-2.46-4c-3.5-3.5-7.16-6.83-10.15-9.65-2.51-18.41-1.8-36.34-7.39-53.36-1.94-5.94-4.79-11.58-7.59-18.23-13.73,22.73-13,47.3-15.18,68.36-4.46,5.37-7.31,9.35-10.74,12.75a12.4,12.4,0,0,0-3.35,7.6c-1.79,13-3.37,26-4.94,38.45L203,445.81v17.11c6.59,7.13,14.26,13,24,16.13,10.94,3.51,21.75,7.42,32.45,11.1,1.29,2.65.32,4.21-1.4,5.89-6.5,6.32-12.87,12.8-19,19v19.55l26.68,7.77c3-3,6.12-6.18,9.33-9.31,2.62-2.55,5.07-5.39,8.05-7.43,5-3.39,6.83-7.76,6-13.59a52.6,52.6,0,0,1,0-6.53c5.4,2.8,10.34,5.79,15,0l1,.54c0,3-.16,6,.05,9s-1.47,6.62,3.58,8.47c5-5.13.49-12.17,3.33-18,4.54,5.09,9,3.51,13.54.49" transform="translate(-202 -252.3)"/><path class="cls-1" d="M308,294.5l-4.63,3.09c-1.11,5.56-2.91,11.2-3.17,16.92a55,55,0,0,0,2,15.94c.56,2.15,3.47,3.71,5.36,5.58,3.3-2.34,5.91-3.59,6.67-8,1.78-10.23.73-20.09-1.45-30l-4.29-3.53" transform="translate(-202 -252.3)"/>
            </clipPath>
        </defs>
    </svg>

you can see my actual code!

A: I followed the @Robert Longson's comment... so I use the property clipPathUnits="objectBoundingBox" but in addition I had to use the transform="scale(0.00472344244, 0.00343548165) property to work, the code at the end looks like...
html

  <svg version="1.1" id="Layer_1" class="hidden" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="50px" y="50px" viewBox="50 50 1534 1503" style="enable-background:new 50 50 1534 503;" xml:space="preserve">
        <defs>
            <clipPath id="clip" clipPathUnits="objectBoundingBox" transform="scale(0.00472344244, 0.00343548165)>
                <path class="cls-1" d="M326,506c.33,3.33.83,6.66.94,10,.08,2.15.38,4,2,5.52,7.18,6.83,14.27,13.74,21.5,20.71l26.27-7.54V514.8c-7.95-7.6-17.82-13.73-25.13-23.68,11.3-4,21.23-7.94,31.41-11,11.14-3.37,21.78-7.26,29.72-18.19V445.58l-70.87-65.12c-1.24-9.74-2.51-20.11-3.9-30.46-.52-3.85-1.16-7.7-2.06-11.48a9.1,9.1,0,0,0-2.46-4c-3.5-3.5-7.16-6.83-10.15-9.65-2.51-18.41-1.8-36.34-7.39-53.36-1.94-5.94-4.79-11.58-7.59-18.23-13.73,22.73-13,47.3-15.18,68.36-4.46,5.37-7.31,9.35-10.74,12.75a12.4,12.4,0,0,0-3.35,7.6c-1.79,13-3.37,26-4.94,38.45L203,445.81v17.11c6.59,7.13,14.26,13,24,16.13,10.94,3.51,21.75,7.42,32.45,11.1,1.29,2.65.32,4.21-1.4,5.89-6.5,6.32-12.87,12.8-19,19v19.55l26.68,7.77c3-3,6.12-6.18,9.33-9.31,2.62-2.55,5.07-5.39,8.05-7.43,5-3.39,6.83-7.76,6-13.59a52.6,52.6,0,0,1,0-6.53c5.4,2.8,10.34,5.79,15,0l1,.54c0,3-.16,6,.05,9s-1.47,6.62,3.58,8.47c5-5.13.49-12.17,3.33-18,4.54,5.09,9,3.51,13.54.49" transform="translate(-202 -252.3)"/><path class="cls-1" d="M308,294.5l-4.63,3.09c-1.11,5.56-2.91,11.2-3.17,16.92a55,55,0,0,0,2,15.94c.56,2.15,3.47,3.71,5.36,5.58,3.3-2.34,5.91-3.59,6.67-8,1.78-10.23.73-20.09-1.45-30l-4.29-3.53" transform="translate(-202 -252.3)"/>
            </clipPath>
        </defs>
    </svg>

Css
body {
  margin: 0;
  display: flex;
  justify-content: center;
  align-items: center;
  height: 100vh;
  background: #333;
}
.hidden {
    position: absolute;
    overflow: hidden;
    width: 0;
    height: 0;
    pointer-events: none;
}
.mid-panel{
  background: #1d1d1d;
  height:60vh;
  position: relative;
  width: 100%;
  min-height: 350px;
  iframe {
    position: absolute;
    top: 0;
    left: 0;
    right: 0;
    margin: auto;
    -webkit-clip-path:url(#clip);
    clip-path:url(#clip);
    height: 60vh;
    max-width: 500px;
    min-width: 400px;
    min-height: 350px;
  }
}


text:
Q: Icon-font not visible in firefox I have added with @fontface an iconfont. The only problem is, that it is not visible under firefox. I have the newest version of firefox. Safari and Chrome are working perfectly fine.
@font-face {
    font-family: 'iconFont';
    src: url('../fonts/iconFont.eot');
    src: url('../fonts/iconFont.eot?#iefix') format('embedded-opentype'), url('../fonts/iconFont.svg#iconFont') format('svg'), url('../fonts/iconFont.woff') format('woff'), url('/fonts/iconFont.ttf') format('truetype');
    font-weight: normal;
    font-style: normal;
}

A: @font-face {
    font-family: 'iconFont';
    src: url('../fonts/iconFont.eot');
    src: url('../fonts/iconFont.eot?#iefix') format('embedded-opentype'),
        url('../fonts/iconFont.woff') format('woff'),
        url('../fonts/iconFont.ttf') format('truetype'),
        url('../fonts/iconFont.svg#iconFont') format('svg');
    font-weight: normal;
    font-style: normal;
}

For font-face kit generations you can use:

*

*http://www.fontsquirrel.com/fontface/generator

*http://fontface.codeandmore.com/index.php

A: Try this:
@font-face {
    font-family: 'iconFont';
    src: url('../fonts/iconFont.eot');
    src: url('../fonts/iconFont.eot') format('embedded-opentype'), url('../fonts/iconFont.svg') format('svg'), url('../fonts/iconFont.woff') format('woff'), url('/fonts/iconFont.ttf') format('truetype');
    font-weight: normal;
    font-style: normal;
}

avoid the shorthand font format
@font-face
{
font-family: myFirstFont;
src: url('Sansation_Light.ttf'),
     url('Sansation_Light.eot'); /* IE9 */
}


text:
Q: Multiple .CSS Jquery lines I'm quite new to jquery, but managed to do some effects the designer wanted, take a look at this fiddle:
jsFiddle
So I would like the words to come to the front, change color and fade back to the background and change color again.
The .animate method i'm using now is not really smooth, I searched stackoverflow and found no good solution, i however found the reason why it's not working:  something with rendering and browser compatibility. Details can be found here
I read something about the jquery .css method and started to try that out. Much smoother transition indeed. The only problem is that i can't seem to combine several styles so they will be animated one after another as i could do with the .animate method: as shown in the jfiddle above
var duration = 1400;
$('.dgrey').css({"-webkit-transform":"scale(2)","-webkit-transition-timing-function":"ease-in", "-webkit-transition-duration": duration + "ms", "color":"#E5352D"});
$('.dgrey').css({"-webkit-transform":"scale(0.5)","-webkit-transition-timing-function":"ease-in", "-webkit-transition-duration": duration + "ms", "color":"#DDDDDD"});

Does anyone have a resolution for this?
thanks in advance

A: You can try to use jQuery transit plugin: jQuery Transit

A: There are webkit transition-end events etc, but for what you're doing where you already have the duration, a simple timeout seems easier :
var duration = 1400;

function ani() {
     $('.dgrey').css({'animation here'});
     setTimeout(function() {
         $('.dgrey').css({next animation here});
         setTimeout(ani, duration);
     }, duration);
}

ani();

FIDDLE

A: Iwill do something like that, for smooth transition:
HTML

TEXT TO ANIMATE

CSS
#x{
position:absolute;
top: 100px;
left: 200px;
color: #0000FF;
font-size: 40px;
}

@-webkit-keyframes pulse_animation {
    50% { -webkit-transform: scale(2); }
    100% { -webkit-transform: scale(0.5); }
}

JavaScript
var duration = 2800;
$('#x').css({"-webkit-animation-duration": duration + "ms", "color":"#E5352D", "-webkit-animation-name": 'pulse_animation', "-webkit-animation-timing-function": "ease-in"});

Fiddler View


text:
Q: New form fails to appear Can someone explain why my new form doesn't appear? The old form closes but it isn't replaced by my new one:
namespace Pong
{
    public partial class Menu : Form
    {
        public Menu()
        {
            InitializeComponent();
        }

        private void pictureBox1_Click(object sender, EventArgs e)
        {

        }

        private void PlayButton_Click(object sender, EventArgs e)
        {
            PongForm form = new PongForm();
            form.Show();
            this.Close();
        }

        private void ExitButton_Click(object sender, EventArgs e)
        {
            this.Close();
        }
    }
}

A: Is "Menu" the startup form for your application? If yes, then when it closes the entire application closes.
One solution would be to hide the current form, display "form" with ShowDialog(), then call Close():
    private void PlayButton_Click(object sender, EventArgs e)
    {
        this.Visible = false; // hide the current form
        Application.DoEvents();

        PongForm form = new PongForm();
        form.ShowDialog(); // code stops here until PongForm is dismissed

        this.Close(); // now close the form (and presumable the whole app)
    }


text:
Q: Upgrade project to target .net framework to 4.7.2 without upgrading nuget references I have a solution with more than 50 projects, some are targeting .net framework 4.5 and some 4.
Primary executable project is targeting to 4.5.
I want to upgrade all projects to 4.7.2.
When project is upgraded from 4 to 4.7.2, I get message that some nuget packages are downloaded for .net 4 and should be downloaded again for 4.5 or 4.7.2, which I don't want to do because it triggers so many nuget upgrades and that's getting me into lots of compilation and dependency issues.
Basically, I would like to upgrade target framework of project, but keep nuget referring to older framework 4.(update nuget incrementally in future).
Question: Are there any downsides to this approach? Will nuget referring to mixed .net framework cause any issue?


text:
Q: jquery/ javascript slider realtime based I would like to implement a slider on top of every page of my website.
however this might be disturbing and annoying when a user starts browsing from page to page, since the slideshow will start over and over again.
I though the solution could lay in the builtin date function:
    var d = new Date();
    var n = d.getSeconds();
    if(n%10 == 0)
    {
    rotate
    }

Or perhaps I could set a cookie with the values from the slider when a user leaves that page (name=topSlider value=1 time=300), so the new page could pick it up and go further in  it's sequence.


text:
Q: SQL Server field calculation based on multiple condition Here is my scenario:
I have a Person table with following fields.
create table Person(PersonID int primary key identity(1,1),
                     Age int,
                     height decimal(4,2),
                     weight decimal(6,2)
                    );

 insert into Person(Age,height,weight) values (60,6.2,169); -- 1
 insert into Person(Age,height,weight) values (15,5.1,100); -- 2
 insert into Person(Age,height,weight) values (10,4.5,50); -- 3

What I need to do is,
if the person Age >= 18 and height >= 6 then calculationValue = 20
if the person Age >= 18 and height < 6 then calculationValue = 15
if the person Age < 18 and weight >= 60 then calculationValue = 10
if the person Age < 18 and weight < 60 then calculationValue = 5

based on these condition I need to find the calculationValue and do some math.
I tried to make a flexible model so in future it would be easier to add any more conditions and can easily change the constant values (like 18, 6, 60 etc)
I created couple of tables as below:
create table condTable(condTableID int primary key identity(1,1),
                        condCol varchar(20),
                        startValue int,
                        endValue int
                      );

  insert into condTable(condCol,startValue,endValue) values ('Age',18,999) -- 1
  insert into condTable(condCol,startValue,endValue) values ('Height',6,99) -- 2
  insert into condTable(condCol,startValue,endValue) values ('Height',0,5.99) -- 3
  insert into condTable(condCol,startValue,endValue) values ('Age',0,17) -- 4
  insert into condTable(condCol,startValue,endValue) values ('Weight',60,999) -- 5
  insert into condTable(condCol,startValue,endValue) values ('Weight',0,59) -- 6

I join two condition to make it one in the following table as given by the requirement.(ie. if age >=18 and height >=6 then calculationValue = 20. etc)
create table CondJoin(CondJoin int,condTableID int,CalculationValue int)

insert into CondJoin values (1,1,20)
insert into CondJoin values (1,2,20)
insert into CondJoin values (2,1,15)
insert into CondJoin values (2,3,15)
insert into CondJoin values (3,4,10)
insert into CondJoin values (3,5,10)
insert into CondJoin values (4,4,5)
insert into CondJoin values (4,6,5)

I think this model will provide the flexibility of adding more conditions in future. But I am having difficulties on implementing it in SQL Server 2005. Anyone can write a sql that process in set basis and compare the value in Person table with CondJoin table and provide the corresponding calculationvalue. For eg. for person ID 1 it should look at CondJoin table and give the calculationValue 20 since his age is greater than 18 and height is greater than 6.

A: this looks like you are headed towards dynamic sql generation.
i think maybe you would be better off with a row for each column and cutoff values for the ranges, and a value if true ... maybe something like:
age_condition
-----------------
min_age
max_age
value

this is something that you could populate and then query without some dynamic generation.

A: The following is extremely rough but it should get the point across. It normalizes the data and moves towards a semi-object oriented (attribute/value/attribute value) structure. I'll leave it up to you to reinforce referential integrity, but the following is flexible and will return the results you want:
CREATE TABLE Person (
    PersonID INT PRIMARY KEY IDENTITY(1,1)
    ,Name NVARCHAR(255)
    );

GO

CREATE TABLE PersonAttribute (
    PersonID INT
    ,CondAttributeID INT
    ,Value NVARCHAR(255)
    );

GO

CREATE TABLE CondAttribute (
AttributeID INT PRIMARY KEY IDENTITY(1,1)
,Attribute NVARCHAR(255));

GO

CREATE TABLE CondTable (
    CondTableID INT PRIMARY KEY IDENTITY(1,1)
    ,CondAttributeID INT
    ,StartValue MONEY
    ,EndValue MONEY
    );

GO

CREATE TABLE CalculationValues (
    CalculationID INT PRIMARY KEY IDENTITY(1,1)
    ,CalculationValue INT
    );

GO

CREATE TABLE CondCalculation (
    CondTableID INT
    ,CalculationID INT
    );

INSERT Person (Name)

VALUES ('Joe')
        ,('Bob')
        ,('Tom');

INSERT PersonAttribute (
    PersonID
    ,CondAttributeID
    ,Value
)

VALUES  (1, 1, '60')
        ,(1, 2, '6.2')
        ,(1, 3, '169')
        ,(2, 1, '15')
        ,(2, 2, '5.1')
        ,(2, 3, '100')
        ,(3, 1, '10')
        ,(3, 2, '4.5')
        ,(3, 3, '50');

INSERT CondAttribute (Attribute)

VALUES ('Age')
        ,('height')
        ,('weight');

INSERT CondTable (
    CondAttributeID
    ,StartValue
    ,EndValue)

VALUES (1,18,999) --Age
        ,(2,6,99) --Height
        ,(2,0,5.99) -- Height
        ,(1,0,17) -- Age
        ,(3,60,999) -- Weight
        ,(3,0,59); -- Weight

INSERT CalculationValues (CalculationValue)
VALUES (5)
        ,(10)
        ,(15)
        ,(20);

INSERT CondCalculation (CondTableID, CalculationID)
VALUES (1,4)
       ,(2,4)
       ,(1,3)
       ,(3,3)
       ,(4,2)
       ,(5,2)
       ,(5,1)
       ,(6,1);

SELECT *
FROM Person AS p
JOIN PersonAttribute AS pa ON p.PersonID = pa.PersonID
JOIN CondAttribute AS ca ON pa.CondAttributeID = ca.AttributeID
JOIN CondTable AS ct ON ca.AttributeID = ct.CondAttributeID
    AND CONVERT(money,pa.Value) BETWEEN ct.StartValue AND ct.EndValue
JOIN CondCalculation AS cc ON cc.CondTableID = ct.CondTableID
JOIN CalculationValues AS c ON cc.CalculationID = c.CalculationID
WHERE p.PersonID = 1

A: The following solution uses PIVOT (twice) to transform the combination of CondJoin and condTable into a chart, then joins the chart to the Person table to calculate the target value. I believe, a series of CASE expressions could be used instead just as well. Anyway...
All the tables have been turned into table variables, for easier testing. So first, DDL and data preparation:
declare @Person table(PersonID int primary key identity(1,1),
                     Age int,
                     height decimal(4,2),
                     weight decimal(6,2)
                    );
insert into @Person(Age,height,weight) values (60,6.2,169); -- 1
insert into @Person(Age,height,weight) values (15,5.1,100); -- 2
insert into @Person(Age,height,weight) values (10,4.5,50); -- 3

declare @condTable table(condTableID int primary key identity(1,1),
                        condCol varchar(20),
                        startValue int,
                        endValue int
                      );
insert into @condTable(condCol,startValue,endValue) values ('Age',18,999) -- 1
insert into @condTable(condCol,startValue,endValue) values ('Height',6,99) -- 2
insert into @condTable(condCol,startValue,endValue) values ('Height',0,5.99) -- 3
insert into @condTable(condCol,startValue,endValue) values ('Age',0,17) -- 4
insert into @condTable(condCol,startValue,endValue) values ('Weight',60,999) -- 5
insert into @condTable(condCol,startValue,endValue) values ('Weight',0,59) -- 6
declare @CondJoin table(CondJoin int,condTableID int,CalculationValue int);
insert into @CondJoin values (1,1,20)
insert into @CondJoin values (1,2,20)
insert into @CondJoin values (2,1,15)
insert into @CondJoin values (2,3,15)
insert into @CondJoin values (3,4,10)
insert into @CondJoin values (3,5,10)
insert into @CondJoin values (4,4,5)
insert into @CondJoin values (4,6,5)

And now the query:
;with startValues as (
  select
    CondJoin,
    Age,
    Height,
    Weight,
    CalculationValue
  from (
    select
      j.CondJoin,
      j.CalculationValue,
      t.condCol,
      t.startValue
    from @CondJoin j
      inner join @condTable t on j.condTableID = t.condTableID
  ) j
  pivot (
    max(startValue) for condCol in (Age, Height, Weight)
  ) p
),
endValues as (
  select
    CondJoin,
    Age,
    Height,
    Weight,
    CalculationValue
  from (
    select
      j.CondJoin,
      j.CalculationValue,
      t.condCol,
      t.endValue
    from @CondJoin j
      inner join @condTable t on j.condTableID = t.condTableID
  ) j
  pivot (
    max(endValue) for condCol in (Age, Height, Weight)
  ) p
),
combinedChart as (
select
  s.CondJoin,
  AgeFrom    = s.Age,
  AgeTo      = e.Age,
  HeightFrom = s.Height,
  HeightTo   = e.Height,
  WeightFrom = s.Weight,
  WeightTo   = e.Weight,
  s.CalculationValue
from startValues s
  inner join endValues e on s.CondJoin = e.CondJoin
)
select
  p.*,
  c.CalculationValue
from @Person p
  left join combinedChart c
    on (c.AgeFrom    is null or p.Age    between c.AgeFrom    and c.AgeTo)
   and (c.HeightFrom is null or p.Height between c.HeightFrom and c.HeightTo)
   and (c.WeightFrom is null or p.Weight between c.WeightFrom and c.WeightTo)


text:
Q: How to programatically get table structure with pyscopg2 when I execute the program application.py ,I get this error :
psycopg2.ProgrammingError
ProgrammingError: ERREUR:  erreur de syntaxe sur ou près de « "/d" »
LINE 1: "/d" carto."BADGES_SFR"

(in english "ProgrammingError: ERREUR:  syntax error at or near  « "/d" » ") My objective for this program is to get the table structure
this is the following code application.py :
#!/usr/bin/python 2.7.6
# -*- coding:utf-8 -*-
import os
import sys
from flask import Flask,render_template
import psycopg2
reload(sys)
sys.setdefaultencoding('utf8')
app = Flask(__name__)

@app.route('/')
def fihoum():
    conn = psycopg2.connect(database="carto", user="postgres", password="daed5Aemo", host="192.168.12.54")
    cur = conn.cursor()
    #cur.execute("SELECT * FROM carto.\"BADGES_SFR\"")
    cur.execute("/d carto.\"BADGES_SFR\"")
    rows = cur.fetchall()
    return render_template('hello.html', titre="Données du client BADGES_SFR !",mots=rows)

if __name__=="__main__":
    app.run(host=os.getenv('IP', '0.0.0.0'),
            port=int(os.getenv('PORT',5000)),
            debug=True)

A: First of all it's \d <table_name> (note the backslash \d not /d), but that's only available in the psql interactive terminal

My [objective] with this program is to get the table structure

You can use the SQL table information_schema.columns which carries the information you want
SELECT column_name, data_type, is_nullable
FROM information_schema.columns
WHERE table_name = '<table_name>';

    column_name     |     data_type     | is_nullable
--------------------+-------------------+-------------
 column_a           | integer           | YES
 column_b           | boolean           | NO

Here's a list of the columns available for use: https://www.postgresql.org/docs/9.4/static/infoschema-columns.html
Snippet
import psycopg2
conn = psycopg2.connect(database='carto', user=..., password=...)

q = """
SELECT column_name, data_type, is_nullable
FROM information_schema.columns
WHERE table_name = %s;
"""

cur = conn.cursor()
cur.execute(q, ('BADGES_SFR',))  # (table_name,) passed as tuple
cur.fetchall()

# Example Output
[('column_a', 'integer', 'YES'),
 ('column_b', 'boolean', 'NO'),
 ...,
]

Tip for Jinja2/Flask integration:
Consider using psycopg2.extras.DictCursor, it'll make it easier for you to pull the information based on the column name (dict key) in your Flask template, because your'll a have a dict that you can access e.g. using row['data_type'], row['column_name'] etc.

A: If you only need the columns description:
>>> cur.execute('select * from t')
>>> description = cur.description
>>> print description
(Column(name='record_timestamptz', type_code=1184, display_size=None, internal_size=8, precision=None, scale=None, null_ok=None),)
>>> columns = description[0]
>>> print columns.name, columns.type_code
record_timestamptz 1184

For the data type name cache the pg_type relation at the application start and make it a dictionary:
cursor = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)

cursor.execute ('select oid, * from pg_type')

pg_type = dict([(t['oid'], t['typname']) for t in cursor.fetchall()])

cursor.execute ('select * from t')

for t in cursor.description:
    print t.name, t.type_code, pg_type[t.type_code]


text:
Q: Android: how can I add different layer to a image view dynamically? I have one fragment layout with a background image. Programmatically I have to add the different layer to this image because I want to add some particular details depending of some data. There are three different details that it can change in 3 different way.  ( there is a thread that works at 5/10hz and it loads the right details)
The following is my first implementation. In this version, I have done all of works in XML layout and in a programmatically way, when the user clicks on a certain button, I set the visibility of each image view. I have seen that is much expensive, and sometimes the inflate layout return null:
   <RelativeLayout
       android:id="@+id/container"
       android:layout_width="match_parent"
       android:layout_height="match_parent"
       android:background="@drawable/background"
     >

     <ImageView
        android:id="@+id/layer1"
         android:layout_width="match_parent"
         android:layout_height="match_parent"
         android:visibility="gone"
        android:src="@drawable/ballgreen"/>
<ImageView
        android:id="@+id/layer2"
         android:layout_width="match_parent"
         android:layout_height="match_parent"
         android:visibility="gone"
         android:src="@drawable/ballred"/>
<ImageView
        android:id="@+id/layer3"
         android:layout_width="match_parent"
         android:layout_height="match_parent"
         android:visibility="gone"
          android:src="@drawable/ballyellow"/>

            ... 6 more image view layer....

</RelativeLayout>

Then I thought to use an async task to decode all drawable image, make a bitmap fusion and in a post execute load the result bitmap in a single image view.
But the asynctask is not the better way to implement, because the user can click on that button many times and each time I have to call new asynctask ..
Are there a clever ways to implement all of this?
Goodbye and thanks a lot :)
EDIT : each layer is a detail that thread can add to that image.
For example : a car, a tree, a red ball ecc ecc. In the xml example for me each imageview is a layer, and this is wrong. But I don't know the clever way to implement ..
EDIT2 : Moreover I know the maximum amount of layer, but there are many combinations. there are 3 point and for each point I can choice between 3 differents details
Edit3:  i add some details , there is a thread that change laYer and not the user

A: Use the LayerDrawable class, adding new layers as the user enables them. I don't think you can remove layers from it, but you can create a new LayerDrawable each time the user disables a layer. Or you can use setDrawable(int, Drawable) to replace a layer.
This means you don't have to complicate your layouts, and should get rid of those pesky extra inflation/measure/layout passes.
As for the image loading / async task stuff: use an LRU cache for your bitmaps. When a layer is enabled, check the cache, and if it's there, just use it. Otherwise, use an async task to load it.


text:
Q: Access data in multiple tabs - datatable I'm using jquery datatable . Table is inside a form.When i submit the form i can only access to the data on the tab which is currently opened . data in other tabs are not submitted.
 this is how i initialize the data table
<script defer>
   $(document).ready(function(){
        $('#tbl_location').dataTable({
            "bSort": false
        });
    });
</script>

I want to access the data in all tabs. I tried to find this issue in their forum but i didn't see anything related to this topic.I would be glad if anyone can give me an idea.

A: try:
('#form').submit( function() {
          var table = $.fn.dataTable.fnTables(false); //get all datatables
          var list = Array()
           $.each(table, function(key, value) {
              list.push($(value).dataTable().fnGetData())
          });
            $('<input />').attr('type', 'hidden')
            .attr('name', "tablesdata")
            .attr('value',  list)
            .appendTo('#form');
            return true;
        } );
    });

Hope it'll be helpful


text:
Q: Best way to copy new daily data from historical table in Databricks Can you please review my method and advise me the best way to copy new daily data from historical fact table to another one.
I have 1 big parquet table let's call it "Table1" that somehow refreshes daily with some transactions with date.
I need to copy new data from "Table1" and keep it in my separated table let's call it "Table2" every day.
I did like this:

*

*Read the historical table "Table1" and saved it in my place in my data lake as "Table2".

*Next day I created a new notebook with code where I read "Table2", find MAX date and declare it as a variable "last_date".

*I read "Table1" where is already new data and filter it by date > "last date" variable.

*Save the filtered data to "Table2" with "append" mode and schedule this job daily.

Is that correct way in data engineering practice? Or there is a better and stable solution?


text:
Q: SELECT IF query I have simple query:
SELECT max(`id`) as `name_id` , LEFT(`name`,LENGTH(`name`)-2) AS `name`, `status`
 FROM `users`
WHERE `site` = 1
  AND `category` = 'some'

And I need select only those, where supp == YES
SELECT `model`.`supp`
  FROM `model`, `users`
 WHERE `users`.`name_id` = `name_id_from_the_first_query`
   AND `model`.`model_id` = `users`.`model_id`;

So I need return values of the first query only if second returns YES for the current id.

A: Try this query:
SELECT max(`id`) as `name_id` , LEFT(`name`, LENGTH(`name`)-2) AS `name`
       , `status`
 FROM `users`
     JOIN `model`
       ON `model`.`model_id` = `users`.`model_id`
    WHERE `site` = 1 AND `category` = 'some'
      AND `model`.`supp` = 'YES';

A: Try,
SELECT  max(a.`id`) as `name_id` ,
        LEFT(`name`,LENGTH(a.`name`)-2) AS `name`,
        a.`status`
FROM    `model` a
        INNER JOIN `users` b
            ON a.`model_id` = b.`model_id`
WHERE    a.`site` = 1 AND
         a.`category` = 'some' AND
         b.`supp` = 'YES'
GROUP BY a.`name`, a.`status`

A: Here you have another option:
SELECT max(`id`) as `name_id` , LEFT(`name`,LENGTH(`name`)-2) AS `name`, `status`
 FROM `users`
WHERE `site` = 1
  AND `category` = 'some'
  AND id IN (SELECT `model_id`
               FROM `model`
              WHERE `supp` = 'YES'
             )
GROUP BY `name`, `status`;

A: SELECT max(`id`) as `name_id` , LEFT(`name`,
LENGTH(`name`)-2) AS `name`, `status`
FROM `users` WHERE `site` = 1 AND `category` = 'some'
     and EXISTS
     (
       SELECT `model`.`supp` FROM `model`
          WHERE `model`.`model_id` = `users`.`model_id`
                 and `model`.`supp`='Yes'
      )


text:
Q: Error : Unable to open module file System Error &H80070002& I am working on a web based dynamic vb form which is of around 20,000 lines suddenly my system got shutdown due to power loss, When I reopened the system then I found my .vb file is only left with Hexadecimal-Binary kind of text as in the screen shot.
When I am running the code in Visual Studio it is running fine and show the form in web browser but I need to edit it as I am in development phase.
On opening the file in notepad it is showing blank page with lots of blank lines.
I have tried to recover using 'restoring previous versions' but I found 'There are no previous versions available'.
I also have tried to recover using 'Open system restore' in which I found a list of restore point but even after successful completion issue is remain same.
Error screen shot

A: I didn't any solution for this problem.
I have some my code in distributed manner so I merge those pieces of code and wrote rest of the code.
If any body knows better solution than this, please share it.


text:
Q: MVC5 Google+ Authentication - Force Google Signin after logout I have just wired up MVC5 with google+ authentication exactly as per this excelent guide:
http://www.oauthforaspnet.com/providers/google-plus/
My question is this. Once my Google Account has been authenticated and I am logged in to my application, I cant seem to force the user to be able to logout totally.
Clicking logout drops me back to my external login screen with the Google+ button, but clicking the button takes me straight in to my application again.
Is it possible to clear the authenticated user completely once logout is pressed?
I tried adding this prior to the login procedure, but to no avail:
AuthenticationManager.SignOut();

Any help or advice greatly appreciated.

A:
Is it possible to clear the authenticated user completely once logout is pressed?

No, that's not possible. The user is still authenticated in his Google account, having all his cookies present. You have absolutely no control over that. It is the responsibility of the user to decide whether he wants to sign out from his Google account, not yours. Since your application integrates a Google OAuth authentication you will have to live with that. The AuthenticationManager.SignOut(); simply clears the cookies out from your application domain.


text:
Q: JMock - several invocations with different arguments The method I want to test is calling a mock method with different arguments:
public void methodToTest(){
   getMock().doSomething(1);
   getMock().doSomething(2);
   getMock().doSomething(3);
}

In my unit test I want to know, if methodToTest really is calling those methods with exactly those arguments. This is the code I wrote:
@Test
public void myMockTest(){
    oneOf(mock).doSomething(1);
    oneOf(mock).doSomething(2);
    oneOf(mock).doSomething(3);
}

At (2) I get an "Unexpected invocation" - as if it couldn't distinguish different arguments. So I've tried that one:
exactly(3).of(mock).doSomething(with(Matchers.anyOf(same(1), same(2), same(3))));

But this also didn't do what I've expected.
Finally, this one worked:
exactly(3).of(mock).doSomething(with(any(Integer.class)));

So I know, that my method was called 3 times with any Integer number. Is there any way to make sure, it's exactly the argument(s) I have passed?

A: Did you surround the expectations with a checking block?
context.checking(new Expectations() {{
  oneOf(mock).doSomething(1);
  oneOf(mock).doSomething(2);
  oneOf(mock).doSomething(3);
}});

Also, are you aware the jmock does not enforce sequence unless you do so explicitly?


text:
Q: Compilation error in C on Linux I'm  writing a simple program in Linux and when compiling I keep getting this error and don't understand why — I put all ; or {} in place:
ex1.c:49.1: error:expected declaration or statement at end of input
My code:
#include<sys/stat.h>
#include<sys/types.h>
#include<unistd.h>
#include<stdio.h>
#include<fcntl.h>

int main (int argc, char* argv[])
{
        if (argc!=3)
                printf("Number of arugements passed is wrong - not 2\n");

        char * file1 = argv[1];
        char * file2 = argv[2];

        int fdout1 = open(file1, O_RDONLY);
        int fdout2 = open(file2, O_RDONLY);

        struct stat stat_p1,stat_p2;

        char * f1;
        char * f2;
        int i=0;

        if (fdout1 < 0 || fdout2<0)     /* means file open did not take place */
        {
                perror("ERROR opening files");   /* text explaining why */
                return 1;
        }

        if ( (stat (file1, &stat_p1) == -1) ||( stat (file2, &stat_p2) == -1) ) /* declare the 'stat' structure */
        {
                printf(" Error occurred attempting to stat files");
                return 1;
        {

        if((read(fdout1,f1,stat_p1.st_size)==-1) ||(read(fdout1,f1,stat_p1.st_size)==-1))
        {
                printf("ERROR reading files");   /* text explaining why */
                return 1;
        }
        if (stat_p1.st_size!=stat_p2.st_size)
                return 1;

        if (strcmp(f1,f2)!=0)
                return 1;

        return 0;
}

Can anyone tell me why I keep getting this error?

A: It seems there is a typo
    if ( (stat (file1, &stat_p1) == -1) ||( stat (file2, &stat_p2) == -1) ) /* declare the 'stat' structure */
    {
            printf(" Error occurred attempting to stat files");
            return 1;
    {

The last brace shall be a closing brace.
    if ( (stat (file1, &stat_p1) == -1) ||( stat (file2, &stat_p2) == -1) ) /* declare the 'stat' structure */
    {
            printf(" Error occurred attempting to stat files");
            return 1;
    }

Take into account that if argc is less than 3 you have to exit the program
if (argc!=3)
        printf("Number of arugements passed is wrong - not 2\n");

Otherwise these statements are invalid
char * file1 = argv[1];
char * file2 = argv[2];

A:             if ( (stat (file1, &stat_p1) == -1) ||( stat (file2, &stat_p2) == -1) ) /* declare the 'stat' structure */
            {
                    printf(" Error occurred attempting to stat files");
                    return 1;
           >>>> { >>>>> change this to '}'

A: This error occurs when you have missing closing braces. As can be seen here:
        if ( (stat (file1, &stat_p1) == -1) ||( stat (file2, &stat_p2) == -1) ) /* declare the 'stat' structure */
        {
                printf(" Error occurred attempting to stat files");
                return 1;
        {

You seem to have a typo, you should have closing brace instead of opening brace.

A: It should be like this
if ( (stat (file1, &stat_p1) == -1) ||( stat (file2, &stat_p2) == -1) ) /* declare the 'stat' structure */
            {
                    printf(" Error occurred attempting to stat files");
                    return 1;
            }

The closing brace is the problem, here it was opening brace on your program.


text:
Q: .range.Formula of a Concatenate() to a server path I'm trying to fill in a cell with the =CONCATENATE() formula which gave me errors,
strformula = "=CONCATENATE("\\SERVER\PATH\DIR\";$A5;".pdf")"

I tried to fix this with the incode concatenate & method
but I somehow have an error with the Sub

"Method or data member not found"

Any ideas?
Sub Fillcells()
   Dim pthstr As String
   Dim strformula As String
      With sheetNAME
        pthstr = "\\SERVER\PATH\DIR"
        strformula = "=CONCATENATE(" & pthstr & ";$A5;\" \ .pdf \ ")"

        .Range("N5:N5").Formula = strformula
        '.Range("N5", "N" & GetLastRow(sheetNAME)).FillDown
    End With
End Sub

A: You have a couple issues with how you're handling your string text. I think this will work as a formula for you or at least allow you to avoid errors.
strformula = "=CONCATENATE(""" & pthstr & """,$A5,""\ .pdf \ "")"
Note that for quotes within the formula, you have to use 2x double quotes. And if this is the end of your quote, you;ll have THREE sets, followed by an & sign.
However, I'm not sure concatenate is really what you want. You might just consider typing ="\\SERVER\PATH\DIR\"&A5&".PDF"
There's almost no reason one should ever use Concatenate when compared to alternative functions such as textjoin or Concat. This article explains. Good luck.


text:
Q: Chinese remainder theorem when modulus are not distinct, is my solution right? also - is this a good way to solve such kind of questions? i learnt about the Chinese remainder theorem, and im trying to solve the following question:
find the minimal solution x for
(1)x = 11 mod 24
(2)x = 5 mod 18
(3)x = 5 mod 30
i know that in order to use the Chinese reminder theorem i need the modulos to be distinct
how can i solve it regardless?
i tried the following(and i think this is good):
24 = 8*3
18 = 2*9
30 = 2 * 3 * 5
so(1) is the same as:
(1*)x = 11 = 3 mod 8
(2*)x = 11 = 2 mod 3
(2) is the same as:
(3*)x = 5 = 1 mod 2
(4*)x = 5 mod 9
(3) is the same as:
(5*)x = 5 = 1 mod 2
(6*)x = 5 = 2 mod 3
(7*)x = 5 = 0 mod 5
from 1* we get x = 8k+3 so it also means x = 1 mod 2 so if x satisfies 2* it also satisfies 3*,5*)
from 4* we get x = 9k + 5 and this implies x = 2 mod 3, so if x satisfies 4* it also satisfies 2*,6*
so it is enough to solve :
x = 3 mod 8
x = 5 mod 9
x = 0 mod 5
and from here i can use the Chinese remainder theorem
this way of solving is fine right? is there a better way? is this a good way to approach solving such equations when i cant use the chinese remainder theorem because the modulus are not distinct?
thanks!!


text:
Q: How to find an element in an array of a document in mongodb? I have a code like the following.
router.get('/Child_Profile/:SchoolId/:childId',function(req,res,next){
childModel.find({"schoolid":req.params.SchoolId,"students[]":req.params.ChildId}, function (err, result) {
        if (err)
            {
            return console.log(err);
        }
    res.json(result);
        });
  });

Ex Collection
{
"schoolid":"1wer",
"students":["121","232"],
"profilepic":"http://wed"
}

While i am finding the student info using schoolid and studentid the above schema was finding with only schoolid, not with the studentid. Thanks in advance

A: You have reffered students as an array - students[], use just students and pass on the value of the studentid, the query will return you results.
Mongo shell query for the collection

{
        "_id" : ObjectId("59e9cfd4f4896dbabfc15f45"),
        "schoolid" : "1wer",
        "students" : [
                "121",
                "232"
        ],
        "profilepic" : "http://wed"
}

db.collection.find({students:"121"});

Modified query
router.get('/Child_Profile/:SchoolId/:childId',function(req,res,next){
childModel.find({"schoolid":req.params.SchoolId,"students":req.params.ChildId}, function (err, result) {
        if (err)
            {
            return console.log(err);
        }
    res.json(result);
        });
  });


text:
Q: Earth Engine: projects exceeding storage quotas within the new "Cloud Asset" framework The new "cloud projects" feature seems promising in terms of connecting large GEE projects with Google cloud services. However, the allotted usage quotas are roughly the same as in the older "legacy projects" feature.
Is there a user friendly way to  increase the storage quota for a project through the new "cloud projects" feature or to connect it to a google cloud bucket?


text:
Q: Combine offsetLimitPagination with filtering Apollo Graphql I'm implementing a pretty advanced table (using React-Table) for a large, complex set of data. I started by following Apollo's guide implementing offset-based pagination, I got sorting to work as well. What I'm stuck at is combining that with server-side filtering.
My definition of InMemoryCache looks like this - I'm querying a field Targets:
const cache = new InMemoryCache({
    typePolicies: {
      Query: {
        fields: {
          Targets: {
            ...offsetLimitPagination(),
            read(existing, { args }): any {
              if (args && args.limit !== undefined && args.offset !== undefined) {
                return existing && existing.slice(args.offset, args.offset + args.limit);
              }
            },
          },
        },
      },
    },
  });

which is pretty much the outcome for what the guide told me to do for Pagination. My component queries backend:
const { networkStatus, error, data, fetchMore, refetch } = useQuery<GetTargetsQuery, GetTargetsQueryVariables>(GET_TARGETS, {
    ...
    notifyOnNetworkStatusChange: true,
    variables: {
      limit: tableState.pageSize,
      offset: tableState.pageNumber * tableState.pageSize,
      orderBy: tableState.orderBy,
      where: {
        _and: [initialFilters, queryFilters],
      },
    },
  });

The issue is, when I'm modifying the queryFilters and data gets refetched, I'm seeing correct data in my Network Tab in browser, but my component still reads the old data from the cache. It seems like the offsetLimitPagination hook is not exactly crafted for incorporating filtering(?).
I can't use the React-Table's built-in filtering, as it only operates on the data that has been queried (which in my case is part of the entire set). How do I modify my InMemoryCache to overwrite the data in cache if there are new filters set? Or is there a better way to tackle this or better question to ask to get this done?

A: To clarify the keyArgs, you'd want to specify the keys that would connect one cache from another.
In your case, you have the variables limit, offset, orderBy & where.
So in the case of changing the limit & offset, you do not want Apollo to create a separate cache when limit & offset changes. So you leave that out of the keyArgs.
The keyArgs that you want to watch is your changes with orderBy & where. From my understanding, the new keyArgs are what you want to base your cache from. So if anything in the orderBy & where changes, you'd want Apollo to treat it as somewhat of a separate dataset.
Also, offsetLimitPagination accepts a keyArgs prop.
So in terms of nested variables like the variable where, you can configure it in the keyArgs as a nested value so Apollo has an idea of what the inside of those values are.
The nested array syntax applies to the previous argument value (where).
const cache = new InMemoryCache({
    typePolicies: {
      Query: {
        fields: {
          Targets: {
            ...offsetLimitPagination(["orderBy", "where", ["_and"]]),
            read(existing, { args }): any {
              if (args && args.limit !== undefined && args.offset !== undefined) {
                return existing && existing.slice(args.offset, args.offset + args.limit);
              }
            },
          },
        },
      },
    },
  });


text:
Q: How can I completely change the underlying disks on each Hadoop DataNode? I have N Hadoop DataNodes. Each has attached two 3TB volumes. Each volume is formatted with XFS. Each volume is configured as a data directory for the DataNode. Example:
$ df -Th
/dev/xvdb1     xfs       3.0T  2.0T  1.1T  65% /mnt/vol_a
/dev/xvdc1     xfs       3.0T  1.3T  1.8T  41% /mnt/vol_b

$ cat hdfs-site.xml
...
<property>
  <name>dfs.data.dir</name>
  <value>/mnt/vol_a/hdfs/datanode,/mnt/vol_b/hdfs/datanode</value>
</property>
...

Goal: I want to replace the two 3 TB volumes completely with a new logical volume comprised of different disks, which I can then expand in the future.
Problem: For each node, assuming the new volume is attached, how do I fully migrate the old volumes to the new volume without breaking Hadoop (i.e. HDFS, DataNodes, NameNode, etc.)?
I'm aware of Hadoop's Hot Swap capabilities, but the documentation is a bit lacking and it doesn't explicitly state that it moves blocks from the old volumes to the new volumes.
I do not need to keep the cluster running. If I can stop everything and somehow copy all blocks, that's great.

A: *

*If all the files in your cluster have 2+ replicas. It is safe to simply shutdown one DN at a time and replace the disks with new ones. HDFS will take care re-replication process. You need to wait the file blocks being fully replicated though after the DN restarts (i.e., by running hdfs fsck).

*Or you can run DataNode decommission process, to wait DataNode to move all blocks out of this machine. And after you restart this DataNode with new disks. You need to run Balancer to move some data back.
Up to today (Sep 2016), there is no official way to move all blocks from one disk to another disk. However, it is perfectly safe to just run mv /mnt/vol_a/hdfs/datanode  /mnt/large_vol_c/hdfs/datanode by yourself when the DataNode (or the cluster) is shutdown. And then configure dfs.data.dir to the new location.


text:
Q: Error: QQmlApplicationEngine failed to load component "Network problem" I'm new to qml and python This is the code and error I get:
Error:
QQmlApplicationEngine failed to load component
c:%5CUsers%5Cpcmcb%5COneDrive%5CDesktop%5Capp%5Capp%5Cqml/main.qml: Network error

main.py:
mport sys
import os

from PySide6.QtGui import QGuiApplication
from PySide6.QtQml import QQmlApplicationEngine

if __name__ == "__main__":
app = QGuiApplication(sys.argv)
engine = QQmlApplicationEngine()
engine.load(os.path.join(os.path.dirname(__file__), "qml/main.qml"))

if not engine.rootObjects():
    sys.exit(-1)
sys.exit(app.exec_())

main.qml:
import QtQuick 2.15
import QtQuick.Window 2.15
import QtQuick.Controls 2.15
import QtQuick.Controls.Material 2.15

ApplicationWindow{
    id: window
    width: 400
    height: 580
    visible: true
    title: qsTr("Login Page")
}

When I try to run the code it gives me the error that I typed above

A: load() method requires a QUrl so passing it a string can cause problems, change to:
import os
from pathlib import Path
import sys

from PySide6.QtCore import QCoreApplication, Qt, QUrl
from PySide6.QtGui import QGuiApplication
from PySide6.QtQml import QQmlApplicationEngine

CURRENT_DIRECTORY = Path(__file__).resolve().parent

def main():
    app = QGuiApplication(sys.argv)

    engine = QQmlApplicationEngine()

    filename = os.fspath(CURRENT_DIRECTORY / "qml" / "main.qml")
    url = QUrl.fromLocalFile(filename)

    def handle_object_created(obj, obj_url):
        if obj is None and url == obj_url:
            QCoreApplication.exit(-1)

    engine.objectCreated.connect(handle_object_created, Qt.QueuedConnection)
    engine.load(url)

    sys.exit(app.exec())

if __name__ == "__main__":
    main()


text:
Q: how to display image uploaded in a submitted form with ploneformgen? I'm using ploneformgen.
[edited] I'm using a custom script adapter from ploneformgen to create a page using the information from submitted forms. The problem is: the page generated when i submit, doesn't show the image. I would like to create a page/document with text AND one image. How can I do this?
I've tried this code:
#Creating the folder in zope instance. I gave the name 'conteudo' to it.
targetdir = context.conteudo

#Here asking for the form,  so we can access its contents
form = request.form

#Creating an unique ID
from DateTime import DateTime
uid= str(DateTime().millis())

# Creating a new Page (Document)
targetdir.invokeFactory('Document',id=uid, title= 'Minha Página', image= form['foto-de-perfil_file'],text='Nome Completo:
'+form
['nome-completo']+'

'+'Formação:
'+form
['formacao']+'

'+'Áreas de Atuação:
'+form['areas-de-    atuacao']+'

'+'Links:
'+form['links'])

# Set the reference for our new Page
doc = targetdir.get('uid',targetdir)

# Reindexed the Page in the site
doc.reindexObject()

It only shows the text, but the picture isn't there.
I tried using "setImage", but it shows an attribute error. I tried making another invokefactory but instead of 'Document', I put 'Image' then it only shows the image.
What should I modify on my code to display both the text and the picture?
Thanks in advance.

A: On a standard Plone Document content type does not own any image field.
If you are on Plone 4 using old ATContentTypes you must install and configure collective.contentleadimage.
On Plone 5 or if you are using Dexterity based contents you must add the new image field to your schema.

A: The only thing that worked well for me was instead of creating 'pages', I switched it for 'news items'.


text:
Q: Box2D distanceJoint not maintaining shape in AndEngine I am using Box2D distanceJoint with AndEngine to maintain a shape. But the shape is not maintained the pieces just fall to the floor. It is supposed to maintain a circular/elliptical shape. Can you let me know what I am doing wrong? Here I am maintaining the distance with the neighbor. Should I maintain the distance with every other member?
    for (int i=1; i<nBodies; ++i) {
        FIXTURE_DEF = PhysicsFactory.createFixtureDef(25f, 0.0f, 1f);

        float angle = map(i, 0, nBodies, 0, 2*3.1415f);
        float x = cx + rx * (float)Math.sin(angle);
        float y = cy + ry * (float)Math.cos(angle);
        // Add plate
        float x1 = (x + 10) * 30;
        float y1 = y * 20;
        centers[i][0] = x1;
        centers[i][1] = y1;

          Vector2 v1 = new Vector2(x1,y1);

          circle = new AnimatedSprite(x1, y1, this.mCircleFaceTextureRegion, this.getVertexBufferObjectManager());
          circleBody[i] = PhysicsFactory.createCircleBody(this.mPhysicsWorld, circle, BodyType.DynamicBody, FIXTURE_DEF);
          this.mPhysicsWorld.registerPhysicsConnector(new PhysicsConnector(circle, circleBody[i], true, true));

          this.mScene.attachChild(circle);

    }

    for(int i= 1;i < nBodies -1; i++)  {

        Vector2 v1 = new Vector2(centers[i][0]/PIXEL_TO_METER_RATIO_DEFAULT,centers[i][1]/PIXEL_TO_METER_RATIO_DEFAULT);
        Vector2 v2 = new Vector2(centers[i+1][0]/PIXEL_TO_METER_RATIO_DEFAULT,centers[i+1][1]/PIXEL_TO_METER_RATIO_DEFAULT);
        Log.d("Vals","vals x:" + centers[i][0] + "y:" + centers[i][1] + "x:" +centers[i+1][0] + "y:" + centers[i+1%19][1]);
        int j = i+1;
        Log.d("i","i" + i + "   " + j);
        distanceJoint.initialize(circleBody[i], circleBody[(i+1)], v1, v2);
        distanceJoint.collideConnected = true;
        distanceJoint.dampingRatio = 10.0f;
        distanceJoint.frequencyHz = 1.0f;
        this.mPhysicsWorld.createJoint(distanceJoint);
    }

    Vector2 v1 = new Vector2(centers[1][0]/PIXEL_TO_METER_RATIO_DEFAULT,centers[1][1]/PIXEL_TO_METER_RATIO_DEFAULT);
    Vector2 v2 = new Vector2(centers[19][0]/PIXEL_TO_METER_RATIO_DEFAULT,centers[19][1]/PIXEL_TO_METER_RATIO_DEFAULT);
    distanceJoint.initialize(circleBody[1], circleBody[19], v1, v2);
    distanceJoint.collideConnected = true;
    distanceJoint.dampingRatio = 10.0f;
    distanceJoint.frequencyHz = 1.0f;
    this.mPhysicsWorld.createJoint(distanceJoint);

Am I missing anything? Do let me know.

A: I think I have solved it by maintaining a distance joint between every body. I was trying to recreate the Blob demo in http://www.jbox2d.org/. This demo used a ConstantVolumeJoint. I have been able to recreate this with a distance joint between every body as follows
    for(int i= 0;i < nBodies-1; i++)  {

       for(int j=i+1; j< nBodies ; j++){
           Vector2 v1 = new Vector2(centers[i][0]/PIXEL_TO_METER_RATIO_DEFAULT,centers[i][1]/PIXEL_TO_METER_RATIO_DEFAULT);
           Vector2 v2 = new Vector2(centers[j][0]/PIXEL_TO_METER_RATIO_DEFAULT,centers[j][1]/PIXEL_TO_METER_RATIO_DEFAULT);
           Log.d("Vals","vals x:" + centers[i][0] + "y:" + centers[i][1] + "x:" +centers[j][0] + "y:" + centers[j][1]);

           Log.d("i","i:" + i + " j:" + j);
           distanceJoint.initialize(circleBody[i], circleBody[(j)], v1, v2);

           distanceJoint.collideConnected = true;
           distanceJoint.dampingRatio = 10.0f;
           distanceJoint.frequencyHz = 0.5f;
           this.mPhysicsWorld.createJoint(distanceJoint);
       }
    }

I now need to have a connecting line between the bodies. Otherwise the blob does bounce off a body dropping on it.
Please let me know how to draw a connecting line between adjacent bodies that maintain the connection as the whole shape moves?