text:
package org.apache.abdera.protocol.client;

import java.io.IOException;
import java.io.InputStream;
import java.io.Reader;
import java.util.Date;

import org.apache.abdera.model.Document;
import org.apache.abdera.model.Element;
import org.apache.abdera.parser.ParseException;
import org.apache.abdera.parser.Parser;
import org.apache.abdera.parser.ParserOptions;
import org.apache.abdera.protocol.Response;

public interface ClientResponse extends Response {

    /**
     * Return the request method
     */
    String getMethod();

    /**
     * Return the request URI. The request was redirected, this will return the new URI
     */
    String getUri();

    /**
     * Release the resources associated with this response
     */
    void release();

    /**
     * Returns the inputstream used to read data from this response
     */
    InputStream getInputStream() throws IOException;

    /**
     * Returns a reader used to read data from this response. Will use the character set declared in the Content-Type to
     * create the reader
     */
    Reader getReader() throws IOException;

    /**
     * Returns a reader used to read data from this response. Will use the character set specified to create the reader
     */
    Reader getReader(String charset) throws IOException;

    void setInputStream(InputStream in);

    /**
     * If the response contains an XML document, parse the document
     */
    <T extends Element> Document<T> getDocument() throws ParseException;

    /**
     * If the response contains an XML document, parse the document using the specified ParserOptions
     */
    <T extends Element> Document<T> getDocument(ParserOptions options) throws ParseException;

    /**
     * If the response contains an XML document, parse the document using the specified Parser
     */
    <T extends Element> Document<T> getDocument(Parser parser) throws ParseException;

    /**
     * If the response contains an XML document, parse the document using the specified Parser and ParserOptions
     */
    <T extends Element> Document<T> getDocument(Parser parser, ParserOptions options) throws ParseException;

    /**
     * Return the server-specified date returned in the response
     */
    Date getServerDate();

    /**
     * Return the character set encoding specified in the ContentType header, if any
     */
    String getCharacterEncoding();

}


Abstract: The bodies of smokers are suffering even if you do not see it.
How is it possible that some people don't seem to suffer visible damage from smoking?
It may seem that some smokers don't suffer from anything, but their bodies are really suffering from the devastating effects of tobacco smoke. Airways, lungs, heart and blood vessels suffer daily from the effects of the intake of numerous damaging substances. Almost all smokers cough more than healthy non-smokers. By coughing, the body tries to remove the filth from the lungs. This is called smoker's cough. Most smokers ignore that. But it is, in fact, a sign that something is very wrong: chronic bronchitis or the beginning of that.
Other smoker's diseases are not noticeable in advance. A heart infarct seemingly happens to somebody unexpectedly. The fact that some people don't seem to have been hit by these symptoms doesn't mean that they are immune to tobacco smoke. It certainly doesn't prove that smoking is harmless for one's health. It proves at the very most that some people are apparently less troubled by the damaging effects of smoking than others.
A well-known example in the immediate surroundings appeals more to most people than data and figures obtained from large and long-lasting research. That same exceptional person is likely to be known by many others as well.
A number of investigations have demonstrated that the chance of getting diseases of the lungs, respiratory organs and heart and blood circulation are much larger for smokers than for non-smokers. Half of all deaths due to this kind of smoker's diseases are caused by smoking. In the Netherlands nearly 1% of all deaths are caused by smoking.
Many smokers forget that they don't only live shorter; the quality of their lives also deteriorates considerably. Smoker's diseases can make people disabled for a long time. Only 10 to 15% of heavy smokers have reasonable health at the age of 60.


Do bankers deserve their bonuses? Can archaeology prove or disprove the Bible? These are the types of questions Oxford University asks interview candidates looking to study at the prestigious institution.
"Interviews are designed to give candidates a chance to show their real ability and potential"
Oxford's director of admissions Samina Khan said that the interviews are mainly an "academic conversation" between tutors and a candidate and they relate to the course the student has applied for.
Dr Khan said students looking to excel at an interview should look beyond just "reciting" what they already know. She said the questions "are designed to give candidates a chance to show their real ability and potential".
These year's questions - published just days before the deadline for students to submit an application to begin their studies next autumn - include a conundrum for would-be economics and management students on whether bankers deserve their bonuses and whether the government should take any action to limit how much they receive.
Interviewer Brian Bell of Lady Margaret Hall said: "A simple answer might be that since banks are generally private firms and workers are free to work where they wish, then the pay they receive is just the outcome of a competitive labour market.
"In this story, bankers earn a lot because they are very skilled and have rare talents. It is hard to see a reason for government intervention in this case - though on equity grounds one may want to have a progressive income tax system that redistributes some of this income.
"A good candidate would wonder why it is that seemingly equivalently talented people can get paid so much more in banking than in other occupations. Do we really believe that bankers are so much better than other workers in terms of skill?
"The key point about this question is trying to get candidates to think about the economics of pay rather than just whether they think it is fair or not."
Alison Salveson of Mansfield College said she may ask someone applying for Oriental Studies whether they think archaeology can prove or disprove the Bible.
Ms Salveson said: "For this particular question I would be looking for an answer that showed the candidate could appreciate that the Bible was a collection of documents written and transmitted over several centuries, and containing important traditions that have a bearing on history, but that academic study of the Bible means that it has to be examined carefully to see when and where these traditions had come from and for what purpose they had been written," she said.
This isn't the first time Oxford publishes similar questions. Students have been asked to elaborate on the consequences of the extinction of tigers, to describe a cactus and to explain why many animals have stripes.
Interviewer: The question is focused on perhaps the most important economic question there is: why are some countries rich and some countries poor? As with most economics questions, there is no simple or unique answer. Candidates need to think about all the potential reasons why such income gaps exist.
Interviewer: This question builds on general knowledge and material studied at school in biology and chemistry to assess how students approach a clinically-relevant problem. It's commonly known that diabetes is associated with sugar (glucose) in the urine; this question asks students to think about why this occurs.
Interviewer: I like this as a question for experimental psychology because answering it brings in a range of skills relevant to the subject. Partly it involves numerical and analytical skills: the question implies that the answer will be 2/3 of some other number, but which one? Some people's first guess is 2/3 of 100, i.e., 66 or 67, in which case I'd ask them what numbers everyone else would have to pick for them to win. In this case, everyone else would have to choose 100, which is unlikely.
More often people first guess 2/3 of 50 (= 33), which seems intuitively more likely. At this point, and usually without prompting, the recursive nature of the solution becomes clear: If there is good reason for me to choose 33, then maybe everyone else will choose 33 too, in which case I should choose 2/3 of 33... but then everyone will think this and choose 2/3 of 33 too, so I should choose 2/3 of that number.. and so on. Assuming everyone thinks like this, then everyone will eventually settle on 0 as their choice – this is the formal 'game theory' solution.
At this point, I'd ask questions that bring out the candidate's broader reasoning skills in terms of thinking how we could define what it is rational to do in this game.
Interviewer: This would never be the opening question in an interview - we usually start with a first question that gives the candidate an opportunity to get comfortable by discussing something familiar. We then ask more technical questions based on material in the GCSE and A-level syllabi. This question would come later in the interview, when we present candidates with an unfamiliar scenario and ask them to use what they know about familiar concepts (such as friction) to explain something.


Warlocks 1507 - The 2019 season has begun!
On Saturday, the Warlocks kicked off the 2019 season with the game Destination: Deep Space. Missed the kickoff? Watch the game animation here!
We're excited to begin brainstorming ideas for our newest robot! During the kickoff event, the team was broken up into smaller groups and we went through a variety of scenarios, such as the Dream Team, Win in Auto, and Play the Game, to determine the best functions for our robot. Each group had time to work out the scenario specific to their group and then they reported back to the rest of the group. At the end of the day, the team gathered in the gym to watch a group of students "play" the game.
On Sunday, the team met at MAHLE to continue brainstorming ideas. By the end of the meeting, we determined what functions we believe are absolutely necessary to make our robot function smoothly.
As Week 1 begins, the students will continue brainstorming until we figure out the design we are going to pursue. Check back in on this section of the website during the season to see how the team progresses through build season and competitions.


...Not only structural member must withstand the impact of earthquake but also the owner should consider seismic zones equipment needs for the building. Seismic zones equipment needs should not be overlooked, since impact of earthquake can create numerous hazard situations that can injury and even kill occupants before and after the earthquake wave. For example, if there is a fuel tank of some sort as part of the design in or around the building, the entire system must be seismically certified.
In the event of an earthquake when power source could be non operational, emergency power systems must be in place, especially for those owners that critically need it; for example, in health-care settings special code-compliant backup power system must be in place.
As part of seismic zones equipment needs, it is required that lighting fixtures that are lay-in tile ceiling system must be independently supported directly from the structure and be supported by building's support ceiling grid system. The system must be sway-braced from at least two points of connection.
A HVAC system is another concern in seismic zones equipment needs. Proper bracing of round duct with diameter of 28 in and larger must be made. When duct work is unbraced, it must be installed with at least 6 in of clearance to ceiling-hanger wires. Proper Anchor bolts must be used to support equipment to prevent a pullout from earthquake effect. Fans, pumps, and a like, must have housing to resist lateral load forces from an earthquake.
Seismic Gas Shutoff Valve 60 psi Horizontal NPT 3/4"


The G/ATOR provides the US Marine Corp with a single multi-mission radar capable of performing the tasks of five legacy radars. This reduces the operation and maintenance costs as well as ensuring a modern capability. The system is highly mobile and highly transportable by a variety of vehicles. It can also augment sea-based air defense sensors and C2 capabilities while providing naval and joint forces with an expeditionary radar and cruise missile detection capability that extends landward battlespace coverage.[vii]Incorporating GaN technology makes the radar more efficient and advanced than other radars.
September 2018: Initial Operating Capacity (IOC) of Block 2.
May 2018: Operational Assessment of Block 2.
February 2018: IOC of G/ATOR Block 1 .
October 2017: Operational Assessment of Block 1.


In that case, I'd suggest you consider hiring someone who can give you faster help.
Otherwise, I'm sorry, but you'll have to be patient – as I said these are volunteer forums – no one gets paid to help out here. So it depends who has time and knows about your question (I don't or I'd answer it.) Any bumps or duplicate threads will be deleted by moderators.
You need a very extensive custom design for that code.
https://wordpress.org/plugins/cimy-user-extra-fields/ MAY be able to help you.
The topic 'conditional forms redirection' is closed to new replies.


Hotels of Pakistan Karachi Lahore slamabad Swat. Hotels of Pakistan,Pakistan hotels,Pakistan hotel reservations online,hotels in - Pakistan hotels. In the Freedom Fight of 1857, The people of this area resisted against the British.
Avari Hotel is the highest hotel in Pakistan with 22 floors.
Karachi is the house of a beautiful hotel called "Dreamworld Resort Hotel & Golf Club". It is on Super Highway Karachi, Pakistan.
Serena Hotel is a five star in Islamabad.
PC hotel or Pearl Continental is the most expensive hotel group of Pakistan with president suites. Rooms charges between US$ 250 to US $500 per night. Swiming pools have crystal clear water. Spa it is situated in all hotels of Pearl continental group.
Bhurban, Lahore, Karachi, Peshawar, Rawalpindi and Quetta are offering PC Hotel services.
PC is called Paradise Station.


Kaolin Crusher,Kaolin Grinding Machine,Kaolin Crushing Machine .Shanghai is a professional Kaolin grinding mill & crusher manufacturer. . Shanghai Company supply different models of grinding machines. . the companies crushing kaolin in China have exceeded 700 whose annual ore production capacity is more than 5.5 million tons and mineral processing capacity 1.8.brazil kaolin processing manufacturer machine,kaolin processing plant supplier usa gnexid orgGrinding Kaolin Clay vajirasri. Suppliers and Kaolin Kaolin clay processing plant is used for making kaolin into powder by Kaolin pulverizing Machine, grinding mill in USA, Brazil, UK. Check price.
Brazil - Mining Equipment | exportAug 10, 2017 . Brazil has a very limited market for turnkey machinery in general, as a large number of leading multinational manufacturers have factories in Brazil. .. Copelmi started last year the construction of the "Seival" mine, which will supply 2 million metric tons/year of processed coal to a new power plant that is.brazil kaolin processing manufacturer machine,brazil kaolin processing manufacturer machine,Kaolin and halloysite deposits of China - CiteSeerXboard-coating clays is not up to the quality of imported material from Brazil and the USA and there .. Shahe Mine Processing. Sedimentary. 23. Paper and ceramics. Mill. Shahe Baicuo Paper. Sedimentary. 20. Mainly for paper. Making Kaolin Mill. Kaolin and halloysite deposits of ... This is because the paper machines.
Nov 9, 2001 . China production and consumption is most difficult to define due to large number of operations reported and some debate between kaolin and ballclay production totals. Major suppliers to Japan represent all the leading producers including: Engelhard ~300kt, Imerys ~340kt, Huber ~180kt, Brazilian.
Kinc Mineral Technologies Private Limited | Mineral Processing Equipment & Machinery - Exporter of Rotary Kiln, Rotary Kiln Calcination Plant and Calcination Plants from . We provide the a state of art Rotary Calcination Plant, which adds value to the industrial minerals like Kaolin, Bauxite, Bentotine, Ball Clay etc.
earth, and kaolin. Ball clays consist primarily of kaolinite with minor to major amounts of illite, chlorite, smectite minerals, quartz, and organic materials. Bentonites .. the common clay and shale producers were manufacturers of structural clay .. Brazil (processed sales), Germany (processed ore), Uzbekistan. (processed.
The United States is the source of around one-third of all global exports of kaolin and kaolin clay. . on the kaolin. Wet processing of kaolin is used in the paper making industry. . As the kaolin comes out of processing and goes to the loading dock, it is put into a bag and has the air pressure squeezed out of it by machines.
kaolin jaw crusher repair in india -. kaolin portable crusher manufacturer in angola; brazil kaolin processing manufacturer machine; . kaolin jaw crusher repair in india Hot Product. HPC Cone Crusher. read more.
Kaolin: Processing, Technology, Applications, Equipment Suppliers, Market, Reports, Company Profiles, Consultants, Projects, Reports, Mining, Equipments Suppliers. . Kaolinite is one of the most common minerals; it is mined, as kaolin, in Brazil, France, Britain, Germany, India, Australia, Japan , the People's Republic of.
Sep 21, 2016 . Kaolin Washing Plant In The Ukraine Process Crusher, Mining sand washing machine manufacturer kaolin mining plant - Processing equipment, mining equipment, processing kaolin mining plant. crushing machine also named kaolin crusher is used widely for crushing kaolin clay into particle in kaolin.
Grinding Kaolin Clay vajirasri. Suppliers and Kaolin Kaolin clay processing plant is used for making kaolin into powder by Kaolin pulverizing Machine, grinding mill in USA, Brazil, UK. Check price.
equipment and method of kaolin production crusherasia. As a professional manufacturer of mining Kaolin processing equipment,grinding machine,processing . kaolin mines in brazil superspare. kaolin mines in brazil crusherasia. kaolin mines in brazil. Kaolin mines in brazil mineral processing system Machine.
Apr 26, 1999 . Kaolin. 6. Kaolin, or china clay, is mined and then ground and refined into a variety of grades. Kaolin is mostly used in paper-making applications, in "speciality applications" . On the other hand, in a paper-machine that is . processing, whilst kaolin in the US and Brazil can typically be found in the open air.
IIED is a company limited by guarantee and incorporated in England. Reg. No. 2188452. VAT Reg. No. GB 440 4948 50. Registered. Charity No. 800066. Mining . Kaolin. 2. Bentonite. 5. Palygorskite - Sepiolite. 7. Industrial Clays. Kaolin. Kaolin is a soft, white plastic clay consisting mainly of the mineral kaolinite which is a.
ADDRESS PARTICIPANT COMPANY'S BUSINESS ACTIVITIES .
SPECIFIC INTERESTS in Brazil and. Argentina. 1. BULGARIAN . Companies manufacturers of machines and equipment for printing and .. machines for essential oil extraction;. Industrial ventilation and air conditioning systems as well as air purification systems. 12. KAOLIN AD. 8, Dabrava str., 7038 Senovo,. District Ruse.
JCB do Brasil | Supply Chain Digital - Supply Chain News .
Nov 7, 2016 . When Jose Luis Gonçalves took over the presidency of JBC of Brazil in 2015, he announced a new wave of investments in the country. "As a family company JCB always think long term. We believe in the potential of the country and we see a huge opportunity growth for machinery market in the future.
Cranston Machinery Co. | Welcome to Cranston Machinery .
Cranston Machinery Company Inc. designs, manufactures and supports custom machinery and tools for users with unique packaging, handling and production applications. Cranston's scope for supporting its clients and these products include consumable supplies, replacement parts and in-field or factory services for.
Jan 30, 2016 . Republic of Bulgaria on his official visit to the Federative Republic of Brazil and the Republic of . Bulgarian company in processing of paddy rice. .. .kaolin.bg. 14.LB BULGARICUM Plc. 9 Saborna, Sofia 1000, Bulgaria. Tel: +359 882292556 nandreevlbbulgaricum.bg. NIKOLAY ANDREEV.


text:
\section{Introduction }

Nonparametric estimation in economic and statistical problems is common because it is appealing in applications for which functional forms are unavailable. In many problems, the primary quantities of interest can be computed from the conditional expectation function of an outcome variable $y$ given a regressor of interest $x$. Nonparametric methods are often attractive for estimating such conditional expectations since assuming an incorrect simple parametric model between the variables of interest will lead to incorrect inference.

In many econometric models, it is important to take into account conditioning information, $z$.
When $x$ is not randomly assigned, estimates of partial effects of $x$ on $y$ will often be incorrect if $z$ is ignored and at the same time, partly influences both $x$ and $y$.
However, if $x$ can be considered as approximately randomly assigned given $z$, then the conditional expectation of $y$ given $x$ and $z$ can be used to calculate causal effects of $x$ on $y$ and to evaluate counterfactuals. Therefore, properly accounting for  conditioning information $z$ is of primary importance.

When conditioning information is important to the problem, it is necessary to replace the simple objective of learning the conditional mean function $E[y|x]=g(x)$ with the new objective of learning a family of conditional mean functions \begin{equation}\label{nonpar_family}E[y|x,z]=g_z(x)\end{equation} indexed by $z$.
Properly accounting for conditioning information in $z$ may be done in several ways and leaves the researcher with important modeling decisions. For the sake of illustration, four potential ways to account for conditioning information are (1) by specifying a partially linear model $g_z(x)=g(x) + z'\beta$, (2) by specifying an additive model $g_z(x) = g(x)+h(z)$, (3) by specifying a multiplicative model $g_z(x)=g(x)h(z)$, or (4) by specifying a fully nonparametric model $g_z(x)=g(x,z)$. The fully nonparametric model suffers from the curse of dimensionality even for moderately many covariates, while the partially linear model may be too rigid and may miss important conditioning information.

Many applications have potentially large conditioning sets. A large conditioning set in economics may arise because the researcher wishes to control for many measured characteristics, like demographics, at the observation level. In the extreme cases, the dimension of $z$ can be large enough so that all four example specifications for $g_z(x)$ listed above require an infeasible amount of data in order to avoid statistical overfitting and ensure good inference.

This paper is restricted to studying the partially linear model and the additive model shown above.\footnote{These models have a particular structure which make them convenient for the problem of selecting a conditioning set. The two alternative models are likely to require conditions considerably different than the ones considered here.} Therefore, the interest is the specialization of model (\ref{nonpar_family}) to the case \begin{equation}\label{add_sep} E[y|x,z] = g(x) + h(z).\end{equation} When additive the model provides a good approximation to the underlying data generating structure, it is useful since many quantities describing the relationship of $x$ and $y$ conditional on $z$ can be learned with a good understanding of $g(x)$ alone. In addition, it provides a clear description of how the conditional relation between $x$ and $y$ changes as $z$ changes.

An important structure that has been used in recent econometrics is approximate sparsity. See for example \cite{BellChernHans:Gauss}, \cite{BellChenChernHans:nonGauss}, and  \cite{BCH-PLM}. In the context of this paper, approximate sparsity informally refers to the condition that the conditional expectation function $g_z(x)$ can be approximated by a family of functions which depend only on a small (though a priori unknown) subset of the conditioning information contained in $z$. Sparsity is useful because in principal, a researcher can address statistical overfitting problems by locating and controlling for only correct conditioning information. The focus of this paper is on providing a formal model selection technique over models for the conditioning set $z$ which retrieve the relevant conditioning information. The retrieval of relevant conditioning information will be done in such a way so that standard estimation techniques performed after model selection will provide correct inference for nonparametric partial effects of $x$.

This paper takes model (1.2) as a starting point and assumes that $h(z)$, a complicated function of many conditioning variables, has sparse structure. The strategy for identifying a sparse structure is to search for a small subset of relevant terms within a long series expansion for $h(z)$. To formalize this, a high dimensional framework, allowing the long series expansion for $h(z)$ to have more terms than the sample size is particularly convenient.  Once a simple model for $h(z)$ is found, the focus returns to the estimation of $g(x)$.
The paper contributes to the nonparametrics literature by establishing rates of convergence of estimates and asymptotic normality for functionals of $g(x)$ after formal model selection has been performed to simplify the way the conditioning variable $z$ affects the conditional expectation relation between $x$ and $y$.

In addition to addressing questions about flexible selection of conditioning set in a nonparametric setting, this paper contributes to a broader program aimed at conducting inference in the context of high-dimensional models. Statistical methods in high dimensions have been well developed for the purpose of prediction. Two widely used methods for estimating high dimensional predictive relationships and are important for the present paper are Lasso and Post Lasso. The Lasso is a shrinkage procedure which estimates regression coefficients by minimizing a loss function plus a penalty for the size of the coefficient. Post-Lasso fits an ordinary least squares regression on variables with non-identically-zero estimated Lasso coefficients. For theoretical and simulation results about the performance of these two methods, see \cite{FF:1993} \cite{T1996}, \cite{elements:book} \cite{CandesTao2007} \cite{BaiNg2008}, \cite{BaiNg2009b}, \cite{BickelRitovTsybakov2009},  \cite{BuneaTsybakovWegkamp2006}, \cite{BuneaTsybakovWegkamp2007b} \cite{BuneaTsybakovWegkamp2007}, \cite{CandesTao2007}, \cite{horowitz:lasso}, \cite{knight:shrinkage}, \cite{Koltchinskii2009}, \cite{Lounici2008}, \cite{LouniciPontilTsybakovvandeGeer2009}, \cite{MY2007}, \cite{RosenbaumTsybakov2008}, \cite{T1996}, \cite{vdGeer}, \cite{Wainright2006}, \cite{ZhangHuang2006}, \cite{BC-PostLASSO},  \cite{BuhlmannGeer2011}, \cite{BC-PostLASSO}, among many more. Regularized estimation buys stability through reduction in estimate variability at the cost of a modest bias in estimates.  Regularized estimators like the Lasso  where many parameter values are set identically to zero, also favor parsimony. Recently, several authors have begun the task of assessing uncertainties or estimation error of model parameter estimates in a wide variety of models models with high dimensional regressors (see, for example, \cite{BellChernHans:Gauss}; \cite{BellChenChernHans:nonGauss}; \cite{ZhangZhang:CI}; \cite{BCH2011:InferenceGauss};  \cite{BCH-PLM}; \cite{vdGBRD:AsymptoticConfidenceSets}; \cite{JM:ConfidenceIntervals}; and \cite{BCFH:Policy}).

Quantifying estimation precision has been shown to be difficult theoretically (for formal statements, see \cite{potscher}, \cite{leeb:potscher:pms}) because  model selection mistakes and regularization typically bias estimates to the same order of magnitude (relative to the sample size) as estimation variability. This paper builds on  methodology found in \cite{BCH-PLM} (named Post-Double-Selection) which gives robust statistical inference for the slope parameter of a treatment variable $x$ with high-dimensional confounders $z$ in the context of a partially linear model $E[y|x,z] = \alpha x + z'\eta$.\footnote{Their formulation is slightly more general because it allows the equality to be an approximate equality.} The method selects elements of $z$ in two steps: step 1 selects the terms in $z$ that are most useful for predicting $x$, and step 2 selects elements of $z$ most useful for predicting $y$ in a second step. The use of two model selection steps is motivated partially by the intuition that two necessary conditions for omitted variables bias to occur: an omitted variable exists which is (1) correlated with the treatment $x$, and (2) correlated with the outcome $y$.  Each selection step addresses one of the two concerns. In their paper, they prove that under the regularity right conditions, the two described model selection steps can be used to obtain asymptotically normal estimates of $\alpha$ and in turn to construct correctly sized confidence intervals. This paper generalizes the approach from estimating a linear treatment model to estimating a component in nonparametric additively separable models. The main technical contribution is providing conditions under which nonparametric estimates for functionals of $g(x)$ are uniformly asymptotically normal after model selection on a conditioning set over a large set of data generating processes.

\section{A high dimensional additively separable model}

This section provides an intuitive discussion of the additively separable nonparametric model explored in this paper. Recall the additive conditional expectation model described in the introduction:
$$E[y|x,z] = g_z(x) = g(x) + h(z).$$
The interest is in recovering the function $g(x)$ which describes the conditional relationship between the treatment variable of interest, $x$, and the outcome $y$. The component functions $g$ and $h$ belong to ambient spaces $g \in \mathcal G, h \in \mathcal H$ which restricted sufficiently to allow $f$ and $g$ to be uniquely identified.\footnote{For example, it necessary to require additional conditions like $g(0)=0$, otherwise, at best, $g$ and $h$ are identified up to addition by constants} The function $h$ and the variable $z$ will be allowed to depend on $n$ to facilitate a high-dimensional thought experiment for the conditioning set. As an example, this allows estimation of models of the form $E[y|x,z] = g(x) + z_{n}'\beta_n$ with $\dim(z_{n}) {\rightarrow} \infty$. Dependence on $n$ will be supressed for ease of notation.
The formulation will allow $x$ and $z$ to share variables to some extent.  For instance, the setup will allow for additive interaction models like those found in Andrews and Whang (1991) so that the model $E[y | x ,z] = g(x) + \gamma \cdot x \cdot z + h(z) $ where $\gamma$ is an unknown scalar.

The estimation of $(g,h)$ proceeds by a series approximation with a dictionary that is partially data-dependent. As a review,
a standard series estimator of the conditional expectation function without conditioning set, $g(x) = E[y|x]$, is obtained with the aid of a dictionary of transformations $p^K(x) = (p_{1K}(x), ..., p_{KK}(x))'$. The dictionary consists of a set of $K$ functions of $x$ with the property that a linear combination of the $p_{jK}(x)$ can approximate $g$ to an increasing level of precision that depends on $K$. $K$ is permitted to depend on $n$ and $p^K(x)$ may include splines, fourier series, orthogonal polynomials or other functions which may be useful for approximating $g$. The series estimator is simple and implemented with standard least squares regression.  Given data $(y_i, x_i)$ for $i = 1,...,n$, a series estimator for $g$ is takes the form: $\hat g(x) = p^K (x)' \hat \beta$ for $P = [p^K(x),...,p^K(x_n)]'$, $Y = (y_1, ...,y_n)'$ and $\hat \beta = (P'P)^{-1} P'Y$. Traditionally, the number of series terms, chosen in a way to simultaneously reduce bias and increase precision, must be small relative to the sample size. Thus the function of interest must be sufficiently smooth or simple in order for nonparametric estimation to work well. The econometric theory for nonparametric regression estimation using an approximating series expansion is well-understood under standard regularity conditions; see, for example, \cite{stone1985},\cite{chen:Chapter}, \cite{newey:series}.

Series estimation is particularly convenient for estimating models of the form of (\ref{add_sep}) because they can be approached using two dictionaries, $p^K(x), q^L(z)$ consisting of $K$ and $L$ terms which individually approximate $g(x)$ and $h(z)$. The dictionaries can simply be combined into one larger dictionary. To describe the estimation procedure in this paper, suppose such a dictionary, $(p^K(x),q^L(z) )= (p_{1K}(x),...,p_{KK}(x), q_{1L}(x),...,q_{LL}(z))$, compatible with the additively separable decomposition exists and is known. In what follows, dependence on $K$ and $L$ is suppressed in the notation so that $p^{K}(x) = p(x)$ and $q^{L}(x) = q(x)$.

The two dictionaries differ in nature. The first dictionary, $p(x)$ is traditional, and follows standard conditions imposed on series estimators, for example,\cite{newey:series}, requiring among other conditions, that $K\rightarrow \infty, K/n \rightarrow 0$. The first dictionary must be chosen to approximate the function $g(x)$ sufficiently well so that if $h(z)$ were known exactly, $g(x)$ could be estimated in the traditional nonparametric way and inferences on functionals of $g(x)$ would be reliable.

The second dictionary, $q(z)$, is afforded much more flexibility. This is convenient and appropriate when entertaining a high dimensional conditioning set $z$. When the problem of interest is in recovering and performing inference for $g(x)$, the second component $h(z)$ may be considered a nuisance parameter. In particular,
this paper will not be concerned with constructing confidence intervals for $h(z)$, and therefore the requirements on the magnitude of bias in estimating $h(z)$ will less stringent.  As a consequence, model selection bias of estimates of $h(z)$, when done according to the method below, will have negligible impact on the coverage probabilities of condifence sets.  Increased flexibility in modeling $h(z)$ by allowing $L>n$ can make subsequent inference for $g(x)$ more robust, but requires additional structure on $q(z)$.  The key additional conditions are sparse approximation conditions. The first sparsity requirement is that there is a small number of components of $q(z)$ that adequately approximate the function $h(z)$. The second sparsity requirement is that information about functions $h \in \mathcal H $ conditional on $\mathcal G$ can be suitably approximated using a small number of terms in $q(z)$. The identities of the contributing terms, however, can be unknown to the researcher a priori.

Aside from estimating an entire component of conditional expectation function $g(x)$ itself, a goal of this paper is to obtain asymptotically normal estimates of certain functionals of $g(x)$. Given a functional $a(g)$, that satisfies certain regularity conditions, the model selection procedure on $q(z)$ will deliver a model such that subsequent plug-in estimates $a(\hat g)$ will be asymptotical normal around $a(g)$. Such functionals include integrals of $g$, weighted average derivatives of $g(x)$, evaluation of $g(x)$ at a point $x^0$, and $\arg \max g(x)$.

\section{Estimation}

When the number of free parameters is larger than the sample size, model selection or regularization is necessary. There are a variety of different model selection techniques available to researchers. A popular approach is via the Lasso estimator given by \cite{FF:1993} and \cite{T1996} which in the context of regression, simultaneously performs regularization and model selection. The Lasso is used in many areas of science and image processing and has demonstrated good predictive performance.  Lasso allows the estimation of regression coefficients even when the sample size is smaller than the number of parameters by adding to the quadratic objective function a penalty term which mechanically favors regression coefficients that contain zero elements. By taking advantage of ideas in regularized regression, this paper demonstrates that quality estimation of $g(x)$ can be attained even when $K + L$, the effective number of parameters, exceeds the sample size $n$. Estimating proceeds by a model selection step that effectively reduces the number of parameters to be estimated. There are many other sensible candidates for model selection devices in the statistics and econometrics literature. The appropriate choice of model selection methodology can be tailored to the application. In addition to the Lasso, variants of Lasso like the group-Lasso, the Scad (see \cite{FanLv2006}), the BIC, the AIC all feasible examples. In the exposition of the results, the model selection procedure used will be specifically the Lasso because it is simple and widely used. The section 3.2 below provides a brief review of Lasso, especially those that arise in econometric applications.

Estimation of $E[y|x,z]$ will be based on a reduced dictionary $(\tilde p(x), \tilde q(z))$ comprised of a subset of the series terms in $p(x)$ and $q(x)$. Because the primary object of interest is $g(x)$, it is natural to include all terms belonging to $p(x)$ in the reduced dictionary, giving $\tilde p(x) \equiv p(x)$. Therefore, the main selection step involves choosing a subset of terms from $q(z)$. Given a model selection procedure which provides a new reduced dictionary $\tilde q(z)$, containing $\tilde L < L$ series terms, the post-model selection estimate of $g(x)$ is defined by
$$\hat g(x) = p(x)'\hat \beta$$
where $[\hat \beta' \ \hat \eta']' := ([ P \ \tilde Q] '[P \ \tilde Q])^{-1}[ P \ \tilde Q]'Y$.

Since estimation of $h(z)$ is of secondary concern, only the components of $h(z)$ predictive of $g(x)$ and $y$ need to be estimated. These two predictive goals will guide the choice of model selection procedure as described in the upcoming sections. The results will demonstrate that under standard regularity, the post-model selection estimates give convergence rates for $\hat g(x)$ which are the same as in classic nonparametric estimation as well as asymptotic normality results for plug in estimators, $\hat a(g)= a(\hat g)$ of nonlinear functionals $a$ of the underlying conditional expectation function.

\subsection{Nonparametric Post-Double Selection in the Additive Model}

The main challenge in statistical inference after model selection is in attaining robustness to model selection errors. When coefficients are small relative to the sample size (ie statistically indistinguishable from zero), model selection mistakes are unavoidable.\footnote{Under some restrictive conditions, for example beta-min conditions which constrain nonzero coefficients to have large magnitudes, perfect model selection can be attained.} When such errors are not accounted for, subsequent inference has been shown to be potentially severely misldeading.
This intuition is formally developed in \cite{potscher} and \cite{leeb:potscher:pms}. Offering solutions to this problem is the focus of a number of recent papers; see, for example, \cite{BellChernHans:Gauss}, \cite{BellChenChernHans:nonGauss}, \cite{ZhangZhang:CI}, \cite{BCH2011:InferenceGauss},  \cite{BCH-PLM}, \cite{vdGBRD:AsymptoticConfidenceSets}, \cite{JM:ConfidenceIntervals}, \cite{BCFH:Policy}, and \cite{BCHK:Panel}.\footnote{Citations are ordered by date of first appearance on arXiv.} This section extends the approach of  \cite{BCH-PLM} to the nonparametric setting.

Informally, model selection in the additively separable model proceeds in two steps. The two selection steps are based on the observation that the functional relation $g(x)$ can be learned with knowledge of the conditional expectations
\begin{equation} \label{LinearFE1}E[\varphi(x)|z]\end{equation}
\begin{equation}\label{LinearFE2} E[y|z] \end{equation} for a robust enough family of test functions $\varphi(x)$, for instance, smooth functions with compact support. Equivalently, the relation $g(x)$ can be learned by projecting out the variable $z$ and working with residuals.
In the additively separable model, the two selection steps are summarized as follows:

\

(1) \textit{First Stage Model Selection Step} - Select those terms in $q$ which are relevant for predicting terms in $p$.

\

(2) \textit{Reduced Form Model Selection Step} - Select those terms in $q$ which are relevant for predicting $y$.

\

To further describe the selection stages, it is convenient to ease notation by introducing an operator $T$ on functions that belong to $\mathcal G$:
$$ T\varphi(z) = E[\varphi(x) | z] $$
This notion is convenient for understanding the validity behind post double selection in the additively separable model. The operator $T$ measures dependence between functions in the ambient spaces $\mathcal G, \mathcal H$ which house the functions $g, h$ and the conditioning is understood to be on all function $\varphi \in \mathcal H$.

If the operator $T$ can be suitably well approximated, then the post double selection methodology generalizes to the nonparametric additively separable case. The operator $T$ on $\phi$ will be approximated as a linear combination, given by $\Gamma_\phi$ of basis terms $q$ so that
$$T\varphi(z) \approx q(z)'\Gamma_\varphi.$$
Meanwhile, $\Gamma_\varphi$ is approximated with linear combinations of $\Gamma_{p_k}$, $1 \leq k \leq K$.
The final selected model $\tilde q$ consists of the union of terms selected during the first stage model selection step and the reduced form model selection step.  A practical implementation algorithm is provided in Section3.3

 \cite{BCH-PLM} develop and discuss the post-double-selection method in detail for partially linear model. They note that including the union of the variables selected in each variable selection step helps address the issue that model selection is inherently prone to errors unless stringent assumptions are made. As noted by \cite{leeb:potscher:pms}, the possibility of model selection mistakes precludes the possibility of valid post-model-selection inference based on a single Lasso regression within a large class of interesting models. The chief difficulty arises with covariates whose effects in (\ref{LinearFE1}) are small enough that the variables are likely to be missed if only (\ref{LinearFE1}) is considered but have large effects in (\ref{LinearFE2}). The exclusion of such variables may lead to substantial omitted variables bias if they are excluded which is likely if variables are selected using only (\ref{LinearFE1}).\footnote{The same is true if only (\ref{LinearFE2}) is used for variable selection exchanging the roles of (\ref{LinearFE1}) and (\ref{LinearFE2}).} Using both model selection steps guards against such model selection mistakes and guarantees that the variables excluded in both model selection steps have a neglible contribution to omitted variables bias under the conditions listed below.

\subsection{Brief overview of Lasso methods}
The following description of the Lasso estimator is a review of the particular implementation given in \cite{BellChenChernHans:nonGauss}. Consider the conditional expectation $E[y|w] = f(w)$ and assume that $\varrho (w)$ is an approximating dictionary for the function $f(w)$, so that $f(w) \approx \varrho (w) ' \vartheta$, with dimension $M=\dim(\varrho(w))$. The Lasso estimates for $\vartheta$ and $f(w)$ are defined by
$$\hat \vartheta \in \text{arg} \min_{t\in \mathbb R^M} \sum_{i=1}^n (y_i - \varrho(w_i)' t)^2 + \lambda \sum_{j=1}^M | \hat \Psi_j t_j |$$
$$ \hat f_{\text{Lasso}}(w) = \varrho(w)'\hat \vartheta$$
where $\lambda$ and $\hat \Psi_j$ are tuning parameters named the penalty level and the penalty loadings. \cite{BellChenChernHans:nonGauss} provided estimation methodology as well as results guaranteeing performance for the Lasso estimator under conditions which are common in econometrics including heteroskedastic and non-Gaussian disturbances. Tuning parameters are chosen to balance regularization and bias considerations.\footnote{ For the simple heteroskedastic Lasso above,\cite{BellChenChernHans:nonGauss} recommend setting
$$\lambda = 2c\sqrt n \Phi^{-1}(1-\gamma/2M), \hspace{3mm} \Psi_j = \sqrt{ \sum_{i=1}^n \varrho_j(w_i)^2 (y_i-f(w_i))^2/n}$$
with $\gamma \rightarrow 0$ sufficiently slowly, and $c>1$. The choices $\gamma = \log^{-1} n$ and $c = 1.1$ are acceptable. The exact values $f(w_i)$ are unobserved, and so a crude preliminary estimate $\hat f(w_i) = \frac{1}{n} \sum_{i=1}^n y_i $ is used to give $\hat \Psi_j = \sqrt{ \sum_{i=1}^n \varrho_j(w_i)^2 (y_i-\hat f(w_i))^2}$. Estimates of $\hat f(w_i)$ can be iterated on as suggested by \cite{BellChenChernHans:nonGauss}. The validity of the of the crude preliminary estimate as well as iterative estimates are detailed in the appendix.} Performance bounds for the Lasso, including rates at which $\sum_{j=1}^L |\vartheta_j - \vartheta|, \sum_{j=1}^L |\vartheta_j - \vartheta|^2, \sum_{i=1}^n |f(w_i) - \hat f(w_i)|^2$ approach zero, are derived on the what is called the Regularization event.  The Regularization event is defined by $\mathscr R = \{ \lambda > 2c \hat \Psi_j ^{-1} \mathcal S_j \ for \ each \ j\leq L \}$, for a fixed constant $c>1$ where $\mathcal S_j$ is the partial derivative of the least squares part of the objective in the $j$th direction.   Informally, under the regularization event, the penalty level is high enough so that coefficients which cannot be statistically separated from zero are mechanically set to identically zero as a consequence of the aboslute values in the above objective function.  Because performance bounds are directly proportional to $\lambda$, Lasso can be shown to perform well for small values of $\lambda$ which are nevertheless large enough so that the regularization event occurs with high probability.

Lasso performs particularly well relative to some more traditional regularization schemes (eg. ridge regression) under sparsity: the parameter $ \varrho$ satisfies $|\{j : \varrho_j \neq 0 \}| \leq s$ for some sequence $s <<n$. A feature of the nature of the Lasso penalty that has granted Lasso success is that it sets some components of $\hat \varrho$ to exactly zero in many cases. Under general conditions,
$$\hat I = |j : \hat \vartheta_j \neq 0| \leq Cs \ \ \text{with probability } 1-o(1)$$
for a constant $C$ that depends on the problem.
The Post-Lasso estimator is defined as the least squares series estimator that considers only terms selected by Lasso (ie terms with nonzero coefficients):

$$\hat f_{\text{Post-Lasso}}(w) = \varrho(w)' \hat \vartheta_{\text{Post-Lasso}}; \ \
\vartheta_{\text{Post-Lasso}} \in \underset{{\{t: t_j = 0, \ \forall j \notin \hat I\}}}{\text{argmin}} \sum_{i=1}^n (y_ i - \varrho(w_i)'t)^2$$

\subsection{Lasso in post-double selection in the additively separable model}

In this section, the use of Lasso is applied directly to the first and second stage problems described in section 3.2 Starting with constructing an approximation to the operator $T$. Each component of $p$ is regressed onto the dictionary $q$ giving an approximation for $Tp_{k}(x)$ as a linear combination of elements $q(x)$ for $1 \leq k \leq K$. If this can be done with all $p_{k},$ for each $1 \leq k \leq K$, then $T$ applied to a linear combination of $p(x)$, namely $Tp(x)'\beta$, can also be approximated by a linear combination of elements of $q$. The estimation can be summarized with one optimzation problem which is equivalent to $K$ separate Lasso problems. All nonzero components of the solution to the optimization are collected and included as elements of the refined dictionary $\tilde p$.

$$\hat \Gamma = \text{arg} \min_{\Gamma \in \mathbb R^{L \times K}} \sum_{k=1}^{K} \sum_{i=1}^n (p_{k}(x_i) - q(z_i)' \Gamma_{k})^2 + \lambda^{FS} \sum_{k=1}^{K} \sum_{j=1}^{L} | \hat \Psi_{jk}^{FS} \Gamma_{kj} |.$$ Note that the estimate $\hat \Gamma$ approximates $T$ in the sense that $(q(x)' \hat \Gamma)'\beta $ approximates $Tp_1(x)' \beta$. The first stage tuning parameters $\lambda^{FS}, \hat \Psi_{jk}^{FS}$ are chosen similarly to the method outlined above but account for the need to estimate effectively $K$ different regressions. Set $$\lambda^{FS} = 2c\sqrt n \Phi^{-1}(1-\gamma/2KL), $$ $$\hspace{3mm} \Psi^{FS}_{jk} = \sqrt{ \sum_{i=1}^n q_j(z_i)^2 ( p_k(x_i) - Tp_k(x_i))^2/n}.$$ As before, the $\Psi^{FS}_{jk}$ are not directly observable and so estimates $\hat \Psi^{FS}_{jk}$ are used in their place. The mechanical implementation for calculating $\hat \Psi^{FS}$ is described in the appendix. Details for the constants involved in choosing $\lambda$ are also given in the appendix. The appearence of $K$ term in $\lambda$ ensures that the performance of the Lasso model selection works uniformly well over over the $K$ different Lassos of the first stage.

Running the regression above will yield coefficient estimates of exactly zero for many of the $\Gamma_{kj}$. For each $1 \leq j \leq K$ let $\hat I_k = \{j: \Gamma_{kj} \neq 0\}$. Then the first stage model selection step selects exactly those terms which belong in the union $\hat I^{FS} = \hat I_1 \cup ... \cup \hat I_{K}$.

The reduced form selection step proceeds after the first stage model selection step. For this step, let

$$\hat \pi = \arg \min_\pi \sum_{i=1}^n (y_i - q(x_i)'\pi)^2 + \lambda^{RF}\sum_{j=1}^{L} |\hat \Psi_j^{RF} \pi_j|$$

Where the reduced form tuning parameters $\lambda^{RF}, \hat \Psi_{jk}^{RF}$ are chosen according to the method outlined above with $$\lambda^{RF} = 2c\sqrt n \Phi^{-1}(1-\gamma/2L), $$ $$\hspace{3mm} \Psi_j^{RF} = \sqrt{ \sum_{i=1}^n q_j(z_i)^2 (y_i - E[y_i | z_i])^2/n}.$$ Let $\hat I^{RF} = \{j: \pi_{j} \neq 0\}$ be the outcome of the reduced form step of model selection.

Considering the set of dictionary terms selected in the first stage and reduced form model selection steps. Let $\hat I$ be the union of all dictionary terms:  $\hat I = \hat I^{FS} \cup \hat I^{RF}$. Then define the refined dictionary by $(p(x), \tilde q(z)) \equiv  (p(x), \{ q(z) \} _{j \in \hat I} )$. Let $[P \ \ \tilde Q]$ be the $n \times (K + |\hat I|)$ matrix with the observations of the refined dictionary stacked. The post-double-model selection estimate for $g(x)$ is defined by
\begin{equation}\label{post-double-estimate}\hat g(x) = p(x)'\hat \beta \end{equation}
where $[\hat \beta' \ \hat \eta']' := ([ P \ \tilde Q] '[P \ \tilde Q])^{-1}[ P \ \tilde Q]'Y$.

\section{Regularity and Approximation Conditions}

In this section, the model described above is written formally and conditions guaranteeing convergence and asymptotic normality of the Post-Double Selection Series Estimator are given.

\begin{assumption} (i) $(y_i, x_i,z_z) $ are i.i.d. random variables and satisfy $E[y_i | x_i,z_i] = g(x_i) + h(z_i)$ with $g \in \mathcal G$ and $h \in \mathcal H$ for pre-specified classes of functions $\mathcal G, \mathcal H$.
\end{assumption}

The first assumption specifies the model. The observations are required to be identically distributed, which is stronger than the treatment of i.n.i.d variables given in Belloni, Chernozhukov and Hansen (2011).

\subsection{Regularity and approximation conditions concerning the first dictionary}

The following few definitions help characterize smoothness properties of target function $g$ and approximating functions $p$. Let$f$ be a function defined on the support $\mathcal X$ of $x$.  Define the Sobolev norm $|f|_d = \sup_{x\in X} \max_{|a| \leq d} \partial ^{|a|} f / \partial x^a$. In addition, let $\zeta_d(K) = \max_{|a| \leq d} \sup_{x \in X} \| \partial^{|a|} p(x) /\partial x^a \| $ where $\| \cdot \|$ denotes the Euclidean norm. Throughout the exposition, all assumptions will be required to hold for each $n$ with the same set of implied constants.

\begin{assumption} There is an integer $d \geq 0$, a real number $ \alpha>0,$ and vectors $ \beta = \beta_{K}$ such that $\| \beta \| = O(1)$ and $|g - p' \beta|_d = O(K^{-\alpha})$ as $K \rightarrow \infty$.
\end{assumption}

Assumption 2 is standard in nonparametric estimation. It requires that the dictionary $p$ can approximate $g$ at a pre-specified rate. Values of $d$ and $\alpha$ can be derived for particular classes of functions. \cite{newey:series} gives approximation rates for several leading examples, for instance orthogonal polynomials, regression splines, etc.

\begin {assumption}
For each $K$, the smallest eigenvalue of the matrix $$E\[ (p(x) - Tp(x)(z) ) (p(x) - Tp(x) (z)) ' \]$$ is bounded uniformly away from zero in $K$. In addition, there is a sequence of constants $\zeta_0(K)$ satisfying
$\sup_{ x \in \mathcal X } \| p(x)\| \leq \zeta_0(K)$
and $\zeta_0(K)^2K/n \rightarrow 0$ as $n \rightarrow \infty$.
\end{assumption}

This condition is a direct analogue of a combination of Assumption 2 from Newey (1997) and the necessary and sufficient conditions for estimation of partially linear models from \cite{robinson}. Requiring the eigenvalues of $E\[ (p(x) - Tp(x) (z)) (p(x) - Tp(x)(z) ) ' \]$ to be uniformly bounded away from zero is effectively an identifiability condition. It is an analogue of the standard condition that $E[p(x)p(x)']$ have eigenvalues bounded away from zero specialized to the residuals of $p(x)$ after conditioning on $z$. The second condition of Assumption 3 is a standard regularity condition on the first dictionary.

\subsection{Sparsity Conditions} The next assumptions concern sparsity properties surrounding the second dictionary $q(z)$, used for approximating $h(z)$. As outlined above, sparsity will be required along two dimensions in the second dictionary: both with respect to the outcome equation (1) and with respect to the functional $T$. Consider a sequence $s=s_n$ that controls the number of nonzero coefficients in a vector. A vector $X$ is $s-$sparse if $|\{j: X_j \neq 0 \}| \leq s$. The following give formal restrictions regarding the sparsity of the outcome equation relative to the second approximating dictionary as well as a sparse approximation of the operator $T$ described above.

\begin{assumption} Sparsity Conditions: there is a sequence $s=s_n$ and $\phi = s \log(\max\{KL, n\})$ such that

(i) Approximate sparsity in the outcome equation: there is a sequence of vectors $\eta = \eta_{L}$ that are $s$-sparse and the approximation $\sqrt{\sum_{i=1}^n( h(z_i) - q(z_i) ' \eta)^2/n } := \xi_{0} = O_P(\sqrt {\phi/n})$ holds.  In addition, $\max_{i\leq n} |h(z_i) - q(z_i)'\eta| = o_P(1)$.

(ii) Approximate sparsity in the first stage. There are $s-$sparse $\Gamma_k = \Gamma_{k,L}$ such that $\max_{k \leq K} \sqrt {\sum_{i=1}^n \( E[p_{k}(x_i) | z_i] - q(z_i)'\Gamma_k \)^2 /n } := \xi_{FS} =O_P( \sqrt {\phi/n}). $  In addition, $\max_{i \leq n, k \leq K} |E[p_k(x_i)|z_i] - q(z_i)'\Gamma_k| = o_P(1)$.

(iii) s=o(n)
\end{assumption}

The assumption above imposes only a mild condition on the sparsity $s$ and in a sense may be thought of as definitional. In the discussion that follows, additional conditions on the size of the sparsity level $s$ will be imposed. As a preview, the conditions listed in Assumption 7 will require that $K\phi n^{-1/2} = Ks \log(\max\{KL,n\})n^{-1/2} \rightarrow 0$, among other conditions.

The first statement requires that the second dictionary can approximate $h$ using a small number of terms.  The average squared approximation error from using a sparse $\eta$ must be smaller than the conjectured estimation error when the subset of the correct terms is known.  This restriction on the approximation error follows the convention used by \cite{BellChenChernHans:nonGauss}.  The second restriction on the maximum approximation error is used to simplify the proofs.  The second statement of Assumption 4 generalizes the first approximate sparsity requirement. It requires that each component of the dictionary $p$ can be approximated by a linear combination of a small set of terms in $q$.

Additional discussion of the sparsity assumptions are given in Section 6 which addresses issues arising in the implementation of non-parametric post-double estimates.

\subsection{Regularity conditions concerning the second dictionary}
The following conditions restrict the sample Gram matrix of the second dictionary. A standard condition for nonparametric estimation is that for a dictionary $P$, the Gram matrix $P'P/n$ eventually has eigenvalues bounded away from zero uniformly in $n$ with high probability. If $K+L >n$, then the matrix $[P Q] '[P Q]/n$ will be rank deficient. However, in the high-dimensional setting, to assure good performance of Lasso, it is sufficient to only control certain moduli of continuity of the empirical Gram matrix. There are multiple formalizations of moduli of continuity that are useful in different settings, see  \cite{BickelRitovTsybakov2009}, \cite{zhou2009a} for explicit examples. This paper focuses on a simple condition that seems appropriate for econometric applications. In particular the assumption that only small submatrices of $Q' Q /n$ have well-behaved eigenvalues will be sufficient for the results that follow. In the sparse setting, it is convenient to define the following sparse eigenvalues of a positive semi-definite matrix $M$:

$$\varphi_{\min} (m)(M) := \min_{1 \leq \|\delta \|_0 \leq m} \frac{\delta ' M \delta }{\|\delta\|^2}, \hspace{5mm} \varphi_{\max} (m)(M) := \max_{1 \leq \|\delta \|_0 \leq m} \frac{\delta ' M \delta }{\|\delta\|^2}$$

In this paper, favorable behavior of sparse eigenvalues is taken as a high level condition and the following is imposed.

\begin{assumption} For every constant $C>0$ there are constants $\kappa'' > \kappa' > 0$ which may depend on $C$ such that with probability $\rightarrow 1$, the sparse eigenvalues obey $$\kappa' \leq \varphi_{\min}( C s K)(Q' Q/n ) \leq \varphi_{\max}(C s K)(Q' Q/n ) \leq \kappa''.$$
\end{assumption}

\noindent Assumption 5 requires only that certain ``small" $CsK \times CsK$ submatrices of the large $p \times p$ empirical Gram matrix $Q'Q/n$ are well-behaved. This condition seems reasonable and will be sufficient for the results that follow. Informally it states that now small subset of covariates in $q$ suffer a multicollinearity problem. The could be shown to hold under more primitive conditions by adapting arguments found in \cite{BC-PostLASSO} which build upon results in \cite{ZhangHuang2006} and \cite{RudelsonVershynin2008}; see also \cite{RudelsonZhou2011}.

\subsection{Moment Conditions}The next conditions are high level conditions about moments and the convergence of certain sample averages which ensure good performance of the Lasso as a model selection device. They allow the use of moderate deviation results given in \cite{jing:etal} which ensures good performance of Lasso under non-Gaussian and heteroskedastic errors.  \cite{BCH-PLM} discuss plausibility of these types of moment conditions for various models for the case $K$ = 1. For common approximating dictionaries for a single variable, the condition can be readily checked in a similar manner.

\begin{assumption} Set $\epsilon = y - g(x)-h(z)$. For each $k \leq K$ let $W_k = p_{k}(x) - Tp_{k}(x)$ and define $\varpi_{jk}^{FS} = \( E\[|q_j(z_i)W_{ik}|^3\]\)^{1/3}, \ \varpi_{j}^{RF} = \( E\[|q_j(z_i)\epsilon_{i}|^3\]\)^{1/3}$. Let $c,C$ be constants that do not depend on $n$. The following conditions are satisfied with probability $1-o(1)$ for each $1 \leq k \leq K, 1 \leq j \leq L, :$
\begin{align*}
(i) \ \ & c<E[q_j(z_i)^2\epsilon_i^2], E[q_j(z_i)^2W_{ik}^2]<C \\
(ii) \ & 1 \leq \max_{j \leq L} \Psi_{j}^{RF} / \min_{j \leq L} \Psi_{j}^{RF} , \max_{j \leq L} \Psi_{jk}^{FS} / \min_{j \leq L} \Psi_{jk}^{FS} \leq C \\
(iii) \ & 1 \leq \max_{j \leq L} \varpi_{jk}^{FS} / \sqrt{ E \Psi_{jk}^{FS2} },\max_{j \leq L} \varpi_{j}^{RF} / \sqrt{ E \Psi_{j}^{RF2} } \leq C \\
(iv) \ & \log ^3(KL) = o(n) \ and \ s\log(KL) = o(1) \\
(v) \ \ & \max_{k \leq K} \max_{j \leq L} \frac{ | \Psi_{jk}^{FS} - \sqrt{ E (\Psi_{jk}^{FS} )^2 } | }{ \sqrt{ E (\Psi_{jk}^{FS})^2 } } , \max_{j \leq L} \frac{| \Psi_{j}^{RF} - \sqrt{ E ( \Psi_{j}^{RF})^2 } | }{ \sqrt{ E (\Psi_{j}^{RF})^2 } } = o(1) \\
(vi) \ & \ell \Psi_{jk}^{FS} \leq \hat \Psi_{jk}^{FS} \leq u \Psi_{jk}^{FS}, \ell \Psi_{j}^{RF} \leq \hat \Psi_{j}^{RF} \leq u \Psi_{j}^{RF}
\end{align*}
\end{assumption}

\subsection{Global Convergence} The first result is a preliminary result which gives bounds on convergence rates for the estimator $\hat g$.  They are used in the course of the proof of Theorem 1 below, the main inferential result of this paper.  The proposition is a direct analogue of the rates given in Theorem 1 of \cite{newey:series} which considers estimation of a conditional expectation $g$ without model selection over a conditioning set.   The rates obtained in Proposition 1 match the rates in \cite{newey:series}.

\begin{proposition}Under assumptions listed above, the post-double-model-selection estimates for the function $g$ given in equation \ref{post-double-estimate} satisfy
$$\int (g(x) - \hat g(x))^2 dF(x) = O_p(K/n + K^{-2\alpha})$$
$$|\hat g - g |_d = O_P(\zeta_d(n) \sqrt{K }/ \sqrt{n} + K^{-\alpha}).$$
\end{proposition}

\section{Inference and asymptotic normality}
In this section, formal results concerning inference are stated. Consider estimation of a functional $a$ on the class of functions $\mathcal G$. The quantity of interest, $\theta = a(g)$, is estimated by $$\hat \theta = a(\hat g).$$

The following assumptions on the functional $a$ are imposed. They are regularity assumptions that imply that $a$ attains a certain degree of smoothness. For example, they imply that $a$ is Fr\'{e}chet differentiable.

\begin{assumption} Either (i) $a$ is linear over $\mathcal G_1;$
or
(ii) for $d$ as in Assumption 2, $\zeta_d(K)^4 K^2/n \rightarrow 0$. In addition, there is a linear function $D(f, \tilde f)$ that is linear in $f$ and such that for some constants $C, \nu>0$ and all $\bar f, \tilde f$ with $|\tilde f - g|_d < \nu$, $|\bar f - g|_d < \nu$, it holds that $\|a(f) - a(\tilde f) - D(f-\tilde f;\tilde f) \| \leq C(|f-\tilde f|_d)^2 $ and $\|D(f;\tilde f) - D(f;\bar f)\|\leq L |f|_d |\tilde f - \bar f|_d.$
\end{assumption}

The function $D$ is related to the functional derivative of $a$. The following assumption imposes further regularity on the continuity of the derivative. For shorthand, let $D(g) = D(g;g_0)$.

\begin{assumption} Either (i) $a$ is scalar, $|D(g)| \leq C |g|_d$. There is $\bar \beta$ dependent on $K$ such that for $\bar g (x) = p(x)'\bar \beta$, it holds that $E[\bar g(x)^2] \rightarrow 0$ and $D(\bar g) \geq C > 0;$ or
(ii) There is $v(x)$ with $E[v(x)v(x)']$ finite and nonsingular with $D(g) = E[v(x)g(x)]$ and $D(p_{k}) = E[v(x)p_{k}(x)]$ for every $k$. There is $\tilde \beta$ so that $E[\|v(x) - p(x)'\tilde \beta \|^2] \rightarrow 0$.
\end{assumption}

In order to use $\hat \theta$ for inference on $\theta$, an approximate expression for the variance $var(\hat \theta)$ is necessary. As is standard, the expression for the variance will be approximated using the delta method. An approximate expression for the variance of the estimator $\hat \theta$ therefore requires an appropriate derivative of the function $a$, (rather, an estimate). Let $A$ denote the derivatives of the functions belonging to the approximating dictionary, $A = (D(p_{1}),...,D(p_{K}))'.$ Let $\hat A = \frac{\partial a( p(x) ' b) }{\partial b}(\hat \beta) $.  The approximate variance, from the delta method is given by $V=V_{K}$:
\begin{align*}
& V = AQ^{-1}\Sigma Q^{-1} A \\
& \Omega = E[(p(x) - Tp(x)) (p(x) - Tp(x)) '] \\
& \Sigma = E[(p(x) - Tp(x)) (p(x) - Tp(x)) ' (y-g(x))^2] \\
\end{align*}
These quantities are unobserved but can be estimated:
\begin{align*}
&\hat V = \hat A \hat \Omega ^{-1} \hat \Sigma \hat \Omega^{-1} \hat A\\
&\hat \Omega = \sum_{i=1}^n (p(x_i) - \hat p(x_i)) (p(x_i) - \hat p(x_i)) '/n \\
&\hat \Sigma = \sum_{i=1}^n (p(x_i) - \hat p(x_i)) (p(x_i) - \hat p(x_i)) ' (y-\hat g(x_i))^2 /n
\end{align*}
The elements $\hat p_k(x_i)$ are obtaining as the predictions from the least squares regresson of $p_k(x_i)$ onto the selected $\tilde q(z_i)$. Then $\hat V$ is used as an estimator of the asymptotic variance of $\hat \theta$ and assumes a sandwich form.

The following assumptions are needed in order to bound $\hat V - V$:

\begin{assumption} Define the moments $\varpi_{j}^{q\epsilon} = \( E\[|q_j(z_i)\epsilon_{i}|^3\]\)^{1/3}$, $\varpi_{k}^{W\epsilon} = \( E\[|W_{ki}\epsilon_{i}|^3\]\)^{1/3}$. Let $q>4, C>0$ be constants. Then for $k \leq K, j \leq L$
\begin{align*}
(i) \ \ & E[|\epsilon_{i}|^q|x_i,z_i ] \leq C \\
(ii) \ \ & \varpi_{jk}^{q\epsilon}/\sqrt{E[q_{j}(z_i)^2\epsilon_{i}^2]}, \varpi^{W\epsilon}_k/\sqrt{E[W_{ki}^2\epsilon_{i}^2]}\leq C \\
(iii) \ \ & n^{2/q} K\phi/\sqrt{n} = o(1) \\
(iv) \ \ & \(\zeta_0(K) \sqrt{K}/\sqrt{n}\) \(s\sqrt{K/n} + \sqrt{n}K^{-\alpha}\)\max_{i,j}|q_j(z_i)|=o_P(1)\\
\end{align*}
\end{assumption}

The assumption that $q>4$ moments of $\epsilon$ are bounded slightly strengthens the condition in \cite{newey:series} that fourth moments are bounded. The condition is necessary for the estimation of the final standard errors. Similarly, condition (iii) is stronger than the rate condition listed in Assumption 3. The more stringent condition is also useful for estimating standard errors. Finally, condition (ii) is analogous to Assumption 7, condition (iii) and is again useful for controlling the tail behavior of certain self-normalized sums.

The next result is the main result of the paper. It establishes the validity of standard inference procedure after model selection as well as validity of the plug in variance estimator.

\begin{theorem} Under the Assumptions 1-7,9 and Assumption 8(i), and in addition $\sqrt{n} K^{-\alpha} \rightarrow 0$ then $\hat \theta = \theta + O_P(\zeta_d(K) \sqrt{n})$ and $$ \sqrt{n} V^{-1/2} (\hat \theta - \theta) \overset{d} \rightarrow N(0,1), \hspace{5mm} \sqrt{n} \hat V^{-1/2} (\hat \theta - \theta) \overset{d} \rightarrow N(0,1)$$

If Assumptions 1-7,9 and Assumption 8(ii) hold with $d=0$ and in addition $\sqrt n K^{-\alpha} \rightarrow 0$ then for $\bar V = E[v(x)v(x)' var (y|x)]$, the following convergences hold.
$$\sqrt n (\hat \theta - \theta) \overset{d}{\rightarrow}  N(0,\bar V), \hspace{ 1cm} \| \hat V - \bar V \| \overset{p}{\rightarrow}  0$$
\end{theorem}

The theorem shows that the outlined procedure gives a valid method for performing inference for functionals after selection of series terms. Note that under assumption 8(i) the $\sqrt n$ rate is not achieved because the functional $a$ does not have a mean square continuous derivative. By contrast, Assumption 8(ii) is sufficient for $\sqrt n$-consistency. Conditions under which the particular assumptions regarding the approximation of $g$ hold are well known. For example, conditions on $K$ for various common approximating dictionaries including power series or regression splines etc follow those directly derived in \cite{newey:series}. Asymptotic normality of these types of estimates under the high dimensional additively separable setting should therefore be viewed as a corollary to the above result.

Consider one example with the functional of interest being evaluation of $g$ at a point $x^0$: $a(g) = g(x_0)$. In this case, $a$ is linear and $D(\bar g) = \bar g(x_0)$ for all functions $\bar g$. This particular example does not attain a $\sqrt n$ convergence rate provided there is a sequence of functions $g_{K}$ in the linear span of $p=p^K$ such that $E[g_{K}(x)^2]$ converges to zero but $g_{K}(x_0)$ is positive for each $K$. Another example is the weighted average derivative $a(g) = \int w(x) \partial g(x) / \partial x$ for a weight function $w$ which satisfies regularity conditions. For example, the theorem holds if $w$ is differentiable, vanishes outside a compact set, and the density of $x$ is bounded away from zero wherever $w$ is positive. In this case, $a(g) = E[v(x)g(x)]$ for $v(x) = - f(x)^{-1} \partial w(x) / \partial x$ by a change of variables provided that $x$ is continuously distributed with non vanishing density $f$. These are one possible set of sufficient conditions under which the weighted average derivative does achieve $\sqrt n$-consistency.

\section{Additional discussion of implementation}

The theorem above states that for a fixed (nonrandom) sequence $K=K_n$, asymptotically normal estimates are achieved for certain functionals of interest of $g(x)$ under the right regularity conditions.  In practice, the choice of $K_n$ is important and it is useful to have a data-driven means by which to choose $K_n = \hat K = \hat K(\{(x_i,y_i)\}_{i=1}^n)$.  This section provides suggestions for choosing such $\hat K$.  This paper leaves these suggestions as heuristics and does not derive formal theory for their asymptotic performance; however, the finite sample performance of these heuristics is explored in the simulations in Section 7. \footnote{There is also the question of whether the penalty levels $\lambda^{FS}$ and $\lambda^{RF}$ in the Lasso optimizations can be chosen in a more data-driven way.  There is  less flexibility in these choices, since the Lasso bounds are predicated on the Regularization event.  Using a smaller penalty level than suggested leads to over-selection of control variables, which can bias estimates.   Further discussion of the effects of post-model-selection inference with over-selection can be found in \cite{BCHK:Panel}.}

Suppose that candidates for $K$ belong to the integer set $\{\underline K,..., \overline K\}$.  Suppose that $L = L_n$ is nonrandom as before and does not vary with $K \in \{\underline K,..., \overline K\}$.  A simple proposal is as follows: for each $K$, construct using the post-double model selection routine, a reduced second dictionary $\tilde q^L_K$.  This results in a set of selected dictionaries which are candidates for a final estimation step:

$$\Big \{ (p^{\underline K}(x), \tilde q^L_{\underline K}(z)), ... , (p^{\overline K}(x), \tilde q^L_{\overline K}(z)) \Big \}.$$

\noindent Then $\hat K$ can be chosen by selecting from the dictionaries in the above set.  In principal this can be done in many ways, including cross validation or BIC, possibly with additional over-smoothing (i.e. choosing $\hat K$ larger than say the mean-square error optimal.)

The informal reasoning behind this proposal is that for each $K$, the post-double model selection selects confounders in a way so that the omitted variables bias resulting from possibly omitting covariates predictive of elements of $p^K(x)$ is small relative to sampling variability.  But there can be another component of omitted variables bias:  namely, the omitted variables biases resulting in excluding all confounders predictive only of signal in $x$ which is not accounted by $p(x)$.  However, this second component of omitted variables bias will plausibly be small if $K$ is chosen appropriately, (i.e. whenever $\sqrt{n}K^{-\alpha} \rightarrow 0$, a similar bound on such omitted variables bias may be expected to hold.)

In addition to issues arising from choosing $K$,  the sparsity conditions imposed in the previous sections are restrictive.  However, without such a sparse structure, it is difficult to construct meaningful estimates of $g(x)$.  Furthermore, at the current time, there are no widely used procedures to test the null hypothesis of a sparse model that the author is aware of.

A major restriction implicit in the sparsity assumption is that the sparse approximation errors are small for \textit{all} terms in $p(x)$.  For instance, if $p(x) = (x,x^2,x^3,...)$, it is much less demanding to ask that there exists a good sparse predictor of $x$ based on $q(z)$, than it is to ask that $x$, $x^2$, $x^3$... \textit{all} have quality sparse predictors.   On the other hand, it is possible that the transformations $x^2+x^3$ or $x^3 - x^2$ might have much better sparse representations in terms of $q(z)$ than $x^2$ and $x^3$ have individually.

Therefore, an alternative strategy for the first stage is to have a model selection step for many distinct linear combinations $\alpha'p(x)$ given by $\alpha \in \mathcal A$.  The strategy is outlined as follows.   First, gather the set $\{ \alpha' p(x): \alpha \in \mathcal A \subset \mathbb R^K\}$ into an extended first stage dictionary:  $p_{\text{FS}}(x)$.  Select a reduced conditioning dictionary $\tilde q(z)$ with the nonparametric post-double selection method described above, except using $p_{FS}(x)$ in the first stage model selection step.  Finally, in the post-model-selection estimation step, estimate using $(p(x), \tilde q(z))$.  This strategy is potentially useful since it further reduces the possibility for omitted variables bias.  A clear tradeoff with using a distinct first stage dictionary is that due to the additional model selection steps introduced, more variables from $p(z)$ can potentially be selected, leading to higher variability of the final estimate of $g(x)$.

As with the data-driven choice of $K$, this suggestion is kept at the level of a heuristic at this moment.  However, arguments in the proofs of the main results can easily be extended to allow an extended to allow a first stage dictionary $p_{\text{FS}}(x)$ provided that it has a subdictionary, $p^K(x)$ for which Assumptions 1-4 hold, and that the number of selected conditioning variables $\tilde q(z)$ remains $O(sK)$ with high probability.   For example, if $\text{dim}(p_{\text{FS}}(x)) \leq C \text{dim}(p(x))$ then no substantive modification to the proof are necessary.

The finite sample performance of these heuristics is explored in the simulations in Section 7.

\section{Simulation study}

The results stated in the previous section suggest that post double selection type series estimation should exhibit good inference properties for additively separable conditional expectation models when the sample size $n$ is large. The following simulation study is conducted in order to illustrate the implementation and performance of the outlined procedure. Results from several other candidate estimators are also calculated to provide a comparison between the post-double method and other methods. Estimation and inference for two functionals of the conditional expectation function are considered. Two simulation designs are considered. In one design, the high dimensional component over which model selection is performed is a large series expansion in four variables. In the other design, the high dimensional component is a linear function of a large number of different covariates.

\subsection{Low Dimensional Additively Separable Design}

Consider the following model of continuous variables $x, z$ of form:
$$E[y|x] = E[y|x,z] = g(x) + h(z )$$
where in this simulation, the true function of interest, $g(x)$, and the conditioning function $h(z)$ are given by :
\begin{align*}
g(x) &= \text{logistic}(x) - \frac{1}{2} \\
h(z) &= \text{logistic}\(\sum_{j=1}^{\text{dim}(z)} z_{j}\)-\frac{1}{2}
\end{align*}
where $\text{logistic}(x) = \frac{\exp(x)}{1+\exp(x)}$ and the $\frac{1}{2}$ terms in the expression for $g$ is used to ensure identifiability via $g(0) = 0$. Ex post, the function is simple, however, for the sake of the simulation, knowledge of the logistic form is assumed unknown. Importantly, the logistic function will not belong exactly in the span of any finite series expansion used in the below simulation.
The second function $h$ is similar, being defined by a combination of a logistic function of a linear combination of the $z$ variables. The logistic part can potentially require many interaction terms unknown in advance to produce an accurate model. The component functions $g$ and $h$ will be used throughout the simulation. The remaining parameters, eg. dictating the data generating processes for $(y,x,z)$ will be changed across simulation to give an illustration of performance across different settings.

The objective is to estimate a population average derivative, $\theta^{(1)}$, and a function evaluation, $\theta^{(2)}$ given by

\

\textit{(i)} \ $\theta_1 =a_1(g)= \int_{\text{supp}(x)} \frac{\partial g}{ \partial x} (x) dF(x) $

\textit{(ii)} $ \theta_2 = a_2(g)= g(\text{quantile}(x,.75)) - g(\text{quantile}(x,.25)).$

\

\noindent $\hat \theta_1$ and $\hat \theta_2$ and estimates of standard errors are obtained using the post-double methodology outlined in the paper with approximating dictionaries specified below.  The plug in estimate of the average derivative, $\theta_1$, is integrated against the empirical distribution of $x$.

The covariates and outcome are drawn as follows. The distribution of the conditioning set $z$ is set at $z \sim N(0,S_{\text{dim}(z)}^{1/2})$ where $S_{\text{dim}(z)}^{1/2}$ is the Toepliz matrix of size dim$(z)$ with decay base of $\frac{1}{2}$; therefore, the correlations between $z_j$ and $z_k$ are set to $\text{corr}(z_j, z_k) = \(\frac{1}{2}\) ^ {-|j-k|}$. The variable of interest, $x$ is determined by $x = h(z) + v $ with $v \sim N(0,1)\cdot \sigma_v$ independent of $z$. The structural errors $\epsilon = y - E[y|x]$ are drawn $N(0,1) \cdot \sigma_\epsilon$ independent of $x$. Several simulations of this model are conducted by varying the sample size $n$, the dependence between $x$ and the remaining regressors $z$, as well as the size of the residual errors $\epsilon$. The sample size varied and is set to $n \in \{500, 1000\}$. The dependence between $x$ and the remaining covariates is dictated by $\sigma_v$. To capture high and low dependence between the covariates, the values $\sigma_v \in \{ 1,2 \}$ are used. Finally, the variability of the structural shocks are set to $\sigma_\epsilon \in\{1,2\}$.

Estimation is based on series expansion using Hermite polynomials. $K$ is set to $K = \text{floor}(n^{1/3})$. Therefore, $g(x)$ is approximated using a $K$ order polynomial. The series expansion $q(z)$ for the function $h(z)$ spans all polynomials in $z$ of order $\leq K$. Specifically,  using  interactions (products) of univariate $m$-order Hermite polynomials, $H_m$, on single components of $z$, the elements of $q(z)$ consist of terms in $\left \{ \prod_{j=1}^{\dim(z)} H_{m_j}(z_j): \ \sum_{j=1}^{\dim(z)} m_j \leq K \right \}$.

In addition to the standard post-double lasso based model, several alternative estimates are calculated for comparison. We give three ad hoc estimators which seem to be sensible data-driven ways to choose the number of series terms (see the description and discussion of these in Section 6).  The first is the Post-Double selection where $K = \hat K$ is chosen from a set of values $\text{floor}(\frac{1}{2}n^{1/3}) \leq \hat K \leq \text{floor}(2n^{1/3})$.   The choice is by the following procedure.  First, run post-double selection for each choice of $K$.  This produces a set of candidate models $\{ (p^{\underline K}, \tilde q^L_{\underline K}), ... , (p^{\overline K}, \tilde q^L_{\overline K}) \}$ which can be compared each other.  In this simulation, the preferred method of comparison is BIC since it is simple computationally, relative, for example, to cross-validation.  Then the final value  $\hat K$ is chosen to be $\hat K = K_{\text{BIC}} + 1$.  This estimator is referred to as the Post-Double Set estimator in the simulation results tables.

In addition, the second suggestion from Section 6, which is to augment the first dictionary in the first stage model selection step, is considered.  For given $p(x)$, the extended first stage dictionary $p_{FS}$ is constructed by $$p_{FS}(x) = p(x) \cup \{  p_j (x)+ p_{j'}(x) : j < j' \} \cup  \{ p_j(x) - p_{j'}(x):  j < j'\}$$ which is used in place of $p(x)$ in the first stage model selection step.
This gives for fixed $K$ an estimate which is called the Post-Double Ext estimate in the simulation results tables.

Finally, we consider a hybrid of the Post-Double Set estimator and the Post-Double Ext estimator where a model is selected using the Post-Double-Extend method for each $K \in \{\underline K,...,\overline K\}$.  These models are compared and the choice $\hat K$ is made according to $\hat K = K_{\text{BIC}} + 1$.  This estimator is called the Post-Double Set+Ext estimator in the simulation results tables.

In addition to Post-Double-based estimators, two additional series estimators which are designed to approximate the function $g(x)+h(z)$ are compared. The first is based on a series approximation which assumes that $h(z)$ is additively separable so that $h(z) = \sum_{j=1}^{\text{dim}(z)}h_j(z_j)$. The approximating series for each term $h_j(z_j)$ consist of Hermite polynomials in $z_j$ of order $\leq K$. The second series estimator uses identical series as the post-double selection model, but performs no model selection; that is, it proceeds under the same mechanical procedure as would be used for a standard series estimator and uses the Moore-Penrose pseudo-inverse if it needs to invert a singular matrix. Second, a single step selection estimator is provided. The single step estimator is done by performing a first stage lasso on the union of the two dictionaries, then re-estimating coefficients of the remaining dictionary terms in the second stage. Finally, an infeasible estimator is provided, where estimation proceeds as standard series estimation given the dictionary $p$ and as if $h(x_i)$ where known for each $i$.

Results for estimating $\theta_1$ and $\theta_2$ are based on 500 simulations for each setting described earlier. For each estimator, the median bias, median absolute deviation, and rejection probability for a 5-percent level test of $H_{01}: \theta_1 = \theta_{01}$ or $H_{02}: \theta_2 = \theta_{02}$ are presented. Results for estimating $\theta_1$ are presented in Table 5.4. In each design, estimates of $\theta$ based on post double selection exhibit small median absolute deviation relative to the competing estimates. With the exception of the infeasible estimates the post double selection estimates are also the only estimates which exhibit reasonable rejection frequencies consistently across all settings. Results for estimation of $\theta_{2}$ are reported in Table 5.5. The results are qualitatively similar to those for $\theta_{1}$. The only reasonable rejection frequencies are obtained with the post-double selection. A small amount of size distortion can be seen as rejection frequencies are closer to 10-percent in most simulations. The distortion in the post double estimator matches that in the infeasible estimator suggesting that they are driven by bias in approximating a nonlinear function with a small bias rather than a consequence of model selection.

\subsection{High Dimensional Additively Separable Design}

In this design, a high dimensional setting is considered:
$$E[y|x,z] = E[y|x, z] = g(x) + h(z).$$
For the functions
\begin{align*}
g(x) &= \text{logistic}(x) - \frac{1}{2} \\
h(z) &= \sum_{j=1}^{\text{dim}(z)} \(\frac{1}{2}\)^{j-1} z_{j}
\end{align*}

In this model, the dimension of $z$ is large compared to the sample size and is set to $\text{dim}(z) = \text{floor}\(2n\)$ for each scenario. The variables $z$ are drawn as above from a Gaussian with $z \sim N\(0,S_{\text{dim}(z)} ^{1/2}\)$ so that, again, the correlation structure is $\text{corr}(x_j, x_k) = \(\frac{1}{2}\) ^ {-|j-k|}$. The target function $g(x)$ remains the same as before. The dependence between $x$ and $z$ is defined with $x =h(z) + v$. The specifications for $v$, $\sigma_v$, $\epsilon$, and $\sigma_\epsilon$ are the same as they were in the low dimensional example. Estimation is again based on a series expansion in terms of Hermite polynomials in $x$. $K$ is taken to be $K = \text{floor}(n^{1/3})$. The second dictionary comprises of the variables $z_j$ themselves. The simulation evaluates the performance of the estimator when the goal is to control for many distinct possible sources of confounding. Results are recorded in Table 5.6 for sample size $n=500$ and Table 5.7 for $n=1000$. In this simulation, the single selection and infeasible estimators are defined as they were in the low dimensional simulation. The series estimator (Series I) uses the first $\text{floor}(4n/5)$ covariates set of controls in estimation. The second series estimator (Series II) randomly selects $\text{floor}(4n/5)$ of the covariates to use in estimation. The results are qualitatively similar to those for the first design.  Relative to other methods, the post-double selection method exhibits substantially lower bias.  The rejection frequencies obtained with the post-double selection are considerably closer to the target 5\% level than with other methods.

\section{  Empirical Application: Estimating the Effect of Export on HIV}

The results in the preceding sections show how variable selection methods can be used to estimate additively separable models, in which the component of interest can be considered effectively randomly assigned conditional on observables.  This section illustrates use of the results by reexamining \cite{Oster:HIV:export}, which studies of the impact of trade on HIV incidence rates in a sample of African countries.  The original results are briefly reviewed, after which estimates using the methods developed in this paper are calculated and discussed.

The arguments presented in \cite{Oster:HIV:export} propose two causal mechanisms relating trade to HIV incidence rates in African countries.  The first is simply that increased income provided by more exports leads to an increase in risky behavior.  The second mechanism is that increased trade leads to an increase in trucking and movement of people who engage in risky behavior.  The basic problem in estimating the causal impact of trade on HIV is that country-level export rates are not randomly assigned.  It is likely that there are factors like government specific policies that are associated to both trade rates and HIV rates.

The baseline model estimated in \cite{Oster:HIV:export} is for country-level incidence rates running from 1985 to 2007, which conditions on country level fixed effects, time level effects as well as lagged HIV prevelance.\footnote{The arguments in this paper, in conjuction with those given for selection of controls in high dimensional panel models in \cite{BCHK:Panel}, could be used to justify allowing dependence within countries over time. The standard errors in the current analysis are clustered by country whereas \cite{Oster:HIV:export} assumes an AR(1) structure and a linear model and proceeds with Prais-Winston regression.  }  In the present analysis, the argument given in \cite{Oster:HIV:export},  that the export rates defined above may be taken as exogenous relative to HIV incidence once observables and government or policy variables are controlled for, is taken for granted.  While \cite{Oster:HIV:export}  controls for these possible confounds by including country-specific and year-specific fixed effects and lagged prevelance, this paper allows a much richer set of variables to be used as controls.  All of the new control variables are described below and can be conveniently constructed from the original dataset used by \cite{Oster:HIV:export}. An additional benefit is that selection of a conditioning set offers a very attractive complimentary analysis to the standard practice of robustness checks.  Robustness checks typically estimate several perturbations of the baseline model.  If the value of causal estimates are insensitive to these perturbations, then this is considered evidence in favor of the researcher's original conclusions.  While robustness checks are useful, it is very difficult to come up with a principaled means for choosing a correct set of perturbed models to use.  The methods here give one such formalized procedure and serve to compliment the robustness checks performed in \cite{Oster:HIV:export}.

This paper considers a model specified by:

$$y_{it} = g(x_{it}) + \alpha_i + \gamma_t + w_{it}'\delta + z_{it}'  \theta + \varepsilon_{it}$$

\

\noindent where $i$ indexes country, $t$ indexes times, $\alpha_i$ are country-specific effects that control for any time-invariant country-specific characteristics, $\gamma_t$ are time-specific effects that control flexibly for any aggregate trends, $w_{it}$ is the lagged HIV prevelance rate.  The outcome $y_{it}$ is log measure of HIV incidence calculated in two different ways.  The first measure, named UNAIDS-based incidence, is based on data from UNAIDS, the United Nations organization responsible for reporting on the global HIV epidemic.  The second way, named Death-based incidence, extrapolates backwards from mortality rates from the Demographic Yearbook Historical Suppliment.  The $x_{it}$ are logs of three different export measures: total export value reported by the World Development Indicators; total export value reported by NBER - United Nations Trade Data; and total export volume reported by NBER-United Nations Trade Data.  The measures are labeled Log Value (WDI), Log Value (NBER) Value, and Log Volume (NBER).  For details, please refer to \cite{Oster:HIV:export}.  This paper abstracts away from important issues arising from measurement error in both incidence rates and export rates in order to provide a clearer illustration of the model selection methods.

The set of controls represented by $z_{it}$ is richer than the controls considered in the original paper.  $z_{it}$ is constructed from interactions of an aggregate linear time trend, $t$, an aggregate quadratic time trend, $t^2$, aggregate periodic time trends with periods 4 and 8 years, $\sin(2\pi t/4), \cos(2\pi t/4), \sin(2\pi t/8), \cos(2\pi t/8)$, log-population, indicators for larger regions in Africa, intial values of HIV prevalence, GDP, log-population, and export levels.  The final set of controls consists of 78 variables and can be obtained by request.  The samples consists of between 720-747 observations for the UNAIDS-based incidence measure and 161-166 for the Death-based incidence measure, with discrepencies arising from missing observations.  Therefore, a conditioning set of 78 variables, though more robust than the baseline, is large enough to potentially cause statistical problems.   Therefore, this analysis will apply the model selection procedure discussed in the earlier sections. The estimates of $g(x_{it})$ are based on a series expansion for $g$ in terms of Hermite polynomials.  The order of the approximating polynomial is chosen by $K = \text{floor}(n^{1/4})$, where $n$ is the total number of observations (not observational units).

Estimates for the sample average derivative, $\frac{1}{n} \sum \hat g'(x_{it})$, based on the UNAIDS measure are presented in Table 5 and estimates based on Death-based incidence are presented in Table 6.  In each table, the first panel uses the Log Value (WDI) measure, the second panel uses the Log Value (NBER) measure, and the third panel uses the Log Volume (NBER) measure.  The estimate is interpreted as the average percent increase in HIV incidence resulting from an exogenously given percent increase in export over the sample export values.  The tables present estimates and 95\% confidence intervals, as well as lists of selected variables for each analysis.  In addition to the post-model selection estimates, a baseline estimate, corresponding to the baseline model in \cite{Oster:HIV:export} (excluding $z_{it}$, yielding $y_{it} = g(x_{it}) + \alpha_i + \gamma_t + w_{it}'\delta + \varepsilon_{it}$), and a model using the full set of controls without model selection are presented.

The estimates presented in Table 5 are not significantly different from 0 in any of the circumstances.  This is partly consistent with the findings in \cite{Oster:HIV:export}, in that the significance does not withstand the robustness checks.  The estimates here are, in principal, more variable relative to \cite{Oster:HIV:export} since this paper considers non-parametric specifications for $g$ whereas \cite{Oster:HIV:export}  considers a linear specification.
The estimates in Table 6 are considerably more interesting.  In particular, the interepretation of the estimates changes depending on which set of controls is used.   Using the first measure of export, Log Value (WDI), the estimated average derivative is significantly different from zero under the baseline model.  However, the estimate fails to maintiain significant separation from zero when the full set of controls is used.  This can be because the true causal effect based on the richer conditioning set is null, or because the inclusion of many control variables presents a serious limitation in terms of statistical precision.  In this situation, using a selected set of controls yields a statistically significant, positive estimate.  In addition, the corresponding confidence interval in more narrow.  Therefore, for these particular measures of incidence and export, the model selection method supports the conclusion of the baseline estimate of a positive average effect.  Using the other two measures of export, the interpretations of the estimates, presented in Panels 2-3, do not change between the three estimation strategies.

\section{Conclusion}

This paper considers the problem of choosing a parsimonious conditioning set in the context of nonparametric regression.    Convergence rates and inference results are provided for series estimators of additively separable models with a high dimensional component.   Lasso continues to have good selection properties in this context and can be used in post-model selection inference when two model selection steps are performed.

redpajama_set_name: RedPajamaArXiv


text:
«Бриллиант Раджи» () — польский приключенческий телевизионный фильм, снятый в 1971 году режиссёром Сильвестром Хенциньским на студии «Iluzjon».

Экранизация одноименной повести Роберта Льюиса Стивенсона.

Премьера фильма состоялась 15 октября 1971 года.

Сюжет
Телефильм из цикла «Необычайные истории» Роберта Льюиса Стивенсона об огромном алмазе, переходившем из рук в руки. Продолжение цикла рассказов «Клуб самоубийц» о приключениях Богемского принца Флоризеля и его верного шталмейстера.

Британский генерал сэр Томас Венделер привёз из Индии драгоценный камень под названием «Алмаз раджи». Однажды ночью драгоценный камень украла жена генерала, которая строит козни против генерала. Поиски бриллианта полны необычных перипетий.

В ролях
 Владислав Ханьча — генерал сэр Томас Венделер
 Ежи Карашкевич — пастор
 Януш Клосиньский — Венделер, брат генерала, бывший диктатор Парагвая
 Кшиштоф Вакулиньский — Гарри Хартли, секретарь леди Венделер
 Чеслав Воллейко — принц Флоризель
 Анна Дымна — Джоан, дочь Венделера бывшего диктатора
 Эва Вишневская — леди Клара Венделер
 Владислав Девойно — эпизод
 Адам Цесляк — эпизод
 Хенрик Хунко — официант / полицейский
 Густав Крон — официант
 Здислав Кузняр — садовник пастора
 Эдвард Линде-Любашенко — пастор
 Мечислав Лоза — эпизод
 Тадеуш Плюциньский — ''брат леди Венделер

Съёмочная группа
 Главный режиссёр — Сильвестр Хенциньский
 Сценарий — Анджей Ярецкий
 Главный оператор — Ежи Ставицкий
 Композитор — Адам Валациньский
 Ассистенты режиссёра — Герард Залевский, Евгения Яшкевич, Беата Бильска
 Грим — Ядвига Райнска-Петровска
 Ассистенты оператора — Яцек Стахлевский, Юлиан Магда, Зигмунт Крушницкий
 Художник-постановщик — Тадеуш Косаревич
 Звукооператор — Рышард Патковский
 Художник по костюмам — Малгожата Спичальская-Комар
 Монтажёр — Кшиштоф Осецкий
 Большой симфонический оркестр Польского радио и телевидения (Катовице)
 Директор — Рышард Страшевский

Ссылки
 Diament radży

Экранизации произведений Роберта Льюиса Стивенсона
Короткометражные фильмы Польши
Телефильмы Польши
Фильмы Польши 1971 года
Приключенческие фильмы Польши

redpajama_set_name: RedPajamaWikipedia


This series of workshops introduces graduate supervision policies and discusses best practices. Individual sessions cover issues and resources relevant to supervising students at University of Waterloo, such as writing resources, ethics and integrity requirements, and the management of milestones, as well as more general issues, such as supervisory alignment, honing research questions, the supervision life cycle, working with students and co-supervisors in the context of cultural and gender diversity, and career support for various job markets.
This course covers a number of topics important to effective university teaching. In addition to policies related to Australian higher education, it addresses the following pedagogical issues: reflective teaching practices, fundamentals of research-led education, how students learn, constructive alignment of learning outcomes and assessment, flexible and technology-enhanced learning, evaluation, student support, and course design.
This workshop is designed for scholars interested in building electronic research and production models that reach new audiences within and beyond academia. Workshop meetings combine practical hands-on experience with theoretical studies concerning the ways in which digital archives, electronic databases, interactive design tools, and open-access publishing remediate fields and objects of knowledge.
Digital video is a tool that facilitates collaboration across time and space. Some scholars give cameras to participants in their study so that they can represent themselves on their own terms, while others use video to enhance their abilities to share knowledge and narratives. Visual literacy is a continuous thread throughout this workshop, which focuses on the challenges of planning a shoot, framing a shot, and creating meanings through scripting and editing decisions.
The Institute is a comprehensive seven-day, 60-hour intensive training program that covers everything necessary to develop a course in the Inside-Out model: curriculum development, setting parameters, institutional relationships, group dynamics, interactive pedagogical approaches, and much more. Two or more of the days provide an opportunity to learn this educational method through observation, hands-on experience, and engagement with the Think Tank, a working group inside Graterford Prison, a large maximum-security facility.
Each year, the Division of Teaching Excellence and Innovation selects a cadre of excellent and experienced Teaching Assistant mentors from various disciplines to serve as Pedagogical Fellows. This yearlong course (rebadged as the Certificate in Teaching Excellence Program in 2016) is part of the required training. It covers social constructivist learning epistemology, a wide range of active learning strategies, and how to design a learning-centered curriculum. Participants also receive training to provide teaching consultation to their peers.