text:
\section{Introduction}

Consider the following min-max problem
\begin{equation}
\label{minmax}
\min_{x\in X} \max_{y\in Y}~  f(x,y),
\end{equation}
where $X\subseteq \mathbb{R}^n$ and $Y\subseteq \mathbb{R}^m$ are nonempty, closed and convex sets, $f:\mathbb{R}^{n}\times \mathbb{R}^{m}\to \mathbb{R}$ is a locally Lipschitz continuous function. Define an envelope function
$$\varphi(x):=\max_{y\in Y}f(x,y).$$
In this paper, we assume that $\varphi(x)$ is finite-valued for any $x\in X$.
We say problem \eqref{minmax} is nonconvex-nonconcave if for a fixed $x\in X$, $f(x,\cdot)$ is not concave, and for a fixed $y\in Y$, $f(\cdot,y)$ is not convex.

The min-max problem \eqref{minmax} has many applications in machine learning and games \cite{JC2021pure,Qian2019Robust,ZHWZ2020generalization}, for instance, the popular generative adversarial networks (GANs) in machine learning  \cite{ACB2017wasserstein,C2018generative,G2016nips,G2014generative,M2021statistical}.
Let $D:\mathbb{R}^m\times \mathbb{R}^{s_1} \to (0,1)$ be a parameterized discriminator, $G:\mathbb{R}^n\times \mathbb{R}^{s_2} \to \mathbb{R}^{s_1}$ be a parameterized generator and $\xi_i$ be a $s_i$-valued random vector with probability distribution $P_i$ and support $\Xi_i\subseteq \mathbb{R}^{s_i}$ for each $i=1,2$. Then the plain vanilla GAN model can be formulated as
\begin{equation}
\label{GAN}
\min_{x \in X} \max_{y\in Y}~  \mathbb{E}_{P_{1}}\Big[\log\big(D(y,\xi_1)\big)\Big] +  \mathbb{E}_{P_2}\Big[\log\Big( 1 - D\big(y,G(x,\xi_2)\big)\Big)\Big],
\end{equation}
where $x$ and $y$ are the parameters to control $D$ and $G$ with ranges $X$ and $Y$, respectively. Here $\mathbb{E}_{P_i}[\cdot]$ denotes the expected value  with probability distribution $P_i$ over $\Xi_i$ for $i=1,2$. We assume that the expected values are finite for any fixed $x\in X$ and $y\in Y$.  Since the range of $D$ is $(0,1)$, for any fixed $x$,
 $$\varphi(x)=\max_{y\in Y}~  \mathbb{E}_{P_{1}}\Big[\log\big(D(y,\xi_1)\big)\Big] +  \mathbb{E}_{P_2}\Big[\log\Big( 1 - D\big(y,G(x,\xi_2)\big)\Big)\Big]$$
is real-valued.  The functions $D$ and $G$ are usually defined by deep neural networks (see Section \ref{Sec4} for a specific example).

Since the pioneering work \cite{N1928theorie} by Von Neumann in 1928, convex-concave min-max problems have been investigated extensively, based on the concept of saddle points (see e.g. \cite{CLO2014optimal,N2004prox,ZHWZ2020generalization,ZHZ2019lower} and the references therein).
In the recent years, driving by important applications, nonconvex-nonconcave min-max problems have attracted considerable attention  \cite{JNJ2020local,LLRY2018solving,LTH2019block,RLLY2021weakly}.
However, it is well-known that a nonconvex-nonconcave min-max problem may not have a saddle point.
How to properly define its local optimal points and optimality conditions has been of great concern. In \cite{ADLH2019local,DP2018limit,MRS2020gradient},  the concept of local saddle points was studied, but it is pointed out in \cite{JNJ2020local} that the concept of local saddle points is not suitable
for most applications of min-max optimization in machine learning.
A nonconvex-nonconcave min-max problem may not have a local saddle point (see Example \ref{Eg1} in this paper).
In \cite{JNJ2020local}, the authors argued that a local solution cannot be determined just based on the function value in an arbitrary small neighborhood of a given point. For that reason, they proposed the concept of local minimax points of unconstrained smooth nonconvex-nonconcave min-max problems and studied the first-order and second-order optimality conditions.

The main contributions of this paper can be summarized as follows.
\begin{itemize}

\item We define the first-order and second-order optimality conditions of local minimax points of constrained min-max problem (\ref{minmax}) by using directional derivatives. These conditions reduce to smooth min-max problems with Fr{\'e}chet derivatives. Our optimality conditions extend the work \cite{JNJ2020local} for unconstrained smooth min-max problems to constrained nonsmooth min-max problems. Moreover, we rigorously describe the relationship between saddle points, local saddle points, global minimax points, local minimax points and stationary points defined by these first-order and second-order optimality conditions. The relationship among these points is illustrated by interesting examples and summarized in Figure \ref{fig-3}.

\item We establish new mathematical optimization theory for the GAN model with both smooth and nonsmooth activation functions. In particular, we give new properties of global minimax points, local minimax points and stationary points of problem (\ref{GAN}) under some specific settings. Examples with the sample average approximation approach  show that our results are helpful and efficient for training GANs.
\end{itemize}

The reminder of the paper is organized as follows.
In section \ref{Sec2}, we give some notations and preliminaries. In section \ref{Sec3}, we study the first-order and second-order optimality conditions of nonsmooth and smooth min-max problems, respectively. In section \ref{Sec4}, we apply our results to GANs and use examples to show the effectiveness of our results. Finally, we make some concluding remarks in section \ref{Sec5}.

\section{Notations and preliminaries}\label{Sec2}

In this paper, $\mathbb{N}$ denotes the natural numbers. $\mathbb{R}_+^n$ denotes the nonnegative part of $\mathbb{R}^n$. $\norm{\cdot}$ denotes the Euclidean norm. $\cl(\Omega)$, $\mathrm{int} (\Omega)$ and $\bd (\Omega)$ denote the closure, the interior and the boundary of set $\Omega$, respectively.  $o(|t|)$ denotes the infinitesimal of a higher order than $|t|$ as $t\to 0$. $O(|t|)$ denotes the same order as $|t|$ as $t\to 0$.
$\mathbb{B}(x,r)$ denotes the closed ball centred at $x$ with radius $r>0$.

Let $\Omega\subseteq \mathbb{R}^n$ be a closed and convex set. The tangent cone \cite[Definition 6.1]{RW2009variational} to $\Omega$ at $x\in \Omega$, denoted by $\mathcal{T}_\Omega(x)$, is defined as
\begin{align*}
\mathcal{T}_\Omega(x) &= \left\{ w: \exists~ x^k\overset{\Omega}{\to} x, t^k\downarrow 0 ~ \text{such that}~ \lim_{ k\to \infty } \frac{x^k-x}{t^k} = w \right\}.
\end{align*}

The normal cone \cite[Definition 6.3]{RW2009variational} to $\Omega$ at $x\in \Omega$, denoted by $\mathcal{N}_\Omega(x)$, is
$$\mathcal{N}_\Omega(x):=\{y\in \mathbb{R}^n: \inp{y, \omega-x}\leq 0, \forall \omega \in \Omega \}.$$
It also knows from \cite[Proposition 6.5]{RW2009variational} that $\mathcal{N}_\Omega(x)=\{v: \inp{v,\omega}\leq 0,~\text{for}~ \forall \omega\in \mathcal{T}_\Omega(x) \}$.

\begin{definition}\label{Def1}
We say that $(\hat{x},\hat{y})\in X\times Y$ is a saddle point of problem \eqref{minmax}, if
\begin{equation}
\label{saddp}
f(\hat{x},y) \leq f(\hat{x},\hat{y}) \leq  f(x,\hat{y})
\end{equation}
holds for any $(x,y)\in X\times Y$.
\end{definition}

\begin{definition}\label{Def2}
We say that $(\hat{x},\hat{y})\in X\times Y$ is a local saddle point of problem \eqref{minmax}, if there exists a $\delta>0$ such that, for any $(x,y)\in X\times Y$ satisfying $\norm{x-\hat{x}}\leq \delta$ and $\norm{y-\hat{y}}\leq \delta$, \eqref{saddp} holds.
\end{definition}

In the convex-concave setting, saddle points are usually used to describe the optimality of min-max problems.  However, one significant drawback of considering (local) saddle points of nonconvex-nonconcave problems is that such points might not exist \cite[Proposition 6]{JNJ2020local}. Also, (local) saddle points correspond to simultaneous game, but many applications (such as GANs and adversarial training) correspond to sequential games. In view of this, we consider in what follows global and local minimax points proposed in \cite{JNJ2020local}, which are from the viewpoint of sequential games.

\begin{definition}
\label{Def3}
We say that $(\hat{x},\hat{y})\in X\times Y$ is a global minimax point of problem \eqref{minmax}, if
$$f(\hat{x},y) \leq f(\hat{x},\hat{y}) \leq \max_{y'\in Y}f(x,y')$$
holds for any $(x,y)\in X\times Y$.
\end{definition}

\begin{definition}
\label{Def4}
We say that $(\hat{x},\hat{y})\in X\times Y$ is a local minimax point of problem \eqref{minmax}, if there exist a $\delta_0>0$ and a function $\tau:\mathbb{R}_+\rightarrow \mathbb{R}_+$ satisfying $\tau(\delta)\rightarrow 0$ as $\delta\rightarrow 0$, such that for any $\delta\in (0,\delta_0]$ and any $(x,y)\in X\times Y$ satisfying $\norm{x-\hat{x}}\leq \delta$ and $\norm{y-\hat{y}}\leq \delta$, we have
$$f(\hat{x},y) \leq f(\hat{x},\hat{y}) \leq \max_{y'\in\{y\in Y: \norm{y-\hat{y}}\leq \tau(\delta)\}}f(x,y').$$
\end{definition}

\begin{remark}
It is noteworthy that the function $\tau$ in Definition \ref{Def4} can be further restricted to be monotone or continuous without changing Definition \ref{Def4} \cite[Remark 15]{JNJ2020local}. Hereafter, we always assume that $\tau$ is monotone and continuous.

Global or local minimax points are motivated by many practical applications and the probable nonconvexity-nonconcavity of the min-max problem. Obviously, a saddle point is a global minimax point and a local saddle point is a local minimax point. However, problem (\ref{minmax}) may not have a local saddle point.    The following proposition gives some sufficient conditions for the existence of global (local) minimax points.
Note that the existence of a global (local) minimax point does not imply the existence of a local saddle point.
\end{remark}

\begin{proposition}\label{Prop1}
\begin{enumerate}
\item[(i)] If $\Phi_u:=\{x\in X: \varphi(x) \leq u\}$ is nonempty and bounded for some scalar $u$ and $\{y\in Y: f(x,y) \geq l_x\}$ is bounded for every $x\in \Phi_u$ and some scalar $l_x$, then problem \eqref{minmax} has at least a global minimax point.

\item[(ii)] (\cite[Lemma 16]{JNJ2020local}) $(x^*,y^*)\in X\times Y$ is a local minimax point if and only if $y^*$ is a local maximizer of $f(x^*,\cdot)$ and there exists a $\delta_0>0$ such that $x^*$ is a local minimizer of $\varphi_\delta(x):=\max_{y'\in\{y\in Y:\norm{y-y^*}\leq \delta\}} f(x,y')$ for any $\delta\in(0,\delta_0]$.
\end{enumerate}
\end{proposition}

\begin{proof}
(i) According to the continuity of $f(x,y)$, $\varphi$ is lower semicontinuous. We know from \cite[Theorem 1.9]{RW2009variational} that $\argmin_{x\in X} \varphi(x)\subseteq \Phi_u$ is nonempty and compact. Let $x^*\in \argmin_{x\in X} \varphi(x)$ and consider the set $\argmax_{y\in Y} f(x^*,y)$. Since $\{y\in Y: f(x^*,y) \geq l_{x^*}\}$ is bounded, we know from the continuity of $f(x^*,\cdot)$ that the maximum can be achieved. Let $y^*\in \argmax_{y\in Y} f(x^*,y)$. It is easy to check that $(x^*,y^*)$ is a global minimax point.
\end{proof}

Specifically, if both $X$ and $Y$ are bounded, then all conditions in (i) of Proposition \ref{Prop1} hold. Thus problem \eqref{minmax} has a global minimax point. However, a local minimax point may not exist even $X$ and $Y$ are bounded (see Example \ref{Eg2}). Also, a global minimax point may not be a local minimax point (see Example \ref{Eg2}). The following example tells that the global and local minimax points exist but (local) saddle points do not.

\begin{example}[{\cite[Figure 1]{JNJ2020local}}]\label{Eg1}
Let $n=m=1$ and $X = Y = [-1,1]$. Consider $f(x,y)=-x^2 + 5xy -y^2$. Note that
\begin{equation*}
\varphi(x)= \max_{y\in[-1,1]} (-x^2+5xy-y^2) =
\begin{cases}
-x^2 - 5x -1 , & x\in \left[-1, -\frac{2}{5}\right];\\
\frac{21}{4}x^2, & x\in \left[-\frac{2}{5}, \frac{2}{5}\right];\\
-x^2 + 5x - 1, & x\in \left[\frac{2}{5}, 1\right].\\
\end{cases}
\end{equation*}
It is not difficult to examine that $\min_{x\in [-1,1]} \varphi(x) = 0$ when $x=0$. In this case, $y=0$. Therefore, $(0,0)$ is a global minimax point. Moreover, let $\delta_0=\frac{2}{5}$ and $\tau(\delta)=\frac{5}{2}\delta$ in Definition \ref{Def4}. Then for any $\delta\leq \delta_0$, $(x,y)\in [-1,1]\times [-1,1]$ satisfying $\abs{x}\leq \delta$ and $\abs{y}\leq \delta$, we have
$$\max_{y'\in\{y\in Y: \abs{y}\leq \frac{5}{2}\delta\}}f(x,y') = \frac{21}{4}x^2$$
when $y= \frac{5}{2}x$. Thus, we obtain
$$-y^2=f(0,y) \leq f(0,0)=0 \leq \max_{y'\in\{y\in Y: \abs{y}\leq \frac{5}{2}\delta\}}f(x,y') = \frac{21}{4}x^2,$$
which implies that $(0,0)$ is also a local minimax point.

Note that the solutions of $\max_{y\in[-\delta,\delta]}\min_{x\in[-\delta,\delta]} f(x,y)$ are $(\delta,0)$ and $(-\delta,0)$ for any $\delta\in(0,1]$. Thus, we have
\begin{equation}
\label{gs5}
\max_{y\in[-\delta,\delta]}\min_{x\in[-\delta,\delta]} f(x,y) = -\delta^2 \neq 0 = \min_{x\in[-\delta,\delta]}\max_{y\in[-\delta,\delta]} f(x,y),
\end{equation}
which implies that $(0,0)$ is neither a saddle point (i.e., \eqref{gs5} holds with $\delta=1$) nor a local saddle point (i.e., \eqref{gs5} holds with sufficiently small $\delta$).

This example tells us (local) saddle points are inadequate to characterize the optimality of nonconvex-nonconcave min-max problems. Global and local minimax points defined in Definitions \ref{Def3} and \ref{Def4} respectively are good supplements of (local) saddle points.
\end{example}

\section{Optimality and stationarity}\label{Sec3}

In this section, we first discuss the first-order and second-order  optimality conditions when $f$ in \eqref{minmax} is nonsmooth. The smooth case is considered as a special case of the nonsmooth ones when the directional derivatives can be represented by Fr\'echet derivatives. The results extend  the study of necessary optimality conditions of min-max problems in \cite{JNJ2020local}, where the unconstrained smooth min-max problems are addressed. We illustrate these theoretical results by three examples.

To proceed further, we give the description of tangents to convex sets.

\begin{lemma}[{\cite[Theorem 6.9]{RW2009variational}}]\label{Lem3}
If $\Omega\subseteq \mathbb{R}^n$ is convex and $\bar{x}\in \Omega$, then
\begin{align*}
\mathcal{T}_{\Omega}(\bar{x})=\cl\{w: \exists \lambda >0~\text{with}~ \bar{x} + \lambda w \in \Omega\}, \,
\mathrm{int}\left(\mathcal{T}_{\Omega}(\bar{x}) \right)=\{w: \exists \lambda >0~\text{with}~ \bar{x} + \lambda w \in \mathrm{int}(\Omega)\}.
\end{align*}
\end{lemma}

Denote
$$\mathcal{T}_{\Omega}^\circ(\bar{x}):=\{w: \exists \lambda >0~\text{with}~ \bar{x} + \lambda w \in \Omega \}.$$
 It is not difficult to verify that $\mathcal{T}_{\Omega}(\bar{x})$, $\mathrm{int}(\mathcal{T}_{\Omega}(\bar{x}))$ and $\mathcal{T}_{\Omega}^\circ(\bar{x})$ are convex cones if $\Omega$ is convex. Moreover, we have the following relationship
\begin{equation*}
\mathrm{int}\left(\mathcal{T}_{\Omega}(\bar{x}) \right) \subseteq \mathcal{T}_{\Omega}^\circ(\bar{x}) \subseteq \mathcal{T}_{\Omega}(\bar{x}).
\end{equation*}
If $\Omega$ is polyhedral, then $\mathcal{T}_{\Omega}^\circ(\bar{x}) = \mathcal{T}_{\Omega}(\bar{x})$.

\subsection{Nonsmooth case}

In this subsection, we give some necessary optimality conditions of local minimax points of problem \eqref{minmax} for the case that $f$ is not differentiable, which lead to definitions of the first-order and second-order d-stationary points. For this purpose, we introduce some definitions for nonsmooth analysis.

Let $g:\mathbb{R}^n \to \mathbb{R}$. The (first-order) \emph{subderivative} $\d g(x)(v)$ at $x\in \mathbb{R}^n$ for $v\in\mathbb{R}^n$ is defined as \cite[Definition 8.1]{RW2009variational}
$$\d g(x)(v) :=\liminf_{v'\to v, t\downarrow 0} \frac{g(x+ t v') - g(x)}{t}.$$
The function $g$ is \emph{semidifferentiable} at $x$ for $v$ \cite[Definition 7.20]{RW2009variational} if the (possibly infinite) limit
$$\lim_{v'\to v, t\downarrow 0} \frac{g(x+ t v') - g(x)}{t}$$
exists. Further, if the above limit exists for every $v\in \mathbb{R}^n$, we say that $g$ is semidifferentiable at $x$. It is easy to see that if $g$ is Lipschitz continuous in a neighborhood of $x$, then this limit is finite.

There are two types of second-order subderivatives \cite[Definition 13.3]{RW2009variational}. The second-order subderivative at $x\in \mathbb{R}^n$ for $w$ and $v$ is
$$\d^2g(x|v)(w):=\liminf_{w'\to w, t\downarrow 0} \frac{g(x+ t w') - g(x) - t \inp{v,w'}}{\frac{1}{2}t^2}.$$
The second-order subderivative at $x\in \mathbb{R}^n$ for $w$ (without mention of $v$) is
$$\d^2g(x)(w):=\liminf_{w'\to w, t\downarrow 0} \frac{g(x+ t w') - g(x) - t \d g(x)(w')}{\frac{1}{2}t^2}.$$

We say that $g$ is \emph{twice semidifferentiable} at $x$ if it is semidifferentiable at $x$ and the (possibly infinite) limit
$$\lim_{w'\to w, t\downarrow 0} \frac{g(x+ t w') - g(x) - t \d g(x)(w')}{\frac{1}{2}t^2}$$
exists for any $w\in \mathbb{R}^n$.

The one-side \emph{directional derivative} $g'(x;v)$ at $x\in \mathbb{R}^n$ along the direction $v\in\mathbb{R}^n$ is defined as
$$g'(x;v):=\lim_{t\downarrow 0} \frac{g(x+ t v) - g(x)}{t}.$$
The function $g$ is directionally differentiable at $x$ if $g'(x;v)$ exists for all directions $v\in\mathbb{R}^n$. If $g$ is locally Lipschitz continuous near $x$, then semidifferentiability at $x$ is equivalent to directional differentiability at $x$.

The \emph{second-order directional derivative} of $g$ at $x\in \mathbb{R}^n$ along the direction $v\in\mathbb{R}^n$  is defined as \cite[Chapter 13.B]{RW2009variational}
$$g^{(2)}(x;v):=\lim_{t\downarrow 0} \frac{g(x+ t v) - g(x) - t g'(x;v)}{\frac{1}{2}t^2}.$$

Obviously, if $g$ is semidifferentiable at $x$, then $\d g(x)(v)=g'(x;v)$; if $g$ is twice semidifferentiable at $x$, then $\d^2 g(x)(w)=g^{(2)}(x;w)$.

As a generalization of classical directional derivatives, \emph{the (Clarke) generalized directional derivative} of $g$ at $x\in \mathbb{R}^n$ along the direction $v \in \mathbb{R}^n$ is defined as \cite[Section 2.1]{C1990optimization}
$$g^{\comp}(x;v) := \limsup_{x' \to x \atop  t\downarrow 0} \frac{g(x'+ tv) - g(x') }{ t}.$$
We say that $g$ is \emph{Clarke regular} at $x$ \cite[Definition 2.3.4]{C1990optimization} if $g'(x;v)$ exists and $g^{\comp}(x;v)=g'(x;v)$ for all $v$. By using the generalized directional derivative, we can define the (Clarke)\emph{ generalized subdifferential} as
$$\partial g(x):= \{z\in \mathbb{R}^n: \inp{z,v} \leq g^{\comp}(x;v)~ \forall v\in \mathbb{R}^n\}.$$

The \emph{generalized second-order directional derivative} of  $g$ at $x\in \mathbb{R}^n$ along the direction $(u, v) \in \mathbb{R}^n\times \mathbb{R}^n$ is defined as \cite[Definition 1.1]{CC1990generalized} and \cite[Theorem 13.52]{RW2009variational}
$$g^{\comp\comp}(x;u,v) := \limsup_{x' \to x \atop  t\downarrow 0, \delta\downarrow 0} \frac{g(x'+ \delta u + t v) - g(x'+ \delta u) - g(x'+t v) + g(x')}{\delta t}.$$

Especially, when $u=v$, we write $g^{\comp\comp}(x;v,v)$ as $g^{\comp\comp}(x;v)$ for simplicity.

\begin{remark}
When $f$ is continuously differentiable at $(\hat{x},\hat{y})$,  $f_x^{\comp}(\hat{x},\hat{y};v) = \d_x f(\hat{x},\hat{y}) (v)= \nabla_x f(\hat{x},\hat{y})^\top v$ and $f_y^{\comp}(\hat{x},\hat{y};w) =\d_y f(\hat{x},\hat{y}) (w) = \nabla_y f(\hat{x},\hat{y})^\top w$ (\cite[Exercise 8.20]{RW2009variational}). Moreover, if $f$ is twice continuously differentiable at $(\hat{x},\hat{y})$, we know from \cite[Example 13.8, Proposition 13.56]{RW2009variational} that $f_x^{\comp\comp}(\hat{x},\hat{y};v) = \d_x^2 f(\hat{x},\hat{y}) (v)= v^\top \nabla_x^2 f(\hat{x},\hat{y}) v$ and $f_y^{\comp\comp}(\hat{x},\hat{y};w) = \d_y^2 f(\hat{x},\hat{y}) (w) = w^\top \nabla_y^2 f(\hat{x},\hat{y}) w$.

\end{remark}

\begin{example}\label{Eg3}
Consider a two-layer neural network with the ReLU activation function as follows:
$$F(W,b):=\rho(W_2(W_1\xi+b_1)_+ + b_2),$$
where $\xi\in \mathbb{R}^s$, $W_1\in \mathbb{R}^{s_1\times s}$, $b_1\in \mathbb{R}^{s_1}$, $W_2\in \mathbb{R}^{s_2\times s_1}$, $b_2\in \mathbb{R}^{s_2}$, $\rho:\mathbb{R}^{s_2} \to \mathbb{R}$
is a continuously differentiable function, $W=(W_1,W_2)$ and $b=(b_1,b_2)$. Obviously, $F$ is locally Lipschitz continuous. Consider
\begin{align*}
&F'(W,b;\overline{W},\bar{b})=\lim_{t\downarrow 0}\frac{F(W + t \overline{W},b+ t \bar{b}) - F(W,b)}{t} \\
&=\lim_{t\downarrow 0}\frac{\rho((W_2 + t\overline{W}_2)((W_1 + t\overline{W}_1)\xi+b_1+ t\overline{b}_1)_+ + b_2 + t\overline{b}_2) - \rho(W_2(W_1\xi+b_1)_+ + b_2)}{t}
\end{align*}
and
\begin{small}
\begin{align*}
&~~~\lim_{t\downarrow 0}\frac{(W_2 + t\overline{W}_2)((W_1 + t\overline{W}_1)\xi+b_1+ t\overline{b}_1)_+ + b_2 + t\overline{b}_2 - (W_2(W_1\xi+b_1)_+ + b_2)}{t} \\
&= \lim_{t\downarrow 0}\frac{W_2 \left( ((W_1 + t\overline{W}_1)\xi+ b_1+ t\overline{b}_1)_+ - (W_1\xi+b_1)_+\right) + t\left( \overline{W}_2((W_1 + t\overline{W}_1)\xi+ b_1+ t\overline{b}_1)_+ + \overline{b}_2 \right) }{t} \\
&= W_2 \left( \lim_{t\downarrow 0}\frac{ ((W_1 + t\overline{W}_1)\xi+ b_1+ t\overline{b}_1)_+ - (W_1\xi+b_1)_+ }{t} \right) +  \overline{W}_2(W_1\xi+ b_1)_+  + \overline{b}_2.
\end{align*}
\end{small}
Since for $i=1,\cdots,s_1$,
\begin{align*}
&\lim_{t\downarrow 0}\frac{ ((W_1^i + t\overline{W}_1^i)^\top \xi+ b_1^i+ t\overline{b}_1^i)_+ - ((W_1^i)^\top\xi+b_1^i)_+ }{t} \\
&=
\begin{cases}
(\overline{W}_1^i)^\top \xi + \bar{b}_1^i & \text{if}~ (W_1^i)^\top \xi+ b_1^i>0 ~\text{or}~ (W_1^i)^\top \xi+ b_1^i=0~\text{and}~ (\overline{W}_1^i)^\top \xi + \bar{b}_1^i>0; \\
0 &   \text{if}~ (W_1^i)^\top \xi+ b_1^i< 0 ~\text{or}~ (W_1^i)^\top \xi+ b_1^i=0~\text{and}~ (\overline{W}_1^i)^\top \xi + \bar{b}_1^i\leq 0,
\end{cases}
\end{align*}
where $\overline{W}_1^i$ and $W_1^i$ are the $i$th row vectors of $\overline{W}_1$ and $W_1$ respectively and  $\bar{b}_1^i$ and $b_1^i$ are the $i$th components of $\bar{b}_1$ and $b_1$ respectively, we have that the following limit
$$\Upsilon:=W_2 \left( \lim_{t\downarrow 0}\frac{ ((W_1 + t\overline{W}_1)\xi+ b_1+ t\overline{b}_1)_+ - (W_1\xi+b_1)_+ }{t} \right) +  \overline{W}_2(W_1\xi+ b_1)_+  + \overline{b}_2$$
exists. Therefore, we have that $F$ is semidifferentiable based on the locally Lipschitz continuity.

If, moreover, $\rho$ is twice continuously differentiable, we have
\begin{align*}
\d^2F(W,b)(\overline{W},\bar{b})&=\liminf_{t\downarrow 0 \atop \overline{W}'\to \overline{W},\bar{b}' \to \bar{b}}\frac{F(W + t \overline{W}',b+ t \bar{b}') - F(W,b) - t\d F(W,b)(\overline{W}',\bar{b}')}{\frac{1}{2}t^2} \\
&= \Upsilon^\top  \nabla^2 \rho( W_2(W_1\xi+b_1)_+ + b_2 )  \Upsilon,
\end{align*}
which implies that $F$ is twice semidifferentiable.

\end{example}

The following lemma tells the necessary optimality conditions for an unconstrained minimization problem by using subderivatives.

\begin{lemma}[{\cite[Theorem 10.1 \& Theorem 13.24]{RW2009variational}}]\label{Lem5}
Let $g:\mathbb{R}^n \to (-\infty,+\infty]$ be a proper extended-valued function. If $\bar{x}$ is a local minimizer of $g$ over $\mathbb{R}^n$, then $\d g(\bar{x})(v)\geq 0$ and $\d^2g(\bar{x}|0)(v)\geq 0$ for any $v\in \mathbb{R}^n$.
\end{lemma}

The following lemma shows that we can replace $\d^2g(\bar{x}|0)(v)\geq 0$ by $\d^2g(\bar{x})(v)\geq 0$ under certain mild conditions.

\begin{lemma}\label{Lem6}

Let $g:\mathbb{R}^n \to (-\infty,+\infty]$ be twice semidifferentiable at $\bar{x}$. If $\d g(\bar{x})(v) = 0$, then $\d^2g(\bar{x}|0)(v)=\d^2g(\bar{x})(v)$.
\end{lemma}

\begin{proof}
Let $\d g(\bar{x})(v) = 0$. Note that
\begin{align*}
\d^2g(\bar{x})(v)&=\liminf_{v'\to v, t\downarrow 0} \frac{g(\bar{x}+ t v') - g(\bar{x}) - t \d g(\bar{x})(v')}{\frac{1}{2}t^2}\\
&=\lim_{v'\to v, t\downarrow 0} \frac{g(\bar{x}+ t v') - g(\bar{x}) - t \d g(\bar{x})(v')}{\frac{1}{2}t^2}\\
&=\lim_{t\downarrow 0} \frac{g(\bar{x}+ t v) - g(\bar{x}) - t \d g(\bar{x})(v)}{\frac{1}{2}t^2}\\
&=\lim_{t\downarrow 0} \frac{g(\bar{x}+ t v) - g(\bar{x})}{\frac{1}{2}t^2}=\d^2g(x|0)(v),
\end{align*}
where the second equality follows from the twice semidifferentiability of $g$ at $\bar{x}$ and the third equality follows from the existence of the limit.
\end{proof}

\begin{lemma}[{\cite[Theorem 8.2]{RW2009variational}}]\label{Lem7}
For the indicator function $\delta_\mathcal{X}$ of a set $\mathcal{X}\subseteq \mathbb{R}^n$ and any point $x\in \mathcal{X}$, one has
$\d \delta_\mathcal{X} (x)(v) = \delta_{\mathcal{T}_\mathcal{X}(x)}(v)$ for any $v\in \mathbb{R}^n$.
\end{lemma}

A function $g:\mathbb{R}^n\to \mathbb{R}$ is called \emph{positively homogeneous of degree $p > 0$} if $g(\lambda w) = \lambda^p g(w)$ for all $\lambda > 0$ and $w\in \mathbb{R}^n$ (see \cite[Definition 13.4]{RW2009variational}).

The following lemma shows the expansion of a function via subderivatives.

\begin{lemma}[{\cite[Theorem 7.21 \& Exercise 13.7]{RW2009variational}}]\label{Lem9}
Let $g:\mathbb{R}^n\to \mathbb{R}$. Then
\begin{enumerate}
\item[(i)] $g$ is semidifferentiable at $\bar{x}$ if and only if
$$g(x)=g(\bar{x}) + \d g(\bar{x})(x-\bar{x}) + o(\norm{x-\bar{x}}),$$
where $\d g(\bar{x})(\cdot)$ is a finite, continuous, positively homogeneous function.

\item[(ii)] Suppose that $g$ is semidifferentiable at $\bar{x}$. Then $g$ is twice semidifferentiable at $\bar{x}$ if and only if
$$g(x)=g(\bar{x}) + \d g(\bar{x})(x-\bar{x}) + \frac{1}{2}\d^2 g(\bar{x})(x-\bar{x}) +  o(\norm{x-\bar{x}}^2),$$
where $\d^2 g(\bar{x})(\cdot)$ is a finite, continuous, positively homogeneous of degree $2$ function.
\end{enumerate}
\end{lemma}

 The following lemma gives the first-order and second-order optimality conditions for minimizing a semidifferentiable function, which extends \cite[Proposition 2.3]{CCHP2020study} with a polyhedral set $\mathcal{X}$. For completeness, we give a simple proof.
\begin{lemma}\label{Prop7}
Let $\mathcal{X}\subseteq \mathbb{R}^n$ be a closed and convex set, $g: \mathbb{R}^n\to \mathbb{R}$ be semidifferentiable at $\bar{x}\in \mathcal{X}$, and $\bar{x}$ be a local minimizer of $g$ over $\mathcal{X}$. Then
 $\d g(\bar{x})(v)\geq 0$ for all $v\in \mathcal{T}_{\mathcal{X}}(\bar{x})$.
Moreover, if $g$ is twice semidifferentiable at $\bar{x}$, then $\d^2g(\bar{x})(v)\geq 0$ for all $v\in  \mathcal{T}_{\mathcal{X}}^\circ(\bar{x}) \cap \{v: \d g(\bar{x})(v) =0\}$.

\end{lemma}

\begin{proof}
From Lemma \ref{Lem5} and Lemma \ref{Lem7}, we have
$$0\leq \d (g(\bar{x}) + \delta_{\mathcal{X}}(\bar{x}))(v)= \d g(\bar{x})(v) + \delta_{\mathcal{T}_{\mathcal{X}}(\bar{x})}(v) = \d g(\bar{x})(v).$$
Moreover, if $g$ is twice semidifferentiable at $\bar{x}$, for  $v\in  \mathcal{T}_{\mathcal{X}}^\circ(\bar{x}) \cap \{v: \d g(\bar{x})(v) =0\}$,
from Lemma \ref{Lem6}, we have
\begin{align*}
0&\leq
\liminf_{v'\to v, t\downarrow 0} \frac{g(\bar{x}+ t v') + \delta_{\mathcal{X}}(\bar{x}+ t v') - g(\bar{x}) - \delta_{\mathcal{X}}(\bar{x}) - t \d g(\bar{x})(v)}{\frac{1}{2} t^2} \\
&\leq \liminf_{t\downarrow 0} \frac{g(\bar{x}+ t v) + \delta_{\mathcal{X}}(\bar{x}+ t v) - g(\bar{x}) - \delta_{\mathcal{X}}(\bar{x}) - t \d g(\bar{x})(v)}{\frac{1}{2} t^2} \\
&= \lim_{t\downarrow 0} \frac{g(\bar{x}+ t v) - g(\bar{x}) - t \d g(\bar{x})(v) }{\frac{1}{2} t^2}\\
&= \d^2 g(\bar{x})(v).
\end{align*}
\end{proof}

The following lemma gives a description of the generalized second-order directional derivative by using directional
derivatives.

\begin{lemma}[{\cite[Proposition 1.3]{CC1990generalized}}]\label{Lem10}
Let $g:\mathbb{R}^n\to \mathbb{R}$ be a continuous function that admits a directional
derivative at every point near $x$. Then $g^{\comp\comp}(x;u,v)$ is the generalized directional derivative of $g'(\cdot,v)$ at $x$ along direction $u$, that is
$$g^{\comp\comp}(x;u,v) = \limsup_{x' \to x \atop  t\downarrow 0} \frac{g'(x'+ t u;v) - g'(x';v) }{ t}.$$
\end{lemma}

\begin{remark}\label{Rem1}
Note that
\begin{align*}
g^{\comp\comp}(x;v) \geq  \lim_{ t\downarrow 0} \frac{g(x+t v + t v) - g(x+ t v) - g(x+t v) + g(x)}{t^2} = g^{(2)}(x;v).
\end{align*}

Recall that $g:\mathbb{R}^n\to \mathbb{R}$ is \emph{twice subregular} at $x$ \cite[Definition 3.1]{CC1990generalized} if the limit
$$\lim_{ t\downarrow 0, \delta\downarrow 0} \frac{g(x+ \delta u + t v) - g(x+ \delta u) - g(x+t v) + g(x)}{\delta t}$$
exists and the above limit equals to $g^{\comp\comp}(x;u,v)$. Thus, we know that $g^{\comp\comp}(x;v) = g^{(2)}(x;v)$ if $g$ is twice subregular at $x$.
\end{remark}

Now we are ready to give the main results of this subsection.

\begin{theorem}\label{Th5}
Let the tuple $(\hat{x},\hat{y})\in X\times Y$ be a local minimax point of problem \eqref{minmax}.

\begin{enumerate}
\item[(i)] If $f$ is semidifferentiable at $(\hat{x},\hat{y})$, then
\begin{subequations}
\label{NonS1st}
\begin{align}
&f_x^{\comp}(\hat{x},\hat{y};v) \geq 0~\text{for all}~v\in \mathcal{T}_{X}(\hat{x}),\label{NonS1st-1}\\
&\d_y f(\hat{x},\hat{y}) (w) \leq 0~\text{for all}~w\in \mathcal{T}_{Y}(\hat{y}),\label{NonS1st-2}
\end{align}
\end{subequations}
where $f_x^{\comp}(\hat{x}, \hat{y};v)$ denotes the generalized directional derivative of $f$ with respect to $x$ at $\hat{x}$ along the direction $v$ for fixed $\hat{y}$.

\item[(ii)] Assume, further, that $f$ is twice semidifferentiable at $(\hat{x},\hat{y})$ and $f$ is Clarke regular in a neighborhood of $(\hat{x},\hat{y})$. Then
    \end{enumerate}
\begin{subequations}
\label{NonS2ed}
\begin{align}
&f_x^{\comp\comp}(\hat{x},\hat{y};v) \geq 0~\text{for all}~v\in \mathcal{T}_{X}^\circ(\hat{x}) \cap \{v: \exists \delta>0, \d_x f(\hat{x},y') (v) =0,\forall y' \in \mathbb{B}(\hat{y},\delta)\cap Y\}, \label{NonS2ed-1}\\
&\d_y^2 f(\hat{x},\hat{y}) (w) \leq 0~\text{for all}~w\in \mathcal{T}_{Y}^\circ(\hat{y}) \cap \{w: \d_y f(\hat{x},\hat{y}) (w) =0 \},\label{NonS2ed-2}
\end{align}
\end{subequations}
where $f_x^{\comp\comp} (\hat{x},\hat{y}; v)$ denotes the generalized second-order directional derivative of $f$ with respect to $x$ at $\hat{x}$ along the direction $(v,v)$ for fixed $\hat{y}$.

\end{theorem}

\begin{proof}
\eqref{NonS1st-2} and \eqref{NonS2ed-2} directly follow from Proposition \ref{Prop7}. Therefore, we only focus on  \eqref{NonS1st-1} and \eqref{NonS2ed-1}, respectively.

(i) Since $(\hat{x},\hat{y})$ is a local minimax point, there exist a $\delta_0>0$ and a function $\tau:\mathbb{R}_+\rightarrow \mathbb{R}_+$ satisfying $\tau(\delta)\rightarrow 0$ as $\delta\rightarrow 0$, such that for any $\delta\in (0,\delta_0]$ and $(x,y)\in X\times Y$ satisfying $\norm{x-\hat{x}}\leq \delta$ and $\norm{y-\hat{y}}\leq \delta$, we have
\begin{equation}\label{stationary1}
f(\hat{x},y) \leq f(\hat{x},\hat{y}) \leq \max_{y'\in\{y\in Y: \norm{y-\hat{y}}\leq \tau(\delta)\}}f(x,y').
\end{equation}

For any $v\in\mathcal{T}_X(\hat{x})$, according to the convexity of $X$, there exist $\{v^k\}_{k\geq 1}$ with $v^k\to v$ as $k\to \infty$ and $\{t_k\}_{k\geq 1}$ with $t_k \downarrow 0$ as $k\to \infty$, such that $x^k:=\hat{x}+ t_kv^k \in X$ (see Lemma \ref{Lem3}). Let $\delta_k=\norm{x^k -\hat{x}}$ and $\tilde{y}^k$ be defined by
\begin{equation}
\label{gs10}
\tilde{y}^k\in \argmax_{y' \in \{y\in Y:  \norm{y-\hat{y}}\leq \tau(\delta_k)\}} f(x^k,y').
\end{equation}
Obviously, $\delta_k\to 0$ and $\norm{\tilde{y}^k \to \hat{y}}\to 0$ as $k\to \infty$. According to the second inequality of (\ref{stationary1}), we have (for sufficiently large $k$) that
\begin{equation}
\label{gs18}
\begin{aligned}
0 \leq f(x^k, \tilde{y}^k) - f(\hat{x}, \hat{y}) &= f(x^k, \tilde{y}^k) - f(\hat{x}, \tilde{y}^k) +  f(\hat{x}, \tilde{y}^k) - f(\hat{x}, \hat{y}) \\
&\leq f(x^k, \tilde{y}^k) - f(\hat{x}, \tilde{y}^k) \\
& \in \inp{\partial f(\tilde{x}^k, \tilde{y}^k), \begin{pmatrix}t_k v^k\\ 0 \end{pmatrix}},
\end{aligned}
\end{equation}
where the last inclusion follows from the mean-value theorem \cite[Theorem 2.3.7]{C1990optimization} and $\tilde{x}^k$ lies in the segment between $x^k$ and $\hat{x}$. Thus, by dividing $t_k$ in both sides and letting $k\to \infty$, due to the upper semicontinuity of $\partial f(\cdot,\cdot)$ (see \cite[Proposition 2.1.5]{C1990optimization}), we obtain
\begin{align*}
0 &\leq \sup_{\zeta\in \partial f(\hat{x}, \hat{y})} \inp{\zeta,\begin{pmatrix}v\\ 0 \end{pmatrix}}
= f^{\comp} (\hat{x}, \hat{y};v,0)
= f_x^{\comp}(\hat{x}, \hat{y};v),
\end{align*}
where $f_x^{\comp}(\hat{x}, \hat{y};v)$ denotes the Clarke generalized directional derivative of $f$ with respect to $x$ at $\hat{x}$  along the direction $v$ for fixed $\hat{y}$.

(ii) Let $v \in \mathcal{T}_{X}^\circ(\hat{x}) \cap \{v: \exists \delta>0, \d_x f(\hat{x},y') (v) =0,\forall y' \in \mathbb{B}(\hat{y},\delta)\cap Y\}$. Then there exists a sequence $\{t_k\}_{k\geq 1}$ with $t_k \downarrow 0$, such that $x^k:=\hat{x} + t_kv \in X$. Let $\delta_k=\norm{x^k-\hat{x}}$, and $\tilde{y}^k$ be defined in \eqref{gs10}.

From mean-value theorem, there is $\zeta_k\in(0,t_k)$ such that
\begin{equation}
\label{gs12}
f(\hat{x}+t_k v,\tilde{y}^k) - f(\hat{x}, \tilde{y}^k) \in   \partial f(\hat{x}+ \zeta_k v,\tilde{y}^k) \begin{pmatrix}v\\0 \end{pmatrix}.
\end{equation}
Similar to \eqref{gs18}, we have $f(\hat{x} + t_k v, \tilde{y}^k) - f(\hat{x}, \tilde{y}^k) \geq 0$. Then, according to the Clarke regularity of $f$ near $(\hat{x},\hat{y})$ and the definition of generalized directional derivative, we have from \eqref{gs12} that
\begin{align*}
0 &\leq \limsup_{k\to \infty} \frac{ f'(\hat{x}+ \zeta_k v,\tilde{y}^k;v,0) }{\zeta_k} = \limsup_{k\to \infty} \frac{ f'(\hat{x}+ \zeta_k v,\tilde{y}^k;v,0) - f'(\hat{x}, \tilde{y}^k;v,0) }{\zeta_k}\\
& \leq f^{\comp\comp} (\hat{x},\hat{y}; v,0) = f_x^{\comp\comp} (\hat{x},\hat{y}; v),
\end{align*}
where the first equality follows from $f'(\hat{x}, \tilde{y}^k;v,0)=0$ for sufficiently large $k$, the second inequality follows from Lemma \ref{Lem10} and $f_x^{\comp\comp} (\hat{x},\hat{y}; v)$ denotes the generalized second-order directional derivative of $f$ with respect to $x$  at $\hat{x}$ along the direction $(v,v)$ for fixed $\hat{y}$.
\end{proof}

\begin{remark}\label{Rem2}
We know from \cite[Page 10]{C1990optimization}  that, for any $v$,
$f_x^{\comp}(\hat{x},\hat{y};v)=\max_{z\in \partial_x f(\hat{x},\hat{y})} \inp{z,v}$.
Thus, \eqref{NonS1st-1} can be equivalently reformulated as
$\max_{z\in \partial_x f(\hat{x},\hat{y})} \inp{z,v} \geq 0,~\forall v\in \mathcal{T}_{X}(\hat{x})$,
which, based on the definition of normal cone, is equivalent to $0\in \partial_x f(\hat{x},\hat{y}) + \mathcal{N}_X(\hat{x})$.

Generally, \eqref{NonS1st-2} implies the Clarke stationary condition
$
0\in -\partial_y f(\hat{x},\hat{y}) + \mathcal{N}_Y(\hat{y}),
$
but not vice versa. Moreover, by using the (generalized) directional derivatives, we can establish the second-order necessary optimality conditions for the nonsmooth case. Therefore, the (generalized) directional derivatives are employed in Theorem \ref{Th5}.
\end{remark}

\begin{remark}
It is noteworthy that the necessary optimality conditions \eqref{NonS1st-1}-\eqref{NonS1st-2} and \eqref{NonS2ed-1}-\eqref{NonS2ed-2} with respect to $x$ and $y$ are not symmetric. Generally, \eqref{NonS1st-1} and \eqref{NonS2ed-1} are weaker than
\begin{equation}
\label{gs16}
\d_x f(\hat{x},\hat{y};v) \geq 0~\text{for all}~v\in \mathcal{T}_{X}(\hat{x})
\end{equation}
and
\begin{equation}
\label{gs17}
\d_x^2 f(\hat{x},\hat{y};v) \geq 0~\text{for all}~v\in \mathcal{T}_{X}^\circ(\hat{x}) \cap \{v: \d_x f(\hat{x},\hat{y}) (v) =0 \},
\end{equation}
respectively, because $f_x^{\comp}(\hat{x},\hat{y};v) \geq \d_x f(\hat{x},\hat{y};v)$, $f_x^{\comp\comp}(\hat{x},\hat{y};v)\geq \d_x^2 f(\hat{x},\hat{y};v)$ (Remark \ref{Rem1}) and
$$\mathcal{T}_{X}^\circ(\hat{x}) \cap \{v: \exists \delta>0, \d_x f(\hat{x},y') (v) =0,\forall y' \in \mathbb{B}(\hat{y},\delta)\cap Y\} \subseteq \mathcal{T}_{X}^\circ(\hat{x}) \cap \{v: \d_x f(\hat{x},\hat{y}) (v) =0 \}.$$
The main reason is that a local minimax point may not be a local saddle point. If we replace \eqref{NonS1st-1} and \eqref{NonS2ed-1} by \eqref{gs16} and \eqref{gs17} respectively, the necessary optimality conditions for local saddle points are derived.

If, in addition, $f$ is Clarke regular at $(\hat{x},\hat{y})$, \eqref{NonS1st-1} can be replaced by (\ref{gs16}).

If, in addition, $f$ is twice subregular at $(\hat{x},\hat{y})$, \eqref{NonS2ed-1} can be replaced by
$$\d_x^2 f(\hat{x},\hat{y}) (v) \geq 0~\text{for all}~v\in \mathcal{T}_{X}^\circ(\hat{x}) \cap \{v: \exists \delta>0, \d_x f(\hat{x},y') (v) =0,\forall y' \in \mathbb{B}(\hat{y},\delta)\cap Y\}.$$

\end{remark}

Based on Theorem \ref{Th5}, we define the first-order and second-order d-stationary points of min-max problems.

\begin{definition}\label{Def6}
We call that $(\hat{x},\hat{y})\in X\times Y$ is a first-order d-stationary point of problem \eqref{minmax} if it satisfies \eqref{NonS1st-1}-\eqref{NonS1st-2}. If $(\hat{x},\hat{y})$ also satisfies \eqref{NonS2ed-1}-\eqref{NonS2ed-2}, we call it a second-order d-stationary point of problem \eqref{minmax}.
\end{definition}

\begin{example}
Let $X=[-1,1]$ and $Y=[-1,1]$. We consider
$$\min_{x\in [-1,1]}\max_{y\in [-1,1]} f(x,y):=-\abs{x}^9 + \frac{3}{5}\abs{x}^3\abs{y}^3- \abs{y}^5.$$
Take $\tau(\delta)=\frac{3}{5}(\sqrt{\delta})^3$. Then for any $\abs{x}\leq \delta$ and $\abs{y}\leq \delta$ with sufficiently small $\delta\in(0,1)$ we have
\begin{align*}
-\abs{y}^5 = f(0,y) \leq f(0,0) &\leq  \max_{y\in [-\tau(\delta),\tau(\delta)]} -\abs{x}^9 + \frac{3}{5}\abs{x}^3\abs{y}^3- \abs{y}^5=-\abs{x}^9 + \frac{2}{5}\left( \frac{3}{5} \right)^4(\sqrt{\abs{x}})^{15}
\end{align*}
where $\pm\frac{3}{5}(\sqrt{\abs{x}})^3$ is the maximizer of the above maximization problem. This implies that $(0,0)$ is a local minimax point.
Obviously, $f(x,y)$ is not differentiable at $(0,0)$. In what follows, we examine the necessary optimality conditions in Theorem \ref{Th5}. Since $\mathcal{T}_{X}(0) = \mathbb{R}$ and $\mathcal{T}_{Y}(0) = \mathbb{R}$, we have for any $v\in \mathcal{T}_{X}(0)$ that
\begin{align*}
f_x^{\comp}(0,0;v)&= \limsup_{x'\to 0, t\downarrow 0}\frac{-\abs{x'+tv}^9 + \abs{x'}^9}{t}= 0,
\end{align*}
which implies that $f_x^{\comp}(0,0;v)= f'_x(0,0;v)$, i.e., the Clarke regularity holds.

Similarly, we have for any $w\in \mathcal{T}_{Y}(0)$ that
\begin{align*}
\d_y f(0,0) (w) &=  \liminf_{w'\to w, t\downarrow 0} \frac{f(0,t w') - f(0,0)}{t} =\liminf_{w'\to w, t\downarrow 0} \frac{-\abs{tw'}^5}{t} = 0.
\end{align*}

Next consider the second-order optimality conditions. Note that $\mathcal{T}_{X}^\circ(0) =\mathbb{R}$ and for any fixed $y'$, we have
\begin{align*}
\d_x f(0,y') (v) &= \liminf_{v'\to v, t\downarrow 0} \frac{f(t v', y') - f(0,y')}{t} \\
&= \liminf_{v'\to v, t\downarrow 0} \frac{-t^9\abs{ v'}^9 + \frac{3}{5}t^3\abs{v'}^3\abs{y'}^3 - \abs{y'}^5 + \abs{y'}^5 }{t} \\
&=0
\end{align*}
for any $v$, which implies that $\{v: \d_x f(0,y') (v) =0 \} = \mathbb{R}$. Thus, for any $\delta>0$
$$\mathcal{T}_{X}^\circ(0) \cap \{v: \d_x f(0,y') (v) =0,\forall y' \in \mathbb{B}(0,\delta)\cap Y\} = \mathbb{R}.$$
Notice that
\begin{align*}
f_x^{\comp\comp}(0,0;v) &= \limsup_{x' \to 0 \atop  t\downarrow 0, \delta\downarrow 0} \frac{f(x'+ \delta v + t v,0) - f(x'+ \delta v,0) - f(x'+t v,0) + f(x',0)}{\delta t} \\
&= \limsup_{x' \to 0 \atop  t\downarrow 0, \delta\downarrow 0} \frac{-\abs{x'+ \delta v + t v}^9 + \abs{x'+ \delta v}^9 + \abs{x'+t v}^9 - \abs{x'}^9 }{\delta t} \geq 0
\end{align*}
for any $v\in \mathbb{R}$. Similarly, we have $\mathcal{T}_{Y}^\circ(0) \cap \{w: \d_y f(0,0) (w) =0 \} =\mathbb{R}$ and
\begin{align*}
\d_y^2 f(0,0) (w) &= \liminf_{w'\to w, t\downarrow 0} \frac{f(0, t w') - f(0,0) - t \d_y f(0,0)(w')}{\frac{1}{2}t^2}
= \liminf_{w'\to w, t\downarrow 0} \frac{-\abs{t w'}^5}{\frac{1}{2}t^2} = 0
\end{align*}
for any $w\in \mathbb{R}$.
\end{example}

\subsection{Smooth case}
In this subsection, we consider the necessary optimality conditions of problem \eqref{minmax} when $f$ is (twice) continuously differentiable.
For any $(x,y)\in X\times Y$, denote
\begin{align*}
\Gamma_1^\circ(x,y)&=\{v\in \mathcal{T}_X^\circ(x): v\bot \nabla_x f(x,y)\}, \quad \,
\Gamma_1(x,y)=\{v\in \mathcal{T}_X(x): v\bot \nabla_x f(x,y)\},\\
\Gamma_2^\circ(x,y)&=\{w\in \mathcal{T}_Y^\circ(y): w\bot \nabla_y f(x,y)\}, \quad
\Gamma_2(x,y)=\{w\in \mathcal{T}_Y(y): w\bot \nabla_y f(x,y)\}.
\end{align*}

It is noteworthy that $\cl( \Gamma_1^\circ(x,y) ) \neq \Gamma_1(x,y)$ and $\cl( \Gamma_2^\circ(x,y) ) \neq \Gamma_2(x,y)$ generally even if we have $\cl( \mathcal{T}_X^\circ(x) ) = \mathcal{T}_X(x)$ and $\cl( \mathcal{T}_Y^\circ(y) ) = \mathcal{T}_Y(y)$. We summarize their relationships as follows.

\begin{lemma}
\label{Lem1}
Let $(x,y)\in X\times Y$. Then $\Gamma_1^\circ(x,y)$, $\Gamma_1(x,y)$, $\Gamma_2^\circ(x,y)$ and $\Gamma_2(x,y)$ are convex cones, and we have
$$\cl\Gamma_1^\circ(x,y) \subseteq \Gamma_1(x,y)~\text{and}~\cl\Gamma_2^\circ(x,y) \subseteq \Gamma_2(x,y).$$

Moreover, if $X$ and $Y$ are polyhedral, then $\Gamma_1^\circ(x,y)= \cl\Gamma_1^\circ(x,y) = \Gamma_1(x,y)$ and $\Gamma_2^\circ(x,y) = \cl\Gamma_2^\circ(x,y) = \Gamma_2(x,y)$.
\end{lemma}

The proof is based on Lemma \ref{Lem3}, and thus we omit it here.

\begin{theorem}\label{Th1}
Let $f$ be continuously differentiable and the tuple $(\hat{x},\hat{y})\in X\times Y$ be a local minimax point of problem \eqref{minmax}.
\begin{enumerate}
\item[(i)] Then it holds that
\begin{subequations}
\label{gs2}
\begin{align}
0&\in \nabla_x f(\hat{x},\hat{y}) + \mathcal{N}_{X}(\hat{x}),\label{gs2-1}\\
0&\in - \nabla_y f(\hat{x},\hat{y}) + \mathcal{N}_{Y}(\hat{y}).\label{gs2-2}
\end{align}
\end{subequations}

\item[(ii)]  Assume, further, that $f$ is twice continuously differentiable. Then
\begin{subequations}
\label{gs6}
\begin{align}
&\inp{v, \nabla_{xx}^2 f(\hat{x},\hat{y}) v} \geq 0~\text{for all}~v\in \cl \left\{\bar{v}: \exists \delta>0, \bar{v} \in \Gamma_1^\circ(\hat{x},y'), \forall y'\in \mathbb{B}(\hat{y},\delta) \right\},\label{gs6-1}\\
&\inp{w, \nabla_{yy}^2 f(\hat{x},\hat{y}) w} \leq 0~\text{for all}~w\in \cl\Gamma_2^\circ(\hat{x},\hat{y}).\label{gs6-2}
\end{align}
\end{subequations}
\end{enumerate}
\end{theorem}

\begin{proof}
(i) The proof is similar to Theorem \ref{Th5}. Here we give a simple proof of \eqref{gs2-1} and \eqref{gs6-1} for completeness. For any $x^k\overset{X}{\to}\hat{x}$ as $k\to \infty$, denote $\delta_k=\norm{x^k-\hat{x}}$ and $\tilde{y}^k$ is defined in \eqref{gs10}.
Obviously, $\delta_k\to 0$ and $\norm{\tilde{y}^k \to \hat{y}}\to 0$ as $k\to \infty$. From the continuous differentiability of $f$, we have
\begin{align*}
0 \leq f(x^k, \tilde{y}^k) - f(\hat{x}, \tilde{y}^k)
= \nabla f(\bar{x}^k, \tilde{y}^k)^\top \begin{pmatrix} x^k-\hat{x}\\ \tilde{y}^k-\tilde{y}^k \end{pmatrix}
= \nabla_x f(\hat{x}, \hat{y})^\top (x^k-\hat{x})   + o\left(\norm{x^k-\hat{x}}\right),
\end{align*}
where $\bar{x}^k$ is some point lying in the segment between $\hat{x}$ and $x^k$.
Thus, we obtain
$$-\nabla_x f(\hat{x}, \hat{y})^\top (x^k-\hat{x}) \leq  o\left(\norm{x^k-\hat{x}}\right).$$
We know from \cite[Definition 6.3]{RW2009variational} that $-\nabla_x f(\hat{x}, \hat{y}) \in \mathcal{N}_X(\hat{x})$,
which verifies \eqref{gs2-1}.

(ii) We only need to prove that \eqref{gs6-1} holds with $v\in \Gamma_1^\circ(\hat{x},y')$ for all $y'\in \mathbb{B}(\hat{y},\delta)$ and some $\delta>0$. According to the definition of $\mathcal{T}_X^\circ(\hat{x})$, there exists a sequence $\{t_k\}_{k\geq 1}$ with $t_k \downarrow 0$ as $k\to \infty$, such that $x^k:=\hat{x} + t_kv \in X$. Let $\delta_k=t_k\norm{v}$, and $\tilde{y}^k$ is denoted in \eqref{gs10}. Similarly, we have that
\begin{align*}
0 & \leq  f(x^k, \tilde{y}^k) - f(\hat{x}, \tilde{y}^k) \\
&\overset{(a)}{=}\nabla_x f(\hat{x}, \tilde{y}^k)^\top (x^k-\hat{x}) + \frac{1}{2} (x^k-\hat{x})^\top \nabla_{xx}^2 f(\tilde{x}^k, \tilde{y}^k)(x^k-\hat{x}) \\
&\overset{(b)}{=}\nabla_x f(\hat{x}, \tilde{y}^k)^\top (x^k-\hat{x}) + \frac{1}{2} (x^k-\hat{x})^\top \nabla_{xx}^2 f(\hat{x}, \hat{y})(x^k-\hat{x}) + o\left(\norm{x^k-\hat{x}}^2\right),
\end{align*}
where (a) follows from Taylor's theorem for multivariate functions with Lagrange's remainder, and $\tilde{x}^k$ is some point lying in the segment between $\hat{x}$ and $x^k$; (b) follows from the twice continuous differentiability of $f$ and $\tilde{x}^k \to \hat{x}$ as $k\to \infty$. Thus, we obtain
\begin{align*}
t_k \nabla_x f(\hat{x}, \tilde{y}^k)^\top v + t_k^2 \frac{1}{2} v^\top \nabla_{xx}^2 f(\hat{x}, \hat{y})v + \norm{v}^2o(t_k^2)\geq 0.
\end{align*}
Since  $\nabla_x f(\hat{x}, \tilde{y}^k)^\top v = 0$ for sufficiently large $k$,
dividing by $t_k^2$ in both sides and letting $k\to \infty$, we complete the proof.
\end{proof}

\begin{remark}
Conversely, if these conditions in (ii) of Theorem \ref{Th1} hold except that $\cl \left\{w: \exists \delta>0, w \in \Gamma_1^\circ(\hat{x},y'), \forall y'\in \mathbb{B}(\hat{y},\delta) \right\}$ and $\cl\Gamma_2^\circ(\hat{x},\hat{y})$ are replaced by $\Gamma_1(\hat{x},\hat{y})$ and $\Gamma_2(\hat{x},\hat{y})$, respectively, and the inequality is strict when $v\neq 0$ and $w\neq 0$, then $(\hat{x},\hat{y})$ is a local saddle point. In that case, \eqref{gs6} together with \eqref{gs2} are the so-called second-order sufficient condition for a local saddle point.

If both $X$ and $Y$ are polyhedral, We can replace
$$\cl \left\{w: \exists \delta>0, w \in \Gamma_1^\circ(\hat{x},y'), \forall y'\in \mathbb{B}(\hat{y},\delta) \right\} \quad {\rm and} \quad  \cl\Gamma_2^\circ(\hat{x},\hat{y})$$
in Theorem \ref{Th1} by $\cl \left\{w: \exists \delta>0, w \in \Gamma_1(\hat{x},y'), \forall y'\in \mathbb{B}(\hat{y},\delta) \right\}$ and $\Gamma_2(\hat{x},\hat{y})$, respectively (see Lemma \ref{Lem1}).
\end{remark}

\begin{corollary}
Let $f$ be twice continuously differentiable. If, further, for local minimax point $(\hat{x},\hat{y})$, there exists an $\tau$ such that $\tau(\delta)=O(\delta)$ as $\delta\downarrow 0$, then \eqref{gs6-1} can be replaced by
\begin{equation*}
\inp{v, \nabla_{xx}^2 f(\hat{x},\hat{y}) v} \geq 0~\text{for all}~v\in \cl \Gamma_1^\circ(\hat{x},\hat{y}).
\end{equation*}
\end{corollary}

\begin{proof}
Let $0\neq v\in\Gamma_1^\circ(\hat{x},\hat{y})$. According to the definition of $\mathcal{T}_X^\circ(\hat{x})$, there exists a sequence $\{t_k\}_{k\geq 1}$ with $t_k \downarrow 0$ as $k\to \infty$, such that $x^k:=\hat{x} + t_kv \in X$. Let $\delta_k:=\norm{x^k -\hat{x}}$, and $\tilde{y}^k$ be denoted in \eqref{gs10}. Since $\tau(\delta)=O(\delta)$ as $\delta\downarrow 0$, we have $\norm{\tilde{y}^k - \hat{y}} =O\left(\norm{x^k - \hat{x}}\right)$ for sufficiently large $k$.
We know from the twice continuous differentiability of $f$ that
\begin{align*}
f(x^k, \tilde{y}^k) &=  f(\hat{x},\hat{y}) + \nabla_x f(\hat{x},\hat{y})^\top (x^k-\hat{x}) + \nabla_y f(\hat{x},\hat{y})^\top (\tilde{y}^k-\hat{y}) \\
&~~~+\frac{1}{2} (x^k-\hat{x})^\top \nabla_{xx}^2 f(\hat{x}, \hat{y})(x^k-\hat{x}) + (x^k-\hat{x})^\top \nabla_{xy}^2 f(\hat{x}, \hat{y})(\tilde{y}^k-\hat{y}) \\
&~~~+ \frac{1}{2} (\tilde{y}^k-\hat{y})^\top \nabla_{yy}^2 f(\hat{x}, \hat{y})(\tilde{y}^k-\hat{y}) + o\left(\norm{x^k-\hat{x}}^2 + \norm{\tilde{y}^k-\hat{y}}^2 \right), \\
f(\hat{x}, \tilde{y}^k) &=f(\hat{x},\hat{y}) + \nabla_y f(\hat{x},\hat{y})^\top (\tilde{y}^k-\hat{y}) + \frac{1}{2} (\tilde{y}^k-\hat{y})^\top \nabla_{yy}^2 f(\hat{x}, \hat{y})(\tilde{y}^k-\hat{y}) \\
&~~~+ o\left(\norm{\tilde{y}^k-\hat{y}}^2 \right).
\end{align*}
Using $t_k\nabla_x f(\hat{x}, \hat{y})^\top v =\nabla_x f(\hat{x}, \hat{y})^\top (x^k-\hat{x})= 0$  for  $ v\in\Gamma_1^\circ(\hat{x},\hat{y})$,  we have
\begin{align*}
0 &\leq f(x^k, \tilde{y}^k) - f(\hat{x}, \tilde{y}^k) \\
&=\frac{1}{2} (x^k-\hat{x})^\top \nabla_{xx}^2 f(\hat{x}, \hat{y})(x^k-\hat{x}) + (x^k-\hat{x})^\top \nabla_{xy}^2 f(\hat{x}, \hat{y})(\tilde{y}^k-\hat{y})  \\
&~~~+ o\left(\norm{x^k-\hat{x}}^2 + \norm{\tilde{y}^k-\hat{y}}^2 \right) - o\left(\norm{\tilde{y}^k-\hat{y}}^2 \right)\\
&\overset{(a)}{=} \frac{1}{2} (x^k-\hat{x})^\top \nabla_{xx}^2 f(\hat{x}, \hat{y})(x^k-\hat{x}) + (x^k-\hat{x})^\top \nabla_{xy}^2 f(\hat{x}, \hat{y})(\tilde{y}^k-\hat{y})+ o\left(\norm{x^k-\hat{x}}^2 \right) \\
&= t_k^2 \frac{1}{2} v^\top \nabla_{xx}^2 f(\hat{x}, \hat{y})v + o(t_k^2),
\end{align*}
where (a) follows from the fact that $\norm{\tilde{y}^k - \hat{y}} = O\left(\norm{x^k - \hat{x}}\right)$ for sufficiently large $k$.
Dividing by $t_k^2$ in both sides and letting $t_k\to 0$, we
complete the proof.
\end{proof}

The following example illustrates $\cl \left\{w: w \in \Gamma_1^\circ(\hat{x},y'), \forall y'\in \mathbb{B}(\hat{y},\delta) \right\}$ for some $\delta>0$.

\begin{example}
Let $n=m=1$, $X = Y = [-1,1]$.
Consider
\begin{equation*}
\min_{x\in [-1,1]} \max_{y\in [-1,1]}~ f(x,y):= -x^4 + 4x^2y^2 -y^4.
\end{equation*}
We have
\begin{equation*}
\varphi(x)= \max_{y\in[-1,1]} (-x^4 + 4x^2y^2 -y^4) =
\begin{cases}
3 x^4, & x\in \left[-\frac{\sqrt{2}}{2}, \frac{\sqrt{2}}{2}\right]~(y^*=\pm \sqrt{2}x);\\
-x^4 + 4x^2 - 1, & [-1,1]\big\backslash \left[-\frac{\sqrt{2}}{2}, \frac{\sqrt{2}}{2}\right]~(y^*=1),
\end{cases}
\end{equation*}
which is not a convex function over $[-1,1]$.
Moreover, it can be examined that $(0,0)$ is a global minimax point. In fact, it is also a local minimax point. Let $\tau(\delta)=2\delta^2$ and $\delta_0=\frac{\sqrt{2}}{2}$. Then, for any $\delta\in (0,\delta_0]$ and any $(x,y)\in [-1,1]^2$ satisfying $\abs{x}\leq \delta$ and $\abs{y}\leq \delta$, we have
$$-y^4=f(0,y) \leq f(0,0) \leq \max_{y'\in\{y\in Y: \abs{y}\leq \tau(\delta)\}}f(x,y')=3x^4.$$

Therefore, for any $\delta\in(0,1]$,
\begin{align*}
\cl \left\{w: w \in \Gamma_1^\circ(0,y'), \forall y'\in \mathbb{B}(0,\delta) \right\}
&= \cl \left(\bigcap_{y'\in \mathbb{B}(0,\delta)}\{w_1\in \mathcal{T}_{[-1,1]}^\circ(0): w_1\bot \nabla_x f(0,y')\} \right)= \mathbb{R}.
\end{align*}
Similarly, we have
$$\cl\Gamma_2^\circ(0,0)=\left\{w_2\in \mathcal{T}_{[-1,1]}^\circ(0): w_2\bot \nabla_y f(0,0) \right\} = \mathbb{R}.$$

In this case, the second-order optimality  condition \eqref{gs6} means
$\nabla_{xx}^2 f(0,0)\ge 0 $ and $ \nabla_{yy}^2 f(0,0)\le 0$.
\end{example}

In Theorem \ref{Th1}, the first-order and second-order optimality necessary conditions are given in a sense of geometry. With specific structures of sets $X$ and $Y$, we can derive the corresponding Karush-Kuhn-Tucker (KKT) systems. Specially, we assume that $X$ and $Y$ are polyhedral defined as follows:
\begin{equation}
\label{gs3}
X=\{x\in\mathbb{R}^n:Ax\leq b\}~\text{and}~Y=\{y\in\mathbb{R}^m:Cy\leq d\},
\end{equation}
where $A\in \mathbb{R}^{p\times n}$, $b\in \mathbb{R}^{p}$, $C\in \mathbb{R}^{q\times m}$ and $d\in \mathbb{R}^{q}$.

The following proposition establishes the relationship between tangent/normal cones and algebra systems when $X$ and $Y$ are defined in \eqref{gs3}.

\begin{proposition}[\cite{FP2007finite}]\label{Prop6}
Let $X$ and $Y$ be defined in \eqref{gs3}. Then,

\begin{align*}
\mathcal{T}_X(x) &= \left\{ \lambda \in \mathbb{R}^n: -A_i^\top \lambda \geq 0 ~\text{for}~ \forall i\in \mathcal{A}_X(x) \right\},\\
\mathcal{T}_Y(y) &= \left\{ \mu \in \mathbb{R}^m: -C_j^\top \mu \geq 0 ~\text{for}~ \forall j\in \mathcal{A}_Y(y) \right\},\\
\mathcal{N}_X(x) &= \left\{ -\sum_{i=1}^p \alpha_i A_i: \alpha \in \mathcal{N}_{\mathbb{R}_+^p}(b - Ax)\right\},\\
\mathcal{N}_Y(y) &= \left\{ -\sum_{j=1}^q \beta_j C_j: \beta \in \mathcal{N}_{\mathbb{R}_+^q}(d - Cy)\right\},
\end{align*}
where $A_i$ is the $i$th row vector of matrix $A$ and $C_j$ is the $j$th row vector of matrix $C$ respectively for $i=1,\cdots,p$ and $j=1,\cdots,q$, and $\mathcal{A}_X(x)$ and $\mathcal{A}_Y(y)$ are active sets of $X$ at $x$ and $Y$ at $y$, respectively.
\end{proposition}

\begin{theorem}\label{Th3}
Let the tuple $(\hat{x},\hat{y})\in X\times Y$ be a local minimax point of problem \eqref{minmax} with $X$ and $Y$ being defined in \eqref{gs3}. Then there exist multipliers $\alpha\in \mathbb{R}^p$ and $\beta\in\mathbb{R}^q$ such that
\begin{equation}
\label{FKKT}
\begin{cases}
\nabla_x f(\hat{x},\hat{y}) - \sum_{i=1}^p \alpha_i A_i=0,\\
- \nabla_y f(\hat{x},\hat{y}) - \sum_{j=1}^q \beta_j C_j=0,\\
\alpha \in \mathcal{N}_{\mathbb{R}_+^p}(b - A\hat{x}),\\
\beta \in \mathcal{N}_{\mathbb{R}_+^q}(d - C\hat{y}).
\end{cases}
\end{equation}

If, moreover, $f$ is twice continuously differentiable, we have, for any $\delta>0$, that
\begin{equation}
\label{SKKT}
\begin{cases}
\inp{v, \nabla_{xx}^2 f(\hat{x},\hat{y}) v} \geq 0 ~\text{for all}\\
~~~~~v\in \left\{ \lambda \in \mathcal{T}_X(\hat{x}): \exists\delta>0, ~ \lambda^\top \nabla_x f(\hat{x},y')=0 ~\text{for}~ y'\in \mathbb{B}(\hat{y},\delta) \right\},\\
\inp{w, \nabla_{yy}^2 f(\hat{x},\hat{y})w} \leq 0
~\text{for all}~w\in \left\{ \mu \in \mathcal{T}_Y(\hat{y}): \mu^\top \nabla_y f(\hat{x},\hat{y})=0 \right\}.
\end{cases}
\end{equation}
\end{theorem}

The proof of Theorem \ref{Th3} directly follows from Lemma \ref{Lem1}, Theorem \ref{Th1} and Proposition \ref{Prop6}. Thus, we neglect it here.

We call \eqref{FKKT} the first-order KKT system of problem \eqref{minmax} (see also \cite[Proposition 1.2.1]{FP2007finite} for the KKT system for variational inequalities with polyhedral sets) and \eqref{FKKT}-\eqref{SKKT} the second-order KKT system of problem \eqref{minmax}.

\begin{definition}
We call that $(\hat{x},\hat{y})\in X\times Y$ is a first-order stationary point of problem \eqref{minmax} if it satisfies \eqref{gs2-1}-\eqref{gs2-2}. Moreover, if $(\hat{x},\hat{y})$ also satisfies \eqref{gs6-1}-\eqref{gs6-2}, we call it a second-order stationary point of problem \eqref{minmax}.
\end{definition}

The existence results of the first-order stationary points can be obtained by using existing results in \cite[Proposition 2.2.3, Corollary 2.2.5]{FP2007finite}.
Let $F(x,y)=\begin{pmatrix}
\nabla_x f(x,y)\\
- \nabla_y f(x,y)
\end{pmatrix}$.
\begin{enumerate}
\item[(i)] If there exits a bounded open set $\mathcal{Z}\subseteq X\times Y$ and a point $(\bar{x},\bar{y})\in (X\times Y)\cap \mathcal{Z}$ such that
    $$\inp{F(x,y),\begin{pmatrix}x-\bar{x}\\y-\bar{y}\end{pmatrix}}\geq 0,~ \forall (x,y)\in (X\times Y)\cap \bd(\mathcal{Z}),$$
    then problem \eqref{minmax} has at least a first-order stationary point.

\item[(ii)] Specially, if $X$ and $Y$ are bounded, the first-order stationary point set of problem \eqref{minmax} is nonempty.
\end{enumerate}

We know from \cite[Proposition 21]{JNJ2020local} that a global minimax point can be neither a local minimax point nor a stationary point. However, some global minimax points can be the first-order stationary points.

The following proposition claims that under mild conditions a class of global minimax points are first-order stationary points.

\begin{proposition}
\label{Prop2}
Let $f$ be continuously differentiable over $X\times Y$, and $(\hat{x},\hat{y})$ be a global minimax point of \eqref{minmax} satisfying
\begin{equation*}
\hat{y} \in \limsup_{x\to \hat{x}} \left(\argmax_{y'\in Y} f(x,y') \right),
\end{equation*}
where ``\,$\limsup$'' denotes outer limit (\cite[Definition 4.1]{RW2009variational}), then $(\hat{x},\hat{y})$ is a first-order stationary point.
\end{proposition}

\begin{proof}
Since $(\hat{x},\hat{y})$ is a global minimax point, we have
\begin{equation}\label{Gminimax}
f(\hat{x},y) \overset{(a)}{\leq} f(\hat{x},\hat{y})  \overset{(b)}{\leq} \max_{y'\in Y}f(x,y').
\end{equation}
The inequality (a) of \eqref{Gminimax} implies \eqref{gs2-2}. In the sequel, we only consider \eqref{gs2-1} through inequality (b) of \eqref{Gminimax}.
Since
$$\hat{y} \in \limsup_{x\to \hat{x}} \left(\argmax_{y'\in Y} f(x,y') \right),$$
without loss of generalization, we assume that there exists a sequence $\tilde{y}^k\in \mathcal{Y}^k$ such that $\tilde{y}^k \to  \hat{y}$.
By a similar procedure to the proof for (i) of Theorem \ref{Th1}, we have
\begin{align*}
0 &\leq \nabla_x f(\hat{x}, \tilde{y}^k)^\top (x^k-\hat{x})  + o\left(\norm{x^k-\hat{x}}\right)\\
&= \nabla_x f(\hat{x}, \hat{y})^\top (x^k-\hat{x}) + (\nabla_x f(\hat{x}, \tilde{y}^k) - \nabla_x f(\hat{x}, \hat{y}) )^\top (x^k-\hat{x})  + o\left(\norm{x^k-\hat{x}}\right)\\
&= \nabla_x f(\hat{x}, \hat{y})^\top (x^k-\hat{x})   + o\left(\norm{x^k-\hat{x}}\right),
\end{align*}
which implies that $-\nabla_x f(\hat{x}, \hat{y}) \in \mathcal{N}_X(\hat{x})$.
\end{proof}

In general, a global minimax point can be neither a local minimax point nor a stationary point \cite[Proposition 21]{JNJ2020local}. We give the following example to show this.

\begin{example}[{\cite[Figure 2]{JNJ2020local}}]\label{Eg2}
Let $n=m=1$, $X = [-1,1]$ and $Y = [-5,5]$. Consider the following minimax problem
\begin{equation}
\label{eg-gs1}
\min_{x\in[-1,1]}\max_{y\in [-5,5]}~f(x,y):=  xy-\cos(y).
\end{equation}

By direct calculation, we have
\begin{align*}
\varphi(x) &= \max_{y\in[-5,5]} (xy-\cos(y)) \\
&=
\begin{cases}
x\cdot( \pi -  \arcsin(-x)) - \cos(\pi-\arcsin(-x)), & x\in[0,1];\\
x\cdot( -\pi -  \arcsin(-x)) - \cos(-\pi-\arcsin(-x)), & x\in[-1,0],
\end{cases}
\end{align*}
where the optima is achieved when $y= \pi - \arcsin(-x)$ and $y= -\pi - \arcsin(-x)$, respectively. It can observe from the definition of $\varphi(x)$ that $x=0$ is the minimizer. In this case, $(0,-\pi)$ and $(0,\pi)$ are two global minimax points. However, they both fail to satisfy \eqref{gs2-1}-\eqref{gs2-2}, that is,
\begin{equation*}
  \begin{cases}
  0\in  x + \sin(y) + \mathcal{N}_{[-1,1]}(x),\\
  0\in y + \mathcal{N}_{[-5,5]}(y),
  \end{cases}
\end{equation*}
which has a unique solution $(0,0)$. Thus, neither $(0,-\pi)$ nor $(0,\pi)$ is a first-order stationary point, which implies from Theorem \ref{Th1} that they cannot be local minimax points either. Therefore, a global minimax point can be neither a local minimax point nor a first-order stationary point.

Next, we show that even $(0,0)$ is not a local minimax point. For any $y$ satisfying $0<\abs{y}\leq \delta$ with any sufficiently small $\delta>0$, we have
$$-\cos(y)=f(0,y) > f(0,0)=-1,$$
which, according to the definition of local minimax points in Definition \ref{Def4}, concludes that $(0,0)$ is not a local minimax point. Therefore, problem \eqref{eg-gs1} here does not have a local minimax point even both $X$ and $Y$ are bounded.
\end{example}

Sometimes we can find that a global minimax point may be a stationary point (Example \ref{Eg1}). In the following proposition, we conclude some sufficient conditions such that a global minimax point is a local minimax point.

\begin{proposition}
\label{Prop3}
Let $(\hat{x},\hat{y})$ be a global minimax point and $f$ be Lipschitz continuous over $X\times Y$. If $\max_{y'\in Y} f(x,y')$ has a unique solution for all $x$ in a neighborhood of $\hat{x}$, then $(\hat{x},\hat{y})$ is a local minimax point.
\end{proposition}

\begin{proof}
Since $\max_{y'\in Y} f(x,y')$ has a unique solution for all $x$ in a neighborhood of $\hat{x}$, we use $\bar{y}(x)$ to denote this unique solution. By \cite[Lemma 4.1]{X2010uniform}, we know that
$$\norm{\bar{y}(x) - \hat{y}} \to 0 ~\text{as}~x\to \hat{x},$$
which implies that there exists a $\delta_0>0$ such that for any $x\in X$ satisfying $\norm{x-\hat{x}} \leq \delta\leq \delta_0$, $\tau(\delta)\to 0$ where $\tau(\delta):=\sup_{\{x\in X:\norm{x-\hat{x}} \leq \delta\}}\norm{\bar{y}(x) - \hat{y}}$. As $(\hat{x},\hat{y})$ is a global minimax point, we have for any $x\in X$ and $y\in Y$ that
\begin{equation*}
f(\hat{x},y) \leq f(\hat{x},\hat{y})  \leq \max_{y'\in Y}f(x,y').
\end{equation*}
This indicates that for $x$ satisfying $\norm{x-\hat{x}} \leq \delta (\leq \delta_0)$ and $y$ satisfying $\norm{y - \hat{y}} \leq \tau(\delta)$, we have
\begin{equation*}
f(\hat{x},y) \leq f(\hat{x},\hat{y})  \leq \max_{y'\in Y}f(x,y')= f(x,\bar{y}(x)) = \max_{y'\in \{y\in Y: \norm{y - \hat{y}} \leq \tau(\delta)\}}f(x,y').
\end{equation*}
Thus, $(\hat{x},\hat{y})$ is a local minimax point based on Definition \ref{Def4}.
\end{proof}

Obviously, when $f(x,\cdot)$ is strictly concave for all $x$ in a neighborhood of $\hat{x}$, the  condition for the uniqueness of the maximization problem holds.

To end this section, we summarize relationships between saddle points, local saddle points, global minimax points, local minimax points and first-order and second-order stationary points in Figure \ref{fig-3}.
\begin{figure}[htp]
\begin{center}
\begin{tikzpicture}
\draw[fill=red,fill opacity=0.3] (0,0) ellipse (4.6cm and 1.4cm);
\draw (0.1,0.3)node[right=80pt,black,fill opacity=1]{\tiny{second-order}};
\draw (0.1,0.05)node[right=80pt,black,fill opacity=1]{\tiny{stationary}};
\draw (0.1,-0.2)node[right=80pt,black,fill opacity=1]{\tiny{point}};

\draw[fill=magenta,fill opacity=0.3] (0,0) ellipse (5.9cm and 1.5cm);
\draw (1.35,0.3)node[right=90pt,black,fill opacity=1]{\tiny{first-order}};
\draw (1.35,0.05)node[right=90pt,black,fill opacity=1]{\tiny{stationary}};
\draw (1.35,-0.2)node[right=90pt,black,fill opacity=1]{\tiny{point}};

\draw[fill=cyan,fill opacity=0.3] (0,0) ellipse (1.1cm and 2.3cm);
\draw (0,2.05) node[black,fill opacity=1]{\tiny{global}};
\draw (0,1.85) node[black,fill opacity=1]{\tiny{minimax}};
\draw (0,1.65) node[black,fill opacity=1]{\tiny{point}};

\draw[fill=blue,fill opacity=0.3] (0,0) ellipse (3cm and 1.3cm);
\draw (-0.2,0.25) node[right=60pt,black,fill opacity=1]{\tiny{local}};
\draw (-0.2,0) node[right=60pt,black,fill opacity=1]{\tiny{minimax}};
\draw (-0.2,-0.25) node[right=60pt,black,fill opacity=1]{\tiny{point}};

\draw[fill=orange,fill opacity=0.3] (0,0) ellipse (2cm and 1.1cm);
\draw (-0.5,0.25) node[right=45pt,black,fill opacity=1]{\tiny{local}};
\draw (-0.5,0) node[right=45pt,black,fill opacity=1]{\tiny{saddle}};
\draw (-0.5,-0.25) node[right=45pt,black,fill opacity=1]{\tiny{point}};

\draw[fill=green,fill opacity=0.3] (0,0) circle (1cm) node[black,fill opacity=1]{\tiny{saddle point}};
\end{tikzpicture}
\end{center}
\caption{Venn diagram for saddle points, minimax points and stationary points} \label{fig-3}
\end{figure}

\section{Generative adversarial networks}\label{Sec4}

In this section, we consider the first-order and second -order optimality conditions of the GAN using nonsmooth activation functions, which can be formulated as nonsmooth nonconvex-nonconcave  min-max problem (\ref{minmax}).

The GAN is one of the most popular generative models in machine learning. It is comprised of two ingredients: the generator, which creates samples that are intended to follow the same distribution as the training data, and the discriminator, which examines samples to determine whether they are real or fake. For more motivations and advantages of GANs, one can refer to \cite{G2014generative}. Recently, Wang gave a mathematical introduction to GANs in \cite{W2020mathematical}.

The plain vanilla GAN model can be formulated as \eqref{GAN}, where $D$ and $G$ are given by feedforward neural networks with parameters $x$ and $y$, respectively. Theoretically, the universal approximation theorem tells us that, under certain mild conditions, a feedforward neural network can approximate any Borel measurable function. Moreover, the derivatives of the feedforward neural network can also approximate the derivatives of this function arbitrarily well. The activation function is a function from $\mathbb{R}$ to $\mathbb{R}$ that is used to compute the hidden layer values and introduce the nonlinear property. There are several commonly-used activation functions, such as ReLU $\sigma(z)=\max\{0,z\}$, the logistic sigmoid $\sigma(z) = 1/(1+ \exp(-z))$, the softplus activation function $\sigma(z) = \ln(1+ \exp(z))$, etc.

We give an intuition for $D$ and $G$ which are consist of linear models with activation functions in the following example.

\begin{example}\label{Eg4}
Consider that the discriminator $D$ is a single-layer network with a logistic sigmoid activation function \cite{GLWM2020limiting} and the generator $G$ is a two-layer network with an activation function  $\sigma$ as follows
\begin{align*}
G(x,\xi_2):= W_2 \bm{\sigma} (W_1\xi_2+b_1) + b_2,   \quad \quad
D(y,\xi_1):= \frac{1}{1+ \exp(y^\top\xi_1 )},
\end{align*}
where $x=(\mathrm{vec}(W_1)^\top, \mathrm{vec}(W_2)^\top,b_1^\top,b_2^\top)^\top$ and $\mathrm{vec}(\cdot)$ denotes the columnwise vectorization operator of matrices, $W_1\in \mathbb{R}^{s \times s_2}$, $b_1\in \mathbb{R}^{s }$, $W_2\in \mathbb{R}^{s_1\times s}$, $b_2\in \mathbb{R}^{s_1}$ and $\bm{\sigma}:\mathbb{R}^s\to \mathbb{R}^s$.
  Here, $\bm{\sigma}$ is a separable vector activation function that aggregates the individual neuron activations in the layer.

In this case, the GAN model \eqref{GAN} can be explicitly written as
\begin{equation}
\label{GAN-2}
\begin{aligned}
\min_{x \in X} \max_{y \in Y}~  &f(x,y)=\Bigg( \mathbb{E}_{P_{1}}\left[ \log\left( \frac{1}{1+ \exp(y^\top \xi_1)} \right) \right]  \\
&+  \mathbb{E}_{P_2}\left[\log\left( 1 - \frac{1}{1+ \exp(y^\top (W_2 \bm{\sigma} (W_1\xi_2+b_1) + b_2) )} \right)\right] \Bigg).
\end{aligned}
\end{equation}
If $X$ and $Y$ are compact and $\bm{\sigma}$ is continuous, by Proposition \ref{Prop1}, problem (\ref{GAN-2}) has a global minimax point.
\end{example}

Obviously, if $D(\cdot,\xi_1)$ and $G(\cdot,\xi_2)$ are smooth (i.e. $\bm{\sigma}$ is smooth), the necessary optimality conditions in Theorem \ref{Th1} hold. Next, we focus on the nonsmooth case with the ReLU activation function.

\begin{proposition}\label{Prop8}
Let $f$ be defined in (\ref{GAN-2})  with $\bm{\sigma}(\cdot)=(\cdot)_+$. Assume that support sets $\Xi_1$ and $\Xi_2$ are bounded. Then the following statements hold.
\begin{enumerate}
\item[(i)] $f$ is locally Lipschitz continuous and twice semidifferentiable in $X\times Y$.
\item[(ii)] In addition, if $\xi_2$ is a continuous random variable, $\Xi_2$ has a nonempty interior and $\|(W_1)_i\|+|(b_1)_i|>0$,  $i=1,\cdots,s$, then $f$ is twice continuously differentiable at $(x,y)$.
\end{enumerate}
\end{proposition}

\begin{proof}
Let
\begin{align*}
\rho_1(y)=\mathbb{E}_{P_{1}}\left[ \log\left( D(y, \xi_1) \right) \right],~ \rho_2(x,y)= \mathbb{E}_{P_2}\left[\log\left( 1 - D(y, G(x,\xi_2))\right)\right].
\end{align*}
(i) Since for any fixed  $\xi_2\in \Xi_2$, $G(x,\xi_2)$ and
$\log\left( 1 - \frac{1}{1+ \exp(y^\top G(x,\xi_2))} \right)$ are locally Lipschitz continuous in $X\times Y$,  the local Lipschitz continuity of $f(x,y)=\rho_1(y) + \rho_2( x,y)$ follows the continuous differentiability of \emph{log} and \emph{exp} functions. Moreover, the twice semidifferentiability follows directly from Example \ref{Eg3}.

(ii) Obviously, $f$ is twice continuously differentiable with respect to $y$. In what follows, we only consider the continuous differentiability of $f$ with respect to $x$. Since $\Xi_2$ has a nonempty interior and $\xi_2$ is a continuous random variable, we know that $P_2(\{\xi_2: (W_1\xi_2+b_1)_i=0\}) = 0$. Thus, $\Xi_0:=\cup_{i=1}^s\{\xi_2: (W_1\xi_2+b_1)_i=0\}$ is $P_2$-zero. Note that $\log\left( 1 - D(y, G(x,\xi_2))\right)$ is continuously differentiable with respect to $x$ for every $\xi_2\in \Xi_2\backslash \Xi_0$. Since $P_2(\Xi_0)=0$, we have
\begin{align*}
\rho_2(x,y) = \int_{\Xi_2\backslash \Xi_0 } \log\left( 1 - D(y, G(x,\xi_2))\right) P_2(\d\xi_2).
\end{align*}
According to \cite[Theorem 7.57]{SDR2014lectures}, we have $\int_{\Xi_2\backslash \Xi_0 } \log\left( 1 - D(y, G(x,\xi_2))\right) P_2(\d\xi_2)$ is continuously differentiable. Thus, $\rho_2(x,y)$ is continuously differentiable and
\begin{align*}
\nabla_x \rho_2(x,y) &= \nabla_x \int_{\Xi_2\backslash \Xi_0 } \log\left( 1 - D(y, G(x,\xi_2))\right) P_2(\d\xi_2)\\
&= \int_{\Xi_2\backslash \Xi_0 } \nabla_x  \log\left( 1 - D(y, G(x,\xi_2))\right) P_2(\d\xi_2).
\end{align*}
By using \cite[Theorem 7.57]{SDR2014lectures} again, we have $\nabla_x\rho_2(x,y)$ is continuously differentiable at $(x,y)$. Thus, $f$ is twice continuously differentiable at $(x,y)$.
\end{proof}

By directly using Proposition \ref{Prop8}, we can apply Theorems \ref{Th5} and \ref{Th1} to problem \eqref{GAN-2}.

\begin{proposition}
Suppose the assumptions of Proposition \ref{Prop8} hold. Let $(\hat{x},\hat{y})$ be a local minimax point of problem \eqref{GAN-2}.  Then the first-order necessary optimality conditions \eqref{NonS1st-1}-\eqref{NonS1st-2} hold at $(\hat{x},\hat{y})$. If, in addition, $\xi_2$ is a continuous random variable, $\Xi_2$ has a nonempty interior and $\|(W_1)_i\|+|(b_1)_i|>0$,  $i=1,\cdots,s$, then the first-order and second-order necessary optimality conditions  \eqref{gs2-1}-\eqref{gs2-2} and \eqref{gs6-1}-\eqref{gs6-2} hold at $(\hat{x},\hat{y})$.
\end{proposition}

In the remainder of this section, we show how to verify whether a pair $(x,y)\in X\times Y$ is a first-order or second-order stationary point of problem \eqref{GAN-2} in Example \ref{Eg4} by Proposition \ref{Prop8}. To this end, we assume  $\|(W_1)_i\|+|(b_1)_i|>0$,  $i=1,\cdots,s$, $\xi_1, \xi_2$ are  continuous random variables with bounded support sets, and $\Xi_2$ has a nonempty interior.
Let $X=[a, b]$ and $Y=[c, d]$ where $a, b \in \mathbb{R}^n,
c, d\in\mathbb{R}^m$, $a < b$, and $c<d$ with $n=(s+1)(s_1+s_2)$ and $m=s_1$.

Since $\|(W_1)_i\|+|(b_1)_i|>0$,  $i=1,\cdots,s$, we can generate independent identically distributed (iid) samples $\{\xi_1^j\}_{j=1}^N$ of $\xi_1$ and $\{\xi_2^j\}_{j=1}^N$ of $\xi_2$ such that $(W_1\xi_2^j+b_1)_i\neq 0$ for $i=1,\cdots,s$ and $j=1,\cdots,N$.
We consider the following min-max problem
\begin{equation}
\label{SAA-GAN-2}
\begin{aligned}
\min_{x\in X}\max_{y\in Y}~\hat{f}_N(x,y):= &\frac{1}{N}\sum_{i=1}^N \Bigg( \log\left( \frac{1}{1+ \exp(y^\top \xi_1^i)} \right) \\
&+  \log\left( 1 - \frac{1}{1+ \exp(y^\top (W_2 (W_1\xi_2^i+b_1)_+ + b_2) )} \right) \Bigg).
\end{aligned}
\end{equation}

Use the existing automatic differentiation technique, such as back-propagation, we can compute
$\nabla_x \hat{f}_N(x,y)$,  $\nabla_y \hat{f}_N(x,y)$, $\nabla_{xx}^2 \hat{f}_N(x,y)$,  $\nabla_{yy}^2 \hat{f}_N(x,y)$. Moreover,
we have
\begin{equation*}
\mathcal{T}_X(x) = \mathcal{T}_X^\circ(x)= \left\{ v \in \mathbb{R}^n \, :\,
v_i \in\begin{cases}
[0, \infty), & \text{if}~x_i = a_i\\
(-\infty, \infty), & \text{if}~a_i < x_i <b_i\\
(-\infty, 0], & \text{if}~x_i = b_i
\end{cases}\right\},
\end{equation*}
\begin{equation*}
\mathcal{T}_Y(y) = \mathcal{T}_Y^\circ(y)= \left\{ w \in \mathbb{R}^m \, :
\,  w_j \in \begin{cases}
[0, \infty), & \text{if}~y_j = c_j\\
(-\infty, \infty), & \text{if}~c_j < y_j <d_j\\
(-\infty, 0], & \text{if}~y_j = d_j
\end{cases}\right\}
\end{equation*}
and
\begin{align*}
\Gamma_1^\circ(x,y)&=\Gamma_1(x,y)=\{v\in \mathcal{T}_X(x): v\bot \nabla_x \hat{f}_N(x,y)\},\\
\Gamma_2^\circ(x,y)&=\Gamma_2(x,y)=\{w\in \mathcal{T}_Y(y): w\bot \nabla_y \hat{f}_N(x,y)\}.
\end{align*}
By Theorem \ref{Th1}, if $(\hat{x},\hat{y})$ is a local minimax point of problem (\ref{SAA-GAN-2}), then $(\hat{x},\hat{y})$ must satisfy the first-order and second-order optimality  conditions:
$$\begin{cases}(\nabla_x \hat{f}_N(\hat{x},\hat{y}))_i\ge 0,
 & \text{if}~x_i = a_i;\\
(\nabla_x \hat{f}_N(\hat{x},\hat{y}))_i=0, & \text{if}~a_i < x_i <b_i;\\
(\nabla_x \hat{f}_N(\hat{x},\hat{y}))_i\le 0, & \text{if}~x_i = b_i
\end{cases}
~\text{and}~
\begin{cases}(\nabla_y \hat{f}_N(\hat{x},\hat{y}))_j\le 0,
 & \text{if}~y_j = c_j;\\
(\nabla_y \hat{f}_N(\hat{x},\hat{y}))_j=0, & \text{if}~c_j < y_j <d_j;\\
(\nabla_y \hat{f}_N(\hat{x},\hat{y}))_j\ge 0, & \text{if}~y_j = d_j
\end{cases}
$$
for $i=1,\ldots, n$, $j=1,\ldots, m$,  and
\begin{align*}
&\inp{v, \nabla_{xx}^2 \hat{f}_N(\hat{x},\hat{y}) v} \geq 0~\text{for all}~v\in \left\{\bar{v}: \exists \delta>0, \bar{v} \in \Gamma_1(\hat{x},y'), \forall y'\in \mathbb{B}(\hat{y},\delta) \right\},\\
&\inp{w, \nabla_{yy}^2 \hat{f}_N(\hat{x},\hat{y}) w} \leq 0~\text{for all}~w\in \Gamma_2(\hat{x},\hat{y}).
\end{align*}

The following proposition tells that the above procedures can ensure an exponential rate of convergence with respect to sample size $N$.

\begin{proposition}Suppose the assumptions of Proposition \ref{Prop8} hold.
 If $(x_N, y_N)$ is a first-order (or second-order stationary point) of problem \eqref{SAA-GAN-2} with iid samples $ \{\xi_1^j\}_{j=1}^N$ and $\{\xi_2^j\}_{j=1}^N$ of $\xi_1$ and $\xi_2$ respectively,  then $(x_N, y_N)$ converges to a first-order (or second-order stationary point) of problem \eqref{GAN-2} exponentially with respect to $N$.
\end{proposition}

\begin{proof}
Denote
\begin{align*}
h(z)=
\begin{pmatrix}
\nabla_x f(x,y)\\
-\nabla_y f(x,y)
\end{pmatrix}, \quad \quad
H(z)=
\begin{pmatrix}
\sup_{v\in \mathcal{V}(x,y)}\inp{v, -\nabla_{xx}^2 f(x,y) v}\\
\sup_{w\in \mathcal{W}(x,y)}\inp{w, \nabla_{yy}^2 f(x,y) w}
\end{pmatrix},\\
\hat{h}_N(z)=
\begin{pmatrix}
\nabla_x \hat{f}_N(x,y)\\
-\nabla_y \hat{f}_N(x,y)
\end{pmatrix}, \quad \quad
\widehat{H}_N(z)=
\begin{pmatrix}
\sup_{v\in \mathcal{V}(x,y)}\inp{v, -\nabla_{xx}^2 \hat{f}_N(x,y) v}\\
\sup_{w\in \mathcal{W}(x,y)}\inp{w, \nabla_{yy}^2 \hat{f}_N(x,y) w}
\end{pmatrix},
\end{align*}
where $z=(x^\top,y^\top)^\top$, $\mathcal{V}(x,y):=\mathbb{B}(0,1) \cap  \cup_{\delta>0} \cl \left\{\bar{v}: \exists \delta>0, \bar{v} \in \Gamma_1^\circ(x,y'), \forall y'\in \mathbb{B}(y,\delta) \right\}$ and $\mathcal{W}(x,y):=\mathbb{B}(0,1) \cap \cl\Gamma_2^\circ(x,y)$.

According to the twice continuous differentiability of $f$ (see Proposition \ref{Prop8}) and the boundedness of $\Xi_1$ and $\Xi_2$, we have $\hat{h}_N(z) \to h(z)$ and $\widehat{H}_N(z) \to H(z)$ exponentially fast uniformly in any compact subset of $\mathcal{Z}\subseteq Z:=X\times Y$ (\cite[Theorem 7.73]{SDR2014lectures}). That is, for any given $\epsilon >0$, there exist $C=C(\epsilon)$ and $\beta=\beta(\epsilon)$, such that
\begin{equation*}
\prob\left\{ \sup_{z\in \mathcal{Z}}\norm{\hat{h}_N(z) - h(z)} \geq \epsilon \right\} \leq C e^{-N\beta} ~\text{and}~ \prob\left\{ \sup_{z\in \mathcal{Z}}\abs{\widehat{H}_N(z)- H(z)} \geq \epsilon \right\} \leq C e^{-N\beta}.
\end{equation*}
Without loss of generality, we assume that $z_N=(x_N^\top, y_N^\top)^\top\in \mathcal{Z}$. Denote the following general growth functions:
\begin{align*}
\psi_1(\tau)&:=\inf\{\d(0, h(z) + \mathcal{N}_Z(z)): z\in \mathcal{Z},\,  \d(z, \mathcal{S}_1)\geq \tau \},\\
\psi_2(\tau)&:=\inf\{\norm{(H(z))_+}: z\in \mathcal{Z}, \, \d(z, \mathcal{S}_2)\geq \tau \},
\end{align*}
where $\mathcal{S}_1$ and $\mathcal{S}_2$ are the sets satisfying \eqref{gs2-1}-\eqref{gs2-2} and \eqref{gs6-1}-\eqref{gs6-2}, respectively, and ``$\d$'' denotes the distance from a point to a set. Let the related functions
\begin{equation*}
\Psi_1(t):=\psi_1^{-1}(t) + t ~\text{and}~ \Psi_2(t):=\psi_2^{-1}(t) + t,
\end{equation*}
where $\psi_i^{-1}(t):=\sup\{\tau: \psi_i(\tau) \leq \eta\}$ for $i=1,2$, which satisfy $\Psi_i(t) \to 0$ as $t\downarrow 0$ for $i=1,2$.

Then, by a conventional discussion (see e.g. \cite{CSS2019convergence}), we have
\begin{equation*}
\d(z_N, \mathcal{S}_1) \leq  \Psi_1\left( \sup_{z\in Z}\norm{\hat{h}_N(z) - h(z)} \right)~\text{and}~\d(z_N, \mathcal{S}_2) \leq  \Psi_2\left( \sup_{z\in Z}\abs{\widehat{H}_N(z)- H(z)} \right).
\end{equation*}
Thus, we have
\begin{equation*}
\prob\left\{ \d(z_N, \mathcal{S}_1) \geq \Psi_1(\epsilon) \right\} \leq C e^{-N\beta} ~\text{and}~ \prob\left\{ \d(z_N, \mathcal{S}_2) \geq \Psi_2(\epsilon) \right\} \leq C e^{-N\beta},
\end{equation*}
which shows that $z_N$ converges to a first-order stationary point in $\mathcal{S}_1$ (or a first-order stationary point in $\mathcal{S}_2$)  exponentially with respect to $N$.
\end{proof}

\section{Conclusions}\label{Sec5}

Many nonconvex-nonconcave min-max problems in dada sciences do not have saddle points. In this paper, we provide sufficient conditions for the existence of global and local minimax points of constrained nonsmooth nonconvex-nonconcave min-max problem (\ref{minmax}). Moreover, we give the first-order and second-order optimality conditions of local minimax points of problem (\ref{minmax}), and use these conditions to define the first-order and second-order stationary points of (\ref{minmax}). The relationships between saddle points, local saddle points, global minimax points, local minimax points, stationary points are summarized in Figure \ref{fig-3}. Several examples are employed to illustrate our theoretical results. To demonstrate applications of these optimality conditions in deep learning, we propose a method to verify the optimality conditions at any given point of generative adversarial network (\ref{GAN-2}).

redpajama_set_name: RedPajamaArXiv


A few days ago, I saw a lady wearing a tshirt that said, "blah, blah, blah" near the Ferry entrance in Manhattan. Thinking about that today yielded this funny tshirt as the perfect reminder of the many, many, many conversations I've overheard on trains, waiting for buses, in public spaces, and walking down the street. Loudly, I might add, the people having the conversation whether by phone or in person walking with someone had to make sure everyone within hearing distance got this information. So, this will save time and be easier – yes, this must be important information to keep – we all get it, including me.


BIJ.TV caught up with former Mayor of London Ken Livingstone at MIPIM Japan 2015, an inaugural conference for professionals in the real estate industry. Livingstone, who was mayor when London successfully bid for the 2012 Olympic and Paralympic Games, was a guest speaker at the event, addressing a number of topics including city investment, sustainability, legacy and lessons learned from the Games. With Tokyo 2020 approaching, now is a great opportunity to learn from the man who began the UK's journey to London 2012— one of the most successful host cities.


text:
Product, Pricing & Promotions
Bid Support
Trade Promotion Optimization
Product Mix and Profitability
Production Planning / Scheduling
Natural Resources / Mining
Supply Optimization
Cross-functional applications
Ultimate Guide to Prescriptive Analytics
Quality Plan / Blend Optimization
The Stream by River Logic
What Is Predictive Analytics and Who Is Winning With It
November 7, 2019 | By River Logic
Tags #PrescriptiveAnalytics #PredictiveModelingAndAnalytics #AdvancedAnalytics #PredictiveAnalytics #BusinessAnalytics
Predictive analytics is an important analytical technique used by many organizations to determine risk, identify future business trends and recognize when maintenance is needed. Using historical data as their source, data scientists apply various regression analyses and machine learning techniques to identify patterns and trends contained within that data.
The primary purpose of predictive analytics is to identify, with a high degree of probability, what will happen in the future. This separates predictive analytics from descriptive analytics, which helps analysts understand what has already happened, and prescriptive analytics, which uses optimization software to determine the best decisions to deal with the trends revealed by predictive analytics.
The term predictive analytics applies to a range of advanced analytical techniques that analyze data to determine the likelihood of certain events taking place. For example, in a macro sense, predictive analytics can be used to determine future population trends. Conversely, it can also be used in the micro sense to determine the credit risk of an individual home loan applicant.
Using advanced statistical techniques, predictive analytics calculates the probability of a particular event taking place. In the example of population trends above, the analyst could access available historical population data to establish the probability of growth in a particular city or metropolitan area.
The reliability of predictions depends largely on the predictive analytics model chosen and the quality of the data used. There are many different forms of predictive analysis, and it's important to choose one suited to the problem at hand.
Predictive Analytics Techniques
Predictive analytics techniques can be broken up into three specific areas:
Although organizations may have access to a great deal of data, much is unstructured and must be prepared so it can be processed. This includes cleaning up data to remove incorrect and garbled information as well as organizing the data in an appropriate format.
The most commonly used analytical techniques include regression analysis and machine learning. With regression analysis, analysts try to determine which of several independent variables are statistically correlated to an event. Machine learning takes this a step further in that it uses various forms of decision-making techniques, such as neural networks, Naïve Bayes conditional probability rules and geospatial modeling to depict complex relationships.
All analyses are based on certain assumptions that need to be relevant and correct. For example, some may deal with how people behave, and this will change over time. The importance of confirming that assumptions are correct cannot be overstated. History has many examples, such as the 2008 financial crisis and the last presidential elections where critical assumptions were incorrect, leading to wrong predictions.
Developing Predictive Analytics Models
As powerful as predictive analytics is, developing a model takes time and a degree of care. The first step is that executive buy-in to the concept is obtained. This is closely followed by appointing a capable internal team that has an adequate degree of advanced analytics knowledge and experience, possibly with support from an external vendor. At this point, it's crucial to precisely define the objective of the program and specify exactly what information the model should determine.
The next step is to prepare the data, also known as data mining. This includes identifying what data is needed, cleaning it, and storing the data in a usable form in an integrated data warehouse. Data preparation accounts for a significant percentage of the overall work. The next phase includes performing a statistical analysis to validate assumptions followed by preparation of a predictive model. A crucial aspect is model validation, which uses historical data with known outcomes to check and calibrate the model. This may entail the need to investigate alternative analytical techniques to establish a model that closely correlates with historical data.
Once the model's integrity has been established, it is deployed and used to guide managerial decisions. Periodically, it's essential to review model performance to ensure assumptions are still valid and that the model continues to provide credible predictions.
The Analytics Hierarchy
A key aspect of appreciating predictive analytics is understanding where predictive analytics fits into the hierarchy of data analytics. Data analytics can be defined as the process of analyzing data to determine meaningful trends and metrics so managers can use that information to manage and optimize performance.
The simplest form of analytics is descriptive analytics, which measures historical performance indicators such as sales and margins. The next type of analytics is diagnostic analytics, which analyzes reasons for past performance. These analytics measure past performance, while predictive analytics is used to determine future metrics, such as the probability of increasing sales.
Although descriptive and diagnostic information is important, as is knowing what's likely to happen in the future, what's missing is the answer to the question, "What should I do?" This is where prescriptive analytics fits in. Using information derived from descriptive, diagnostic and predictive analytics, prescriptive analytics determines which of a range of decisions is the best in terms of achieving specific organizational goals, such as how to increase sales.
Who's Using Predictive Analytics
Many organizations are using predictive analytics to enhance their business and reduce risk, and here are some typical predictive analytics examples:
Retailers use analytics to understand customer preferences and implement strategies to encourage customer retention, increase sales and satisfy customers.
Medical specialists use predictive analytics to determine patients who are at risk of developing diseases such as diabetes and asthma.
Banks use analytics to evaluate credit risk to determine whether to extend or withdraw credit.
Insurance companies use analytics to determine long-term insurance risks.
Airlines use predictive analytics to determine aircraft maintenance requirements.
Utilities vary electricity prices depending on demand forecasts.
Manufacturers use predictive analytics to monitor supplier performance, predict maintenance requirements and optimize production capacity.
Benefits of Predictive Analytics
Businesses invested in predictive analytics report significant benefits. For example, Gartner reports that a significant percentage of organizations that invest in supply chain analytics achieve a high return on their investment. Some specific benefits reported include:
Businesses gain a competitive advantage by analyzing internal data to better understand customer needs and behavior.
Companies reduce costs by better understanding future demand and adjusting production and inventory accordingly.
Banks and other financial institutions reduce fraud through predicting suspicious behavior.
Retail outlets increase turnover through accurately predicting customer preferences and offering customer-specific services and products.
All reduce risk through a better understanding of their business environment and those factors that could potentially impact the organization's profitability.
Some Limitations of Predictive Analytics and Why Prescriptive Analytics Is Important
While predictive analytics offers many benefits, it's important to appreciate the limitations of this technique. Possibly the most significant drawback is that predictive analytics tools can't inform managers of the right decisions; they only alert them to the fact that decisions are needed. While in some instances decisions may be obvious, in many others they aren't. The benefits of predictive analytics can be easily negated by poor decision-making based on gut feel. In such situations, a prescriptive analytics solution may be a better answer because of its ability to determine the best decision in any circumstance.
Other limitations of predictive analytics include the risk of poor design and validation of models, which can result in incorrect predictions and financial loss. This is particularly a concern expressed by Deloitte over what the company terms industrialized analytics, which may not be fit for purpose nor correctly reflect the organization's environment.
This can be avoided by careful project specification and model design. It's vital to have the right people on the team. If your organization doesn't have people with the right predictive analytical design skills, consider working with a trusted vendor with the appropriate skills and experience. In such instances, make certain that, over a period time, you bring these skills in-house.
In any event, it's essential you develop the in-house capability to revise and update models and avoid the black box syndrome where no one has a clue how the model works. While this is often an issue with hard-coded solutions, it's less of a problem with newer predictive and prescriptive analytics solutions that have an intuitive programming interface.
You May Also Enjoy Reading
[Infographic] Implementing Prescriptive Analytics: 6 Easy Tips for Success…(Read More)
Financial Optimization: What Do You Need to Succeed in 2020?…(Read More)
What Do Top Supply Chain Companies Know About Analytics?…(Read More)
The 10 Must-Read Supply Chain Blogs for 2020…(Read More)
Top 11 Supply Chain Trends You Need to Know in 2020…(Read More)
How to Select the Best Supply Chain Optimization Software…(Read More)
It's Time to Move Beyond Rough Cut Capacity Planning…(Read More)
6 Big Myths You Need to Know about Prescriptive Analytics…(Read More)
10 Prescriptive Analytics Examples for the Food and Beverage Industry…(Read More)
3 Risks Facing Consumer Packaged Goods Companies in 2019…(Read More)
Integrated Business Planning: 2019 Expert Round-up…(Read More)
River Logic's Rohan Guthal Named to the 2019 Food Logistics Champions: Rock Stars of the Supply Chain…(Read More)
The Limitations of Predictive Analytics Tools and Why Execs Should Care…(Read More)
What is Integrated Business Planning and Why is It Crucial in 2020?…(Read More)
2018 Year in Review: The Evolution of Types of Analytics…(Read More)
Five Key Questions a Successful S&OP Process Strategy Should Ask…(Read More)
Business Analytics: Real-World Applications Get Prescriptive…(Read More)
Prescriptive Analytics (104)
Integrated Business Planning (68)
S&OP (62)
Enterprise Optimizer (36)
Decision Analysis (30)
Predictive Modeling and Analytics (30)
Advanced Analytics (24)
Constraint-Based Modeling (21)
EO Users (18)
Sales and Operations Planning (16)
Supply Chain Optimization (15)
Partners (14)
Gartner (11)
Trade Promotion Management and Optimization (10)
Scenario Management (9)
Supply Optimization (8)
Supply Planning (8)
Demand Optimization (7)
Enterprise Optimization (7)
IBP (7)
Supply Chain Excellence (7)
Demand Planning (6)
Optimization software (6)
modeling software (5)
Algorithmic Business (4)
Citizen Data Scientist (4)
Demand Shaping (4)
Integrated Delivery Systems (4)
Scenario Analysis (4)
what-if analysis (4)
CSCO (3)
Consumer Packaged Goods (3)
Global Planning (3)
Supply Chain Network Design (3)
supply chain modeling (3)
Corporate Strategy (2)
Customer Resources (2)
EO+ (2)
Energy Trading (2)
Network Design (2)
Petrochemicals (2)
Supply Chain Planning (2)
Supply chain analytics (2)
Trade Promotion Optimization (2)
optimization modeling (2)
Balanced Scorecard (1)
Bimodal (1)
Business Optimization (1)
Commodity Trading (1)
Decisions support (1)
Financial Optimization (1)
Food and Beverage (1)
Inventory Optimization (1)
Magic Quadrant (1)
Marketing Research (1)
Mixed Integer Optimization (1)
Network Optimization (1)
Omni Channel (1)
Production Planning (1)
Return on Assets (1)
Rough Cut Capacity Planning (1)
SCOR Model (1)
Supply Chain Network Optimization (1)
Supply Chain Talent (1)
TPO (1)
Theory of Constraints (1)
control tower (1)
decision support system (1)
decision support system example (1)
forest products (1)
product portfolio (1)
wood products (1)
working capital (1)
COMMODITIES TRADING & LOGISTICS
Supply Chain Network Design
© 2019 River Logic, Inc. All rights reserved.
Enterprise Optimizer® is a registered trademark of River Logic, Inc.
8150 N.Central Expressway, Suite M2025

redpajama_set_name: RedPajamaCommonCrawl


Unique property! 3 bed/2bath fully renovated home along with 2 legal rental units for extra income!! Too many great features. A must see property! *one apartment currently rented, home and the 2nd apartment are vacant.


Suppliers:Guangzhou Sainayasi Electronic Technology Co., Ltd.
Suppliers:Shenzhen LES Technology Co., Ltd.
Suppliers:Guangzhou Konmison Electronic Technology Co., Ltd.
1-9 shift setting of rotating speeding.
Suppliers:Xiamen Best Fortune Import & Export Co., Ltd.


Download Star Wars The Last from ZippyShare, Uploaded, Torrent & Direct Download. View the links and download below.
Full Download Results For "Star Wars The Last"
| Download Star Wars The Last Full Version Now!
| Download Star Wars The Last Full Download Now!
| Download Star Wars The Last Newest Release With Updates Now!
| Download Star Wars The Last From Highspeed Server Now!
Warez Download Results For "Star Wars The Last"
Download Star Wars The Last Via File Download Club Now!
Download Star Wars The Last Via Full Software Download Now!
If your are not satisfied with the results for Star Wars The Last, please use one part of the name only to find better results.
Please avoid common search terms such as "Star Wars The Last Crack", "Star Wars The Last Serial", "Star Wars The Last Keygen", "Star Wars The Last Warez", "Star Wars The Last ZippyShare", "Star Wars The Last Uploaded", "Star Wars The Last MediaFire", "Star Wars The Last Full Version".


text:
Les Ilergetes étaient l'un des peuples qui occupaient une partie de la péninsule Ibérique avant l'arrivée des Romains. Formés à partir du substrat ethnique indigène, ils incorporèrent des influences multiples provenant de l'âge du bronze et de certaines des tribus indo-européennes qui immigrèrent dans la péninsule dans le  millénaire  (âge du fer).

Situation géographique
Ils se situaient dans une partie de ce qu'on appellera plus tard la Tarraconaise, de la Basse-Urgel à la rivière Èbre, qui occupe aujourd'hui les provinces de Huesca et Lérida, tout comme les bassins riches du fleuve Sègre, de la rivière Noguera Pallaresa, du Noguera Ribagorzana, du Cinca et de l'Alcanadre, mais il reste inapproprié de tenter d'établir les limites définies à l'époque.

Situation politique
La société politique était fondée sur l'existence d'un Roi, cette société était très hiérarchisée avec des oligarques qui maintenaient la structure sociale et un fort composant militaire. Il s'agissait d'une société très avancée à l'arrivée des Romains, peuplée de bons commerçants. Quelques grandes villes semblaient disposer d'une certaine autonomie avec des organes de gouvernement semblable à la Curie à partir du , bien que celles-ci puissent avoir été influencées par le processus de romanisation accentué à partir de 195

La capitale de ce peuple était Atanagrum, dont l'emplacement exact n'a pas été identifié jusqu'aux travaux archéologiques les plus récents. Une autre ville importante de ce peuple était Ilerda, également appelée Iltirta ou Iltirda, (maintenant Lleida-Lérida), et à un certain moment, cette cité a été la capitale des Ilergetes, car elle était considérée à certaines périodes comme la plus grande ville sur la péninsule Ibérique au nord de Sagonte.

Situation économique
L'économie des Ilergetes était basée sur l'agriculture et sur la culture des céréales. D'ailleurs, des moulins manuels tournants ont été retrouvés datant de cette période. Le développement de la métallurgie, de l'orfèvrerie et de l'industrie textile a été également très important chez ce peuple. Du côté de la céramique, des composants phéniciens et grecs avec des motifs ornementaux géométriques ont été trouvés grâce à l'archéologie dans la zone géographique des Ilergetes. Ce peuple utilisait également la monnaie, en bronze (as) et en argent (drachme), au moins dès le , ce qui a favorisé le commerce et la perception des impôts.

Situation historique

Sa figure la plus remarquable a été le roi Indibilis (258  - 205 ) qui a d'abord été allié de Carthage ; il a d'ailleurs fait partie de divers affrontements contre les Romains durant la deuxième guerre punique. La situation stratégique du territoire des Ilergetes a favorisé les alliances avec Rome ou Carthage (presque toujours d'ailleurs avec cette dernière), bien que l'on affirme actuellement et d'une manière unanime que l'objectif des Ilergetes et de ses chefs était de favoriser les propres intérêts de ce peuple, de défendre ses villes et de s'étendre au-delà de ses frontières en conquérant les peuples voisins en marge des deux grandes puissances de la Méditerranée occidentale du moment.

En plus des batailles où Indibilis et Mandonius (ce dernier étant le lieutenant d'Indibilis) sont intervenus, les Ilergetes se sont encore élevés en 195  contre Rome et ont dû être soumis par Caton. À partir de ce moment, les Ilergetes entrent dans une décadence vers 192 , et ils doivent demander à Rome de l'aide pour se défendre contre leurs voisins, car ce peuple avaient ses rois soumis à une vassalité romaine et les défenses des villes étaient assurées par les Romains.

Situation archéologique
Le gisement archéologique d'une installation ilergete qui nous a offert le plus d'informations sur l'histoire de ce peuple (en plus des œuvres des auteurs antiques romains) est le site connu sous le nom de La Pedrera de Vallfogona.

Voir aussi

Articles connexes
 Hispanie romaine
 Histoire de l'Aragon
 Histoire de la Catalogne
 Ibères
 Indibilis
 Mandonius

Notes et références

Bibliographie
  Martín Almagro et autres, Protohistoria de la Península Ibérica. Madrid, 2001.
  Sanmartí, J.&Santacana, J. Els Ibers del Nord, Rafael Dalmau, éd. Barcelona, 2005.
 .

Liens externes

  Carte détaillé des populations de la péninsule ibérique avant la domination romaine (vers 200
  Ilergetes, enciclopedia-aragonesa.

Peuple ancien
Peuple pré-romain de Catalogne
Peuple ibère
Histoire de l'Aragon
Histoire de la Catalogne
Histoire de Lérida
Histoire de l'Espagne antique

redpajama_set_name: RedPajamaWikipedia


text:
Q: Reshape data for histograms (multhist) I want to convert my data to plot different histograms next to each other (probably with multhist).
My data now looks like:
    Code Pt
     0   5
     0   0
     0   2
     1   0
     1   3
     1   1
    30   0
    30   6
    30   2

and so on.
Each code number stands for one species and I would like to plot histograms of Pt for each species (code number) next to each other.
Therefore, I want the data to be reshaped like:
    Code Pt
     0   5
     0   5
     0   5
     0   5
     0   5
     0   0
     0   2
     0   2
     1   0
     1   3
     1   3
     1   3

and so on.
I have already tried
    hp<-read.table("C:/Users/file.csv", header=T, sep=";")
    hp$Code<-as.factor(hp$Code)
    with(hp,data.frame(
    Code=rep(Code,Pt),Ptdet=rep(Pt,Code)))

But I get the error: arguments imply differing number of rows. Maybe this is due to the "0" in my data frame?
Probably it is an easy thing, nevertheless I am struggeling and I appreciate any help.

A: You can try:
library(data.table)
setDT(df)[,rep(Pt, max(Pt,1)),by=.(Code, Pt)][,.(Code, Pt)]

#    Code Pt
# 1:    0  5
# 2:    0  5
# 3:    0  5
# 4:    0  5
# 5:    0  5
# 6:    0  0
# 7:    0  2
# 8:    0  2
# 9:    1  0
#10:    1  3
#11:    1  3
#12:    1  3
#13:    1  1
#14:   30  0
#15:   30  6
#16:   30  6
#17:   30  6
#18:   30  6
#19:   30  6
#20:   30  6
#21:   30  2
#22:   30  2

redpajama_set_name: RedPajamaStackExchange


Capture Server requires additional configuration when installed in an Azure VM. If all of the servers in your farm are located on-premises, skip these instructions and reference the On-Premises Prerequisite Article.
Add a firewall exception for the Capture Server ports.
On the Azure VM, open Windows Firewall with Advanced Security.
On the Actions panel, click New Rule.
On the New Inbound Rule Wizard, select Port and then click Next.
On the Protocol and Ports page, select TCP.
Select Specific local ports, type "7210-7216" or your range of custom ports, and then click Next.
On the Action page, select Allow the connection, and then click Next.
On the Profile page, select Domain, Private, and Public, and then click Next.
On the name page, type a Name and a Description for the rule, and then click Finish.
Add Azure VM endpoints for Capture Server Apps and PowerShell.
In the Virtual Machines section, select the Capture Server VM.
In the VM's Endpoints section, click the Add button at the bottom of the page.
On the Add Endpoint wizard, select Stand-Alone Endpoint and then click Continue.
Type "Apps" in the NAME box, select TCP in the PROTOCOL box, type "7210" in the PUBLIC PORT box and the PRIVATE PORT box, and then click Complete.
Select "PowerShell" in the NAME box, keep the default values in all of the other boxes, and then click Complete.
Ensure network traffic is allowed between the Azure VM and any on-premises servers in the farm.