# Dual norm

In functional analysis, the dual norm is a measure of the "size" of each continuous linear functional defined on a normed vector space.

## Definition

Let $X$  be a normed vector space with norm $|\cdot |$  and let $X^{*}$  be the dual space. The dual norm of a continuous linear functional $f$  belonging to $X^{*}$  is defined to be the real number

$\|f\|:=\sup\{|f(x)|:x\in X,|x|\leq 1\}$

where $\sup$  denotes the supremum.

The map $f\mapsto \|f\|$  defines a norm on $X^{*}$ . (See Theorems 1 and 2 below.)

The dual norm is a special case of the operator norm defined for each (bounded) linear map between normed vector spaces.

The topology on $X^{*}$  induced by $|\cdot |$  turns out to be as strong as the weak-* topology on $X^{*}$ .

If the ground field of $X$  is complete then $X^{*}$  is a Banach space.

## The double dual of a normed linear space

The double dual (or second dual) $X^{**}$  of $X$  is the dual of the normed vector space $X^{*}$ . There is a natural map $\varphi :X\to X^{**}$ . Indeed, for each $w^{*}$  in $X^{*}$  define

$\varphi (v)(w^{*}):=w^{*}(v).$

The map $\varphi$  is linear, injective, and distance preserving. In particular, if $X$  is complete (i.e. a Banach space), then $\varphi$  is an isometry onto a closed subspace of $X^{**}$ .

In general, the map $\varphi$  is not surjective. For example, if $X$  is the Banach space $L^{\infty }$  consisting of bounded functions on the real line with the supremum norm, then the map $\varphi$  is not surjective. (See $L^{p}$  space). If $\varphi$  is surjective, then $X$  is said to be a reflexive Banach space. If $1  then the space $L^{p}$  is a reflexive Banach space.

## Mathematical Optimization

Let $\|\cdot \|$  be a norm on $\mathbb {R} ^{n}.$  The associated dual norm, denoted $\|\cdot \|_{*},$  is defined as

$\|z\|_{*}=\sup\{z^{\intercal }x\;|\;\|x\|\leq 1\}.$

(This can be shown to be a norm.) The dual norm can be interpreted as the operator norm of $z^{\intercal }$ , interpreted as a $1\times n$  matrix, with the norm $\|\cdot \|$  on $\mathbb {R} ^{n}$ , and the absolute value on $\mathbb {R}$ :

$\|z\|_{*}=\sup\{|z^{\intercal }x|\;|\;\|x\|\leq 1\}.$

From the definition of dual norm we have the inequality

$z^{\intercal }x=\|x\|\left(z^{\intercal }{\frac {x}{\|x\|}}\right)\leq \|x\|\|z\|_{*}$

which holds for all x and z. The dual of the dual norm is the original norm: we have $\|x\|_{**}=\|x\|$  for all x. (This need not hold in infinite-dimensional vector spaces.)

The dual of the Euclidean norm is the Euclidean norm, since

$\sup\{z^{\intercal }x\;|\;\|x\|_{2}\leq 1\}=\|z\|_{2}.$

(This follows from the Cauchy–Schwarz inequality; for nonzero z, the value of x that maximises $z^{\intercal }x$  over $\|x\|_{2}\leq 1$  is ${\tfrac {z}{\|z\|_{2}}}$ .)

The dual of the $\ell _{\infty }$ -norm is the $\ell _{1}$ -norm:

$\sup\{z^{\intercal }x\;|\;\|x\|_{\infty }\leq 1\}=\sum _{i=1}^{n}|z_{i}|=\|z\|_{1},$

and the dual of the $\ell _{1}$ -norm is the $\ell _{\infty }$ -norm.

More generally, Hölder's inequality shows that the dual of the $\ell _{p}$ -norm is the $\ell _{q}$ -norm, where, q satisfies ${\tfrac {1}{p}}+{\tfrac {1}{q}}=1$ , i.e., $q={\tfrac {p}{p-1}}.$

As another example, consider the $\ell _{2}$ - or spectral norm on $\mathbb {R} ^{m\times n}$ . The associated dual norm is

$\|Z\|_{2*}=\sup\{\mathrm {\bf {tr}} (Z^{\intercal }X)|\|X\|_{2}\leq 1\},$

which turns out to be the sum of the singular values,

$\|Z\|_{2*}=\sigma _{1}(Z)+\cdots +\sigma _{r}(Z)=\mathrm {\bf {tr}} ({\sqrt {Z^{\intercal }Z}}),$

where $r=\mathrm {\bf {rank}} Z.$  This norm is sometimes called the nuclear norm.

## Examples

### Dual norm for matrices

The Frobenius norm defined by

$\|A\|_{\text{F}}={\sqrt {\sum _{i=1}^{m}\sum _{j=1}^{n}\left|a_{ij}\right|^{2}}}={\sqrt {\operatorname {trace} (A^{*}A)}}={\sqrt {\sum _{i=1}^{\min\{m,n\}}\sigma _{i}^{2}}}$

is self-dual, i.e., its dual norm is $\|\cdot \|'_{\text{F}}=\|\cdot \|_{\text{F}}.$

The spectral norm, a special case of the induced norm when $p=2$ , is defined by the maximum singular values of a matrix, i.e.,

$\|A\|_{2}=\sigma _{\max }(A),$

has the nuclear norm as its dual norm, which is defined by

$\|B\|'_{2}=\sum _{i}\sigma _{i}(B),$

for any matrix $B$  where $\sigma _{i}(B)$  denote the singular values[citation needed].

## Some basic results about the operator norm

More generally, let $X$  and $Y$  be topological vector spaces, and $L(X,Y)$  be the collection of all bounded linear mappings (or operators) of $X$  into $Y$ . In the case where $X$  and $Y$  are normed vector spaces, $L(X,Y)$  can be normed in a natural way.

Theorem 1. Let $X$  and $Y$  be normed spaces, and associate to each $f\in L(X,Y)$  the number:
$\|f\|=\sup\{|f(x)|:x\in X,\|x\|\leq 1\}.$
This turns $L(X,Y)$  into a normed space. Moreover if $Y$  is a Banach space, so is $L(X,Y)$ .

Proof. A subset of a normed space is bounded if and only if it lies in some multiple of the unit sphere; thus $\|f\|<\infty$  for every $f\in L(X,Y)$  if $\alpha$  is a scalar, then $(\alpha f)(x)=\alpha \cdot fx$  so that

$\|\alpha f\|=|\alpha |\|f\|$

The triangle inequality in $Y$  shows that

{\begin{aligned}\|(f_{1}+f_{2})x\|&=\|f_{1}x+f_{2}x\|\\&\leq \|f_{1}x\|+\|f_{2}x\|\\&\leq (\|f_{1}\|+\|f_{2}\|)\|x\|\\&\leq \|f_{1}\|+\|f_{2}\|\end{aligned}}

for every $x\in X$  with $\|x\|\leq 1$ . Thus

$\|f_{1}+f_{2}\|\leq \|f_{1}\|+\|f_{2}\|$

If $f\neq 0$ , then $fx\neq 0$  for some $x\in X$ ; hence $\|f\|>0$ . Thus, $L(X,Y)$  is a normed space.

Assume now that $Y$  is complete, and that $\{f_{n}\}$  is a Cauchy sequence in $L(X,Y)$ . Since

$\|f_{n}x-f_{m}x\|\leq \|f_{n}-f_{m}\|\|x\|$

and it is assumed that $\|f_{n}-f_{m}\|\to 0$  as $n,m\to \infty$ , $\{f_{n}x\}$  is a Cauchy sequence in $Y$  for every $x\in X$ . Hence

$fx=\lim _{n\to \infty }f_{n}x$

exists. It is clear that $f:X\to Y$  is linear. If $\varepsilon >0$ , $\|f_{n}-f_{m}\|\|x\|\leq \varepsilon \|x\|$  for sufficiently large n and m. It follows

$\|fx-f_{m}x\|\leq \varepsilon \|x\|$

for sufficiently large m. Hence $\|fx\|\leq (\|f_{m}\|+\varepsilon )\|x\|$ , so that $f\in L(X,Y)$  and $\|f-f_{m}\|\leq \varepsilon$ . Thus $f_{m}\to f$  in the norm of $L(X,Y)$ . This establishes the completeness of $L(X,Y).$ 

When $Y$  is a scalar field (i.e. $Y=\mathbb {C}$  or $Y=\mathbb {R}$ ) so that $L(X,Y)$  is the dual space $X^{*}$  of $X$ .

Theorem 2. Suppose $B$  is the closed unit ball of normed space $X$ . For every $x^{*}\in X^{*}$  define:
$\|x^{*}\|=\sup\{|\langle {x,x^{*}}\rangle |:x\in B\}$
Then
(a) This norm makes $X^{*}$  into a Banach space.
(b) Let $B^{*}$  be the closed unit ball of $X^{*}$ . For every $x\in X$ ,
$\|x\|=\sup\{|\langle {x,x^{*}}\rangle |:x^{*}\in B^{*}\}.$
Consequently, $x^{*}\to \langle {x,x^{*}}\rangle$  is a bounded linear functional on $X^{*}$  of norm $\|x\|$ .
(c) $B^{*}$  is weak*-compact.

Proof. Since $L(X,Y)=X^{*}$ , when $Y$  is the scalar field, (a) is a corollary of Theorem 1. Fix $x\in X$ . There exists $y^{*}\in B^{*}$  such that

$\langle {x,y^{*}}\rangle =\|x\|.$

but,

$|\langle {x,x^{*}}\rangle |\leq \|x\|\|x^{*}\|\leq \|x\|$

for every $x^{*}\in B^{*}$ . (b) follows from the above. Since the open unit ball $U$  of $X$  is dense in $B$ , the definition of $\|x^{*}\|$  shows that $x^{*}\in B^{*}$  if and only if $|\langle {x,x^{*}}\rangle |\leq 1$  for every $x\in U$ . The proof for (c) now follows directly.