Question: **Rules**

1. Place some pentominoes into an 8 x 8 grid. They do not touch each other. They can touch only diagonally (with corner).
2. Pentominoes cannot repeat in the grid. Rotations and reflections of a pentomino are considered the same shape.
3. Grid is 8 x 8.

Answer: 
With integer programming, I managed to place

> 
>  8 pieces, proved to be optimal
> 
> 
> 

like this.

> 
> $$\begin{array}{cccccccc} 3&3&3&3& &5&5&5\\ 3& & & & &5& &5\\ &4&4&4&4& &A&\\ 6& & &4& &A&A&A\\ 6&6&6& &8& &A&\\ & &6& &8&8& &B\\ 2&2& &8&8& &B&B\\2&2&2& & &B&B&\\ \end{array}$$
> 
> 
> 

Here is my formulation. I happened to solve a similar model to solve a puzzle called One puzzle a day. Let $B$ be the set of cells in the 8x8 board, and $P$ be the set of all kinds of pieces (rotating and flipping count difference here), so I have $|P|=63$, then we need to choose a subset of $P$ and place them only by translation. Let binary variable $x\_{pb}$ indicate whether the piece $p$ is placed on the cell $b$ (some predefined reference point on the piece being on $b$). Let the set $P\_i\subset P$ represent all flipped and rotated version of a same pentomino, and binary variable $y\_i$ indicates whether the pentomino $i$ is placed on the board, so we have
$$
y\_i = \sum\_{p\in P\_i} \sum\_{b\in B} x\_{pb}
$$
and the objective is to maximize $\sum\_i y\_i$.

Putting the reference point on some cells make the tile out of the board, let that set be $F\_p$, we have
$$
x\_{pb} = 0 \quad\forall b \in F\_p.
$$

Now the main part. Actually in every cell $b$, we could find all possible $(p,b')$ that will cover the cell $b$, say the set is $Covered(b)$. By constraining $\sum\_{Covered(b)} x\_{pb'} \leq 1$, we could prevent overlap. To prevent neighboring, I build sets $Edge(b) = \{(p, b')\}$ similarly, which means if putting $p$ in $b'$, the cell $b$ will be on the edge of that piece. Then we need some constraints like
$$
\alpha \sum\_{(p,b')\in Covered(b)} x\_{pb'} + \beta \sum\_{(p,b') \in Edge(b)} x\_{pb'} \leq \delta
$$

The parameters $\alpha, \beta, \delta >0$should satisfy that $\alpha \leq \delta$, $4\beta \leq \delta$, $\alpha + \beta > \delta$, $2\alpha > \delta$, meaning that "one cover is allowed", "four edges on the same cell is allowed", "one cover and one edge is not allowed" (that is when two pieces are next to each other) and "no overlap".
I chose $\alpha=\delta=1, \beta=0.25$ (actually any $\beta > 0$ works...).

That is all constraints we need.
I can manage

> 
>  seven pentominoes, in a few different ways:  
> 
>   
> 
> [![seven pentominoes](https://i.stack.imgur.com/FPIEH.png)](https://i.stack.imgur.com/FPIEH.png) [![seven again](https://i.stack.imgur.com/rVcqQ.png)](https://i.stack.imgur.com/rVcqQ.png)
> 
> 
> 

There is an obvious upper bound of

> 
>  twelve (the total number of distinct pentominoes, and $12\times5$ is still less than the number of squares on the board),
> 
> 
> 

but this is nowhere near achievable given the restriction that pentominoes are not allowed to touch each other orthogonally. This means that

> 
>  every placed pentomino creates at least 4 squares around it that no other pentomino can occupy. *At least* 4, in the best possible scenario where the pentomino is placed in a corner - usually more. (I think 4 is only achievable with the P-pentomino in the corner and 5 is only achievable with the symmetric L-pentomino in the corner, otherwise it's more.)
> 
> 
> 

Because of this, I'm pretty sure the number I managed above is optimal and it's not possible to place more.