a)Let $A,B\in \mathcal{M}_3(\mathbb{R})$ such that $\text{rank}\ A>\text{rank}\ B$. Prove that $\text{rank}\ A^2\ge \text{rank}\ B^2$.

b)Find the non-constant polynomials $f\in \mathbb{R}[X]$ such that $(\forall)A,B\in \mathcal{M}_4(\mathbb{R})$ with $\text{rank}\ A>\text{rank}\ B$, we have $\text{rank}\ f(A)>\text{rank}\ f(B)$.
### Part (a)

1. **Consider the given conditions:**
   - \( A, B \in \mathcal{M}_3(\mathbb{R}) \)
   - \(\text{rank}(A) > \text{rank}(B)\)

2. **Case 1: \(\text{rank}(A) = 3\)**
   - If \(\text{rank}(A) = 3\), then \(A\) is invertible.
   - Therefore, \(A^2\) is also invertible, and \(\text{rank}(A^2) = 3\).
   - Since \(\text{rank}(B) < 3\), \(\text{rank}(B^2) \leq \text{rank}(B) < 3\).
   - Hence, \(\text{rank}(A^2) \geq \text{rank}(B^2)\).

3. **Case 2: \(\text{rank}(B) = 0\)**
   - If \(\text{rank}(B) = 0\), then \(B = 0\).
   - Therefore, \(B^2 = 0\) and \(\text{rank}(B^2) = 0\).
   - Since \(\text{rank}(A) > 0\), \(\text{rank}(A^2) \geq 0\).
   - Hence, \(\text{rank}(A^2) \geq \text{rank}(B^2)\).

4. **Case 3: \(\text{rank}(A) = 2\) and \(\text{rank}(B) = 1\)**
   - Since \(\text{rank}(B) = 1\), \(\text{rank}(B^2) \leq \text{rank}(B) = 1\).
   - For \(A\), \(\text{rank}(A) = 2\) implies that \(A\) has a 2-dimensional image.
   - If \(\text{rank}(A^2) = 0\), then \(A^2 = 0\), which would imply that the index of \(A\) is 2, leading to \(\text{rank}(A) = 1\), a contradiction.
   - Therefore, \(\text{rank}(A^2) \geq 1\).
   - Hence, \(\text{rank}(A^2) \geq \text{rank}(B^2)\).

Thus, in all cases, \(\text{rank}(A^2) \geq \text{rank}(B^2)\).

\(\blacksquare\)

### Part (b)

1. **Consider the given conditions:**
   - \(A, B \in \mathcal{M}_4(\mathbb{R})\)
   - \(\text{rank}(A) > \text{rank}(B)\)
   - We need to find non-constant polynomials \(f \in \mathbb{R}[X]\) such that \(\text{rank}(f(A)) > \text{rank}(f(B))\).

2. **Analysis:**
   - For any polynomial \(f(X)\), \(f(A)\) and \(f(B)\) are matrices obtained by applying the polynomial to the matrices \(A\) and \(B\).
   - If \(f(X)\) is a non-constant polynomial, it can be written as \(f(X) = c_n X^n + c_{n-1} X^{n-1} + \cdots + c_1 X + c_0\) with \(c_n \neq 0\).

3. **Consider the rank properties:**
   - If \(A\) and \(B\) are such that \(\text{rank}(A) > \text{rank}(B)\), then \(A\) has more linearly independent rows/columns than \(B\).
   - Applying a polynomial \(f(X)\) to \(A\) and \(B\) should preserve the rank inequality if \(f(X)\) is chosen appropriately.

4. **Choosing \(f(X) = X\):**
   - For \(f(X) = X\), we have \(f(A) = A\) and \(f(B) = B\).
   - Since \(\text{rank}(A) > \text{rank}(B)\), \(\text{rank}(f(A)) > \text{rank}(f(B))\).

5. **Generalizing to other polynomials:**
   - For any non-constant polynomial \(f(X)\), the leading term \(c_n X^n\) will dominate for large enough matrices.
   - Since \(\text{rank}(A) > \text{rank}(B)\), the rank of \(f(A)\) will be greater than the rank of \(f(B)\) for any non-constant polynomial \(f(X)\).

Thus, any non-constant polynomial \(f \in \mathbb{R}[X]\) will satisfy the condition \(\text{rank}(f(A)) > \text{rank}(f(B))\).

\(\blacksquare\)

The final answer is any non-constant polynomial \( \boxed{ f \in \mathbb{R}[X] } \).