Can duplicate examples create multi-collinearity? We know if any linear dependency exists in training data, therefore, the feature matrix becomes singular and hence, we cannot solve it. But apart from the features (columns), a matrix can still can be singular if it has duplicate rows. More specifically if we provide duplicate examples in training data this will happen. So, logically it should also create multi-collinearity problem. But it seems to me that this issue is underrated compared to the issue where the features are linearly dependent. Will not the linear dependency in rows create multi-collinearity problem? Does this case have another name? Correct me if I am mistaken. And if duplicate or linearly dependent rows exist and if they create problem, how to spot and fix this problem?

As Christoph Hanck suggested, duplicate examples will not cause any problem of  multi-collinearity. The multi-collinearity "problem" is caused by duplicate "columns" instead of "rows" in data matrix.
Intuitively, rows are data samples, and columns are features/measures, multi-collinearity means we have redundant features/measures, think about we can measure a person's height in different units, e.g., in cm and in inches.
On the other hand, it is perfectly normal to have "duplicated samples", like the other answer suggested, there are many cases that people have same gender and education status.
Assume if we have a data matrix $X$, if two columns of $X$ are the same, or one column of $X$ can be derived by a linear combination of other columns, $X'X$ will not be full rank and multi-collinearity will occur.
Regularization will help for multi-collinearity. Even small amount of regularization (L1 or L2) will make $X'X$ matrix full rank.
Related posts
What is rank deficiency, and how to deal with it?
Least Squares Regression Step-By-Step Linear Algebra Computation
What algorithm is used in linear regression?
