Since the time I started solving problems in physics nearly 5 years ago, the most common mistake I've seen people make is trying to introduce 'signs'(into equations). The positive or negative value is purely a matter of convention, I.e., it is based on the coordinate system chosen by you. So I suggest that instead of trying to give the work done by friction a negative sign, just use your axes to give signs to all the forces, displacements and velocities and leave the unknowns with a positive sign. The values of work done and energies(kinetic and potential) will then have signs accordingly.
EDIT: I'm sorry, but I hadn't read your question completely, so I will endeavour to answer your question as completely as possible. I strongly believe that instead of directly taking equations from an outside source, they should derive the equations themselves. Now the equation given to you by your professor is derived from the most fundamental law of physics, The Law of Conservation of Energy. It states that the sum of energies of the initial state is equal to the sum of energies in the final state. In case we bring in forces other than the gravitational force, such as friction, we have another term relating to the work done to overcome friction. Now the term $f_k$h relates to the done in overcoming friction as the first mass slides along the table. So the law of conservation of energy for the above case can be written as: $KE_1$ + $PE_1$ + $W_f$ = $KE_2$ + $PE_2$
Here, the state 1 refers to the initial position while the state 2 refers to the position of the system after the second mass has moved by 1.4 m vertically.
A small clarification of the work done against friction $W_f$. 499
By definiton, the force of friction acts against the direction of motion. Just keep this in mind.
Now let us set our coordinate system. As the pulley is stationary, the centre of the pulley is chosen as the origin. The direction to its right is the positive x-direction while the direction going above it is the positive y. Also the mass moving horizontally is chosen as $m_1$ while the other mass is $m_2$. Now as both the masses are connected by the same rope, their distances and displacements are the same. Hence if $m_2$ moves along the negative y direction by 1.4 m, the mass $m_1$ moves by 1.4 m along the positive x direction. Also let the mass $m_2$ be at a distance of y m from the pulley. Let us assume the coefficient of friction $\mu$ = 0.3.
We then move on to the calculation of the various terms of the above mentioned equation.
$KE_1$ = 0.5$m_1$$v_1^2$ + 0.5$m_2$$v_1^2$
$v_1$ = 0 as the masses are at rest initially. Hence, $KE_1$ = 0
$PE_1$ = $m_1$g(0) + $m_2$g(-y) = -$m_2$gy = -47.4y J
$W_f$ = $F_f$(1.4)
$F_f$ = -$\mu m_1 g$ as the force friction is in the opposite direction to displacement and hence in the negative x direction.
$W_f$ = -$\mu m_1 g$(1.4) = -11.53 J
$KE_2$ = 0.5$m_1$$v_2^2$ + 0.5$m_2$$v_2^2$ = 0.5$v_2^2$($m_1 + m_2$) = 3.8$v_2^2$ J
$PE_2$ = $m_1$g(0) + $m_2$g(-y - 1.4) = -47.4y - 65.86 J
Now if we plug in the above values into the law of conservation of energy equation, we get-
0 - 47.4y - 11.53 = 3.8$v_2^2$ - 47.4y - 65.86
Simplifying, we get-
3.8$v_2^2$ = 65.86 - 11.53 = 54.33
and $v_2$ = $\sqrt(\frac{54.33}{3.8})$ = 3. 512