A voltage source acts as an electron pump. Suppose we take a battery as an example, then in the battery a chemical reaction pumps electrons from the cathode to the anode. The cathode becomes depleted in electrons and becomes positive while the anode acquires an excess of electrons and becomes negative. As soon as enough electrons have been pumped for the resulting potential difference to equal the reaction potential the reaction stops.
The point of all this is that suppose we attach some conductors to the battery in the way you describe. We'll make them spheres for simplicity:
If a charge $Q$ is transferred then the voltage on conductor A is:
$$ V_A = \frac{Q}{C_A}$$
where $C_A$ is the capacitance of conductor A, and likewise:
$$ V_B = -\frac{Q}{C_B}$$
At equilibrium the potential difference $V_A-V_B$ will be equal to the battery voltage so:
$$ V_\text{batt} = Q\left(\frac{1}{C_A} + \frac{1}{C_B}\right) $$
So given the voltage of the battery and the capacitance of the two conductors we can calculate how much charge is transferred. 531
This will be non-zero, so the conductors will become charged and if you disconnect them from the battery they will keep that charge. So the answer to your question:
After removing the source, will the conductor stay charged from the source if is not earthed?
is yes.
But you need the capacitances on both sides of the voltage source to calculate how much charge is transferred, and you only say what is connected to one side of your 12kV voltage source. If the other side is connected to earth the capacitance is effectively infinite i.e. $1/C_\text{earth}=0$ in our equation above. However if the other side of your 12kV source is not connected to anything the capacitance is close to zero so $1/C\approx\infty$ and plugging this into our equation we gind $Q\approx 0$ i.e. a negligable charge is transferred.
So while it's generally true that in the situation you describe the conductor will end up charged, you haven't given us enough information to say what that charge is. 531