If you would like to use matplotlib and just visualize the figures, one way would be to save then as svg data and send them to visdom.
Using this approach you wouldn’t need to recreate the figures in visdom.
Here is a small example:
import re
import io
from visdom import Visdom

labels = np.random.randint(0, 2, (10,))
scores = np.random.uniform(0, 1, (10,))
precision, recall, thresholds = precision_recall_curve(
    labels, scores)

fig, ax = plt.subplots()
ax.step(recall,precision,color='r',alpha=0.99,where='post')
ax.fill_between(recall, precision, alpha=0.2, color='b', step='post')
plt.xlabel('Recall')
plt.ylabel('Precision')
plt.ylim([0.0, 1.05])
plt.xlim([0.0, 1.0])

viz = Visdom()
imgdata = io.StringIO()
fig.savefig(imgdata, format='svg')

svg_str = imgdata.getvalue()
# Scale the figure
svg_str = re.sub('width=".*pt"', 'width="100%"', svg_str)
svg_str = re.sub('height=".*pt"', 'height="100%"', svg_str)
viz.svg(svg_str)