I am building a turn-based strategy game using Libtcod and Python. The game map has variable terrain and each tile can be 1 of 5 types:

Plains - Costs 1 to move over
Forest - Costs 2
River - Costs 4
Hill - Costs 3
Mountain - Impassable

Each type has its own movement cost, so that it costs less "move points" to move through plains than through a forest, for example. I want to display all the squares a unit can move to given its movement range/starting move points. 
Libtcod has pathfinding functionality built for both A* and Dijtskra and it is trivial to display all the squares in a given range, without accounting for terrain. 
However, I cannot figure out how I can implement the terrain costs, without having to write my own pathfinding algorithm. Looking at the docs I know it has something to do with:
def path_func(xFrom,yFrom,xTo,yTo,userData) : ...
path_new_using_function(width, height, path_func, user_data=0, diagonalCost=1.41)
dijkstra_new_using_function(width, height, path_func, user_data=0,  diagonalCost=1.41)
but I cannot figure out what the custom function is supposed to do. According to the docs, it supposed to 

...return the walk cost from coordinates xFrom,yFrom to coordinates
  xTo,yTo. The cost must be > 0.0f if the cell xTo,yTo is walkable. It
  must be equal to 0.0f if it's not.

but, isn't that the point of the dijtskra algorithm to begin with? That is, the algorithm is supposed to take into account the variable costs of each tile and then build a path accordingly. 
The map itself already has the terrain and move costs applied to it, I just need a way to bridge that data with the pathfinding.
As far as I understand, what you want to achieve is very well possible with tcod's built-in pathfinding function.
path_new_using_function will call your path_func with adjacent cells, so you can simply have it return the values you listed above depending on the terrain below (xFrom, yFrom) and/or (xTo, yTo).
Python Libtcod: How to do pathfinding with variable move cost terrain?