Given a list of words, construct a python dictionary of all the words with their length as the value.
words = ["apple", "strawberry", "coffee"]
word_lengths = {
    "apple": 5,
    "strawberry": 10,
    "coffee": 6
}