For the model with context features, I used the following to create the index and query it.
(Hope it’s correct)
index = tfrs.layers.factorized_top_k.BruteForce(model.query_model)
index.index_from_dataset(
    candidates.batch(100).map(lambda c: (c['candidate_id'], model.candidate_model(c)))
    )

query = {"user_id": np.array(['42'])} # the query is a hash in this case. i'm not sure why the user id needs to be a numpy array though :(
_scores, titles = index(query)
titles[0].numpy().tolist()