Figure 9: Prompt used to generate interview-style programming questions.

G.2 Evaluation prompts

32

Prompt:
[INST] Your task is to write 5 tests to check the correctness of a function that solves a programming
problem.
The tests must be between [TESTS] and [/TESTS] tags.
You must write the comment "#Test case n:" on a separate line directly above each assert statement,
where n represents the test case number, starting from 1 and increasing by one for each subsequent
test case.

Problem: Write a Python function to get the unique elements of a list.
[/INST]
[TESTS]
# Test case 1:
assert get_unique_elements([]) == []
# Test case 2:
assert get_unique_elements([1]) == [1]
# Test case 3:
assert get_unique_elements([1, 2, 3, 2, 1]) == [1, 2, 3]
[/TESTS]

[INST] Problem: %%%question%%%
[/INST]

Figure 10: Prompt template used to generate unit tests. The substring %%%question%%% is a placeholder
for an interview-style programming question we replace at runtime.
CodeLlama2