NE-F1

Sensitive-F1

ROUGE

ROUGE-1 ROUGE-L BLEU-1 BLEU-2 BLEU-4

+date+msg_id+subject
+date+subject
+date+msg_id
+msg_id+subject
+date
+msg_id
+subject

3.61
3.77
2.58
3.18
2.73
2.52
3.13

2.44
2.65
2.71
2.02
2.39
1.92
2.46

12.35
13.34
11.98
12.96
12.58
11.79
12.26

7.95
8.70
7.55
8.27
8.02
7.65
7.94

6.93
7.47
6.98
7.31
6.79
7.04
7.09

BLEU

1.48
1.41
1.04
1.40
0.98
1.21
1.52

0.14
0.36
0.00
0.06
0.05
0.00
0.21

Table 7: The ablation study on email content recovery. All results are measured in %. For each email, we combine
the email addresses of its sender and receiver with a subset of {date, msg_id, subject} as queried indentifers.
Multi-step Jailbreaking Privacy Attacks on ChatGPT