MikeDoes (Michael Anthony)

posted an update about 3 hours ago

Post

42

Traditional data leak prevention is failing. A new paper has a solution-oriented approach inspired by evolution.

The paper introduces a genetic-algorithm-driven method for detecting data leaks. To prove its effectiveness, the researchers Anatoliy Sachenko, Petro V., Oleg Savenko, Viktor Ostroverkhov, Bogdan Maslyyak from Casimir Pulaski Radom University and others needed a real-world, complex PII dataset. We're proud that the AI4Privacy PII 300k dataset was used as a key benchmark for their experiments.

This is the power of open-source collaboration. We provide complex, real-world data challenges, and brilliant researchers develop and share better solutions to solve them. It's a win for every organization when this research helps pave the way for more adaptive and intelligent Data Loss Prevention systems.

🔗 Read the full paper to see the data and learn how genetic algorithms are making a difference in cybersecurity: https://ceur-ws.org/Vol-4005/paper19.pdf

#OpenSource
#DataPrivacy
#LLM
#Anonymization
#AIsecurity
#HuggingFace
#Ai4Privacy
#Worldslargestopensourceprivacymaskingdataset

reacted to their post with ❤️ 5 days ago

Post

1947

Anonymizing a prompt is half the battle. Reliably de-anonymizing the response is the other.

To build a truly reliable privacy pipeline, you have to test it. A new Master's thesis does just that, and our data was there for every step.

We're excited to showcase this work on handling confidential data in LLM prompts from Nedim Karavdic at Mälardalen University. To build their PII anonymization pipeline, they first trained a custom NER model. We're proud that the Ai4Privacy pii-masking-200k dataset was used as the foundational training data for this critical first step.

But it didn't stop there. The research also used our dataset to create the parallel data needed to train and test the generative "Seek" models for de-anonymization. It's a win-win when our open-source data not only helps build the proposed "better solution" but also helps prove why it's better by enabling a rigorous, data-driven comparison.

🔗 Check out the full thesis for a great deep-dive into building a practical, end-to-end privacy solution: https://www.diva-portal.org/smash/get/diva2:1980696/FULLTEXT01.pdf

#OpenSource
#DataPrivacy
#LLM
#Anonymization
#AIsecurity
#HuggingFace
#Ai4Privacy
#Worldslargestopensourceprivacymaskingdataset

posted an update 6 days ago

Post

1947

Anonymizing a prompt is half the battle. Reliably de-anonymizing the response is the other.

To build a truly reliable privacy pipeline, you have to test it. A new Master's thesis does just that, and our data was there for every step.

We're excited to showcase this work on handling confidential data in LLM prompts from Nedim Karavdic at Mälardalen University. To build their PII anonymization pipeline, they first trained a custom NER model. We're proud that the Ai4Privacy pii-masking-200k dataset was used as the foundational training data for this critical first step.

But it didn't stop there. The research also used our dataset to create the parallel data needed to train and test the generative "Seek" models for de-anonymization. It's a win-win when our open-source data not only helps build the proposed "better solution" but also helps prove why it's better by enabling a rigorous, data-driven comparison.

🔗 Check out the full thesis for a great deep-dive into building a practical, end-to-end privacy solution: https://www.diva-portal.org/smash/get/diva2:1980696/FULLTEXT01.pdf

#OpenSource
#DataPrivacy
#LLM
#Anonymization
#AIsecurity
#HuggingFace
#Ai4Privacy
#Worldslargestopensourceprivacymaskingdataset

reacted to their post with 👀👀 6 days ago

Post

4167

What if an AI agent could be tricked into stealing your data, just by reading a tool's description? A new paper reports it's possible.

The "Attractive Metadata Attack" paper details this stealthy new threat. To measure the real-world impact of their attack, the researchers needed a source of sensitive data for the agent to leak. We're proud that the AI4Privacy corpus was used to create the synthetic user profiles containing standardized PII for their experiments.

This is a perfect win-win. Our open-source data helped researchers Kanghua Mo, 龙昱丞, Zhihao Li from Guangzhou University and The Hong Kong Polytechnic University to not just demonstrate a new attack, but also quantify its potential for harm. This data-driven evidence is what pushes the community to build better, execution-level defenses for AI agents.

🔗 Check out their paper to see how easily an agent's trust in tool metadata could be exploited: https://arxiv.org/pdf/2508.02110

#OpenSource
#DataPrivacy
#LLM
#Anonymization
#AIsecurity
#HuggingFace
#Ai4Privacy
#Worldslargestopensourceprivacymaskingdataset

reacted to their post with 🔥 7 days ago

Post

1280

The tools we use to audit AI for privacy might be easier to fool than we think.

We're highlighting a critical paper that introduces "PoisonM," a novel attack that could make Membership Inference tests unreliable. The direct connection to our work is explicit: the researchers, Neal M., Atul Prakash, Amrita Roy Chowdhury, Ashish Hooda, Kassem Fawaz, Somesh Jha, Zhuohang Li, and Brad Malin used the AI4Privacy dataset as the "canary" dataset in their experiments to test the effectiveness of their attack on realistic, sensitive information.

This is the power of a healthy open-source ecosystem. We provide the foundational data that helps researchers pressure-test our collective assumptions about AI safety. It's a win for everyone when this leads to a more honest conversation about what our tools can and can't do, pushing us all to create better solutions.

🔗 Read the full paper to understand the fundamental flaws in current MI testing: https://arxiv.org/pdf/2506.06003

#OpenSource
#DataPrivacy
#LLM
#Anonymization
#AIsecurity
#HuggingFace
#Ai4Privacy
#Worldslargestopensourceprivacymaskingdataset

reacted to their post with 🔥 7 days ago

Post

4167

What if an AI agent could be tricked into stealing your data, just by reading a tool's description? A new paper reports it's possible.

The "Attractive Metadata Attack" paper details this stealthy new threat. To measure the real-world impact of their attack, the researchers needed a source of sensitive data for the agent to leak. We're proud that the AI4Privacy corpus was used to create the synthetic user profiles containing standardized PII for their experiments.

This is a perfect win-win. Our open-source data helped researchers Kanghua Mo, 龙昱丞, Zhihao Li from Guangzhou University and The Hong Kong Polytechnic University to not just demonstrate a new attack, but also quantify its potential for harm. This data-driven evidence is what pushes the community to build better, execution-level defenses for AI agents.

🔗 Check out their paper to see how easily an agent's trust in tool metadata could be exploited: https://arxiv.org/pdf/2508.02110

#OpenSource
#DataPrivacy
#LLM
#Anonymization
#AIsecurity
#HuggingFace
#Ai4Privacy
#Worldslargestopensourceprivacymaskingdataset

posted an update 7 days ago

Post

1280

The tools we use to audit AI for privacy might be easier to fool than we think.

We're highlighting a critical paper that introduces "PoisonM," a novel attack that could make Membership Inference tests unreliable. The direct connection to our work is explicit: the researchers, Neal M., Atul Prakash, Amrita Roy Chowdhury, Ashish Hooda, Kassem Fawaz, Somesh Jha, Zhuohang Li, and Brad Malin used the AI4Privacy dataset as the "canary" dataset in their experiments to test the effectiveness of their attack on realistic, sensitive information.

This is the power of a healthy open-source ecosystem. We provide the foundational data that helps researchers pressure-test our collective assumptions about AI safety. It's a win for everyone when this leads to a more honest conversation about what our tools can and can't do, pushing us all to create better solutions.

🔗 Read the full paper to understand the fundamental flaws in current MI testing: https://arxiv.org/pdf/2506.06003

#OpenSource
#DataPrivacy
#LLM
#Anonymization
#AIsecurity
#HuggingFace
#Ai4Privacy
#Worldslargestopensourceprivacymaskingdataset

posted an update 10 days ago

Post

4167

What if an AI agent could be tricked into stealing your data, just by reading a tool's description? A new paper reports it's possible.

The "Attractive Metadata Attack" paper details this stealthy new threat. To measure the real-world impact of their attack, the researchers needed a source of sensitive data for the agent to leak. We're proud that the AI4Privacy corpus was used to create the synthetic user profiles containing standardized PII for their experiments.

This is a perfect win-win. Our open-source data helped researchers Kanghua Mo, 龙昱丞, Zhihao Li from Guangzhou University and The Hong Kong Polytechnic University to not just demonstrate a new attack, but also quantify its potential for harm. This data-driven evidence is what pushes the community to build better, execution-level defenses for AI agents.

🔗 Check out their paper to see how easily an agent's trust in tool metadata could be exploited: https://arxiv.org/pdf/2508.02110

#OpenSource
#DataPrivacy
#LLM
#Anonymization
#AIsecurity
#HuggingFace
#Ai4Privacy
#Worldslargestopensourceprivacymaskingdataset

reacted to their post with 👀🚀 13 days ago

Post

1808

How do you protect your prompts without breaking them? You need a smart sanitizer. A new system called Prϵϵmpt shows how.

The first, critical step in their solution is a high-performance Named Entity Recognition (NER) model to find the sensitive data. We're proud to see that these researchers, Amrita Roy Chowdhury, David Glukhov, Divyam Anshumaan, Prasad Chalasani, Nicolas Papernot, Somesh Jha, and Mihir Bellare from the University of Michigan, University of Toronto, University of Wisconsin-Madison, University of California, San Diego - Rady School of Management and Langroid Incorporated fine-tuned their NER model on 10 high-risk categories from the AI4Privacy dataset.

This is a perfect win-win. Our open-source data helps provide the foundation for the critical detection engine, which in turn enables the community to build and test better solutions like Prϵϵmpt's innovative use of encryption and Differential Privacy.

🔗 Check out their paper for a deep dive into a formally private, high-utility prompt sanitizer: https://arxiv.org/pdf/2504.05147

#OpenSource
#DataPrivacy
#LLM
#Anonymization
#AIsecurity
#HuggingFace
#Ai4Privacy
#Worldslargestopensourceprivacymaskingdataset

posted an update 14 days ago

Post

1808

How do you protect your prompts without breaking them? You need a smart sanitizer. A new system called Prϵϵmpt shows how.

The first, critical step in their solution is a high-performance Named Entity Recognition (NER) model to find the sensitive data. We're proud to see that these researchers, Amrita Roy Chowdhury, David Glukhov, Divyam Anshumaan, Prasad Chalasani, Nicolas Papernot, Somesh Jha, and Mihir Bellare from the University of Michigan, University of Toronto, University of Wisconsin-Madison, University of California, San Diego - Rady School of Management and Langroid Incorporated fine-tuned their NER model on 10 high-risk categories from the AI4Privacy dataset.

This is a perfect win-win. Our open-source data helps provide the foundation for the critical detection engine, which in turn enables the community to build and test better solutions like Prϵϵmpt's innovative use of encryption and Differential Privacy.

🔗 Check out their paper for a deep dive into a formally private, high-utility prompt sanitizer: https://arxiv.org/pdf/2504.05147

#OpenSource
#DataPrivacy
#LLM
#Anonymization
#AIsecurity
#HuggingFace
#Ai4Privacy
#Worldslargestopensourceprivacymaskingdataset

reacted to their post with 🔥 21 days ago

Post

3138

Making LLMs fast with KV-cache sharing is great. A new paper reports it's also a huge privacy risk.

That's why we're excited to see the "SafeKV" paper from researchers at the University of Connecticut, Peking University, and others. Their solution-oriented framework selectively shares non-sensitive data while isolating PII. To validate the "Safe" part of their system, they needed a robust, multilingual privacy benchmark.

We're proud that the Ai4Privacy pii-masking dataset was used for this critical evaluation related to privacy.

This is a perfect win-win. Our open-source data enables researchers to build and validate more effective security solutions for core AI infrastructure. Their work, in turn, helps make the entire LLM ecosystem safer, showing that performance and privacy don't have to be mutually exclusive.

Kudos to Kexin Chu, Zecheng Lin, Dawei Xiang, 沈子旭, Jianchang Su, cheng chu, Yiwei Yang, Wenhui Zhang, Wenfei Wu, and Wei Zhang on this beautiful work.

🔗 Check out their paper to see the future of secure, high-performance LLM inference: https://arxiv.org/pdf/2508.08438

#OpenSource
#DataPrivacy
#LLM
#Anonymization
#AIsecurity
#HuggingFace
#Ai4Privacy
#Worldslargestopensourceprivacymaskingdataset

posted an update 21 days ago

Post

3138

Making LLMs fast with KV-cache sharing is great. A new paper reports it's also a huge privacy risk.

That's why we're excited to see the "SafeKV" paper from researchers at the University of Connecticut, Peking University, and others. Their solution-oriented framework selectively shares non-sensitive data while isolating PII. To validate the "Safe" part of their system, they needed a robust, multilingual privacy benchmark.

We're proud that the Ai4Privacy pii-masking dataset was used for this critical evaluation related to privacy.

This is a perfect win-win. Our open-source data enables researchers to build and validate more effective security solutions for core AI infrastructure. Their work, in turn, helps make the entire LLM ecosystem safer, showing that performance and privacy don't have to be mutually exclusive.

Kudos to Kexin Chu, Zecheng Lin, Dawei Xiang, 沈子旭, Jianchang Su, cheng chu, Yiwei Yang, Wenhui Zhang, Wenfei Wu, and Wei Zhang on this beautiful work.

🔗 Check out their paper to see the future of secure, high-performance LLM inference: https://arxiv.org/pdf/2508.08438

#OpenSource
#DataPrivacy
#LLM
#Anonymization
#AIsecurity
#HuggingFace
#Ai4Privacy
#Worldslargestopensourceprivacymaskingdataset

reacted to their post with 🚀 about 1 month ago

Post

322

Are you sure the open-source LLM model you just downloaded is safe?

A recent paper on "Privacy Backdoors" reports a new vulnerability where pre-trained models can be poisoned before fine-tuning them. This is a serious challenge for everyone building on open-source AI.

Instead of just pointing out problems, we believe in finding better solutions. To understand this threat, the researchers needed to test their attack on realistic data structures. They needed a dataset that could effectively simulate a high-stakes privacy attack, and we're proud that our Ai4Privacy dataset was used to provide this crucial benchmark. The paper reports that for our complex dataset, the privacy leakage on a non-poisoned model was almost zero. After the backdoor attack, that number reportedly jumped to 87%.

Ai4Privacy dataset provided a realistic benchmark for their research. Our dataset, composed of synthetic identities, helped them demonstrate how a poisoned model could dramatically amplify privacy leakage.

This is why we champion open source: it enables the community to identify these issues and develop better, safer solutions together.

Kudos to the research team behind this study: Yuxin Wen, Leo Marchyok, Sanghyun Hong, Jonas Geiping, Tom Goldstein, and Nicholas Carlini, Oregon State University, University of Maryland, Google DeepMind, and ELLIS Institute Tubingen & MPI Intelligent Systems.

🔗 Read the research to understand this new challenge: https://arxiv.org/pdf/2404.01231

#DataPrivacy #AI #OpenSource #Anonymization #MachineLearning #Ai4Privacy #Worldslargestopensourceprivacydataset