Insights

Takeaways:

Within the same series of LLMs trained on identical data in the same order, as the size of the models increases, their capacities on language tasks also increase. Concurrently, these larger models exhibit enhanced extraction accuracy with existing Data Extraction Attacks, due to their advanced memorization capacities. Notably, the rate of increase in data extraction accuracy on Enron out-paces the improvements in ARC-Easy for Pythia, suggesting a growing privacy risk as models scale.

Position and Type of Private Data

Get a better experience on larger screens

Data Length

Get a better experience on larger screens

For Enron, short emails have higher perplexity due to their informal nature and variability, which provides less context and makes them harder for the model to predict accurately. For ECHR, longer legal documents have higher perplexity due to their complexity and dense information, making them challenging for the model.

Higher perplexity indicates the model struggles more, creating distinct patterns between training and non-training data, leading to increased MIA AUC and higher privacy risks for these samples.

Pretraining data size

Figure 6. DEA accuracy with different training tokens.
Besides the model size, when increasing the number of training tokens, LLM’s memorization capacity also increases. Consequently, this leads to a rise in data extraction accuracy

Takeaways:

Our findings reveal that data type, data position, data length, and pretraining data size collectively impact privacy risks on Llama-2. Data with richer contextual information (e.g., locations) tends to be more susceptible to memorization during fine-tuning. Private data at the end of a sample is more vulnerable to extraction. Data samples that are harder to predict, indicated by higher perplexity, are more easily identified in MIAs. Additionally, increasing the size of the training data enhances the model’s memorization capacity, leading to higher privacy risks. These insights highlight the necessity for targeted privacy strategies that address the specific characteristics of different data types in LLMs.

Please view this table on a larger device

Takeaways:

Our investigation shows that scrubbing and DP effectively reduce the privacy risks of MIA while degrading model performance. This underscores the need for further research to develop techniques that achieve a better privacy-utility tradeoff.

Please view this table on a larger device

Takeaways:

While model-generated attack prompts are more effective than manually created ones for jailbreak attacks, the evaluated poisoning attack is less effective than pure querybased method, potentially due to suboptimal poison data pattern design. Moreover, the trend of attack success rate changes with model sizes is consistent among different types of attacks.

Figure 7. The FuzzRate of different attacks on different models.
The ignore_print and spell_check are the two strongest attacks on Llama2-70b-chat.

Figure 8. The leakage ratio (%) of samples that have FuzzRate over 90.
Consistent with results measured by the average FuzzRate, ignore_print is the strongest attack on Llama-2-70b-chat.

Takeaways:

Prompts can be easily leaked with prompting attacks. Directly asking LLMs to ignore and print the previous instructions can leak to serious prompt leakage in many LLMs.

Takeaways:

For the same series of models, the larger model has a higher risk of prompt leakage, potentially because they are better at following the PLA instructions to output the private prompts.

Takeaways:

Using manually designed defensive prompts to protect the private prompts has limited effects. It is essential to develop a rigorous mechanism that can preserve the privacy of prompts.

Takeaways:

LLMs can extract user data from input context due to their advanced reasoning capabilities. Developing techniques that aim to enable the private usage of LLMs while safeguarding query prompts is necessary.