{"author":{"@type":"Person","name":"Alexander Bock","url":"https://abxk.ai/en/autor/alexander-bock/"},"categories":["AI Security"],"content":{"markdown":"[Part 2 of this level](/en/ai-security/learn/adversarial-machine-learning/) showed that a trained model can give away its training data through membership inference, inversion or verbatim extraction. Removing names from a prompt — the basics from our guide on [protecting personal data with AI tools](/en/ai-security/how-to-protect-personal-data-using-ai-tools/) — does not help against that. You need techniques that limit what a model can learn about any single person, or that keep data hidden even while it is processed. Together they are called **privacy-enhancing technologies** (PETs).\n\nThis is part 3 of Level 4 (Expert) in our AI Security learning path. You will learn what epsilon and a privacy budget mean, and what federated learning protects and what it does not. You will also see where secure enclaves and homomorphic encryption fit, and why synthetic data is not automatically anonymous. The online shop of this level wants to train a better fraud model on customer data and to work with partner shops. At the end, you will choose a technique for each of its goals.\n\n## Why removing names is not enough\n\nData without names can still point to one person. A classic study estimated that 87% of the US population could be uniquely identified by ZIP code, gender and date of birth alone (Source: Sweeney, \"Simple Demographics Often Identify People Uniquely\", Carnegie Mellon University, 2000). In 2008, researchers re-identified people in an anonymized movie-rating dataset that Netflix had published, by comparing it with public reviews (Source: Narayanan and Shmatikov, IEEE S\u0026P 2008).\n\nThe law reflects this. As [Level 2](/en/ai-security/learn/data-leakage-controls-for-ai-tools/) showed, pseudonymized data — names replaced by codes — remains personal data under the GDPR for anyone who can link the codes back to people. Only truly anonymous data falls outside the regulation. The European Data Protection Board applies the same logic to AI models. A model trained on personal data counts as anonymous only if the chance of extracting personal data from it, directly or through queries, is insignificant. This has to be shown case by case (Source: EDPB Opinion 28/2024, December 2024, edpb.europa.eu).\n\nSo the question changes. It is no longer \"Did we remove the identifiers?\" but \"What can anyone learn about one person from our data, our model or its outputs?\" Differential privacy answers that question with a number.\n\n## Differential privacy: a limit on what one person can change\n\nA process is **differentially private** if its results look almost the same whether or not any single person's data is included (Source: Dwork, McSherry, Nissim and Smith, 2006). The parameter **epsilon** (ε) defines \"almost\": for every possible result, the probability with one person's data may be at most e^ε times the probability without it. Smaller ε means stronger privacy.\n\nThe oldest example shows the idea. In **randomized response**, a survey method from the 1960s, each person flips a coin before answering a sensitive question:\n\n![Randomized response as a tree: a first coin decides whether a person answers truthfully or lets a second coin decide the answer; a yes is three times as likely from a person whose true answer is yes than from a person whose true answer is no, which gives epsilon equal to ln 3, about 1.1](/images/learn/en/privacy-preserving-ai-techniques-randomized-response.webp \"Randomized response: every single answer is deniable, but the group statistics still work.\")\n\nWith fair coins, a person whose true answer is \"yes\" says \"yes\" with a probability of 3/4. A person whose true answer is \"no\" says \"yes\" with a probability of 1/4. The ratio is 3, so ε = ln 3, about 1.1. Every person can deny their answer, but the noise is known, so the share of true \"yes\" answers in a large group can still be estimated.\n\nThree practical rules follow:\n\n- **e^ε grows fast.** At ε = 1, one person can change the odds of a result by a factor of about 2.7. At ε = 10, the factor is over 20,000, so the formal guarantee is weak.\n- **Every use spends budget.** Each query or training run uses part of the **privacy budget**. In the simplest accounting, budgets add up: two queries at ε = 0.5 each cost ε = 1 in total. When the budget is spent, you stop or accept a weaker guarantee.\n- **The unit matters.** Does ε protect one record, or one person with many records? NIST explains how to evaluate such claims, including the unit of privacy and common pitfalls in implementations (Source: NIST SP 800-226, Guidelines for Evaluating Differential Privacy Guarantees, March 2025, nist.gov).\n\n![Two charts: on the left, the worst-case factor e to the power of epsilon rises from 1 at epsilon 0 to 148 at epsilon 5 and over 22,000 at epsilon 10, with randomized response marked at epsilon 1.1 and factor 3; on the right, a schematic curve shows usefulness rising with epsilon and then leveling off](/images/learn/en/privacy-preserving-ai-techniques-epsilon.webp \"The privacy side is exact math. The usefulness curve is a schematic shape, not a measurement.\")\n\nFor machine learning, **DP-SGD** limits the influence of each training example and adds noise during training, so that the finished model is differentially private (Source: Abadi et al., CCS 2016). The price is accuracy, and it is not paid evenly: rare groups and rare patterns lose the most (Source: Bagdasaryan, Poursaeed and Shmatikov, NeurIPS 2019). Large datasets soften the cost. Differential privacy is in real use: the US Census Bureau applied it to the 2020 Census results, and Apple and Google use it to collect usage statistics from devices.\n\n## Federated learning: train without collecting the data\n\n**Federated learning** sends the model to the data instead of the data to the model. Devices or organizations train a copy locally and send back only model updates; a server combines them into a new model (Source: McMahan et al., AISTATS 2017). Google used this approach to improve next-word prediction in its Gboard keyboard (Source: Hard et al., 2018).\n\n![Federated learning: a server sends the model to three participants, each trains on its own data and sends back only an update, and the server combines the updates; markers show that updates can still leak data and that secure aggregation and differential privacy reduce this risk](/images/learn/en/privacy-preserving-ai-techniques-federated.webp \"Federated learning keeps raw data local — but the updates still need protection.\")\n\nFederated learning keeps raw data out of a central store. That helps when data cannot leave a device, a hospital or a country. It does not make the result private by itself:\n\n- **Updates leak.** Researchers reconstructed training images and text from shared gradients (Source: Zhu, Liu and Han, \"Deep Leakage from Gradients\", NeurIPS 2019).\n- **Participants can poison** the shared model, as described in part 2.\n- **Data differs between participants.** Training slows down, and the shared model may fit some participants worse than others.\n\nTwo add-ons close part of the gap. **Secure aggregation** lets the server see only the sum of many updates, not each single one (Source: Bonawitz et al., CCS 2017). Differential privacy on the updates limits what even the sum can reveal.\n\n## Protecting data in use: enclaves and homomorphic encryption\n\nEncryption usually protects data at rest and in transit. During processing, however, the data sits decrypted in memory. Two technologies address this gap.\n\nA **trusted execution environment** (TEE), also called a secure enclave, runs code in hardware-isolated memory that even the cloud operator cannot read. **Remote attestation** proves which code runs inside. This approach is called **confidential computing**. CPU technologies from Intel and AMD support it, and so do recent NVIDIA data center GPUs such as the H100. Apple's Private Cloud Compute, announced in 2024, builds on similar ideas for AI requests. The limits: you trust the hardware vendor, attestation helps only if you actually verify it, and side-channel attacks have broken enclaves before (Source: Van Bulck et al., \"Foreshadow\", USENIX Security 2018).\n\n**Homomorphic encryption** (HE) allows computing directly on encrypted data; only the key holder can read the result. Fully homomorphic encryption has been possible since 2009 (Source: Gentry, STOC 2009). It is still far slower than computing on plain data, and many operations inside neural networks can only be approximated. Today it fits narrow tasks, such as scoring with a small model — not running a large language model at a practical speed. **Secure multi-party computation** lets several parties compute a joint result without showing each other their inputs; it also adds heavy computing and communication costs.\n\n## Synthetic data: useful, but not anonymous by default\n\n**Synthetic data** consists of generated records that mimic the statistics of real data. It is useful for testing, development and sharing. But the generator learns from real records and can reproduce them, especially unusual ones. A 2022 study found that synthetic data does not offer a better trade-off between privacy and usefulness than traditional anonymization, and that its privacy gain is hard to predict (Source: Stadler, Oprisanu and Troncoso, USENIX Security 2022). If you need a privacy guarantee, train the generator with differential privacy and check its output for copied records.\n\n{{\u003c callout type=\"warning\" title=\"Embeddings are personal data too\" \u003e}}\nA vector database stores **embeddings**: numeric representations of your documents. They look anonymous, but they are not. Researchers recovered 92% of 32-token text inputs exactly from their embeddings (Source: Morris et al., \"Text Embeddings Reveal (Almost) As Much As Text\", EMNLP 2023). Give a vector index, such as one in Pinecone, the same access rights, retention and deletion rules as the source documents.\n{{\u003c /callout \u003e}}\n\n## Choosing a technique: the shop's decisions\n\nStart from the goal, not from the technology:\n\n| Goal | Technique | Protects against | Main cost |\n|---|---|---|---|\n| Publish statistics about customers | Differential privacy for queries | Re-identification from results | Noise, budget management |\n| Train a model that must not memorize individuals | DP-SGD | Membership inference, extraction | Accuracy, especially for rare cases |\n| Data must stay with its owners | Federated learning with secure aggregation | Central data collection | Leaky updates, complex operations |\n| The cloud must not see data during processing | Confidential computing (TEE) | Access by the operator | Trust in hardware, side channels |\n| Compute without ever decrypting | Homomorphic encryption, multi-party computation | Everyone without the key | Very slow, narrow tasks |\n| Test and share data without real records | Synthetic data, generated with DP | Exposure of real records | Less realism, checks for copies |\n| Keep prompts in-house | Self-hosted open-weight model | Access by the model provider | Your own security and operations |\n\nFor the shop, this leads to four decisions. The **fraud model** is a poor fit for DP-SGD: fraud is rare, and noise hurts rare patterns most. The shop protects it with strict access control, minimal API outputs and regular membership inference tests instead. The **joint fraud signals** with two partner shops use federated learning with secure aggregation. The **fraud dashboard** for partners uses differential privacy with a fixed yearly budget. The **test data** for developers is synthetic, generated with differential privacy and checked for copied records.\n\nThe last row of the table needs infrastructure. Our [LLM database](/en/llm/) lists {{\u003c stat \"models-open\" \u003e}} models with open weights that you can run yourself. According to the vendor, Runpod rents out GPUs on demand for such workloads, and Airia, by its own description, governs agents, models and data access in enterprises. Both move trust rather than remove it, so they belong in your threat model. [Part 5](/en/ai-security/learn/build-a-secure-ai-program/) shows where these decisions sit in a security program.\n\n{{\u003c tools \"airia runpod pinecone\" \u003e}}","text":"Part 2 of this level showed that a trained model can give away its training data through membership inference, inversion or verbatim extraction. Removing names from a prompt — the basics from our guide on protecting personal data with AI tools — does not help against that. You need techniques that limit what a model can learn about any single person, or that keep data hidden even while it is processed. Together they are called privacy-enhancing technologies (PETs).\nThis is part 3 of Level 4 (Expert) in our AI Security learning path. You will learn what epsilon and a privacy budget mean, and what federated learning protects and what it does not. You will also see where secure enclaves and homomorphic encryption fit, and why synthetic data is not automatically anonymous. The online shop of this level wants to train a better fraud model on customer data and to work with partner shops. At the end, you will choose a technique for each of its goals.\nWhy removing names is not enough Data without names can still point to one person. A classic study estimated that 87% of the US population could be uniquely identified by ZIP code, gender and date of birth alone (Source: Sweeney, \u0026ldquo;Simple Demographics Often Identify People Uniquely\u0026rdquo;, Carnegie Mellon University, 2000). In 2008, researchers re-identified people in an anonymized movie-rating dataset that Netflix had published, by comparing it with public reviews (Source: Narayanan and Shmatikov, IEEE S\u0026amp;P 2008).\nThe law reflects this. As Level 2 showed, pseudonymized data — names replaced by codes — remains personal data under the GDPR for anyone who can link the codes back to people. Only truly anonymous data falls outside the regulation. The European Data Protection Board applies the same logic to AI models. A model trained on personal data counts as anonymous only if the chance of extracting personal data from it, directly or through queries, is insignificant. This has to be shown case by case (Source: EDPB Opinion 28/2024, December 2024, edpb.europa.eu).\nSo the question changes. It is no longer \u0026ldquo;Did we remove the identifiers?\u0026rdquo; but \u0026ldquo;What can anyone learn about one person from our data, our model or its outputs?\u0026rdquo; Differential privacy answers that question with a number.\nDifferential privacy: a limit on what one person can change A process is differentially private if its results look almost the same whether or not any single person\u0026rsquo;s data is included (Source: Dwork, McSherry, Nissim and Smith, 2006). The parameter epsilon (ε) defines \u0026ldquo;almost\u0026rdquo;: for every possible result, the probability with one person\u0026rsquo;s data may be at most e^ε times the probability without it. Smaller ε means stronger privacy.\nThe oldest example shows the idea. In randomized response, a survey method from the 1960s, each person flips a coin before answering a sensitive question:\nRandomized response: every single answer is deniable, but the group statistics still work. With fair coins, a person whose true answer is \u0026ldquo;yes\u0026rdquo; says \u0026ldquo;yes\u0026rdquo; with a probability of 3/4. A person whose true answer is \u0026ldquo;no\u0026rdquo; says \u0026ldquo;yes\u0026rdquo; with a probability of 1/4. The ratio is 3, so ε = ln 3, about 1.1. Every person can deny their answer, but the noise is known, so the share of true \u0026ldquo;yes\u0026rdquo; answers in a large group can still be estimated.\nThree practical rules follow:\ne^ε grows fast. At ε = 1, one person can change the odds of a result by a factor of about 2.7. At ε = 10, the factor is over 20,000, so the formal guarantee is weak. Every use spends budget. Each query or training run uses part of the privacy budget. In the simplest accounting, budgets add up: two queries at ε = 0.5 each cost ε = 1 in total. When the budget is spent, you stop or accept a weaker guarantee. The unit matters. Does ε protect one record, or one person with many records? NIST explains how to evaluate such claims, including the unit of privacy and common pitfalls in implementations (Source: NIST SP 800-226, Guidelines for Evaluating Differential Privacy Guarantees, March 2025, nist.gov). The privacy side is exact math. The usefulness curve is a schematic shape, not a measurement. For machine learning, DP-SGD limits the influence of each training example and adds noise during training, so that the finished model is differentially private (Source: Abadi et al., CCS 2016). The price is accuracy, and it is not paid evenly: rare groups and rare patterns lose the most (Source: Bagdasaryan, Poursaeed and Shmatikov, NeurIPS 2019). Large datasets soften the cost. Differential privacy is in real use: the US Census Bureau applied it to the 2020 Census results, and Apple and Google use it to collect usage statistics from devices.\nFederated learning: train without collecting the data Federated learning sends the model to the data instead of the data to the model. Devices or organizations train a copy locally and send back only model updates; a server combines them into a new model (Source: McMahan et al., AISTATS 2017). Google used this approach to improve next-word prediction in its Gboard keyboard (Source: Hard et al., 2018).\nFederated learning keeps raw data local — but the updates still need protection. Federated learning keeps raw data out of a central store. That helps when data cannot leave a device, a hospital or a country. It does not make the result private by itself:\nUpdates leak. Researchers reconstructed training images and text from shared gradients (Source: Zhu, Liu and Han, \u0026ldquo;Deep Leakage from Gradients\u0026rdquo;, NeurIPS 2019). Participants can poison the shared model, as described in part 2. Data differs between participants. Training slows down, and the shared model may fit some participants worse than others. Two add-ons close part of the gap. Secure aggregation lets the server see only the sum of many updates, not each single one (Source: Bonawitz et al., CCS 2017). Differential privacy on the updates limits what even the sum can reveal.\nProtecting data in use: enclaves and homomorphic encryption Encryption usually protects data at rest and in transit. During processing, however, the data sits decrypted in memory. Two technologies address this gap.\nA trusted execution environment (TEE), also called a secure enclave, runs code in hardware-isolated memory that even the cloud operator cannot read. Remote attestation proves which code runs inside. This approach is called confidential computing. CPU technologies from Intel and AMD support it, and so do recent NVIDIA data center GPUs such as the H100. Apple\u0026rsquo;s Private Cloud Compute, announced in 2024, builds on similar ideas for AI requests. The limits: you trust the hardware vendor, attestation helps only if you actually verify it, and side-channel attacks have broken enclaves before (Source: Van Bulck et al., \u0026ldquo;Foreshadow\u0026rdquo;, USENIX Security 2018).\nHomomorphic encryption (HE) allows computing directly on encrypted data; only the key holder can read the result. Fully homomorphic encryption has been possible since 2009 (Source: Gentry, STOC 2009). It is still far slower than computing on plain data, and many operations inside neural networks can only be approximated. Today it fits narrow tasks, such as scoring with a small model — not running a large language model at a practical speed. Secure multi-party computation lets several parties compute a joint result without showing each other their inputs; it also adds heavy computing and communication costs.\nSynthetic data: useful, but not anonymous by default Synthetic data consists of generated records that mimic the statistics of real data. It is useful for testing, development and sharing. But the generator learns from real records and can reproduce them, especially unusual ones. A 2022 study found that synthetic data does not offer a better trade-off between privacy and usefulness than traditional anonymization, and that its privacy gain is hard to predict (Source: Stadler, Oprisanu and Troncoso, USENIX Security 2022). If you need a privacy guarantee, train the generator with differential privacy and check its output for copied records.\n!Embeddings are personal data too\nA vector database stores embeddings: numeric representations of your documents. They look anonymous, but they are not. Researchers recovered 92% of 32-token text inputs exactly from their embeddings (Source: Morris et al., \u0026ldquo;Text Embeddings Reveal (Almost) As Much As Text\u0026rdquo;, EMNLP 2023). Give a vector index, such as one in Pinecone, the same access rights, retention and deletion rules as the source documents.\nChoosing a technique: the shop\u0026rsquo;s decisions Start from the goal, not from the technology:\nPrivacy-Preserving AI: Differential Privacy, Federated Learning and More Goal Technique Protects against Main cost Publish statistics about customers Differential privacy for queries Re-identification from results Noise, budget management Train a model that must not memorize individuals DP-SGD Membership inference, extraction Accuracy, especially for rare cases Data must stay with its owners Federated learning with secure aggregation Central data collection Leaky updates, complex operations The cloud must not see data during processing Confidential computing (TEE) Access by the operator Trust in hardware, side channels Compute without ever decrypting Homomorphic encryption, multi-party computation Everyone without the key Very slow, narrow tasks Test and share data without real records Synthetic data, generated with DP Exposure of real records Less realism, checks for copies Keep prompts in-house Self-hosted open-weight model Access by the model provider Your own security and operations For the shop, this leads to four decisions. The fraud model is a poor fit for DP-SGD: fraud is rare, and noise hurts rare patterns most. The shop protects it with strict access control, minimal API outputs and regular membership inference tests instead. The joint fraud signals with two partner shops use federated learning with secure aggregation. The fraud dashboard for partners uses differential privacy with a fixed yearly budget. The test data for developers is synthetic, generated with differential privacy and checked for copied records.\nThe last row of the table needs infrastructure. Our LLM database lists 172 models with open weights that you can run yourself. According to the vendor, Runpod rents out GPUs on demand for such workloads, and Airia, by its own description, governs agents, models and data access in enterprises. Both move trust rather than remove it, so they belong in your threat model. Part 5 shows where these decisions sit in a security program.\nAiriaSecurity \u0026amp; Governance · AI at the coreOrchestration platform for enterprises to build and manage AI solutions securely. RunpodInfrastructure \u0026amp; Hosting · AI at the coreGPU cloud for training and running AI models, billed by usage. PineconeInfrastructure \u0026amp; Hosting · AI at the coreVector database for AI applications such as search and retrieval. Short descriptions from our tool comparison, based on the vendors' information — not tested by ABXK.\n","words":1760},"date":"2026-10-14","description":"Epsilon and privacy budgets, federated learning, secure enclaves, homomorphic encryption and synthetic data: what each protects, what it costs and how to choose.","image":"https://abxk.ai/images/cover/privacy-preserving-ai-techniques.webp","json":"https://abxk.ai/en/ai-security/learn/privacy-preserving-ai-techniques/index.json","kind":"page","language":"en","learning_path":{"category":"ai-security","level":4,"level_name":"Expert"},"modified":"2026-10-14","related_tools":["Airia","Runpod","Pinecone"],"section":"ai-security","seo_title":"Privacy-Preserving AI: Differential Privacy and More","tags":["privacy","differential-privacy","federated-learning","confidential-computing","expert"],"title":"Privacy-Preserving AI: Differential Privacy, Federated Learning and More","translations":[{"json":"https://abxk.ai/de/ai-security/learn/privacy-preserving-ai-techniques/index.json","language":"de","url":"https://abxk.ai/de/ai-security/learn/privacy-preserving-ai-techniques/"}],"url":"https://abxk.ai/en/ai-security/learn/privacy-preserving-ai-techniques/"}