c2.5 Discover sensitive data in training sets with Macie

Find sensitive data before the model memorizes it

A fine-tuned model can memorize and later reproduce fragments of its training data. If that data contains PII, secrets, or regulated content, the model becomes a leak path. Amazon Macie uses managed ML and pattern matching to discover sensitive data in S3, so you can catch and remove it from a training set before customization — not after the model has already learned it.

Scan the training bucket before a job

Run a one-time sensitive data discovery job against the bucket that holds your training data:

aws macie2 create-classification-job \
  --name "scan-training-data-pre-finetune" \
  --job-type ONE_TIME \
  --s3-job-definition '{
    "bucketDefinitions": [
      {
        "accountId": "111122223333",
        "buckets": ["my-bedrock-training-data"]
      }
    ]
  }'

Macie reports findings by sensitive-data type (names, emails, credentials, financial and health identifiers, and custom data identifiers you define with regex). Review the findings and scrub or tokenize the flagged records before you launch the customization job.

Make it a gate, not a one-off

• Wire Macie findings to EventBridge → SNS/Lambda so a high-severity finding on a training bucket blocks or flags the pipeline automatically.
• For ongoing pipelines, use automated sensitive data discovery (continuous sampling) instead of re-running full jobs, which is far cheaper at scale.
• Add custom data identifiers for organization-specific secrets (internal ID formats, project code names) that the built-in types won't catch.

Combine with the guardrail

Macie protects the data going into the model; the Bedrock guardrail's sensitive-information filter (see the guardrails control) protects data coming out at inference. Use both: scrub the training corpus with Macie, and redact PII at runtime with the guardrail.

Naming note: replace the account id and bucket name. Start with a ONE_TIME job to gauge finding volume and cost before scheduling recurring scans.