Before You Paste Production Logs into ChatGPT

It's 2 a.m., a service is throwing stack traces, and the on-call engineer has a chat window open. Paste the logs, ask what's wrong, get a plausible answer in seconds. It works well enough that it's quickly becoming a reflex.

It's also one of the fastest ways for production data to leave your organization without anyone noticing. Logs carry customer email addresses, IP addresses, session tokens, internal hostnames and, more often than anyone admits, credentials. Once pasted into a third-party service, that data sits outside your retention policy, your access controls and your incident scope.

This spring made the risk concrete. Before worrying about model accuracy, most teams have a simpler operational problem to solve: knowing what data is going where.

Three warnings in three months

On May 2, Bloomberg reported that Samsung was banning staff use of generative AI tools like ChatGPT after discovering employees had uploaded sensitive code. The concern, according to the company's memo, was that data sent to these platforms is stored on external servers, making it difficult to retrieve and delete.

In March, OpenAI published a post-mortem on a March 20 incident in which a bug in the open-source redis-py client library let some users see titles from other users' chat history. The same bug may have made payment-related information visible for 1.2% of ChatGPT Plus subscribers who were active during a specific nine-hour window. Whatever your view of the vendor, the lesson is general: anything you send to a hosted service is exposed to that service's bugs.

On March 31, Italy's data protection authority announced a temporary limitation on OpenAI's processing of Italian users' data, citing that breach, the lack of a legal basis for mass collection of personal data to train the models, and the absence of age verification. ChatGPT returned in Italy on April 28 after OpenAI added an age-verification tool at sign-up and a form for EU users to object to their data being used for training, but the regulator said its investigation would continue. Regulators are paying close attention to where personal data goes, and that includes data your staff submit.

Meanwhile, the tools keep getting more capable and more embedded. GitHub's Copilot X announcement in March previewed chat inside the editor, AI-generated pull request descriptions and a command-line assistant, some of it powered by OpenAI's newly released GPT-4. The more useful these tools become, the more context people will want to hand them.

Treat AI usage policy as an operational control

Many organizations respond with a memo that says not to put confidential data into AI tools, and stop there. That's a start, but it's a statement of intent, not a control. An engineer under pressure at 2 a.m. will not reread the acceptable use policy.

A workable approach treats AI tools like any other third-party data processor and puts controls where the data actually flows:

• An approved-tools list. Name the tools and account types that are allowed, and why. Business tiers with clear data-handling terms carry a different risk from personal accounts.
• Data classes, not vague rules. Spell out what may never be submitted: credentials, customer personal data, regulated data, source code for sensitive systems. Keep it short enough to remember.
• A sanctioned path. If you ban everything and offer nothing, usage goes underground. Give people an approved option so they don't need a workaround.
• Vendor review. Check retention, use of submitted data for training, data residency and breach notification for every tool on the list, the same way you would for a logging or monitoring vendor.
• An incident playbook entry. Decide in advance what happens when someone pastes something they shouldn't: who is told, which keys are rotated, and whether the vendor is contacted.

Redact at the source

The most effective control is making sure sensitive data isn't in the logs to begin with. If logs are clean, a pasted log is a much smaller problem.

• Structured logging with field-level redaction. Mask emails, tokens, card numbers and authorization headers in the logging library or collector, rather than relying on developers to remember.
• Secrets never logged. Audit for request dumps and debug statements that print headers or environment variables. Add tests that fail if known secret patterns appear in log output.
• Scrubbing in the pipeline. Apply redaction rules again at the log shipper or aggregator as a second line of defence.
• Least-privilege log access. Not everyone who can read dashboards needs raw production logs. Fewer people with raw access means fewer chances to paste.
• Short retention for raw logs. Keep detailed logs as long as operations and compliance require, and no longer.

These are good practices with or without AI. Generative tools simply raise the cost of not having them.

What we recommend

Don't start with a blanket ban unless you have a specific reason. Start with visibility: find out which AI tools your teams already use, and for what. Then publish a one-page usage standard with an approved-tools list and clear data classes, review the data-handling terms of each approved tool, and invest in redaction so the data that matters never reaches a log line in the first place.

Finally, rehearse it. Add a scenario where sensitive data is pasted into an external tool to your next incident tabletop exercise. If the response is improvised, the policy isn't finished.

If you'd like a second pair of eyes on your logging pipeline or your AI usage policy, talk to us.