Data Security in AI: 8 Steps for Using AI with Sensitive Business Info

Anyone who’s dealt with InfoSec questionnaires knows they’re painful. Still, they’re necessary.

As part of a GSA group, Fleetio deals with a lot of sensitive data as a company working directly with different departments of the US federal government. Fleetio stores sensitive information and provides secure access to data, so thorough oversight must be firmly in place.

As a result, Fleetio is constantly fielding complex IT and InfoSec questionnaires as each of its clients decides whether to work with Fleetio’s platform.

These documents are highly technical in nature, requiring tons of scrutiny and review and lots of back-and-forth between documents and teammates.

They were being hit with so many InfoSec questionnaires they were simply overwhelmed. And the worst part is they had to deal with so many different format types, from:

• XLSX
• PDF
• Docx

And more. So they’re seeing these docs in various formats and having to adapt and engage quickly. Each doc may include:

• Dropdowns
• Tables
• Multipart questions

And more. So the time to wade through them moves from hours to days and sometimes even longer.

Tellennium’s customer base consists of leaders in healthcare, energy, telecoms, and other highly regulated industries. Every opportunity comes with a long list of questions about security, compliance, technology, and regulatory requirements.

Security questionnaires can significantly slow down deals as the sale turns into a compliance review process.

Like most people, you can’t help but be concerned about how sensitive data is handled. It’s become a hot topic in business circles. Sure, generative AI tools can drastically boost productivity. But they can introduce new risks, especially when used with proprietary or sensitive business information. 

Whether you’re deploying AI internally or integrating it into your customer-facing products, security should always come first. In this post, we break down data security as it pertains to Large Language Models (LLMs) to help you:

• Maintain compliance
• Mitigate risk across new and existing AI systems
• Maximize AI productivity gains while minimizing sensitive data exposure

We cover everything from working with LLMs to securing knowledge sources and sanitizing data. Here are 8 practical steps to protect your sensitive business data while using AI.

1. Classify What Counts as “Sensitive” First 

Before uploading any information into an AI system, ensure that you define what your organization considers sensitive. This baseline will help you determine how to handle every subsequent step.

Examples of Sensitive Data: 

• Customer PII (Personally Identifiable Information)
• Financial reports or revenue numbers
• Internal strategic documents
• Unreleased product plans or roadmaps
• Employee performance records or HR data

Perhaps surprisingly, not everything needs to be locked down. Your public website content or marketing assets are likely not sensitive. Once you take the time to classify data types, the rest of your security process gets so much easier to manage. 

It also enhances communication across departments, so everyone, from IT to marketing, knows what to protect and how.

2. Sanitize the Data Before You Connect It 

Now, if you’re worried about data exposure, simply clean it before you use it with AI tools. The easiest way to sanitize is simply by removing.

Consider removing/redacting: 

• Customer names and IDs
• Employee or stakeholder names
• Financial figures
• Account numbers or client references
• Any identifiable internal markers

Although, this is going overboard:

Other than removing the data, you can also sanitize data by:

• Replacing: Swap names or values with generic nouns like “Customer A” or “Project X”.
• Structuring it intentionally: Use templates or pre-cleaned formats that strip out sensitive information by default.

3. Always Disable Model Training

A huge misconception is that AI models learn from you by default. Sadly, that’s not always the case. However, if you don’t actively disable model training, you might be contributing to it. 

Here are some tips for the best ways to disable model training: 

• Disable training in the model’s configuration if the option exists.
• Avoid free-tier versions of AI tools. This is not the area to save money. 
• Ask your vendor directly: “Do your platform use our inputs for future training?” 
• Prefer RAG (Retrieval Augmented Generation) over fine-tuned models when privacy is key.

It’s in your best interest to invest in a paid plan that includes proper data governance. It’s worth the cost to protect your business IP. Additionally, always review terms of service and data policies. What feels like a simple query might end up in a training dataset if you’re not careful.

4. Host Files in Integrations Rather than Direct Uploads

A good enterprise-grade AI system will allow you to host files on your own provider of choice. So, avoid uploading files directly into an AI tool unless it’s absolutely necessary. 

This reduces headaches with GDPR and data governance concerns. Since you’re already using a familiar tool to host files, this allows you to have one less worry when it comes to uploading files directly.

You want to choose a provider who can allow you to connect file sources such as: 

• Google Drive 
• OneDrive
• Confluence
• Box
• SharePoint

The AI provider will likely utilize that information in its original format or in a vectorized format. But the data itself should not need to be uploaded directly to the model provider.

In short, let the AI access what it needs, but keep the data where you trust it.

5. Log All Interactions with Your AI

If you’re concerned about data leakage or accountability, ensure that every interaction with your AI system is logged. For example, if you’re using a model provider, enable whatever logging is available. You want to be able to have a record of all interactions with the model. 

Here’s what you should be logging:

• 3rd party Integrations
• File uploads/downloads
• Queries
• Answers
• Revisions/corrections made to the AI

These logs can help with audits, trace data exposure incidents, and improve trust in your system. This might sound like a lot of unnecessary information at first. But if your concern is data leakage, you’ll be glad you have detailed records.

6. Always Review AI-Generated Outputs

This is less of a technical tip and more of a habit your team needs to build. AI can hallucinate, misattribute sources, or miss the context entirely. Encourage (insist) your team to always review AI-generated outputs. Even from the strongest knowledge base.

Because even the best models get it wrong, here’s what you should be looking for:

• Incorrect citation:  AI attributes an answer to a source that doesn’t contain the stated information.
• Content hallucinations:  AI generates plausible-sounding but nonexistent facts due to vague or low-relevance context.
• Context omission:  AI answers without grounding in provided documents when retrieval fails or is misaligned. 

Reviewing output isn’t optional. It’s a safety measure. Train your team to treat AI responses as drafts, not gospel. 

Also, build a checklist-based review process for common failure modes. Then, assign responsibility for AI output verification to specific roles, especially for client-facing or compliance-critical work. 

It’s better to spend a few extra minutes validating than risk reputational damage, legal exposure, or poor decision-making based on flawed information.

7. Use an Enterprise-Grade RAG with Governance Controls

You’re likely already using a mix of foundational models and RAG (Retrieval Augmented Generation). With RAG, there are a few key elements that can really boost confidence when it comes to sensitive data:

• RBAC: User abilities should be role-based. Some users might not be able to upload content or “teach” the model new information. You may only want some users to get answers from the AI, not train it. Other users might not have read/write access to the source material. For example, you wouldn’t want an engineer asking about sales figures they shouldn’t have. Or a sales rep inquiring about the engineering roadmap.
• Tiered file protections: You want to instruct the AI on which files are “more sensitive” than others. For example, you may not want to return links to certain sensitive files.
• Source tagging and classification: Categorizing your content with tags is important. This can make it much easier to narrow down how files are used in what situation.

Also consider audit logging and usage tracking, so you can monitor how data is accessed and flagged if misuse occurs. Granular insights into queries and user behavior create a clear accountability trail. This is an essential feature for regulated industries or companies managing high-stakes data.

With proper controls, RAG becomes a safe and scalable way to power AI within your organization.

8. Keep Your Prompt Context Small

We’re often told that bigger context windows are better. This might sound nice, but keeping context windows small can actually benefit you. How? By forcing users to leverage only the most relevant content. Smaller is smarter. 

By limiting the context the AI has access to, you: 

• Force precision in data usage
• Avoid loading irrelevant or risky data
• Reduce computation costs
• Improve output quality by narrowing focus

Encourage users to select only the most relevant content when querying the AI. This reduces noise and keeps sensitive information out of reach unless explicitly needed. 

Bonus: Use SSO to Lock Down Access

Single sign-on (SSO) is a no-brainer when it comes to access control. Integrate your AI provider with your company’s identity system. 

Benefits of SSO: 

• Centralized access control: Manage permissions from one dashboard
• Faster onboarding/offboarding: Employees instantly gain or lose access
• Fewer passwords = fewer breaches
• Consistent security policies: Enforce MFA, session timeouts, and login monitoring across tools
• Better audit trails: Track user activity through a unified identity provider

If your AI platform supports SSO, enable it. If it doesn’t, it might be time to look for a new vendor. 

How 1up Emphasizes Data Security

Using AI in a business setting brings huge benefits, but only if you’re crystal clear on how your data is being handled. 

At 1up we’ve been implementing these rules since day 1. Our platform is purpose-built with enterprise-grade security to give organizations full control over their data. From encryption at rest (AES-256) to strict data isolation within your workspace, every layer is designed to protect sensitive business information.

No data is ever shared outside your organization. Uploaded documents are never used to train our AI models, and all interactions are logged securely within your workspace for transparency and traceability. 1up also meets SOC 2 compliance standards and includes SSO support at no additional cost.

1up ensures your data remains private, protected, and only used to deliver accurate results from sources you trust. Read more on 1up’s security here.

Data Security in AI FAQs