Why Enterprise Search is so Difficult

Workplace search is seeing rapid growth.

Mainstream adoption of Large Language Models has brought about a wave of productivity tools that address the challenges of enterprise knowledge management. At a basic level, these platforms enable a business to search an index of their own files and data.

The more recent addition of chatbots has expanded search to include analysis and answer generation. This combination of search and Generative AI chat is commonly known as RAG (Retrieval Augmented Generation).

But despite a ton of investment and effort, we have yet to see mass adoption of search at a business level. Have you personally used an enterprise search tool today? Maybe not. But chances are you’ve used Google at least once.

We often get asked by business leaders why does it seem so hard to adopt this tech? The answer is a mix of technical challenges and public perception of search.

The “Google for B2B” trope isn’t new.

It has been pitched for years with buyers being told that all you need are a few key ingredients:

Language Model + Database + Data Connectors + User Interface = Enterprise search

As it turns out, getting search to run well is harder than people think. And it’s not just the nerdy stuff. There’s also the parts about selling, marketing, supporting, securing, maintaining, and building an actual business around the tech.

Good search requires clean, structured data. This is why Google mandates that webpages meet stringent requirements around machine readability. It must be easy for a web crawler to understand the data before it can retrieve anything.

The problem here is that within (most) organizations there’s too much data at any given moment. Most of it is very poorly structured. Some of it is completely irrelevant.

This creates a roadblock for workplace search tools. Don’t believe me? Just take a look at the G2 reviews for business search tools. A common complaint is that it’s difficult to find what you’re looking for.

This is where the “Garbage in = Garbage out” saying makes total sense. You either need to clean up your data or be highly selective about what goes into your search index.

Business data is scattered. That’s just life.

Think of how much work-related information you have sitting on your desktop, Google Drive, email inbox, Sharepoint, Slack, or any file storage system.

Search sits on top of these existing systems. That means a search tool needs to ingest the data, keep it synced, and ensure that it’s always pulling quality information. Reading all of this material requires 3rd party integrations that can introduce new technical pains.

Constantly building and maintaining connectors. Keeping a team dedicated to maintaining these integrations. Accounting for changes made to the connectors. The list goes on and on.

Controlling access to files is hard enough. Introducing AI into the mix has opened a new can of worms for businesses. Just scroll the CIO subreddit and you’ll find no shortage of complaints around AI data governance.

We’re not just talking about controlling access to files. LLM search systems have introduced a new challenge of controlling for answers sourced from outdated content, sensitive documents, and even opinions generated by an AI.

Imagine sensitive financial information is ingested by your search tool. A user conducts a search that erroneously answers their question using this document.

Who is to blame?

No one is to blame. This is a bi-product of using enterprise search without strict governance controls. In a search context, the sensitive data is not just exposed to a group of users. It’s made available to anyone interacting with the tool.

This is the number 1 challenge of automated knowledge management. Good results are subjective. They may vary from company to company, and even at the department level. Doug Turnbull explains it well in his post about the hidden dangers that kill search products.

There are many different approaches to search. The two most often discussed are:

As a business, your employees could be searching for 3-letter terms or for long complex phrases. Which search method will yield the best results?

There’s also the always present problem of garbage in/garbage out. Results depend heavily on user-generated content, so you’re fighting a constant battle to keep good data in the system.

One example of this is when searching a company chat in Slack or MS Teams. You would think that these message logs are a treasure trove of quality data. Sometimes they are. But the amount of noise that you find in these data sets can introduce incorrect answers, unintended opinions, or private conversations that you never wanted to be searchable in the first place.

For example, do you want your engineers to be able to ask questions about how the Sales Org is doing? Should a sales rep be able to inquire about how the product works or what capabilities are on the horizon?

For some teams these are good aspects of a search system. Employees are learning things about their own company they wouldn’t know otherwise. Good!

On the other hand, departments might not want this level of transparency about their roadmap or strategy. It’s all a matter of preference, which isn’t easy to define in a search tool.

The promise of enterprise search is to unearth tribal knowledge. Which introduces the unintended consequences of unearthing tribal knowledge.

Great workplace search platforms come at a high cost. Tools like Coveo and Elastic are complex systems with deeply technical teams. Given the problem space, it’s no surprise that they focus on big businesses as their target audience.

This has led to Small and Medium Businesses being priced out of the adoption curve. The good news is we expect to see SMBs adopt more knowledge automation systems as costs decrease and self-service onboarding becomes easier.

Is workplace search a real problem for most businesses?

On paper, of course it is. But speak with business leaders and you’ll hear mixed opinions. Urgency varies at a department level.

Some folks feel like the need to make everything connected and searchable is a nice-to-have for their organization. In rare cases you might find that mature internal tooling has enabled a team to search for knowledge effectively.

Our take is that the true benefit of enterprise search lies in connecting these disparate sources and allowing for cross-department collaboration. How much of a problem that is remains up to the organization and can vary based on stage, size, and maturity. It definitely aligns with why platforms are targeting the big enterprise segment.

There can be alot of effort involved in getting a search system off the ground.

Take a 1,000 person organization as an example. There could be upwards of 50,000 files to index across multiple departments.

Among those departments you might have a mix of Sharepoint, Google Drive, OneDrive, and even the occasional Dropbox user.

Then there’s the engineering team who document everything in Confluence. The sales reps who live in Salesforce. Let’s not forget the Slack channels which contain a vast amount of files and valuable conversations.

Your system needs to pull from all of these sources, index the data, make it searchable, handle the complexities of data governance, and finally – get people to adopt it. This is a high bar for setup which can take multiple departments weeks or even months to complete.

This is true of any knowledge management tool, but in the context of LLMs, the problem is compounded by the need to not only house the data but to make it machine-readable for natural language search. That means embedding and re-embedding the information as it gets updated and pulled in.

This is the part no one likes to discuss because you can’t really force it.

Users who are satisfied with their current approach to retrieving knowledge will have no incentive to use a new system, even if it’s noticeably better.

Despite the technical and business challenges, enterprise search is a $12B TAM that’s growing rapidly.

That’s because the underlying pain is very real. There is urgency to solve it. Here’s where businesses are seeing immediate value from these tools:

This is the traditional use case where teams are trying to make knowledge accessible internally. It’s where you find tools like Guru, Notion, and Learning Management Systems (LMS) such as Docebo.

This is becoming more recent and more popular among sales, IT, security, and compliance teams. An automated questionnaire is really just a series of search + answer loops. Examples include 1up, Vanta, and Drata.

Using AI to search and analyze documents is a huge use case. It’s also where a verticalized approach makes sense, with tools like Harvey and Spellbook specializing in legal documents.

This is probably the most obvious use case for search. Examples include Intercom, Zendesk, and new players like Maven.

We’re seeing support improve in multiple ways:

Beyond the application layer there are a number of successful infrastructure tools such Algolia, Coveo, and Elastic. Many of these are known for their highly customizable (and successful) search platforms.

As Doug Turnbull writes, search infrastructure carries its own hidden challenges.

“If you sell customers a full solution, they will expect their subjective, unconscious, and domain specific definition of success. Customers assume you read their mind, projecting their definition of “good search” onto your product.

Multiply that by every stakeholder in that company. Multiply that by every domain—medical search is completely dissimilar from e-commerce. Even in e-commerce- book search differs from fashion search. Passive discovery for consumers differs from B2B. Book e-commerce for librarians differs from search for sci-fi nerds. And on and on.”

Enablement is a broad category of sales tools, but in the context of knowledge retrieval we’re seeing modern search tools displace legacy players like Highspot and Seismic.

Here’s a video of what AI powered Sales Enablement looks like:

Going “Narrow” on a problem set is a faster path to adoption and clear ROI.

The wider you’re trying to go, the harder it is to get value out of Enterprise Search. A broad set of use cases makes everything harder for the buyer and the vendor. For business leaders to unlock the true value of search, they should align it with a clear set of problems.

Examples of narrow search use cases include tools like Intercom for Customer Support, Harvey for Legal document analysis, or 1up for automating sales enablement. In all of these cases, success is compounded by urgency and pain within the specific problem space.

There are many reasons to be optimistic about the future of search within businesses. We think focusing will go a long way for businesses adopting this technology.