Google’s search business is finally up against a real competitor.

A new AI search company named Perplexity, focused on building and expanding knowledge, has reached 10M monthly active users (MAU) as early 2024, experiencing a staggering month-over-month growth rate of over 40%.

As of when this article was written, its MAU can be somewhere between 40M to 50M. Compared with Google search MAU, it is still tiny. However, what’s really promising is the new search experience it offers, it is not only focused on what you know, e.g. returning results based on keyword search, but it also expands on your knowledge when you are not sure what to search, e.g. don't know the keyword to search for. 

That is powered by hybrid search. 

What is Hybrid Search?

Hybrid search is an advanced search technique that combines the strengths of traditional keyword search (keyword-based) with modern semantic search capabilities (intent-based). 

Search engine results mainly depend on keyword matching. For instance, if you search for the best smartphones with high-definition cameras, the traditional keyword search only shows results with keywords with  “smartphones” and “high-definition camera” but you might miss the information like its reviews, comparison, and other context-specific insights like low-light performance, video capabilities and more.

However, semantic search understands your intention behind buying a smartphone but you can achieve more accurate results and a comprehensive set of results by combining keyword search and semantic search. And this is what hybrid search is.

Why Hybrid Search Matters in 2024?

Do you know even the top e-commerce companies like Amazon and eBay use hybrid search algorithms for better recommendations and improved experience? On the startup front, they move even faster. For example, UK-based startup Moonsift is leveraging hybrid search to help online shoppers discover the products they love. Moonsift offers an e-commerce browser extension for users to curate shoppable boards with products from across the internet, and that’s vital to deliver users with the precise results or services they wish. 

Giving users the perfect experience and making your users feel understood is essential, and that’s why hybrid search matters in 2024. 

Top Hybrid Search Solutions in 2024

There are plenty of hybrid search tools available in the market. Below we have researched the three top hybrid search solutions that you will find worth checking out.

#1 Pinecone

The Pinecone platform is a cloud-based vector database designed for search applications.  It combines vector search with keywords and familiar metadata filters to get the latest and relevant results. It offers a user API interface for semantic and multi-modal search capabilities as well as candidate generation services. Creating AI solutions is made simple with its hassle-free infrastructure. 

Key Features of Pinecone

• All-in-One Solution: Combines keyword and semantic search in a single system, simplifying implementation and management.

• Customizable Relevance: Easily adjust the balance between exact matches and related concepts to suit your business needs.

• Versatile Application: Works across various content types including text, images, and audio, making it suitable for diverse business use cases.

• Scalability: Handles large volumes of data efficiently, growing with your business without performance issues.

• User-Friendly: Integrates seamlessly with existing systems through a straightforward API, reducing technical complexity.

• Improved Accuracy: Enhances search precision by considering both specific terms and overall context, leading to better user experiences.

• Cost-Effective: Eliminates the need for multiple search solutions, potentially reducing operational costs and complexity.

• Adaptable: Supports various industry-standard search models, allowing flexibility in implementation based on specific business requirements.

Use cases:

Pinecone is useful in providing personalized recommendations, real-time search similarity, and creating AI applications that require fast and accurate searching capabilities. Some of the use cases of pinecones are: 

• E-commerce Product Search: Improving product discovery and relevance.

• Open Domain Question Answering: Enhancing accuracy in general knowledge queries.

• Contextual Chatbots: Providing more relevant responses in conversational AI.

• Personalized Search Experiences: Tailoring results based on user preferences and behavior.

• Retrieval Augmented Generation (RAG): Enhancing language model outputs with relevant information retrieval.

• Enterprise Search: Improving information retrieval across diverse corporate data.

• Content Recommendation Systems: Suggesting relevant content to users.

Official website link: https://www.pinecone.io/

#2 Weaviate

Weaviate is an open-source vector database provider, and offers Hybrid search as one of its key features. The team has expanded rapidly to over 80+ employees and servicing both startup and enterprise clients.  

Weaviate's hybrid search uses both sparse vectors (for keyword search) and dense vectors (for semantic search) to represent the meaning and context of search queries and documents.

Key Features of Weaviate:

• Combines multiple search algorithms for improved accuracy and relevance

• Generative feedback loops: Taking results generated from models, vectorizing them, and saving them back into the database for future use. This creates a cycle of data generation, storage, and retrieval that can enhance the capabilities of AI applications

• Real-time processing: Ability to search and update data in real-time, even while data is being imported or modified

• Cost-effective architecture: Strategic balancing between speed and cost, with the ability to manage large datasets without keeping everything in memory

• Flexibility: Supports various programming languages and GraphQL queries

• Scalability: Designed to scale horizontally to handle large datasets and high query volumes

• Multi-modal: Able to handle multiple data types, including text, images, and more, making it versatile for various application

• AI Model Integration: Integrates seamlessly with various AI and machine learning models

Use Cases:

Weaviate is mostly suitable for applications that need contextual understanding such as chatbots or AI-driven search engines. Some of the use cases of Weaviate are:

1. E-commerce Product Search:

   • Improves product discovery by combining exact keyword matches with semantically related items

   • Enhances user experience and potentially increases conversion rates

2. Content Recommendation Systems:

   • Delivers more relevant content suggestions by understanding both specific terms and overall context

   • Increases user engagement and time spent on platform

3. Knowledge Management Systems:

   • Facilitates more efficient information retrieval in corporate environments

   • Improves employee productivity by providing more accurate search results

Official website link: https://weaviate.io/

#3 Elasticsearch 

Elasticsearch is a popular open-source search engine plugin that is capable of handling a diverse range of data types. It is known for its lightning-fast search and fine-tuned relevancy capabilities. The company behind Elasticsearch is Elastic, long established since 2012 has grown significantly since its founding and went public in 2018.

Key Features of Elasticsearch:

• Full-text search capabilities: Leveraging an inverted index structure for fast and efficient searching across large volumes of text data, supporting complex queries and phrase searches.

• Scalability: Ability to scale horizontally across multiple nodes in a cluste

• Real-time processing: offers near real-time search and analytics capabilities, allowing for quick data ingestion and immediate searchability

• Flexibility: RESTful API and JSON support make it easy to integrate with various programming languages and tools

• Schema-free and documented-oriented: Allowing for flexible data storage without requiring a predefined schema, and easy ingestion of structured and unstructured data

• Geospatial support: Ability to handle location-based queries and analytics efficiently

• Automatic node recovery: Built-in feature that helps maintain cluster health when nodes fail or leave the cluster

• Cross-cluster replication: Enables replication of indices from one Elasticsearch cluster to another; useful for disaster recovery, data locality, and centralized reporting scenarios

• Top-notch security: Supports multi-tenancy and provides robust security features, including role-based access control, encryption, and audit logging

Use Cases:

An elastic search plugin is best suited for e-commerce websites, security labs, and more especially those that need advanced product searches, recommendation engines, and enterprise knowledge management systems some of the use cases of elastic search are:

• Geospatial Data Search

• Log and Event Data Analysis

• Website and E-commerce Search Engines

• Business Intelligence

Official website link: https://www.elastic.co/elasticsearch

Comparison of the 3 Hybrid Search Solutions 

Conclusion:

Anticipating Google is facing more scrutiny from the US Department of Justice (DoJ), this will send out a shockwave to the rest of its business including google search business. This will push for more adoption of new search types of experience to match its incoming competitors such as Perplexity. What it means to the world is while constant data growing and user-changing needs, it's essential to go beyond basic keyword searches and adopt hybrid search solutions into your product stack, to enhance the user experience when tackling intricate queries, and stay competitive and relevant.  

If you would like to be kept in the loop, join more than 1500 AI innovators on this journey together.