We were tasked to build a smart search engine to provide fast searches powered by Artificial Intelligence, Machine Learning, Knowledge Graph, and personalized relevance tuning.
Challenge:
A search engine is a complex task, time-consuming, and needs proper DevOps integration
Setting up ElasticSearch cluster, eContext API for Artificial Intelligence / Machine Learning / Natural Language Processing
Fine-tuning on big data set to provide accurate and personalized search results
Solution:
Architect, design and implement ElasticSearch Cluster
Setup web crawler and Indexer applications which consume RSS feed from a various content provider like Bing, Google, Yahoo. etc.
Design and implement Search-API using Golang which extracts search results from Elasticsearch cluster for our end users
Deploy web application with continuous integration using Kubernetes on containerized environment
Setup stream and crawl ingestion cluster by using Kafka and RabbitMQ
Results:
Several hundred users were successfully migrated to new search API within a set timeframe.
Enhanced and improved performance of cloud-based infrastructure, reduction in cost, and improved search results.
We integrated a big data set powered by Artificial Intelligence, Machine Learning, and Natural Language Processing engines.
We reduced load from back end servers to ensure immutable, tamper-proof search results.
