Project WHY
Web, AI Engineering, Full-Stack Development,
High-Performance Computing, Python
LLM-powered chatbot built to provide professionals with digital policy advice. Powered by The University of Warwick
The Objectives
I joined a team of PhD Researchers at The University of Warwick to create an AI-powered tool that helps professionals better understand digital policy. The endeavour, titled Project WHY, was broken down into the following three objectives:
01 Accurate data retrieval
Industry professionals frequently face challenges due to the complexity and volume of digital policy information. Traditional data platforms, which can be cumbersome and inefficient, often result in frustration and overlooked insights. This justifies the need for a system that delivers accurate information easily and efficiently.
Objective 1
Users need a system that provides easy access to accurate digital policy data
02 Simple & modern UX
Current digital policy search systems rely on manual category filters and outdated text-matching algorithms to search large databases, returning the results as simple lists of documents. This project aims to modernize the process by using chatbots to query and display the data.
Objective 2
Create a modern chat-bot interface for querying the database
03 Allow for 3rd-party data
Professionals often possess unique data and follow specific policies that aren't included in the public data collected by this system. Allowing them to securely upload their own data would enhance the system's robustness and usefulness.
Objective 3
Users need to be able to securely upload their own data into the model
Design & Implementation
As the sole developer on the team, I proceeded to tackle each objective - ensuring to learn the necessary skills and best-practices for each step to ensure that our product was the best it could be.
Objective 1
Users need a system that provides easy access to accurate digital policy data
AI & Large Language
Models (LLMs)
Given the goal of creating a chatbot and the significant advancements in AI with large language models like ChatGPT, it made sense to use these sophisticated models for our project. With support from the University of Warwick and access to high-performance computing resources, I was able to deploy and test several large language models on powerful GPUs to find the best fit for our needs.
We chose the latest model from Mistral, which had over 56 billion parameters. These text-based models were selected for their proven and continuously improving accuracy, as well as their capability to process and understand policy documents effectively. I then fine-tuned the model on modern policy data to train it to become a digital policy expert.
Retrieval-Augmented Generation (RAG)
To ensure the model accurately retrieved the right data from the database when asked for specific information, I used a technique called Retrieval Augmented Generation (RAG). This method starts by searching a vector database for all relevant information based on the user's query. The relevant data is then fed to the large language model (LLM), which formats it into a response.
RAG was crucial because it allowed the model to access new or updated data in the database, even if it hadn't been trained on this latest dataset.
Objective 2
Create a modern chat-bot interface for querying the database
Streamlit
The project's code was entirely written in Python and designed to be web-hosted. Due to the prototype nature and tight deadlines, I opted for a simpler approach rather than building a website from scratch. I used a Python library called Streamlit, which enabled me to create a sleek and modern user interface.
This interface allows users to interact with the trained model and retrieve the specific information they need. Additionally, I implemented a traditional search system, similar to existing systems, that lets users pull up entire documents. We also developed a smart assistant feature that guides users through questionnaires and project requirements, providing tailored advice and resources.
Objective 3
Users need to be able to securely upload their own data into the model
Privacy
The decisions made in implementing this product prioritized user privacy. By using locally hosted Mistral models instead of third-party, cloud-hosted OpenAI models, we ensured that user data was not transmitted to or stored in external servers unintentionally.
Furthermore, rather than storing user-uploaded data in a shared index accessible to all users, I established a local index on each user’s device. This index was queried using the Retrieval Augmented Generation (RAG) technique, and the retrieved data was then used by the model to generate responses. This approach significantly enhanced data security and privacy.
Agents & Tools
A key feature of the model was its autonomous capability to determine the best source for retrieving information — whether it be from locally uploaded data, our global database, or directly from the ongoing chat. We also explored capabilities for online searching.
This functionality was managed through an "Agent" approach, where the model decides which tool to use from a set of predefined options, such as searching the web, querying the database, or requesting more information. This method significantly enhanced the model's ability to access diverse sources and integrate them into its responses.
Development
Overview
Working in a team of four, consisting of two social science and digital policy experts and my supervisor, an AI & Natural Language Processing professor, I completed the following steps in developing this project:
- Created a development schedule and plan based on the product description.
- Evaluated several large language models (LLMs) and quickly developed a prototype for experts to test and offer feedback.
- Designed the UI and planned the functionality of the website using Figma.
- Trained the model on a dataset expertly curated by specialists in the field.
- Developed the project's features strictly according to the established plan.
- Iteratively refined the design based on feedback from stakeholders and evolving project needs.
Tech Stack
UI/UX:
Figma, Canva
Front-End:
Streamlit - Python
Back-End:
Python - Transformers, PyTorch, LangChain, LlamaIndex. Running on an Ngrok server
Version Control:
GitHub
Technical Challenges
This project demanded a thorough understanding of contemporary AI practices, a knowledge base I initially lacked. I devoted considerable time to learning new techniques, studying the latest literature, and grasping the terminology and best practices in the field. The greatest technical challenge was definitely acquiring the new skill sets necessary to complete this project effectively.
Additionally, this was my first time working with high-performance computing servers. Learning how to access and leverage their powerful resources efficiently presented an initial obstacle.
Ultimately, integrating everything I had learned into a cohesive product was a complex task. Managing multiple data sources while running a large-scale model in the background required careful coordination to ensure a seamless user experience. This project not only challenged my technical abilities but also improved my capacity to quickly learn and apply new technologies and methodologies when needed.
Conclusion
What have I learnt?
This project immersed me deeply into the world of AI and Large Language Models. I learned to apply new techniques, discover and utilize cutting-edge methods and technologies, and how to host and deploy my own models in products.
Additionally, this experience enhanced my skills in working with stakeholders and adhering to their design specifications. Unlike my previous work, which was often self-directed, this project required me to meticulously ensure that my creations aligned with the designers' visions.
Furthermore, I recognized the importance of collaborating with a team of diverse skill sets. As the only developer in a team of four and without expertise in digital policy, we engaged in extensive discussions to devise solutions that were both feasible and relevant.
