Designing a chatbot to learn AI
In this blog
A little history
The story of WWT Chat is an interesting one. Much like the ever-changing landscape of all things artificial intelligence (AI), it evolved over several iterations. In the beginning, the objective of the Generative AI team was different: we were just trying to come to grips with what engineers could accomplish with AI. The team was not tasked with producing a deliverable product, but with learning. With that goal in mind, we started a simple chat app.
A focus on learning
We decided to make a chat app as a small toy project because it was fairly straightforward. In addition, we wanted the problem domain that we were learning to be AI, not the app we were making.
The architecture was chosen to keep learning at the forefront, a React frontend and a Python backend. At the team's inception, there was one React subject matter expert (SME) and multiple backend developers familiar with Model View Controller (MVC) and backend concepts, but with minimal exposure to Python. Originally, the chat app was developed to answer the question: "How much could AI help a well-versed developer onboard into an unfamiliar tech stack?" The protocol was not to do regular web searches, but to see how far just using AI could take us.
Responsible AI is non-negotiable at WWT. All of the work was done on either freshly installed laptops or within virtual machines. It quickly became apparent that if anyone outside of our small group was ever going to be able to experiment with this new technology safely, we would have to do something different. At that point, our primary purpose for the chat app ceased to be strictly learning and pivoted towards delivering a safe platform where WWT could safely engage with this new technology.
Our next step was to switch to a self-hosted solution that would allow us to keep our data under our control. We chose Microsoft Azure OpenAI for this, mainly because it was so well supported.
A more robust Minimum Viable Product (MVP)
Once we pivoted towards delivering an entire platform, it was time to knuckle down and get the infrastructure in place. We went with Microsoft because their tools are very well-supported and familiar to engineers and customers alike. We containerized the application so that it could be deployed anywhere, improving and expanding flexibility. Every push to the main branch triggered a new container build on a self-hosted registry and initiated a zero-downtime deployment of the application hosted by the Advanced Technology Center (ATC).
We now had a running app that was safe to use and was accessible only to people on WWT networks. The primary MVP objective was complete.
It was time for some polish. An engineer was brought in to add WWT branding, and the styling of the entire app was overhauled. The app was designed from the beginning to be modular, making swapping in and testing new language models easy. Auditing and logging were added. The foundation has been laid.