My main project during my internship at Supportiv was to build and accelerate a knowledge extraction pipeline. With the millions of messages that Supportiv has, it is unreasonable to have a human read through all of these messages to understand what topics people are discussing. Therefore, I used an artificial intelligence model, specifically a named entity recognition (NER) model, to extract entities from all of this text. This model learns to understand the structure of language from the millions of messages present in their database, allowing it to identify pre-existing entities, and even predict new ones. With these entities, we then used a process called "entity linking" to build clusters of related topics and then situate them in the knowledge graph. Equipped with the knowledge graph, Supportiv can better understand what its users are talking about, and thus provide better and more accurate support. The flexibility of the model means that it can predict new topics as they arise, such as the COVID pandemic, allowing Supportiv to adapt to new topics on the fly.

At Columbia, I am studying computer sScience and physics. In general, classes from both of these majors were extremely helpful towards the goal of completing my project this summer. Creating this NER model required the use of complex artificial intelligence algorithms that use a combination of calculus, statistics, and linear algebra as a foundation. The math courses I’ve taken prepared me well to tackle challenges along the way. In addition, the programming classes I’ve taken helped me understand and parse the hundreds of lines of code that express the mathematics in a language that the computers can understand and process. And finally, my physics classes have taught me in general how to reason and persevere in the face of complicated and abstract problems. This solid foundation of tools made it possible for me to deal with numerous issues that arose during the training of this NER model.

There were numerous obstacles — some easy, some hard — that I had to overcome throughout the course of my project. On the one hand, I was working in a new territory of natural language processing. While I had experience programming in general before, and training some machine learning models, I had never truly delved deep into the field of natural language processing, thus I had to familiarize myself in this new terrain. In addition, preparing the training data for our model took quite a bit of work for a few reasons. On the one hand, the input format for the model was quite specific, so I had to spend some time converting the data so that the model could understand it. On the other hand, our data was quite noisy, since the text was very conversational, making it hard sometimes for the model to understand the language. Both of these challenges meant that I had to spend a lot of time iterating and improving on the data I fed the model.

Overall, my internship with Supportiv was a truly incredible experience, and I am extremely grateful to have had the opportunity to pursue this work, even with everything else that is going on in the world right now. Going into this summer, I was hoping to learn more about this new field of artificial intelligence that I had little experience in and thanks, to my generous and welcoming co-workers, I was able to learn a lot about the way to train these natural language processing models. Furthermore, working in a fast-paced startup environment forced me to multitask and adapt to new situations all the time, while learning how to properly organize and manage large code bases. And, perhaps most importantly, the weekly Zoom meetings to discuss the progress of our projects and the frequent collaboration on Slack reminded me how important it is to stick together during such strange times.