Domain-Specific LLM and Embeddings

Speaker: Yi Yang, Associate Professor, Hong Kong University of Science and Technology
Abstract: In this talk, I will present my research on building domain-specific LLMs and embeddings within the context of finance and business. Three key works will be discussed. First, I will introduce domain-specific instruction training for developing a finance-focused LLM. Second, I will explore effective pooling and attention designs for training LLM-based embedding models. Finally, I will address whether domain-specific LLM-based embeddings are necessary, given the superiority of general embedding models.
Bio: Yi Yang is an Associate professor in the Department of Information Systems, Business Statistics and Operations Management at Hong Kong University of Science and Technology. He is the Director of the Center for Business and Social Analytics (CBSA). He received his PhD in computer science from Northwestern University. He designs machine learning methods in his research to solve challenging business and Fintech problems. His work has been published in business discipline journals such as Information Systems Research, Management Information Systems Quarterly, Journal of Marketing, Contemporary Accounting Research and INFORMS Journal on Computing. His work has also been published in top-tier machine learning and natural language processing conferences such as Annual Meeting of the Association for Computational Linguistics (ACL), Conference on Empirical Methods in Natural Language Processing (EMNLP) and International Conference on Artificial Intelligence and Statistics (AISTATS).