Yilun Liu

PERFT: Parameter-Efficient Routed Fine-Tuning for Mixture-of-Expert Model

» arXiv preprint arXiv:2411.08212

• A unified framework for integrating PEFT with MoE LLMs
• PERFT as a family of adaptation strategies, with extensive experiments showing effectiveness and scalability

SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification

» Findings of the Association for Computational Linguistics ACL 2024, 4666-4682

• A lightweight plug-and-play text classification framework using LLM’s internal representations
• Consistently outperforming conventional methods in terms of performance, efficiency, and interpretability

Leveraging Intermediate Neuron Activations from Pretained Large Language Models » Details
for Application Project: A Solution for Large-Scale Analysis on Tweets
Professorship of Computational Social Science and Big Data, Technical University of Munich
Tutor: Daniel Matter, Prof. Jürgen Pfeffer

• A light weight method that leverages activation patterns of identified salient neurons from feed-forward neuron networks within transformer blocks across different layers of pretrained LLMs for text classification tasks
• Experiment results suggest that proposed method notably outperforms the original paradigm of LLM for sequence classification

Munich, Bavaria, Germany

Multidimensional Analysis on the Political Spectrum of Media Ideologies » Details
for Master Seminar: Computational Social Science
Professorship of Computational Social Science and Big Data, Technical University of Munich
Tutor: Prof. Jürgen Pfeffer

• Large-scale NLP analysis over 5,000 news articles from the Daily Mail and the New York Times concerning the topic of climate changing over the period 2018–2022
• Representation learning process consisting of sentence embedding, PCA and clustering to automatically mine vector representations of different sub-topic clusters
• Sentiment analysis and vector projection to obtain the sentiment representation components of each article on different sub-topic dimensions
• Time-series analysis and visualization reveals that different media outlets hold different positions on the spectrum of some sub-topics, and such positions shift over time

Munich, Bavaria, Germany

Master Praktikum: Machine Learning in Crowd Modeling & Simulation » Details
Chair of Scientific Computing in Computer Science (SCCS), Technical University of Munich
Tutor: Dr. Felix Dietrich

• Systematically studied and implemented algorithms about modeling of crowds, dynamical systems theory, machine learning techniques, and numerical analysis of complex systems
• Team projects on development of extensions for simulation software (Vadere), validation of models and data, and implementation of neural networks applied to simulation results for modelling pedestrian movement speed based on distances to the nearest neighbors

Munich, Bavaria, Germany

Bachelor’s Thesis: Design and Implementation of Deep-Learning Based Algorithms for Anomaly Detection in Industrial Scenarios » Details
School of Software Engineering, Xi’an Jiaotong University
Tutor: Prof. Jizhong Zhao, Prof. Wei Xi

• Designed and implemented a deep neural network system for anomaly detection in industrial images, which combines the structures of Convolutional Neural Network (CNN) and Generative Adversarial Network (GAN)
• By reconstructing normal image patterns based on test samples and generating pixel-level masks of potential anomalous regions, the system could simultaneously perform classification, location, and segmentation of anomalies in images, and addresses issues such as small sample problems and background noise.
• Results tested on datasets including MNIST, CIFAR-10, and MVTecAD and has shown high performance with F1 index of 98.76%, 89.16%, and 79.45% respectively.

Xi’an, Shaanxi, China

Lab Research: Data Mining in Social Networks » Details
Ministry of Education Key Lab for Intelligent Networks and Network Security, Xi’an Jiaotong University
Tutor: Prof. Xiaohong Guan, Prof. Chao Shen, Assoc. Prof. Xiaoming Liu

• Conducted research on linking the users across different social networks
• Participated in developing systems for dynamically demonstrating data collected from social networks, introducing textual information to jointly perform alignment on heterogeneous networks, and revealing the potential relationships between different users

Xi’an, Shaanxi, China

International Polytechnic Summer School:
Machine Learning – Theory and Application » Details
Peter the Great St. Petersburg Polytechnic University
Tutor: Dr. Nikita Kudriashov, Dr. Oğul Ünal

• Conducted studies on regression predictions with a variety of machine learning methodologies

Saint Petersburg, Russia

Remote Collaboration Project: Chatbot Design and Development » Details
MIT IBM Watson AI Lab
Tutor: Prof. Fan Zhang

• A Rasa-NLP based chatbot, with built and trained language models from scratch in specific domains

Cambridge, MA, U.S.

International Genetically Engineered Machine Competition (iGEM) 2019
XJTU-CHINA Team Project: Bacterial Fragrance Generator » Details
School of Life Science & Technology, Xi’an Jiaotong University

• Involved in designing a synthetic biology system consisting of engineered E. coli and light-controlled gene circuit to improve sleep quality by selectively producing 2 fragrance molecules: linalool at night and limonene in the morning
• The Team XJTU-CHINA won a gold medal award in iGEM 2019.

Xi’an, Shaanxi, China

Exchange Visit: Unsupervised Training on Graphs » Details
Tsinghua University Knowledge Engineering Group (THUKEG)
Tutor: Prof. Jie Tang, Dr. Chenhui Zhang

• Studied cutting-edge methodologies of graph mathematics and data mining in social networks, participated in research on graph pretraining, especially representative graph convolutional networks

Beijing, China

Team Project: Huihe – “Cloud + Client” Intelligent Visual Recognition Solutions » Details
Heils Technology Co., Ltd, Shenzhen

• A Computer Vision based commercial solution with combination of hardware and software, being tested and implemented in coal mining and tyre industries and proven to improve productivity
• This team project won a silver award in the 2018 National Undergraduate Entrepreneurship Competition and a gold award in the Shaanxi Provincial Semi-Final of the 4th China “Internet+” College Students Innovation & Entrepreneurship Competition

Shenzhen, Guangdong, China; Xi’an, Shaanxi, China

Mobile Application Team: XJ-LINK Development and Maintenance » Details
Data and Information Center, Xi’an Jiaotong University

• Involved in developing, maintaining, and updating the campus mobile application “XJ-LINK” for students to check schedules, grades, library borrowings, teaching assessment, club activities, etc.
• By 2018, the application has over 12,000 users registered, with a peak of 7,000 monthly active users
• The team project XJ-LINK: Design & Implementation of Digital Campus Information Systems Basing on Mobile Internet won a silver award in the 2018 Entrepreneurship Practice Competition of XJTU.

Xi’an, Shaanxi, China