Researcher basic information

• PhD (Engineering), Huazhong University of Science and Technology, Dec. 2020
• Bachelor (Engineering), Hefei University of Technology, Jun. 2014

• https://researchmap.jp/LingX?lang=en

• https://orcid.org/0000-0002-4650-8841
• https://jglobal.jst.go.jp/detail?JGLOBAL_ID=202401020380361198

• 40946787

• https://orcid.org/0000-0002-4650-8841

• 202401020380361198

• Robotic AI
• Large multimodal model
• Multimodal Processing
• Recommendation System
• Defect Detection
• Artificial Intelligence
• Computer Vision

• Informatics, Intelligent robotics
• Informatics, Intelligent informatics

• Bachelor's degree program, School of Engineering
• Master's degree program, Graduate School of Information Science and Technology
• Doctoral (PhD) degree program, Graduate School of Information Science and Technology

Research activity information

• May 2026, シドニー工科大学（University of Technology Sydney, UTS）, UTS Visiting Fellowship
• Mar. 2026, NVIDIA, NVIDIA Academic Grant Program Award
  Ling Xiao
• Feb. 2026, 画像工学研究会, IEICE IE Award
  A Multimodal Dataset for Socially Compliant Navigation in Urban Environments
  Zhuonan Liu, Zishuo Wang, Xinyu Zhang, Tomohito Kawabata, Ling Xiao
• Jun. 2025, The 15th ACM International Conference on Multimedia Retrieval (ICMR), Best paper award
  TourMLLM: A Retrieval-Augmented Multimodal Large Language Model for Multitask Learning in the Tourism Domain
  Hiromasa Yamanishi;Ling Xiao;Toshihiko Yamasaki
• Feb. 2024, 画像工学研究会, IEICE IE賞
  継続学習における敵対的頑健性の向上
  向井皇喜;熊野創一郎;Nicolas Michel;Ling Xiao;山崎俊彦

• HumAIN: Human-Aware Implicit Social Robot Navigation
  Daeun Song; Nhat Le; Jeffrey Chen; Mohammad Nazeri; Amirreza Payandeh; Rohan Chandra; Reuth Mirsky; Ross Mead; Ling Xiao; Xuesu Xiao
  Proceedings of the 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2026), Sep. 2026, [Peer-reviewed]
  English, International conference proceedings
• LLM-Advisor: An LLM Advisor for Cost-efficient Path Planning across Multiple Terrains
  Ling Xiao; Toshihiko Yamasaki
  IEEE Transactions on Automation Science and Engineering, Aug. 2026, [Peer-reviewed], [Lead author]
  English, Scientific journal, 54594061
• SOPD-SocialNav: Selective On-Policy Distillation for Vision-Language Social Navigation
  2026 WRC Symposium on Advanced Robotics and Automation (WRC SARA) (WRC SARA 2026), Aug. 2026, [Peer-reviewed], [Last author, Corresponding author]
  English, International conference proceedings
• LLM Guided Multi Style Typography and Layout Generation via Dynamic Direct Preference Optimization
  Chen FU; Shengzhou Yi; Ling Xiao; Toshihiko Yamasaki
  Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings, 5725, 5734, Jun. 2026, [Peer-reviewed]
  English, International conference proceedings
• Deconstructing the Failure of Ideal Noise Correction: A Three-Pillar Diagnosis
  Chen Feng, Zhuo Zhi, Zhao Huang, Jiawei Ge, Ling Xiao, Nicu Sebe, Georgios Tzimiropoulos, Ioannis Patras
  Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 34512, 34523, Jun. 2026, [Peer-reviewed]
  English, International conference proceedings
• E-SocialNav: Efficient Socially Compliant Navigation with Language Models
  Ling Xiao; Daeun Song; Xuesu Xiao; Toshihiko Yamasaki
  Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 20077, 20081, May 2026, [Peer-reviewed], [Lead author]
  English, International conference proceedings
• A Multihead Continual Learning Framework for Fine-Grained Fashion Image Retrieval with Contrastive Learning and Exponential Moving Average Distillation
  Ling Xiao; Toshihiko Yamasaki
  IEEE Transactions on Multimedia, 1, 10, Apr. 2026, [Peer-reviewed], [Lead author]
  English, Scientific journal, 47287350
• MAction-SocialNav: Multi-Action Socially Compliant Navigation via Reasoning-enhanced Prompt Tuning
  Zishuo Wang; Xinyu Zhang; Zhuonan Liu; Tomohito Kawabata; Daeun Song; Xuesu Xiao; Ling Xiao
  2026, [Peer-reviewed], [Last author, Corresponding author]
  English, Scientific journal, 47287350
• TourMLLM: A Retrieval-Augmented Multimodal Large Language Model for Multitask Learning in the Tourism Domain
  Hiromasa Yamanishi; Ling Xiao; Toshihiko Yamasaki
  Proceedings of the 2025 International Conference on Multimedia Retrieval, 1654, 1663, ACM, 30 Jun. 2025, [Peer-reviewed], [Corresponding author]
  International conference proceedings
• ActRecognition-GPT: Utilizing Multimodal Large Language Models for Spatiotemporal Action Recognition in Nursery Videos
  Kenta Watanabe; Shuntaro Masuda; Ling Xiao; Toshihiko Yamasaki
  Proceedings of the IEEE 19th International Conference on Automatic Face and Gesture Recognition (FG), 1, 10, IEEE, 26 May 2025, [Peer-reviewed]
  International conference proceedings
• Explainable AI for Image Aesthetic Evaluation Using Vision-Language Models
  Supatta Viriyavisuthisakul; Shun Yoshida; Kaede Shiohara; Ling Xiao; Toshihiko Yamasaki
  Proceedings of the Conference on Artificial Intelligence x Multimedia (AIxMM), 62, 65, IEEE, 03 Feb. 2025, [Peer-reviewed]
  International conference proceedings
• Multi-level knowledge distillation for fine-grained fashion image retrieval
  Ling Xiao; Toshihiko Yamasaki
  Knowledge-Based Systems, 310, 112955, 112955, Elsevier BV, Feb. 2025, [Peer-reviewed], [Lead author]
  Scientific journal, 47287350
• Combining Non-Numerical Text and Numerical Sequences in LLM-based Survival Prediction
  Zijie Zhou; Guoqing Qian; Xinyi Jiang; Guoming Wang; Rongxing Lu; Ling Xiao; Siliang Tang
  Proceedings of the 22nd Pacific Rim International Conference Series on Artificial Intelligence (PRICAI 2025), 256, 271, 2025, [Peer-reviewed]
  International conference proceedings
• Incorporating Semantic Visual Content into Click-Through Rate Prediction for Video Advertisements
  Yoshiaki Tanabe; Shuntaro Masuda; Gakumatsu Ryu; Naoto Tanji; Hiroyuki Seshime; Ling Xiao; Toshihiko Yamasaki
  Proceedings of the 17th Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC 2025), 1916, 1921, 2025, [Peer-reviewed]
  International conference proceedings
• GeoDCL: Weak Geometrical Distortion based Contrastive Learning for Fine-grained Fashion Image Retrieval
  Ling Xiao; Toshihiko Yamasaki
  IEEE Transactions on Artificial Intelligence, 1, 13, Institute of Electrical and Electronics Engineers (IEEE), 2025, [Peer-reviewed], [Lead author]
  Scientific journal, 47287350
• LITA: LMM-Guided Image-Text Alignment for Art Assessment
  Tatsumi Sunada; Kaede Shiohara; Ling Xiao; Toshihiko Yamasaki
  Proceedings of the International Conference on Multimedia Modeling (MMM), 268, 281, Springer Nature Singapore, 31 Dec. 2024, [Peer-reviewed]
  International conference proceedings
• LLaVA-Tour: A Large Multimodal Model for Japanese Tourist Spot Prediction and Review Generation
  Hiromasa Yamanishi; Ling Xiao; Toshihiko Yamasaki
  Proceedings of the IEEE International Conference on Visual Communications and Image Processing (VCIP), 1, 5, IEEE, 08 Dec. 2024, [Peer-reviewed], [Corresponding author]
  International conference proceedings
• Language-Guided Self-Supervised Video Summarization Using Text Semantic Matching Considering the Diversity of the Video
  Tomoya Sugihara; Shuntaro Masuda; Ling Xiao; Toshihiko Yamasaki
  Proceedings of the 6th ACM International Conference on Multimedia in Asia, abs/2405.08890, 1, ACM, 03 Dec. 2024, [Peer-reviewed], [Corresponding author]
  International conference proceedings, Current video summarization methods primarily depend on supervised computer
  vision techniques, which demands time-consuming manual annotations. Further,
  the annotations are always subjective which make this task more challenging. To
  address these issues, we analyzed the feasibility in transforming the video
  summarization into a text summary task and leverage Large Language Models
  (LLMs) to boost video summarization. This paper proposes a novel
  self-supervised framework for video summarization guided by LLMs. Our method
  begins by generating captions for video frames, which are then synthesized into
  text summaries by LLMs. Subsequently, we measure semantic distance between the
  frame captions and the text summary. It's worth noting that we propose a novel
  loss function to optimize our model according to the diversity of the video.
  Finally, the summarized video can be generated by selecting the frames whose
  captions are similar with the text summary. Our model achieves competitive
  results against other state-of-the-art methods and paves a novel pathway in
  video summarization.
• SCOMatch: Alleviating Overtrusting in Open-Set Semi-supervised Learning
  Zerun Wang; Liuyu Xiang; Lang Huang; Jiafeng Mao; Ling Xiao; Toshihiko Yamasaki
  Proceedings of the European Conference on Computer Vision, 217, 233, Springer Nature Switzerland, 29 Oct. 2024, [Peer-reviewed]
  English, International conference proceedings
• Adversarially Robust Continual Learning with Anti-Forgetting Loss
  Koki Mukai; Soichiro Kumano; Nicolas Michel; Ling Xiao; Toshihiko Yamasaki
  Proceedings of the IEEE International Conference on Image Processing (ICIP), 1085, 1091, IEEE, 27 Oct. 2024, [Peer-reviewed]
  International conference proceedings
• E-ReaRev: Adaptive Reasoning for Question Answering over Incomplete Knowledge Graphs by Edge and Meaning Extensions
  Xiaotong Ye; Ling Xiao; Chi Zhang; Toshihiko Yamasaki
  Proceedings of the International Conference on Applications of Natural Language to Information Systems, 85, 95, Springer Nature Switzerland, 20 Sep. 2024, [Peer-reviewed]
  International conference proceedings
• Improving Plasticity in Online Continual Learning via Collaborative Learning
  Maorong Wang; Nicolas Michel; Ling Xiao; Toshihiko Yamasaki
  Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 23460, 23469, IEEE, 16 Jun. 2024, [Peer-reviewed]
  International conference proceedings
• Rethinking Momentum Knowledge Distillation in Online Continual Learning.
  Nicolas Michel; Maorong Wang; Ling Xiao 0001; Toshihiko Yamasaki
  Proceedings of the International Conference on Machine Learning (ICML), abs/2309.02870, 35607, 35622, 2024, [Peer-reviewed]
  International conference proceedings
• Boosting Fine-grained Fashion Retrieval with Relational Knowledge Distillation
  Ling Xiao; Toshihiko Yamasaki
  Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 8229, 8234, 2024, [Peer-reviewed], [Lead author]
  International conference proceedings
• A Multimodal Dataset and Benchmark for Tourism Review Generation
  Hiromasa Yamanishi; Ling Xiao; Toshihiko Yamasaki
  Proceedings of the ACM International Conference on Recommender Systems Workshops, 3886, 49, 67, 2024, [Peer-reviewed], [Corresponding author]
  International conference proceedings
• HetSpot: Analyzing Tourist Spot Popularity with Heterogeneous Graph Neural Network.
  Hiromasa Yamanishi; Ling Xiao 0001; Toshihiko Yamasaki
  Proceedings of the 6th International Conference on Image, Video and Signal Processing, 111, 120, 2024, [Peer-reviewed], [Corresponding author]
  International conference proceedings
• LiFSO-Net: A lightweight feature screening optimization network for complex-scale flat metal defect detection.
  Hao Zhong; Ling Xiao 0001; Haifeng Wang; Xin Zhang; Chenhui Wan; Youmin Hu; Bo Wu
  Knowl. Based Syst., 304, 112520, 112520, 2024, [Peer-reviewed]
  Scientific journal
• STFE-Net: A multi-stage approach to enhance statistical texture feature for defect detection on metal surfaces.
  Hao Zhong; Daxing Fu; Ling Xiao; Fang Zhao; Jie Liu 0017; Youmin Hu; Bo Wu 0006
  Advanced engineering informatics, 61, 102437, 102437, Elsevier BV, 2024, [Peer-reviewed]
  Scientific journal
• Attribute-Guided Multi-Level Attention Network for Fine-Grained Fashion Retrieval
  Ling Xiao; Toshihiko Yamasaki
  IEEE Access, 12, 48068, 48080, 2024, [Peer-reviewed], [Lead author]
  Scientific journal, Fine-grained fashion retrieval searches for items that share a similar
  attribute with the query image. Most existing methods use a pre-trained feature
  extractor (e.g., ResNet 50) to capture image representations. However, a
  pre-trained feature backbone is typically trained for image classification and
  object detection, which are fundamentally different tasks from fine-grained
  fashion retrieval. Therefore, existing methods suffer from a feature gap
  problem when directly using the pre-trained backbone for fine-tuning. To solve
  this problem, we introduce an attribute-guided multi-level attention network
  (AG-MAN). Specifically, we first enhance the pre-trained feature extractor to
  capture multi-level image embedding, thereby enriching the low-level features
  within these representations. Then, we propose a classification scheme where
  images with the same attribute, albeit with different values, are categorized
  into the same class. This can further alleviate the feature gap problem by
  perturbing object-centric feature learning. Moreover, we propose an improved
  attribute-guided attention module for extracting more accurate
  attribute-specific representations. Our model consistently outperforms existing
  attention based methods when assessed on the FashionAI (62.8788% in MAP),
  DeepFashion (8.9804% in MAP), and Zappos50k datasets (93.32% in Prediction
  accuracy). Especially, ours improves the most typical ASENet_V2 model by 2.12%,
  0.31%, and 0.78% points in FashionAI, DeepFashion, and Zappos50k datasets,
  respectively. The source code is available in
  https://github.com/Dr-LingXiao/AG-MAN.
• Online Open-set Semi-supervised Object Detection via Semi-supervised Outlier Filtering.
  Zerun Wang; Ling Xiao; Liuyu Xiang; Zhaotian Weng; Toshihiko Yamasaki
  CoRR, abs/2305.13802, 2023
  Scientific journal
• Learning Fashion Compatibility with Color Distortion Prediction
  Ling Xiao; Xiaofeng Zhang; Toshihiko Yamasaki
  Proceedings of the IEEE 6th International Conference on Multimedia Information Processing and Retrieval (MIPR), 81, 84, 2023, [Peer-reviewed], [Lead author]
  English, International conference proceedings
• Toward a More Robust Fine-Grained Fashion Retrieval.
  Ling Xiao; Xiaofeng Zhang; Toshihiko Yamasaki
  Proceedings of the IEEE 6th International Conference on Multimedia Information Processing and Retrieval (MIPR), 1, 4, 2023, [Peer-reviewed], [Lead author]
  English, International conference proceedings
• Bridging the Capacity Gap for Online Knowledge Distillation.
  Maorong Wang; Hao Yu; Ling Xiao; Toshihiko Yamasaki
  Proceedings of the IEEE 6th International Conference on Multimedia Information Processing and Retrieval (MIPR), 1, 4, 2023, [Peer-reviewed]
  English, International conference proceedings
• Sat: Self-Adaptive Training for Fashion Compatibility Prediction
  Ling Xiao; Toshihiko Yamasaki
  Proceedings of the IEEE International Conference on Image Processing (ICIP), 2431, 2435, IEEE, 16 Oct. 2022, [Peer-reviewed], [Lead author]
  English, International conference proceedings
• Missing Small Fastener Detection Using Deep Learning
  Ling Xiao; Bo Wu; Youmin Hu
  IEEE Transactions on Instrumentation and Measurement, 70, 1, 9, Institute of Electrical and Electronics Engineers ({IEEE}), 2021, [Peer-reviewed], [Lead author]
  English, Scientific journal
• Surface Defect Detection Using Image Pyramid
  Ling Xiao; Bo Wu; Youmin Hu
  IEEE Sensors Journal, 20, 13, 7181, 7188, Institute of Electrical and Electronics Engineers ({IEEE}), 01 Jul. 2020, [Peer-reviewed], [Lead author]
  Scientific journal
• A Hierarchical Features-Based Model for Freight Train Defect Inspection
  Ling Xiao; Bo Wu; Youmin Hu; Jie Liu
  IEEE Sensors Journal, 1, 1, Institute of Electrical and Electronics Engineers ({IEEE}), 01 Mar. 2020, [Peer-reviewed], [Lead author]
  English, Scientific journal
• OSED: Object-specific edge detection.
  Ling Xiao; Bo Wu 0006; Youmin Hu
  Journal of Visual Communication and Image Representation, 72, 102918, 102918, Elsevier BV, 2020, [Peer-reviewed], [Lead author]
  English, Scientific journal
• Detection of powder bed defects in selective laser sintering using convolutional neural network
  Xiao, Ling; Lu, Mingyuan; Huang, Han
  The International Journal of Advanced Manufacturing Technology, 107, 5–6, 2485, 2496, Springer, 2020, [Peer-reviewed], [Lead author]
  English, Scientific journal
• Surface Defect Detection using Hierarchical Features.
  Ling Xiao; Tao Huang; Bo Wu 0006; Youmin Hu; Jiehan Zhou
  Proceedings of the 15th IEEE International Conference on Automation Science and Engineering(CASE), 1592, 1596, IEEE, 2019, [Peer-reviewed], [Lead author]
  English, International conference proceedings

• Enhancing Lightweight Vision Language Models through Group Competitive Learning for Socially Compliant Navigation
  Xinyu Zhang; Atsushi Konno; Toshihiko Yamasaki; Ling Xiao, 12 Mar. 2026
  Social robot navigation requires a sophisticated integration of scene semantics and human social norms. Scaling up Vision Language Models (VLMs) generally improves reasoning and decision-making capabilities for socially compliant navigation. However, increased model size incurs substantial computational overhead, limiting suitability for real-time robotic deployment. Conversely, lightweight VLMs enable efficient inference but often exhibit weaker reasoning and decision-making performance in socially complex environments. Achieving both strong reasoning ability and efficiency remains an open challenge. To bridge this gap, we propose Group Competitive Learning (GCL), a strategy designed to amplify the capabilities of lightweight VLMs. Our strategy introduces the Group Competitive Objective (GCO) to harmonize global semantics with distributional regularization, alongside Asymmetric Group Optimization (AGO) to explore the upper limits of model performance. Empirical evaluations on social navigation benchmarks demonstrate that GCL significantly elevates VLM performance. Specifically, GCL enables the Qwen2.5-VL-3B learner model and guide Qwen3-VL-4B to achieve an F1 score of 0.968 and 0.914, representing 40\% and 12\% improvement over vanilla supervised fine-tuning (SFT). Notably, under vanilla SFT, the 3B model initially trails the 8B model (F1: 0.692 vs. 0.755). However, through the GCL, the 3B model outperforms (28\%) the 8B baseline model. These results suggest that GCL provides an effective solution for achieving both high accuracy and computational efficiency in real-world deployment.
• Spectral Probing of Feature Upsamplers in 2D-to-3D Scene Reconstruction
  Ling Xiao; Yuliang Xiu; Yue Chen; Guoming Wang; Toshihiko Yamasaki, 06 Mar. 2026
  A typical 2D-to-3D pipeline takes multi-view images as input, where a Vision Foundation Model (VFM) extracts features that are spatially upsampled to dense representations for 3D reconstruction. If dense features across views preserve geometric consistency, differentiable rendering can recover an accurate 3D representation, making the feature upsampler a critical component. Recent learnable upsampling methods mainly aim to enhance spatial details, such as sharper geometry or richer textures, yet their impact on 3D awareness remains underexplored. To address this gap, we introduce a spectral diagnostic framework with six complementary metrics that characterize amplitude redistribution, structural spectral alignment, and directional stability. Across classical interpolation and learnable upsampling methods on CLIP and DINO backbones, we observe three key findings. First, structural spectral consistency (SSC/CSC) is the strongest predictor of NVS quality, whereas High-Frequency Spectral Slope Drift (HFSS) often correlates negatively with reconstruction performance, indicating that emphasizing high-frequency details alone does not necessarily improve 3D reconstruction. Second, geometry and texture respond to different spectral properties: Angular Energy Consistency (ADC) correlates more strongly with geometry-related metrics, while SSC/CSC influence texture fidelity slightly more than geometric accuracy. Third, although learnable upsamplers often produce sharper spatial features, they rarely outperform classical interpolation in reconstruction quality, and their effectiveness depends on the reconstruction model. Overall, our results indicate that reconstruction quality is more closely related to preserving spectral structure than to enhancing spatial detail, highlighting spectral consistency as an important principle for designing upsampling strategies in 2D-to-3D pipelines.
• Probing Prompt Design for Socially Compliant Robot Navigation with Vision Language Models
  Ling Xiao; Toshihiko Yamasaki, 21 Jan. 2026
  Language models are increasingly used for social robot navigation, yet existing benchmarks largely overlook principled prompt design for socially compliant behavior. This limitation is particularly relevant in practice, as many systems rely on small vision language models (VLMs) for efficiency. Compared to large language models, small VLMs exhibit weaker decision-making capabilities, making effective prompt design critical for accurate navigation. Inspired by cognitive theories of human learning and motivation, we study prompt design along two dimensions: system guidance (action-focused, reasoning-oriented, and perception-reasoning prompts) and motivational framing, where models compete against humans, other AI systems, or their past selves. Experiments on two socially compliant navigation datasets reveal three key findings. First, for non-finetuned GPT-4o, competition against humans achieves the best performance, while competition against other AI systems performs worst. For finetuned models, competition against the model's past self yields the strongest results, followed by competition against humans, with performance further influenced by coupling effects among prompt design, model choice, and dataset characteristics. Second, inappropriate system prompt design can significantly degrade performance, even compared to direct finetuning. Third, while direct finetuning substantially improves semantic-level metrics such as perception, prediction, and reasoning, it yields limited gains in action accuracy. In contrast, our system prompts produce a disproportionately larger improvement in action accuracy, indicating that the proposed prompt design primarily acts as a decision-level constraint rather than a representational enhancement.
• MUSON: A Reasoning-oriented Multimodal Dataset for Socially Compliant Navigation in Urban Environments
  Zhuonan Liu; Xinyu Zhang; Zishuo Wang; Tomohito Kawabata; Xuesu Xiao; Ling Xiao, 28 Dec. 2025
  Socially compliant navigation requires structured reasoning over dynamic pedestrians and physical constraints to ensure safe and interpretable decisions. However, existing social navigation datasets often lack explicit reasoning supervision and exhibit highly long-tailed action distributions, limiting models' ability to learn safety-critical behaviors. To address these issues, we introduce MUSON, a multimodal dataset for short-horizon social navigation collected across diverse indoor and outdoor campus scenes. MUSON adopts a structured five-step Chain-of-Thought annotation consisting of perception, prediction, reasoning, action, and explanation, with explicit modeling of static physical constraints and a rationally balanced discrete action space. Compared to SNEI, MUSON provides consistent reasoning, action, and explanation. Benchmarking multiple state-of-the-art Small Vision Language Models on MUSON shows that Qwen2.5-VL-3B achieves the highest decision accuracy of 0.8625, demonstrating that MUSON serves as an effective and reusable benchmark for socially compliant navigation. The dataset is publicly available at https://huggingface.co/datasets/MARSLab/MUSON
• SocialNav-MoE: A Mixture-of-Experts Vision Language Model for Socially Compliant Navigation with Reinforcement Fine-Tuning
  Tomohito Kawabata; Xinyu Zhang; Ling Xiao, 15 Dec. 2025
  For robots navigating in human-populated environments, safety and social compliance are equally critical, yet prior work has mostly emphasized safety. Socially compliant navigation that accounts for human comfort, social norms, and contextual appropriateness remains underexplored. Vision language models (VLMs) show promise for this task; however, large-scale models incur substantial computational overhead, leading to higher inference latency and energy consumption, which makes them unsuitable for real-time deployment on resource-constrained robotic platforms. To address this issue, we investigate the effectiveness of small VLM and propose SocialNav-MoE, an efficient Mixture-of-Experts vision language model for socially compliant navigation with reinforcement fine-tuning (RFT). We further introduce a semantic similarity reward (SSR) to effectively leverage RFT for enhancing the decision-making capabilities. Additionally, we study the effectiveness of different small language model types (Phi, Qwen, and StableLM), routing strategies, and vision encoders (CLIP vs. SigLIP, frozen vs. fine-tuned). Experiments on the SNEI dataset demonstrate that SocialNav-MoE achieves an excellent balance between navigation accuracy and efficiency. The proposed SSR function is more effective than hard-level and character-level rewards. Source code will be released upon acceptance.
• 大規模言語モデルを活用した自己教師あり学習によるビデオ要約
  杉原, 朋弥; 増田, 俊太郎; 肖, 玲; 山崎, 俊彦, 第86回全国大会講演論文集, 2024, 1, 653, 654, 01 Mar. 2024
  既存のビデオ要約手法は重要シーン抽出にコンピュータービジョン技術をベースにしていて、大量のアノテーションデータが必要である。しかし、人手によるアノテーションは主観的である上にコストが高いため教師データの作成難易度が高い。そこで本研究では、近年の大規模言語モデルの進歩を活用した、自己教師あり学習に基づく新しいフレームワークを提案する。具体的には、フレームからキャプションを生成して映像を言語化し、大規模言語モデルにより映像の要約を作成する。この要約を教師データとして使用して、自然言語処理による新しいビデオ要約手法を実現した。本研究はビデオ要約の分野に新しい方向性を示し、既存の課題の解決に寄与する。, Japanese
• Rethinking Momentum Knowledge Distillation in Online Continual Learning
  Nicolas Michel; Maorong Wang; Ling Xiao; Toshihiko Yamasaki, Forty-first International Conference on Machine Learning, abs/2309.02870, 2024
  Online Continual Learning (OCL) addresses the problem of training neural
  networks on a continuous data stream where multiple classification tasks emerge
  in sequence. In contrast to offline Continual Learning, data can be seen only
  once in OCL, which is a very severe constraint. In this context, replay-based
  strategies have achieved impressive results and most state-of-the-art
  approaches heavily depend on them. While Knowledge Distillation (KD) has been
  extensively used in offline Continual Learning, it remains under-exploited in
  OCL, despite its high potential. In this paper, we analyze the challenges in
  applying KD to OCL and give empirical justifications. We introduce a direct yet
  effective methodology for applying Momentum Knowledge Distillation (MKD) to
  many flagship OCL methods and demonstrate its capabilities to enhance existing
  approaches. In addition to improving existing state-of-the-art accuracy by more
  than $10\%$ points on ImageNet100, we shed light on MKD internal mechanics and
  impacts during training in OCL. We argue that similar to replay, MKD should be
  considered a central component of OCL. The code is available at
  \url{https://github.com/Nicolas1203/mkd_ocl}.
• Online Open-set Semi-supervised Object Detection with Dual Competing Head
  Zerun Wang; Ling Xiao; Liuyu Xiang; Zhaotian Weng; Toshihiko Yamasaki, 23 May 2023
  Open-set semi-supervised object detection (OSSOD) task leverages practical
  open-set unlabeled datasets that comprise both in-distribution (ID) and
  out-of-distribution (OOD) instances for conducting semi-supervised object
  detection (SSOD). The main challenge in OSSOD is distinguishing and filtering
  the OOD instances (i.e., outliers) during pseudo-labeling since OODs will
  affect the performance. The only OSSOD work employs an additional offline OOD
  detection network trained solely with labeled data to solve this problem.
  However, the limited labeled data restricts the potential for improvement.
  Meanwhile, the offline strategy results in low efficiency. To alleviate these
  issues, this paper proposes an end-to-end online OSSOD framework that improves
  performance and efficiency: 1) We propose a semi-supervised outlier filtering
  method that more effectively filters the OOD instances using both labeled and
  unlabeled data. 2) We propose a threshold-free Dual Competing OOD head that
  further improves the performance by suppressing the error accumulation during
  semi-supervised outlier filtering. 3) Our proposed method is an online
  end-to-end trainable OSSOD framework. Experimental results show that our method
  achieves state-of-the-art performance on several OSSOD benchmarks compared to
  existing methods. Moreover, additional experiments show that our method is more
  efficient and can be easily applied to different SSOD frameworks to boost their
  performance.
• MetaMixer: A Regularization Strategy for Online Knowledge Distillation
  Maorong Wang; Ling Xiao; Toshihiko Yamasaki, CoRR, abs/2303.07951, 14 Mar. 2023
  Online knowledge distillation (KD) has received increasing attention in
  recent years. However, while most existing online KD methods focus on
  developing complicated model structures and training strategies to improve the
  distillation of high-level knowledge like probability distribution, the effects
  of the multi-level knowledge in the online KD are greatly overlooked,
  especially the low-level knowledge. Thus, to provide a novel viewpoint to
  online KD, we propose MetaMixer, a regularization strategy that can strengthen
  the distillation by combining the low-level knowledge that impacts the
  localization capability of the networks, and high-level knowledge that focuses
  on the whole image. Experiments under different conditions show that MetaMixer
  can achieve significant performance gains over state-of-the-art methods.
• Semi-supervised Fashion Compatibility Prediction by Color Distortion Prediction
  Ling Xiao; Toshihiko Yamasaki, CoRR, abs/2212.14680, 27 Dec. 2022
  Supervised learning methods have been suffering from the fact that a
  large-scale labeled dataset is mandatory, which is difficult to obtain. This
  has been a more significant issue for fashion compatibility prediction because
  compatibility aims to capture people's perception of aesthetics, which are
  sparse and changing. Thus, the labeled dataset may become outdated quickly due
  to fast fashion. Moreover, labeling the dataset always needs some expert
  knowledge; at least they should have a good sense of aesthetics. However, there
  are limited self/semi-supervised learning techniques in this field. In this
  paper, we propose a general color distortion prediction task forcing the
  baseline to recognize low-level image information to learn more discriminative
  representation for fashion compatibility prediction. Specifically, we first
  propose to distort the image by adjusting the image color balance, contrast,
  sharpness, and brightness. Then, we propose adding Gaussian noise to the
  distorted image before passing them to the convolutional neural network (CNN)
  backbone to learn a probability distribution over all possible distortions. The
  proposed pretext task is adopted in the state-of-the-art methods in fashion
  compatibility and shows its effectiveness in improving these methods' ability
  in extracting better feature representations. Applying the proposed pretext
  task to the baseline can consistently outperform the original baseline.

• Exploring new areas of artificial intelligence
  Ling Xiao
  Forum for HUASHAN Scholars (IFHS2024), 27 Oct. 2024, Chinese, Public discourse
  25 Oct. 2024 - 28 Oct. 2024, [Invited]
• Large multimodal model powered recommendation systems
  Ling Xiao
  The 4th International Computational Imaging Conference (CITA 2024), 22 Sep. 2024, English, Invited oral presentation
  20 Sep. 2024 - 22 Sep. 2024, [Invited]
• The Past, Current, and Future of Fashion Retrieval: Toward a User-Centered Orientation
  Ling Xiao
  MVE, 13 Mar. 2024, English, Invited oral presentation
  13 Mar. 2024 - 15 Mar. 2024, [Invited]

• IEEE, ACM, IEICE, IEEE SPS, IEEE Robotics and Automation Society

• 困難な環境における作業を実現するヒューマノイドの統合的研究開発
  科学技術振興機構 ムーンショット 目標3
  Jul. 2026 - Nov. 2030
  原田 研介(PM, PI); 川角 祐一郎(PI); 渡辺 哲陽(PI); 梶田 秀司(PI); 小水内 俊介(PI); 築地原 里樹(PI); 長谷川 泰久(PI); 清川 拓哉(PI); 垣内 洋平(PI); 妹尾 拓(PI); Ling Xiao(PI); 松原 崇充(PI); 赤坂 亮太(PI)
  Principal investigator, JPMJMS263E
• 自律型ロボティック基盤モデルの構築
  豊田理研スカラ
  Apr. 2026 - Mar. 2028
  公益財団法人豊田理化学研究所
• スマート農業ロボット
  HU-UMA Joint Research Seed Fund
  Apr. 2026 - Mar. 2027
  Hokkaido University
• Multimodal Socially Compliant Open-World Navigation with Continual Reinforcement Learning
  NII戦略研究公募型研究
  Apr. 2026 - Mar. 2027
  国立情報学研究所(NII), Principal investigator
• Efficient and accurate scaling Graph Neural Networks for giant graphs
  JSPS Grants-in-Aid for Scientific Research
  Apr. 2024 - Mar. 2026
  Japan Society for the Promotion of Science, Grant-in-Aid for Early-Career Scientists, The University of Tokyo, 24K20787

• 静脈穿刺に用いる穿刺ターゲットの認識および位置決め方法
  Patent right, 肖玲,欧阳浩,叶霖,韩斌,陈学东,杨新
  特願202210202422.1, 03 Mar. 2022
• 視覚に基づく鋼材コイルの位置決め方法および装置
  Patent right, 胡友民,肖玲,吴波
  特願201811059328.5, 19 Mar. 2019
• 鋼材コイルのステレオビジョン位置決め方法および装置
  Patent right, 胡友民,肖玲,吴波
  特願201810094718.X, 31 Jan. 2018
• 光フロー画像に対する高速なFCM画像分割手法
  Patent right, 胡友民; 胡中旭; 吴波; 武敏健; 刘颉; 肖玲; 王诗杰; 李雪莲
  特願201710530461.3, 28 Jun. 2017
• 可視化されたクレーン吊り下げ位置決めシステム
  Patent right, 胡友民; 肖玲; 吴波; 刘颉
  特願201611246219.5, 29 Dec. 2016
• 溶接溶融池オンライン監視プラットフォーム用の多機能治具
  Utility model right, 胡友民; 唐松; 肖玲; 谷勇; 刘颉
  特願201620434683.6, 13 May 2016
• 溶接溶融池の動的プロセスをオンライン監視するシステムおよび方法
  Patent right, 胡友民; 刘颉; 肖玲; 唐松; 谷勇
  特願201610288460.8, 04 May 2016