2023年10月, Bronze Prize GCCE2023 Excellent Student Poster Award　　　　　　　　　　　　　　　
Tatsuki Seino, Naoki Saito, Takahiro Ogawa, Satoshi Asamizu, Miki Haseyama

2023年10月, Silver Prize GCCE2023 Excellent Paper Award　　　　　　　　　　　　　　　
Haruka Matsuda, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama

2023年07月, Best Paper Award Honorable Mention　　　　　　　　　　　　　　　
Ryota Goka, Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama

2023年01月, The 2022 IEEE Sapporo Section Encouragement Award　　　　　　　　　　　　　　　
Nao Nakagawa;Ren Togo;Takahiro Ogawa;Miki Haseyama

2023年01月, The 2022 IEEE Sapporo Section Encouragement Award　　　　　　　　　　　　　　　
Naoki Ogawa;Keisuke Maeda;Takahiro Ogawa;Miki Haseyama

2023年01月, The 2022 IEEE Sapporo Section Encouragement Award　　　　　　　　　　　　　　　
Kyohei Kamikawa;Keisuke Maeda;Takahiro Ogawa;Miki Haseyama

2023年01月, Best Paper Award　　　　　　　　　　　　　　　
Teruhisa Yamashiro;Yuki Honma;Ren Togo;Takahiro Ogawa;Miki Haseyama;International;Workshop on;Advanced Image Technology;IWAIT

2022年12月, 優秀研究発表賞　　　　　　　　　　　　　　　
ユーザの嗜好を考慮した強化学習と知識グラフに基づく楽曲プレイリスト生成に関する検討, 映像情報メディア学会技術報告, vol.46, no.6, pp.109-112
櫻井 慶悟, 藤後 廉, 小川 貴弘, 長谷山 美紀

2022年12月, 若手優秀論文発表賞　　　　　　　　　　　　　　　
五箇 亮太;諸戸 祐哉;前田 圭介;小川 貴弘;長谷山 美紀

2022年12月, 若手優秀論文発表賞　　　　　　　　　　　　　　　
七田 亮, 藤後 廉, 前田 圭介, 小川 貴弘, 長谷山 美紀

2022年12月, 若手優秀論文発表賞　　　　　　　　　　　　　　　
山本一輝;前田 圭介;藤後 廉;小川 貴弘;長谷山 美紀

2022年11月, Bronze Prize GCCE2022 Excellent Student Paper Award　　　　　　　　　　　　　　　
Cross-platform recommendation considering common users’ preferences based on preference propagation graphnet, 2022 IEEE 11th Global Conference on Consumer Electronics (GCCE 2022), pp.133-134
Kazuki Yamamoto;Keisuke Maeda;Ren Togo;Takahiro Ogawa;Miki Haseyama

2022年11月, 第1回 北海道大学医療AIシンポジウム 優秀研究賞　　　　　　　　　　　　　　　
李 広;藤後 廉;小川 貴弘;長谷山 美紀

2022年10月, Silver Prize GCCE2022 Excellent Poster Award　　　　　　　　　　　　　　　
Free-viewpoint sports video generation based on dynamic NeRF considering time series, 2022 IEEE 11th Global Conference on Consumer Electronics (GCCE 2022), pp.419-420
Masato Kawai;Rintaro Yanagi;Togo Ren;Takahiro Ogawa;Miki Haseyama

2022年10月, Silver Prize GCCE2022 Excellent Student Poster Award　　　　　　　　　　　　　　　
Content-based image retrieval using effective synthesized images from different camera views via pixelNeRF, 2022 IEEE 11th Global Conference on Consumer Electronics (GCCE 2022), pp.415-416
Yuki Era;Ren Togo;Keisuke Maeda;Takahiro Ogawa;Miki Haseyama

2022年09月, 土木学会 土木情報学システム開発賞　　　　　　　　　　　　　　　

2022年08月, MIRU 2022 学生奨励賞2件　　　　　　　　　　　　　　　

2022年03月, IEEE LifeTech 2022 WIE Excellent Poster Award　　　　　　　　　　　　　　　

2022年02月, 2021 IEEE Sapporo Section Student Paper Contest, Best Presentation Award　　　　　　　　　　　　　　　

2022年02月, 2021 IEEE Sapporo Section Encouragement Award 2件　　　　　　　　　　　　　　　

2022年01月, International Workshop on Advanced Image Technology (IWAIT2022) Best Paper Award　　　　　　　　　　　　　　　

2021年12月, 映像情報メディア学会 優秀研究発表賞　　　　　　　　　　　　　　　

2021年12月, 令和3年度電気・情報関係学会北海道支部連合大会 若手優秀論文発表賞 3件　　　　　　　　　　　　　　　

2021年10月, The 1st Hokkaido Young Professionals Workshop Best Student Presentation Award　　　　　　　　　　　　　　　

2021年10月, 2021 IEEE 10th Global Conference on Consumer Electronics, Gold Prize GCCE2021 Excellent Poster Award　　　　　　　　　　　　　　　

2021年10月, 2021 IEEE 10th Global Conference on Consumer Electronics, Gold Prize GCCE2021 Excellent Student Poster Award　　　　　　　　　　　　　　　

2021年10月, 2021 IEEE 10th Global Conference on Consumer Electronics, Silver Prize GCCE2021 Excellent Student Poster Award　　　　　　　　　　　　　　　

2021年10月, 2021 IEEE 10th Global Conference on Consumer Electronics, GCCE2021 Outstanding Paper Award　　　　　　　　　　　　　　　

2021年06月, 映像情報メディア学会丹羽高柳賞論文賞　　　　　　　　　　　　　　　

2021年03月, ACM Multimedia Asia 2020, Best Paper Runner-up Award　　　　　　　　　　　　　　　

2021年03月, 2021 IEEE 3rd Global Conference on Life Sciences and Technologies, Excellent Poster (On-site) Award Winners: Bronze Prize　　　　　　　　　　　　　　　

2021年03月, IEEE LifeTech 2021 Excellent Student Paper Award for Oral Presentation, 2nd Prize　　　　　　　　　　　　　　　

2021年02月, 2020 IEEE Sapporo Section Student Paper Awards, Encouragement Paper Award　　　　　　　　　　　　　　　

2021年02月, 2020 IEEE Sapporo Section Student Paper Awards, Best Paper Award　　　　　　　　　　　　　　　

2020年11月, 令和2年度電気・情報関係学会北海道支部連合大会 若手優秀論文発表賞 3件　　　　　　　　　　　　　　　

2020年10月, 2020 IEEE 9th Global Conference on Consumer Electronics, Bronze Prize GCCE2020 Excellent Paper Award　　　　　　　　　　　　　　　

2020年10月, 2020 IEEE 9th Global Conference on Consumer Electronics, Silver Prize IEEE GCCE2020 Excellent Paper Award　　　　　　　　　　　　　　　

2020年10月, 2020 IEEE 9th Global Conference on Consumer Electronics, Gold Prize IEEE GCCE2020 Excellent Demo! Award　　　　　　　　　　　　　　　

2020年10月, 2020 IEEE 9th Global Conference on Consumer Electronics, Gold Prize GCCE2020 Excellent Poster Award　　　　　　　　　　　　　　　

2020年10月, 2020 IEEE 9th Global Conference on Consumer Electronics, Gold Prize IEEE GCCE2020 Excellent Student Paper Award　　　　　　　　　　　　　　　

2020年06月, 映像情報メディア学会丹羽高柳賞論文賞　　　　　　　　　　　　　　　

2020年05月, 2020 ICCE-TW Best Paper Award Honorable Metion　　　　　　　　　　　　　　　

2020年02月, The 2019 IEEE Sapporo Section Student Paper Contest Encouraging Prize 3件　　　　　　　　　　　　　　　

2020年02月, The 2019 IEEE Sapporo Section Encouragement Award　　　　　　　　　　　　　　　

2019年12月, 映像情報メディア学会 優秀研究発表賞　　　　　　　　　　　　　　　

2019年12月, 令和元年度電気・情報関係学会北海道支部連合大会 若手優秀論文発表賞 2件　　　　　　　　　　　　　　　

2019年10月, 2019 IEEE 8th Global Conference on Consumer Electronics, Silver Prize IEEE GCCE 2019 Excelent Paper Award　　　　　　　　　　　　　　　

2019年10月, 2019 IEEE 8th Global Conference on Consumer Electronics, Silver Prize IEEE GCCE 2019 Excelent Poster Award　　　　　　　　　　　　　　　

2019年10月, 2019 IEEE 8th Global Conference on Consumer Electronics, Outstanding Prize IEEE GCCE 2019 Excelent Demo! Award　　　　　　　　　　　　　　　

2019年03月, 2019 IEEE 1st Global Conference on Life Sciences and Technologies, 2nd Prize IEEE Lifetech 2019 Excellent Paper Award　　　　　　　　　　　　　　　

2019年02月, The 2018 IEEE Sapporo Section Student Paper Contest Encouraging Prize　　　　　　　　　　　　　　　

2019年02月, The 2018 IEEE Sapporo Section Encouragement Award 2件　　　　　　　　　　　　　　　

2019年01月, The 2019 joint International Workshop on Advanced Image Technology & International Forum on Medical Imaging in Asia IWAIT Best Paper Award　　　　　　　　　　　　　　　

2018年12月, 平成30年度電気・情報関係学会北海道支部連合大会 優秀論文発表賞　　　　　　　　　　　　　　　

2018年12月, 映像情報メディア学会 優秀研究発表賞　　　　　　　　　　　　　　　

2018年10月, 2018 IEEE 7th Global Conference on Consumer Electronics, IEEE GCCE 2018 Outstanding Paper Award　　　　　　　　　　　　　　　

2018年10月, 2018 IEEE 7th Global Conference on Consumer Electronics, 1st Prize IEEE GCCE 2018 Excellent Poster Award　　　　　　　　　　　　　　　

2018年, 平成29年度電気・情報関係学会北海道支部連合大会 優秀論文発表賞　　　　　　　　　　　　　　　

2018年, The 2017 IEEE Sapporo Section Student Paper Contest Encouraging Prize　　　　　　　　　　　　　　　

2018年, The 2017 IEEE Sapporo Section Encouragement Award (2件)　　　　　　　　　　　　　　　

2017年, International Workshop on Advanced Image Technology (IWAIT2017) Best Paper Award　　　　　　　　　　　　　　　

2017年, 平成28年度電気・情報関係学会北海道支部連合大会 優秀論文発表賞　　　　　　　　　　　　　　　

2017年, The 2016 IEEE Sapporo Section Student Paper Contest Encouraging Prize　　　　　　　　　　　　　　　

2017年, The 2016 IEEE Sapporo Section Encouragement Award　　　　　　　　　　　　　　　

2017年, 電子情報通信学会 学術奨励賞　　　　　　　　　　　　　　　

2017年, 精密工学会画像応用技術専門委員会・映像情報メディア学会メディア工学研究委員会合同サマーセミナー 優秀発表賞　　　　　　　　　　　　　　　

2017年, 2017 IEEE 6th Global Conference on Consumer Electronics, IEEE GCCE 2017 Outstanding Poster Award　　　　　　　　　　　　　　　

2016年, 2016 IEEE 5th Global Conference on Consumer Electronics 1st Prize IEEE GCCE 2016 Excellent Poster Award　　　　　　　　　　　　　　　

2016年, 平成27年度電気・情報関係学会北海道支部連合大会 優秀論文発表賞 (2件)　　　　　　　　　　　　　　　

2016年, The 2015 IEEE Sapporo Section Student Paper Contest Encouraging Prize　　　　　　　　　　　　　　　

2016年, The 2015 IEEE Sapporo Section Encouragement Award (2件)　　　　　　　　　　　　　　　

2015年05月, 映像情報メディア学会, 丹羽高柳賞　　　　　　　　　　　　　　　
業績賞
長谷山 美紀

2015年, International Workshop on Advanced Image Technology (IWAIT2015) Best Paper Award　　　　　　　　　　　　　　　

2015年, IEEE GCCE 2015 Excellent Poster Award　　　　　　　　　　　　　　　

2015年, IEEE GCCE 2015 Outstanding Poster Award　　　　　　　　　　　　　　　

2015年, The 2014 IEEE Sapporo Section Student Paper Contest Best Presentation Award　　　　　　　　　　　　　　　

2015年, 平成27年度 映像情報メディア学会 優秀研究発表賞　　　　　　　　　　　　　　　

2014年06月, 総務省北海道総合通信局, 平成26年度情報通信月間 北海道総合通信局長表彰　　　　　　　　　　　　　　　
長谷山 美紀

2014年, IEEE GCCE 2014 Undergraduate Poster Award　　　　　　　　　　　　　　　

2013年, 平成25年度電気・情報関係学会北海道支部連合大会 優秀論文発表賞　　　　　　　　　　　　　　　

2011年, 平成23年度信号処理学生奨励賞 (2件)　　　　　　　　　　　　　　　

2011年, 平成23年度電気関係学会北海道支部連合大会 若手優秀論文発表賞　　　　　　　　　　　　　　　

2011年, 映像情報メディア学会 学生優秀発表賞　　　　　　　　　　　　　　　

2011年, SIP学生奨励賞　　　　　　　　　　　　　　　
日本国

2011年, 平成23年度電気・情報関係学会北海道支部 優秀論文発表賞　　　　　　　　　　　　　　　
日本国

2011年, 映像情報メディア学会年次大会 学生優秀発表賞　　　　　　　　　　　　　　　
日本国

2010年, 平成22年度電気関係学会北海道支部連合大会 若手優秀論文発表賞　　　　　　　　　　　　　　　

2010年, 2010 IEEE Sapporo Section Student Member Best Presentation Award　　　　　　　　　　　　　　　

2009年, 電子情報通信学会論文賞　　　　　　　　　　　　　　　

2009年, 平成21年度電気・情報関係学会北海道支部連合大会 優秀論文発表賞　　　　　　　　　　　　　　　
日本国

2009年, 平成20年度電子情報通信学会論文賞　　　　　　　　　　　　　　　
日本国

2008年, IEEE CE-Society 日本支部 若手論文賞　　　　　　　　　　　　　　　

2008年, 平成20年度電気関係学会北海道支部連合大会 若手優秀論文発表賞　　　　　　　　　　　　　　　

2008年, 2008 IEEE Sapporo Section Student Member Encouraging Prize　　　　　　　　　　　　　　　

2007年, 平成19年度電気関係学会北海道支部連合大会 若手優秀論文発表賞　　　　　　　　　　　　　　　

2007年, IEEE International Conference on Consumer Electronics, IEEE Consumer Electronics Society Japan Chapter Young Scientist Paper Award　　　　　　　　　　　　　　　

2006年, 2006 IEEE Sapporo Section Student Paper Contest Award　　　　　　　　　　　　　　　

2005年, 精密工学会画像応用技術専門委員会・映像情報メディア学会メディア工学研究委員会合同サマーセミナー優秀発表賞　　　　　　　　　　　　　　　

2005年, 平成17年度電気情報関係学会北海道支部連合大会 若手優秀論文発表賞　　　　　　　　　　　　　　　

2005年, 映像情報メディア学会 研究奨励賞　　　　　　　　　　　　　　　

The 2022 IEEE Sapporo Section Student Paper Contest Encouraging Prize　　　　　　　　　　　　　　　
河合 雅斗;柳 凜太郎;藤後 廉;小川 貴弘;長谷山 美紀

Manta: Enhancing Mamba for Few-Shot Action Recognition of Long Sub-Sequence
Wenbo Huang, Jinghui Zhang, Guang Li, Lei Zhang, Shuoyuan Wang, Fang Dong, Jiahui Jin, Takahiro Ogawa, Miki Haseyama
Proceedings of the AAAI Conference on Artificial Intelligence, 39, 4, 3751, 3759, Association for the Advancement of Artificial Intelligence (AAAI), 2025年04月11日
研究論文（学術雑誌）, In few-shot action recognition (FSAR), long sub-sequences of video naturally express entire actions more effectively. However, the high computational complexity of mainstream Transformer-based methods limits their application. Recent Mamba demonstrates efficiency in modeling long sequences, but directly applying Mamba to FSAR overlooks the importance of local feature modeling and alignment. Moreover, long sub-sequences within the same class accumulate intra-class variance, which adversely impacts FSAR performance. To solve these challenges, we propose a Matryoshka MAmba and CoNtrasTive LeArning framework (Manta). Firstly, the Matryoshka Mamba introduces multiple Inner Modules to enhance local feature representation, rather than directly modeling global features. An Outer Module captures dependencies of timeline between these local features for implicit temporal alignment. Secondly, a hybrid contrastive learning paradigm, combining both supervised and unsupervised methods, is designed to mitigate the negative effects of intra-class variance accumulation. The Matryoshka Mamba and the hybrid contrastive learning paradigm operate in two parallel branches within Manta, enhancing Mamba for FSAR of long sub-sequence. Manta achieves new state-of-the-art performance on prominent benchmarks, including SSv2, Kinetics, UCF101, and HMDB51. Extensive empirical studies prove that Manta significantly improves FSAR of long sub-sequence from multiple perspectives.

Generative Dataset Distillation Based on Self-knowledge Distillation
Longzhen Li, Guang Li, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1, 5, IEEE, 2025年04月06日
研究論文（国際会議プロシーディングス）

Continual Self-supervised Learning Considering Medical Domain Knowledge in Chest CT Images
Ren Tasai, Guang Li, Ren Togo, Minghui Tang, Takaaki Yoshimura, Hiroyuki Sugimori, Kenji Hirata, Takahiro Ogawa, Kohsuke Kudo, Miki Haseyama
ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1, 5, IEEE, 2025年04月06日
研究論文（国際会議プロシーディングス）

Multistage deep learning for classification of Helicobacter pylori infection status using endoscopic images
Guang Li, Ren Togo, Katsuhiro Mabe, Shunpei Nishida, Yoshihiro Tomoda, Fumiyuki Shiratani, Masashi Hirota, Takahiro Ogawa, Miki Haseyama
Journal of Gastroenterology, 60, 4, 408, 415, Springer Science and Business Media LLC, 2025年01月15日
研究論文（学術雑誌）

Lung Disease Classification with Limited Training Data Based on Weight Selection Technique
Ayaka Tsutsumi, Guang Li, Ren Togo, Takahiro Ogawa, Satoshi Kondo, Miki Haseyama
2024 IEEE 13th Global Conference on Consumer Electronics (GCCE), 460, 461, IEEE, 2024年10月29日
研究論文（国際会議プロシーディングス）

Generative Dataset Distillation Based on Large Model Pool
Longzhen Li, Guang Li, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
2024 IEEE 13th Global Conference on Consumer Electronics (GCCE), 458, 459, IEEE, 2024年10月29日
研究論文（国際会議プロシーディングス）

Lung Cancer Classification Using Masked Autoencoder Pretrained on J-MID Database
Ren Tasai, Guang Li, Ren Togo, Minghui Tang, Takaaki Yoshimura, Hiroyuki Sugimori, Kenji Hirata, Takahiro Ogawa, Kohsuke Kudo, Miki Haseyama
2024 IEEE 13th Global Conference on Consumer Electronics (GCCE), 456, 457, IEEE, 2024年10月29日
研究論文（国際会議プロシーディングス）

Graph Convolutional Network-based Sports Skill-level Recognition via Deep Metric Learning
Tatsuki Seino, Naoki Saito, Takahiro Ogawa, Satoshi Asamizu, Miki Haseyama
2024 International Technical Conference on Circuits/Systems, Computers, and Communications (ITC-CSCC), 15, 1, 4, IEEE, 2024年07月02日
研究論文（国際会議プロシーディングス）

Multimodal Transformer Model Using Time-Series Data to Classify Winter Road Surface Conditions.
Yuya Moroto, Keisuke Maeda, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
Sensors, 24, 11, 3440, 3440, 2024年06月
研究論文（学術雑誌）

Trial Analysis of Brain Activity Information for the Presymptomatic Disease Detection of Rheumatoid Arthritis.
Keisuke Maeda, Takahiro Ogawa, Tasuku Kayama, Takuya Sasaki, Kazuki Tainaka, Masaaki Murakami, Miki Haseyama
Bioengineering (Basel, Switzerland), 11, 6, 2024年05月21日, ［国際誌］
英語, 研究論文（学術雑誌）, This study presents a trial analysis that uses brain activity information obtained from mice to detect rheumatoid arthritis (RA) in its presymptomatic stages. Specifically, we confirmed that F759 mice, serving as a mouse model of RA that is dependent on the inflammatory cytokine IL-6, and healthy wild-type mice can be classified on the basis of brain activity information. We clarified which brain regions are useful for the presymptomatic detection of RA. We introduced a matrix completion-based approach to handle missing brain activity information to perform the aforementioned analysis. In addition, we implemented a canonical correlation-based method capable of analyzing the relationship between various types of brain activity information. This method allowed us to accurately classify F759 and wild-type mice, thereby identifying essential features, including crucial brain regions, for the presymptomatic detection of RA. Our experiment obtained brain activity information from 15 F759 and 10 wild-type mice and analyzed the acquired data. By employing four types of classifiers, our experimental results show that the thalamus and periaqueductal gray are effective for the classification task. Furthermore, we confirmed that classification performance was maximized when seven brain regions were used, excluding the electromyogram and nucleus accumbens.

Expert–Novice Level Classification Using Graph Convolutional Network Introducing Confidence-Aware Node-Level Attention Mechanism
Tatsuki Seino, Naoki Saito, Takahiro Ogawa, Satoshi Asamizu, Miki Haseyama
Sensors, 24, 10, 3033, 3033, MDPI AG, 2024年05月10日
研究論文（学術雑誌）, In this study, we propose a classification method of expert–novice levels using a graph convolutional network (GCN) with a confidence-aware node-level attention mechanism. In classification using an attention mechanism, highlighted features may not be significant for accurate classification, thereby degrading classification performance. To address this issue, the proposed method introduces a confidence-aware node-level attention mechanism into a spatiotemporal attention GCN (STA-GCN) for the classification of expert–novice levels. Consequently, our method can contrast the attention value of each node on the basis of the confidence measure of the classification, which solves the problem of classification approaches using attention mechanisms and realizes accurate classification. Furthermore, because the expert–novice levels have ordinalities, using a classification model that considers ordinalities improves the classification performance. The proposed method involves a model that minimizes a loss function that considers the ordinalities of classes to be classified. By implementing the above approaches, the expert–novice level classification performance is improved.

Analysis of Continual Learning Techniques for Image Generative Models with Learned Class Information Management.
Taro Togo, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
Sensors, 24, 10, 3087, 3087, 2024年05月
研究論文（学術雑誌）

A Novel Frame-Selection Metric for Video Inpainting to Enhance Urban Feature Extraction.
Yuhu Feng, Jiahuan Zhang, Guang Li 0008, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
Sensors, 24, 10, 3035, 3035, 2024年05月
研究論文（学術雑誌）

Algal Bed Region Segmentation Based on a ViT Adapter Using Aerial Images for Estimating CO2 Absorption Capacity.
Guang Li 0008, Ren Togo, Keisuke Maeda, Akinori Sako, Isao Yamauchi, Tetsuya Hayakawa, Shigeyuki Nakamae, Takahiro Ogawa 0001, Miki Haseyama
Remote. Sens., 16, 10, 1742, 1742, 2024年05月
研究論文（学術雑誌）

Confidence-Aware Spatial-Temporal Attention Graph Convolutional Network for Skeleton-Based Expert-Novice Level Classification
Tatsuki Seino, Naoki Saito, Takahiro Ogawa, Satoshi Asamizu, Miki Haseyama
ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2024年04月14日
研究論文（国際会議プロシーディングス）

Parameter-efficient tuning of cross-modal retrieval for a specific database via trainable textual and visual prompts.
Huaying Zhang, Rintaro Yanagi, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
Int. J. Multim. Inf. Retr., 13, 1, 14, 14, 2024年03月
研究論文（学術雑誌）

Flexibly manipulating popularity bias for tackling trade-offs in recommendation.
Hiroki Okamura, Keisuke Maeda, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
Inf. Process. Manag., 61, 2, 103606, 103606, 2024年03月
研究論文（学術雑誌）

Text-Guided Image Editing Based on Post Score for Gaining Attention on Social Media.
Yuto Watanabe, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
Sensors, 24, 3, 921, 921, 2024年02月
研究論文（学術雑誌）

MLLM-based Automatic Exploration of Editing Prompt for High Engagement Image Generation.
Kenta Kubota, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 1165, 1166, 2024年
研究論文（国際会議プロシーディングス）

An Evaluation Metric for Single Image-to-3D Models Based on a Class Confidence Score of Object Detection Models.
Yuiko Uchida, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 1163, 1164, 2024年
研究論文（国際会議プロシーディングス）

Improving Zero-shot Adversarial Robustness via Integrating Image Features of Foundation Models.
Koshiro Toishi, Keisuke Maeda, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 148, 149, 2024年
研究論文（国際会議プロシーディングス）

Emotion-conditional Image Generation Reflecting Semantic Alignment with Text-to-Image Models.
Kaede Hayakawa, Keisuke Maeda, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 10, 11, 2024年
研究論文（国際会議プロシーディングス）

Structured Polyphonic Music Generation with Diffusion Transformer.
Jinlong Zhu, Keigo Sakurai, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 8, 9, 2024年
研究論文（国際会議プロシーディングス）

ブルーカーボンによるCO₂吸収量推計へ向けた大規模セマンティックセグメンテーションモデルに基づく藻場領域の自動認識
李 広, 藤後 廉, 前田 圭介, 酒向 章哲, 山内 功, 早川 哲也, 中前 茂之, 小川 貴弘, 長谷山 美紀
土木学会論文集, 80, 17, n/a, 公益社団法人 土木学会, 2024年
日本語, 沿岸藻場のCO₂吸収量の測定は，カーボンニュートラル実現に向けた重要課題の一つであるが，UAV画像からの藻場領域の特定には多大な労力と経験を必要とする．本研究では，UAV画像を用いた藻場領域の自動認識手法を提案する．本手法では，画素レベルの意味的領域分割を可能とするモデルを利用し，最新モデルの一つであるViT-Adapterを採用する．本技術の利点は，学習済み大規模モデルの知識を有効活用して藻場領域認識を行う点であり，モデルのパラメータ調整により画素レベルでの海藻の識別を実現する．本研究では，航空写真から目視により藻場領域を特定したマスク画像を用いた学習を行い，さらに，UAV画像へ適応するためにデータ拡張等の処理を検討した．北海道えりも沿岸のUAV画像を用いた実証を行い，有効性を確認した．

Generative Dataset Distillation: Balancing Global Structure and Local Details.
Longzhen Li, Guang Li 0008, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
CVPR Workshops, 7664, 7671, 2024年
研究論文（国際会議プロシーディングス）

Cross-domain Few-shot In-context Learning for Enhancing Traffic Sign Recognition.
Yaozong Gan, Guang Li 0008, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2407.05814, 2024年
研究論文（学術雑誌）

Zero-shot Composed Image Retrieval Considering Query-target Relationship Leveraging Masked Image-text Pairs.
Huaying Zhang, Rintaro Yanagi, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2406.18836, 2024年
研究論文（学術雑誌）

Prompt-based Personalized Federated Learning for Medical Visual Question Answering.
He Zhu, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2402.09677, 2024年
研究論文（学術雑誌）

Importance-Aware Adaptive Dataset Distillation.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2401.15863, 2024年
研究論文（学術雑誌）

Multi-Object Editing in Personalized Text-To-Image Diffusion Model Via Segmentation Guidance.
Haruka Matsuda, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICASSP, 8140, 8144, 2024年
研究論文（国際会議プロシーディングス）

Caption Unification for Multi-View Lifelogging Images Based on In-Context Learning with Heterogeneous Semantic Contents.
Masaya Sato, Keisuke Maeda, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ICASSP, 8085, 8089, 2024年
研究論文（国際会議プロシーディングス）

Enhancing Noisy Label Learning Via Unsupervised Contrastive Loss with Label Correction Based on Prior Knowledge.
Masaki Kashiwagi, Keisuke Maeda, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ICASSP, 6235, 6239, 2024年
研究論文（国際会議プロシーディングス）

Prompt-Based Personalized Federated Learning for Medical Visual Question Answering.
He Zhu, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ICASSP, 1821, 1825, 2024年
研究論文（国際会議プロシーディングス）

Importance-aware adaptive dataset distillation.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
Neural Networks, 172, 106154, 106154, 2024年
研究論文（学術雑誌）

Dataset Distillation Using Parameter Pruning.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
IEICE Trans. Fundam. Electron. Commun. Comput. Sci., 107, 6, 936, 940, 2024年
研究論文（学術雑誌）

地下鉄トンネル点検における技術者のモーションデータを用いた熟練度分類 -説明可能なGraph Convolutional Networkの導入-
清野 竜生, 斉藤 直輝, 前田 圭介, 小川 貴弘, 長谷山 美紀
AI・データサイエンス論文集, 5, 1, 101, 109, 公益社団法人 土木学会, 2024年
日本語, インフラ設備の正確かつ効率的な維持管理業務を遂行するためには，熟練技術者が有する優れた点検技術(熟練技術)を若手技術者へ継承する必要がある．多くの時間と労力を要する熟練技術の継承を支援するためには，点検に重要な動作を明確にする必要がある．しかしながら，従来の研究では，視線・動作などの生体情報を用いた熟練度分類と点検技術の熟練度と生体情報間の関連性分析に留まっている．そこで本論文では，若手技術者への技術継承支援に向け，熟練度分類および分類に重要な要素を可視化する手法を提案する．提案手法では，点検中の技術者から得られたモーションデータを用いて，熟練度分類の確信度を考慮することで重要な要素を強調するAttention機構を導入したグラフ畳み込みネットワークにより，説明可能な熟練度分類を実現する．

工事現場映像における深度情報を活用した重機接触事故リスクの推定
五箇 亮太, 前田 圭介, 藤後 廉, 小川 貴弘, 長谷山 美紀
人工知能学会全国大会論文集, JSAI2024, 2C6GS701, 2C6GS701, 一般社団法人 人工知能学会, 2024年
日本語, 建設業において，労働災害リスクの低減や安全性の向上は急務であり，近年では工事現場の安全性向上を目的とした重機との接触事故リスクを推定する手法が提案されている．既存研究では，工事現場で撮影された映像中より検出された作業員や重機の関係から接触事故リスクを推定しているものの，検出物体間の奥行きを考慮していなかった．これに対して本稿では，映像中の空間情報を補完するために深度情報を活用した接触事故リスク推定を可能とする深層学習モデルを提案する．実験の結果，映像中の作業員・重機間の空間的関係を考慮した接触事故リスクの推定において，提案手法は既存手法よりも高い性能を達成することが示された．

Reinforcing Pre-trained Models Using Counterfactual Images.
Xiang Li, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2406.13316, 2024年
研究論文（学術雑誌）

Generative Dataset Distillation: Balancing Global Structure and Local Details.
Longzhen Li, Guang Li 0008, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2404.17732, 2024年
研究論文（学術雑誌）

Enhancing Generative Class Incremental Learning Performance with Model Forgetting Approach.
Taro Togo, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2403.18258, 2024年
研究論文（学術雑誌）

Automatic Findings Generation for Distress Images Using In-Context Few-Shot Learning of Visual Language Model Based on Image Similarity and Text Diversity.
Yuto Watanabe, Naoki Ogawa, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
J. Robotics Mechatronics, 36, 2, 353, 364, 2024年
研究論文（学術雑誌）

Individual Persistence Adaptation for User-Centric Evaluation of User Satisfaction in Recommender Systems.
Nozomu Onodera, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
IEEE Access, 12, 23626, 23635, 2024年
研究論文（学術雑誌）

Zero-Shot Traffic Sign Recognition Based on Midlevel Feature Matching
Yaozong Gan, Guang Li, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
Sensors, 23, 23, 9607, 9607, MDPI AG, 2023年12月04日
研究論文（学術雑誌）, Traffic sign recognition is a complex and challenging yet popular problem that can assist drivers on the road and reduce traffic accidents. Most existing methods for traffic sign recognition use convolutional neural networks (CNNs) and can achieve high recognition accuracy. However, these methods first require a large number of carefully crafted traffic sign datasets for the training process. Moreover, since traffic signs differ in each country and there is a variety of traffic signs, these methods need to be fine-tuned when recognizing new traffic sign categories. To address these issues, we propose a traffic sign matching method for zero-shot recognition. Our proposed method can perform traffic sign recognition without training data by directly matching the similarity of target and template traffic sign images. Our method uses the midlevel features of CNNs to obtain robust feature representations of traffic signs without additional training or fine-tuning. We discovered that midlevel features improve the accuracy of zero-shot traffic sign recognition. The proposed method achieves promising recognition results on the German Traffic Sign Recognition Benchmark open dataset and a real-world dataset taken from Sapporo City, Japan.

Manipulation Direction: Evaluating Text-Guided Image Manipulation Based on Similarity between Changes in Image and Text Modalities.
Yuto Watanabe, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
Sensors, 23, 22, 9287, 9287, 2023年11月
研究論文（学術雑誌）

Skill Level Classification Using Motion Data via Spatial Temporal Graph Convolutional Network
Tatsuki Seino, Naoki Saito, Takahiro Ogawa, Satoshi Asamizu, Miki Haseyama
2023 IEEE 12th Global Conference on Consumer Electronics (GCCE), IEEE, 2023年10月10日
研究論文（国際会議プロシーディングス）

Visual Emotion Recognition Through Multimodal Cyclic-Label Dequantized Gaussian Process Latent Variable Model.
Naoki Saito 0006, Keisuke Maeda, Takahiro Ogawa 0001, Satoshi Asamizu, Miki Haseyama
Journal of Robotics and Mechatronics, 35, 5, 1321, 1330, 2023年10月
研究論文（学術雑誌）

Self-supervised learning for gastritis detection with gastric X-ray images.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
International Journal of Computer Assisted Radiology and Surgery, 18, 10, 1841, 1848, 2023年10月
研究論文（学術雑誌）

Zero-Shot Neural Decoding with Semi-Supervised Multi-View Embedding.
Yusuke Akamatsu, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
Sensors, 23, 15, 6903, 6903, 2023年08月
研究論文（学術雑誌）

Off-Screen Sound Separation Based on Audio-visual Pre-training Using Binaural Audio
Masaki Yoshida, Ren Togo, Takahiro Ogawa, Miki Haseyama
SENSORS, 23, 9, 4540, 4540, MDPI, 2023年05月
英語, 研究論文（学術雑誌）, This study proposes a novel off-screen sound separation method based on audio-visual pre-training. In the field of audio-visual analysis, researchers have leveraged visual information for audio manipulation tasks, such as sound source separation. Although such audio manipulation tasks are based on correspondences between audio and video, these correspondences are not always established. Specifically, sounds coming from outside a screen have no audio-visual correspondences and thus interfere with conventional audio-visual learning. The proposed method separates such off-screen sounds based on their arrival directions using binaural audio, which provides us with three-dimensional sensation. Furthermore, we propose a new pre-training method that can consider the off-screen space and use the obtained representation to improve off-screen sound separation. Consequently, the proposed method can separate off-screen sounds irrespective of the direction from which they arrive. We conducted our evaluation using generated video data to circumvent the problem of difficulty in collecting ground truth for off-screen sounds. We confirmed the effectiveness of our methods through off-screen sound detection and separation tasks.

Multimodal Natural Language Explanation Generation for Visual Question Answering Based on Multiple Reference Data
He Zhu, Ren Togo, Takahiro Ogawa, Miki Haseyama
ELECTRONICS, 12, 10, MDPI, 2023年05月
英語, 研究論文（学術雑誌）, As deep learning research continues to advance, interpretability is becoming as important as model performance. Conducting interpretability studies to understand the decision-making processes of deep learning models can improve performance and provide valuable insights for humans. The interpretability of visual question answering (VQA), a crucial task for human-computer interaction, has garnered the attention of researchers due to its wide range of applications. The generation of natural language explanations for VQA that humans can better understand has gradually supplanted heatmap representations as the mainstream focus in the field. Humans typically answer questions by first identifying the primary objects in an image and then referring to various information sources, both within and beyond the image, including prior knowledge. However, previous studies have only considered input images, resulting in insufficient information that can lead to incorrect answers and implausible explanations. To address this issue, we introduce multiple references in addition to the input image. Specifically, we propose a multimodal model that generates natural language explanations for VQA. We introduce outside knowledge using the input image and question and incorporate object information into the model through an object detection module. By increasing the information available during the model generation process, we significantly improve VQA accuracy and the reliability of the generated explanations. Moreover, we employ a simple and effective feature fusion joint vector to combine information from multiple modalities while maximizing information preservation. Qualitative and quantitative evaluation experiments demonstrate that the proposed method can generate more reliable explanations than state-of-the-art methods while maintaining answering accuracy.

Boosting automatic COVID-19 detection performance with self-supervised learning and batch knowledge ensembling.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
Comput. Biol. Medicine, 158, 106877, 106877, 2023年05月
研究論文（学術雑誌）

画像の感情推定のためのラベル逆量子化を導入した正準相関分析
斉藤 直輝, 前田 圭介, 小川 貴弘, 浅水 仁, 長谷山 美紀
電子情報通信学会論文誌D 情報・システム, J106-D, 5, 337, 348, The Institute of Electronics, Information and Communication Engineers, 2023年05月01日
日本語, 本論文では，画像の感情推定のためのラベル逆量子化を導入した正準相関分析であるSupervised Multi-view Canonical Correlation Analysis via Cyclic Label Dequantization (sMVCCA-CLD)を提案する．ラベルから算出される特徴量(ラベル特徴量)の次元数は他の特徴量と比較して小さいため，従来のCCAでは，構築する空間の次元数の低下により特徴量間の相関関係の表現が困難となる問題点が存在する．そこで，sMVCCA-CLDでは，ラベル特徴量の次元数をラベル逆量子化により増加させながら，特徴量間の相関を最大化することで，次元数の制約を受けない共通潜在空間の構築を可能とする．更に，感情が円環状に配置されることを考慮してラベル逆量子化を行うことで，感情推定に適した共通潜在空間の構築を可能とする．以上で構築された空間に射影された新たな特徴量を利用することで，高精度な感情推定が可能となる．

COVID-19 detection based on self-supervised transfer learning using chest X-ray images.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
Int. J. Comput. Assist. Radiol. Surg., 18, 4, 715, 722, 2023年04月
研究論文（学術雑誌）

Estimation of Degradation Degree in Road Infrastructure Based on Multi-Modal ABN Using Contrastive Learning.
Takaaki Higashi, Naoki Ogawa, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
Sensors, 23, 3, 1657, 1657, 2023年02月
研究論文（学術雑誌）

Diversity Learning Based on Multi-Latent Space for Medical Image Visual Question Generation.
He Zhu, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
Sensors, 23, 3, 1057, 1057, MDPI, 2023年02月
英語, 研究論文（学術雑誌）, Auxiliary clinical diagnosis has been researched to solve unevenly and insufficiently distributed clinical resources. However, auxiliary diagnosis is still dominated by human physicians, and how to make intelligent systems more involved in the diagnosis process is gradually becoming a concern. An interactive automated clinical diagnosis with a question-answering system and a question generation system can capture a patient's conditions from multiple perspectives with less physician involvement by asking different questions to drive and guide the diagnosis. This clinical diagnosis process requires diverse information to evaluate a patient from different perspectives to obtain an accurate diagnosis. Recently proposed medical question generation systems have not considered diversity. Thus, we propose a diversity learning-based visual question generation model using a multi-latent space to generate informative question sets from medical images. The proposed method generates various questions by embedding visual and language information in different latent spaces, whose diversity is trained by our newly proposed loss. We have also added control over the categories of generated questions, making the generated questions directional. Furthermore, we use a new metric named similarity to accurately evaluate the proposed model's performance. The experimental results on the Slake and VQA-RAD datasets demonstrate that the proposed method can generate questions with diverse information. Our model works with an answering model for interactive automated clinical diagnosis and generates datasets to replace the process of annotation that incurs huge labor costs.

Visual language modelを用いた変状画像に対する所見の自動生成—類似画像検索によるFew-shot learningの導入—
渡邉 優宇人, 小川 直輝, 前田 圭介, 小川 貴弘, 長谷山 美紀
AI・データサイエンス論文集, 4, 3, 223, 232, 公益社団法人 土木学会, 2023年
日本語, 本研究では，インフラ施設の点検過程で作成される点検記録に必要な所見の効率的な作成支援のため，Visual language modelを活用した所見の自動生成手法を提案する．変状画像から認識可能な事柄に加えて，技術者の判断や意見を含む文章である所見は点検記録の作成に不可欠であるが，その直接的な生成に対する議論は依然として少なく，所見の効率的な作成支援に向けた生成技術の実現が期待されている．そこで本稿では，近年，注目されている大規模言語モデルの応用研究である視覚と言語の双方を高精度に理解したテキスト出力を可能とするVisual language modelに対して，変状画像に対する類似画像検索に基づくFew-shot learningを導入することで，少数の変状画像および所見の組から効率的にその関係性を把握可能とする．本稿の最後では，実際の橋梁点検記録に含まれる変状画像から所見を生成する検証により，提案手法の有効性を確認する．

Attention-based Multiple Instance Learningに基づく背景の多様性に頑健な道路附属物の異状判定
渡部 航史, 小川 直輝, 前田 圭介, 小川 貴弘, 長谷山 美紀
AI・データサイエンス論文集, 4, 3, 482, 489, 公益社団法人 土木学会, 2023年
日本語, 本稿では，道路附属物の画像から異状の有無を判定する手法について提案を行う．道路標識や照明などの設備である道路附属物は，その設置数と設置範囲の広大さから点検の効率化が望まれており，現在ドローンを用いた自動点検技術の実現が期待されている．ドローンより撮像される道路附属物の画像には地面や空，道路など，多様な背景が含まれているが，従来研究ではこれらの背景の多様性を考慮できていなかった．そこで，本研究ではAttention機構を導入したMultiple Instance Learningを適用した異状判定手法を構築する．パッチ分割した撮像画像を入力し，各パッチの重要度を推定することで，背景領域と道路附属物の領域を区別した異状の学習を実現する．本稿の最後では，実際の道路附属物の画像を用いた実験を行い，7割程度の分類精度を達成し本研究のアプローチの有効性が確認された．

社会インフラ点検時の変状画像を対象とした変状種と劣化レベルのマルチタスク分類
小川 直輝, 前田 圭介, 小川 貴弘, 長谷山 美紀
AI・データサイエンス論文集, 4, 3, 807, 814, 公益社団法人 土木学会, 2023年
日本語, 本論文では，変状の種類（変状種）と進行度（劣化レベル）を同時に分類するマルチタスク分類手法を提案する．従来では，変状種毎に劣化レベルを分類するモデルの学習を行っていた．これに対し，提案手法では，学習時に変状種と劣化レベルを用いた損失の最小化を行うことで，あらかじめ画像に変状種を付与せずとも変状種の分類結果を考慮して劣化レベルを分類可能とする．本論文の最後では，実際に社会インフラに発生した変状を撮影した画像を用いた実験により，提案手法のモデルが変状種毎に構築したモデルに匹敵する劣化レベルの分類精度を達成可能であることを確認する．

社会インフラ維持管理効率化に向けた最先端AI研究
前田 圭介, 小川 貴弘, 長谷山 美紀
AI・データサイエンス論文集, 4, 3, 982, 989, 公益社団法人 土木学会, 2023年
日本語, AI技術の発展・高度化により，社会インフラ維持管理分野へのAI応用に関する研究が活発に進められている．これらの研究の多くは，維持管理業務等で得られる画像の性質を考慮した学習理論の構築に注目しており，ひび割れ検出や変状の種類分類，劣化レベル推定等，様々なタスクにおいて，AIの有効性が示されている．一方，真に業務効率化を実現するAIを構築するためには，実務に必要な機能を備えたAIを構築する必要がある．さらに，構築したAIを高度化し，継続的に活用するためには，実務においてAI構築に適した画像を取得する必要がある．そこで，本稿では，維持管理業務等で得られる画像を対象に構築されてきた学習理論，実務に必要な機能を備えたAI構築に関する従来研究および画像の取得方法に関する著者らの考えについて紹介する．

セマンティックセグメンテーションに基づく道路走行中の車載映像を用いた枯損木の検出
小川 直輝, 前田 圭介, 小川 貴弘, 長谷山 美紀
AI・データサイエンス論文集, 4, 3, 686, 693, 公益社団法人 土木学会, 2023年
日本語, 本稿では，道路走行中の車両から撮影した映像を用いた枯損木の自動検出に関する手法を提案する．提案手法では，多様な物体が撮影された映像中からセマンティックセグメンテーションに基づき樹木領域を抽出する．さらに，抽出した領域から色彩情報を用いて枯損木を検出する．提案手法により検出した枯損木の領域を技術者へ提示することで，技術者が効率的に枯損木を発見できるように支援する．本稿の最後では，実際の車載映像を用いた実験により提案手法の有効性を検証する．

Feature Integration via Back-Projection Ordering Multi-Modal Gaussian Process Latent Variable Model for Rating Prediction.
Kyohei Kamikawa, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICIP, 3125, 3129, 2023年
研究論文（国際会議プロシーディングス）

Multi-View Variational Recurrent Neural Network for Human Emotion Recognition Using Multi-Modal Biological Signals.
Yuya Moroto, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICIP, 2925, 2929, 2023年
研究論文（国際会議プロシーディングス）

Video-Music Retrieval with Fine-Grained Cross-Modal Alignment.
Yuki Era, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICIP, 2005, 2009, 2023年
研究論文（国際会議プロシーディングス）

Text-Guided Facial Image Manipulation for Wild Images via Manipulation Direction-Based Loss.
Yuto Watanabe, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICIP, 361, 365, 2023年
研究論文（国際会議プロシーディングス）

Text-to-image Diffusion Model Suppressing Catastrophic Forgetting via Elastic Weight Consolidation.
Haruka Matsuda, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 831, 832, 2023年
研究論文（国際会議プロシーディングス）

Deterioration Level Estimation for Infrastructures Considering Noisy Labels via DivideMix.
Masaki Kashiwagi, Keisuke Maeda, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 829, 830, 2023年
研究論文（国際会議プロシーディングス）

Novel Feature Extraction for Classification of Auditory-visual Stimuli from fNIRS Signals.
Taro Togo, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 759, 760, 2023年
研究論文（国際会議プロシーディングス）

A Controllable Recoloring Method for Novel Views Using Segment Anything Model.
Haoyang Wang, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 587, 588, 2023年
研究論文（国際会議プロシーディングス）

Caption Unification for Multiple Viewpoint Lifelogging Images and Its Verification.
Masaya Sato, Keisuke Maeda, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 415, 416, 2023年
研究論文（国際会議プロシーディングス）

Improving Visual Counterfactual Explanation Models for Image Classification via CLIP.
Xiang Li, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 390, 391, 2023年
研究論文（国際会議プロシーディングス）

画像の感情推定のためのラベル逆量子化を導入した正準相関分析
斉藤直輝, 前田圭介, 小川貴弘, 浅水仁, 長谷山美紀
電子情報通信学会論文誌 D(Web), J106-D, 5, 2023年

Few-Shot Personalized Saliency Prediction Using Tensor Regression for Preserving Structural Global Information.
Yuya Moroto, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2307.02799, 2023年
研究論文（学術雑誌）

Personalized Content Recommender System via Non-verbal Interaction Using Face Mesh and Facial Expression.
Yuya Moroto, Rintaro Yanagi, Naoki Ogawa, Kyohei Kamikawa, Keigo Sakurai, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ACM Multimedia, 9399, 9401, 2023年
研究論文（国際会議プロシーディングス）

Gromov-Wasserstein Autoencoders.
Nao Nakagawa, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ICLR, 2023年
研究論文（国際会議プロシーディングス）

Proficiency-level Estimation Using Heterogeneous Features via Label Dequantized CCA.
Tatsuki Seino, Naoki Saito 0006, Takahiro Ogawa 0001, Satoshi Asamizu, Miki Haseyama
ICCE-Taiwan, 813, 814, 2023年
研究論文（国際会議プロシーディングス）

Parameter-efficient Tuning of a Pre-trained Model via Prompt Learning in Cross-modal Retrieval.
Huaying Zhang, Rintaro Yanagi, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ICCE-Taiwan, 811, 812, 2023年
研究論文（国際会議プロシーディングス）

Binaural Audio Generation with Data Augmentation from 360° Videos.
Masaki Yoshida, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ICCE-Taiwan, 795, 796, 2023年
研究論文（国際会議プロシーディングス）

Prediction of Shoot Events by Considering Spatio-temporal Relations of Multimodal Features.
Ryota Goka, Yuya Moroto, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICCE-Taiwan, 793, 794, 2023年
研究論文（国際会議プロシーディングス）

Shoot Event Prediction in Soccer Considering Expected Goals Based on Players' Positions.
Ryota Goka, Yuya Moroto, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICCE-Taiwan, 449, 450, 2023年
研究論文（国際会議プロシーディングス）

Estimation of Amyloid-β Positivity Using QSM Images Considering Age Information.
Tsubasa Kunieda, Ren Togo, Noriko Nishioka, Yukie Shimizu, Shiro Watanabe, Kenji Hirata, Keisuke Maeda, Takahiro Ogawa 0001, Kohsuke Kudo, Miki Haseyama
ICCE-Taiwan, 165, 166, 2023年
研究論文（国際会議プロシーディングス）

A Medical Domain Visual Question Generation Model via Large Language Model.
He Zhu, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ICCE-Taiwan, 163, 164, 2023年
研究論文（国際会議プロシーディングス）

Defense Against Black-Box Adversarial Attacks Via Heterogeneous Fusion Features.
Jiahuan Zhang, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICASSP, 1, 5, 2023年
研究論文（国際会議プロシーディングス）

Binauralization Robust To Camera Rotation Using 360° Videos.
Masaki Yoshida, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ICASSP, 1, 5, 2023年
研究論文（国際会議プロシーディングス）

Learning Graph Laplacian from Intrinsic Patterns via Gaussian Process.
Koshi Watanabe, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICASSP, 1, 5, 2023年
研究論文（国際会議プロシーディングス）

Estimation of Visual Contents from Human Brain Signals via VQA Based on Brain-Specific Attention.
Ryo Shichida, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICASSP, 1, 5, 2023年
研究論文（国際会議プロシーディングス）

Class-Aware Shared Gaussian Process Dynamic Model.
Ryosuke Sawata, Takahiro Ogawa 0001, Miki Haseyama
ICASSP, 1, 5, 2023年
研究論文（国際会議プロシーディングス）

Improving Dropout in Graph Convolutional Networks for Recommendation via Contrastive Loss.
Hiroki Okamura, Keisuke Maeda, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ICASSP, 1, 5, 2023年
研究論文（国際会議プロシーディングス）

Multi-Label Classification in Anime Illustrations Based on Hierarchical Attribute Relationships.
Ziwen Lan, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
Sensors, 23, 10, 4798, 4798, 2023年
研究論文（学術雑誌）

Prediction of Shooting Events in Soccer Videos Using Complete Bipartite Graphs and Players' Spatial-Temporal Relations.
Ryota Goka, Yuya Moroto, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
Sensors, 23, 9, 4506, 4506, 2023年
研究論文（学術雑誌）

Material Compound-Property Retrieval Using Electron Microscope Images for Rubber Material Development.
Rintaro Yanagi, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
IEEE Access, 11, 88258, 88264, 2023年
研究論文（学術雑誌）

Hierarchical Multi-Label Attribute Classification With Graph Convolutional Networks on Anime Illustration.
Ziwen Lan, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
IEEE Access, 11, 35447, 35456, 2023年
研究論文（学術雑誌）

SpectralMAP: Approximating Data Manifold With Spectral Decomposition
Koshi Watanabe, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
IEEE ACCESS, 11, 31530, 31540, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2023年
英語, 研究論文（学術雑誌）, Dimensionality reduction is widely used to visualize complex high-dimensional data. This study presents a novel method for effective data visualization. Previous methods depend on local distance measurements for data manifold approximation. This leads to unreliable results when a data manifold locally oscillates because of some undesirable effects, such as noise effects. In this study, we overcome this limitation by introducing a dual approximation of a data manifold. We roughly approximate a data manifold with a neighborhood graph and prune it with a global filter. This dual scheme results in local oscillation robustness and yields effective visualization with explicit global preservation. We consider a global filter based on principal component analysis frameworks and derive it with the spectral information of the original high-dimensional data. Finally, we experiment with multiple datasets to verify our method, compare its performance to that of state-of-the-art methods, and confirm the effectiveness of our novelty and results.

Summarizing Data Structures with Gaussian Process and Robust Neighborhood Preservation
Koshi Watanabe, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
MACHINE LEARNING AND KNOWLEDGE DISCOVERY IN DATABASES, ECML PKDD 2022, PT V, 13717, 157, 173, SPRINGER INTERNATIONAL PUBLISHING AG, 2023年
英語, 研究論文（国際会議プロシーディングス）, Latent variable models summarize high-dimensional data while preserving its many complex properties. This paper proposes a locality-aware and low-rank approximated Gaussian process latent variable model (LolaGP) that can preserve the global relationship and local geometry in the derivation of the latent variables. We realize the global relationship by imitating the sample similarity non-linearly and the local geometry based on our newly constructed neighborhood graph. Formally, we derive LolaGP from GP-LVM and implement a locality-aware regularization to reflect its adjacency relationship. The neighborhood graph is constructed based on the latent variables, making the local preservation more resistant to noise disruption and the curse of dimensionality than the previous methods that directly construct it from the high-dimensional data. Furthermore, we introduce a new lower bound of a log-posterior distribution based on low-rank matrix approximation, which allows LolaGP to handle larger datasets than the conventional GP-LVM extensions. Our contribution is to preserve both the global and local structures in the derivation of the latent variables using the robust neighborhood graph and introduce the scalable lower bound of the log-posterior distribution. We conducted an experimental analysis using synthetic as well as images with and without highly noise disrupted datasets. From both qualitative and quantitative standpoint, our method produced successful results in all experimental settings.

Recallable Question Answering-Based Re-Ranking Considering Semantic Region for Cross-Modal Retrieval
Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama
IEEE OPEN JOURNAL OF SIGNAL PROCESSING, 4, 1, 11, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2023年
英語, 研究論文（学術雑誌）, Question answering (QA)-based re-ranking methods for cross-modal retrieval have been recently proposed to further narrow down similar candidate images. The conventional QA-based re-ranking methods provide questions to users by analyzing candidate images, and the initial retrieval results are re-ranked based on the user's feedback. Contrary to these developments, only focusing on performance improvement makes it difficult to efficiently elicit the user's retrieval intention. To realize more useful QA-based re-ranking, considering the user interaction for eliciting the user's retrieval intention is required. In this paper, we propose a QA-based re-ranking method with considering two important factors for eliciting the user's retrieval intention: query-image relevance and recallability. Considering the query-image relevance enables to only focus on the candidate images related to the provided query text, while, focusing on the recallability enables users to easily answer the provided question. With these procedures, our method can efficiently and effectively elicit the user's retrieval intention. Experimental results using Microsoft Common Objects in Context and computationally constructed dataset including similar candidate images show that our method can improve the performance of the cross-modal retrieval methods and the QA-based re-ranking methods.

Interpretable Visual Question Answering Referring to Outside Knowledge.
He Zhu, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2303.04388, 2023年
研究論文（学術雑誌）

Text-Guided Image Manipulation via Generative Adversarial Network With Referring Image Segmentation-Based Guidance.
Yuto Watanabe, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
IEEE Access, 11, 42534, 42545, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2023年
英語, 研究論文（学術雑誌）, This study proposes a novel text-guided image manipulation method that introduces referring image segmentation into a generative adversarial network. The proposed text-guided image manipulation method aims to manipulate images containing multiple objects while preserving text-unrelated regions. The proposed method assigns the task of distinguishing between text-related and unrelated regions in an image to segmentation guidance based on referring image segmentation. With this architecture, the adversarial generative network can focus on generating new attributes according to the text description and reconstructing text-unrelated regions. For the challenging input images with multiple objects, the experimental results demonstrate that the proposed method outperforms conventional methods in terms of image manipulation precision.

Cross-Modal Image Retrieval Considering Semantic Relationships With Many-to-Many Correspondence Loss.
Huaying Zhang, Rintaro Yanagi, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
IEEE Access, 11, 10675, 10686, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2023年
英語, 研究論文（学術雑誌）, A cross-modal image retrieval that explicitly considers semantic relationships between images and texts is proposed. Most conventional cross-modal image retrieval methods retrieve the target images by directly measuring the similarities between the candidate images and query texts in a common semantic embedding space. However, such methods tend to focus on a one-to-one correspondence between a predefined image-text pair during the training phase, and other semantically similar images and texts are ignored. By considering the many-to-many correspondences between semantically similar images and texts, a common embedding space is constructed to assure semantic relationships, which allows users to accurately find more images that are related to the input query texts. Thus, in this paper, we propose a cross-modal image retrieval method that considers semantic relationships between images and texts. The proposed method calculates the similarities between texts as semantic similarities to acquire the relationships. Then, we introduce a loss function that explicitly constructs the many-to-many correspondences between semantically similar images and texts from their semantic relationships. We also propose an evaluation metric to assess whether each method can construct an embedding space considering the semantic relationships. Experimental results demonstrate that the proposed method outperforms conventional methods in terms of this newly proposed metric.

Similar interior coordination image retrieval with multi-view features
Ren Togo, Yuki Honma, Maiku Abe, Takahiro Ogawa, Miki Haseyama
International Journal of Multimedia Information Retrieval, 11, 4, 731, 740, Springer Science and Business Media LLC, 2022年08月26日
英語, 研究論文（学術雑誌）, This paper presents a novel similar image retrieval method for interior coordination. Interior coordination is very familiar; however, it is still an abstract and difficult concept. Even if we are involved in coordination every day, it does not mean we can become professional coordinators. By realizing the retrieval that can provide similar interior coordination images from a query room image, inspiring users' ideas for interior coordination becomes feasible. In the proposed method, we extract image features specialized for interior coordination and realize similar interior coordination image retrieval. We employ multi-view features: object-based, color-based, and semantic-based features, in the feature extraction phase. The extracted features are used to calculate similarity between the query image and the database images for the retrieval. We conducted experiments using a sophisticated real-world interior coordination image dataset. Furthermore, we qualitatively and quantitatively evaluated the effectiveness of the proposed method.

Brain Decoding of Multiple Subjects for Estimating Visual Information Based on a Probabilistic Generative Model
Takaaki Higashi, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
SENSORS, 22, 16, 6148, 6148, MDPI, 2022年08月
英語, 研究論文（学術雑誌）, Brain decoding is a process of decoding human cognitive contents from brain activities. However, improving the accuracy of brain decoding remains difficult due to the unique characteristics of the brain, such as the small sample size and high dimensionality of brain activities. Therefore, this paper proposes a method that effectively uses multi-subject brain activities to improve brain decoding accuracy. Specifically, we distinguish between the shared information common to multi-subject brain activities and the individual information based on each subject's brain activities, and both types of information are used to decode human visual cognition. Both types of information are extracted as features belonging to a latent space using a probabilistic generative model. In the experiment, an publicly available dataset and five subjects were used, and the estimation accuracy was validated on the basis of a confidence score ranging from 0 to 1, and a large value indicates superiority. The proposed method achieved a confidence score of 0.867 for the best subject and an average of 0.813 for the five subjects, which was the best compared to other methods. The experimental results show that the proposed method can accurately decode visual cognition compared with other existing methods in which the shared information is not distinguished from the individual information.

Defect Detection of Subway Tunnels Using Advanced U-Net Network
An Wang, Ren Togo, Takahiro Ogawa, Miki Haseyama
SENSORS, 22, 6, 2330, 2330, MDPI, 2022年03月
英語, 研究論文（学術雑誌）, In this paper, we present a novel defect detection model based on an improved U-Net architecture. As a semantic segmentation task, the defect detection task has the problems of background-foreground imbalance, multi-scale targets, and feature similarity between the background and defects in the real-world data. Conventionally, general convolutional neural network (CNN)-based networks mainly focus on natural image tasks, which are insensitive to the problems in our task. The proposed method has a network design for multi-scale segmentation based on the U-Net architecture including an atrous spatial pyramid pooling (ASPP) module and an inception module, and can detect various types of defects compared to conventional simple CNN-based methods. Through the experiments using a real-world subway tunnel image dataset, the proposed method showed a better performance than that of general semantic segmentation including state-of-the-art methods. Additionally, we showed that our method can achieve excellent detection balance among multi-scale defects.

Microscopy and biomimetics: the NanoSuit® method and image retrieval platform
Takahiko Hariyama, Yasuharu Takaku, Hideya Kawasaki, Masatsugu Shimomura, Chiyo Senoh, Yumi Yamahama, Atsushi Hozumi, Satoru Ito, Naoto Matsuda, Satoshi Yamada, Toshiya Itoh, Miki Haseyama, Takahiro Ogawa, Naoki Mori, Shuhei So, Hidefumi Mitsuno, Masahiro Ohara, Shuhei Nomura, Masao Hirasaka
Microscopy, 71, 1, 1, 12, Oxford University Press (OUP), 2022年01月29日
研究論文（学術雑誌）, Abstract

This review aims to clarify a suitable method towards achieving next-generation sustainability. As represented by the term ‘Anthropocene’, the Earth, including humans, is entering a critical era; therefore, science has a great responsibility to solve it. Biomimetics, the emulation of the models, systems and elements of nature, especially biological science, is a powerful tool to approach sustainability problems. Microscopy has made great progress with the technology of observing biological and artificial materials and its techniques have been continuously improved, most recently through the NanoSuit® method. As one of the most important tools across many facets of research and development, microscopy has produced a large amount of accumulated digital data. However, it is difficult to extract useful data for making things as biomimetic ideas despite a large amount of biological data. Here, we would like to find a way to organically connect the indispensable microscopic data with the new biomimetics to solve complex human problems.

Popularity-Aware Graph Social Recommendation for Fully Non-Interaction Users.
Nozomu Onodera, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
Proceedings of the 4th ACM International Conference on Multimedia in Asia(MMAsia), 30, 5, ACM, 2022年
研究論文（国際会議プロシーディングス）

Affective Embedding Framework with Semantic Representations from Tweets for Zero-Shot Visual Sentiment Prediction.
Yingrui Ye, Yuya Moroto, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
Proceedings of the 4th ACM International Conference on Multimedia in Asia(MMAsia), 6, 7, ACM, 2022年
研究論文（国際会議プロシーディングス）

Visual Sentiment Prediction Using Cross-Way Few-Shot Learning Based on Knowledge Distillation.
Yingrui Ye, Yuya Moroto, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
2022 IEEE International Conference on Image Processing(ICIP), 3838, 3842, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Human-Centric Image Retrieval with Gaze-Based Image Captioning.
Yuhu Feng, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
2022 IEEE International Conference on Image Processing(ICIP), 3828, 3832, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Few-Shot Personalized Saliency Prediction with Similarity of Gaze Tendency Using Object-Based Structural Information.
Yuya Moroto, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
2022 IEEE International Conference on Image Processing(ICIP), 3823, 3827, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Gaussian Distributed Graph Constrained Multi-Modal Gaussian Process Latent Variable Model for Ordinal Labeled Data.
Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
2022 IEEE International Conference on Image Processing(ICIP), 3798, 3802, IEEE, 2022年
研究論文（国際会議プロシーディングス）

GCN-Based Multi-Modal Multi-Label Attribute Classification in Anime Illustration Using Domain-Specific Semantic Features.
Ziwen Lan, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
2022 IEEE International Conference on Image Processing(ICIP), 2021, 2025, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Trend Prediction of Students' Mock Examination Results Using Matrix Completion.
Yutaka Yamada, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
11th IEEE Global Conference on Consumer Electronics(GCCE), 891, 892, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Shoot Event Prediction from Soccer Videos by Considering Players' Spatio-Temporal Relations.
Ryota Goka, Yuya Moroto, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
11th IEEE Global Conference on Consumer Electronics(GCCE), 406, 407, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Refinement of Gaze-based Image Caption for Image Retrieval.
Yuhu Feng, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
11th IEEE Global Conference on Consumer Electronics(GCCE), 272, 273, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Boosting Automatic COVID-19 Detection Performance with Self-Supervised Learning and Batch Knowledge Ensembling.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2212.09281, 2022年
研究論文（学術雑誌）

COVID-19 Detection Based on Self-Supervised Transfer Learning Using Chest X-Ray Images.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2212.09276, 2022年
研究論文（学術雑誌）

Union-set Multi-source Model Adaptation for Semantic Segmentation.
Zongyao Li, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2212.02785, 2022年
研究論文（学術雑誌）

RGMIM: Region-Guided Masked Image Modeling for COVID-19 Detection.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2211.00313, 2022年
研究論文（学術雑誌）

Rubber Material Retrieval System using Electron Microscope Images for Rubber Material Development.
Rintaro Yanagi, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
MMAsia, 44, 3, ACM, 2022年
研究論文（国際会議プロシーディングス）

Disentangled Image Attribute Editing in Latent Space via Mask-Based Retention Loss.
Shunya Ohaga, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
MMAsia, 25, 7, ACM, 2022年
研究論文（国際会議プロシーディングス）

Assessment of Image Manipulation Using Natural Language Description: Quantification of Manipulation Direction.
Yuto Watanabe, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICIP, 1046, 1050, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Improving Model Adaptation for Semantic Segmentation by Learning Model-Invariant Features with Multiple Source-Domain Models.
Zongyao Li, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ICIP, 421, 425, IEEE, 2022年
研究論文（国際会議プロシーディングス）

A Multimodal Interpretable Visual Question Answering Model Introducing Image Caption Processor.
He Zhu, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 777, 778, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Cross-modal Image Retrieval Considering Semantic Relationships with Object Information.
Huaying Zhang, Rintaro Yanagi, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 775, 776, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Free-viewpoint Sports Video Generation Based on Dynamic NeRF Considering Time Series.
Masato Kawai, Rintaro Yanagi, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 408, 409, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Content-based Image Retrieval Using Effective Synthesized Images from Different Camera Views via pixelNeRF.
Yuki Era, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 404, 405, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Analysis of Relationships between Visual Cognitive Contents and Response of Each Brain Region via Visual Question Answering.
Ryo Shichida, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 402, 403, IEEE, 2022年
研究論文（国際会議プロシーディングス）

GCN-based Collaborative Filtering Considering Personality Bias.
Hiroki Okamura, Keisuke Maeda, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 278, 279, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Prediction of Amyloid-β Positivity Using QSM Images Based on Bootstrap Your Own Latent.
Tsubasa Kunieda, Ren Togo, Noriko Nishioka, Yukie Shimizu, Shiro Watanabe, Kenji Hirata, Keisuke Maeda, Takahiro Ogawa 0001, Kohsuke Kudo, Miki Haseyama
GCCE, 137, 138, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Cross-platform Recommendation Considering Common Users' Preferences Based on Preference Propagation GraphNet.
Kazuki Yamamoto, Keisuke Maeda, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 135, 136, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Union-Set Multi-source Model Adaptation for Semantic Segmentation.
Zongyao Li, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ECCV (29), 579, 595, Springer, 2022年
研究論文（国際会議プロシーディングス）

Trial Analysis of the Relationship between Taste and Biological Information Obtained While Eating Strawberries for Sensory Evaluation.
Keisuke Maeda, Ren Togo, Takahiro Ogawa 0001, Shin-ichi Adachi, Fumiaki Yoshizawa, Miki Haseyama
Sensors, 22, 23, 9496, 9496, 2022年
研究論文（学術雑誌）

Distress Detection in Subway Tunnel Images via Data Augmentation Based on Selective Image Cropping and Patching.
Keisuke Maeda, Saya Takada, Tomoki Haruyama, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
Sensors, 22, 22, 8932, 8932, 2022年
研究論文（学術雑誌）

Compressed gastric image generation based on soft-label dataset distillation for medical data sharing.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
Comput. Methods Programs Biomed., 227, 107189, 107189, 2022年
研究論文（学術雑誌）

Dataset Complexity Assessment Based on Cumulative Maximum Scaled Area Under Laplacian Spectrum.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2209.14743, 2022年
研究論文（学術雑誌）

Compressed Gastric Image Generation Based on Soft-Label Dataset Distillation for Medical Data Sharing.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2209.14635, 2022年
研究論文（学術雑誌）

Dataset Distillation using Parameter Pruning.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2209.14609, 2022年
研究論文（学術雑誌）

Dataset Distillation for Medical Dataset Sharing.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2209.14603, 2022年
研究論文（学術雑誌）

Gromov-Wasserstein Autoencoders.
Nao Nakagawa, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2209.07007, 2022年
研究論文（学術雑誌）

TriBYOL: Triplet BYOL for Self-Supervised Representation Learning.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2206.03012, 2022年
研究論文（学術雑誌）

Self-Knowledge Distillation based Self-Supervised Learning for Covid-19 Detection from Chest X-Ray Images.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
CoRR, abs/2206.03009, 2022年
研究論文（学術雑誌）

Generating Captions of Imagined content from Human Brain Activities Applying An Image Captioning Model.
Saya Takada, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
4th IEEE Global Conference on Life Sciences and Technologies(LifeTech), 614, 615, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Knowledge-Guided Sequential Recommendation with Reinforcement Learning Using Empirical Distribution Function.
Keigo Sakurai, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
4th IEEE Global Conference on Life Sciences and Technologies(LifeTech), 187, 188, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Transformer Based Multimodal Scene Recognition in Soccer Videos.
Yaozong Gan, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ICME Workshops, 1, 6, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Scene Retrieval in Soccer Videos by Spatial-temporal Attention with Video Vision Transformer.
Yaozong Gan, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ICCE-TW, 453, 454, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Multi-scale Defect Detection from Subway Tunnel Images with Spatial Attention Mechanism.
An Wang, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ICCE-TW, 305, 306, IEEE, 2022年
研究論文（国際会議プロシーディングス）

橋梁点検時の技術者の一人称および三人称視点映像を用いた点検動作の分類に関する検討
増田毅, 前田圭介, 藤後廉, 小川貴弘, 長谷山美紀
映像情報メディア学会技術報告, 46, 6(MMS2022 1-37/ME2022 26-62/AIT2022 1-37), 303, 304, IEEE, 2022年

TriBYOL: Triplet BYOL for Self-Supervised Representation Learning.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ICASSP, 3458, 3462, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Divergence-Guided Feature Alignment for Cross-Domain Object Detection.
Zongyao Li, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ICASSP, 2240, 2244, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Self-Knowledge Distillation based Self-Supervised Learning for Covid-19 Detection from Chest X-Ray Images.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ICASSP, 1371, 1375, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Regularization Meets Enhanced Multi-Stage Fusion Features: Making CNN More Robust against White-Box Adversarial Attacks.
Jiahuan Zhang, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
Sensors, 22, 14, 5431, 5431, MDPI, 2022年
英語, 研究論文（学術雑誌）, Regularization has become an important method in adversarial defense. However, the existing regularization-based defense methods do not discuss which features in convolutional neural networks (CNN) are more suitable for regularization. Thus, in this paper, we propose a multi-stage feature fusion network with a feature regularization operation, which is called Enhanced Multi-Stage Feature Fusion Network (EMSF(2)Net). EMSF(2)Net mainly combines three parts: multi-stage feature enhancement (MSFE), multi-stage feature fusion (MSF2), and regularization. Specifically, MSFE aims to obtain enhanced and expressive features in each stage by multiplying the features of each channel; MSF2 aims to fuse the enhanced features of different stages to further enrich the information of the feature, and the regularization part can regularize the fused and original features during the training process. EMSF(2)Net has proved that if the regularization term of the enhanced multi-stage feature is added, the adversarial robustness of CNN will be significantly improved. The experimental results on extensive white-box attacks on the CIFAR-10 dataset illustrate the robustness and effectiveness of the proposed method.

Controllable Music Playlist Generation Based on Knowledge Graph and Reinforcement Learning.
Keigo Sakurai, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
Sensors, 22, 10, 3722, 3722, 2022年
研究論文（学術雑誌）

Learning intra-domain style-invariant representation for unsupervised domain adaptation of semantic segmentation.
Zongyao Li, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
Pattern Recognit., 132, 108911, 108911, 2022年
研究論文（学術雑誌）

Dataset complexity assessment based on cumulative maximum scaled area under Laplacian spectrum.
Guang Li 0008, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
Multimedia Tools and Applications, 81, 22, 32287, 32303, 2022年
研究論文（学術雑誌）

User-centric multimodal feature extraction for personalized retrieval of tumblr posts.
Kazuma Ohtomo, Ryosuke Harakawa, Takahiro Ogawa 0001, Miki Haseyama, Masahiro Iwahashi
Multimedia Tools and Applications, 81, 2, 2979, 3003, 2022年
研究論文（学術雑誌）

Chain centre loss: A psychology inspired loss function for image sentiment analysis.
Yun Liang 0014, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
Neurocomputing, 495, 118, 128, 2022年
研究論文（学術雑誌）

Generative Adversarial Network Including Referring Image Segmentation For Text-Guided Image Manipulation.
Yuto Watanabe, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICASSP, 4818, 4822, IEEE, 2022年
英語, 研究論文（国際会議プロシーディングス）, This paper proposes a novel generative adversarial network to improve the performance of image manipulation using natural language descriptions that contain desired attributes. Text-guided image manipulation aims to semantically manipulate an image aligned with the text description while preserving text-irrelevant regions. To achieve this, we newly introduce referring image segmentation into the generative adversarial network for image manipulation. The referring image segmentation aims to generate a segmentation mask that extracts the text-relevant region. By utilizing the feature map of the segmentation mask in the network, the proposed method explicitly distinguishes the text-relevant and irrelevant regions and has the following two contributions. First, our model can pay attention only to the text-relevant region and manipulate the region aligned with the text description. Second, our model can achieve an appropriate balance between the generation of accurate attributes in the text-relevant region and the reconstruction in the text-irrelevant regions. Experimental results show that the proposed method can significantly improve the performance of image manipulation.

Human Emotion Recognition Using Multi-Modal Biological Signals Based On Time Lag-Considered Correlation Maximization.
Yuya Moroto, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICASSP, 4683, 4687, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Distributed Label Dequantized Gaussian Process Latent Variable Model for Multi-View Data Integration.
Koshi Watanabe, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICASSP, 4643, 4647, IEEE, 2022年
英語, 研究論文（国際会議プロシーディングス）, In this paper, we present a novel method for multi-view data analysis, distributed label dequantized Gaussian process latent variable model (DLDGP). DLDGP can integrate multi-view data and class information into a common latent space. In the previous multi-view methods, the dimension of label features transformed from the class information is much smaller than those of the other modalities, which causes a dimensionality-limitation problem in the latent space. DLDGP extends the dimension of the label features by a distributed label dequantization scheme. Additionally, DLDGP calculates correlation between different classes by encoding class information into distributed features. DLDGP can correctly capture the relationship between multi-view data and obtain the latent features with high expression ability. Experimental results show the effectiveness of our method by using the open dataset.

Variational Bayesian Graph Convolutional Network for Robust Collaborative Filtering.
Nozomu Onodera, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICASSP, 3908, 3912, IEEE, 2022年
研究論文（国際会議プロシーディングス）

Time-Lag Aware Latent Variable Model for Prediction of Important Scenes Using Baseball Videos and Tweets.
Kaito Hirasawa, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
Sensors, 22, 7, 2465, 2465, 2022年
研究論文（学術雑誌）

Refining Graph Representation for Cross-Domain Recommendation Based on Edge Pruning in Latent Space.
Taisei Hirakawa, Keisuke Maeda, Takahiro Ogawa 0001, Satoshi Asamizu, Miki Haseyama
IEEE Access, 10, 12503, 12509, 2022年
研究論文（学術雑誌）

Interactive Re-ranking via Object Entropy-Guided Question Answering for Cross-Modal Image Retrieval.
Rintaro Yanagi, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
ACM Trans. Multim. Comput. Commun. Appl., 18, 3, 68, 17, ASSOC COMPUTING MACHINERY, 2022年
英語, 研究論文（学術雑誌）, Cross-modal image-retrieval methods retrieve desired images from a query text by learning relationships between texts and images. Such a retrieval approach is one of the most effective ways of achieving the easiness of query preparation. Recent cross-modal image-retrieval methods are convenient and accurate when users input a query text that can be used to uniquely identify the desired image. However, in reality, users frequently input ambiguous query texts, and these ambiguous queries make it difficult to obtain desired images. To overcome these difficulties, in this study, we propose a novel interactive cross-modal image-retrieval method based on question answering. The proposed method analyzes candidate images and asks users questions to obtain information that can narrow down retrieval candidates. By only answering questions generated by the proposed method, users can reach their desired images, even when using an ambiguous query text. Experimental results show the proposed method's effectiveness.

Deterioration Level Estimation Based on Convolutional Neural Network Using Confidence-Aware Attention Mechanism for Infrastructure Inspection.
Naoki Ogawa, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
Sensors, 22, 1, 382, 382, 2022年
研究論文（学術雑誌）

Domain Adaptive Cross-Modal Image Retrieval via Modality and Domain Translations
Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, E104A, 6, 866, 875, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2021年06月
英語, 研究論文（学術雑誌）, Various cross-modal retrieval methods that can retrieve images related to a query sentence without text annotations have been proposed. Although a high level of retrieval performance is achieved by these methods, they have been developed for a single domain retrieval setting. When retrieval candidate images come from various domains, the retrieval performance of these methods might be decreased. To deal with this problem, we propose a new domain adaptive cross-modal retrieval method. By translating a modality and domains of a query and candidate images, our method can retrieve desired images accurately in a different domain retrieval setting. Experimental results for clipart and painting datasets showed that the proposed method has better retrieval performance than that of other conventional and state-of-the-art methods.

Database-adaptive Re-ranking for Enhancing Cross-modal Image Retrieval.
Rintaro Yanagi, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
MM '21: ACM Multimedia Conference, 3816, 3825, ACM, 2021年
研究論文（国際会議プロシーディングス）

Interpretable Representation Learning on Natural Image Datasets via Reconstruction in Visual-Semantic Embedding Space.
Nao Nakagawa, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
2021 IEEE International Conference on Image Processing(ICIP), 2473, 2477, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Cross-Domain Recommendation Method Based On Multi-Layer Graph Analysis With Visual Information.
Taisei Hirakawa, Keisuke Maeda, Takahiro Ogawa 0001, Satoshi Asamizu, Miki Haseyama
ICIP, 2688, 2692, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Time-Lag Aware Multi-Modal Variational Autoencoder Using Baseball Videos And Tweets For Prediction Of Important Scenes.
Kaito Hirasawa, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICIP, 2678, 2682, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Segmentation-Aware Text-Guided Image Manipulation.
Tomoki Haruyama, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICIP, 2433, 2437, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Few-Shot Personalized Saliency Prediction using Person Similarity based on Collaborative Multi-Output Gaussian Process Regression.
Yuya Moroto, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICIP, 1469, 1473, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Interest Level Estimation via Multi-Modal Gaussian Process Latent Variable Factorization.
Kyohei Kamikawa, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICIP, 1209, 1213, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Deep Metric Network Via Heterogeneous Semantics for Image Sentiment Analysis.
Yun Liang 0014, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICIP, 1039, 1043, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Correlation-Aware Attention Branch Network Using Multi-Modal Data For Deterioration Level Estimation Of Infrastructures.
Naoki Ogawa, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
ICIP, 1014, 1018, IEEE, 2021年
研究論文（国際会議プロシーディングス）

User Background Information-Aware Music Recommendation with Reinforcement Learning-Based Knowledge Graph Exploration.
Keigo Sakurai, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
IEEE International Conference on Consumer Electronics-Taiwan(ICCE-TW), 1, 2, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Degradation Level Estimation of Road Structures via Attention Branch Network with Text Data.
Naoki Ogawa, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
IEEE International Conference on Consumer Electronics-Taiwan(ICCE-TW), 1, 2, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Cross-view Self-supervised Learning via Momentum Statistics in Batch Normalization.
Guang Li, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
IEEE International Conference on Consumer Electronics-Taiwan(ICCE-TW), 1, 2, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Triplet Self-Supervised Learning for Gastritis Detection with Scarce Annotations.
Guang Li, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 787, 788, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Defense Against Image Captioning Attacks via A Robust and Stable Recurrent Neural Network.
Jiahuan Zhang, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 785, 786, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Text-Guided Image Manipulation for Desired Region Using Referring Image Segmentation.
Yuto Watanabe, Ren Togo, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 661, 662, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Captioning特徴を利用したグラフ畳み込みネットワークに基づくアニメイラストのマルチラベル画像分類に関する検討
LAN Ziwen, 前田圭介, 小川貴弘, 長谷山美紀
映像情報メディア学会技術報告, 46, 6(MMS2022 1-37/ME2022 26-62/AIT2022 1-37), 273, 274, IEEE, 2021年

A Trial of Fine-grained Classification of Expert-novice Level Using Bio-signals While Inspecting Subway Tunnels.
Kaito Hirasawa, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 204, 205, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Listener Recommendation for Artist Based on Knowledge Graph and Reinforcement Learning.
Keigo Sakurai, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 202, 203, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Movie Rating Estimation Based on Weakly Supervised Multi-modal Latent Variable Model.
Koshi Watanabe, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 195, 196, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Detection of Off-screen Sound Based on Loss Function of Self-supervised Audio-visual Spatialization.
Masaki Yoshida, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 193, 194, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Visual Sentiment Prediction Using Few-shot Learning via Distribution Relations of Visual Features.
Yingrui Ye, Yuya Moroto, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 191, 192, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Multi-class Similar Scene Retrieval in Soccer Videos: A Scene Confusion Reduction Method Based on Combination of Long and Short Frame Sequences.
Yaozong Gan, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 117, 118, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Sports Action Detection Based on Self-Supervised Feature Learning and Object Detection.
Tsuyoshi Masuda, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 54, 55, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Analysis of Social Trends Related to COVID-19 Pandemic Utilizing Social Media Data.
Taisei Hirakawa, Keisuke Maeda, Takahiro Ogawa 0001, Satoshi Asamizu, Miki Haseyama
GCCE, 43, 44, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Estimating Imagined Images from Brain Activities via Visual Question Answering.
Saya Takada, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 35, 36, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Action Classification from Egocentric Videos Using Reinforcement Learning-based Pose Estimation.
Shunya Ohaga, Ren Togo, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 9, 10, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Graph Analysis-based Recommendation via Entity Embeddings Using Wikipedia.
Nozomu Onodera, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
GCCE, 5, 6, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Reliable Estimation of Deterioration Levels via Late Fusion Using Multi-View Distress Images for Practical Inspection.
Keisuke Maeda, Naoki Ogawa, Takahiro Ogawa 0001, Miki Haseyama
Journal of Imaging, 7, 12, 273, 273, 2021年
研究論文（学術雑誌）

Feature Integration Through Semi-Supervised Multimodal Gaussian Process Latent Variable Model With Pseudo-Labels for Interest Level Estimation.
Kyohei Kamikawa, Keisuke Maeda, Takahiro Ogawa 0001, Miki Haseyama
IEEE Access, 9, 163843, 163850, 2021年
研究論文（学術雑誌）

[Paper] Personalized Recommendation of Tumblr Posts Using Graph Convolutional Networks with Preference-aware Multimodal Features
Kazuma Ohtomo, Ryosuke Harakawa, Takahiro Ogawa, Miki Haseyama, Masahiro Iwahashi
ITE Transactions on Media Technology and Applications, 9, 1, 54, 61, Institute of Image Information and Television Engineers, 2021年
研究論文（学術雑誌）

光電容積脈波による連続血圧推定における血流の流量と抵抗指標を用いた精度向上方式の提案と評価
川上健, 川上健, 川上健, 住友和弘, 菅野厚博, 小川貴弘, 南重信, 長谷山美紀
電気学会論文誌 E, 141, 6, 2021年

User-selectable Event Summarization in Unedited Raw Soccer Video via Multimodal Bidirectional LSTM
Tomoki Haruyama, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
ITE TRANSACTIONS ON MEDIA TECHNOLOGY AND APPLICATIONS, 9, 1, 42, 53, INST IMAGE INFORMATION & TELEVISION ENGINEERS, 2021年
英語, 研究論文（学術雑誌）, A new method that generates user-selectable event summaries from unedited raw soccer videos is presented in this paper. Since there are more unedited raw soccer videos than broadcasted/distributed soccer videos and unedited videos have various viewers, it is necessary to analyze these videos for meeting the demands of various viewers. The proposed method introduces a multimodal CNN-BiLSTM architecture for analyzing unedited raw soccer videos. This architecture extracts candidate scenes for event summarization from unedited soccer videos and classifies these scenes into typical events. Finally, our method generates user-selectable event summaries by simultaneously considering the importance of candidate scenes and the event classification results. Experimental results using real unedited raw soccer videos show the effectiveness of our method.

Cross-domain Recommendation Based on Multilayer Graph Analysis Using Subgraph Representation
Taisei Hirakawa, Keisuke Maeda, Takahiro Ogawa, Satoshi Asamizu, Miki Haseyama
INTERNATIONAL WORKSHOP ON ADVANCED IMAGING TECHNOLOGY (IWAIT) 2021, 11766, SPIE-INT SOC OPTICAL ENGINEERING, 2021年
英語, 研究論文（国際会議プロシーディングス）, This paper presents cross-domain recommendation based on multilayer graph analysis using subgraph representation. The proposed method constructs two graphs in source and target domains utilizing user-item embedding and trains link relationships between the users' embedding features on each above graph via graph convolutional networks considering subgraph representation. Thus, the proposed method can obtain features with high representation ability, and this is the main contribution of this paper. Then the proposed method can estimate the user's embedding features in the target domain from those in the source domain and recommend items to users by using the estimated features. Experiments on real-world e-commerce datasets verify the effectiveness of the proposed method.

Interest Estimation Method Based on 2D Pose Features on Security Camera
Yuki Honma, Ren Togo, Maiku Abe, Takahiro Ogawa, Miki Haseyama
INTERNATIONAL WORKSHOP ON ADVANCED IMAGING TECHNOLOGY (IWAIT) 2021, 11766, SPIE-INT SOC OPTICAL ENGINEERING, 2021年
英語, 研究論文（国際会議プロシーディングス）, This paper proposes a customer interest estimation method using security camera to meet the demand of the retail industry. In the field of retail industry, it is considered that the understanding of customers' interests in the real store can be used for various marketing activities such as the product development and the layout of the store. Then, it is important to pay attention to customers' behavior in the real store. Their behavior is often recorded by the cameras installed in the store for security purposes. A method for estimating their interests from the videos of the security camera is presented in this paper. The novelty of our method is three-fold. Firstly, the experimental data of subjects in our group were taken by using the security camera already installed in the real store. Secondly, we used a pre-trained posture estimation model and treated the results as the features to be trained by a two-layer neural network model. Finally, a professional have annotated the subjects' interests. The effectiveness of our method was confirmed by comparing with benchmark supervised machine learning models.

Preliminary study of AI-assisted diagnosis using FDG-PET/CT for axillary lymph node metastasis in patients with breast cancer
Zongyao Li, Kazuhiro Kitajima, Kenji Hirata, Ren Togo, Junki Takenaka, Yasuo Miyoshi, Kohsuke Kudo, Takahiro Ogawa, Miki Haseyama
EJNMMI RESEARCH, 11, 1, SPRINGER, 2021年01月
英語, 研究論文（学術雑誌）, Background To improve the diagnostic accuracy of axillary lymph node (LN) metastasis in breast cancer patients using 2-[F-18]FDG-PET/CT, we constructed an artificial intelligence (AI)-assisted diagnosis system that uses deep-learning technologies. Materials and methods Two clinicians and the new AI system retrospectively analyzed and diagnosed 414 axillae of 407 patients with biopsy-proven breast cancer who had undergone 2-[F-18]FDG-PET/CT before a mastectomy or breast-conserving surgery with a sentinel lymph node (LN) biopsy and/or axillary LN dissection. We designed and trained a deep 3D convolutional neural network (CNN) as the AI model. The diagnoses from the clinicians were blended with the diagnoses from the AI model to improve the diagnostic accuracy. Results Although the AI model did not outperform the clinicians, the diagnostic accuracies of the clinicians were considerably improved by collaborating with the AI model: the two clinicians' sensitivities of 59.8% and 57.4% increased to 68.6% and 64.2%, respectively, whereas the clinicians' specificities of 99.0% and 99.5% remained unchanged. Conclusions It is expected that AI using deep-learning technologies will be useful in diagnosing axillary LN metastasis using 2-[F-18]FDG-PET/CT. Even if the diagnostic performance of AI is not better than that of clinicians, taking AI diagnoses into consideration may positively impact the overall diagnostic accuracy.

A Note on Detection of Sports Action Based on Temporal Cycle Consistency Learning
Tsuyoshi Masuda, Ren Togo, Takahiro Ogawa, Miki Haseyama
INTERNATIONAL WORKSHOP ON ADVANCED IMAGING TECHNOLOGY (IWAIT) 2021, 11766, SPIE-INT SOC OPTICAL ENGINEERING, 2021年
英語, 研究論文（国際会議プロシーディングス）, This paper presents a method for action detection based on Temporal Cycle Consistency(TCC) Learning. The proposed method realizes the action detection of flexible length segments based on a frame-level action prediction technique. We enable calculation of similarities for spatio-temporal features based on TCC to detect target actions from input videos. Finally, our method determines temporal segments by smoothing the frame-level action detection result. Experimental results show the validity of the proposed method.

Interior Coordination Image Retrieval with Object-Detection-Based and Color Features
Ren Togo, Takahiro Ogawa, Miki Haseyama
INTERNATIONAL WORKSHOP ON ADVANCED IMAGING TECHNOLOGY (IWAIT) 2021, 11766, SPIE-INT SOC OPTICAL ENGINEERING, 2021年
英語, 研究論文（国際会議プロシーディングス）, This paper presents a new interior coordination image retrieval method using object-detection-based and color features. Interior coordination requires consideration of objects' positional information and the overall atmosphere of the room simultaneously. However, similar image retrieval methods considering the coordination characteristics have not been proposed. In the proposed method, we extract different types of features from interior coordination images and realize the similar interior coordination image retrieval based on our newly derived features.

Self-Supervised Learning for Gastritis Detection with Gastric X-Ray Images.
Guang Li, Ren Togo, Takahiro Ogawa, Miki Haseyama
CoRR, abs/2104.02864, 2021年
研究論文（学術雑誌）

Soft-Label Anonymous Gastric X-ray Image Distillation.
Guang Li, Ren Togo, Takahiro Ogawa, Miki Haseyama
CoRR, abs/2104.02857, 305, 309, IEEE, 2021年
研究論文（学術雑誌）

IR Questioner: QA-based Interactive Retrieval System.
Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama
ICMR '21: International Conference on Multimedia Retrieval(ICMR), 611, 614, ACM, 2021年
英語, 研究論文（国際会議プロシーディングス）, Image retrieval from a given text query (text-to-image retrieval) is one of the most essential systems, and it is effectively utilized for databases (DBs) on the Web. To make them more versatile and familiar, a retrieval system that is adaptive even for personal DBs such as images in smartphones and lifelogging devices should be considered. In this paper, we present a novel text-to-image retrieval system that is specialized for personal DBs. With the cross-modal scheme and the question-answering scheme, the developed system enables users to obtain the desired image effectively even from personal DBs. Our demo is available at https://sites.google.com/view/ir-questioner/.

Human Emotion Estimation Using Multi-Modal Variational AutoEncoder with Time Changes.
Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
2021 IEEE 3RD GLOBAL CONFERENCE ON LIFE SCIENCES AND TECHNOLOGIES (IEEE LIFETECH 2021), 67, 68, IEEE, 2021年
英語, 研究論文（国際会議プロシーディングス）, A human emotion estimation method via feature integration using multi-modal variational autoencoder (MVAE) with time changes is presented in this paper. To utilize multi-modal information such as gaze and brain activity data including some noises, the proposed method newly introduces MVAE into the human emotion estimation. Furthermore, the proposed MVAE can consider the changes in bio-signals with time and reduce the effect of noises caused in bio-signals by using the probabilistic variation. Experimental results with that of some state-of-the-art methods indicate that the proposed method is effective.

メタデータを考慮した楽曲グラフ探索に基づく強化学習を用いた楽曲プレイリスト生成に関する検討
櫻井慶悟, 藤後廉, 小川貴弘, 長谷山美紀
電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2021, 53, 54, IEEE, 2021年
英語, Spreading of music streaming platforms that use playlists to make recommendations, automatic playlist generation has been actively researched. Recently, it has been reported that playlists that have high diversity and smooth track transitions increase user satisfaction. Our previous method that used a two-dimensional space as a reinforcement learning environment has achieved these demands, but there remains the problem that the content of multi-dimensional acoustic features cannot be retained accurately. To solve this problem, in this paper, we present a new method of music playlist generation based on reinforcement learning using a graph structure constructed from multi-dimensional acoustic features directly. The new playlist generation provides greater diversity and smoother track transitions than the previous method. Experimental results are shown for verifying the effectiveness of the proposal method.

Question Answering from Brain Activity Data via Decoder Based on Neural Networks.
Saya Takada, Ren Togo, Takahiro Ogawa, Miki Haseyama
2021 IEEE 3RD GLOBAL CONFERENCE ON LIFE SCIENCES AND TECHNOLOGIES (IEEE LIFETECH 2021), 51, 52, IEEE, 2021年
英語, 研究論文（国際会議プロシーディングス）, We build a model that can estimate what subjects recognize from functional magnetic resonance imaging (fMRI) data via a visual question answering (VQA) model. The VQA model can generate an answer to a question about an image. We convert fMRI signals into image features via an fMRI decoder based on the relationship between the fMRI signals and the image features extracted from the gazed image. Then this allows the VQA model to answer a visual question from the fMRI signals measured while the subject is gazing at the image. Though brain decoding, which interprets what humans recognize, has become overwhelmingly popular in neuroscience, they often suffer from the small datasets of brain activity data. To overcome the small size of datasets of fMRI signals, we introduce an fMRI decoder based on neural networks that have a high expressive ability. Even when we do not have enough fMRI signals, the proposed method derives the answer to what a person is looking at from fMRI signals. Experimental results on several datasets show that our method allows us to answer a question about gazed images from fMRI signals.

Cross-Domain Semi-Supervised Deep Metric Learning for Image Sentiment Analysis.
Yun Liang 0014, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP), 4150, 4154, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Feature Integration via Semi-Supervised Ordinally Multi-Modal Gaussian Process Latent Variable Model.
Kyohei Kamikawa, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP), 4130, 4134, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Multi-Modal Label Dequantized Gaussian Process Latent Variable Model for Ordinal Label Estimation.
Masanao Matsumoto, Keisuke Maeda, Naoki Saito 0006, Takahiro Ogawa, Miki Haseyama
IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP), 3985, 3989, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Semantic-Aware Unpaired Image-to-Image Translation for Urban Scene Images.
Zongyao Li, Ren Togo, Takahiro Ogawa, Miki Haseyama
IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP), 2150, 2154, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Classification of Expert-Novice Level Using Eye Tracking And Motion Data via Conditional Multimodal Variational Autoencoder.
Yusuke Akamatsu, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP), 1360, 1364, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Estimation of Visual Features of Viewed Image From Individual and Shared Brain Information Based on FMRI Data Using Probabilistic Generative Model.
Takaaki Higashi, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP), 1335, 1339, IEEE, 2021年
英語, 研究論文（国際会議プロシーディングス）, This paper presents a method for estimation of visual features based on brain responses measured when subjects view images. The proposed method estimates visual features of viewed images by using both individual and shared brain information from functional magnetic resonance imaging (fMRI) data when subjects view images. To extract an effective latent space shared by multiple subjects from high dimensional fMRI data, a probabilistic generative model that can provide a prior distribution to the space is introduced into the proposed method. Also, the extraction of a robust feature space with respect to noise for the individual information becomes feasible via the proposed probabilistic generative model. This is the first contribution of our method. Furthermore, the proposed method constructs a decoder transforming brain information into visual features based on collaborative use of both estimated spaces for individual and shared brain information. This is the second contribution of our method. Experimental results show that the proposed method improves the estimation accuracy of the visual features of viewed images.

Human-Centered Favorite Music Classification Using EEG-Based Individual Music Preference Via Deep Time-Series CCA.
Ryosuke Sawata, Takahiro Ogawa, Miki Haseyama
IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP), 1320, 1324, IEEE, 2021年
研究論文（国際会議プロシーディングス）

Rubber Material Property Prediction Using Electron Microscope Images of Internal Structures Taken under Multiple Conditions.
Ren Togo, Naoki Saito 0006, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
Sensors, 21, 6, 2088, 2088, MDPI, 2021年
英語, 研究論文（学術雑誌）, A method for prediction of properties of rubber materials utilizing electron microscope images of internal structures taken under multiple conditions is presented in this paper. Electron microscope images of rubber materials are taken under several conditions, and effective conditions for the prediction of properties are different for each rubber material. Novel approaches for the selection and integration of reliable prediction results are used in the proposed method. The proposed method enables selection of reliable results based on prediction intervals that can be derived by the predictors that are each constructed from electron microscope images taken under each condition. By monitoring the relationship between prediction results and prediction intervals derived from the corresponding predictors, it can be determined whether the target prediction results are reliable. Furthermore, the proposed method integrates the selected reliable results based on Dempster-Shafer (DS) evidence theory, and this integration result is regarded as a final prediction result. The DS evidence theory enables integration of multiple prediction results, even if the results are obtained from different imaging conditions. This means that integration can even be realized if electron microscope images of each material are taken under different conditions and even if these conditions are different for target materials. This nonconventional approach is suitable for our application, i.e., property prediction. Experiments on rubber material data showed that the evaluation index mean absolute percent error (MAPE) was under 10% by the proposed method. The performance of the proposed method outperformed conventional comparative property estimation methods. Consequently, the proposed method can realize accurate prediction of the properties with consideration of the characteristic of electron microscope images described above.

Detection of Important Scenes in Baseball Videos via a Time-Lag-Aware Multimodal Variational Autoencoder.
Kaito Hirasawa, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
Sensors, 21, 6, 2045, 2045, MDPI, 2021年
英語, 研究論文（学術雑誌）, A new method for the detection of important scenes in baseball videos via a time-lag-aware multimodal variational autoencoder (Tl-MVAE) is presented in this paper. Tl-MVAE estimates latent features calculated from tweet, video, and audio features extracted from tweets and videos. Then, important scenes are detected by estimating the probability of the scene being important from estimated latent features. It should be noted that there exist time-lags between tweets posted by users and videos. To consider the time-lags between tweet features and other features calculated from corresponding multiple previous events, the feature transformation based on feature correlation considering such time-lags is newly introduced to the encoder in MVAE in the proposed method. This is the biggest contribution of the Tl-MVAE. Experimental results obtained from actual baseball videos and their corresponding tweets show the effectiveness of the proposed method.

Deterioration level estimation via neural network maximizing category-based ordinally supervised multi-view canonical correlation.
Keisuke Maeda, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
Multim. Tools Appl., 80, 15, 23091, 23112, SPRINGER, 2021年
英語, 研究論文（学術雑誌）, A deterioration level estimation method via neural network maximizing category-based ordinally supervised multi-view canonical correlation is presented in this paper. This paper focuses on real world data such as industrial applications and has two contributions. First, a novel neural network handling multi-modal features transforms original features into features effectively representing deterioration levels in transmission towers, which are one of the infrastructures, with consideration of only correlation maximization. It can be realized by setting projection matrices maximizing correlations between multiple features into weights of hidden layers. That is, since the proposed network has only a few hidden layers, it can be trained from a small amount of training data. Second, since there exist diverse characteristics and an ordinal scale in deterioration levels, the proposed method newly derives category-based ordinally supervised multi-view canonical correlation analysis (Co-sMVCCA). Co-sMVCCA enables estimation of effective projection considering both within-class divergence and the ordinal scale between classes. Experimental results showed that the proposed method realizes accurate deterioration level estimation.

Disentangled Representation Learning in Real-World Image Datasets via Image Segmentation Prior.
Nao Nakagawa, Ren Togo, Takahiro Ogawa, Miki Haseyama
IEEE Access, 9, 110880, 110888, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2021年
英語, 研究論文（学術雑誌）, We propose a novel method that can learn easy-to-interpret latent representations in real-world image datasets using a VAE-based model by splitting an image into several disjoint regions. Our method performs object-wise disentanglement by exploiting image segmentation and alpha compositing. With remarkable results obtained by unsupervised disentanglement methods for toy datasets, recent studies have tackled challenging disentanglement for real-world image datasets. However, these methods involve deviations from the standard VAE architecture, which has favorable disentanglement properties. Thus, for disentanglement in images of real-world image datasets with preservation of the VAE backbone, we designed an encoder and a decoder that embed an image into disjoint sets of latent variables corresponding to objects. The encoder includes a pre-trained image segmentation network, which allows our model to focus only on representation learning while adopting image segmentation as an inductive bias. Evaluations using real-world image datasets, CelebA and Stanford Cars, showed that our method achieves improved disentanglement and transferability.

Detection of Important Scenes in Baseball Videos via Bidirectional Time Lag Aware Deep Multiset Canonical Correlation Analysis.
Kaito Hirasawa, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
IEEE Access, 9, 84971, 84981, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2021年
英語, 研究論文（学術雑誌）, A novel method for detection of important scenes in baseball videos based on correlation maximization between heterogeneous modalities via bidirectional time lag aware deep multiset canonical correlation analysis (BiTl-dMCCA) is presented in this paper. The proposed method enables detection of important scenes by collaboratively using baseball videos and their corresponding tweets. The technical contributions of this paper are twofold. First, since there are time lags between not only "tweets and corresponding multiple previous events" but also "events and corresponding multiple following posted tweets", the proposed method considers these bidirectional time lags. Specifically, the representation of such bidirectional time lags into the derivation of their covariance matrices is newly introduced. Second, the proposed method adopts textual, visual and audio features calculated from tweets and videos as multi-modal time series features. Important scenes are detected as abnormal scenes via anomaly detection based on a generative adversarial network using multi-modal features projected by BiTl-dMCCA. The proposed method does not need any training data with annotation. Experimental results obtained by applying the proposed method to actual baseball matches show the effectiveness of the proposed method.

Distress Image Retrieval for Infrastructure Maintenance via Self-Trained Deep Metric Learning Using Experts' Knowledge.
Naoki Ogawa, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
IEEE Access, 9, 65234, 65245, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2021年
英語, 研究論文（学術雑誌）, Distress image retrieval for infrastructure maintenance via self-trained deep metric learning using experts' knowledge is proposed in this paper. Since engineers take multiple images of a single distress part for inspection of road structures, it is necessary to construct a similar distress image retrieval method considering the input of multiple images to support determination of the level of deterioration. Thus, the construction of an image retrieval method while selecting an effective input from multiple images is described in this paper. The proposed method performs deep metric learning by using a small number of effective images labeled by experts' knowledge with information about their effectiveness and a large number of unlabeled images via a self-training approach. Specifically, an end-to-end learning approach that performs retraining of the model by assigning pseudo-labels to these unlabeled images according to the output confidence of the model is achieved. Thus, the proposed method can select an effective image from multiple images that are input at the retrieval as a query image. This is the main contribution of this paper. As a result, the proposed method realizes highly accurate retrieval of similar distress images considering the actual situation of inspection in which multiple images of a distress part are input.

Text-Guided Style Transfer-Based Image Manipulation Using Multimodal Generative Models.
Ren Togo, Megumi Kotera, Takahiro Ogawa, Miki Haseyama
IEEE Access, 9, 64860, 64870, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2021年
英語, 研究論文（学術雑誌）, A new style transfer-based image manipulation framework combining generative networks and style transfer networks is presented in this paper. Unlike conventional style transfer tasks, we tackle a new task, text-guided image manipulation. We realize style transfer-based image manipulation that does not require any reference style images and generate a style image from the user's input sentence. In our method, since an initial reference input sentence for a content image can automatically be given by an image-to-text model, the user only needs to update the reference sentence. This scheme can help users when they do not have any images representing the desired style. Although this text-guided image manipulation is a new challenging task, quantitative and qualitative comparisons showed the superiority of our method.

Perceived Image Decoding From Brain Activity Using Shared Information of Multi-Subject fMRI Data.
Yusuke Akamatsu, Ryosuke Harakawa, Takahiro Ogawa, Miki Haseyama
IEEE Access, 9, 26593, 26606, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2021年
英語, 研究論文（学術雑誌）, Decoding a person's cognitive contents from evoked brain activity is becoming important in the field of brain-computer interaction. Previous studies have decoded a perceived image from functional magnetic resonance imaging (fMRI) activity by constructing brain decoding models that were trained with a single subject's fMRI data. However, accurate decoding is still challenging since fMRI data acquired from only a single subject have several disadvantageous characteristics such as small sample size, noisy nature, and high dimensionality. In this article, we propose a method to decode categories of perceived images from fMRI activity using shared information of multi-subject fMRI data. Specifically, by aggregating fMRI data of multiple subjects that contain a large number of samples, we extract a low-dimensional latent representation shared by multi-subject fMRI data. Then the latent representation is nonlinearly transformed into visual features and semantic features of the perceived images to identify categories from various candidate categories. Our approach leverages rich information obtained from multi-subject fMRI data and improves the decoding performance. Experimental results obtained by using two public fMRI datasets showed that the proposed method can more accurately decode categories of perceived images from fMRI activity than previous approaches using a single subject's fMRI data.

Supervised Fractional-Order Embedding Multiview Canonical Correlation Analysis via Ordinal Label Dequantization for Image Interest Estimation.
Masanao Matsumoto, Naoki Saito 0006, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
IEEE Access, 9, 21810, 21822, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2021年
英語, 研究論文（学術雑誌）, Supervised fractional-order embedding multiview canonical correlation analysis via ordinal label dequantization (SFEMCCA-OLD) for image interest estimation is presented in this paper. SFEMCCA-OLD is a CCA method that realizes accurate integration of features including low-dimensional ordinal label features. In general, since information is lost due to a limitation of the number of classes, i.e., the dimension of ordinal label information is smaller than those of other features, derivation of highly accurate integration of features is difficult. In SFEMCCA-OLD, the dimension of the ordinal label information can be increased by estimation of the canonical correlation between multiview features. We call this approach ordinal label dequantization. In addition, by introducing a fractional-order technique, our method can calculate optimal projections for noisy data such as real data. Experimental results show that the accuracy of SFEMCCA-OLD for image interest estimation is better than that of recent CCA-based methods.

Chronic atrophic gastritis detection with a convolutional neural network considering stomach regions
Misaki Kanai, Ren Togo, Takahiro Ogawa, Miki Haseyama
WORLD JOURNAL OF GASTROENTEROLOGY, 26, 25, 3650, 3659, BAISHIDENG PUBLISHING GROUP INC, 2020年07月
英語, 研究論文（学術雑誌）, BACKGROUND The risk of gastric cancer increases in patients withHelicobacter pylori-associated chronic atrophic gastritis (CAG). X-ray examination can evaluate the condition of the stomach, and it can be used for gastric cancer mass screening. However, skilled doctors for interpretation of X-ray examination are decreasing due to the diverse of inspections. AIM To evaluate the effectiveness of stomach regions that are automatically estimated by a deep learning-based model for CAG detection. METHODS We used 815 gastric X-ray images (GXIs) obtained from 815 subjects. The ground truth of this study was the diagnostic results in X-ray and endoscopic examinations. For a part of GXIs for training, the stomach regions are manually annotated. A model for automatic estimation of the stomach regions is trained with the GXIs. For the rest of them, the stomach regions are automatically estimated. Finally, a model for automatic CAG detection is trained with all GXIs for training. RESULTS In the case that the stomach regions were manually annotated for only 10 GXIs and 30 GXIs, the harmonic mean of sensitivity and specificity of CAG detection were 0.955 +/- 0.002 and 0.963 +/- 0.004, respectively. CONCLUSION By estimating stomach regions automatically, our method contributes to the reduction of the workload of manual annotation and the accurate detection of the CAG.

胃X線画像を用いたAIによるH.pylori感染識別と今後の展望　　　　　　　　　　　　　　　
藤後 廉, 小川 貴弘, 間部 克裕, 加藤 元嗣, 長谷山 美紀
日本消化器がん検診学会雑誌, 58, 2, 127, 127, (一社)日本消化器がん検診学会, 2020年03月
日本語

Multimodal Important Scene Detection in Far-view Soccer Videos Based on Single Deep Neural Architecture
Tomoki Haruyama, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
ITE TRANSACTIONS ON MEDIA TECHNOLOGY AND APPLICATIONS, 8, 2, 89, 99, INST IMAGE INFORMATION & TELEVISION ENGINEERS, 2020年
英語, 研究論文（学術雑誌）, The details of the matches of soccer can be estimated from visual and audio sequences, and they correspond to the occurrence of important scenes. Therefore, the use of these sequences is suitable for important scene detection. In this paper, a new multimodal method for important scene detection from visual and audio sequences in far-view soccer videos based on a single deep neural architecture is presented. A unique point of our method is that multiple classifiers can be realized by a single deep neural architecture that includes a Convolutional Neural Network-based feature extractor and a Support Vector Machine-based classifier. This approach provides a solution to the problem of not being able to simultaneously optimize different multiple deep neural architectures from a small amount of training data. Then we monitor confidence measures output from this architecture for the multimodal data and enable their integration to obtain the final classification result.

Important Scene Detection Based on Anomaly Detection using Long Short-Term Memory for Baseball Highlight Generation
Kaito Hirasawa, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
2020 IEEE INTERNATIONAL CONFERENCE ON CONSUMER ELECTRONICS - TAIWAN (ICCE-TAIWAN), 1, 2, IEEE, 2020年
英語, 研究論文（国際会議プロシーディングス）, This paper presents an important scene detection method based on anomaly detection using a Long Short-Term Memory (LSTM) for baseball highlight generation. In order to deal with multi-view time series features calculated from tweets and videos, we adopt an anomaly detection method using LSTM. LSTM which can maintain a long-term memory is effective for training such features. Introduction of LSTM into important scene detection of baseball videos is the biggest contribution of this paper. Experimental results show high detection performance by our method.

Interpretable Convolutional Neural Network Including Attribute Estimation for Image Classification
Kazaha Horii, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
ITE TRANSACTIONS ON MEDIA TECHNOLOGY AND APPLICATIONS, 8, 2, 111, 124, INST IMAGE INFORMATION & TELEVISION ENGINEERS, 2020年
英語, 研究論文（学術雑誌）, An interpretable convolutional neural network (CNN) including attribute estimation for image classification is presented in this paper. Although CNNs perform highly accurate image classification, the reason for the classification results obtained by the neural networks is not clear. In order to provide interpretation of CNNs, the proposed method estimates attributes, which explain elements of objects, in an intermediate layer of the network. This enables improvement of the interpretability of CNNs, and it is the main contribution of this paper. Furthermore, the proposed method uses the estimated attributes for image classification in order to enhance its accuracy. Consequently, the proposed method not only provides interpretation of CNNs but also realizes improvement in the performance of image classification.

Image Retrieval Based on Supervised Local Regression and Global Alignment with Relevance Feedback for Insect Identification
Keisuke Maeda, Susumu Genma, Takahiro Ogawa, Miki Haseyama
ITE TRANSACTIONS ON MEDIA TECHNOLOGY AND APPLICATIONS, 8, 3, 140, 150, INST IMAGE INFORMATION & TELEVISION ENGINEERS, 2020年
英語, 研究論文（学術雑誌）, A method for image retrieval based on supervised local regression and global alignment (sLRGA) with relevance feedback for insect identification is presented in this paper. Based on the novel sLRGA, which is an extended version of LRGA, the proposed method estimates ranking scores for image retrieval in such a way that the neighborhood structure of a feature space of the database can be optimally preserved with consideration of class information. This is the main contribution of this paper. By measuring the relevance between all of the images and the query image in the database, sLRGA realizes accurate image retrieval. Furthermore, when positive/negative labels to retrieved images are given by users, the proposed method can improve image retrieval performance considering the query relevance information via use of both relevance feedback and sLRGA. This is the second contribution of this paper. Experimental results show the effectiveness of the proposed method.

Estimation of Person-Specific Visual Attention via Selection of Similar Persons
Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
2020 IEEE INTERNATIONAL CONFERENCE ON CONSUMER ELECTRONICS - TAIWAN (ICCE-TAIWAN), 1, 2, IEEE, 2020年
英語, 研究論文（国際会議プロシーディングス）, This paper presents a method for estimation of person-specific visual attention based on estimated similar persons' visual attention. For improving the estimation performance of person-specific visual attention, the proposed method uses the dataset including the large number of images and corresponding gaze data of many persons not including the target person and trains an estimation model based on deep learning. By using the estimated visual attention of similar persons for the target image, the proposed method estimates the visual attention of the target person with the small amount of gaze data. Experimental results show that the proposed method is effective for estimation of person-specific visual attention.

A Method for Player Importance Prediction from Player Network Using Gaze Position Estimated by LSTM
Genki Suzuki, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
ITE TRANSACTIONS ON MEDIA TECHNOLOGY AND APPLICATIONS, 8, 3, 151, 160, INST IMAGE INFORMATION & TELEVISION ENGINEERS, 2020年
英語, 研究論文（学術雑誌）, A novel method for player importance prediction from a player network using gaze positions estimated by Long Short-Term Memory (LSTM) in soccer videos is presented in this paper. By newly using an estimation model of gaze positions trained by gaze tracking data of experienced persons, it is expected that the importance of each player can be predicted. First, we generate a player network by utilizing the estimated gaze positions and first-arrival regions representing players' connections, e.g., passes between players. The gaze positions are estimated by LSTM that is newly trained from the gaze tracking data of experienced persons. Second, the proposed method predicts the importance of each player by applying the Hypertext Induced Topic Selection (HITS) algorithm to the constructed network. Consequently, prediction of the importance of each player based on soccer tactic knowledge of experienced persons can be realized without constantly obtaining gaze tracking data.

An Estimation Method of Candidate Region for Superimposing Information Based on Gaze Tracking Data in Soccer Videos
Genki Suzuki, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
2020 IEEE INTERNATIONAL CONFERENCE ON CONSUMER ELECTRONICS - TAIWAN (ICCE-TAIWAN), 1, 2, IEEE, 2020年
英語, 研究論文（国際会議プロシーディングス）, A novel method estimating candidate regions for superimposing information in soccer videos based on gaze tracking data is presented in this paper. The proposed method generates a likelihood map based on visual attention regions based on the gaze tracking data and detection results of objects such as players and soccer goals in soccer videos. Candidate regions for superimposing information are estimated by using the likelihood map. Experimental results show that the proposed method realizes effective candidate region estimation.

Image Retrieval with Data Augmentation of Sentence Labels Based on Paraphrasing
Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama
2020 IEEE INTERNATIONAL CONFERENCE ON CONSUMER ELECTRONICS - TAIWAN (ICCE-TAIWAN), 1, 2, IEEE, 2020年
英語, 研究論文（国際会議プロシーディングス）, Text-based image retrieval is a fundamental study in the field of information retrieval. Recent text-based image retrieval methods employ deep neural networks (here-inafter referred to as deep neural TBIR) to retrieve a desired image from a sentence query and achieve the state-of-the-art performance in TBIR. To improve the retrieval performance of the deep neural TBIR method further, it is essential to prepare diverse sentence labels in training data. However, it takes a lot of effort to prepare diverse sentence labels in training data. To address this problem, we propose a novel deep neural TBIR method with data augmentation of the sentence labels in training data. Experimental results show the effectiveness of the proposed method.

Interactive re-ranking for cross-modal retrieval based on object-wise question answering.
Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama
MMAsia 2020: ACM Multimedia Asia(MMAsia), 37, 7, ACM, 2020年
研究論文（国際会議プロシーディングス）

Similar scene retrieval in soccer videos with weak annotations by multimodal use of bidirectional LSTM.
Tomoki Haruyama, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
MMAsia 2020: ACM Multimedia Asia(MMAsia), 27, 8, ACM, 2020年
研究論文（国際会議プロシーディングス）

Quantitative Analysis of Engineer's Skill Using Wearable Sensor Data while Inspecting Highway Bridge.
Genki Suzuki, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
2nd IEEE Global Conference on Life Sciences and Technologies(LifeTech), 111, 112, IEEE, 2020年
研究論文（国際会議プロシーディングス）

Estimating Viewed Images with Natural Language Question Answering from fMRI Data.
Saya Takada, Ren Togo, Takahiro Ogawa, Miki Haseyama
2nd IEEE Global Conference on Life Sciences and Technologies(LifeTech), 99, 100, IEEE, 2020年
研究論文（国際会議プロシーディングス）

Distress Level Classification of Road Infrastructures via CNN Generating Attention Map.
Naoki Ogawa, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
2nd IEEE Global Conference on Life Sciences and Technologies(LifeTech), 97, 98, IEEE, 2020年
研究論文（国際会議プロシーディングス）

Interest Estimation for Images Using Eye Gaze-based Visual and Text Features via DLPCCA.
Masanao Matsumoto, Naoki Saito 0006, Takahiro Ogawa, Miki Haseyama
2nd IEEE Global Conference on Life Sciences and Technologies(LifeTech), 3, 4, IEEE, 2020年
研究論文（国際会議プロシーディングス）

Mvgan Maximizing Time-Lag Aware Canonical Correlation for Baseball Highlight Generation.
Kaito Hirasawa, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
2020 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO WORKSHOPS (ICMEW), 1, 6, IEEE, 2020年
英語, 研究論文（国際会議プロシーディングス）, This paper presents multi-view unsupervised generative adversarial network maximizing time-lag aware canonical correlation (MvGAN) for baseball highlight generation. MvGAN has the following two contributions. First, MvGAN utilizes textual, visual and audio features calculated from tweets and videos as multi-view features. MvGAN which adopts these multi-view features is the effective work for highlight generation of baseball videos. Second, since there is a temporal difference between posted tweets and the corresponding events, MvGAN introduces a novel feature embedding scheme considering a time-lag between textual features and other features. Specifically, the proposed method newly derives the timelag aware canonical correlation maximization of these multi-view features. This is the biggest contribution of this paper. Furthermore, since MvGAN is an unsupervised method for highlight generation, a large amount of training data with annotation is not needed. Thus, the proposed method has high applicability to the real world.

Generation of Viewed Image Captions From Human Brain Activity Via Unsupervised Text Latent Space.
Saya Takada, Ren Togo, Takahiro Ogawa, Miki Haseyama
2020 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2521, 2525, IEEE, 2020年
英語, 研究論文（国際会議プロシーディングス）, Generation of human cognitive contents based on the analysis of functional magnetic resonance imaging (fMRI) data has been actively researched. Cognitive contents such as viewed images can be estimated by analyzing the relationship between fMRI data and semantic information of viewed images. In this paper, we propose a new method generating captions for viewed images from human brain activity via a novel robust regression scheme. Unlike conventional generation methods using image feature representations, the proposed method makes use of more semantic text feature representations, which are more suitable for the caption generation. We construct a text latent space with unlabeled images not used for the training, and the fMRI data are regressed to the text latent space. Besides, we newly make use of unlabeled images not used for the training phase to improve caption generation performance. Finally, the proposed method can generate captions from the fMRI data measured while subjects are viewing images. Experimental results show that the proposed method enables accurate caption generation for viewed images.

Multimodal Image-to-Image Translation for Generation of Gastritis Images.
Ren Togo, Takahiro Ogawa, Miki Haseyama
2020 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2466, 2470, IEEE, 2020年
英語, 研究論文（国際会議プロシーディングス）, We present a new multimodal image-to-image translation model for the generation of gastritis images using X-ray and blood inspection results. In clinical situations, clinicians estimate the prognosis of the target disease by considering multiple inspection results. Similarly, we take a multimodal approach in the task of gastric cancer risk prediction. Visual characteristics of the gastric X-ray image and blood index values are highly related in the evaluation of gastric cancer risk. If we can generate a prediction image from blood index values, it contributes to the clinicians' sophisticated and integrated diagnosis. Hence, we learn a model that can map non-gastritis images to gastritis images based on the blood index values. Although this is a challenging multimodal task in medical image analysis, experimental results showed the effectiveness of our model.

Image Retrieval With Lingual And Visual Paraphrasing Via Generative Models.
Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama
2020 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2431, 2435, IEEE, 2020年
英語, 研究論文（国際会議プロシーディングス）, A new approach that improves text-based image retrieval (hereinafter referred to as TBIR) performance is proposed in this paper. TBIR methods aim to retrieve a desired image related to a query text. Especially, recent TBIR methods allow us to retrieve images considering word relationships by using a sentence as a query. In these TBIR methods, it is necessary to uniquely identify a desired image from similar images using a single query sentence. However, the diverse expressive styles for a query sentence make it difficult to uniquely identify a desired image. In this paper, we propose a novel TBIR method with paraphrasing on multiple representation spaces. Specifically, by paraphrasing a query sentence on lingual and visual representation spaces, the proposed method can retrieve a desired image from various perspectives and then it can uniquely identify a desired image from similar images. Comprehensive experimental results show the effectiveness of the proposed method.

Variational Autoencoder Based Unsupervised Domain Adaptation For Semantic Segmentation.
Zongyao Li, Ren Togo, Takahiro Ogawa, Miki Haseyama
2020 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2426, 2430, IEEE, 2020年
英語, 研究論文（国際会議プロシーディングス）, Unsupervised domain adaptation, which transfers supervised knowledge from a labeled domain to an unlabeled domain, remains a tough problem in the field of computer vision, especially for semantic segmentation. Some methods inspired by adversarial learning and semi-supervised learning have been developed for unsupervised domain adaptation in semantic segmentation and achieved outstanding performances. In this paper, we propose a novel method for this task. Like adversarial learning-based methods using a discriminator to align the feature distributions from different domains, we employ a variational autoencoder to get to the same destination but in a non-adversarial manner. Since the two approaches are compatible, we also integrate an adversarial loss into our method. By further introducing pseudo labels, our method can achieve state-of-the-art performances on two benchmark adaptation scenarios, GTA5-to-CITYSCAPES and SYNTHIA-to-CITYSCAPES.

Important Scene Detection Of Baseball Videos Via Time-Lag Aware Deep Multiset Canonical Correlation Maximization.
Kaito Hirasawa, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
2020 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 1236, 1240, IEEE, 2020年
英語, 研究論文（国際会議プロシーディングス）, This paper presents a new important scene detection method of baseball videos based on correlation maximization between heterogeneous modalities via time-lag aware deep multiset canonical correlation analysis (Tl-dMCCA). The technical contributions of this paper are twofold. First, textual, visual and audio features calculated from tweets and videos are adopted as multi-view time series features. Since Tl-dMCCA which utilizes these features includes the unsupervised embedding scheme via deep networks, the proposed method can flexibly express the relationship between heterogeneous features. Second, since there is the time-lag between posted tweets and the corresponding multiple previous events, Tl-dMCCA considers the time-lag relationships between them. Specifically, we newly introduce the representation of such time-lags into the derivation of their covariance matrices. By considering time-lags via Tl-dMCCA, the proposed method correctly detects important scenes.

Soft-Label Anonymous Gastric X-Ray Image Distillation.
Guang Li, Ren Togo, Takahiro Ogawa, Miki Haseyama
2020 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 305, 309, IEEE, 2020年
英語, 研究論文（国際会議プロシーディングス）, This paper presents a soft-label anonymous gastric X-ray image distillation method based on a gradient descent approach. The sharing of medical data is demanded to construct high-accuracy computer-aided diagnosis (CAD) systems. However, the large size of the medical dataset and privacy protection are remaining problems in medical data sharing, which hindered the research of CAD systems. The idea of our distillation method is to extract the valid information of the medical dataset and generate a tiny distilled dataset that has a different data distribution. Different from model distillation, our method aims to find the optimal distilled images, distilled labels and the optimized learning rate. Experimental results show that the proposed method can not only effectively compress the medical dataset but also anonymize medical images to protect the patient's private information. The proposed approach can improve the efficiency and security of medical data sharing.

Estimation Of Visual Contents Based On Question Answering From Human Brain Activity.
Saya Takada, Ren Togo, Takahiro Ogawa, Miki Haseyama
2020 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 61, 65, IEEE, 2020年
英語, 研究論文（国際会議プロシーディングス）, We propose an estimation method for free-form Visual Question Answering (VQA) from human brain activity, brain decoding VQA. The task of VQA in the field of computer vision is generating an answer given an image and a question about its contents. The proposed method can realize answering arbitrary visual questions about images from brain activity measured by functional Magnetic Resonance Imaging (fMRI) while viewing the same images. We enable estimating various information from brain activity via a unique VQA model, which can realize a more detailed understanding of images and complex reasoning. In addition, we newly make use of un-labeled images not used in the training phase to improve the performance of the transformation, since fMRI datasets are generally small. The proposed method can answer a visual question from a little amount of fMRI data measured while subjects are viewing images.

Feature Integration Via Geometrical Supervised Multi-View Multi-Label Canonical Correlation For Incomplete Label Assignment.
Keisuke Maeda, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
2020 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 46, 50, IEEE, 2020年
英語, 研究論文（国際会議プロシーディングス）, This paper presents feature integration via geometrical supervised multi-view multi-label canonical correlation analysis (GSM2CCA) for incomplete label assignment. The problem of incomplete labels is frequently encountered in the multi-label classification problem where the training labels are obtained via crowd-sourcing. In such a situation, consideration of only the label correlation, which is the basic approach, is not suitable for improvement of representation ability of features. For dealing with the incomplete label assignment, GSM2CCA constructs effective feature embedding space providing the discriminant ability by introducing both the multi-label correlation and feature similarity of the original feature space into its objective function. Since novel integrated features with high discriminant ability can be calculated by our GSM2CCA, performance improvement of multi-label classification with the incomplete label assignment is realized. The main contribution of this paper is the realization of the effective feature integration via the adoption of the combination use of label similarity and locality preserving projection of heterogeneous features for solving the problem of the incomplete label assignment. The effectiveness of GSM2CCA by applying GSM2CCA-based feature integration to heterogeneous features calculated from various convolutional neural network models is verified via experimental results.

Unsupervised Domain Adaptation for Semantic Segmentation with Symmetric Adaptation Consistency.
Zongyao Li, Ren Togo, Takahiro Ogawa, Miki Haseyama
2020 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, 2263, 2267, IEEE, 2020年
英語, 研究論文（国際会議プロシーディングス）, Unsupervised domain adaptation, which leverages label information from other domains to solve tasks on a domain without any labels, can alleviate the problem of the scarcity of labels and expensive labeling costs faced by supervised semantic segmentation. In this paper, we utilize adversarial learning and semi-supervised learning simultaneously to solve the task of unsupervised domain adaptation in semantic segmentation. We propose a new approach that trains two segmentation models with the adversarial learning symmetrically and further introduces the consistency between the outputs of the two models into the semi-supervised learning to improve the accuracy of pseudo labels which significantly affect the final adaptation performance. We achieve state-of-the-art semantic segmentation performance on the GTA5-to-Cityscapes scenario, a widely used benchmark setting in unsupervised domain adaptation.

Multi-View Bayesian Generative Model for Multi-Subject FMRI Data on Brain Decoding of Viewed Image Categories.
Yusuke Akamatsu, Ryosuke Harakawa, Takahiro Ogawa, Miki Haseyama
2020 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, 1215, 1219, IEEE, 2020年
英語, 研究論文（国際会議プロシーディングス）, Brain decoding studies have demonstrated that viewed image categories can be estimated from human functional magnetic resonance imaging (fMRI) activity. However, there are still limitations with the estimation performance because of the characteristics of fMRI data and the employment of only one modality extracted from viewed images. In this paper, we propose a multi-view Bayesian generative model for multi-subject fMRI data to estimate viewed image categories from fMRI activity. The proposed method derives effective representations of fMRI activity by utilizing multi-subject fMRI data. In addition, we associate fMRI activity with multiple modalities, i:e:, visual features and semantic features extracted from viewed images. Experimental results show that the proposed method outperforms existing state-of-the-art methods of brain decoding.

Interest Level Estimation Based on Feature Integration Considering Distribution of Partially Paired User's Behavior, Videos and Posters.
Kyohei Kamikawa, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
9th IEEE Global Conference on Consumer Electronics(GCCE), 944, 945, IEEE, 2020年
研究論文（国際会議プロシーディングス）

Music Playlist Generation Based on Reinforcement Learning Using Acoustic Feature Map.
Keigo Sakurai, Ren Togo, Takahiro Ogawa, Miki Haseyama
9th IEEE Global Conference on Consumer Electronics(GCCE), 942, 943, IEEE, 2020年
研究論文（国際会議プロシーディングス）

Estimation of Images Matched with Audio-Induced Brain Activity via Modified DGCCA.
Yun Liang 0014, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
9th IEEE Global Conference on Consumer Electronics(GCCE), 940, 941, IEEE, 2020年
研究論文（国際会議プロシーディングス）

Estimation of User-Specific Visual Attention Considering Individual Tendency toward Gazed Objects.
Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
9th IEEE Global Conference on Consumer Electronics(GCCE), 745, 746, IEEE, 2020年
研究論文（国際会議プロシーディングス）

Estimation of Viewed Images Using Individual and Shared Brain Responses.
Takaaki Higashi, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
9th IEEE Global Conference on Consumer Electronics(GCCE), 716, 717, IEEE, 2020年
研究論文（国際会議プロシーディングス）

Cross-domain Recommendation via Multi-layer Graph Analysis Using User-item Embedding.
Taisei Hirakawa, Keisuke Maeda, Takahiro Ogawa, Satoshi Asamizu, Miki Haseyama
9th IEEE Global Conference on Consumer Electronics(GCCE), 714, 715, IEEE, 2020年
研究論文（国際会議プロシーディングス）

Question Answering for Estimation of Seen Image Contents from Multi-subject fMRI Responses.
Saya Takada, Ren Togo, Takahiro Ogawa, Miki Haseyama
9th IEEE Global Conference on Consumer Electronics(GCCE), 712, 713, IEEE, 2020年
研究論文（国際会議プロシーディングス）

Face Synthesis via User Manipulation of Disentangled Latent Representation.
Nao Nakagawa, Ren Togo, Takahiro Ogawa, Miki Haseyama
9th IEEE Global Conference on Consumer Electronics(GCCE), 692, 693, IEEE, 2020年
研究論文（国際会議プロシーディングス）

Complexity Evaluation of Medical Image Data for Classification Problem Based on Spectral Clustering.
Guang Li, Ren Togo, Takahiro Ogawa, Miki Haseyama
9th IEEE Global Conference on Consumer Electronics(GCCE), 667, 669, IEEE, 2020年
研究論文（国際会議プロシーディングス）

Important Scene Prediction of Baseball Videos Using Twitter and Video Analysis Based on LSTM.
Kaito Hirasawa, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
9th IEEE Global Conference on Consumer Electronics(GCCE), 636, 637, IEEE, 2020年
研究論文（国際会議プロシーディングス）

Brain Decoding of Viewed Image Categories via Semi-Supervised Multi-View Bayesian Generative Model.
Yusuke Akamatsu, Ryosuke Harakawa, Takahiro Ogawa, Miki Haseyama
IEEE Trans. Signal Process., 68, 5769, 5781, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2020年
英語, 研究論文（学術雑誌）, Brain decoding has shown that viewed image categories can be estimated from evoked functional magnetic resonance imaging (fMRI) activity. Recent studies attempted to estimate viewed image categories that were not used for training previously. Nevertheless, the estimation performance is limited since it is difficult to collect a large amount of fMRI data for training. This paper presents a method to accurately estimate viewed image categories not used for training via a semi-supervised multi-view Bayesian generative model. Our model focuses on the relationship between fMRI activity and multiple modalities, i.e., visual features extracted from viewed images and semantic features obtained from viewed image categories. Furthermore, in order to accurately estimate image categories not used for training, our semi-supervised framework incorporates visual and semantic features obtained from additional image categories in addition to image categories of training data. The estimation performance of the proposed model outperforms existing state-of-the-art models in the brain decoding field and achieves more than 95% identification accuracy. The results also have shown that the incorporation of additional image category information is remarkably effective when the number of training samples is small. Our semi-supervised framework is significant for the brain decoding field where brain activity patterns are insufficient but visual stimuli are sufficient.

Few-Shot Personalized Saliency Prediction Based on Adaptive Image Selection Considering Object and Visual Attention.
Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
Sensors, 20, 8, 2170, 2170, MDPI, 2020年
英語, 研究論文（学術雑誌）, A few-shot personalized saliency prediction based on adaptive image selection considering object and visual attention is presented in this paper. Since general methods predicting personalized saliency maps (PSMs) need a large number of training images, the establishment of a theory using a small number of training images is needed. To tackle this problem, although finding persons who have visual attention similar to that of a target person is effective, all persons have to commonly gaze at many images. Thus, it becomes difficult and unrealistic when considering their burden. On the other hand, this paper introduces a novel adaptive image selection (AIS) scheme that focuses on the relationship between human visual attention and objects in images. AIS focuses on both a diversity of objects in images and a variance of PSMs for the objects. Specifically, AIS selects images so that selected images have various kinds of objects to maintain their diversity. Moreover, AIS guarantees the high variance of PSMs for persons since it represents the regions that many persons commonly gaze at or do not gaze at. The proposed method enables selecting similar users from a small number of images by selecting images that have high diversities and variances. This is the technical contribution of this paper. Experimental results show the effectiveness of our personalized saliency prediction including the new image selection scheme.

Tensor-Based Emotional Category Classification via Visual Attention-Based Heterogeneous CNN Feature Fusion.
Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
Sensors, 20, 7, 2146, 2146, MDPI, 2020年
英語, 研究論文（学術雑誌）, The paper proposes a method of visual attention-based emotion classification through eye gaze analysis. Concretely, tensor-based emotional category classification via visual attention-based heterogeneous convolutional neural network (CNN) feature fusion is proposed. Based on the relationship between human emotions and changes in visual attention with time, the proposed method performs new gaze-based image representation that is suitable for reflecting the characteristics of the changes in visual attention with time. Furthermore, since emotions evoked in humans are closely related to objects in images, our method uses a CNN model to obtain CNN features that can represent their characteristics. For improving the representation ability to the emotional categories, we extract multiple CNN features from our novel gaze-based image representation and enable their fusion by constructing a novel tensor consisting of these CNN features. Thus, this tensor construction realizes the visual attention-based heterogeneous CNN feature fusion. This is the main contribution of this paper. Finally, by applying logistic tensor regression with general tensor discriminant analysis to the newly constructed tensor, the emotional category classification becomes feasible. Since experimental results show that the proposed method enables the emotional category classification with the F1-measure of approximately 0.6, and about 10% improvement can be realized compared to comparative methods including state-of-the-art methods, the effectiveness of the proposed method is verified.

Chronic gastritis classification using gastric X-ray images with a semi-supervised learning method based on tri-training.
Zongyao Li, Ren Togo, Takahiro Ogawa, Miki Haseyama
Medical Biol. Eng. Comput., 58, 6, 1239, 1250, SPRINGER HEIDELBERG, 2020年
英語, 研究論文（学術雑誌）, High-quality annotations for medical images are always costly and scarce. Many applications of deep learning in the field of medical image analysis face the problem of insufficient annotated data. In this paper, we present a semi-supervised learning method for chronic gastritis classification using gastric X-ray images. The proposed semi-supervised learning method based on tri-training can leverage unannotated data to boost the performance that is achieved with a small amount of annotated data. We utilize a novel learning method named Between-Class learning (BC learning) that can considerably enhance the performance of our semi-supervised learning method. As a result, our method can effectively learn from unannotated data and achieve high diagnostic accuracy for chronic gastritis.

Multi-Task Convolutional Neural Network Leading to High Performance and Interpretability via Attribute Estimation.
Keisuke Maeda, Kazaha Horii, Takahiro Ogawa, Miki Haseyama
IEICE Trans. Fundam. Electron. Commun. Comput. Sci., 103-A, 12, 1609, 1612, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2020年
英語, 研究論文（学術雑誌）, A multi-task convolutional neural network leading to high performance and interpretability via attribute estimation is presented in this letter. Our method can provide interpretation of the classification results of CNNs by outputting attributes that explain elements of objects as a judgement reason of CNNs in the middle layer. Furthermore, the proposed network uses the estimated attributes for the following prediction of classes. Consequently, construction of a novel multi-task CNN with improvements in both of the interpretability and classification performance is realized.

Inpainting via Sparse Representation Based on a Phaseless Quality Metric.
Takahiro Ogawa, Keisuke Maeda, Miki Haseyama
IEICE Trans. Fundam. Electron. Commun. Comput. Sci., 103-A, 12, 1541, 1551, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2020年
英語, 研究論文（学術雑誌）, An inpainting method via sparse representation based on a new phaseless quality metric is presented in this paper. Since power spectra, phaseless features, of local regions within images enable more successful representation of their texture characteristics compared to their pixel values, a new quality metric based on these phaseless features is newly derived for image representation. Specifically, the proposed method enables spare representation of target signals, i.e., target patches, including missing intensities by monitoring errors converged by phase retrieval as the novel phaseless quality metric. This is the main contribution of our study. In this approach, the phase retrieval algorithm used in our method has the following two important roles: (1) derivation of the new quality metric that can be derived even for images including missing intensities and (2) conversion of phaseless features, i.e., power spectra, to pixel values, i.e., intensities. Therefore, the above novel approach solves the existing problem of not being able to use better features or better quality metrics for inpainting. Results of experiments showed that the proposed method using sparse representation based on the new phaseless quality metric outperforms previously reported methods that directly use pixel values for inpainting.

Heterogeneous-Graph-Based Video Search Reranking Using Topic Relevance.
Soh Yoshida, Mitsuji Muneyasu, Takahiro Ogawa, Miki Haseyama
IEICE Trans. Fundam. Electron. Commun. Comput. Sci., 103-A, 12, 1529, 1540, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2020年
英語, 研究論文（学術雑誌）, In this paper, we address the problem of analyzing topics, included in a social video group, to improve the retrieval performance of videos. Unlike previous methods that focused on an individual visual aspect of videos, the proposed method aims to leverage the "mutual reinforcement" of heterogeneous modalities such as tags and users associated with video on the Internet. To represent multiple types of relationships between each heterogeneous modality, the proposed method constructs three subgraphs: user-tag, video-video, and video-tag graphs. We combine the three types of graphs to obtain a heterogeneous graph. Then the extraction of latent features, i.e., topics, becomes feasible by applying graph-based soft clustering to the heterogeneous graph. By estimating the membership of each grouped cluster for each video, the proposed method defines a new video similarity measure. Since the understanding of video content is enhanced by exploiting latent features obtained from different types of data that complement each other, the performance of visual reranking is improved by the proposed method. Results of experiments on a video dataset that consists of YouTube-8M videos show the effectiveness of the proposed method, which achieves a 24.3% improvement in terms of the mean normalized discounted cumulative gain in a search ranking task compared with the baseline method.

Deep convolutional neural network-based anomaly detection for organ classification in gastric X-ray examination.
Ren Togo, Haruna Watanabe, Takahiro Ogawa, Miki Haseyama
Comput. Biol. Medicine, 123, 103903, 103903, PERGAMON-ELSEVIER SCIENCE LTD, 2020年
英語, 研究論文（学術雑誌）, Aim: The aim of this study was to determine whether our deep convolutional neural network-based anomaly detection model can distinguish differences in esophagus images and stomach images obtained from gastric X-ray examinations.Methods: A total of 6012 subjects were analyzed as our study subjects. Since the number of esophagus X-ray images is much smaller than the number of gastric X-ray images taken in X-ray examinations, we took an anomaly detection approach to realize the task of organ classification. We constructed a deep autoencoding gaussian mixture model (DAGMM) with a convolutional autoencoder architecture. The trained model can produce an anomaly score for a given test X-ray image. For comparison, the original DAGMM, AnoGAN, and a One-Class Support Vector Machine (OCSVM) that were trained with features obtained by a pre-trained Inception-v3 network were used.Results: Sensitivity, specificity, and the calculated harmonic mean of the proposed method were 0.956, 0.980, and 0.968, respectively. Those of the original DAGMM were 0.932, 0.883, and 0.907, respectively. Those of AnoGAN were 0.835, 0.833, and 0.834, respectively, and those of OCSVM were 0.932, 0.935, and 0.934, respectively. Experimental results showed the effectiveness of the proposed method for an organ classification task.Conclusion: Our deep convolutional neural network-based anomaly detection model has shown the potential for clinical use in organ classification.

Human-Centric Emotion Estimation Based on Correlation Maximization Considering Changes With Time in Visual Attention and Brain Activity.
Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
IEEE Access, 8, 203358, 203368, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2020年
英語, 研究論文（学術雑誌）, A human-centric emotion estimation method based on correlation maximization with consideration of changes with time in visual attention and brain activity when viewing images is proposed in this paper. Owing to the recent developments of many kinds of biological sensors, many researchers have focused on multimodal emotion estimation using both eye gaze data and brain activity data for improving the quality of emotion estimation. In this paper, a novel method that focuses on the following two points is introduced. First, in order to reduce the burden on users, we obtain brain activity data from users only in the training phase by using a projection matrix calculated by canonical correlation analysis (CCA) between gaze-based visual features and brain activity-based features. Next, for considering the changes with time in both visual attention and brain activity, we obtain novel features based on CCA-based projection in each time unit. In order to include these two points, the proposed method analyzes a fourth-order gaze and image tensor for which modes are pixel location, color channel and the changes with time in visual attention. Moreover, in each time unit, the proposed method performs CCA between gaze-based visual features and brain activity-based features to realize human-centric emotion estimation with a high level of accuracy. Experimental results show that accurate human emotion estimation is achieved by using our new human-centric image representation.

Estimation of Interest Levels From Behavior Features via Tensor Completion Including Adaptive Similar User Selection.
Keisuke Maeda, Tetsuya Kushima, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
IEEE Access, 8, 126109, 126118, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2020年
英語, 研究論文（学術雑誌）, A method for estimating interest levels from behavior features via tensor completion including adaptive similar user selection is presented in this paper. The proposed method focuses on a tensor that is suitable for data containing multiple contexts and constructs a third-order tensor in which three modes are "products", "users" and "user behaviors and interest levels" for these products. By complementing this tensor, unknown interest level estimation of a product for a target user becomes feasible. For further improving the estimation performance, the proposed method adaptively selects similar users for the target user by focusing on converged estimation errors between estimated interest levels and known interest levels in the tensor completion. Furthermore, the proposed method can adaptively estimate the unknown interest from the similar users. This is the main contribution of this paper. Therefore, the influence of users having different interests is reduced, and accurate interest level estimation can be realized. In order to verify the effectiveness of the proposed method, we show experimental results obtained by estimating interest levels of users holding books.

Supervised Fractional-Order Embedding Geometrical Multi-View CCA (SFGMCCA) for Multiple Feature Integration.
Keisuke Maeda, Yoshiki Ito, Takahiro Ogawa, Miki Haseyama
IEEE Access, 8, 114340, 114353, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2020年
英語, 研究論文（学術雑誌）, Techniques for integrating different types of multiple features effectively have been actively studied in recent years. Multiset canonical correlation analysis (MCCA), which maximizes the sum of pairwise correlations of inter-view (i.e., between different features), is one of the powerful methods for integrating different types of multiple features, and various MCCA-based methods have been proposed. This work focuses on a supervised MCCA variant in order to construct a novel effective feature integration framework. In this paper, we newly propose supervised fractional-order embedding geometrical multi-view CCA (SFGMCCA). This method constructs not only the correlation structure but also two types of geometrical structures of intra-view (i.e., within each feature) and inter-view simultaneously, thereby realizing more precise feature integration. This method also supports the integration of small sample and high-dimensional data by using the fractional-order technique. We conducted experiments using four types of image datasets, i.e., MNIST, COIL-20, ETH-80 and CIFAR-10. Furthermore, we also performed an fMRI dataset containing brain signals to verify the robustness. As a result, it was confirmed that accuracy improvements using SFGMCCA were statistically significant at the significance level of 0.05 compared to those using conventional representative MCCA-based methods.

Enhancing Cross-Modal Retrieval Based on Modality-Specific and Embedding Spaces.
Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama
IEEE Access, 8, 96777, 96786, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2020年
英語, 研究論文（学術雑誌）, A new approach that drastically improves cross-modal retrieval performance in vision and language (hereinafter referred to as & x201C;vision and language retrieval & x201D;) is proposed in this paper. Vision and language retrieval takes data of one modality as a query to retrieve relevant data of another modality, and it enables flexible retrieval across different modalities. Most of the existing methods learn optimal embeddings of visual and lingual information to a single common representation space. However, we argue that the forced embedding optimization results in loss of key information for sentences and images. In this paper, we propose an effective utilization of representation spaces in a simple but robust vision and language retrieval method. The proposed method makes use of multiple individual representation spaces through text-to-image and image-to-text models. Experimental results showed that the proposed approach enhances the performance of existing methods that embed visual and lingual information to a single common representation space.

Context-Aware Network Analysis of Music Streaming Services for Popularity Estimation of Artists.
Yui Matsumoto, Ryosuke Harakawa, Takahiro Ogawa, Miki Haseyama
IEEE Access, 8, 48673, 48685, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2020年, ［査読有り］
英語, 研究論文（学術雑誌）, A novel trial for estimating popularity of artists in music streaming services (MSS) is presented in this paper. The main contribution of this paper is to improve extensibility for using multi-modal features to accurately analyze latent relationships between artists. In the proposed method, a novel framework to construct a network is derived by collaboratively using social metadata and multi-modal features via canonical correlation analysis. Different from conventional methods that do not use multi-modal features, the proposed method can construct a network that can capture social metadata and multi-modal features, i.e., a context-aware network. For effectively analyzing the context-aware network, a novel framework to realize popularity estimation of artists is developed based on network analysis. The proposed method enables effective utilization of the network structure by extracting node features via a node embedding algorithm. By constructing an estimator that can distinguish differences between the node features, the proposed method can archive accurate popularity estimation of artists. Experimental results using multiple real-world datasets that contain artists in various genres in Spotify, one of the largest MSS, are presented. Quantitative and qualitative evaluations show that our method is effective for both classifying and regressing the popularity.

Retrieval of similar scenes based on multimodal distance metric learning in soccer videos
Tomoki Haruyama, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
MMSports 2019 - Proceedings of the 2nd International Workshop on Multimedia Content Analysis in Sports, co-located with MM 2019, 10, 15, ACM, 2019年10月15日, ［査読有り］
研究論文（国際会議プロシーディングス）, © 2019 Association for Computing Machinery. This paper presents a new method for retrieval of similar scenes based on multimodal distance metric learning in far-view soccer videos that broadly capture soccer fields and are not edited. We extract visual features and audio features from soccer video clips, and we extract text features from text data corresponding to these soccer video clips. In addition, distance metric learning based on Laplacian Regularized Metric Learning is performed to calculate the distances for each kind of features. Finally, by determining the final rank by integrating these distances, we realize successful multimodal retrieval of similar scenes from query scenes of soccer video clips. Experimental results show the effectiveness of our retrieval method.

Semantic Shot Classification in Baseball Videos Based on Similarities of Visual Features
K. Hirasawa, K. Maeda, T. Ogawa, M. Haseyama
IEEE Global Conference on Consumer Electronics (GCCE), 663, 664, IEEE, 2019年10月, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）

Region-based Distress Classification of Road Infrastructures via CNN Without Region Annotation
N. Ogawa, K. Maeda, T. Ogawa, M. Haseyama
IEEE Global Conference on Consumer Electronics (GCCE), 764, 765, IEEE, 2019年10月, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）

Aesthetic style transfer through text-to-image synthesis and image-to-image translation
Megumi Kotera, Ren Togo, Takahiro Ogawa, Miki Haseyama
IEEE Global Conference on Consumer Electronics (GCCE), 492, 493, IEEE, 2019年10月, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）

Voice-input multimedia information retrieval system based on text-to-image GAN
Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama
IEEE Global Conference on Consumer Electronics (GCCE), 943, 944, IEEE, 2019年10月, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）

Estimation of drilling energy from tunnel cutting face image based on online learning
Kentaro Yamamoto, Ren Togo, Takahiro Ogawa, Miki Haseyama
IEEE Global Conference on Consumer Electronics (GCCE), 77, 1, 794, 795, IEEE, 2019年10月, ［査読有り］
英語

Detection of distress region from subway tunnel images via U-net-based deep semantic segmentation
An Wang, Ren Togo, Takahiro Ogawa, Miki Haseyama
IEEE Global Conference on Consumer Electronics (GCCE), 766, 767, IEEE, 2019年10月, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）

Estimation of Emotion Labels via Tensor-Based Spatiotemporal Visual Attention Analysis
Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
Proceedings - International Conference on Image Processing, ICIP, 2019-September, 4105, 4109, IEEE, 2019年09月, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, © 2019 IEEE. This paper presents emotion label estimation via tensor-based spatiotemporal visual attention analysis. It has been reported in the fields of psychology and neuroscience that human emotions are related to two elements, their visual attention change and objects included in a target image. Therefore, the proposed method focuses on the spatiotemporal change of visual attention of human gazing at objects in the target image and constructs two neural networks which enable the emotion label estimation considering both of the above two elements. Specifically, the proposed method newly constructs a fourth-order tensor, gaze and image tensor (GIT) whose modes correspond to the width, the height and the color channel of the target image and the time axis of visual attention which is used for representing the time change. Then the first network, which consists of general tensor discriminant analysis (GTDA) and extreme learning machine (ELM), estimates the emotion label from the fourth-order GIT with concerning their visual attention change. Furthermore, the second network, which consists of pre-trained convolutoinal neural network-based feature extraction, GTDA and ELM, enables the estimation from the second-order GIT including visual features obtained from objects focused at each time. Finally, the proposed method estimates emotion labels based on decision fusion of the outputs from the two networks. Experimental results show the effectiveness of the proposed method.

Scene Retrieval for Video Summarization Based on Text-to-Image gan
Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama
Proceedings - International Conference on Image Processing, ICIP, 2019-September, 1825, 1829, IEEE, 2019年09月, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, © 2019 IEEE. We present a new scene retrieval method based on text-to-image Generative Adversarial Network (GAN) and its application to query-based video summarization. Text-to-image GAN is a deep learning method that can generate images from their corresponding sentences. In this paper, we reveal a characteristic that deep learning-based visual features extracted from images generated by text-to-image GAN include semantic information sufficiently. By utilizing the generated images as queries, the proposed method achieves higher scene retrieval performance than those of the stateof-the-art methods. In addition, we introduce a novel architecture that can consider order relationship of the input sentences to our method for realizing a target video summarization. Specifically, the proposed method generates multiple images thorough text-to-image GAN from multiple sentences summarizing target videos. Their summarized video can be obtained by performing the retrieval of corresponding scenes from the target videos according to the generated images with considering the order relationship. Experimental results show the effectiveness of the proposed method in the retrieval and summarization performance.

Gastritis Detection from Gastric X-Ray Images Via Fine-Tuning of Patch-Based Deep Convolutional Neural Network
Misaki Kanai, Ren Togo, Takahiro Ogawa, Miki Haseyama
Proceedings - International Conference on Image Processing, ICIP, 2019-September, 1371, 1375, IEEE, 2019年09月, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, © 2019 IEEE. This paper presents a method for gastritis detection from gastric X-ray images via fine-tuning techniques using a deep convolutional neural network (DCNN). DCNNs can learn parameters to capture high-dimensional features which express semantic contents of images by training on a large number of labeled images. However, lack of gastric X-ray images for training often occurs. To realize accurate detection with a small number of gastric X-ray images, the proposed method adopts fine-tuning techniques and newly introduces simple annotation of stomach regions to gastric X-ray images used for training. The proposed method fine-tunes a pre-trained DCNN with patches and three kinds of patch-level class labels considering not only the image-level ground truth ('gastritis'/'non-gastritis') but also the regions of a stomach since the outside of the stomach is not related to the image-level ground truth. In the test phase, by estimating the patch-level class labels with the fine-tuned DCNN, the proposed method enables the image-level class label estimation which excludes the effect of the unnecessary regions. Experimental results show the effectiveness of the proposed method.

Neural Network Maximizing Ordinally Supervised Multi-View Canonical Correlation for Deterioration Level Estimation
Keisuke Maeda, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
Proceedings - International Conference on Image Processing, ICIP, 2019-September, 919, 923, IEEE, 2019年09月, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, © 2019 IEEE. This paper presents a neural network maximizing ordinally supervised multi-view canonical correlation for deterioration level estimation. The contributions of this paper are twofold. First, in order to calculate features representing deterioration levels on transmission towers, which is one of the infrastructures, a novel neural network handling multi-modal features is constructed from a small amount of training data. Specifically, in our method, effective transformation to features with high discriminant ability without using many hidden layers is realized by setting projection matrices maximizing correlation between multiple features into hidden layer's weights. Second, since there exists ordinal scale in deterioration levels, the proposed method newly derives ordinally supervised multi-view canonical correlation analysis (OsMVCCA). OsMVCCA enables estimation of the effective projection considering not only label information but also their ordinal scales. Experimental results show that the proposed method realizes accurate deterioration level estimation.

Convolutional sparse coding-based deep random vector functional link network for distress classification of road structures.
Keisuke Maeda, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
Comp.-Aided Civil and Infrastruct. Engineering, 34, 8, 654, 676, WILEY, 2019年08月, ［査読有り］
英語, 研究論文（学術雑誌）, © 2019 Computer-Aided Civil and Infrastructure Engineering This paper presents a convolutional sparse coding (CSC)-based deep random vector functional link network (CSDRN) for distress classification of road structures. The main contribution of this paper is the introduction of CSC into a feature extraction scheme in the distress classification. CSC can extract visual features representing characteristics of target images because it can successfully estimate optimal convolutional dictionary filters and sparse features as visual features by training from a small number of distress images. The optimal dictionaries trained from distress images have basic components of visual characteristics such as edge and line information of distress images. Furthermore, sparse feature maps estimated on the basis of the dictionaries represent both strength of the basic components and location information of regions having their components, and these maps can represent distress images. That is, sparse feature maps can extract key components from distress images that have diverse visual characteristics. Therefore, CSC-based feature extraction is effective for training from a limited number of distress images that have diverse visual characteristics. The construction of a novel neural network, CSDRN, by the use of a combination of CSC-based feature extraction and the DRN classifier, which can also be trained from a small dataset, is shown in this paper. Accurate distress classification is realized via the CSDRN.

Bilingual Lexicon Learning Using Tagged Images via Graph Trilateral Filter-based Feature Refinement
Yui Matsumoto, Shota Hamano, Ryosuke Harakawa, Takahiro Ogawa, Miki Haseyama
2019 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-TW 2019, 2019年05月
研究論文（国際会議プロシーディングス）, A novel method to realize bilingual lexicon learning (BLL) using tagged images is presented in this paper. Different from existing methods that require parallel corpora, the proposed method enables extraction of semantically similar words by utilizing not such corpora but tagged images on image sharing services. The main contribution of this paper is derivation of a novel framework to refine visual features of tagged images based on graph trilateral filter-based smoothing. This enables reduction of the influence of noisy tags that are irrelevant to contents of images. As a result, accurate BLL becomes feasible by nearest neighbor search using the refined visual features.

Convolutional Sparse Coding-based Anomalous Event Detection in Surveillance Videos
Masanao Matsumoto, Naoki Saito, Takahiro Ogawa, Miki Haseyama
2019 IEEE International Conference on Consumer Electronics - Taiwan (ICCE-TW), IEEE, 2019年05月
研究論文（国際会議プロシーディングス）

User-Specific Visual Attention Estimation Based on Visual Similarity and Spatial Information in Images　　　　　　　　　　　　　　　
Y. Moroto, K. Maeda, T. Ogawa, M. Haseyama
IEEE International Conference on Consumer Electronics – Taiwan (ICCE-TW), 479, 480, 2019年05月, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）

Estimating Viewed Image Categories from Human Brain Activity via Semi-supervised Fuzzy Discriminative Canonical Correlation Analysis.
Yusuke Akamatsu, Ryosuke Harakawa, Takahiro Ogawa, Miki Haseyama
IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2019, Brighton, United Kingdom, May 12-17, 2019, 2019-May, 1105, 1109, IEEE, 2019年05月, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, © 2019 IEEE. This paper presents a method to estimate viewed image categories from human brain activity via newly derived semi-supervised fuzzy discriminative canonical correlation analysis (Semi-FDCCA). The proposed method can estimate image categories from functional magnetic resonance imaging (fMRI) activity measured while subjects view images by making fMRI activity and visual features obtained from images comparable through Semi-FDCCA. To realize Semi-FDCCA, we first derive a new supervised CCA called FDCCA that can consider fuzzy class information based on image category similarities obtained from WordNet ontology. Second, we adopt SemiCCA that can utilize additional unpaired visual features in addition to pairs of fMRI activity and visual features in order to prevent overfitting to the limited pairs. Furthermore, Semi-FDCCA can be derived by combining FDCCA with SemiCCA. Experimental results show that Semi-FDCCA enables accurate estimation of viewed image categories.

Extraction of regions related to cardiac sarcoidosis in polar map images
Ren Togo, Takahiro Ogawa, Osamu Manabe, Kenji Hirata, Tohru Shiga, Miki Haseyama
2019 IEEE 1st Global Conference on Life Sciences and Technologies, LifeTech 2019, 237, 238, IEEE, 2019年03月, ［査読有り］
研究論文（国際会議プロシーディングス）, © 2019 IEEE. This paper presents a method for extracting important regions for deep learning models in the identification of cardiac sarcoidosis using polar map images. Although deep learning-based detection methods have widely studied, they are still often called black boxes. Since high reliability for provided results from computer-aided diagnosis systems is important toward clinical applications, this problem should be solved. In this paper, we try to visualize important regions for deep learning-based models for improvement of understanding to clinicians. We monitor the variance of confidence of a model constructed with a deep learning-based feature and define it as a contribution value toward the estimated label. We visualize important regions for models based on the contribution value.

Estimation of emotions evoked by images based on multiple gaze-based CNN features
Taiga Matsui, Naoki Saito, Takahiro Ogawa, Satoshi Asamizu, Miki Haseyama
2019 IEEE 1st Global Conference on Life Sciences and Technologies, LifeTech 2019, 194, 195, IEEE, 2019年03月, ［査読有り］
研究論文（国際会議プロシーディングス）, © 2019 IEEE. This paper presents a method for estimating emotions evoked by watching images based on multiple visual features considering relationship with gaze information. The proposed method obtains multiple visual features from multiple middle layers of a Convolutional Neural Network. Then the proposed method newly derives their gaze-based visual features maximizing correlation with gaze information by using Discriminative Locality Preserving Canonical Correlation Analysis. The final estimation result is calculated by integrating multiple estimation results obtained from these gaze-based visual features. Consequently, successful emotion estimation becomes feasible by using such multiple estimation results which correspond to different semantic levels of target images.

Estimation of users' interest levels using tensor completion with SemiCCA
Tetsuya Kushima, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
2019 IEEE 1st Global Conference on Life Sciences and Technologies, LifeTech 2019, 239, 240, IEEE, 2019年03月, ［査読有り］
研究論文（国際会議プロシーディングス）, © 2019 IEEE. This paper presents a new method for estimation of users' interest levels using tensor completion with SemiCCA. The proposed method extracts new features maximizing correlation between features calculated from partially paired users' behavior and contents with semi-supervised canonical correlation analysis (SemiCCA). By this approach, we can successfully use the contents that users have not viewed for the interest level estimation. Moreover, our method utilizes the tensor completion to estimate unknown interest levels. Consequently, in the proposed method, accurate estimation of interest levels using SemiCCA and the tensor completion is realized. Experimental results are shown to verify the effectiveness of the proposed method by using actual data.

Classification of subcellular protein patterns in human cells with transfer learning
Zongyao Li, Ren Togo, Takahiro Ogawa, Miki Haseyama
2019 IEEE 1st Global Conference on Life Sciences and Technologies, LifeTech 2019, 273, 274, IEEE, 2019年03月, ［査読有り］
研究論文（国際会議プロシーディングス）, © 2019 IEEE. In this paper, we present a deep learning method for classifying subcellular protein patterns in human cells. Our method is mainly based on transfer learning and utilizes a newly proposed loss function named focal loss to deal with the problem of severe class imbalance existing in the task. The performance of our method is evaluated by a MacroF1 score of total 28 classes, and the final MacroF1 score of our method is 0.706.

Estimation of visual attention via canonical correlation between visual and gaze-based features
Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
2019 IEEE 1st Global Conference on Life Sciences and Technologies, LifeTech 2019, 229, 230, IEEE, 2019年03月, ［査読有り］
研究論文（国際会議プロシーディングス）, © 2019 IEEE. This paper presents a method for estimating visual attention via canonical correlation between visual and gaze-based features. The proposed method estimates user-specific visual attention by comparing a test image with training images including their corresponding individual eye gaze data in a common space. Specifically, canonical correlation analysis can derive projections which enable comparison between visual and gaze-based features in the common space. Therefore, given the new test image, our method projects its visual features to the common space and can estimate visual attention. Experimental results show the effectiveness of the proposed method.

Chronic gastritis detection from gastric X-ray images via deep autoencoding Gaussian mixture models
Masanao Matsumoto, Naoki Saito, Takahiro Ogawa, Miki Haseyama
2019 IEEE 1st Global Conference on Life Sciences and Technologies, LifeTech 2019, 231, 232, IEEE, 2019年03月, ［査読有り］
研究論文（国際会議プロシーディングス）, © 2019 IEEE. This paper presents a detection method of chronic gastritis from gastric X-ray images. The conventional method cannot detect chronic gastritis accurately since the number of non-gastritis images is overwhelmingly larger than the number of gastritis images. To deal with this problem, the proposed method performs the detection of chronic gastritis by using Deep Autoencoding Gaussian Mixture Models (DAGMM) which is an anomaly detection approach. DAGMM enables construction of chronic gastritis detection model using only non-gastritis images. In addition, DAGMM is superior to conventional anomaly detection methods since the models of dimensionality reduction and density estimation can be learned simultaneously. Therefore, the proposed method realizes accurate detection of chronic gastritis by utilizing DAGMM.

Fine-tuning of pre-trained DCNN for gastritis detection from gastric X-ray images
Misaki Kanai, Ren Togo, Takahiro Ogawa, Miki Haseyama
2019 IEEE 1st Global Conference on Life Sciences and Technologies, LifeTech 2019, 196, 197, IEEE, 2019年03月, ［査読有り］
研究論文（国際会議プロシーディングス）, © 2019 IEEE. This paper presents a detection method of gastritis from gastric X-ray images using fine-tuning techniques. With the development of deep convolutional neural networks (DCNNs), DCNN-based methods have achieved more accurate performance than conventional machine learning methods using hand-crafted features in the field of medical image analysis. However, lack of training images often occurs in clinical situations even though DCNNs require a large amount of training images to avoid overfitting. Therefore, the proposed method aims to consider the clinical situations that a limited amount of the training images are available. By fine-tuning a DCNN pre-trained with a large amount of annotated natural images, we avoid overfitting and realize accurate detection of the gastritis with a small amount of the training images.

Bone metastatic tumor detection based on AnoGAN using CT images
Haruna Watanabe, Ren Togo, Takahiro Ogawa, Miki Haseyama
2019 IEEE 1st Global Conference on Life Sciences and Technologies, LifeTech 2019, 235, 236, IEEE, 2019年03月, ［査読有り］
研究論文（国際会議プロシーディングス）, © 2019 IEEE. In this paper, we propose a method to detect bone metastatic tumors using computed tomography (CT) images. Bone metastatic tumors spread from primary cancer to other organs, and they can cause severe pain. Therefore, it is important to detect metastatic tumors earlier in addition to primary cancer. However, since metastatic tumors are very small, and they emerge from unpredictable regions in the body, collecting metastatic tumor images is difficult compared to primary cancer. In such a case, it can be considered that the idea of anomaly detection is suitable. The proposed method based on a generative adversarial network model trains with only non-metastatic bone tumor images and detects bone metastatic tumor in an unsupervised manner. Then the anomaly score is defined for each test CT image. Experimental results show the anomaly scores between non-metastatic bone tumor images and metastatic bone tumor images are clearly different. The anomaly detection approach may be effective for the detection of bone metastatic tumors in CT images.

Semi-supervised discriminative CCA for estimating viewed image categories from fMRI data
Yusuke Akamatsu, Ryosuke Harakawa, Takahiro Ogawa, Miki Haseyama
2019 IEEE 1st Global Conference on Life Sciences and Technologies, LifeTech 2019, 233, 234, IEEE, 2019年03月, ［査読有り］
研究論文（国際会議プロシーディングス）, © 2019 IEEE. This paper presents a method that estimates viewed image categories from functional magnetic resonance imaging (fMRI) data via semi-supervised discriminative canonical correlation analysis (Semi-DCCA). We newly derive Semi-DCCA that enables direct comparison of fMRI data and visual features extracted from viewed images while taking into account the class information and additional visual features to avoid overfitting. The proposed method enables estimation of image categories from fMRI data measured when subjects view images by comparing fMRI data with visual features through Semi-DCCA. Experimental results show that Semi-DCCA can improve estimation performance of the viewed image categories.

Video classification based on user preferences with soft-bag multiple instance learning
Akira Toyoda, Takahiro Ogawa, Miki Haseyama
2019 IEEE 1st Global Conference on Life Sciences and Technologies, LifeTech 2019, 198, 199, IEEE, 2019年03月, ［査読有り］
研究論文（国際会議プロシーディングス）, © 2019 IEEE. This paper presents a method to classify videos based on user preferences with soft-bag multiple instance learning (MIL). Our method classifies videos that a user has watched into two classes (preferred and not-preferred) with two-modal features extracted from the videos and brain signals measured while the user is watching the videos. Our method splits videos and brain signals into fixed-length segments and computes features used for classification from only a fixed-number of segments selected based on the idea of soft-bag MIL. By using the features computed from the selected segments, our method makes it possible to classify videos in the case that some videos that a user prefers contain some scenes the user does not prefer, and vice versa. Our main contribution allows methods classifying videos based on user preferences to treat such a case unlike conventional methods.

The Friction Properties of Firebrat Scales
Yuji Hirai, Naoto Okuda, Naoki Saito, Takahiro Ogawa, Ryuichiro Machida, Shuhei Nomura, Masahiro Ohara, Miki Haseyama, Masatsugu Shimomura
BIOMIMETICS, 4, 1, MDPI, 2019年01月
英語, 研究論文（学術雑誌）, Friction is an important subject for sustainability due to problems that are associated with energy loss. In recent years, micro- and nanostructured surfaces have attracted much attention to reduce friction; however, suitable structures are still under consideration. Many functional surfaces are present in nature, such as the friction reduction surfaces of snake skins. In this study, we focused on firebrats, Thermobia domestica, which temporary live in narrow spaces, such as piled papers, so their body surface (integument) is frequently in contact with surrounding substrates. We speculate that, in addition to optical, cleaning effects, protection against desiccation and enemies, their body surface may be also adapted to reduce friction. To investigate the functional effects of the firebrat scales, firebrat surfaces were observed using a field-emission scanning electron microscope (FE-SEM) and a colloidal probe atomic force microscope (AFM). Results of surface observations by FE-SEM revealed that adult firebrats are entirely covered with scales, whose surfaces have microgroove structures. Scale groove wavelengths around the firebrat's head are almost uniform within a scale but they vary between scales. At the level of single scales, AFM friction force measurements revealed that the firebrat scale reduces friction by decreasing the contact area between scales and a colloidal probe. The heterogeneity of the scales' groove wavelengths suggests that it is difficult to fix the whole body on critical rough surfaces and may result in a "fail-safe" mechanism.

Multimodal Retrieval of Similar Soccer Videos Based on Optimal Combination of Multiple Distance Measures.
Tomoki Haruyama, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
IEEE 8th Global Conference on Consumer Electronics(GCCE), 665, 666, IEEE, 2019年
研究論文（国際会議プロシーディングス）

Similarity Calculation Based on Pass Regions in Soccer Videos.
Sho Takahashi, Marco Bertini, Alberto Del Bimbo, Miki Haseyama, Toru Hagiwara
IEEE 8th Global Conference on Consumer Electronics(GCCE), 515, 516, IEEE, 2019年
研究論文（国際会議プロシーディングス）

Interest Estimation for Images Based on Eye Gaze-based Visual and Text Features.
Masanao Matsumoto, Naoki Saito 0006, Takahiro Ogawa, Miki Haseyama
IEEE 8th Global Conference on Consumer Electronics(GCCE), 481, 482, IEEE, 2019年
研究論文（国際会議プロシーディングス）

Performance Prediction Method of Examinees Based on Matrix Completion.
Yutaka Yamada, Takahiro Ogawa, Miki Haseyama
IEEE 8th Global Conference on Consumer Electronics(GCCE), 229, 230, IEEE, 2019年, ［査読有り］
研究論文（国際会議プロシーディングス）

Estimating Viewed Image Categories from fMRI Activity via Multi-view Bayesian Generative Model.
Yusuke Akamatsu, Ryosuke Harakawa, Takahiro Ogawa, Miki Haseyama
IEEE 8th Global Conference on Consumer Electronics(GCCE), 127, 128, IEEE, 2019年, ［査読有り］
研究論文（国際会議プロシーディングス）

The Extraction of Individual Music Preference Based on Deep Time-series CCA.
Ryosuke Sawata, Takahiro Ogawa, Miki Haseyama
IEEE 8th Global Conference on Consumer Electronics(GCCE), 15, 16, IEEE, 2019年, ［査読有り］
研究論文（国際会議プロシーディングス）

Estimation of User-Specific Visual Attention Based on Gaze Information of Similar Users.
Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
IEEE Global Conference on Consumer Electronics (GCCE), 477, 478, IEEE, 2019年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）

Effectiveness Evaluation of Deep Features for Image Reconstruction from fMRI Signals.
Saya Takada, Ren Togo, Takahiro Ogawa, Miki Haseyama
IEEE Global Conference on Consumer Electronics (GCCE), 479, 480, IEEE, 2019年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）

Scene Retrieval Using Text-to-image GAN-based Visual Similarities and Image-to-text Model-based Textual Similarities.
Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama
IEEE Global Conference on Consumer Electronics (GCCE), 13, 14, IEEE, 2019年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）

Query is GAN: Scene Retrieval with Attentional Text-To-Image Generative Adversarial Network
Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama
IEEE Access, 7, 153183, 153193, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2019年, ［査読有り］
英語, 研究論文（学術雑誌）, © 2013 IEEE. Scene retrieval from input descriptions has been one of the most important applications with the increasing number of videos on the Web. However, this is still a challenging task since semantic gaps between features of texts and videos exist. In this paper, we try to solve this problem by utilizing a text-To-image Generative Adversarial Network (GAN), which has become one of the most attractive research topics in recent years. The text-To-image GAN is a deep learning model that can generate images from their corresponding descriptions. We propose a new retrieval framework, 'Query is GAN', based on the text-To-image GAN that drastically improves scene retrieval performance by simple procedures. Our novel idea makes use of images generated by the text-To-image GAN as queries for the scene retrieval task. In addition, unlike many studies on text-To-image GANs that mainly focused on the generation of high-quality images, we reveal that the generated images have reasonable visual features suitable for the queries even though they are not visually pleasant. We show the effectiveness of the proposed framework through experimental evaluation in which scene retrieval is performed from real video datasets.

Synthetic Gastritis Image Generation via Loss Function-Based Conditional PGGAN.
Array,Array,Miki Haseyama
IEEE Access, 7, 87448, 87457, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2019年, ［査読有り］
英語, 研究論文（学術雑誌）, © 2013 IEEE. In this paper, a novel synthetic gastritis image generation method based on a generative adversarial network (GAN) model is presented. Sharing medical image data is a crucial issue for realizing diagnostic supporting systems. However, it is still difficult for researchers to obtain medical image data since the data include individual information. Recently proposed GAN models can learn the distribution of training images without seeing real image data, and individual information can be completely anonymized by generated images. If generated images can be used as training images in medical image classification, promoting medical image analysis will become feasible. In this paper, we targeted gastritis, which is a risk factor for gastric cancer and can be diagnosed by gastric X-ray images. Instead of collecting a large amount of gastric X-ray image data, an image generation approach was adopted in our method. We newly propose loss function-based conditional progressive growing generative adversarial network (LC-PGGAN), a gastritis image generation method that can be used for a gastritis classification problem. The LC-PGGAN gradually learns the characteristics of gastritis in gastric X-ray images by adding new layers during the training step. Moreover, the LC-PGGAN employs loss function-based conditional adversarial learning so that generated images can be used as the gastritis classification task. We show that images generated by the LC-PGGAN are effective for gastritis classification using gastric X-ray images and have clinical characteristics of the target symptom.

Semi-Supervised Learning Based on Tri-Training for Gastritis Classification using Gastric X-ray Images.
Zongyao Li, Ren Togo, Takahiro Ogawa, Miki Haseyama
IEEE International Symposium on Circuits and Systems, ISCAS 2019, Sapporo, Japan, May 26-29, 2019, 1, 5, IEEE, 2019年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a method of semi-supervised learning based on tri-training for gastritis classification using gastric X-ray images. The proposed method is constructed based on the tri-training architecture, and the strategies of label smoothing regularization and random erasing augmentation are utilized in the method to enhance the performance. Although the task of gastritis classification is challenging, we report that the proposed semi-supervised learning method using only a small number of labeled data achieves 0.888 harmonic mean of sensitivity and specificity on test data composed of 615 patients.

Synthetic Image Generation for Gastritis Detection Based on Auxiliary Classifier Generative Adversarial Network.
Misaki Kanai, Ren Togo, Takahiro Ogawa, Miki Haseyama
IEEE International Symposium on Circuits and Systems, ISCAS 2019, Sapporo, Japan, May 26-29, 2019, 1, 5, IEEE, 2019年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, With the development of convolutional neural networks (CNNs), CNN-based methods for medical image analysis have achieved more accurate performance than conventional machine learning methods using hand-crafted features. Although these methods utilize a large number of training images and realize high performance, lack of the training images often occurs in medical image analysis due to several reasons. This paper presents a novel image generation method to construct a dataset for gastritis detection from gastric X-ray images. The proposed method effectively utilizes two kinds of training images (gastritis and non-gastritis images) to generate images of each domain by introducing label conditioning into a generative model. Experimental results using real-world gastric X-ray images show the effectiveness of the proposed method.

Text-to-Image GAN-Based Scene Retrieval and Re-Ranking Considering Word Importance.
Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama
IEEE Access, 7, 169920, 169930, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2019年, ［査読有り］
英語, 研究論文（学術雑誌）, In this paper, we propose a novel scene retrieval and re-ranking method based on a text-to-image Generative Adversarial Network (GAN). The proposed method generates an image from an input query sentence based on the text-to-image GAN and then retrieves a scene that is the most similar to the generated image. By utilizing the image generated from the input query sentence as a query, we can control semantic information of the query image at the text level. Furthermore, we introduce a novel interactive re-ranking scheme to our retrieval method. Specifically, users can consider the importance of each word within the first input query sentence. Then the proposed method re-generates the query image that reflects the word importance provided by users. By updating the generated query image based on the word importance, it becomes feasible for users to revise retrieval results through this re-ranking process. In experiments, we showed that our retrieval method including the re-ranking scheme outperforms recently proposed retrieval methods.

Estimating Regions of Deterioration in Electron Microscope Images of Rubber Materials via a Transfer Learning-Based Anomaly Detection Model.
Ren Togo, Naoki Saito 0006, Takahiro Ogawa, Miki Haseyama
IEEE Access, 7, 162395, 162404, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2019年, ［査読有り］
英語, 研究論文（学術雑誌）, A method for estimating regions of deterioration in electron microscope images of rubber materials is presented in this paper. Deterioration of rubber materials is caused by molecular cleavage, external force, and heat. An understanding of these characteristics is essential in the field of material science for the development of durable rubber materials. Rubber material deterioration can be observed by using on electron microscope but it requires much effort and specialized knowledge to find regions of deterioration. In this paper, we propose an automated deterioration region estimation method based on deep learning and anomaly detection techniques to support such material development. Our anomaly detection model, called Transfer Learning-based Deep Autoencoding Gaussian Mixture Model (TL-DAGMM), uses only normal regions for training since obtaining training data for regions of deterioration is difficult. TL-DAGMM makes use of extracted high representation features from a pre-trained deep learning model and can automatically learn the characteristics of normal rubber material regions. Regions of deterioration are estimated at the pixel level by calculated anomaly scores. Experiments on real rubber material electron microscope images demonstrated the effectiveness of our model.

Multi-feature Fusion Based on Supervised Multi-view Multi-label Canonical Correlation Projection.
Keisuke Maeda, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2019, Brighton, United Kingdom, May 12-17, 2019, 3936, 3940, IEEE, 2019年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents multi-feature fusion based on supervised multi view multi-label canonical correlation projection (sM2CP). The proposed method applies sM2CP-based feature fusion to multiple features obtained from various convolutional neural networks (CNNs) whose characteristics are different. Since new fused features with high representation ability can be obtained, performance improvement of multi-label classification is realized. Specifically, in order to tackle the multi-label problem, sM2CP introduces a label similarity information of label vectors into the objective function of supervised multi-view canonical correlation analysis. Thus, sM2CP can deal with complex label information such as multi-label annotation. The main contribution of this paper is the realization of feature fusion of multiple CNN features for the multi-label problem by introducing multi-label similarity information into the canonical correlation analysis-based feature fusion approach. Experimental results show the effectiveness of sM2CP, which enables effective fusion of multiple CNN features.

Team Tactics Estimation in Soccer Videos Based on a Deep Extreme Learning Machine and Characteristics of the Tactics.
Genki Suzuki, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
IEEE Access, 7, 153238, 153248, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2019年, ［査読有り］
英語, 研究論文（学術雑誌）, A novel method for estimating team tactics in soccer videos based on a Deep Extreme Learning Machine (DELM) and unique characteristics of tactics is presented in this paper. The proposed method estimates the tactics of each team from players formations and enables successful training from a limited amount of training data. Specifically, the estimation of tactics consists of two stages. First, by utilizing two DELMs corresponding to the two teams, the proposed method estimates the provisional tactics of each team. Second, the proposed method updates the team tactics based on unique characteristics of soccer tactics, the relationship between tactics of the two teams and information on ball possession. Consequently, since the proposed method estimates the team tactics that satisfy these characteristics, accurate estimation results can be obtained. In an experiment, the proposed method is applied to actual soccer videos to verify its effectiveness.

Interest Level Estimation Based on Tensor Completion via Feature Integration for Partially Paired User's Behavior and Videos.
Tetsuya Kushima, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
IEEE Access, 7, 148576, 148585, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2019年, ［査読有り］
英語, 研究論文（学術雑誌）, A novel method for interest level estimation based on tensor completion via feature integration for partially paired users' behavior and videos is presented in this paper. The proposed method defines a novel canonical correlation analysis (CCA) framework that is suitable for interest level estimation, which is a hybrid version of semi-supervised CCA (SemiCCA) and supervised locality preserving CCA (SLPCCA) called semi-supervised locality preserving CCA (S2LPCCA). For partially paired users' behavior and videos in actual shops and on the Internet, new integrated features that maximize the correlation between partially paired samples by the principal component analysis (PCA)-mixed CCA framework are calculated. Then videos that users have not watched can be used for the estimation of users' interest levels. Furthermore, local structures of partially paired samples in the same class are preserved for accurate estimation of interest levels. Tensor completion, which can be applied to three contexts, videos, users and "canonical features and interest levels," is used for estimation of interest levels. Consequently, the proposed method realizes accurate estimation of users' interest levels based on S2LPCCA and the tensor completion from partially paired training features of users' behavior and videos. Experimental results obtained by applying the proposed method to actual data show the effectiveness of the proposed method.

Consensus Clustering of Tweet Networks via Semantic and Sentiment Similarity Estimation.
Ryosuke Harakawa, Shoji Takimura, Takahiro Ogawa, Miki Haseyama, Masahiro Iwahashi
IEEE Access, 7, 116207, 116217, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2019年, ［査読有り］
英語, 研究論文（学術雑誌）, Although Twitter has become an important source of information, the number of accessible tweets is too large for users to easily find their desired information. To overcome this difficulty, a method for tweet clustering is proposed in this paper. Inspired by the reports that network representation is useful for multimedia content analysis including clustering, a network-based approach is employed. Specifically, a consensus clustering method for tweet networks that represent relationships among the tweets' semantics and sentiment are newly derived. The proposed method integrates multiple clustering results obtained by applying successful clustering methods to the tweet networks. By integrating complementary clustering results obtained based on semantic and sentiment features, the accurate clustering of tweets becomes feasible. The contribution of this work can be found in the utilization of the features, which differs from existing network-based consensus clustering methods that target only the network structure. Experimental results for a real-world Twitter dataset, which includes 65 553 tweets of 25 datasets, verify the effectiveness of the proposed method.

Music Video Recommendation Based on Link Prediction Considering Local and Global Structures of a Network.
Array,Array,Array,Miki Haseyama
IEEE Access, 7, 104155, 104167, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2019年, ［査読有り］
英語, 研究論文（学術雑誌）, A novel method for music video recommendation is presented in this paper. The contributions of this paper are two-fold. (i) The proposed method constructs a network, which not only represents relationships between music videos and users but also captures multi-modal features of music videos. This enables collaborative use of multi-modal features such as audio, visual, and textual features, and multiple social metadata that can represent relationships between music videos and users on video hosting services. (ii) A novel scheme for link prediction considering local and global structures of the network (LP-LGSN) is newly derived by fusing multiple link prediction scores based on both local and global structures. By using the LP-LGSN to predict the degrees to which users desire music videos, the proposed method can recommend users' desired music videos. The experimental results for a real-world dataset constructed from YouTube-8M show the effectiveness of the proposed method.

Scene Retrieval from Multiple Resolution Generated Images Based on Text-to-image GAN
Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama
2019 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS (ISCAS), 1, 5, IEEE, 2019年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, Text-to-image Generative Adversarial Network (GAN) is a deep learning model that generates an image from an input sentence. It is expressly attracting attentions because of its applicability of the generated images. However, many existing studies have still focused on generation of high-quality images, and there are few studies focusing on application of the generated images since text-to-image GANs still cannot produce visually pleasing images in the complicated tasks. In this paper, we apply a text-to-image GAN as a generator of query images for a scene retrieval task to show availability of the visually non-pleasant images. The proposed method utilizes a low-resolution generated image that focuses on a sentence and a high-resolution generated image that focuses on each word of the sentence to retrieve a desired scene. With this mechanism, the proposed method realizes a high-accuracy scene retrieval from a sentence input. Experimental results show the effectiveness of our method.

Cardiac sarcoidosis classification with deep convolutional neural network-based features using polar maps.
Ren Togo, Kenji Hirata, Osamu Manabe, Hiroshi Ohira, Ichizo Tsujino, Keiichi Magota, Takahiro Ogawa, Miki Haseyama, Tohru Shiga
Computers in biology and medicine, 104, 81, 86, PERGAMON-ELSEVIER SCIENCE LTD, 2019年01月, ［査読有り］, ［国際誌］
英語, 研究論文（学術雑誌）, AIMS: The aim of this study was to determine whether deep convolutional neural network (DCNN)-based features can represent the difference between cardiac sarcoidosis (CS) and non-CS using polar maps. METHODS: A total of 85 patients (33 CS patients and 52 non-CS patients) were analyzed as our study subjects. One radiologist reviewed PET/CT images and defined the left ventricle region for the construction of polar maps. We extracted high-level features from the polar maps through the Inception-v3 network and evaluated their effectiveness by applying them to a CS classification task. Then we introduced the ReliefF algorithm in our method. The standardized uptake value (SUV)-based classification method and the coefficient of variance (CoV)-based classification method were used as comparative methods. RESULTS: Sensitivity, specificity and the harmonic mean of sensitivity and specificity of our method with the ReliefF algorithm were 0.839, 0.870 and 0.854, respectively. Those of the SUVmax-based classification method were 0.468, 0.710 and 0.564, respectively, and those of the CoV-based classification method were 0.655, 0.750 and 0.699, respectively. CONCLUSION: The DCNN-based high-level features may be more effective than low-level features used in conventional quantitative analysis methods for CS classification.

Distress classification of class-imbalanced inspection data via correlation-maximizing weighted extreme learning machine
Keisuke Maeda, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
Advanced Engineering Informatics, 37, 79, 87, Elsevier Ltd, 2018年08月01日, ［査読有り］
英語, 研究論文（学術雑誌）, This paper presents distress classification of class-imbalanced inspection data via correlation-maximizing weighted extreme learning machine (CMWELM). For distress classification, it is necessary to extract semantic features that can effectively distinguish multiple kinds of distress from a small amount of class-imbalanced data. In recent machine learning techniques such as general deep learning methods, since effective feature transformation from visual features to semantic features can be realized by using multiple hidden layers, a large amount of training data are required. However, since the amount of training data of civil structures becomes small, it becomes difficult to perform successful transformation by using these multiple hidden layers. On the other hand, CMWELM consists of two hidden layers. The first hidden layer performs feature transformation, which can directly extract the semantic features from visual features, and the second hidden layer performs classification with solving the class-imbalanced problem. Specifically, in the first hidden layer, the feature transformation is realized by using projections obtained by maximizing the canonical correlation between visual and text features as weight parameters of the hidden layer without designing multiple hidden layers. Furthermore, the second hidden layer enables successful training of our classifier by using weighting factors concerning the class-imbalanced problem. Consequently, CMWELM realizes accurate distress classification from a small amount of class-imbalanced data.

Strategy to develop convolutional neural network-based classifier for diagnosis of whole-body FDG PET images
Keisuke Kawauchi, Kenji Hirata, Seiya Ichikawa, Osamu Manabe, Kentaro Kobayashi, Shiro Watanabe, Miki Haseyama, Takahiro Ogawa, Ren Togo, Tohru Shiga, Chietsugu Katoh
Society of Nuclear Medicine and Molecular Imaging Annual Meeting (SNMMI), 59, SOC NUCLEAR MEDICINE INC, 2018年06月, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）

Use of deep convolutional neural network-based features for detection of cardiac sarcoidosis from polar map
Ren Togo, Kenji Hirata, Osamu Manabe, Hiroshi Ohira, Ichizo Tsujino, Takahiro Ogawa, Miki Haseyama, Tohru Shiga
Society of Nuclear Medicine and Molecular Imaging Annual Meeting (SNMMI), 59, SOC NUCLEAR MEDICINE INC, 2018年06月, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）

Estimating the quality of fractal compressed images using lacunarity
Megumi Takezawa, Hirofumi Sanada, Takahiro Ogawa, Miki Haseyama
IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, E101A, 6, 900, 903, Institute of Electronics, Information and Communication, Engineers, IEICE, 2018年06月01日, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, In this paper, we propose a highly accurate method for estimating the quality of images compressed using fractal image compression. Using an iterated function system, fractal image compression compresses images by exploiting their self-similarity, thereby achieving high levels of performance
however, we cannot always use fractal image compression as a standard compression technique because some compressed images are of low quality. Generally, sufficient time is required for encoding and decoding an image before it can be determined whether the compressed image is of low quality or not. Therefore, in our previous study, we proposed a method to estimate the quality of images compressed using fractal image compression. Our previous method estimated the quality using image features of a given image without actually encoding and decoding the image, thereby providing an estimate rather quickly
however, estimation accuracy was not entirely sufficient. Therefore, in this paper, we extend our previously proposed method for improving estimation accuracy. Our improved method adopts a new image feature, namely lacunarity. Results of simulation showed that the proposed method achieves higher levels of accuracy than those of our previous method.

Graph-Based Video Search Reranking with Local and Global Consistency Analysis
Soh Yoshida, Takahiro Ogawa, Miki Haseyama, Mitsuji Muneyasu
IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, E101D, 5, 1430, 1440, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2018年05月, ［査読有り］
英語, 研究論文（学術雑誌）, Video reranking is an effective way for improving the retrieval performance of text-based video search engines. This paper proposes a graph-based Web video search reranking method with local and global consistency analysis. Generally, the graph-based reranking approach constructs a graph whose nodes and edges respectively correspond to videos and their pairwise similarities. A lot of reranking methods are built based on a scheme which regularizes the smoothness of pairwise relevance scores between adjacent nodes with regard to a user's query. However, since the overall consistency is measured by aggregating only the local consistency over each pair, errors in score estimation increase when noisy samples are included within query-relevant videos' neighbors. To deal with the noisy samples, the proposed method leverages the global consistency of the graph structure, which is different from the conventional methods. Specifically, in order to detect this consistency, the propose method introduces a spectral clustering algorithm which can detect video groups, in which videos have strong semantic correlation, on the graph. Furthermore, a new regularization term, which smooths ranking scores within the same group, is introduced to the reranking framework. Since the score regularization is performed by both local and global aspects simultaneously, the accurate score estimation becomes feasible. Experimental results obtained by applying the proposed method to a real-world video collection show its effectiveness.

映像情報メディア年報2017シリーズ(第8回)メディア工学の研究動向
長谷山 美紀, 河村 圭, 田良島 周平, 新井 啓之
映像情報メディア学会誌, Vol.72, No.2, pp.241-246, 2018年03月
日本語, 研究論文（学術雑誌）, 本稿では、(1) 拡張現実・仮想現実の研究動向、(2) 深層学習の研究動向と実用展開、(3) IoT とメディア処理、等、メディア工学分野の研究動向を紹介する。(1)については、デプスセンサやヘッドマウントディスプレイ等のデバイスの登場と性能向上を踏まえつつ、これらを活用した研究開発の進展と動向を紹介する。(2) については、加速度的に進む画像映像メディアへの深層学習技術の適用について、具体的に画像分類や物体検出、さらには画像検索から教師データの検討、実用展開に至るまでを解説する。(3)では、(2) を受けてメディア工学の分野でAI 技術と相乗効果を生み出すIoT 技術について解説する。

A Novel Framework for Estimating Viewer Interest by Unsupervised Multimodal Anomaly Detection
Yuma Sasaka, Takahiro Ogawa, Miki Haseyama
IEEE Access, 6, 8340, 8350, Institute of Electrical and Electronics Engineers Inc., 2018年02月09日, ［査読有り］
英語, 研究論文（学術雑誌）, A reliable method to estimate viewer interest is highly sought after for human-centered video information retrieval. A method that estimates viewer interest while users are watching Web videos is presented in this paper. The method uses a framework for anomaly detection based on collaborative use of facial expression and biological signals such as electroencephalogram (EEG) signals. To the best of our knowledge, there have been no studies that have taken into account two actual mechanisms of the behavior of users while they are watching Web videos. First, whereas most Web videos garner very little attention, a small number attract millions of views. Therefore, a framework for anomaly detection is newly applied to facial expression and EEG in order to model the imbalanced distribution of popularity. Second, since the number of Web videos that are labeled by users as interesting/not interesting is generally too small to estimate viewer interest by a supervised approach, the proposed method utilizes parametric techniques for anomaly detection, which estimates viewer interest in an unsupervised way. Unlike some related studies for estimating viewer interest, our method takes into account actual mechanisms of the behavior of users while they are watching Web videos by utilizing parametric techniques for anomaly detection. Then viewer interest can be estimated on the basis of an anomaly score calculated from our proposed method. Consequently, successful estimation of viewer interest based on a framework for anomaly detection, via collaborative use of facial expression and biological signals, becomes feasible.

Accurate estimation of personalized video preference using multiple users' viewing behavior
Yoshiki Ito, Takahiro Ogawa, Miki Haseyama
IEICE Transactions on Information and Systems, E101D, 2, 481, 490, Institute of Electronics, Information and Communication, Engineers, IEICE, 2018年02月01日, ［査読有り］
英語, 研究論文（学術雑誌）, A method for accurate estimation of personalized video preference using multiple users' viewing behavior is presented in this paper. The proposed method uses three kinds of features: a video, user's viewing behavior and evaluation scores for the video given by a target user. First, the proposed method applies Supervised Multiview Spectral Embedding (SMSE) to obtain lower-dimensional video features suitable for the following correlation analysis. Next, supervised Multi-View Canonical Correlation Analysis (sMVCCA) is applied to integrate the three kinds of features. Then we can get optimal projections to obtain new visual features, "canonical video features" reflecting the target user's individual preference for a video based on sMVCCA. Furthermore, in our method, we use not only the target user's viewing behavior but also other users' viewing behavior for obtaining the optimal canonical video features of the target user. This unique approach is the biggest contribution of this paper. Finally, by integrating these canonical video features, Support Vector Ordinal Regression with Implicit Constraints (SVORIM) is trained in our method. Consequently, the target user's preference for a video can be estimated by using the trained SVORIM. Experimental results show the effectiveness of our method.

Media engineering
Miki Haseyama, Kei Kawamura, Shuhei Tarashima, Hiroyuki Arai
Kyokai Joho Imeji Zasshi/Journal of the Institute of Image Information and Television Engineers, 72, 3, 241, 246, Inst. of Image Information and Television Engineers, 2018年
日本語, 研究論文（学術雑誌）

Sentiment-aware personalized tweet recommendation through multimodal FFM
Ryosuke Harakawa, Daichi Takehara, Takahiro Ogawa, Miki Haseyama
Multimedia Tools and Applications, 77, 14, 18741, 18759, SPRINGER, 2018年, ［査読有り］
英語, 研究論文（学術雑誌）, For realizing quick and accurate access to desired information and effective advertisements or election campaigns, personalized tweet recommendation is highly demanded. Since multimedia contents including tweets are tools for users to convey their sentiment, users' interest in tweets is strongly influenced by sentiment factors. Therefore, successful personalized tweet recommendation can be realized if sentiment in tweets can be estimated. However, sentiment factors were not taken into account in previous works and the performance of previous methods may be limited. To overcome the limitation, a method for sentiment-aware personalized tweet recommendation through multimodal Field-aware Factorization Machines (FFM) is newly proposed in this paper. Successful personalized tweet recommendation becomes feasible through the following three contributions: (i) sentiment factors are newly introduced into personalized tweet recommendation, (ii) users' interest is modeled by deriving multimodal FFM that enables collaborative use of multiple factors in a tweet, i.e., publisher, topic and sentiment factors, and (iii) the effectiveness of using sentiment factors as well as publisher and topic factors is clarified from results of experiments using real-world datasets related to worldwide hot topics, "#trump", "#hillaryclinton" and "#ladygaga". In addition to showing the effectiveness of the proposed method, the applicability of the proposed method to other tasks such as advertisement and social analysis is discussed as a conclusion and future work of this paper.

Favorite Video Estimation Based on Multiview Feature Integration via KMvLFDA.
Array,Array,Miki Haseyama
IEEE Access, 6, 63833, 63842, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2018年, ［査読有り］
英語, 研究論文（学術雑誌）, This paper presents a novel method for favorite video estimation based on multiview feature integration via kernel multiview local fisher discriminant analysis (KMvLFDA). The proposed method first extracts electroencephalogram (EEG) features from users' EEG signals recorded while watching videos and multiple visual features from videos. Then, multiple EEG-based visual features are obtained by applying locality preserving canonical correlation analysis to EEG features and each visual feature. Next, KMvLFDA, which is newly derived in this paper, explores the complementary properties of different features and integrates the multiple EEG-based visual features. In addition, by using KMvLFDA, between-class scatter is maximized and within-class scatter is minimized in the integrated feature space. Consequently, it can be expected that the new features that are obtained by the above integration are more effective than each of the EEG-based visual features for the estimation of users' favorite videos. The main contribution of this paper is the new derivation of KMvLFDA. Successful estimation of users' favorite videos becomes feasible by using the new features obtained via KMvLFDA.

Favorite Video Classification Based on Multimodal Bidirectional LSTM.
Array,Array, Keisuke Maeda, Miki Haseyama
IEEE Access, 6, 61401, 61409, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2018年, ［査読有り］
英語, 研究論文（学術雑誌）, Video classification based on the user's preference (information of what a user likes: WUL) is important for realizing human-centered video retrieval. A better understanding of the rationale of WUL would greatly contribute to the support for successful video retrieval. However, a few studies have shown the relationship between information of what a user watches and WUL. A new method that classifies videos on the basis of WUL using video features and electroencephalogram (EEG) signals collaboratively with a multimodal bidirectional Long Short-Term Memory (Bi-LSTM) network is presented in this paper. To the best of our knowledge, there has been no study on WUL-based video classification using video features and EEG signals collaboratively with LSTM. First, we newly apply transfer learning to the WUL-based video classification since the number of labels (liked or not liked) attached to videos by users is small, and it is difficult to classify videos based on WUL. Furthermore, we conduct a user study for showing that the representation of psychophysiological signals calculated from Bi-LSTM is effective for the WUL-based video classification. Experimental results showed that our deep neural network feature representations can distinguish WUL for each subject.

Selection of Significant Brain Regions Based on MvGTDA and TS-DLF for Emotion Estimation.
Array,Array,Miki Haseyama
IEEE Access, 6, 32481, 32492, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2018年, ［査読有り］
英語, 研究論文（学術雑誌）, In this paper, we propose a novel method for estimating human emotion using functional brain images. The final goal of our study is contribution to affective brain computer interfaces (aBCIs), which use neuropsychological signals. In the proposed method, we newly derive multiview general tensor discriminant analysis (MvGTDA) in order to reveal significant brain regions and accurately estimate human emotion evoked by visual stimuli. This is because it is important to find activation of multiple brain regions for estimating emotional states. Since we regard a Brodmann area as a view and introduce -norm regularization for these views, MvGTDA can eliminate non-crucial Brodmann areas and select significant ones. Moreover, in general studies on functional brain images based on machine learning methodologies, there is an overfitting problem caused by a small sample size. Therefore, revealing significant Brodmann areas based on MvGTDA has another important role, i.e., solving the overfitting problem. By inputting estimation results respectively obtained from the significant areas and the MvGTDA-based feature, tensor-based supervised decision-level fusion (TS-DLF) integrates them and outputs the final estimation result of the users emotion. In experiments, we showed the effectiveness of our method by using actual functional brain images and we revealed the significant brain regions in emotional states.

A Language-Independent Ontology Construction Method Using Tagged Images in Folksonomy.
Array,Takahiro Ogawa, Miki Haseyama
IEEE Access, 6, 2930, 2942, 2018年, ［査読有り］
研究論文（学術雑誌）

Tracking topic evolution via salient keyword matching with consideration of semantic broadness for Web video discovery
Ryosuke Harakawa, Takahiro Ogawa, Miki Haseyama
Multimedia Tools and Applications, 77, 16, 1, 28, Springer New York LLC, 2017年12月06日, ［査読有り］
英語, 研究論文（学術雑誌）, A method to track topic evolution via salient keyword matching with consideration of semantic broadness for Web video discovery is presented in this paper. The proposed method enables users to understand the evolution of topics over time for discovering Web videos in which they are interested. A framework that enables extraction and tracking of the hierarchical structure, which contains Web video groups with various degrees of semantic broadness, is newly derived as follows: Based on network analysis using multimodal features, i.e., features of video contents and metadata, our method extracts the hierarchical structure and salient keywords that represent contents of each Web video group. Moreover, salient keyword matching, which is newly developed by considering salient keyword distribution, semantic broadness of each Web video group and initial topic relevance, is applied to each hierarchical structure obtained in different time stamps. Unlike methods in previous works, by considering the semantic broadness as well as the salient keyword distribution, our method can overcome the problem of the desired semantic broadness of topics being different depending on each user. Also, the initial topic relevance enables correction of the gap from an initial topic at the start of tracking. Consequently, it becomes feasible to track the evolution of topics over time for finding Web videos in which the users are interested. Experimental results for real-world datasets containing YouTube videos verify the effectiveness of the proposed method.

Extracting hierarchical structure of content groups from different social media platforms using multiple social metadata
Daichi Takehara, Ryosuke Harakawa, Takahiro Ogawa, Miki Haseyama
MULTIMEDIA TOOLS AND APPLICATIONS, 76, 19, 20249, 20272, SPRINGER, 2017年10月, ［査読有り］
英語, 研究論文（学術雑誌）, A novel scheme for retrieving users' desired contents, i.e., contents with topics in which users are interested, from multiple social media platforms is presented in this paper. In existing retrieval schemes, users first select a particular platform and then input a query into the search engine. If users do not specify suitable platforms for their information needs and do not input suitable queries corresponding to the desired contents, it becomes difficult for users to retrieve the desired contents. The proposed scheme extracts the hierarchical structure of content groups (sets of contents with similar topics) from different social media platforms, and it thus becomes feasible to retrieve desired contents even if users do not specify suitable platforms and do not input suitable queries. This paper has two contributions: (1) A new feature extraction method, Locality Preserving Canonical Correlation Analysis with multiple social metadata (LPCCA-MSM) that can detect content groups without the boundaries of different social media platforms is presented in this paper. LPCCA-MSM uses multiple social metadata as auxiliary information unlike conventional methods that only use content-based information such as textual or visual features. (2) The proposed novel retrieval scheme can realize hierarchical content structuralization from different social media platforms. The extracted hierarchical structure shows various abstraction levels of content groups and their hierarchical relationships, which can help users select topics related to the input query. To the best of our knowledge, an intensive study on such an application has not been conducted; therefore, this paper has strong novelty. To verify the effectiveness of the above contributions, extensive experiments for real-world datasets containing YouTube videos and Wikipedia articles were conducted.

Wiener-Based Inpainting Quality Prediction
Takahiro Ogawa, Akira Tanaka, Miki Haseyama
IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, E100D, 10, 2614, 2626, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2017年10月, ［査読有り］
英語, 研究論文（学術雑誌）, A Wiener-based inpainting quality prediction method is presented in this paper. The proposed method is the first method that can predict inpainting quality both before and after the intensities have become missing even if their inpainting methods are unknown. Thus, when the target image does not include any missing areas, the proposed method estimates the importance of intensities for all pixels, and then we can know which areas should not be removed. Interestingly, since this measure can be also derived in the same manner for its corrupted image already including missing areas, the expected difficulty in reconstruction of these missing pixels is predicted, i.e., we can know which missing areas can be successfully reconstructed. The proposed method focuses on expected errors derived from the Wiener filter, which enables least-squares reconstruction, to predict the inpainting quality. The greatest advantage of the proposed method is that the same inpainting quality prediction scheme can be used in the above two different situations, and their results have common trends. Experimental results show that the inpainting quality predicted by the proposed method can be successfully used as a universal quality measure.

Visualizing Web Images Using Fisher Discriminant Locality Preserving Canonical Correlation Analysis
Kohei Tateno, Takahiro Ogawa, Miki Haseyama
IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, E100D, 9, 2005, 2016, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2017年09月, ［査読有り］
英語, 研究論文（学術雑誌）, A novel dimensionality reduction method, Fisher Discriminant Locality Preserving Canonical Correlation Analysis (FDLPCCA), for visualizing Web images is presented in this paper. FDLP-CCA can integrate two modalities and discriminate target items in terms of their semantics by considering unique characteristics of the two modalities. In this paper, we focus onWeb images with text uploaded on Social Networking Services for these two modalities. Specifically, text features have high discriminate power in terms of semantics. On the other hand, visual features of images give their perceptual relationships. In order to consider both of the above unique characteristics of these two modalities, FDLPCCA estimates the correlation between the text and visual features with consideration of the cluster structure based on the text features and the local structures based on the visual features. Thus, FDLP-CCA can integrate the different modalities and provide separated manifolds to organize enhanced compactness within each natural cluster.

Biomimetics Image Retrieval Platform
Miki Haseyama, Takahiro Ogawa, Sho Takahashi, Shuhei Nomura, Masatsugu Shimomura
IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, E100D, 8, 1563, 1573, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2017年08月, ［査読有り］
英語, 研究論文（学術雑誌）, Biomimetics is a new research field that creates innovation through the collaboration of different existing research fields. However, the collaboration, i.e., the exchange of deep knowledge between different research fields, is difficult for several reasons such as differences in technical terms used in different fields. In order to overcome this problem, we have developed a new retrieval platform, "Biomimetics image retrieval platform," using a visualization-based image retrieval technique. A biological database contains a large volume of image data, and by taking advantage of these image data, we are able to overcome limitations of text-only information retrieval. By realizing such a retrieval platform that does not depend on technical terms, individual biological databases of various species can be integrated. This will allow not only the use of data for the study of various species by researchers in different biological fields but also access for a wide range of researchers in fields ranging from materials science, mechanical engineering and manufacturing. Therefore, our platform provides a new path bridging different fields and will contribute to the development of biomimetics since it can overcome the limitation of the traditional retrieval platform.

Deterioration Level Estimation on Transmission Towers via Extreme Learning Machine based on Combination Use of Local Receptive Field and Principal Component Analysis　　　　　　　　　　　　　　　
K. Maeda, S. Takahashi, T. Ogawa, M. Haseyama
International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC), 457, 458, 2017年07月, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）

Effectiveness Evaluation of Imaging Direction for Estimation of Gastritis Regions on Gastric X-ray Images　　　　　　　　　　　　　　　
Ren Togo, Kenta Ishihara, Takahiro Ogawa, Miki Haseyama
International Technical Conference on Circuits, Systems, Computers, and Communications (ITC-CSCC), 459, 460, 2017年05月, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）

Helicobacter Pylori infection detection from gastric X-ray images based on feature fusion and decision fusion
Kenta Ishihara, Takahiro Ogawa, Miki Haseyama
COMPUTERS IN BIOLOGY AND MEDICINE, 84, 69, 78, PERGAMON-ELSEVIER SCIENCE LTD, 2017年05月, ［査読有り］
英語, 研究論文（学術雑誌）, In this paper, a fully automatic method for detection of Helicobacter pylori (H. pylori) infection is presented with the aim of constructing a computer-aided diagnosis (CAD) system. In order to realize a CAD system with good performance for detection of H. pylori infection, we focus on the following characteristic of stomach X-ray examination. The accuracy of X-ray examination differs depending on the symptom of H. pylori infection that is focused on and the position from which X-ray images are taken. Therefore, doctors have to comprehensively assess the symptoms and positions. In order to introduce the idea of doctors' assessment into the CAD system, we newly propose a method for detection of H. pylori infection based on the combined use of feature fusion and decision fusion. As a feature fusion scheme, we adopt Multiple Kernel Learning (MKL). Since MKL can combine several features with determination of their weights, it can represent the differences in symptoms. By constructing an MKL classifier for each position, we can obtain several detection results. Furthermore, we introduce confidence-based decision fusion, which can consider the relationship between the classifier's performance and the detection results. Consequently, accurate detection of H. pylori infection becomes possible by the proposed method. Experimental results obtained by applying the proposed method to real X-ray images show that our method has good performance, close to the results of detection by specialists, and indicate that the realization of a CAD system for determining the risk of H. pylori infection is possible.

Tracking hierarchical structure of web video groups based on salient keyword matching including semantic broadness estimation
Ryosuke Harakawa, Takahiro Ogawa, Miki Haseyama
2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings, 1238, 1242, Institute of Electrical and Electronics Engineers Inc., 2017年04月19日, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a novel method to track the hierarchical structure of Web video groups on the basis of salient keyword matching including semantic broadness estimation. To the best of our knowledge, this paper is the first work to perform extraction and tracking of the hierarchical structure simultaneously. Specifically, the proposed method first extracts the hierarchical structure of Web video groups and salient keywords of them on the basis of an improved scheme of our previously reported method. Moreover, to calculate similarities between Web video groups obtained in different time stamps, salient keyword matching is newly developed by considering both co-occurrences of the salient keywords and semantic broadness of each Web video group. Consequently, tracking of the hierarchical structure over time becomes feasible to easily understand popularity trends of many Web videos for realizing effective retrieval.

Distress Classification of Class Imbalanced Data for Maintenance Inspection of Road Structures in Express Way　　　　　　　　　　　　　　　
K. Maeda, S. Takahashi, T. Ogawa, M. Haseyama
International Conference on Civil and Building Engineering Informatics in conjunction with Conference on Computer Applications in Civil and Hydraulic Engineering (ICCBEI & CCACHE), 182, 185, 2017年04月, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）

Human-Centered Video Feature Selection via mRMR-SCMMCCA for Preference Extraction
Takahiro Ogawa, Yoshiaki Yamaguchi, Satoshi Asamizu, Miki Haseyama
IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, E100D, 2, 409, 412, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2017年02月, ［査読有り］
英語, 研究論文（学術雑誌）, This paper presents human-centered video feature selection via mRMR-SCMMCCA (minimum Redundancy and Maximum Relevance-Specific Correlation Maximization Multiset Canonical Correlation Analysis) algorithm for preference extraction. The proposed method derives SCMMCCA, which simultaneously maximizes two kinds of correlations, correlation between video features and users' viewing behavior features and correlation between video features and their corresponding rating scores. By monitoring the derived correlations, the selection of the optimal video features that represent users' individual preference becomes feasible.

PERSONALIZED VIDEO PREFERENCE ESTIMATION BASED ON EARLY FUSION USING MULTIPLE USERS'VIEWING BEHAVIOR
Yoshiki Ito, Takahiro Ogawa, Miki Haseyama
2017 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 3006, 3010, IEEE, 2017年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a novel method for personalized video preference estimation based on early fusion using multiple users' viewing behavior. The proposed method adopts supervised Multi-View Canonical Correlation Analysis (sMVCCA) to estimate correlation between different types of features. Specifically, we estimate optimal projections maximizing the correlation between three features of video, target user's viewing behavior and evaluation scores for video. Then novel video features (canonical video features), which reflect the target user's individual preference, are obtained by the estimated projections. Furthermore, our method computes sMVCCA-based canonical video features by using multiple users' viewing behavior and a target user's evaluation scores. This non-conventional approach using the multiple users' viewing behavior for the preference estimation of the target user is the biggest contribution of our method, and it enables early fusion of the canonical video features. Consequently, successful video recommendation that reflects the users' individual preference can be expected via the evaluation score prediction from the integrated canonical video features. Experimental results show the effectiveness of our method.

EXEMPLAR-BASED IMAGE COMPLETION VIA NEW QUALITY MEASURE BASED ON PHASELESS TEXTURE FEATURES
Takahiro Ogawa, Miki Haseyama
2017 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 1827, 1831, IEEE, 2017年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents an exemplar-based image completion via a new quality measure based on phaseless texture features. The proposed method derives a new quality measure obtained by monitoring errors caused in power spectra, i.e., errors of phaseless texture features, converged through phase retrieval. Even if a target patch includes missing pixels, this measure enables selection of the best matched patch including the most similar texture features for realizing the exemplar-based image completion. Furthermore, since the phaseless texture features are robust to various changes such as spatial gaps and luminance changes, the new quality measure successfully provides the best matched patch from few training examples. Then, by solving an optimization problem that retrieves the phase of the target patch from the phaseless texture features of the best matched patch, its missing areas can be reconstructed. Consequently, accurate image completion using the new quality measure becomes feasible. Subjective and quantitative experimental results are shown to verify the effectiveness of our method using the new quality measure.

EMOTION ESTIMATION VIA TENSOR-BASED SUPERVISED DECISION-LEVEL FUSION FROM MULTIPLE BRODMANN AREAS
Kento Sugata, Takahiro Ogawa, Miki Haseyama
2017 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 999, 1003, IEEE, 2017年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a novel method that estimates human emotion based on tensor-based supervised decision-level fusion (TS-DLF) from multiple Brodmann areas (BAs). From multiple brain data corresponding to these BAs captured by functional magnetic resonance imaging (fMRI), our method performs general tensor discriminant analysis (GTDA) to obtain features which can reflect the user's emotion. Furthermore, since the dimension of the obtained features becomes lower, this can avoid overfitting in the following training procedure of estimators. Next, by separately using the transformed BA data obtained after GTDA, we obtain multiple estimation results of the user's emotion based on logistic tensor regression (LTR). Then our method realizes the decision of the final result based on TS-DLF from the multiple estimation results. This approach, i.e., the integration of the multiple BAs' results for the whole-brain data, is the biggest contribution of this paper. TS-DLF successfully integrates the multiple estimation results with considering the performance of the LTR-based estimator constructed for each BA. Experimental results show that our method outperforms state-of-the-art approaches, and the effectiveness of our method can be confirmed.

Extracting Hierarchical Structure of Web Video Groups Based on Sentiment-Aware Signed Network Analysis
Ryosuke Harakawa, Takahiro Ogawa, Miki Haseyama
IEEE ACCESS, 5, 16963, 16973, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2017年, ［査読有り］
英語, 研究論文（学術雑誌）, Sentiment in multimedia contents has an influence on their topics, since multimedia contents are tools for social media users to convey their sentiment. Performance of applications such as retrieval and recommendation will be improved if sentiment in multimedia contents can be estimated; however, there have been few works in which such applications were realized by utilizing sentiment analysis. In this paper, a novel method for extracting the hierarchical structure of Web video groups based on sentiment-aware signed network analysis is presented to realize Web video retrieval. First, the proposed method estimates latent links between Web videos by using multimodalfeatures of contents and sentiment features obtained from texts attached to Web videos. Thus, our method enables construction of a signed network that reflects not only similarities but also positive and negative relations between topics of Web videos. Moreover, an algorithm to optimize a modularity-based measure, which can adaptively adjust the balance between positive and negative edges, was newly developed. This algorithm detects Web video groups with similar topics at multiple abstraction levels; thus, successful extraction of the hierarchical structure becomes feasible. By providing the hierarchical structure, users can obtain an overview of many Web videos and it becomes feasible to successfully retrieve the desired Web videos. Results of experiments using a new benchmark dataset, YouTube-8M, validate the contributions of this paper, i.e., 1) the first attempt to utilize sentiment analysis for Web video grouping and 2) a novel algorithm for analyzing a weighted signed network derived from sentiment and multimodal features.

Retrieval of similar inspection records based on metric learning using experienced inspectors' evaluation
Ryota Saito, Sho Takahashi, Takahiro Ogawa, Miki Hasayama
2016 IEEE 5th Global Conference on Consumer Electronics, GCCE 2016, 1, 2, Institute of Electrical and Electronics Engineers Inc., 2016年12月27日, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a retrieval method of similar inspection records in road structures based on metric learning using experienced inspectors' evaluation. Inspection records of road structures include images and text-based information such as category of distress, damaged parts and degree of damage. The proposed method calculates distances from query inspection records, and rank lists of retrieval results are obtained for each feature. In this approach, the distance quantification are updated on the basis of experienced inspectors' evaluation. Finally, the proposed method obtains retrieval results by integrating the multiple rank lists. The experimental results show the effectiveness of the proposed method.

Adaptive Subspace-Based Inverse Projections via Division Into Multiple Sub-Problems for Missing Image Data Restoration
Takahiro Ogawa, Miki Haseyama
IEEE TRANSACTIONS ON IMAGE PROCESSING, 25, 12, 5971, 5986, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2016年12月, ［査読有り］
英語, 研究論文（学術雑誌）, This paper presents adaptive subspace-based inverse projections via division into multiple sub-problems (ASIP-DIMSs) for missing image data restoration. In the proposed method, a target problem for estimating missing image data is divided into multiple sub-problems, and each sub-problem is iteratively solved with the constraints of other known image data. By projection into a subspace model of image patches, the solution of each sub-problem is calculated, where we call this procedure "subspace-based inverse projection" for simplicity. The proposed method can use higher dimensional subspaces for finding unique solutions in each sub-problem, and successful restoration becomes feasible, since a high level of image representation performance can be preserved. This is the biggest contribution of this paper. Furthermore, the proposed method generates several subspaces from known training examples and enables derivation of a new criterion in the above framework to adaptively select the optimal subspace for each target patch. In this way, the proposed method realizes missing image data restoration using ASIP-DIMS. Since our method can estimate any kind of missing image data, its potential in two image restoration tasks, image inpainting and super-resolution, based on several methods for multivariate analysis is also shown in this paper.

A Web video retrieval method using hierarchical structure of Web video groups
Ryosuke Harakawa, Takahiro Ogawa, Miki Haseyama
MULTIMEDIA TOOLS AND APPLICATIONS, 75, 24, 17059, 17079, SPRINGER, 2016年12月, ［査読有り］
英語, 研究論文（学術雑誌）, In this paper, we propose a Web video retrieval method that uses hierarchical structure of Web video groups. Existing retrieval systems require users to input suitable queries that identify the desired contents in order to accurately retrieve Web videos; however, the proposed method enables retrieval of the desired Web videos even if users cannot input the suitable queries. Specifically, we first select representative Web videos from a target video dataset by using link relationships between Web videos obtained via metadata "related videos" and heterogeneous video features. Furthermore, by using the representative Web videos, we construct a network whose nodes and edges respectively correspond to Web videos and links between these Web videos. Then Web video groups, i.e., Web video sets with similar topics are hierarchically extracted based on strongly connected components, edge betweenness and modularity. By exhibiting the obtained hierarchical structure of Web video groups, users can easily grasp the overview of many Web videos. Consequently, even if users cannot write suitable queries that identify the desired contents, it becomes feasible to accurately retrieve the desired Web videos by selecting Web video groups according to the hierarchical structure. Experimental results on actual Web videos verify the effectiveness of our method.

Performance Improvement of Error-Resilient 3D DWT Video Transmission Using Invertible Codes
Kotoku Omura, Shoichiro Yamasaki, Tomoko K. Matsushima, Hirokazu Tanaka, Miki Haseyama
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, E99A, 12, 2256, 2265, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2016年12月, ［査読有り］
英語, 研究論文（学術雑誌）, Many studies have applied the three-dimensional discrete wavelet transform (3D DWT) to video coding. It is known that corruptions of the lowest frequency sub-band (LL) coefficients of 3D DWT severely affect the visual quality of video. Recently, we proposed an error resilient 3D DWT video coding method (the conventional method) that employs dispersive grouping and an error concealment (EC). The EC scheme of our conventional method adopts a replacement technique of the lost LL coefficients. In this paper, we propose a new 3D DWT video transmission method in order to enhance error resilience. The proposed method adopts an error correction scheme using invertible codes to protect LL coefficients. We use half-rate Reed-Solomon (RS) codes as invertible codes. Additionally, to improve performance by using the effect of interleave, we adopt a new configuration scheme at the RS encoding stage. The evaluation by computer simulation compares the performance of the proposed method with that of other EC methods, and indicates the advantage of the proposed method.

Classifying Insects from SEM Images Based on Optimal Classifier Selection and D-S Evidence Theory
Takahiro Ogawa, Akihiro Takahashi, Miki Haseyama
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, E99A, 11, 1971, 1980, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2016年11月, ［査読有り］
英語, 研究論文（学術雑誌）, In this paper, an insect classification method using scanning electron microphotographs is presented. Images taken by a scanning electron microscope (SEM) have a unique problem for classification in that visual features differ from each other by magnifications. Therefore, direct use of conventional methods results in inaccurate classification results. In order to successfully classify these images, the proposed method generates an optimal training dataset for constructing a classifier for each magnification. Then our method classifies images using the classifiers constructed by the optimal training dataset. In addition, several images are generally taken by an SEM with different magnifications from the same insect. Therefore, more accurate classification can be expected by integrating the results from the same insect based on Dempster-Shafer evidence theory. In this way, accurate insect classification can be realized by our method. At the end of this paper, we show experimental results to confirm the effectiveness of the proposed method.

Estimation of salient regions related to chronic gastritis using gastric X-ray images
Ren Togo, Kenta Ishihara, Takahiro Ogawa, Miki Haseyama
COMPUTERS IN BIOLOGY AND MEDICINE, 77, 9, 15, PERGAMON-ELSEVIER SCIENCE LTD, 2016年10月, ［査読有り］
英語, 研究論文（学術雑誌）, Since technical knowledge and a high degree of experience are necessary for diagnosis of chronic gastritis, computer-aided diagnosis (CAD) systems that analyze gastric X-ray images are desirable in the field of medicine. Therefore, a new method that estimates salient regions related to chronic gastritis/non-gastritis for supporting diagnosis is presented in this paper. In order to estimate salient regions related to chronic gastritis/non-gastritis, the proposed method monitors the distance between a target image feature and Support Vector Machine (SVM)-based hyperplane for its classification. Furthermore, our method realizes removal of the influence of regions outside the stomach by using positional relationships between the stomach and other organs. Consequently, since the proposed method successfully estimates salient regions of gastric X-ray images for which chronic gastritis and non-gastritis are unknown, visual support for inexperienced clinicians becomes feasible. (C) 2016 Elsevier Ltd. All rights reserved.

画像共有サービスに投稿される画像の観光名所に関するカテゴリー分類
斉藤 直輝, 小川 貴弘, 浅水 仁, 長谷山 美紀
電子情報通信学会論文誌D 情報・システム, J99-D, 9, 848, 860, The Institute of Electronics, Information and Communication Engineers, 2016年09月01日, ［査読有り］
日本語, 本論文では，画像共有サービスに投稿される画像の観光名所に関するカテゴリー（観光カテゴリー）の分類手法を提案する．提案手法では，画像共有サービスにおいて画像とともに投稿される様々なデータの中で，位置座標が観光カテゴリーの分類に最も効果的であることに注目する．更に，このデータに基づいた分類において誤りが発生する場合を，分類結果から導出される確信度に基づいて判定する．誤分類と判定された場合，提案手法では，画像及びタグの特徴に基づいた分類結果を求め，それらを各々の分類精度に注目して統合することで，最終的な分類結果を高精度に推定可能とする．このとき，位置座標による分類結果に基づいて分類候補の観光カテゴリーを限定することで，多クラス分類問題におけるクラス数を減少させ，より正確な分類を可能とする．以上の提案手法によって，異なる種類のデータの分類精度が互いに大きく異なる場合に従来手法では困難であった高精度な最終分類結果の取得が可能となる．

NMF-Based Spectral Reflectance Estimation From Image Pairs Including Near-Infrared Components
Takahiro Ogawa, Yuta Igarashi, Miki Haseyama
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 26, 5, 855, 867, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2016年05月, ［査読有り］
英語, 研究論文（学術雑誌）, In this paper, a novel spectral reflectance estimation method from image pairs including near-infrared (NIR) components based on nonnegative matrix factorization (NMF) is presented. The proposed method enables estimation of spectral reflectance from only two kinds of input images: 1) an image including both visible light components and NIR components and 2) an image including only NIR components. These two images can be easily obtained using a general digital camera without an infrared-cut filter and one with a visible light-cut filter, respectively. Since RGB values of these images are obtained according to spectral sensitivity of the image sensor, the spectrum power distribution of the light source and the spectral reflectance, we have to solve the inverse problem for estimating the spectral reflectance. Therefore, our method approximates spectral reflectance by a linear combination of several bases obtained by applying NMF to a known spectral reflectance data set. Then estimation of the optimal solution to the above problem becomes feasible based on this approximation. In the proposed method, NMF is used for obtaining the bases used in this approximation from a characteristic that the spectral reflectance is a nonnegative component. Furthermore, the proposed method realizes simple approximation of the spectrum power distribution of the light source with direct and scattered light components. Therefore, estimation of spectral reflectance becomes feasible using the spectrum power distribution of the light source in our method. In the last part of this paper, we show some simulation results to verify the performance of the proposed method. The effectiveness of the proposed method is also shown using the method for several applications that are closely related to spectral reflectance estimation. Although our method is based on a simple scheme, it is the first method that realizes the estimation of the spectral reflectance and the spectrum power distribution of the light source from the above two kinds of images taken by general digital cameras and provides breakthroughs to several fundamental applications.

A Most Resource-Consuming Disease Estimation Method from Electronic Claim Data Based on Labeled LDA
Yasutaka Hatakeyama, Takahiro Ogawa, Hironori Ikeda, Miki Haseyama
IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, E99D, 3, 763, 768, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2016年03月, ［査読有り］
英語, 研究論文（学術雑誌）, In this paper, we propose a method to estimate the most resource-consuming disease from electronic claim data based on Labeled Latent Dirichlet Allocation (Labeled LDA). The proposed method models each electronic claim from its medical procedures as a mixture of resource-consuming diseases. Thus, the most resource-consuming disease can be automatically estimated by applying Labeled LDA to the electronic claim data. Although our method is composed of a simple scheme, this is the first trial for realizing estimation of the most resource-consuming disease.

Distress Classification of Road Structures via Multiple Classifier-based Bayesian Network　　　　　　　　　　　　　　　
K. Maeda, S. Takahashi, T. Ogawa, M. Haseyama
International Workshop on Advanced Image Technology (IWAIT), 1, 4, 2016年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）

6. サッカー映像における試合内容の理解を促すデータの可視化
高橋 翔, 長谷山 美紀
映像情報メディア学会誌, 70, 9, 722, 724, 一般社団法人 映像情報メディア学会, 2016年
日本語

Multimodal Interest Level Estimation via Variational Bayesian Mixture of Robust CCA
Yuma Sasaka, Takhiro Ogawa, Miki Haseyama
MM'16: PROCEEDINGS OF THE 2016 ACM MULTIMEDIA CONFERENCE, 387, 391, ASSOC COMPUTING MACHINERY, 2016年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a method which estimates interest level while watching videos, based on collaborative use of facial expression and biological signals such as electroencephalogram (EEG) and electrocardiogram (ECG). To the best of our knowledge, no studies have been carried out on the collaborative use of facial expression and biological signals for estimating interest level. Since training data, which is used for estimating interest level, is generally small and imbalanced, Variational Bayesian Mixture of Robust Canonical Correlation Analysis (VBMRCCA) is newly applied to facial expression and biological signals, which are obtained from users while they are watching the videos. Unlike some related works, VBMRCCA is used to obtain the posterior distributions which represent the latent correlation between facial expression and biological signals in our method. Then, the users' interest level can be estimated by comparing the posterior distributions of the positive class data with those of the negative. Consequently, successful interest level estimation, via collaborative use of facial expression and biological signals, becomes feasible.

Realization of Associative Image Search Development of Image Retrieval Platform for Enhancing Serendipity
Miki Haseyama
2016 IEEE 46TH INTERNATIONAL SYMPOSIUM ON MULTIPLE-VALUED LOGIC (ISMVL 2016), 56, 59, IEEE COMPUTER SOC, 2016年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents "Associative Image Search", a new image retrieval scheme and its specific engineering application, which enable value creation from big data. The main aim of the associative image search is the realization of information retrieval that enhances the potential for serendipities by providing users with new awareness. Thus, this paper presents the details of research for realizing associative image retrieval. Furthermore, as an example of its applications, a Biomimetics image retrieval platform is also introduced in this paper. By associatively and collaboratively using data accumulated in the fields of biology and material science, the Biomimetics image retrieval platform enables acceleration of their knowledge sharing in different research fields. From retrieval results actually obtained from this platform, there is discussion of the potential of serendipities such as new knowledge emergence

A Virtual Vital Signs Sensor "MIRUWS" for Visualization of Healthy to Illness Transition (HIT)
Shigenobu Minami, Miki Haseyama, Hirokazu Tanaka, Toru Takahashi, Tatsuya Komori
2016 10TH INTERNATIONAL SYMPOSIUM ON MEDICAL INFORMATION AND COMMUNICATION TECHNOLOGY (ISMICT), 1, 5, IEEE, 2016年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper proposes a virtual Vital Signs Sensor (VSS) for visualization of half illness (so called "Mibyou") and sudden illness. Since both half and sudden illnesses, which are located in between wellness and illness, are categorized in healthy(H) to illness(I) transition(T) status. So, we name this status as HIT. Since HIT happens in an ordinary life, VSS for HIT visualization should work for 24 hours continuously and unconsciously.
The proposed virtual VSS is named as "MIRUWS" which means (M)multi-sensing, (I)integrated, (R)reliable, and (U)unconscious (V)virtual (V)vital (S) signs sensor. All these key words are meaningful and required for professional HIT (Pro-HIT) visualization which has real demand rather than purely personal one.
MIRUWS is a virtual VSS in a cyber space targeting this Pro-HIT visualization, and is a projection of actual VSSs in a physical space. There are wide varieties of physical sensors such as patch, touch, proximity, and remote types. To handle these wide varieties of VSSs efficiently and consistently, MIRUWS plays as a common and unique virtual VSS in a cyber space.
Unlikely to medical devices which need very high reliability rather than flexibility, MIRUWS is needed to satisfy both reliability and flexibility at the same time to cover wide range of Pro-HIT use-cases. To realize this, MIRUWS visualizes physical VSS's specifications throughout new API, which are dynamically determined by connected and released physical VSSs in a BAN.
To certify MIRUWS performance objectively, conformance testing is desired. This paper also presents MIRUWS test environment example, which measures basic vital signs performances, heart rate variation (HRV) and pulse wave transition time (PWTT) using two target physical VSSs at once.

An Accurate Mortality Prediction Method Based on Decision-level Fusion of Existing ICU Scoring Systems
Yasutaka Hatakeyama, Takahiro Ogawa, Hirokazu Tanaka, Miki Haseyama
PROCEEDINGS OF 2016 INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY AND ITS APPLICATIONS (ISITA 2016), 126, 130, IEEE, 2016年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, In this paper, we propose a mortality prediction method based on decision-level fusion (DLF) of existing intensive unit care (ICU) scoring systems. First, the proposed method obtains severity scores from the existing ICU scoring systems. Furthermore, we construct classifiers that categorize patients into survivors or non-survivors. Next, patient feature vectors are extracted based on the mortality rates that are estimated from the obtained severity scores by using a non-linear least squares method to obtain other types of classification results. In order to obtain the final severity score for each patient, we integrate the obtained multiple classification results based on DLF that can estimate the final severity scores. Finally, we performed the proposed method to actual ICU patient data and verified the effectiveness of the proposed method. Thus, the proposed method can realize accurate mortality prediction without any additional work by using the existing ICU scoring systems.

GRAPH-BASED WEB VIDEO SEARCH RERANKING THROUGH CONSISTENCY ANALYSIS USING SPECTRAL CLUSTERING
Soh Yoshida, Takahiro Ogawa, Miki Haseyaina
2016 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA & EXPO (ICME), 1, 6, IEEE, 2016年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper proposes a graph-based Web video search reranking method through consistency analysis using spectral clustering. Graph-based reranking is effective for refining text-based video search results. Generally, this approach constructs a graph where the vertices are videos and the edges reflect their pairwise similarities. A lot of reranking methods are built based on a scheme which regularizes the smoothness of pairwise ranking scores between adjacent nodes. However, since the overall consistency is measured by aggregating the individual consistency over each pair, errors in score estimation increase when noisy samples are included within their neighbors. To deal with the noisy samples, different from the conventional methods, the proposed method models the global consistency of the graph structure. Specifically, in order to detect this consistency, the propose method introduces a spectral clustering algorithm which can detect video groups, whose videos have strong semantic correlation, on the graph. Furthermore, a new regularization term, which smooths ranking scores within the same group, is introduced to the reranking framework. Since score regularization is performed by both local and global aspect simultaneously, the accurate score estimation becomes feasible. Experimental results obtained by applying the proposed method to a real-world video collection show its effectiveness.

HIERARCHICAL CONTENT GROUP DETECTION FROM DIFFERENT SOCIAL MEDIA PLATFORMS USING WEB LINK STRUCTURE
Daichi Takehara, Ryosuke Harakawa, Takahiro Ogawa, Miki Haseyama
2016 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 479, 483, IEEE, 2016年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a method for hierarchical content group detection from different social media platforms, which can reveal hierarchical structure of content groups. In this paper, content groups are defined as sets of contents with similar topics. Based on the revealed hierarchical structure, our method enables users to efficiently find the desired contents from large amount of contents placed in diversified social media platforms. The main contributions of this paper are twofold. First, effective latent features for comparing the contents placed in different social media platforms can be extracted by the combination use of the correlation between features obtained from different social media platform and the Web link structure. Second, the hierarchical structure of the content groups, which captures their various abstraction levels, can be revealed by hierarchically detecting their content groups. Experimental results on the real-world dataset containing YouTube videos and Wikipedia articles show the effectiveness of our method.

Distress Classification of Road Structures via Decision Level Fusion
Keisuke Maeda, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
2016 IEEE INTERNATIONAL CONFERENCE ON DIGITAL SIGNAL PROCESSING (DSP), 589, 593, IEEE, 2016年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, A distress classification method of road structures via decision level fusion is presented in this paper. In order to classify various kinds of distresses accurately, the proposed method integrates multiple classification results with considering their performance, and this is the biggest contribution of this paper. By introducing this approach, it becomes feasible to adaptively integrate the multiple classification results based on the accuracy of each classifier for a target sample. Consequently, realization of the accurate distress classification can be expected. Experimental results show that our method outperforms existing methods.

Improvement of video coding efficiency based on sparse contractive mapping approach
Zaixing He, Takahiro Ogawa, Sho Takahashi, Miki Haseyama, Xinyue Zhao
NEUROCOMPUTING, 173, 1898, 1907, ELSEVIER SCIENCE BV, 2016年01月, ［査読有り］
英語, 研究論文（学術雑誌）, This paper presents a new method for improving video coding efficiency based on a sparse contractive mapping approach. The proposed method introduces a new sparse contractive mapping approach to replace the traditional intra prediction in the video coding standards such as H.264/AVC. Specifically, the intra- and its following inter-frame are respectively approximated by the sparse representation, satisfying contractive mapping. Then these two frames can be reconstructed from an arbitraryinitial image by utilizing a few representation coefficients. With this advantage, the proposed method reduces the total amount of bits by removing MBs in the target I frame, whose approximation performance is higher than the others in the encoder. Furthermore, by transmitting the representation coefficients of the removed MBs, these MBs can be accurately reconstructed in the decoder. Since the reconstruction performance is better than that of the conventional approach, the proposed method can remove more MBs from the target video sequences, and reduction of total amount of bits can be feasible. Therefore, the proposed method realizes the improvement of the video coding efficiency. Some experimental results are shown to verify the superior performance of the proposed method to that of H.264/AVC. The results also demonstrate that the bit-saving performance of the proposed method is comparable to that of H.2651 HEVC. (C) 2015 Elsevier B.V. All rights reserved.

NOVEL FAVORITE MUSIC CLASSIFICATION USING EEG-BASED OPTIMAL AUDIO FEATURES SELECTED VIA KDLPCCA
Ryosuke Sawata, Takahiro Ogawa, Miki Haseyama
2016 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING PROCEEDINGS, 759, 763, IEEE, 2016年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a novel method of favorite music classification using EEG-based optimal audio features. To select audio features related to user's music preference, our method utilizes a relationship between EEG features obtained from the user's EEG signals during listening to music and their corresponding audio features since EEG signals of human reflect his/her music preference. Specifically, cross-loadings, whose components denote the degree of the relationship, are calculated based on Kernel Discriminative Locality Preserving Canonical Correlation Analysis (KDLPCCA) which is newly derived in the proposed method. In contrast with standard CCA, KDLPCCA can consider (1) non-linear correlation, (2) class information and (3) local structures of input EEG and audio features, simultaneously. Therefore, KDLPCCA-based cross-loadings can reflect best correlation between the user's EEG and corresponding audio signals. Then an optimal set of audio features related to his/her music preference can be obtained by employing the cross-loadings as novel criteria for feature selection. Consequently, our method realizes favorite music classification successfully by using the EEG-based optimal audio features.

Bregman pooling: Feature-space local pooling for image classification
Alameen Najjar, Takahiro Ogawa, Miki Haseyama
International Journal of Multimedia Information Retrieval, 4, 4, 247, 259, Springer London, 2015年12月01日, ［査読有り］
英語, 研究論文（学術雑誌）, In this paper, we propose a novel feature-space local pooling method for the commonly adopted architecture of image classification. While existingmethods partition the feature space based on visual appearance to obtain pooling bins, learning more accurate space partitioning that takes semantics into account boosts performance even for a smaller number of bins. To this end, we propose partitioning the feature space over clusters of visual prototypes common to semantically similar images (i.e., images belonging to the same category). The clusters are obtained by Bregman co-clustering applied offline on a subset of training data. Therefore, being aware of the semantic context of the input image, our features have higher discriminative power than do those pooled from appearance-based partitioning. Testing on four datasets (Caltech-101, Caltech-256, 15 Scenes, and 17 Flowers) belonging to three different classification tasks showed that the proposed method outperforms methods in previous works on local pooling in he feature space for less feature dimensionality. Moreover, when implemented within a spatial pyramid, our method achieves comparable results on three of the datasets used.

Perceptually Optimized Missing Texture Reconstruction via Neighboring Embedding
Takahiro Ogawa, Miki Haseyama
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, E98A, 8, 1709, 1717, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2015年08月, ［査読有り］
英語, 研究論文（学術雑誌）, Perceptually optimized missing texture reconstruction via neighboring embedding (NE) is presented in this paper. The proposed method adopts the structural similarity (SSIM) index as a measure for representing texture reconstruction performance of missing areas. This provides a solution to the problem of previously reported methods not being able to perform perceptually optimized reconstruction. Furthermore, in the proposed method, a new scheme for selection of the known nearest neighbor patches for reconstruction of target patches including missing areas is introduced. Specifically, by monitoring the SSIM index observed by the proposed NE-based reconstruction algorithm, selection of known patches optimal for the reconstruction becomes feasible even if target patches include missing pixels. The above novel approaches enable successful reconstruction of missing areas. Experimental results show improvement of the proposed method over previously reported methods.

本格化するバイオミメティクス材料の開発〔解説‐2〕バイオミメティクス・画像検索システムの実現と活用
長谷山美紀
工業材料, 63, 8, 29, 33, 2015年08月01日
日本語

分類体系に注目した決定木の構築に基づく走査型電子顕微鏡画像を 用いた生物の分類法
朴 君, 小川 貴弘, 長谷山 美紀
電子情報通信学会論文誌 D, 情報・システム, 98, 5, 823, 834, 電子情報通信学会, 2015年05月01日
日本語, 本論文では，走査型電子顕微鏡(Scanning Electron Microscop，SEM)で撮像された画像を用いた生物の分類法を提案する．提案手法では，生物学者が構築した分類体系に注目し，各ノードにその下位のノードへの分類を行う分類器を割り当てることで，決定木を構築する．これにより，構築された決定木を用いて生物の分類が可能となる．このように，生物の分類に有用な分類体系の構造を導入することで，画像特徴のみに注目する生物の分類法と比較して，高精度な分類が期待できる．また，提案手法では，更なる精度向上のため，以下の二つの処理も導入する．まず，分類体系において，画像特徴が類似する異なるノードに注目した決定木の変更を行い，誤分類を抑制する．次に，同種の生物の異なる撮像倍率の画像に対し，モーフィングを施すことで学習データの充足を行う．以上によって，提案手法では，SEMで撮像された生物の高精度な分類が可能となる．本論文の最後では，提案手法の有効性を確認するための実験結果を示す．

ものづくりの発想を支援する―バイオミメティクス・画像検索基盤―
長谷山美紀, 長谷山美紀
現代化学, 529, 31, 34, 2015年04月01日
日本語

生物顕微鏡像からものづくりの発想を支援するバイオミメティクス・画像検索
長谷山美紀
タクサ, 38, 22, 25, 日本動物分類学会, 2015年02月28日
日本語, Biomimetics is a new research area that creates innovation through the collaboration of different existing research fields. Since biomimetics brings together expert researchers with deep knowledge of various research fields, there is a need to facilitate the mutual exchange of that knowledge in order to create new research areas. However, this exchange is difficult due to several reasons, e.g., differences in technical terms between different fields. In order to overcome this problem, we started the development of a new data retrieval platform based on the theory of associative image retrieval. A biological database contains many image data, and by taking advantage of these image data, we are able to overcome limitations of text-only information retrieval. If the development of such a retrieval platform that does not depend on text data can be realized, individual biological databases of various species (insects, fish, etc.) will be integrated. This will allow not only the use for the study of the various species by researchers in different biological fields, but also access for a wide range of researchers in fields ranging from materials science, mechanical engineering and manufacturing.

「バイオミメティクス」日本企業の研究開発の現況:「生物・博物・情報」分野 バイオミメティクス・画像検索システム―生物の微細構造から材料開発の発想を支援する―
長谷山美紀
OHM, 102, 1, 26, 27, 2015年01月12日
日本語

Automatic Martian Dust Storm Detection from Multiple Wavelength Data Based on Decision Level Fusion
Maeda Keisuke, Ogawa Takahiro, Haseyama Miki
Information and Media Technologies, 10, 3, 473, 477, Information and Media Technologies Editorial Board, 2015年
英語, This paper presents automatic Martian dust storm detection from multiple wavelength data based on decision level fusion. In our proposed method, visual features are first extracted from multiple wavelength data, and optimal features are selected for Martian dust storm detection based on the minimal-Redundancy-Maximal-Relevance algorithm. Second, the selected visual features are used to train the Support Vector Machine classifiers that are constructed on each data. Furthermore, as a main contribution of this paper, the proposed method integrates the multiple detection results obtained from heterogeneous data based on decision level fusion, while considering each classifiers detection performance to obtain accurate final detection results. Consequently, the proposed method realizes successful Martian dust storm detection.

NOVEL IMAGE CLASSIFICATION BASED ON INTEGRATION OF EEG AND VISUAL FEATURES VIA MSLPCCA
Takuya Kawakami, Takahiro Ogawa, Miki Haseyama
2015 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING (ICASSP), 957, 961, IEEE, 2015年
英語, 研究論文（国際会議プロシーディングス）, This paper presents a novel image classification method based on integration of EEG and visual features. In the proposed method, we obtain classification results by separately using EEG and visual features. Furthermore, we merge the above classification results based on a kernelized version of Supervised learning from multiple experts and obtain the final classification result. In order to generate feature vectors used for the final image classification, we apply Multiset supervised locality preserving canonical correlation analysis (MSLPCCA), which is newly derived in the proposed method, to EEG and visual features. Our method realizes successful multimodal classification of images by the object categories that they contain based on MSLPCCA-based feature integration.

MISSING INTENSITY RESTORATION VIA ADAPTIVE SELECTION OF PERCEPTUALLY OPTIMIZED SUBSPACES
Takahiro Ogawa, Miki Haseyama
2015 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING (ICASSP), 1628, 1632, IEEE, 2015年
英語, 研究論文（国際会議プロシーディングス）, A missing intensity restoration method via adaptive selection of perceptually optimized subspaces is presented in this paper. In order to realize adaptive and perceptually optimized restoration, the proposed method generates several subspaces of known textures optimized in terms of the structural similarity (SSIM) index. Furthermore, the SSIM-based missing intensity restoration is performed by a projection onto convex sets (POCS) algorithm whose constraints are the obtained subspace and known intensities within the target image. In this approach, a non-convex maximization problem for calculating the projection onto the subspace is reformulated as a quasi-convex problem, and the restoration of the missing intensities becomes feasible. Furthermore, the selection of the optimal subspace is realized by monitoring the SSIM index converged in the POCS algorithm, and the adaptive restoration becomes feasible. Experimental results show that our method outperforms existing methods.

Heterogeneous Graph-based Video Search Reranking using Web Knowledge via Social Media Network
Soh Yoshida, Takahiro Ogawa, Miki Haseyama
MM'15: PROCEEDINGS OF THE 2015 ACM MULTIMEDIA CONFERENCE, 871, 874, ASSOC COMPUTING MACHINERY, 2015年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, Graph-based reranking is effective for refining text-based video search results by making use of the social network structure. Unlike previous works which only focus on an individual video graph, the proposed method leverages the mutual reinforcement of heterogeneous graphs, such as videos and their associated tags obtained by social influence mining. Specifically, propagation of information relevancy across different modalities is performed by exchanging information of inter- and intra-relations among heterogeneous graphs. The proposed method then formulates the video search reranking as an optimization problem from the perspective of Bayesian framework. Furthermore, in order to model the consistency over the modified video graph topology, a local learning regularization with a social community detection scheme is introduced to the framework. Since videos within the same social community have strong semantic correlation, the consistency score estimation becomes feasible. Experimental results obtained by applying the proposed method to a real-world video collection show its effectiveness.

HELICOBACTER PYLORI INFECTION DETECTION FROM MULTIPLE X-RAY IMAGES BASED ON COMBINATION USE OF SUPPORT VECTOR MACHINE AND MULTIPLE KERNEL LEARNING
Kenta Ishihara, Takahiro Ogawa, Miki Haseyama
2015 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 4728, 4732, IEEE, 2015年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a detection method of Helicobacter pylori (H. pylori) infection from multiple gastric X-ray images based on combination use of Support Vector Machine (SVM) and Multiple Kernel Learning (MKL). The proposed method firstly computes some types of visual features from multiple gastric X-ray images taken in several specific directions in order to represent the characteristics of X-ray images with H. pylori infection. Second, based on the minimal-Redundancy-Maximal-Relevance algorithm, we select the effective features for H. pylori infection detection from each type of visual feature and all visual features. The selected features are used to train the SVM classifier and the MKL classifier for each direction of gastric X-ray images. Finally, the proposed method integrates multiple detection results based on a late fusion scheme considering the detection performance of each classifier. Experimental results obtained by applying the proposed method to real X-ray images prove its effectiveness.

AUTOMATIC DETECTION OF MARTIAN DUST STORMS FROM HETEROGENEOUS DATA BASED ON DECISION LEVEL FUSION
Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
2015 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2246, 2250, IEEE, 2015年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents automatic detection of Martian dust storms from heterogeneous data (raw data, reflectance data and background subtraction data of the reflectance data) based on decision level fusion. Specifically, the proposed method first extracts image features from these data and selects optimal features for dust storm detection based on the minimal-Redundancy-Maximal-Relevance algorithm. Second, the selected image features are used to train the Support Vector Machine classifier that is constructed on each data. Furthermore, as a main contribution of this paper, the proposed method combines the multiple detection results obtained from the heterogeneous data based on decision level fusion with considering each classifier's detection performance to obtain accurate final detection results. Consequently, the proposed method realizes automatic and accurate detection of Martian dust storms.

EXTRACTION OF HIERARCHICAL STRUCTURE OF WEB COMMUNITIES INCLUDING SALIENT KEYWORD ESTIMATION FOR WEB VIDEO RETRIEVAL
Ryosuke Harakawa, Takahiro Ogawa, Miki Haseyama
2015 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 1021, 1025, IEEE, 2015年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, In this paper, we present a method for extraction of hierarchical structure of Web communities including salient keyword estimation for Web video retrieval. The following two contributions of the proposed method enable retrieval of the desired Web videos even if users cannot input suitable queries that identify the desired contents. First, our method realizes the extraction of hierarchical structure of Web communities, i.e., Web video sets with similar topics by using heterogeneous features of Web videos and link relationships between Web videos obtained via metadata "related videos". Second, we can estimate salient keywords to identify the contents of each obtained Web community at a glance based on text attached to Web videos such as title, the heterogeneous features of Web videos and the link relationships between Web videos. Experimental results on actual Web videos verify that our method can realize accurate retrieval of the desired Web videos via the hierarchical structure of Web communities with their salient keywords.

Human-centered Favorite Music Estimation: EEG-based Extraction of Audio Features Reflecting Individual Preference
Ryosuke Sawata, Takahiro Ogawa, Miki Haseyama
2015 IEEE INTERNATIONAL CONFERENCE ON DIGITAL SIGNAL PROCESSING (DSP), 818, 822, IEEE, 2015年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a human-centered method for favorite music estimation using EEG-based audio features. In order to estimate user's favorite musical pieces, our method utilizes his/her EEG signals for calculating new audio features suitable for representing the user's music preference. Specifically, projection, which transforms original audio features into the features reflecting the preference, is calculated by applying kernel Canonical Correlation Analysis (CCA) to the audio features and the EEG features which are extracted from the user's EEG signals during listening to favorite musical pieces. By using the obtained projection, the new EEG-based audio features can be derived since this projection provides the best correlation between the user's EEG signals and their corresponding audio signals. Thus, successful estimation of user's favorite musical pieces via a Support Vector Machine (SVM) classifier using the new audio features becomes feasible. Since our method does not need acquisition of EEG signals for obtaining new audio features from new musical pieces after calculating the projection, this indicates the high practicability of our method. Experimental results show that our method outperforms methods using original audio features or EEG features.

Algorithm for Sparse Representation Minimizing Mean Square Error of Power Spectrograms
Yuma Tanaka, Takahiro Ogawa, Miki Haseyama
2015 IEEE INTERNATIONAL CONFERENCE ON DIGITAL SIGNAL PROCESSING (DSP), 618, 622, IEEE, 2015年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, Sparse representation is an idea to approximate a target signal by a linear combination of a small number of sample signals, and it is utilized in various research fields. In this paper, we evaluate the approximation error of signals by the mean square error of power spectrograms (P-MSE). Specifically, we propose a P-MSE minimization algorithm for sparse representation. Our method minimizes the P-MSE by an iterative approach. Specifically, in each iteration, we find the optimal sample signal and optimize the corresponding coefficients by a gradient-based method. In this approach, our method can utilize the result of the previous iteration for fast and stable convergence in the optimization of the coefficients. Based on this algorithm, the sparse representation which minimizes the P-MSE becomes feasible. Experimental results show the effectiveness of our method in terms of the P-MSE minimization.

MISSING INTENSITY RESTORATION VIA ADAPTIVE SELECTION OF PERCEPTUALLY OPTIMIZED SUBSPACES
Takahiro Ogawa, Miki Haseyama
2015 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING (ICASSP), 1628, 1632, IEEE, 2015年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, A missing intensity restoration method via adaptive selection of perceptually optimized subspaces is presented in this paper. In order to realize adaptive and perceptually optimized restoration, the proposed method generates several subspaces of known textures optimized in terms of the structural similarity (SSIM) index. Furthermore, the SSIM-based missing intensity restoration is performed by a projection onto convex sets (POCS) algorithm whose constraints are the obtained subspace and known intensities within the target image. In this approach, a non-convex maximization problem for calculating the projection onto the subspace is reformulated as a quasi-convex problem, and the restoration of the missing intensities becomes feasible. Furthermore, the selection of the optimal subspace is realized by monitoring the SSIM index converged in the POCS algorithm, and the adaptive restoration becomes feasible. Experimental results show that our method outperforms existing methods.

NOVEL IMAGE CLASSIFICATION BASED ON INTEGRATION OF EEG AND VISUAL FEATURES VIA MSLPCCA
Takuya Kawakami, Takahiro Ogawa, Miki Haseyama
2015 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING (ICASSP), 957, 961, IEEE, 2015年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a novel image classification method based on integration of EEG and visual features. In the proposed method, we obtain classification results by separately using EEG and visual features. Furthermore, we merge the above classification results based on a kernelized version of Supervised learning from multiple experts and obtain the final classification result. In order to generate feature vectors used for the final image classification, we apply Multiset supervised locality preserving canonical correlation analysis (MSLPCCA), which is newly derived in the proposed method, to EEG and visual features. Our method realizes successful multimodal classification of images by the object categories that they contain based on MSLPCCA-based feature integration.

Random combination for information extraction in compressed sensing and sparse representation-based pattern recognition
Zaixing He, Xinyue Zhao, Shuyou Zhang, Takahiro Ogawa, Miki Haseyama
NEUROCOMPUTING, 145, 160, 173, ELSEVIER SCIENCE BV, 2014年12月, ［査読有り］
英語, 研究論文（学術雑誌）, In compressed sensing and sparse representation-based pattern recognition, random projection with a dense random transform matrix is widely used for information extraction. However, the complicated structure makes dense random matrices computationally expensive and difficult in hardware implementation. This paper considers the simplification of the random projection method. First, we propose a simple random method, random combination, for information extraction to address the issues of dense random methods. The theoretical analysis and the experimental results show that it can provide comparable performance to those of dense random methods. Second, we analyze another simple random method, random choosing, and give its applicable occasions. The comparative analysis and the experimental results show that it works well in dense cases but worse in sparse cases. Third, we propose a practical method for measuring the effectiveness of the feature transform matrix in sparse representation-based pattern recognition. A matrix satisfying the Representation Residual Restricted Isometry Property can provide good recognition results. (C) 2014 Elsevier B.V. All rights reserved.

Adaptive missing texture reconstruction method based on kernel cross-modal factor analysis with a new evaluation criterion
Takahiro Ogawa, Mild Haseyama
SIGNAL PROCESSING, 103, 69, 83, ELSEVIER SCIENCE BV, 2014年10月, ［査読有り］
英語, 研究論文（学術雑誌）, This paper presents an adaptive missing texture reconstruction method based on kernel cross-modal factor analysis (KCFA) with a new evaluation criterion. The proposed method estimates the latent relationship between two areas, which correspond to a missing area and its neighboring area, respectively, from known parts within the target image and realizes reconstruction of the missing textures. In order to obtain this relationship, KCFA is applied to each cluster containing similar known textures, and the optimal cluster is used for reconstructing each target missing area. Specifically, a new criterion obtained by monitoring errors caused in the latent space enables selection of the optimal cluster. Then each missing texture is adaptively estimated by the optimal cluster's latent relationship, which enables accurate reconstruction of similar textures. In our method, the above criterion is also used for estimating patch priority, which determines the reconstruction order of missing areas within the target image. Since patches, whose textures are accurately modeled by our KCFA-based method, can be selected by using the new criterion, it becomes feasible to perform successful reconstruction of the missing areas. Experimental results show improvements of our KCFA-based reconstruction method over previously reported methods. (C) 2013 Elsevier B.V. All rights reserved.

Player Tracking in Far-View Soccer Videos Based on Composite Energy Function
Kazuya Iwai, Sho Takahashi, Takahiro Ogawa, Miki Haseyama
IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, E97D, 7, 1885, 1892, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2014年07月, ［査読有り］
英語, 研究論文（学術雑誌）, In this paper, an accurate player tracking method in far-view soccer videos based on a composite energy function is presented. In far-view soccer videos, player tracking methods that perform processing based only on visual features cannot accurately track players since each player region becomes small, and video coding causes color bleeding between player regions and the soccer field. In order to solve this problem, the proposed method performs player tracking on the basis of the following three elements. First, we utilize visual features based on uniform colors and player shapes. Second, since soccer players play in such a way as to maintain a formation, which is a positional pattern of players, we use this characteristic for player tracking. Third, since the movement direction of each player tends to change smoothly in successive frames of soccer videos, we also focus on this characteristic. Then we adopt three energies: a potential energy based on visual features, an elastic energy based on formations and a movement direction-based energy. Finally, we define a composite energy function that consists of the above three energies and track players by minimizing this energy function. Consequently, the proposed method achieves accurate player tracking in far-view soccer videos.

A new method for error degree estimation in numerical weather prediction via MKDA-based ordinal regression
Takahiro Ogawa, Shintaro Takahashi, Sho Takahashi, Miki Haseyama
EURASIP JOURNAL ON ADVANCES IN SIGNAL PROCESSING, 2014, 115, 115, SPRINGER INTERNATIONAL PUBLISHING AG, 2014年07月, ［査読有り］
英語, 研究論文（学術雑誌）, This paper presents a new method for estimating error degrees in numerical weather prediction via multiple kernel discriminant analysis (MKDA)-based ordinal regression. The proposed method tries to estimate how large prediction errors will occur in each area from known observed data. Therefore, ordinal regression based on KDA is used for estimating the prediction error degrees. Furthermore, the following points are introduced into the proposed approach. Since several meteorological elements are related to each other based on atmospheric movements, the proposed method merges such heterogeneous features in the target and neighboring areas based on a multiple kernel algorithm. This approach is based on the characteristics of actual meteorological data. Then, MKDA-based ordinal regression for estimating the prediction error degree of a target meteorological element in each area becomes feasible. Since the amount of training data obtained from known observed data becomes very large in the training stage of MKDA, the proposed method performs simple sampling of those training data to reduce the number of samples. We effectively use the remaining training data for determining the parameters of MKDA to realize successful estimation of the prediction error degree.

歌謡番組における映像の構造に注目したシーン分割手法
吉田壮, 小川貴弘, 長谷山美紀
電子情報通信学会論文誌 D(Web), J97-D, 7, WEB ONLY 1177-1188, 2014年07月01日
日本語

バイオミメティクスエンジニアリング 発想を支援するバイオミメティクス・データ検索と技術革新
長谷山美紀
化学工学, 78, 6, 394, 397, 2014年06月05日
日本語

A Cross-Modal Approach for Extracting Semantic Relationships Between Concepts Using Tagged Images
Marie Katsurai, Takahiro Ogawa, Miki Haseyama
IEEE TRANSACTIONS ON MULTIMEDIA, 16, 4, 1059, 1074, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2014年06月, ［査読有り］
英語, 研究論文（学術雑誌）, This paper presents a cross-modal approach for extracting semantic relationships between concepts using tagged images. In the proposed method, we first project both text and visual features of the tagged images to a latent space using canonical correlation analysis (CCA). Then, under the probabilistic interpretation of CCA, we calculate a representative distribution of the latent variables for each concept. Based on the representative distributions of the concepts, we derive two types of measures: the semantic relatedness between the concepts and the abstraction level of each concept. Because these measures are derived from a cross-modal scheme that enables the collaborative use of both text and visual features, the semantic relationships can successfully reflect semantic and visual contexts. Experiments conducted on tagged images collected from Flickr show that our measures are more coherent to human cognition than the conventional measures that use either text or visual features, or the WordNet-based measures. In particular, a new measure of semantic relatedness, which satisfies the triangle inequality, obtains the best results among different distance measures in our framework. The applicability of our measures to multimedia-related tasks such as concept clustering, image annotation and tag recommendation is also shown in the experiments.

メディア理解のための発想支援型検索の試み(パターン認識・メディア理解における組み合わせと統合,文化の振興と教育)
長谷山 美紀
電子情報通信学会技術研究報告. PRMU, パターン認識・メディア理解, 113, 493, 73, 77, 一般社団法人電子情報通信学会, 2014年03月06日
日本語, 本文では,画像や映像などの非構造化データを有機的に連携し,互いの類似性を効果的に抽出し提示することで,検索者に気づきを与え,発想を支援する検索システムについて紹介する.このような検索システムの実現のためには,マルチモーダル処理が有効である.マルチモーダル処理は,音声信号や楽曲信号などの音響情報,画像・映像情報や行動履歴等,複数の異なるメディアから得られるデータを効率的に利用することで,単一の信号を用いるだけでは解決が困難であった問題に新たな解法を与える.本文では,マルチモーダル処理を用いた映像検索について紹介し,望むコンテンツを効率的に取得するために発想を支援する検索システムの実現の試みについて説明する.

映像の構造に注目したMCMC法に基づくシーン分割法
SONG Yan, 小川貴弘, 長谷山美紀
電子情報通信学会論文誌 D, J97-D, 3, 560, 573, 2014年03月
日本語

映像の構造に注目したMCMC法に基づくシーン分割法(画像・映像処理,<特集>学生論文)
宋 妍, 小川 貴弘, 長谷山 美紀
電子情報通信学会論文誌. D, 情報・システム, 97, 3, 560, 573, 一般社団法人電子情報通信学会, 2014年03月01日
日本語, 以前,我々は,映像の類似したショットが隣接せずに出現する構造(映像の構造)に注目したシーン分割手法を提案した.しかしながら,映像の構造に注目したシーン分割手法の共通の問題として,映像の構造を取得する際に用いられるシーンの最長時間幅の設定が困難であるという点が存在した.そこで,本論文では,その改良手法として映像の構造に注目したMCMC法に基づくシーン分割手法を提案する.提案手法では,シーンの最長時間幅を変化させて映像の構造に基づき取得された全ての境界をシーン境界候補とし,それらの中からMCMC法に基づいて最適なシーン境界を推定する.これにより,従来の手法を適用する際に設定が困難であったシーンの最長時間幅を一意に決定することなく,MCMC法により最適なシーン境界を求めることが可能となる.本論文の最後では,実際にテレビで放送された映像に対して提案手法を適用し,その有効性を確認する.

MFCC extraction in AAC domain for audio content analysis
Ai Haojun, Miki Haseyama, Wang Kang
INFORMATION SCIENCE AND MANAGEMENT ENGINEERING, VOLS 1-3, 46, 1413, 1420, WIT PRESS, 2014年
英語, 研究論文（国際会議プロシーディングス）, We focus the attention on the extraction of Mel-frequency cepstral coefficients (MFCC) features from MDCT spectrum in AAC domain for audio content analysis. In particular, a MFCC extraction method is proposed, which is adaptive to the window switch in AAC encoding process, and independent of the audio sampling frequency. We discuss the fusion method of MFCC features from different window type in order to keep the balance of the frequency and temporal resolution. The audio scene segmentation and audio classification experimental results show that such approach based on compression domain can approach the performance of the system based on PCM audio, and the CPU overload decreased dramatically. It is meaningful to the real-time analysis of audio content.

HELICOBACTER PYLORI INFECTION DETECTION FROM MULTIPLE X-RAY IMAGES BASED ON DECISION LEVEL FUSION
Kenta Ishihara, Takahiro Ogawa, Miki Haseyama
2014 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2769, 2773, IEEE, 2014年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents an automatic detection method of Helicobacter pylori (H. pylori) infection from multiple gastric X-ray images. As the biggest contribution of this paper, we combine multiple detection results based on a decision level fusion. In order to obtain multiple detection results, the proposed method first focuses on characteristics of gastric X-ray images with H. pylori infection and computes several visual features from multiple X-ray images taken in several specific directions. Second, we select effective features for H. pylori infection detection from all features based on the minimal-Redundancy-Maximal-Relevance algorithm, and the selected features are used to train the Support Vector Machine (SVM) classifiers that are constructed for each direction of gastric radiography. Therefore, the detection of H. pylori infection becomes feasible, and we can obtain multiple detection results from the SVM classifiers. Furthermore, we combine multiple detection results based on the decision level fusion scheme considering the detection performance of each SVM classifier. Experimental results obtained by applying the proposed method to real X-ray images prove the effectiveness of the proposed method.

2D SEMI-SUPERVISED CCA-BASED INPAINTING INCLUDING NEW PRIORITY ESTIMATION
Takahiro Ogawa, Miki Haseyama
2014 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 1837, 1841, IEEE, 2014年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents an inpainting method based on 2D semi-supervised canonical correlation analysis (2D semi-CCA) including new priority estimation. The proposed method estimates relationship, i.e., the optimal correlation, between missing area and its neighboring area from known parts within the target image by using 2D CCA. In this approach, we newly introduce a semi-supervised scheme into the 2D CCA for deriving the 2D semi-CCA which corresponds to a hybrid version of 2D CCA and 2D principle component analysis (2D PCA). This enables successful relationship estimation even if sufficient number of training pairs cannot be provided. Then, by using the obtained relationship, accurate estimation of the missing intensities can be realized. Furthermore, in the proposed method, errors caused in the new variate space obtained by the 2D semi-CCA are effectively used for deriving patch priority determining inpainting order of missing areas. Experimental results show our inpainting method can outperform previously reported methods.

NOVEL IMAGE CLASSIFICATION BASED ON DECISION-LEVEL FUSION OF EEG AND VISUAL FEATURES
Takuya Kawakami, Takahiro Ogawa, Miki Haseyama
2014 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 5874, 5878, IEEE, 2014年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a novel image classification based on decision-level fusion of EEG and visual features. In the proposed method, we extract the EEG features from EEG signals recorded while users stare at images, and the visual features are computed from these images. Then the classification of images is performed based on Support Vector Machine (SVM) by separately using the EEG and visual features. Furthermore, we merge the above classification results based on Supervised Learning from Multiple Experts to obtain the final classification result. This method focuses on the classification accuracy calculated from each classification result. Therefore, although classification accuracy based on EEG and visual features are different from each other, our method realizes effective integration of these classification results. In addition, we newly derive a kernelized version of the method in order to realize more accurate integration of the classification results. Consequently, our method realizes successful multimodal classification of images by the object categories that they contain.

MISSING INTENSITY RESTORATION VIA PERCEPTUALLY OPTIMIZED SUBSPACE PROJECTION BASED ON ENTROPY COMPONENT ANALYSIS
Takahiro Ogawa, Miki Haseyama
2014 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 175, 179, IEEE, 2014年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, A missing intensity restoration method via perceptually optimized subspace projection based on entropy component analysis (ECA) is presented in this paper. The proposed method calculates the optimal subspace of known patches within a target image based on structural similarity (SSIM) index, and the optimal bases are determined based on ECA. Then missing intensity estimation whose results maximize the SSIM index is realized by using a projection onto convex sets (POCS) algorithm whose constraints are the obtained subspace and known intensities within the target image. In this approach, a non-convex maximization problem for calculating the projection onto the subspace is reformulated as a quasi-convex problem, and the restoration of the missing intensities becomes feasible. Experimental results show that our restoration method outperforms previously reported methods.

Image inpainting based on sparse representations with a perceptual metric
Takahiro Ogawa, Miki Haseyama
EURASIP JOURNAL ON ADVANCES IN SIGNAL PROCESSING, 2013, 179, 179, SPRINGER INTERNATIONAL PUBLISHING AG, 2013年12月, ［査読有り］
英語, 研究論文（学術雑誌）, This paper presents an image inpainting method based on sparse representations optimized with respect to a perceptual metric. In the proposed method, the structural similarity (SSIM) index is utilized as a criterion to optimize the representation performance of image data. Specifically, the proposed method enables the formulation of two important procedures in the sparse representation problem, 'estimation of sparse representation coefficients' and 'update of the dictionary', based on the SSIM index. Then, using the generated dictionary, approximation of target patches including missing areas via the SSIM-based sparse representation becomes feasible. Consequently, image inpainting for which procedures are totally derived from the SSIM index is realized. Experimental results show that the proposed method enables successful inpainting of missing areas.

Exploring and visualizing tag relationships in photo sharing websites based on distributional representations
Marie Katsurai, Miki Haseyama
ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, 3617, 3621, IEEE, 2013年10月18日, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a method for exploring and visualizing tag relationships in photo sharing websites based on distributional representations of tags. First, we find a representative distribution of a tag, which is summarized by the mean and covariance, using features of tagged photos. This distributional representation can jointly consider the semantic meaning of tags and their abstraction levels. Then, based on the representative distributions, we derive two kinds of semantic measures on tag relationships. The extracted information is visualized in a graphical network to facilitate the understanding of tag usage. Experiments conducted using tagged photos collected from Flickr show that our tag network is more coherent to human cognition than other networks constructed by conventional methods. © 2013 IEEE.

Vocal segment estimation in music pieces based on collaborative use of EEG and audio features
Takuya Kawakami, Takahiro Ogawa, Miki Haseyama
ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, 1197, 1201, IEEE, 2013年10月18日, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a novel estimation method of segments including vocals in music pieces based on collaborative use of features extracted from electroencephalogram (EEG) signals recorded while users are listening to music pieces and features extracted from these audio signals. From extracted EEG features and audio features, we estimate segments including vocals based on Support Vector Machine (SVM) by separately utilizing these two features. Furthermore, the final classification results are obtained by integrating these estimation results based on supervised learning from multiple experts. Therefore, our method realizes multimodal estimation of segments including vocals in music pieces. Experimental results show the improvement of our method over the methods utilizing only EEG or audio features. © 2013 IEEE.

胃X線検査における胃背景粘膜の自動解析による胃がんリスク評価の検討
間部克裕, 長谷山美紀, 小川貴弘, 吉澤和哉, 大泉晴史, 中島滋美, 加藤元嗣
日本消化器がん検診学会雑誌, 51, 3, 96, 2013年05月15日
日本語

レベルセット法を用いたサッカー映像における選手追跡手法(画像認識,コンピュータビジョン,<特集>学生論文)
高橋 翔, 林 原局, 長谷山 美紀
電子情報通信学会論文誌. D, 情報・システム, 96, 3, 695, 703, 一般社団法人電子情報通信学会, 2013年03月, ［査読有り］
日本語, 本論文では,サッカー映像からレベルセット法を用いて選手を追跡する手法を提案する.提案手法では,サッカー映像を各フレームが時間軸方向に重なるように連結した三次元データとして扱う.このデータに対して,レベルセット法を適用することで抽出される三次元の領域は,複数フレームに渡って存在する同一選手を包含する.提案手法では,この三次元の領域をサッカー映像から抽出することで,選手の追跡を実現する.したがって,提案手法では,フレームごとに選手を検出する必要がないため,フレームを個別に処理する従来手法における選手の検出と追跡それぞれの誤差によって精度が低下する問題を解決可能である.また,我々は,ユニフォームの色成分をサッカー映像から色コリログラムを用いて推定し,これをレベルセット法を用いて追跡する選手の特徴として導入する.これにより,提案手法では,追跡対象の特徴を事前に与えることなく,選手の頑健な追跡が可能となる.本文の最後では,実際にテレビで放送されたサッカー映像に対する実験により,提案手法の有効性を確認する.

Missing Texture Reconstruction Method Based on Error Reduction Algorithm Using Fourier Transform Magnitude Estimation Scheme
Takahiro Ogawa, Miki Haseyama
IEEE TRANSACTIONS ON IMAGE PROCESSING, 22, 3, 1252, 1257, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2013年03月, ［査読有り］
英語, 研究論文（学術雑誌）, A missing texture reconstruction method based on an error reduction (ER) algorithm, including a novel estimation scheme of Fourier transform magnitudes is presented in this brief. In our method, Fourier transform magnitude is estimated for a target patch including missing areas, and the missing intensities are estimated by retrieving its phase based on the ER algorithm. Specifically, by monitoring errors converged in the ER algorithm, known patches whose Fourier transform magnitudes are similar to that of the target patch are selected from the target image. In the second approach, the Fourier transform magnitude of the target patch is estimated from those of the selected known patches and their corresponding errors. Consequently, by using the ER algorithm, we can estimate both the Fourier transform magnitudes and phases to reconstruct the missing areas.

Super-resolution for simultaneous realization of resolution enhancement and motion blur removal based on adaptive prior settings
Takahiro Ogawa, Daisuke Izumi, Akane Yoshizaki, Miki Haseyama
EURASIP JOURNAL ON ADVANCES IN SIGNAL PROCESSING, 2013, 1, 17, SPRINGER INTERNATIONAL PUBLISHING AG, 2013年02月, ［査読有り］
英語, 研究論文（学術雑誌）, A super-resolution method for simultaneously realizing resolution enhancement and motion blur removal based on adaptive prior settings are presented in this article. In order to obtain high-resolution (HR) video sequences from motion-blurred low-resolution video sequences, both of the resolution enhancement and the motion blur removal have to be performed. However, if one is performed after the other, errors in the first process may cause performance deterioration of the subsequent process. Therefore, in the proposed method, a new problem, which simultaneously performs the resolution enhancement and the motion blur removal, is derived. Specifically, a maximum a posterior estimation problem which estimates original HR frames with motion blur kernels is introduced into our method. Furthermore, in order to obtain the posterior probability based on Bayes' rule, a prior probability of the original HR frame, whose distribution can adaptively be set for each area, is newly defined. By adaptively setting the distribution of the prior probability, preservation of the sharpness in edge regions and suppression of the ringing artifacts in smooth regions are realized. Consequently, based on these novel approaches, the proposed method can perform successful reconstruction of the HR frames. Experimental results show impressive improvements of the proposed method over previously reported methods.

Randomized-Controlled Study on the Effects of Hand Bathing on Stroke Patients in Convalescence - Subjective Improvements in Hand Movement and Increased Positiveness in Talking
Rika Yano, Miki Haseyama
STROKE, 44, 2, LIPPINCOTT WILLIAMS & WILKINS, 2013年02月
英語

Active grid-based method for visualizing pass regions in soccer videos
Sho Takahashi, Miki Haseyama
Electronic Proceedings of the 2013 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2013, 1, 6, IEEE, 2013年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a method for visualizing pass regions that have high probabilities of the pass succeeding from broadcast soccer videos. In soccer matches, players discover pass regions based on player position geometry and player velocities. Therefore, by using player position geometry and player velocities, which are obtained from a broadcast soccer video, we can visualize pass regions. The proposed method is realized by the following two steps. 1) Generation of new three-dimensional data (volume data) for analyzing pass regions, which are not visible. 2) Visualization of pass regions. In the first step, volume data are generated from player position geometry and player velocities. By generating the volume data, which express the player position geometry and the player velocities, analysis of invisible pass regions is enabled. In the second step, by applying Active grid to the generated volume data, pass regions are visualized. Specifically, lattice points of the Active grid converge to the pass regions. Therefore, positions of the pass regions on the pitch can be visualized from densities of the lattice points. In the experiment, the proposed method is applied to actual TV programs to verify its effectiveness. © 2013 IEEE.

An extraction method of hierarchical Web communities for Web video retrieval
Ryosuke Harakawa, Yasutaka Hatakeyama, Takahiro Ogawa, Miki Haseyama
2013 IEEE International Conference on Image Processing, ICIP 2013 - Proceedings, 4397, 4401, IEEE, 2013年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents an extraction method of hierarchical Web communities for Web video retrieval. In the proposed method, Web communities containing Web videos whose topics are similar to each other are extracted by using hyperlinks between Web pages including Web videos and their video features. Furthermore, we focus on graph structure of hyperlinks between Web pages including Web videos which belong to the Web communities. Then, by using strongly connected components and betweenness centrality of the graph, hierarchical structure of the Web communities can be estimated. Consequently, users can easily find Web videos including related topics in each hierarchy, and desired Web videos can be effectively retrieved. © 2013 IEEE.

Insect classification using Scanning Electron Microphotographs considering magnifications
Akihiro Takahashi, Takahiro Ogawa, Miki Haseyama
2013 IEEE International Conference on Image Processing, ICIP 2013 - Proceedings, 3269, 3273, IEEE, 2013年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a method of insect classification using images taken by Scanning Electron Microscope (SEM) considering magnifications. Generally, when images of the same insects are taken by SEM with different magnifications, visual features of these images are different from each other. Thus, the proposed method adopts a new scheme which groups images of different magnifications in such a way that the classification performance becomes the highest. Then a classifier is constructed for each group, and the insect classification becomes feasible based on a target image magnification. In addition, by integrating the classification results of several images obtained from the same sample, i.e., the same insect, performance improvement of the insect classification considering magnifications can be realized. Experimental results show the effectiveness of the proposed method. © 2013 IEEE.

SPECTRAL REFLECTANCE ESTIMATION FROM VISIBLE LIGHT COMPONENTS AND NEAR-INFRARED COMPONENTS
Yuta Igarashi, Takahiro Ogawa, Miki Haseyama
2013 20TH IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP 2013), 2388, 2392, IEEE, 2013年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a novel method for estimating a spectral reflectance from two kinds of input images: an image including both visible light components and near-infrared (NIR) components, and an image including only NIR components. From these input images, we estimate the spectral reflectance based on the Non-negative Matrix Factorization algorithm using spectral sensitivities of a digital camera. The estimated spectral reflectance enables several important applications. In this paper, the e ff ectiveness of the proposed method is verified by using the estimated spectral reflectance in the two image processing applications.

KCFA-BASED MISSING AREA RESTORATION INCLUDING NEW PRIORITY ESTIMATION
Takahiro Ogawa, Miki Haseyama
2013 20TH IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP 2013), 704, 708, IEEE, 2013年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, A kernel cross-modal factor analysis (KCFA) based missing area restoration method including a new priority estimation scheme is presented in this paper. The proposed method estimates latent relationship between missing areas and their neighboring areas by deriving projection matrices minimizing their errors in the latent space based on KCFA. This latent relationship represented by the derived projection matrices is optimal for accurately restoring missing areas within the target image. Furthermore, the proposed method adopts a new priority estimation scheme which determines the restoration order of missing areas. Specifically, this priority is estimated based on the criterion representing the restoration performance derived from KCFA, and it enables adaptive selection of missing areas successfully restored by our method. Consequently, it becomes feasible to accurately perform the restoration of missing areas by using the proposed KCFA-based method. Experimental results show subjective and quantitative improvements of the proposed method over previously reported restoration methods.

Trial Realization of Human-Centered Multimedia Navigation for Video Retrieval
Miki Haseyama, Takahiro Ogawa
INTERNATIONAL JOURNAL OF HUMAN-COMPUTER INTERACTION, 29, 2, 96, 109, TAYLOR & FRANCIS INC, 2013年01月, ［査読有り］
英語, 研究論文（学術雑誌）, A trial realization of human-centered navigation for video retrieval is presented in this article. This system consists of the following functions: (a) multimodal analysis for collaborative use of multimedia data, (b) preference extraction for the system to adapt to users' individual demands, and (c) adaptive visualization for users to be guided to their desired contents. By using these functions, users can find their desired video contents more quickly and accurately than with the conventional retrieval schemes since our system can provide new pathways to the desired contents. Experimental results verify the effectiveness of the proposed system.

車載複数カメラによる夜間雨天時における路面反射グレア領域の検出と画像改善
公文宏明, 長谷山美紀
映像情報メディア学会誌(Web), 67, 3, 2013年

Performance of Spatial and Temporal Error Concealment Method for 3D DWT Video Coding in Packet Loss Channel
Hirokazu Tanaka, Sunmi Kim, Takahiro Ogawa, Miki Haseyama
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, E95A, 11, 2015, 2022, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2012年11月, ［査読有り］
英語, 研究論文（学術雑誌）, A new spatial and temporal error concealment method for three-dimensional discrete wavelet transform (3D DWT) video coding is analyzed. 3D DWT video coding employing dispersive grouping (DG) and two-step error concealment is an efficient method in a packet loss channel [20], [21]. In the two-step error concealment method, the interpolations are only spatially applied however, higher efficiency of the interpolation can be expected by utilizing spatial and temporal similarities. In this paper, we propose an enhanced spatial and temporal error concealment method in order to achieve higher error concealment (EC) performance in packet loss networks. In the temporal error concealment method, structural similarity (SSIM) index is employed for inter group of pictures (GOP) EC and minimum mean square error (MMSE) is used for intra GOP EC. Experimental results show that the proposed method can obtain remarkable performance compared with the conventional methods.

Super-Resolution Reconstruction for Spatio-Temporal Resolution Enhancement of Video Sequences
Miki Haseyama, Daisuke Izumi, Makoto Takizawa
IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, E95D, 9, 2355, 2358, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2012年09月, ［査読有り］
英語, 研究論文（学術雑誌）, A method for spatio-temporal resolution enhancement of video sequences based on super-resolution reconstruction is proposed. A new observation model is defined for accurate resolution enhancement, which enables subpixel motion in intermediate frames to be obtained. A modified optimization formula for obtaining a high-resolution sequence is also adopted.

回路理論における伝送と振動(第7回)インピーダンス変成比をもつ帯域通過フィルタ
永井 信夫, 任 捷, 長谷山 美紀
信号処理, 16, 5, 359, 368, [信号処理学会], 2012年09月
日本語

回路理論における伝送と振動(第6回)回路理論に基づく左手系回路の再考
永井 信夫, 任 捷, 長谷山 美紀
信号処理, 16, 4, 263, 272, [信号処理学会], 2012年07月
日本語

A Novel Framework for Extracting Visual Feature-Based Keyword Relationships from an Image Database
Marie Katsurai, Takahiro Ogawa, Miki Haseyama
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, E95A, 5, 927, 937, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2012年05月, ［査読有り］
英語, 研究論文（学術雑誌）, In this paper, a novel framework for extracting visual feature-based keyword relationships from an image database is proposed. From the characteristic that a set of relevant keywords tends to have common visual features, the keyword relationships in a target image database are extracted by using the following two steps. First, the relationship between each keyword and its corresponding visual features is modeled by using a classifier. This step enables detection of visual features related to each keyword. In the second step, the keyword relationships are extracted from the obtained results. Specifically, in order to measure the relevance between two keywords, the proposed method removes visual features related to one keyword from training images and monitors the performance of the classifier obtained for the other keyword. This measurement is the biggest difference from other conventional methods that focus on only keyword co-occurrences or visual similarities. Results of experiments conducted using an image database showed the effectiveness of the proposed method.

回路理論における伝送と振動(第5回)差分方程式の拡張による1次元格子および左手系回路
永井 信夫, 任 捷, 長谷山 美紀
信号処理, 16, 3, 187, 196, [信号処理学会], 2012年05月
日本語

講義シリーズ 回路理論における伝送と振動(第4回)影像位相がπ/2で生じる固有振動と共鳴
永井 信夫, 任 捷, 長谷山 美紀
信号処理, 16, 2, 101, 110, [信号処理学会], 2012年03月
日本語

研究室紹介 長谷山研究室(北海道大学大学院情報科学研究科)
長谷山 美紀
信号処理, 16, 2, 121, 126, [信号処理学会], 2012年03月
日本語

顕微鏡画像を用いたSVDDによる深海底生物の分類体系の推定法
長谷川尭史, 小川貴弘, 渡邉日出海, 長谷山美紀
電子情報通信学会技術研究報告, 111, 442(IE2011 105-132), 73, 78, 2012年02月
日本語

A cross-modal approach for extracting semantic relationships of concepts from an image database
Marie Katsurai, Takahiro Ogawa, Miki Haseyama
ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, 2373, 2376, IEEE, 2012年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a cross-modal approach for extracting semantic relationships of concepts from an image database. First, canonical correlation analysis (CCA) is used to capture the cross-modal correlations between visual features and tag features in the database. Then, in order to measure inter-concept relationships and estimate semantic levels, the proposed method focuses on the distributions of images under the probabilistic interpretation of CCA. Results of experiments conducted by using an image database showed the improvement of the proposed method over existing methods. © 2012 IEEE.

PERCEPTUALLY OPTIMIZED SUBSPACE ESTIMATION FOR MISSING TEXTURE RECONSTRUCTION
Takahiro Ogawa, Miki Haseyama
2012 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 1141, 1144, IEEE, 2012年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a perceptually optimized subspace estimation method for missing texture reconstruction. The proposed method calculates the optimal subspace of known patches within a target image based on structural similarity (SSIM) index instead of calculating mean square error (MSE)-based eigenspace. Furthermore, from the obtained subspace, missing texture reconstruction whose results maximize the SSIM index is performed. In this approach, the non-convex maximization problem is reformulated as a quasi convex problem, and the reconstruction of the missing textures becomes feasible. Experimental results show that our method overcomes previously reported MSE-based reconstruction methods.

回路理論における伝送と振動(第3回)連成振動とその拡張に関係する固有振動
永井 信夫, 任 捷, 長谷山 美紀
信号処理, 16, 1, 13, 22, [信号処理学会], 2012年01月
日本語

8-3 選手の移動速度を考慮したサッカー映像における3次元パス可能領域の推定に関する検討(第8部門 メディア工学3)
高橋 翔, 長谷山 美紀
映像情報メディア学会冬季大会講演予稿集, 2012, 8, 3-1, 一般社団法人 映像情報メディア学会, 2012年
日本語, This paper realizes an improvement of 3D pass region estimation method by using player velocity in soccer videos. In the previous method, since the pass region was estimated regardless of player velocity, the accuracy was limited. Therefore, by introducing the player velocity to the pass region estimation, we improve the performance of the previous method.

SVDDを用いた顕微鏡画像からの新種深海底生物の検出および分類体系上の位置の推定法
長谷川尭史, 小川貴弘, 渡邉日出海, 長谷山美紀
映像情報メディア学会誌(Web), 66, 7, 2012年

Missing Image Data Reconstruction Based on Adaptive Inverse Projection via Sparse Representation
Takahiro Ogawa, Miki Haseyama
IEEE TRANSACTIONS ON MULTIMEDIA, 13, 5, 974, 992, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2011年10月, ［査読有り］
英語, 研究論文（学術雑誌）, In this paper, a missing image data reconstruction method based on an adaptive inverse projection via sparse representation is proposed. The proposed method utilizes sparse representation for obtaining low-dimensional subspaces that approximate target textures including missing areas. Then, by using the obtained low-dimensional subspaces, inverse projection for reconstructing missing areas can be derived to solve the problem of not being able to directly estimate missing intensities. Furthermore, in this approach, the proposed method monitors errors caused by the derived inverse projection, and the low-dimensional subspaces optimal for target textures are adaptively selected. Therefore, we can apply adaptive inverse projection via sparse representation to target missing textures, i.e., their adaptive reconstruction becomes feasible. The proposed method also introduces some schemes for color processing into the calculation of subspaces on the basis of sparse representation and attempts to avoid spurious color caused in the reconstruction results. Consequently, successful reconstruction of missing areas by the proposed method can be expected. Experimental results show impressive improvement of our reconstruction method over previously reported reconstruction methods.

Cross Low-Dimension Pursuit for Sparse Signal Recovery from Incomplete Measurements Based on Permuted Block Diagonal Matrix
Zaixing He, Takahiro Ogawa, Miki Haseyama
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, E94A, 9, 1793, 1803, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2011年09月, ［査読有り］
英語, 研究論文（学術雑誌）, In this paper, a novel algorithm, Cross Low-dimension Pursuit, based on a new structured sparse matrix, Permuted Block Diagonal (PBD) matrix, is proposed in order to recover sparse signals from incomplete linear measurements. The main idea of the proposed method is using the PBD matrix to convert a high-dimension sparse recovery problem into two (or more) groups of highly low-dimension problems and crossly recover the entries of the original signal from them in an iterative way. By sampling a sufficiently sparse signal with a PBD matrix, the proposed algorithm can recover it efficiently. It has the following advantages over conventional algorithms: (1) low complexity, i.e., the algorithm has linear complexity, which is much lower than that of existing algorithms including greedy algorithms such as Orthogonal Matching Pursuit and (2) high recovery ability, i.e., the proposed algorithm can recover much less sparse signals than even l(1)-norm minimization algorithms. Moreover, we demonstrate both theoretically and empirically that the proposed algorithm can reliably recover a sparse signal from highly incomplete measurements.

回路理論における伝送と振動(第2回)2種類の原子からなる1次元格子の固有振動と共鳴
永井 信夫, 任 捷, 長谷山 美紀
信号処理, 15, 5, 331, 340, 〔信号処理学会〕, 2011年09月
日本語

Adaptive Single Image Superresolution Approach Using Support Vector Data Description
Takahiro Ogawa, Miki Haseyama
EURASIP JOURNAL ON ADVANCES IN SIGNAL PROCESSING, 2011, SPRINGER INTERNATIONAL PUBLISHING AG, 2011年, ［査読有り］
英語, 研究論文（学術雑誌）, An adaptive single image superresolution (SR) method using a support vector data description (SVDD) is presented. The proposed method represents the prior on high-resolution (HR) images by hyperspheres of the SVDD obtained from training examples and reconstructs HR images from low-resolution (LR) observations based on the following schemes. First, in order to perform accurate reconstruction of HR images containing various kinds of objects, training HR examples are previously clustered based on the distance from a center of a hypersphere obtained for each cluster. Furthermore, missing high-frequency components of the target image are estimated in order that the reconstructed HR image minimizes the above distances. In this approach, the minimized distance obtained for each cluster is utilized as a criterion to select the optimal hypersphere for estimating the high-frequency components. This approach provides a solution to the problem of conventional methods not being able to perform adaptive estimation of the high-frequency components. In addition, local patches in the target low-resolution (LR) image are utilized as the training HR examples from the characteristic of self-similarities between different resolution levels in general images, and our method can perform the SR without utilizing any other HR images.

ADAPTIVE KPCA-BASED MISSING TEXTURE RECONSTRUCTION APPROACH INCLUDING CLASSIFICATION SCHEME VIA DIFFERENCE SUBSPACES
Takahiro Ogawa, Miki Haseyama
2011 18TH IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 1133, 1136, IEEE, 2011年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents an adaptive kernel principal component analysis (KPCA) based missing texture reconstruction approach including a classification scheme via difference subspaces. The proposed method utilizes a KPCA-based nonlinear eigenspace, which is obtained from each kind of known texture within a target image, as a constraint for reconstructing missing textures with a constraint of known neighboring areas. Then since these two constraints are convex, we can estimate missing textures based on a projection onto convex sets (POCS) algorithm. Furthermore, in this approach, the proposed method derives a new criterion for selecting the optimal eigenspace by monitoring errors caused in the projection via a difference subspace of each kind of known texture. This provides a solution to conventional problems of not being able to perform accurate texture classification, and the adaptive reconstruction of missing textures can be realized by the proposed method. Experimental results show subjective and quantitative improvement of the proposed method over previously reported reconstruction methods.

LINEAR TIME DECODING OF REAL-FIELD CODES OVER HIGH ERROR RATE CHANNELS
Zaixing He, Takahiro Ogawa, Miki Haseyama
2011 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, 3172, 3175, IEEE, 2011年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper proposes a novel algorithm for decoding real-field codes over erroneous channels, where the encoded message is corrupted by sparse errors, i.e., impulsive noise. The main problem of decoding such a corrupted encoded message is to reconstruct the error vector; recently, a common way to reconstruct it is to find the sparsest solution to an underdetermined system that is constructed using a parity-check matrix. Unlike the conventional approaches reconstructing the high-dimensional error vector directly, the proposed method crossly recovers the elements of error vector from two (or several) groups of low-dimensional equations. Compared with the traditional algorithms, the proposed method can decode an encoded message with a much higher corruption rate. Furthermore, the complexity of our method is linear, which is much lower than those of the traditional methods. The experimental results verified the high error correction ability and speed of the proposed method.

ADAPTIVE RECONSTRUCTION METHOD OF MISSING TEXTURES BASED ON PERCEPTUALLY OPTIMIZED ALGORITHM
Takahiro Ogawa, Miki Haseyama
2011 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, 1157, 1160, IEEE, 2011年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents an adaptive reconstruction method of missing textures based on structural similarity (SSIM) index. The proposed method firstly performs SSIM-based selection of the optimal known local textures to adaptively obtain subspaces for reconstructing missing textures. Furthermore, from the selected known textures, the missing texture reconstruction maximizing the SSIM index is performed. In this approach, the non-convex maximization problem is reformulated as a quasi convex problem, and the adaptive reconstruction of the missing textures becomes feasible. Experimental results show impressive improvement of the proposed method over previously reported reconstruction methods.

Adaptive example-based super-resolution using kernel PCA with a novel classification approach
Takahiro Ogawa, Miki Haseyama
EURASIP JOURNAL ON ADVANCES IN SIGNAL PROCESSING, 2011, 1, 29, SPRINGER INTERNATIONAL PUBLISHING AG, 2011年, ［査読有り］
英語, 研究論文（学術雑誌）, An adaptive example-based super-resolution (SR) using kernel principal component analysis (PCA) with a novel classification approach is presented in this paper. In order to enable estimation of missing high-frequency components for each kind of texture in target low-resolution (LR) images, the proposed method performs clustering of high-resolution (HR) patches clipped from training HR images in advance. Based on two nonlinear eigenspaces, respectively, generated from HR patches and their corresponding low-frequency components in each cluster, an inverse map, which can estimate missing high-frequency components from only the known low-frequency components, is derived. Furthermore, by monitoring errors caused in the above estimation process, the proposed method enables adaptive selection of the optimal cluster for each target local patch, and this corresponds to the novel classification approach in our method. Then, by combining the above two approaches, the proposed method can adaptively estimate the missing high-frequency components, and successful reconstruction of the HR image is realized.

Music recommendation according to human motion based on kernel CCA-based relationship
Hiroyuki Ohkushi, Takahiro Ogawa, Miki Haseyama
EURASIP JOURNAL ON ADVANCES IN SIGNAL PROCESSING, 2011, 121, 121, SPRINGER INTERNATIONAL PUBLISHING AG, 2011年, ［査読有り］
英語, 研究論文（学術雑誌）, In this article, a method for recommendation of music pieces according to human motions based on their kernel canonical correlation analysis (CCA)-based relationship is proposed. In order to perform the recommendation between different types of multimedia data, i.e., recommendation of music pieces from human motions, the proposed method tries to estimate their relationship. Specifically, the correlation based on kernel CCA is calculated as the relationship in our method. Since human motions and music pieces have various time lengths, it is necessary to calculate the correlation between time series having different lengths. Therefore, new kernel functions for human motions and music pieces, which can provide similarities between data that have different time lengths, are introduced into the calculation of the kernel CCA-based correlation. This approach effectively provides a solution to the conventional problem of not being able to calculate the correlation from multimedia data that have various time lengths. Therefore, the proposed method can perform accurate recommendation of best matched music pieces according to a target human motion from the obtained correlation. Experimental results are shown to verify the performance of the proposed method.

足跡を用いた男女識別　　　　　　　　　　　　　　　
画像ラボ, 22, 1, 17, 21, 2011年

固有空間BPLP法の補間精度に関する解析
田中章, 小川貴弘, 長谷山美紀, 宮腰政明
電子情報通信学会 論文誌(A), J94-A, 2, 116, 126, 2011年

Error-Resilient 3-D Wavelet Video Coding with Duplicated Lowest Sub-Band Coefficients and Two-Step Error Concealment Method
Sunmi Kim, Hirokazu Tanaka, Takahiro Ogawa, Miki Haseyama
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, E93A, 11, 2173, 2183, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2010年11月, ［査読有り］
英語, 研究論文（学術雑誌）, In this paper we propose a two step error concealment algorithm based on an error resilient three dimensional discrete wavelet transform (3 D DWT) video coding scheme. The proposed scheme consists of an error resilient encoder duplicating the lowest sub band bit streams for dispersive grouped frames and an error concealment decoder. The error concealment method of this decoder is decomposed of two steps the first step is replacement of erroneous coefficients in the lowest sub band by the duplicated coefficients and the second step is interpolation of the missing wavelet coefficients by minimum mean square error (MMSE) estimation. The proposed scheme can achieve robust transmission over unreliable channels. Experimental results provide performance comparisons in terms of peak signal to noise ratio (PSNR) and demonstrate increased performances compared to state of the art error concealment schemes.

画像・映像意味理解の現状と検索インタフェース(<小特集>ビジョンコンピューティングにおける確率的情報処理の展開)
長谷山 美紀
電子情報通信学会誌, 93, 9, 764, 769, 一般社団法人電子情報通信学会, 2010年09月
日本語, 画像・映像意味理解の研究動向とその検索への応用について紹介する.更に,画像及び映像が持つ固有の多義性とあいまい性から検索結果の可視化システムの必要性を議論し,その実現の試みについて紹介するとともに今後の展開について考える.

SVMを用いた足跡からの男女識別の実現(研究速報)
浅水 仁, 長谷山 美紀
電子情報通信学会論文誌. D, 情報・システム, 93, 5, 642, 646, 一般社団法人電子情報通信学会, 2010年05月01日
日本語, 本論文では,施設や店舗などに出入りする人物の足跡を用いて男女識別する手法について検証する.取得した足跡から算出が可能な特徴量を用いてSVMにより男女を識別する.本手法を用いて被験者実験を行い,90%の識別率を実現した.

ADAPTIVE RECONSTRUCTION METHOD OF MISSING TEXTURES BASED ON INVERSE PROJECTION VIA SPARSE REPRESENTATION
Takahiro Ogawa, Miki Haseyama
2010 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO (ICME 2010), 352, 357, IEEE, 2010年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents an adaptive reconstruction method of missing textures based on an inverse projection via sparse representation. The proposed method approximates original and corrupted textures in lower-dimensional subspaces by using the sparse representation technique. Then, this approach effectively solves problems of not being able to directly estimate an inverse projection for reconstructing missing textures. Furthermore, even if target textures contain missing areas, the proposed method enables adaptive generation of the subspaces by monitoring errors caused in their known neighboring textures by the estimated inverse projection. Consequently, since the optimal inverse projection is adaptively estimated for each texture, successful reconstruction of the missing areas can be expected. Experimental results show impressive improvement of the proposed reconstruction technique over previously reported reconstruction techniques.

THE SIMPLEST MEASUREMENT MATRIX FOR COMPRESSED SENSING OF NATURAL IMAGES
Zaixing He, Takahiro Ogawa, Miki Haseyama
2010 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, 4301, 4304, IEEE, 2010年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, There exist two main problems in currently existing measurement matrices for compressed sensing of natural images, the difficulty of hardware implementation and low sensing efficiency. In this paper, we present a novel simple and efficient measurement matrix, Binary Permuted Block Diagonal (BPBD) matrix. The BPBD matrix is binary and highly sparse (all but one or several "1"s in each column are "0"s). Therefore, it can simplify the compressed sensing procedure dramatically. The proposed measurement matrix has the following advantages, which cannot be entirely satisfied by existing measurement matrices. (1) It has easy hardware implementation because of the binary elements; (2) It has high sensing efficiency because of the highly sparse structure; (3) It is incoherent with different popular sparsity basis' like wavelet basis and gradient basis; (4) It provides fast and nearly optimal reconstructions. Moreover, the simulation results demonstrate the advantages of the proposed measurement matrix.

Spatio-temporal resolution enhancement of video sequence based on super-resolution reconstruction.
Miki Haseyama, Daisuke Izumi, Makoto Takizawa
Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2010, 14-19 March 2010, Sheraton Dallas Hotel, Dallas, Texas, USA, 870, 873, IEEE, 2010年, ［査読有り］
研究論文（国際会議プロシーディングス）

A Genetic Algorithm for Generating Multiple Paths on Mesh Maps
Jun Inagaki, Tomoaki Shirakawa, Tetsuo Shimono, Miki Haseyama
2010 IEEE CONGRESS ON EVOLUTIONARY COMPUTATION (CEC), 1, 4, IEEE, 2010年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, Path generation is an optimization problem mainly performed on grid square maps that combines generation of paths with minimization of their cost. Several methods that belong to the class of exhaustive searches are available; however, these methods are only able to obtain a single path as a solution for each iteration of the search. Hence, this paper proposes a new method using genetic algorithms for this problem with the goal of simultaneously searching for multiple candidate paths.

Missing Texture Reconstruction Method Based on Perceptually Optimized Algorithm
Takahiro Ogawa, Miki Haseyama
EURASIP JOURNAL ON ADVANCES IN SIGNAL PROCESSING, 2010, HINDAWI PUBLISHING CORPORATION, 2010年, ［査読有り］
英語, 研究論文（学術雑誌）, This paper presents a simple and effective missing texture reconstruction method based on a perceptually optimized algorithm. The proposed method utilizes the structural similarity (SSIM) index as a new visual quality measure for reconstructing missing areas. Furthermore, in order to adaptively reconstruct target images containing several kinds of textures, the following two novel approaches are introduced into the SSIM-based reconstruction algorithm. First, the proposed method performs SSIM-based selection of the optimal known local textures to adaptively obtain subspaces for reconstructing missing textures. Secondly, missing texture reconstruction that maximizes the SSIM index in the known neighboring areas is performed. In this approach, the nonconvex maximization problem is reformulated as a quasi convex problem, and adaptive reconstruction of the missing textures based on the perceptually optimized algorithm becomes feasible. Experimental results show impressive improvements of the proposed method over previously reported reconstruction methods.

Erratum: Error-Resilient 3-D Wavelet Video Coding with Duplicated Lowest Sub-Band Coefficients and Two-Step Error Concealment Method [IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences E93.A (2010) , No. 11 pp.2173-2183]
KIM Sunmi, TANAKA Hirokazu, OGAWA Takahiro, HASEYAMA Miki
IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, 93, 12, 2763_e1, 2763_e1, The Institute of Electronics, Information and Communication Engineers, 2010年
英語

生命系異分野共同研究プロジェクト(北の国から明日のICTに架ける橋,<特別小特集>知の創出を支える次世代IT基盤技術-北海道大学グローバルCOEプログラムと北海道内情報通信系研究グループの活動-)
渡邉 日出海, 金子 俊一, 長谷山 美紀
電子情報通信学会誌, 92, 10, 822, 827, 一般社団法人電子情報通信学会, 2009年10月
日本語

Adaptive Missing Texture Reconstruction Method Based on Kernel Canonical Correlation Analysis with a New Clustering Scheme
Takahiro Ogawa, Miki Haseyama
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, E92A, 8, 1950, 1960, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2009年08月, ［査読有り］
英語, 研究論文（学術雑誌）, In this paper, a method for adaptive reconstruction of missing textures based on kernel canonical correlation analysis (CCA) with a new clustering scheme is presented. The proposed method estimates the correlation between two areas, which respectively correspond to a missing area and its neighboring area, from known parts within the target image and realizes reconstruction of the missing texture. In order to obtain this correlation, the kernel CCA is applied to each cluster containing the same kind of textures, and the optimal result is selected for the target missing area. Specifically, a new approach monitoring errors caused in the above kernel CCA-based reconstruction process enables selection of the optimal result. This approach provides a solution to the problem in traditional methods of not being able to perform adaptive reconstruction of the target textures due to missing intensities. Consequently, all of the missing textures are successfully estimated by the optimal cluster's correlation, which provides accurate reconstruction of the same kinds of textures. In addition, the proposed method can obtain the correlation more accurately than our previous works, and more successful reconstruction performance can be expected. Experimental results show impressive improvement of the proposed reconstruction technique over previously reported reconstruction techniques.

An ER Algorithm-Based Method for Removal of Adherent Water Drops from Images Obtained by a Rear View Camera Mounted on a Vehicle in Rainy Conditions
Tomoki Hiramatsu, Takahiro Ogawa, Miki Haseyama
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, E92A, 8, 1939, 1949, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2009年08月, ［査読有り］
英語, 研究論文（学術雑誌）, In this paper, an ER (Error-Reduction) algorithm-based method for removal of adherent water drops from images obtained by a rear view camera mounted on a vehicle in rainy conditions is proposed. Since Fourier-domain and object-domain constraints are needed for any ER algorithm-based method, the proposed method introduces the following two novel constraints for the removal of adherent water drops. The first one is the Fourier-domain constraint that utilizes the Fourier transform magnitude of the previous frame in the obtained images as that of the target frame. Noting that images obtained by the rear view camera have the unique characteristics of objects moving like ripples because the rear view camera is generally composed of a fish-eye lens for a wide view angle, the proposed method assumes that the Fourier transform magnitudes of the target frame and the previous frame are the same in the polar coordinate system. The second constraint is the object-domain constraint that utilizes intensities in an area of the target frame to which water drops have adhered. Specifically, the proposed method models a deterioration process of intensities that are corrupted by the water drop adhering to the rear view camera lens. By utilizing these novel constraints, the proposed ER algorithm can remove adherent water drops from images obtained by the rear view camera. Experimental results that verify the performance of the proposed method are represented.

Estimating Number of People Using Calibrated Monocular Camera Based on Geometrical Analysis of Surface Area
Hiroyuki Arai, Isao Miyagawa, Hideki Koike, Miki Haseyama
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, E92A, 8, 1932, 1938, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2009年08月, ［査読有り］
英語, 研究論文（学術雑誌）, We propose a novel technique for estimating the number of people in a video sequence; it has the advantages of being stable even in crowded situations and needing no ground-truth data. By analyzing the geometrical relationships between image pixels and their intersection volumes in the real world quantitatively, a foreground image directly indicates the number of people. Because foreground detection is possible even in crowded situations, the proposed method can be applied in such situations. Moreover, it can estimate the number of people in an a priori manner, so it needs no ground-truth data unlike existing feature-based estimation techniques. Experiments show the validity of the proposed method.

An Accurate Scene Segmentation Method Based on Graph Analysis Using Object Matching and Audio Feature
Makoto Yamamoto, Miki Haseyama
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, E92A, 8, 1883, 1891, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2009年08月, ［査読有り］
英語, 研究論文（学術雑誌）, A method for accurate scene segmentation using two kinds of directed graph obtained by object matching and audio features is proposed. Generally, in audiovisual materials, such as broadcast programs and movies, there are repeated appearances of similar shots that include frames of the same background, object or place, and such shots are included in a single scene. Many scene segmentation methods based on this idea have been proposed; however, since they use color information as visual features, they cannot provide accurate scene segmentation results if the color features change in different shots for which frames include the same object due to camera operations such as zooming and panning. In order to solve this problem, scene segmentation by the proposed method is realized by using two novel approaches. In the first approach, object matching is performed between two frames that are each included in different shots. By using these matching. results, repeated appearances of shots for which frames include the same object can be successfully found and represented as a directed graph. The proposed method also generates another directed graph that represents the repeated appearances of shots with similar audio features in the second approach. By combined use of these two directed graphs, degradation of scene segmentation accuracy, which results from using only one kind of graph, can be avoided in the proposed method and thereby accurate scene segmentation can be realized. Experimental results performed by applying the proposed method to actual broadcast programs are shown to verify the effectiveness of the proposed method.

動的ファジィ制約充足における解の安定性維持
須藤 康裕, 柳田 拓人, 栗原 正仁, 長谷山 美紀
知能と情報 : 日本知能情報ファジィ学会誌 : journal of Japan Society for Fuzzy Theory and Intelligent Informatics, 21, 3, 372, 380, 日本知能情報ファジィ学会, 2009年06月15日
日本語, ファジィ制約充足問題は人工知能の基礎戦術として知られる制約充足問題（CSP：Constraint Satisfaction Problem）を充足度を持たせることにより拡張した数理モデルである．一方，動的CSPの枠組みは現実世界における問題の部分的な変化と再求解を想定したモデルであり，その特徴は，問題を解きなおすコストの削減と解の安定性が求められることである．これら２つを組み合わせた数理モデルである動的ファジィCSP（DFCSP：Dynamic Fuzzy CSP）に関する研究は既存するが，CSPはNP完全問題として知られ，最悪計算量は指数オーダーとなることから，完全で高速な解法は存在しない． 本論文では，ファジィCSPのハイブリッド解法であるSRSアルゴリズムをDFCSPに適用し，さらに後処理としてのフィルタリングとしてSRSDアルゴリズムを導入することにより，大規模で複雑な問題に対しても，安定した実用的な許容解が得られることを，他の代表的な手法と比較することにより定量的に示す．

アクティブネットを用いたサッカー映像におけるパス可能領域の推定(画像認識,コンピュータビジョン)
高橋 翔, 今 宏史, 長谷山 美紀
電子情報通信学会論文誌. D, 情報・システム, 92, 4, 501, 510, 一般社団法人電子情報通信学会, 2009年04月01日
日本語, 本論文では,チームスポーツ映像からアクティブネットを用いてパス可能領域を推定する手法を提案する.チームスポーツ映像の一つであるサッカー映像の意味内容解析を行うために重要なサッカーの戦術は,選手の移動とボール運びによって表現されるため,ボール運びを実現するパスを分析することは重要である.一般にパスコースはボール保持者と味方チームの選手へとつながる緩やかな曲線で表される.提案手法では,新たなエネルギーの定義とパス可能領域を推定するための画像生成により,アクティブネットを用いて前述の曲線が存在する領域を抽出する.また,パス可能領域は守備の選手から離れるほど,パスが成功する可能性が高いという特徴をもつ.提案手法では,格子点の密度に着眼することで,パスが成功する可能性をパス可能領域の推定と同時に得る.更に,アクティブネットの収束結果は多少の選手位置の誤差を許容するため,選手の動きを用いた従来手法における,選手位置の誤差の影響を受けやすいという問題点を解決することが可能である.したがって,提案手法はカメラワークが存在し,高精度な選手位置の推定が困難であるテレビ映像に対しても,高精度にパス可能領域の推定が可能である.

Kalman Filter-Based Error Concealment for Video Transmission
Shigeki Takahashi, Takahiro Ogawa, Hirokazu Tanaka, Miki Haseyama
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, E92A, 3, 779, 787, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2009年03月, ［査読有り］
英語, 研究論文（学術雑誌）, A novel error concealment method using a Kalman filter is presented ill this paper, In order to successfully utilize the Kalman filter, its state transition and observation models that are suitable for the video error concealment are newly defined as follows. The state transition model represents the video decoding process by a notion-compensated prediction. Furthermore, the new observation model that represents all image blurring process is defined. and calculation of the Kalman gain becomes possible. The problem of the traditional methods is solved by using the Kalman filter in the proposed method, and accurate reconstruction of corrupted video frames, is achieved. Consequently. an effective error concealment method using the Kalman filter is realized. Experimental results showed that the proposed method has better performance than that of traditional methods.

エッジの連続性を考慮したIFSによる画像の拡大(画像・映像処理)
覚幸 典弘, 小川 貴弘, 長谷山 美紀
電子情報通信学会論文誌. D, 情報・システム, 92, 3, 382, 392, 一般社団法人電子情報通信学会, 2009年03月01日
日本語, 本論文では,エッジの連続性を考慮した, Iterated Function System (IFS)に基づく画像拡大法を提案する.従来のIFS画像拡大法では,拡大後の画像において処理の最小単位となるブロックの境界で,本来存在しないはずの輝度値の変化が生じる.また,エッジの連続性を考慮しておらず,拡大後の画像におけるエッジが不連続となる問題が存在した.そこで提案手法では,まず,処理の最小単位となるブロックの重なりを許すことで, IFSによる高近似縮小写像を実現し,ブロック境界での輝度値の変化を抑制する.更に,連続性を保ったエッジの推定が可能であるラインプロセスを新たにIFS画像拡大法に導入する.このとき我々は,ラインプロセスを拡大後の画像のエッジを推定する手法に拡張することで,輝度値が未知である拡大後の画像に対しても,連続性を保ったエッジの推定を可能とする.また,提案手法では得られるエッジの存在を考慮した上でIFSに基づく画像拡大を行うことにより,従来法で発生していた拡大後の画像におけるエッジの不連続を解決し,高精細な拡大を実現する.本論文の最後では,提案手法の有効性を示すため比較実験を行い,その拡大性能を評価する.

A Kalman Filter-Based Method for Restoration of Images Obtained by an In-Vehicle Camera in Foggy Conditions
Tomoki Hiramatsu, Takahiro Ogawa, Miki Haseyama
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, E92A, 2, 577, 584, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2009年02月, ［査読有り］
英語, 研究論文（学術雑誌）, In this paper, a Kalman filter-based method for restoration of video images acquired by an in-vehicle camera in foggy conditions is proposed. In order to realize Kalman filter-based restoration, the proposed method clips local blocks from the target frame by using a sliding window and regards the intensities in each block as elements of the state variable of the Kalman filter. Furthermore, the proposed method designs the following two models for restoration of foggy images. The first one is an observation model, which represents a fog deterioration model. The proposed method automatically determines all parameters of the fog deterioration model from only the foggy images to design the observation model. The second one is a non-linear state transition model, which represents the target frame in the original video image from its previous frame based on motion vectors. By utilizing the observation and state transition models, the correlation between successive frames can be effectively utilized for restoration, and accurate restoration of images obtained in foggy conditions can be achieved. Experimental results show that the proposed method has better performance than that of the traditional method based on the fog deterioration model.

情報大航海プロジェクトにおける共通技術 : 画像・映像処理技術の紹介
長谷山 美紀, 久光 徹
映像情報メディア学会誌 : 映像情報メディア, 63, 1, 42, 47, 一般社団法人映像情報メディア学会, 2009年
日本語, 2007年度に3年計画でスタートした情報大航海プロジェクトは,モデルサービスによる実証を通じて次世代の情報検索・解析技術を開発することを目的としている.同時に,実証を通じた制度的課題の洗い出しにより,市場創出に必要な環境整備を目指している.本稿では,開発された技術の中から,マルチメディア情報処理の中核を担う,画像・映像処理技術を紹介する.

選択情報が不要な適応内挿フィルタを用いた動画像符号化の高性能化
近藤 敏志, 松野 孝也, 長谷山 美紀
映像情報メディア学会誌 : 映像情報メディア = The journal of the Institute of Image Information and Television Engineers, 63, 11, 1592, 1597, 映像情報メディア学会, 2009年
日本語, 研究論文（学術雑誌）, We propose a method to improve performance of video coding using an adaptive interpolation filter technique. The adaptive interpolation technique was based on clustering using the k-means method and did not need filter selection information. To improve the performance of the clustering, we introduced autocorrelation coefficients of the pixel values and the directions of the motion vectors as the new features and the Mahalanobis distance as the distance scale in the k-means method. We also used vector quantization to reduce the number of interpolation filter coefficients. In the simulation, our proposed method was implemented in the MPEG-2 based video codec. The simulation results show that the proposed method can reduce the bit rate by up to nearly 7 % compared to conventional adaptive interpolation filter methods.

Accurate Graph-Based Scene Segmentation Using Object Matching and Audio Feature
Makoto Yamamoto, Miki Haseyama
ISCE: 2009 IEEE 13TH INTERNATIONAL SYMPOSIUM ON CONSUMER ELECTRONICS, VOLS 1 AND 2, 670, 671, IEEE, 2009年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, A method for accurate scene segmentation utilizing two kinds of directed graph obtained by object matching and by using audio features is proposed. Generally, in audiovisual materials, there are repeated appearances of shots that include frames of the same background, object or place. It should be assumed that such shots are included in a single scene. In the proposed method, by performing object matching between two frames that are each included in different shots, multiple shots for which frames include the same object can be successfully found and their repeated appearances are represented as a directed graph. The proposed method also generates another directed graph that represents the repeated appearances of shots with similar audio features. By the combined use of these two graphs, accurate scene segmentation can be realized. The effectiveness of the proposed method is verified by applying this method to news programs and another broadcast program.

Semantic Image Retrieval Based on POCS Algorithm Using Kernel PCA And Its Performance Verification
Takahiro Ogawa, Miki Haseyama
ISCE: 2009 IEEE 13TH INTERNATIONAL SYMPOSIUM ON CONSUMER ELECTRONICS, VOLS 1 AND 2, 342, 343, IEEE, 2009年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a projection onto convex sets (POCS)-based semantic image retrieval method and its performance verification. The main contributions of the proposed method are twofold: introduction of nonlinear eigenspace of visual and semantic features into the constraint of the POCS-based semantic image retrieval algorithm and adaptive selection of the annotated images utilized for this algorithm. Then, by combining these two approaches., the semantic features of the query image are successfully estimated, and accurate image retrieval can be expected. Finally, relationship between the performance of the proposed method and the kinds of the kernel functions utilized for the kernel PICA is shown in this paper.

A new image retrieval interface and its practical use in "View Search Hokkaido"
Miki Haseyama, Toshifumi Murata, Hisashi Ukawa
ISCE: 2009 IEEE 13TH INTERNATIONAL SYMPOSIUM ON CONSUMER ELECTRONICS, VOLS 1 AND 2, 624, +, IEEE, 2009年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, New image retrieval technology, which is used for a service demonstration project "View Search Hokkaido" in "Information Grand Voyage Project" conducted by Ministry of Economy, Trade and Industry, Japan, is presented in this paper. The new technology enables image retrieval based on low-level features without utilizing any tag-based scheme and realizes a 3D interface for image retrieval. This 3D interface can provide new user experience, which has not been provided by the conventional retrieval services. The effectiveness of the 3D interface has been verified in "View Search Hokkaido".

映像検索のための画像及び音響信号によるシーン間の相違度算出--正準相関分析とDynamic Time Warpingを用いたアプローチ
道山 大悟, 長谷山 美紀
画像ラボ, 19, 7, 1, 4, 日本工業出版, 2008年07月
日本語

映像処理技術の変革と映像メディア
長谷山 美紀
映像情報メディア学会誌 : 映像情報メディア, 62, 4, K12, K12, 一般社団法人映像情報メディア学会, 2008年04月01日
日本語

3-2. メディア流通・処理(3.メディア情報,<特集>映像情報メディア年報)
上倉 一人, 長谷山 美紀, 村上 和人
映像情報メディア学会誌 : 映像情報メディア, 62, 8, 1251, 1254, 一般社団法人映像情報メディア学会, 2008年
日本語, 研究論文（学術雑誌）

Estimating the number of people in a video sequence via geometrical model.
Hiroyuki Arai, Isao Miyagawa, Hideki Koike, Miki Haseyama
19th International Conference on Pattern Recognition (ICPR 2008)(ICPR), 1, 4, IEEE Computer Society, 2008年
研究論文（国際会議プロシーディングス）

KERNEL PCA-BASED SEMANTIC FEATURE ESTIMATION APPROACH FOR SIMILAR IMAGE RETRIEVAL
Takahiro Ogawa, Miki Haseyama
2008 IEEE International Conference on Image Processing, Proceedings, 969, 972, IEEE, 2008年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, A kernel PCA-based semantic feature estimation approach for similar image retrieval is presented in this paper. Utilizing database images previously annotated by keywords, tire proposed method estimates unknown semantic features of a query image. First, our method performs semantic clustering of the database images and derives a new map from a nonlinear eigenspace of visual and semantic features in each c aster. This map accurately provides the semantic features for the images belonging to each cluster by using their visual features. Further, in order to select the optional cluster including the query image, the proposed method monitors errors of the visual features caused by the Semantic feature estimation process. Then, even if any semantics of the query image arc unknown, its semantic features are successfully estimated by tire optimal cluster. Experimental results verify the effectiveness of the proposed method for semantic image retrieval.

カルマンフィルタと適応信号処理, 谷萩隆嗣著, (コロナ社, Tel.03-3941-3131, 2005年, A5判, 282頁, 定価4,515円(税込))
長谷山 美紀
日本音響学会誌, 63, 8, 社団法人日本音響学会, 2007年08月01日
日本語

Audio-based shot classification for audiovisual indexing using PCA, MGD and fuzzy algorithm
Naoki Nitanda, Miki Haseyama
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, E90A, 8, 1542, 1548, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2007年08月, ［査読有り］
英語, 研究論文（学術雑誌）, An audio-based shot classification method for audiovisual indexing is proposed in this paper. The proposed method mainly consists of two parts, an audio analysis part and a shot classification part. In the audio analysis part, the proposed method utilizes both principal component analysis (PCA) and Mahalanobis generalized distance (MGD). The effective features for the analysis can be automatically obtained by using PCA, and these features are analyzed based on MGD, which can take into account the correlations of the data set. Thus, accurate analysis results can be obtained by the combined use of PCA and MGD. In the shot classification part, the proposed method utilizes a fuzzy algorithm. By using the fuzzy algorithm, the mixing rate of the multiple audio sources can be roughly measured, and thereby accurate shot classification can be attained. Results of experiments performed by applying the proposed method to actual audiovisual materials are shown to verify the effectiveness of the proposed method.

Players clustering based on graph theory tor tactics analysis purpose in soccer videos
Hiroftuni Kon, Miki Haseyama
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, E90A, 8, 1528, 1533, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2007年08月, ［査読有り］
英語, 研究論文（学術雑誌）, In this paper, a new method for clustering of players in order to analyze games in soccer videos is proposed. The proposed method classifies players who are closely related in terms of soccer tactics into one group. Considering soccer tactics, the players in one group are located near each other. For this reason, the Euclidean distance between the players is an effective measurement for the clustering of players. However, the distance is not sufficient to extract tactics-based groups. Therefore, we utilize a modified version of the community extraction method, which finds community structure by dividing a non-directed graph. The use of this method in addition to the distance enables accurate clustering of players.

小規模データベースにおける学術論文の関連性可視化手法
二反田 直己, 鎌倉 純一, 長谷山 美紀
信号処理, 11, 2, 179, 185, 〔信号処理学会〕, 2007年03月
日本語

Phase retrieval based on a snake for image reconstruction
Keiko Kondo, Miki Haseyama, Hideo Kitajima
IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, E90D, 1, 283, 287, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2007年01月
英語, 研究論文（学術雑誌）, A new phase retrieval method using an active contour model (snake) for image reconstruction is proposed. The proposed method reconstructs a target image by retrieving the phase from the magnitude of its Fourier transform and the measured area of the image. In general, the measured area is different from the true area where the target image exists. Thus a snake, which can extract the shape of the target image, is utilized to renew the measured area. By processing this renewal iteratively, the area obtained by the snake converges to the true area and as a result the proposed method can accurately reconstruct a target image even when the measured area is different from the true area. Experimental results show the effectiveness of the proposed method.

Steady-state properties of a CORDIC-based adaptive ARMA lattice filter
Shin'ichi Shiraishi, Miki Haseyama, Hideo Kitajima
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, E89A, 12, 3724, 3729, IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, 2006年12月
英語, 研究論文（学術雑誌）, This paper analyzes the steady-state properties of a CORDIC-based adaptive ARMA lattice filter. In our previous study, the convergence properties of the filter in the non-steady state were clarified; however, its behavior in the steady state was not discussed. Therefore, we develop a distinct analysis technique based on a Markov chain in order to investigate the steady-state properties of the filter. By using the proposed technique, the relationship between step size and coefficient estimation error is revealed.

基礎シリーズ 線形予測理論とラティスフィルタアルゴリズム(2)ARラティスフィルタの実現とその性質
長谷山 美紀
信号処理, 10, 5, 309, 315, 〔信号処理学会〕, 2006年09月
日本語

輝度こう配ベクトルを用いた画像識別方法(画像認識,コンピュータビジョン)
平本 政夫, 小川 貴弘, 長谷山 美紀
電子情報通信学会論文誌. D, 情報・システム, 89, 6, 1348, 1358, 一般社団法人電子情報通信学会, 2006年06月01日
日本語, 本論文では,撮像素子の多面素化や高画質化の流れを踏まえ,画像の回転・移動等の幾何学的変換にも対応できる大局的な画像識別方法を提案している.提案手法は,ベクトルを利用した投票方式を用いかものであり,画像を輝度こう配を表すベクトルと位置を示すベクトルで表現し,識別のための投票ベクトルと類似度を定義している.また提案手法では,同一画像であれば得票場所が原点に集中し,得票結果が幾何学的な変換に影響されないという特徴がある.原画像に対してガウシアン,メジアンのフィルタリング処理,JPEG圧縮処理を施した画像も含め,自然画像の識別実験を行ったところ,類似性において明確な差が現れ,画像に対して人工的な処理を加えても識別可能であることが分かった.更に提案手法の応用として,最多得票点を利用した画像の識別について検討したところ,識別能力が高く,1画像内に含まれる部分画像の識別も可能であることを示すことができた.

基礎シリーズ 線形予測理論とラティスフィルタアルゴリズム(1)線形予測とARモデル同定
長谷山 美紀
信号処理, 10, 3, 153, 159, 〔信号処理学会〕, 2006年05月
日本語

知的情報アクセスを実現するための映像検索に関する研究動向 (映像・音響信号処理の将来展望と課題)
長谷山 美紀
回路とシステム軽井沢ワークショップ論文集, 19, 199, 203, [電子情報通信学会], 2006年04月24日
日本語

色エッジ及び並列実行の領域統合を用いたWatershed画像領域分割法の改善(画像認識,コンピュータビジョン)
趙 延軍, 長谷山 美紀, 北島 秀夫
電子情報通信学会論文誌. D, 情報・システム, 89, 4, 836, 849, 一般社団法人電子情報通信学会, 2006年04月01日
日本語, 画像内容に基づく画像処理では,画素ではなく,画像中の物体を構成する領域を対象とし,画像の圧縮,検索,認識等様々な処理を行う.そのため,領域を検出し,ラベルを付与する画像分割が必要になる.本論文では画像における輝度値の空間分布を地形とみなすWatershedアルゴリズムによる領域分割を検討する.Watershedアルゴリズムにより,閉じた単一の領域分割線が得られるが,分割結果には領域の未分割及び過剰分割という問題が発生する.本論文においてはこれらの問題を解決するために,原画像を直接処理することではなく,連続エッジと均質部分の両方が強調できる色エッジの強度画像を分割対象とする.更に,未分割と過剰分割とを防ぐために,色エッジの強度画像に対する分割結果を初期領域として,ノイズ抑制を考慮した並列実行の領域統合を行う.提案手法を各種の画像へ適用した結果によりその有効性を示す.

A multi-objective service restoration method for power distribution systems
Jun Inagaki, Jun Nakajima, Miki Haseyama
2006 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS, VOLS 1-11, PROCEEDINGS, 1784, +, IEEE, 2006年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, Service restoration problem in distribution systems is formulated as a multi-objective optimization problem which is demanded not only for minimizing the amount of unrestored total loads but also for minimizing the number of the switching operations. The solution of the multi-objective optimization problem is usually obtained with a set of Pareto optimal solutions. The Pareto optimal solutions for the service restoration problem are useful for users to obtain their desired restoration by comparing a Pareto optimal solution with the others. However, the conventional methods cannot obtain several Pareto optimal solutions in one trial. Therefore, this paper proposes a method for obtaining the Pareto optimal set for the service restoration problem with a genetic algorithm. The genetic algorithm produces many possible solutions in its search process. By utilizing this feature, the proposed method can obtain the Pareto optimal set.

Restoration method of missing areas in still images using GMRF model
T Ogawa, M Haseyama, H Kitajima
2005 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS (ISCAS), VOLS 1-6, CONFERENCE PROCEEDINGS, 4931, 4934, IEEE, 2005年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper proposes a GMRF-model based restoration method of missing areas in still images. The GMRF model used in the proposed method is realized by a new assumption that reasonably holds for an image source. This model can express important image features such as edges because of the use of the new assumption. Therefore, the proposed method restores the missing areas using the modified GMRF model and can correctly reconstruct the missing edges. Consequently, the proposed method achieves more accurate restoration than those of the traditional methods on both objective and subjective measures. Extensive experimental results demonstrate the improvement of the proposed method over the previous methods.

GA-based applications for routing with an upper bound constraint
J Inagaki, M Haseyama
2005 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS (ISCAS), VOLS 1-6, CONFERENCE PROCEEDINGS, 2239, 2242, IEEE, 2005年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper presents a method of searching for the shortest route via the most designated points among the routes whose lengths are less than the upper bound using a genetic algorithm (GA). If chromosomes whose route lengths exceed the upper bound are simply screened out in the GA process, the optimization probability gets worse. For the purpose of solving this problem, this paper proposes a new fitness function including an upper bound constraint which can be flexibly changed during the searching process. By using this function, the optimum is efficiently obtained and the optimization probability can be raised. Furthermore, the effectiveness of the proposed method is verified by the experiments applying it to the actual map data.

Audio signal segmentation and classification for scene-cut detection
N Nitanda, M Haseyama, H Kitajima
2005 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS (ISCAS), VOLS 1-6, CONFERENCE PROCEEDINGS, 4030, 4033, IEEE, 2005年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, A scene is regarded as a basic unit of audiovisual material, and thereby the boundaries between two adjacent scenes, which are called scene-cuts, must be detected in advance for audiovisual indexing. This paper proposes a scene-cut detection method. Since scene-cuts are associated with a simultaneous change of visual and audio characteristics, both audio and visual analyses are required for the scene-cut detection. For the audio signal analysis, the proposed method utilizes an audio signal segmentation and classification method using fuzzy c-means clustering, which has been proposed by the authors. For the visual signal analysis, the proposed method utilizes some visual segmentation methods. By using these methods simultaneously, the proposed method can accurately detect the scene-cuts, and thereby it is highly valuable for the preprocessing of the audiovisual indexing. Experimental results performed by applying the proposed method to real audiovisual material are shown to verify its high performance.

Quality improvement technique for JPEG images with fractal image coding
M Takezawa, H Sanada, K Watanabe, M Haseyama
2005 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS (ISCAS), VOLS 1-6, CONFERENCE PROCEEDINGS, 6320, 6323, IEEE, 2005年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper proposes a quality improvement technique for JPEG images by using fractal image coding. JPEG coding is a commonly used standard method of compressing images. However, in its decoded images, quantization noise is sometimes visible in high frequency regions, such as the edges of objects. Hence, in order for the JPEG coding to become a more powerful image-coding method, the JPEG image quality must be improved. Therefore, our method solves this problem by adding the obtained codes by the fractal image coding to improve the image quality. Some simulation results verify that the proposed method achieved higher coding-performance than the traditional JPEG coding.

A GA-based fast search algorithm for realizing efficiency motion compensation
S Asamizu, M Haseyama
8TH WORLD MULTI-CONFERENCE ON SYSTEMICS, CYBERNETICS AND INFORMATICS, VOL VI, PROCEEDINGS, 185, 190, INT INST INFORMATICS & SYSTEMICS, 2004年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper proposes a fast search technique using a genetic algorithm (GA) [1] for realizing efficiency motion compensation. Previously proposed method[2] that the motion compensation based on table lookup refers to two or more frames with one motion table. Therefore, dispersion of the prediction error of each frame is not only dispersion of the image quality of each prediction image but also influences the search speed of the motion table designing. In this proposed method, the prediction error of each frame is obtained by using the block matching method[3] before the motion table is designed. The weight of each frame is put from the obtained prediction error to the fitness function of the GA, and the GA searches for the optimal motion table. Based on this strategy, the proposed method can search for a motion table more effectively.

New JPEG coding algorithm including fractal image coding for color images
M Takezawa, M Haseyama
8TH WORLD MULTI-CONFERENCE ON SYSTEMICS, CYBERNETICS AND INFORMATICS, VOL VI, PROCEEDINGS, 333, 336, INT INST INFORMATICS & SYSTEMICS, 2004年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper proposes an effective JPEG coding algorithm using fractal image coding for color images. In the JPEG images, quantization noise is sometimes visible in high frequency regions, such as the edges of objects. Hence, in order for the JPEG coding to become a more powerful image-coding method, the JPEG image quality must be improved. Therefore, in this paper, a quality improvement method for the color JPEG images is proposed by using the fractal image coding. Some simulation results verify that the proposed method can improve the image quality. in the high frequency regions and provide the decoded image with 0.4 dB higher quality than the traditional JPEG.

A trainable retrieval system for cartoon character images
M Haseyama, A Matsumura
2003 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOL III, PROCEEDINGS, 673, 676, IEEE, 2003年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper proposes a novel method to retrieve cartoon character images in a database or network. In this method, partial features of an image, defined as Regions and Aspects, are used as keys to identify cartoon character images. The similarities between a query cartoon character image and the images in the database are computed by using these features. Based on the similarities the cartoon images same or similar to the query image are identified and retrieved from the database. Moreover, our method adopts a training scheme to reflect the user's subjectivity. The training emphasizes the signficant Regions or Aspects by assigning more weight based on the user's preferences and actions, such as selecting a desired image or an area of an image. These processes make the retrieval more effective and accurate. Experiment results verify the effectiveness and retrieval accuracy of the method.

2-D functional AR model for image identification
M Haseyama, Kondo, I
2003 INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, VOL II, PROCEEDINGS, 377, 380, IEEE, 2003年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper proposes a 2-D Functional AR Model for image identification. The definition of the proposed model includes functions that can exploit the self-similarity nature in images to throughly extract image features. By introducing the functional scheme into the model, only a few number of parameters, which are called 2-D Functional AR parameters, can describe the image features simply and accurately. These characteristics make the model suitable for image identification applications. Some experiments of image identification are performed, and the results verify that the proposed model accurately represents the image feature, and the image can be correctly, identified. The calculation time is fast enough for practical use in image retrieval.

E cient fixed-valued and random-valued impulse detection for accurate image restoration
K Kondo, M Haseyama, H Kitajima
ISPA 2003: PROCEEDINGS OF THE 3RD INTERNATIONAL SYMPOSIUM ON IMAGE AND SIGNAL PROCESSING AND ANALYSIS, PTS 1 AND 2, 1009, 1012, UNIV ZAGREB, FACULTY MECHANICAL ENGINEERING & NAVAL ARCHITECTURE, 2003年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, This paper proposes a novel impulse detection method for the restoration of images corrupted by impulse noise. Conventional impulse detection methods tend to work well for fixed-valued impulse noise but poorly for random-valued impulse noise. The proposed method can accurately detect not only fixed-valued but also random-valued impulse noise by using two different systems. The first system detects impulse noise by considering the di erences between the intensity of a target pixel and the output of a median filter The second system verifies whether the impulse detection results obtained by the first system are correct. By using these systems, the proposed method can accurately detect both types of impulse noise even in highly corrupted images. Furthermore, the proposed method can be e ectively used as a preprocessor for noise reduction filtering. Experiments are presented to demonstrate the e ectiveness of the proposed method.

Two-dimensional analysis of magnetic microstructures in the DC-demagnetized state and magnetization fluctuations in the transition region using MFM images
Takekuma, I, M Haseyama, K Sueoka, K Mukasa
JOURNAL OF MAGNETISM AND MAGNETIC MATERIALS, 239, 1-3, 359, 362, ELSEVIER SCIENCE BV, 2002年02月, ［査読有り］
英語, 研究論文（学術雑誌）, This study analyzed the directional dependence of magnetic microstructures of media by focusing on the distribution of magnetic poles. Media, whose magnetic poles tend to be distributed along a specific direction in the DC-demagnetized state, have larger magnetization fluctuations in the transition region, larger medium noise and higher partial erasure (PE) probability than media with uniform magnetic pole distribution. (C) 2002 Elsevier Science B.V. All rights reserved.

A new approach with IFS for image restoration.
Miki Haseyama, Megumi Takezawa, Junichi Miura, Hideo Kitajima
10th European Signal Processing Conference(EUSIPCO), 1, 4, IEEE, 2000年
研究論文（国際会議プロシーディングス）

A practical method to reduce a number of reference signals for the ANC system
M Akiho, M Haseyama, H Kitajima
ICASSP '99: 1999 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, PROCEEDINGS VOLS I-VI, 2387, 2390, IEEE, 1999年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, In this paper, we propose a practical method to reduce a number of reference signals for the active noise cancellation (ANC) system and the filter characteristics to generate the reduced number of reference signals, which maintain the original value of the coherence function. This method finds the number of independent noise sources and provides the filter characteristics based on SVD (singular value decomposition) of the power spectrum matrix of the reference signals. Then, we also use the multiple coherence function analysis to select dominant components in the reference signals. The method contributes greatly in reducing the number of reference signals for the ANC system that uses the large number of reference signals. We also discuss the characteristics of the filters that synthesis the new set of reference signals. And an experimental test is performed to confirm the theory.

A genetic algorithm based image segmentation for image analysis
M Haseyama, M Kumagai, H Kitajima
ICASSP '99: 1999 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, PROCEEDINGS VOLS I-VI, 3445, 3448, IEEE, 1999年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）, In this paper a new genetic algorithm (GA) based image segmentation method is proposed for image analysis. This method using a mean square error (MSE) based criterion can segment an image into some regions, while estimating a suitable region representation. The criterion is defined as MSE caused by interpolating each region of an observed image with a parametric model. Since the criterion is expressed with not only the parameters of the model but also shape and location of the regions, the criterion can not be easily minimized by the usual optimization methods, the proposed method minimizes the criterion by a GA. The proposed method also includes a processor to eliminate fragile regions with the Markov random field (MRF) model. Though the thresholds of the existent methods negatively affect image segmentation results; since no thresholds are required in the proposed method, it segments images more accurately than the existent methods.

A realization method of an ARMAX lattice filter
M HASEYAMA, T HIROHKU, H KITAJIMA
1995 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS, VOLS 1-3, 365, 368, I E E E, 1995年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）

FREQUENCY-WEIGHTING MODEL IDENTIFICATION WITH AN ADAPTIVE ARMA LATTICE FILTER
M HASEYAMA, N NAGAI, N MIKI
IEEE INTERNATIONAL CONFERENCE ON SYSTEMS ENGINEERING, 543, 546, I E E E, 1992年, ［査読有り］
英語, 研究論文（国際会議プロシーディングス）

モーションおよび視線データを用いたSpatial Temporal Attention GCNによる熟練度分類に関する検討
清野竜生, 斉藤直輝, 小川貴弘, 浅水仁, 長谷山美紀, 人工知能学会全国大会論文集(Web), 38th, 2024年

Spatial-temporal attentionを導入した再帰型ニューラルネットワークに基づく重機との接触事故リスクの推定
五箇亮太, 前田圭介, 藤後廉, 小川貴弘, 長谷山美紀, AI・データサイエンス論文集(Web), 5, 1, 2024年

工事現場の定点カメラを用いた物体検出および姿勢推定に基づくZero-shot高リスク状況検出
大羽賀駿也, 前田圭介, 藤後廉, 小川貴弘, 長谷山美紀, AI・データサイエンス論文集(Web), 5, 1, 2024年

大規模言語モデルを用いた 通報音声からの事象発生地点予測
吉田将規, 前田圭介, 藤後廉, 小川貴弘, 長谷山美紀, AI・データサイエンス論文集(Web), 5, 1, 2024年

サッカー映像における時空間的関係を考慮したシュート予測の高精度化に関する検討 : 競技者のチーム情報に基づく完全二部グラフの導入—A Note on Accurate Shoot Prediction Considering Players' Spatio-temporal Relations in Soccer Videos : Introduction of Complete Bipartite Graph Based on Players' Team Information—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
五箇 亮太, 諸戸 祐哉, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 47, 6, 23, 28, 2023年02月
東京 : 映像情報メディア学会, 日本語

クラス情報を導入したグラフ表現による教師有り潜在変数モデルの高精度化に関する検討—A Note on Improvement of Supervised Latent Variable Model with Graph-Encoded Class Information—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
渡部 航史, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 47, 6, 29, 33, 2023年02月
東京 : 映像情報メディア学会, 日本語

属性情報の階層関係を考慮したアニメイラストのマルチラベル分類に関する検討—A Note on Multi-label Image Classification in Animation Illustration Considering Hierarchical Relationships of Attributes—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
蘭 子文, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 47, 6, 5, 9, 2023年02月
東京 : 映像情報メディア学会, 日本語

ユーザの視線情報を考慮したコンテンツベースの画像再検索に関する検討—A note on gaze-dependent image re-ranking for content-based image retrieval—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
馮 鈺虎, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 47, 6, 89, 93, 2023年02月
東京 : 映像情報メディア学会, 日本語

Spatial Temporal Attention Graph Convolutional Networkによるモーションデータを用いた熟練度の分類に関する検討
清野竜生, 斉藤直輝, 小川貴弘, 浅水仁, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2023, 2023年

変状の劣化レベル判定支援に向けたグラフニューラルネットワークによるレコードデータの特徴表現の獲得
山本一輝, 前田圭介, 藤後廉, 小川貴弘, 長谷山美紀, AI・データサイエンス論文集(Web), 4, 3, 2023年

地下鉄トンネル点検技術者の新規変状の発見率向上に向けた一人称視点映像からの変状検出
櫻井慶悟, 前田圭介, 藤後廉, 小川貴弘, 長谷山美紀, AI・データサイエンス論文集(Web), 4, 3, 2023年

時系列データを用いたMulti-modal Transformerに基づく冬期路面状態の分類
諸戸祐哉, 前田圭介, 藤後廉, 小川貴弘, 長谷山美紀, AI・データサイエンス論文集(Web), 4, 3, 2023年

北海道大学におけるデータ駆動型融合研究創発の取組
長谷山美紀, 水谷祐輔, 田中晋吾, 映像情報メディア学会技術報告, 47, 6(MMS2023 1-34/ME2023 21-54/AIT2023 1-34), 2023年

データ横断型対照学習を用いた道路構造物における変状画像の劣化レベル分類
東孝明, 小川直輝, 前田圭介, 小川貴弘, 長谷山美紀, AI・データサイエンス論文集(Web), 4, 2, 2023年

属性情報の階層関係を考慮したアニメイラストのマルチラベル分類に関する検討
LAM Ziwen, 前田圭介, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 47, 6(MMS2023 1-34/ME2023 21-54/AIT2023 1-34), 2023年

データベース特化型クロスモーダル画像検索のためのテキストプロンプトチューニングに関する検討
ZHANG Huaying, 柳凛太郎, 藤後廉, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 47, 6(MMS2023 1-34/ME2023 21-54/AIT2023 1-34), 2023年

360度映像におけるマルチビュー学習に基づくバイノーラル音声生成の高精度化に関する検討
吉田将規, 藤後廉, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 47, 6(MMS2023 1-34/ME2023 21-54/AIT2023 1-34), 2023年

クラス情報を導入したグラフ表現による教師有り潜在変数モデルの高精度化に関する検討
渡部航史, 前田圭介, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 47, 6(MMS2023 1-34/ME2023 21-54/AIT2023 1-34), 2023年

ユーザの視線情報を考慮したコンテンツベースの画像再検索に関する検討
FENG Yuhu, 前田圭介, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 47, 6(MMS2023 1-34/ME2023 21-54/AIT2023 1-34), 2023年

サッカー映像における時空間的関係を考慮したシュート予測の高精度化に関する検討-競技者のチーム情報に基づく完全二部グラフの導入-
五箇亮太, 諸戸祐哉, 前田圭介, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 47, 6(MMS2023 1-34/ME2023 21-54/AIT2023 1-34), 2023年

物体検出モデルに基づく視覚表現を用いた解釈可能なVisual Question Answeringモデルに関する検討
ZHU He, 藤後廉, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 47, 6(MMS2023 1-34/ME2023 21-54/AIT2023 1-34), 2023年

知識蒸留を用いたFew-shot Learningに基づく画像の感情ラベル推定に関する検討—A Note on Visual Sentiment Prediction Based on Few-shot Learning Using Knowledge Distillation—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
叶 穎睿, 諸戸 祐哉, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 46, 6, 171, 175, 2022年02月
日本語

Variational Autoencoderに基づく深層生成モデルを用いた潜在表現のDisentanglementに関する検討 : Disentanglement評価指標を含む正則化損失の導入—A Note on Disentanglement Using Deep Generative Model Based on Variational Autoencoder : Introduction of Regularization Losses Based on Metrics of Disentangled Representation—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
中川 真, 藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 46, 6, 97, 102, 2022年02月
日本語

Captioning特徴を利用したグラフ畳み込みネットワークに基づくアニメイラストのマルチラベル画像分類に関する検討—A Note on Multi-label Image Recognition in Anime Illustration Based on Graph Convolutional Networks Using Captioning Features—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
蘭 子文, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 46, 6, 161, 165, 2022年02月
日本語

ゴム材料開発のためのGenerative Adversarial Networkに基づく配合量および物性からの電子顕微鏡画像の生成に関する一検討—A Note on Electron Microscope Image Generation from Mix Proportion and Material Property via Generative Adversarial Network for Rubber Materials—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
柳 凛太郎, 藤後 廉, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 46, 6, 187, 191, 2022年02月
日本語

セマンティックセグメンテーションに対するマルチソースモデル適応に関する検討 : 複数のソースモデルからの不変な特徴表現の学習による適応精度の向上—A note on multi-source model adaptation for semantic segmentation : Improving adaptation performance by learning model-invariant representation from multiple source models—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
李 宗曜, 藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 46, 6, 37, 41, 2022年02月
日本語

地下鉄トンネル点検時の技術者から取得される生体信号と技術者の点検行動の関連性分析—Relevance Analysis between Bio-signals of Engineers Inspecting Subway Tunnels and Their Inspection Behaviors—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
平澤 魁人, 前田 圭介, 藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 46, 6, 365, 370, 2022年02月
日本語

高速道路の遮音壁画像を用いた物体検出手法による変状分類の高精度化に関する検討—A note on improvement of distress classification using noise barrier images on highway via object detection method—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
梁 鋆, 前田 圭介, 藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 46, 6, 359, 363, 2022年02月
日本語

道路構造物の維持管理効率化に向けた変状画像分類の高精度化に関する検討 : テキストデータに基づく類似事例の含有率の導入—A Note on Improvement of Accuracy in Classification of Destress Images for Efficient Inspection of Road Structures : Introduction of Ratio of Similar Cases Based on Text Data—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
平川 泰成, 小川 直輝, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 46, 6, 43, 48, 2022年02月
日本語

地下鉄トンネルの維持管理支援のためのマルチスケール解析を導入した深層学習に基づく変状検出に関する検討—A Note on Distress Detection Based on Deep Learning with Hierarchical Multi-Scale Attention Mechanism for Supporting Maintenance of Subway Tunnels—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
高田 紗弥, 前田 圭介, 藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 46, 6, 377, 381, 2022年02月
日本語

A Note on Automatic Diagnosis of Helicobacter Pylori Infection Based on Self-Supervised Learning and Self-Knowledge Distillation: マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
トウゴ レン, マベ カツヒロ, ニシダ シュンペイ, トモダ ヨシヒロ, オガワ タカヒロ, ハセヤマ ミキ, 46, 6, 49, 52, 2022年02月
英語

A Note on Realizing Adversarial Defense Based on Regularization of Multi-stage Squeeze-and-Excitation Features: マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
マエダ ケイスケ, オガワ タカヒロ, ハセヤマ ミキ, 46, 6, 87, 90, 2022年02月
英語

ユーザの動作情報を用いたコンテンツの関心度推定に関する検討 : 複数ユーザを導入した特徴統合の有効性検証—A Note on Interest level Estimation Using Users' behavior Information : Validating the Effectiveness of Feature Integration with Multiple Users—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
上川 恭平, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 46, 6, 103, 107, 2022年02月
日本語

画像注視時の脳活動信号を用いた圧縮再構成ネットワークに基づく視覚認知内容の推定に関する検討—A Note on Perceived Visual Content Estimation Based on Compressed Reconstruction Network Using Brain Signals While Gazing on Images—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
東 孝明, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 46, 6, 349, 353, 2022年02月
日本語

Attention mapに対する確信度を考慮可能な深層学習を用いた変状分類の高精度化に関する検討—A Note on Accurate Distress Classification Using Deep Learning Considering Confidence in Attention map—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
小川 直輝, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 46, 6, 371, 376, 2022年02月
日本語

A Note on Transformer-based Scene Recognition in Soccer Videos Using Different Lengths of Clips: マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
トウゴ レン, オガワ タカヒロ, ハセヤマ ミキ, 46, 6, 167, 170, 2022年02月
英語

画像中の物体情報を考慮したユーザ類似度に基づく個人に特化した注視領域の推定に関する検討—A Note on Personalized Saliency Prediction Based on User Similarity Considering Object Information in Images—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
諸戸 祐哉, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 46, 6, 181, 186, 2022年02月
日本語

橋梁点検時の技術者の一人称および三人称視点映像を用いた点検動作の分類に関する検討—A Note on Inspection Action Classification Using First and Third Person Video of Engineers Inspecting Bridges—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
増田 毅, 前田 圭介, 藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 46, 6, 177, 180, 2022年02月
日本語

ユーザの嗜好を考慮した強化学習と知識グラフに基づく楽曲プレイリスト生成に関する検討—A Note on User Preference-Aware Music Playlist Generation Based on Reinforcement Learning and Knowledge Graph—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
櫻井 慶悟, 藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 46, 6, 109, 112, 2022年02月
日本語

AIの実社会応用とインフラ分野の将来像
長谷山美紀, 高速道路と自動車, 65, 12, 2022年

異種特徴間の相関およびAttention Mapの確信度を考慮可能な変状画像の劣化レベル分類
小川直輝, 前田圭介, 小川貴弘, 長谷山美紀, AI・データサイエンス論文集(Web), 3, J2, 2022年

テキストおよび画像情報に基づくFocal Lossを導入した深層学習による冬期路面状態の分類
諸戸祐哉, 前田圭介, 藤後廉, 小川貴弘, 長谷山美紀, AI・データサイエンス論文集(Web), 3, J2, 2022年

地下鉄トンネル点検時の一人称視点映像を用いたVision Transformerに基づく変状検出
櫻井慶悟, 前田圭介, 藤後廉, 小川貴弘, 長谷山美紀, AI・データサイエンス論文集(Web), 3, J2, 2022年

インフラ施設の変状の評価を支援する効率的な映像提示に向けた技術者の点検動作分類
上川恭平, 前田圭介, 藤後廉, 小川貴弘, 長谷山美紀, AI・データサイエンス論文集(Web), 3, J2, 2022年

セマンティックセグメンテーションに対するマルチソースモデル適応に関する検討-複数のソースモデルからの不変な特徴表現の学習による適応精度の向上-
LI Zongyao, 藤後廉, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 46, 6(MMS2022 1-37/ME2022 26-62/AIT2022 1-37), 2022年

ユーザの嗜好を考慮した強化学習と知識グラフに基づく楽曲プレイリスト生成に関する検討
櫻井慶悟, 藤後廉, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 46, 6(MMS2022 1-37/ME2022 26-62/AIT2022 1-37), 2022年

Variational Autoencoderに基づく深層生成モデルを用いた潜在表現のDisentanglementに関する検討-Disentanglement評価指標を含む正則化損失の導入-
中川真, 藤後廉, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 46, 6(MMS2022 1-37/ME2022 26-62/AIT2022 1-37), 2022年

協調フィルタリングにおける潜在因子モデルの埋め込み表現と人気バイアスの関係の検討
岡村洋希, 前田圭介, 藤後廉, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2022, 2022年

ドメイン共有ネットワークに基づく嗜好のドメイン依存性に頑健な推薦システムに関する検討
山本一輝, 前田圭介, 藤後廉, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2022, 2022年

pixelNeRFによる生成画像を用いた視点の変化に頑健な画像検索手法に関する検討
江良勇輝, 藤後廉, 前田圭介, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2022, 2022年

フーリエ振幅成分を考慮したNeural Radiance Fieldsのノンリファレンス評価指標に関する検討
河合雅斗, 柳凛太郎, 藤後廉, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2022, 2022年

サッカー競技のスカウティング映像における選手間の時空間的関係を考慮した不確実性に基づくシュートイベント予測に関する検討
五箇亮太, 諸戸祐哉, 前田圭介, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2022, 2022年

画像注視時の脳活動データを用いたVQAモデルに基づく認知内容推定に関する検討-fMRIデコーダに用いる回帰モデルによる推定精度に関する検証-
七田亮, 藤後廉, 前田圭介, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2022, 2022年

眼底画像を用いた教師なしドメイン適応に基づく糖尿病性網膜症の重症度の推定に関する検討
國枝翼, 藤後廉, 前田圭介, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2022, 2022年

ゴム材料開発のためのGenerative Adversarial Networkに基づく配合量および物性からの電子顕微鏡画像の生成に関する一検討
柳凜太郎, 藤後廉, 前田圭介, 前田圭介, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 46, 6(MMS2022 1-37/ME2022 26-62/AIT2022 1-37), 2022年

道路構造物の維持管理効率化に向けた変状画像分類の高精度化に関する検討-テキストデータに基づく類似事例の含有率の導入-
平川泰成, 小川直輝, 前田圭介, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 46, 6(MMS2022 1-37/ME2022 26-62/AIT2022 1-37), 2022年

地下鉄トンネルの維持管理支援のためのマルチスケール解析を導入した深層学習に基づく変状検出に関する検討
高田紗弥, 前田圭介, 藤後廉, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 46, 6(MMS2022 1-37/ME2022 26-62/AIT2022 1-37), 2022年

画像注視時の脳活動信号を用いた圧縮再構成ネットワークに基づく視覚認知内容の推定に関する検討
東孝明, 前田圭介, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 46, 6(MMS2022 1-37/ME2022 26-62/AIT2022 1-37), 2022年

ユーザの動作情報を用いたコンテンツの関心度推定に関する検討-複数ユーザを導入した特徴統合の有効性検証-
上川恭平, 前田圭介, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 46, 6(MMS2022 1-37/ME2022 26-62/AIT2022 1-37), 2022年

知識蒸留を用いたFew-shot Learningに基づく画像の感情ラベル推定に関する検討
YE Yingrui, 諸戸祐哉, 前田圭介, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 46, 6(MMS2022 1-37/ME2022 26-62/AIT2022 1-37), 2022年

地下鉄トンネル点検時の技術者から取得される生体信号と技術者の点検行動の関連性分析
平澤魁人, 前田圭介, 藤後廉, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 46, 6(MMS2022 1-37/ME2022 26-62/AIT2022 1-37), 2022年

画像中の物体情報を考慮したユーザ類似度に基づく個人に特化した注視領域の推定に関する検討
諸戸祐哉, 前田圭介, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 46, 6(MMS2022 1-37/ME2022 26-62/AIT2022 1-37), 2022年

Attention mapに対する確信度を考慮可能な深層学習を用いた変状分類の高精度化に関する検討
小川直輝, 前田圭介, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 46, 6(MMS2022 1-37/ME2022 26-62/AIT2022 1-37), 2022年

AIの実社会応用と次世代インフラ維持管理
長谷山 美紀, 学士会会報, 2022, 1, 78, 83, 2022年01月
学士会, 日本語

招待講演 マルチメディアAI技術に基づく異分野融合研究と実社会応用—Invited Talk : Interdisciplinary Research and Real-world Applications Based on Multimedia AI technologies—情報センシング メディア工学 映像・信号の処理・解析・AI技術とその多分野応用
小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 45, 13, 73, 80, 2021年06月
映像情報メディア学会, 日本語

A Note on Distress Classification of High-resolution Subway Tunnel Images based on HRNet: マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
トウゴ レン, オガワ タカヒロ, ハセヤマ ミキ, 45, 4, 13, 16, 2021年02月
英語

地下鉄トンネルの維持管理支援を目的とした深層学習に基づく変状検出の高精度化に関する検討 : 壁面の施工方法に注目した精度検証—A Note on Improving Performance of Deep Learning-based Distress Detection for Supporting Maintenance of Subway Tunnels : Accuracy Verification Focusing on Tunnel Wall Characteristics—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
春山 知生, 前田 圭介, 藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 45, 4, 1, 6, 2021年02月
映像情報メディア学会, 日本語

特別講演 実店舗の防犯カメラ映像データを用いた顧客の関心推定に関する検討 : 姿勢推定モデルから得られる特徴量を用いた異常検知モデルの精度検証—A Note on Customer's Interest Estimation Method Using Security Camera Video Data from the Real Store : Validation of the accuracy of an anomaly detection model using features obtained from a posture estimation model—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
本間 勇紀, 藤後 廉, 阿部 真育, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 45, 4, 193, 198, 2021年02月
映像情報メディア学会, 日本語

地下鉄トンネル点検時の技術者から取得される視線およびモーションデータに基づく熟練度の推定に関する検討—A Note on Estimation of Expert-Novice Level Based on Eye Tracking and Motion Data from Engineers While Inspecting Subway Tunnel—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
赤松 祐亮, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 45, 4, 7, 12, 2021年02月
映像情報メディア学会, 日本語

特別講演 ゴム材料開発のためのConditional StyleGANに基づく配合量からの電子顕微鏡画像の生成に関する一検討—A Note on Electron Microscope Image Generation from Mix Proportion via Conditional Style Generative Adversarial Network for Rubber Materials—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
柳 凜太郎, 藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 45, 4, 171, 175, 2021年02月
映像情報メディア学会, 日本語

特別講演 路面画像を用いた深層学習に基づく路面状態の分類に関する検討—A Note on Discrimination of Road Surface Conditions Based on Deep Learning Using Road Images—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
諸戸 祐哉, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 45, 4, 165, 169, 2021年02月
映像情報メディア学会, 日本語

TCCに基づく自己教師学習による特徴表現を用いた映像中の人物動作の検出に関する検討 : 動作検出パラメータが与える影響に対する考察—A Note on Action Detection Using Feature Representation via Self-Supervised Learning based on TCC : Effectiveness evaluation of parameters for action detection—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
増田 毅, 藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 45, 4, 47, 51, 2021年02月
映像情報メディア学会, 日本語

電子顕微鏡により撮像されたゴム材料からの異常検知に基づく劣化領域の推定に関する一検討 : 深層学習モデルにより抽出された特徴表現の有効性検証—A Note on Estimation of Deteriorated Regions Based on Anomaly Detection from Rubber Material Electron Microscope Images : Verification of Feature Representations Extracted from Deep Learning Models—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
松本 真直, 藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 45, 4, 43, 46, 2021年02月
映像情報メディア学会, 日本語

サッカー映像における情報の提示位置の推定に関する検討 : 視聴者の注視位置および選手位置に基づいたオブジェクト配置の決定—A Note on Position Estimation of Superimposing Information in Soccer Videos : Decision of Object Placement Based on User's Gaze Position and Players' Positions—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
鈴木 元樹, 高橋 翔, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 45, 4, 77, 81, 2021年02月
映像情報メディア学会, 日本語

A Note on Automatic Diagnosis of Helicobacter Pylori Infection Based on EfficientNet with Flooding Loss: マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
トウゴ レン, マベ カツヒロ, ニシダ シュンペイ, トモダ ヨシヒロ, シミズ ヒカリ, オガワ タカヒロ, ハセヤマ ミキ, 45, 4, 23, 26, 2021年02月
英語

Twitterと映像を用いたMVAEに基づく野球映像の重要シーン予測に関する検討—A Note on Prediction of Important Scenes in Baseball Videos via Multimodal Variational Autoencoder Using Tweets and Videos—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
平澤 魁人, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 45, 4, 71, 75, 2021年02月
映像情報メディア学会, 日本語

Detecting axillary lymph node metastasis of breast cancer with FDG-PET/CT images based on attention mechanism: FDG-PET/CT画像を用いたattention mechanismに基づく乳癌の腋窩リンパ節転移の検出: マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
トウゴ レン, ヒラタ ケンジ, キタジマ カズヒロ, タケナカ ジュンキ, ミヨシ ヤスオ, クドウ コウスケ, オガワ タカヒロ, ハセヤマ ミキ, 45, 4, 33, 36, 2021年02月
英語

特別講演 ユーザの嗜好を反映可能なインテリアコーディネート検索技術の構築 : コーディネートを表現可能な特徴の抽出と実データへの適用—A Note on Interior Coordination Retrieval Reflecting User's Preferences : Extraction of Features Representing Coordination and Application to Real Data—マルチメディアストレージ メディア工学 映像表現&コンピュータグラフィックス
藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 45, 4, 187, 191, 2021年02月
映像情報メディア学会, 日本語

Audio-visual立体化モデルを用いた画面外音声検出に関する検討-自己教師あり学習手法に基づくAudio-visual特徴表現の導入-
吉田将規, 藤後廉, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2021, 2021年

画像特徴の表現能力が世界モデルのパフォーマンスに与える影響の検証
大羽賀駿也, 藤後廉, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2021, 2021年

Shilling attackの状況下におけるグラフ解析に基づく推薦システムの脆弱性の検証
小野寺望, 前田圭介, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2021, 2021年

グラフ畳み込みネットワークに基づくアニメイラストのマルチラベル画像認識に関する検討
LAN Ziwen, 前田圭介, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2021, 2021年

自己教師あり学習を導入した潜在変数モデルによる低次元特徴の抽出に関する検討
渡部航史, 前田圭介, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2021, 2021年

Few-shot Learningを用いた感情ラベル推定における複数のデータセット利用に関する初期検討
YE Yingrui, 諸戸祐哉, 前田圭介, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2021, 2021年

テキスト入力型画像操作における操作領域に着目した精度評価に関する検討
渡邉優宇人, 藤後廉, 前田圭介, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2021, 2021年

画像注視時の脳活動データを用いた質問応答生成モデルに基づく意味内容の推定に関する検討-fine-tuningに基づく高精度化-
高田紗弥, 藤後廉, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 45, 4(MMS2021 1-28/ME2021 1-28/AIT2021 1-28), 27, 31, 2021年
映像情報メディア学会, 日本語

Image Captioningの導入による画像の感情推定の高精度化に関する検討
LIANG Yun, 前田圭介, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 45, 4(MMS2021 1-28/ME2021 1-28/AIT2021 1-28), 65, 69, 2021年
映像情報メディア学会, 日本語

画像特徴を用いた多層グラフ解析に基づくクロスドメイン推薦に関する検討
平川泰成, 前田圭介, 小川貴弘, 浅水仁, 長谷山美紀, 映像情報メディア学会技術報告, 45, 4(MMS2021 1-28/ME2021 1-28/AIT2021 1-28), 59, 63, 2021年
映像情報メディア学会, 日本語

地下鉄トンネルの維持管理支援を目的とした深層学習に基づく変状検出の高精度化に関する検討-壁面の施工方法に注目した精度検証-
春山知生, 前田圭介, 藤後廉, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 45, 4(MMS2021 1-28/ME2021 1-28/AIT2021 1-28), 2021年

実店舗の防犯カメラ映像データを用いた顧客の関心推定に関する検討-姿勢推定モデルから得られる特徴量を用いた異常検知モデルの精度検証-
本間勇紀, 本間勇紀, 藤後廉, 阿部真育, 小川貴弘, 長谷山美紀, 長谷山美紀, 映像情報メディア学会技術報告, 45, 4(MMS2021 1-28/ME2021 1-28/AIT2021 1-28), 2021年

電子顕微鏡により撮像されたゴム材料からの異常検知に基づく劣化領域の推定に関する一検討~深層学習モデルにより抽出された特徴表現の有効性検証~
松本真直, 藤後廉, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 45, 4(MMS2021 1-28/ME2021 1-28/AIT2021 1-28), 2021年

地下鉄トンネル点検時の技術者から取得される視線およびモーションデータに基づく熟練度の推定に関する検討
赤松祐亮, 前田圭介, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 45, 4(MMS2021 1-28/ME2021 1-28/AIT2021 1-28), 2021年

TCCに基づく自己教師学習による特徴表現を用いた映像中の人物動作の検出に関する検討-動作検出パラメータが与える影響に対する考察-
増田毅, 藤後廉, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 45, 4(MMS2021 1-28/ME2021 1-28/AIT2021 1-28), 2021年

路面画像を用いた深層学習に基づく路面状態の分類に関する検討
諸戸祐哉, 前田圭介, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 45, 4(MMS2021 1-28/ME2021 1-28/AIT2021 1-28), 2021年

サッカー映像における情報の提示位置の推定に関する検討-視聴者の注視位置および選手位置に基づいたオブジェクト配置の決定-
鈴木元樹, 高橋翔, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 45, 4(MMS2021 1-28/ME2021 1-28/AIT2021 1-28), 2021年

Twitterと映像を用いたMVAEに基づく野球映像の重要シーン予測に関する検討
平澤魁人, 前田圭介, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 45, 4(MMS2021 1-28/ME2021 1-28/AIT2021 1-28), 2021年

ユーザの嗜好を反映可能なインテリアコーディネート検索技術の構築-コーディネートを表現可能な特徴の抽出と実データへの適用-
藤後廉, 小川貴弘, 長谷山美紀, 長谷山美紀, 映像情報メディア学会技術報告, 45, 4(MMS2021 1-28/ME2021 1-28/AIT2021 1-28), 2021年

Attention Mapを用いた道路構造物の変状画像分類におけるテキストデータの導入に基づく高精度化に関する検討
小川直輝, 前田圭介, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 45, 4(MMS2021 1-28/ME2021 1-28/AIT2021 1-28), 17, 21, 2021年
映像情報メディア学会, 日本語

ゴム材料開発のためのConditional StyleGANに基づく配合量からの電子顕微鏡画像の生成に関する一検討
柳凜太郎, 藤後廉, 小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 45, 4(MMS2021 1-28/ME2021 1-28/AIT2021 1-28), 2021年

AIを中心とした医療デジタル技術基盤の構築へ向けた取り組み
藤後廉, 小川貴弘, 長谷山美紀, 日本消化器がん検診学会雑誌(Web), 59, Supplement 1, 2021年

マルチメディアAI技術に基づく異分野融合研究と実社会応用
小川貴弘, 長谷山美紀, 映像情報メディア学会技術報告, 45, 13(IST2021 22-29/ME2021 53-60), 2021年

消化管がんの検診・スクリーニングにおけるトピックス 5 ピロリ菌感染胃炎の胃X線スクリーニングにおけるAIの活用
竹内千尋, 藤後廉, 小川貴弘, 長谷山美紀, 山道信毅, 臨床消化器内科, 36, 8, 1062, 1065, 2021年
日本メディカルセンター

データサイエンスのあるべき姿とは : 高等教育・研究機関における取り組みを通じての意見交換会—特集 データサイエンス教育の潮流(その1)
木島 正明, 竹村 彰通, 長谷山 美紀, 椿 広計, 蓮池 隆, オペレーションズ・リサーチ = Communications of the Operations Research Society of Japan : 経営の科学, 65, 10, 525, 536, 2020年10月
日本オペレーションズ・リサーチ学会 ; 1956-, 日本語

メディア工学研究委員会—Technical Group on Media Engineering—特集 学会創立70周年記念特集 ; 映像情報メディア学会の研究会活動 : 過去から未来へ
村上 和人, 長谷山 美紀, 田川 憲男, 田良島 周平, 石井 大祐, 映像情報メディア学会誌 = The journal of the Institute of Image Information and Television Engineers, 74, 5, 779, 783, 2020年09月
映像情報メディア学会, 日本語

画像内の物体に着目した画像検索に関する検討 : RetinaNetを用いた物体認識に基づく高精度化—A Note on Image Retrieval Focusing on Objects in Images : Improving Retrieval Performance Based on Object Recognition Using RetinaNet—マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス
柳 凛太郎, 藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 44, 6, 377, 381, 2020年02月
映像情報メディア学会, 日本語

画像注視時の視線データを用いた関心度の推定に関する検討 : ラベルの逆量子化を導入したsMVCCAによる推定精度の高精度化—A Note on Eye Gaze Based User-specific Interest Estimation for Images : Estimation Performance Improvement Based on sMVCCA including Ordinal Label Dequantization—ITS : Intelligent Transport Systems Technology
松本 真直, 斉藤 直輝, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 119, 421, 73, 77, 2020年02月
電子情報通信学会, 日本語

画像注視時の視線データを用いた関心度の推定に関する検討 : ラベルの逆量子化を導入したsMVCCAによる推定精度の高精度化—A Note on Eye Gaze Based User-specific Interest Estimation for Images : Estimation Performance Improvement Based on sMVCCA including Ordinal Label Dequantization—マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス
松本 真直, 斉藤 直輝, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 44, 6, 73, 77, 2020年02月
映像情報メディア学会, 日本語

地下鉄トンネル点検時の生体信号に基づいた熟練および若手技術者の分類に関する検討—A Note on Classification of Experienced and Novice Inspectors Based on Bio-signals While Inspecting in Subway Tunnels—ITS : Intelligent Transport Systems Technology
九島 哲哉, 前田 圭介, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 119, 421, 101, 105, 2020年02月
電子情報通信学会, 日本語

特別講演 社会インフラ維持管理効率化のための最先端AI技術の導入 : 点検データを用いた変状分類の精度向上に向けた取り組み—Introduction of Advanced AI Technology for Efficient Maintenance Inspection of Social Infrastructure : Performance Improvement of Distress Image Classification Utilizing Inspection Data—ITS : Intelligent Transport Systems Technology
前田 圭介, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 119, 421, 359, 362, 2020年02月
電子情報通信学会, 日本語

道路構造物の変状評価における技術者の視線データと熟練度の分析に関する一考察—A Note on Analysis of Gaze Data and Skills of Inspectors in Distress Assessment of Road Structures—マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス
松井 太我, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 44, 6, 97, 100, 2020年02月
映像情報メディア学会, 日本語

道路構造物の維持管理効率化に向けた変状画像分類における信頼性の向上に関する検討—A Note on Improvement of Reliability of Distress Image Classification for Efficient Maintenance of Road Structures—ITS : Intelligent Transport Systems Technology
堀井 風葉, 前田 圭介, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 119, 421, 51, 56, 2020年02月
電子情報通信学会, 日本語

画像内の物体に着目した画像検索に関する検討 : RetinaNetを用いた物体認識に基づく高精度化—A Note on Image Retrieval Focusing on Objects in Images Improving Retrieval Performance Based on Object Recognition Using RetinaNet—ITS : Intelligent Transport Systems Technology
柳 凜太郎, 藤後 廉, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 119, 421, 377, 381, 2020年02月
電子情報通信学会, 日本語

弱異常検知アルゴリズムに基づくCCTV映像を用いた河川利用者の異常行動の検出に関する検討—A Note on Abnormal Motion Detection of River Users on CCTV Videos Based on Weakly Supervised Anomaly Detection Algorithm—マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス
渡邊 はるな, 藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 44, 6, 367, 370, 2020年02月
映像情報メディア学会, 日本語

A note on retrieval of visually similar distress regions in subway tunnel images : Introduction of deep features extracted by semantic segmentation network: ITS : Intelligent Transport Systems Technology
トウゴ レン, オガワ タカヒロ, ハセヤマ ミキ, 119, 421, 65, 68, 2020年02月
英語

画像注視時のヒトの感情推定のための視線特徴の推定に関する検討—A Note on Estimation of Gaze Features for Human Emotion Estimation while Viewing Images—マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス
諸戸 祐哉, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 44, 6, 85, 89, 2020年02月
映像情報メディア学会, 日本語

ゴム材料の配合量を用いたAC-GANに基づく電子顕微鏡画像の生成に関する一検討—A Note on Generation of Electron Microscope Images via Auxiliary Classifier Generative Adversarial Network with Mix Proportions—ITS : Intelligent Transport Systems Technology
金井 美岬, 藤後 廉, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 119, 421, 107, 111, 2020年02月
電子情報通信学会, 日本語

ゴム材料の配合量を用いたAC-GANに基づく電子顕微鏡画像の生成に関する一検討—A Note on Generation of Electron Microscope Images via Auxiliary Classifier Generative Adversarial Network with Mix Proportions—マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス
金井 美岬, 藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 44, 6, 107, 111, 2020年02月
映像情報メディア学会, 日本語

道路構造物の変状評価における技術者の視線データと熟練度の分析に関する一考察—A Note on Analysis of Gaze Data and Skills of Inspectors in Distress Assessment of Road Structures—ITS : Intelligent Transport Systems Technology
松井 太我, 前田 圭介, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 119, 421, 97, 100, 2020年02月
電子情報通信学会, 日本語

エッジの補完を導入したツイートクラスタリングの高精度化に関する検討—A Note on Tweet Clustering by Complementing Edges among Tweets—ITS : Intelligent Transport Systems Technology
滝村 祥司, 原川 良介, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 119, 421, 61, 64, 2020年02月
電子情報通信学会, 日本語

地下鉄トンネル維持管理支援を目的とした類似画像の検索に関する検討 : 技術者の評価を反映可能な距離計量学習の導入—A Note on Retrieval of Similar Images for Supporting Maintenance of Subway Tunnels : Introduction of Distance Metric Learning Reflecting Inspectors' Evaluation—マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス
松本 有衣, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 44, 6, 47, 50, 2020年02月
映像情報メディア学会, 日本語

特別講演 社会インフラ維持管理効率化のための最先端AI技術の導入 : 点検データを用いた変状分類の精度向上に向けた取り組み—Introduction of Advanced AI Technology for Efficient Maintenance Inspection of Social Infrastructure : Performance Improvement of Distress Image Classification Utilizing Inspection Data—マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス
前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 44, 6, 359, 362, 2020年02月
映像情報メディア学会, 日本語

地下鉄トンネル維持管理支援を目的とした類似画像の検索に関する検討 : 技術者の評価を反映可能な距離計量学習の導入—A Note on Retrieval of Similar Images for Supporting Maintenance of Subway Tunnels : Introduction of Distance Metric Learning Reflecting Inspectors' Evaluation—ITS : Intelligent Transport Systems Technology
松本 有衣, 前田 圭介, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 119, 421, 47, 50, 2020年02月
電子情報通信学会, 日本語

穿孔データを用いたオンライン学習に基づく岩盤の圧縮強度指数推定に関する検討—A Note on Estimation of rock compressive strength index from drilling data based on online learning—マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス
山本 健太郎, 藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 44, 6, 57, 60, 2020年02月
映像情報メディア学会, 日本語

穿孔データを用いたオンライン学習に基づく岩盤の圧縮強度指数推定に関する検討—A Note on Estimation of rock compressive strength index from drilling data based on online learning—ITS : Intelligent Transport Systems Technology
山本 健太郎, 藤後 廉, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 119, 421, 57, 60, 2020年02月
電子情報通信学会, 日本語

サッカー映像視聴時の視線データを用いた情報の提示領域の算出に関する検討—Presentation Region Calculation of Information Based on Gaze Tracking Data in Soccer Videos—ITS : Intelligent Transport Systems Technology
鈴木 元樹, 高橋 翔, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 119, 421, 91, 95, 2020年02月
電子情報通信学会, 日本語

地下鉄トンネル点検時の生体信号に基づいた熟練および若手技術者の分類に関する検討—マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス
九島 哲哉, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 44, 6, 101, 105, 2020年02月
映像情報メディア学会, 日本語

Bidirectional LSTMを用いたサッカー映像におけるイベント分類に関する検討—A Note on Event Classification in Soccer Videos Using Bidirectional LSTM—ITS : Intelligent Transport Systems Technology
春山 知生, 高橋 翔, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 119, 421, 371, 375, 2020年02月
電子情報通信学会, 日本語

画像注視時のヒトの感情推定のための視線特徴の推定に関する検討—A Note on Estimation of Gaze Features for Human Emotion Estimation while Viewing Images—ITS : Intelligent Transport Systems Technology
諸戸 祐哉, 前田 圭介, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 119, 421, 85, 89, 2020年02月
電子情報通信学会, 日本語

道路構造物の維持管理効率化に向けた変状画像分類における信頼性の向上に関する検討—A Note on Improvement of Reliability of Distress Image Classification for Efficient Maintenance of Road Structures—マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス
堀井 風葉, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 44, 6, 51, 56, 2020年02月
映像情報メディア学会, 日本語

エッジの補完を導入したツイートクラスタリングの高精度化に関する検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
滝村 祥司, 原川 良介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 44, 6, 61, 64, 2020年02月
映像情報メディア学会, 日本語

MVBGM-MSに基づく画像注視時の脳活動データを用いた画像カテゴリの推定に関する検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
赤松 祐亮, 原川 良介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 44, 6, 79, 83, 2020年02月
映像情報メディア学会, 日本語

サッカー映像視聴時の視線データを用いた情報の提示領域の算出に関する検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
鈴木 元樹, 高橋 翔, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 44, 6, 91, 95, 2020年02月
映像情報メディア学会, 日本語

Bidirectional LSTMを用いたサッカー映像におけるイベント分類に関する検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
春山 知生, 高橋 翔, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 44, 6, 371, 375, 2020年02月
映像情報メディア学会, 日本語

エッジの補完を導入したツイートクラスタリングの高精度化に関する検討
滝村祥司, 原川良介, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 119, 421(ITS2019 30-56), 2020年

Bidirectional LSTMを用いたサッカー映像におけるイベント分類に関する検討
春山知生, 高橋翔, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 119, 421(ITS2019 30-56), 2020年

社会インフラ維持管理効率化のための最先端AI技術の導入-点検データを用いた変状分類の精度向上に向けた取り組み-
前田圭介, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 119, 421(ITS2019 30-56), 2020年

サッカー映像視聴時の視線データを用いた情報の提示領域の算出に関する検討
鈴木元樹, 高橋翔, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 119, 421(ITS2019 30-56), 2020年

道路構造物の維持管理効率化に向けた変状画像分類における信頼性の向上に関する検討
堀井風葉, 前田圭介, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 119, 421(ITS2019 30-56), 2020年

地下鉄トンネル維持管理支援を目的とした類似画像の検索に関する検討-技術者の評価を反映可能な距離計量学習の導入-
松本有衣, 前田圭介, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 119, 421(ITS2019 30-56), 2020年

穿孔データを用いたオンライン学習に基づく岩盤の圧縮強度指数推定に関する検討
山本健太郎, 藤後廉, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 119, 421(ITS2019 30-56), 2020年

地下鉄トンネル点検時の生体信号に基づいた熟練および若手技術者の分類に関する検討
九島哲哉, 前田圭介, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 119, 421(ITS2019 30-56), 2020年

弱異常検知アルゴリズムに基づくCCTV映像を用いた河川利用者の異常行動の検出に関する検討
渡邊はるな, 藤後廉, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 119, 421(ITS2019 30-56), 2020年

画像注視時の視線データを用いた関心度の推定に関する検討-ラベルの逆量子化を導入したsMVCCAによる推定精度の高精度化-
松本真直, 斉藤直輝, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 119, 421(ITS2019 30-56), 2020年

北海道大学における数理・データサイエンス教育と人材育成の取組
長谷山美紀, 大本亨, 湧田雄基, 阿部真育, 長谷山美紀, 大本亨, 電子情報通信学会技術研究報告, 119, 421(ITS2019 30-56), 2020年

ゴム材料の配合量を用いたAC-GANに基づく電子顕微鏡画像の生成に関する一検討
金井美岬, 藤後廉, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 119, 421(ITS2019 30-56), 2020年

道路構造物の変状評価における技術者の視線データと熟練度の分析に関する一考察
松井太我, 前田圭介, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 119, 421(ITS2019 30-56), 2020年

MVBGM-MSに基づく画像注視時の脳活動データを用いた画像カテゴリの推定に関する検討
赤松祐亮, 原川良介, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 119, 421(ITS2019 30-56), 2020年

画像内の物体に着目した画像検索に関する検討-RetinaNetを用いた物体認識に基づく高精度化-
柳凛太郎, 藤後廉, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 119, 421(ITS2019 30-56), 2020年

画像注視時のヒトの感情推定のための視線特徴の推定に関する検討
諸戸祐哉, 前田圭介, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 119, 421(ITS2019 30-56), 2020年

胃X線画像を用いたAIによるH.pylori感染識別と今後の展望
藤後廉, 小川貴弘, 間部克裕, 加藤元嗣, 長谷山美紀, 日本消化器がん検診学会雑誌(Web), 58, 2, 2020年

視線データと点検データの正準相関に基づく道路橋点検のための類似点検データ検索
前田圭介, 斉藤僚汰, 高橋翔, 小川貴弘, 長谷山美紀, 土木学会論文集 F3(土木情報学)(Web), 76, 1, 2020年

自己組織化マップを用いた強化学習に基づく楽曲プレイリスト生成に関する検討
櫻井慶悟, 藤後廉, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2020, 2020年

路面画像を用いた異常検知に基づく路面状態の識別に関する検討
諸戸祐哉, 前田圭介, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2020, 2020年

多層グラフ解析に基づくクロスドメイン推薦に関する検討-埋め込み特徴量の次元数の変化による精度検証-
平川泰成, 前田圭介, 小川貴弘, 浅水仁, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2020, 2020年

Temporal Cycle Consistency Learningに基づく人物動作の検出に関する検討
増田毅, 藤後廉, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2020, 2020年

m-SimGPを用いた特徴統合に基づくユーザの関心度推定に関する検討
上川恭平, 前田圭介, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2020, 2020年

Multi-level Deep Metric Netに基づく画像の感情推定に関する検討
LIANG Yun, 前田圭介, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2020, 2020年

nAULSに基づくデータセットの複雑性評価に関する検討
LI Guang, 藤後廉, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2020, 2020年

RICAPに基づくデータ拡張による地下鉄トンネルにおける変状検出精度の向上に関する検討
春山知生, 前田圭介, 藤後廉, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2020, 2020年

画像注視時のfMRIデータを用いた注視画像の推定に関する検討-確率的生成モデルに基づく複数被験者の共通応答の導入-
東孝明, 前田圭介, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2020, 2020年

学習済みセマンティックセグメンテーションモデルを用いたdisentanglementに関する検討
中川真, 藤後廉, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2020, 2020年

Human-centered image classification via a neural network considering visual and biological features.
Kazaha Horii, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, Multimedia Tools Appl., 79, 7-8, 4395, 4415, 2020年
© 2019, Springer Science+Business Media, LLC, part of Springer Nature. In this paper, we propose a human-centered image classification via a neural network considering visual and biological features. The proposed method has two novelties. Firstly, we apply Group-Sparse Local Fisher Discriminant Analysis (GS-LFDA) to biological features. GS-LFDA realizes dimensionality reduction and noise elimination for biological features with consideration of local structures and class information. Secondly, we construct a Canonical Correlation Analysis (CCA)-based hidden layer via Discriminative Locality Preserving CCA (DLPCCA). DLPCCA transforms visual features into effective features by considering the relationships with biological information and class information. The CCA-based hidden layer enables transformation of visual features into effective features for image classification from a small number of training samples. Furthermore, once the projection can be obtained in the training phase, elimination of the need for biological data acquisition in the test phase is realized. This is another merit of our method., SPRINGER, 英語

胃バリウム検査におけるAIによるHelicobacter pylori診断 (Helicobacter Topics! : 「旬のHelicobacter」を知り,驚き,理解を深め,楽しむ)
藤後 廉, 間部 克裕, 山道 信毅, 大泉 晴史, 小川 貴弘, 長谷山 美紀, 加藤 元嗣, 坂本 直哉, Helicobacter research : Journal of helicobacter research, 23, 2, 92,133, 137, 2019年11月
先端医学社, 日本語

画像の視覚的および空間的特徴に基づくユーザに特化した注視領域推定の高精度化に関する検討 : 視覚的特徴の類似度と推定精度の関係性に関する一考察 (イメージ・メディア・クオリティ)
諸戸 祐哉, 前田 圭介, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 119, 131, 13, 16, 2019年07月19日
電子情報通信学会, 日本語

画像注視時の視線データを用いた関心の識別に関する検討 : 推定結果の統合による高精度化 (イメージ・メディア・クオリティ)
松本 真直, 斉藤 直輝, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 119, 131, 17, 20, 2019年07月19日
電子情報通信学会, 日本語

敵対的生成ネットワークにより文から生成される画像の意味的評価に関する検討 (イメージ・メディア・クオリティ)
柳 凜太郎, 藤後 廉, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 119, 131, 21, 24, 2019年07月19日
電子情報通信学会, 日本語

認識・抽出 センサから得られる視聴行動データを活用したユーザの関心推定の高度化
長谷山 美紀, 小川 貴弘, 髙橋 翔, 原川 良介, 画像ラボ, 30, 7, 8, 12, 2019年07月
日本工業出版, 日本語

Detection of gastritis by a deep convolutional neural network from double-contrast upper gastrointestinal barium X-ray radiography
Ren Togo, Nobutake Yamamichi, Katsuhiro Mabe, Yu Takahashi, Chihiro Takeuchi, Mototsugu Kato, Naoya Sakamoto, Kenta Ishihara, Takahiro Ogawa, Miki Haseyama, Journal of Gastroenterology, 54, 4, 321, 329, 2019年04月
© 2018, Japanese Society of Gastroenterology. Background: Deep learning has become a new trend of image recognition tasks in the field of medicine. We developed an automated gastritis detection system using double-contrast upper gastrointestinal barium X-ray radiography. Methods: A total of 6520 gastric X-ray images obtained from 815 subjects were analyzed. We designed a deep convolutional neural network (DCNN)-based gastritis detection scheme and evaluated the effectiveness of our method. The detection performance of our method was compared with that of ABC (D) stratification. Results: Sensitivity, specificity, and harmonic mean of sensitivity and specificity of our method were 0.962, 0.983, and 0.972, respectively, and those of ABC (D) stratification were 0.925, 0.998, and 0.960, respectively. Although there were 18 false negative cases in ABC (D) stratification, 14 of those 18 cases were correctly classified into the positive group by our method. Conclusions: Deep learning techniques may be effective for evaluation of gastritis/non-gastritis. Collaborative use of DCNN-based gastritis detection systems and ABC (D) stratification will provide more reliable gastric cancer risk information., SPRINGER JAPAN KK, 英語

電子顕微鏡により撮像されたゴム材料からの異常検知に基づく劣化領域の推定に関する一検討 (ITS)
藤後 廉, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 118, 449, 265, 268, 2019年02月19日
電子情報通信学会, 日本語

高性能マルチセンシングバンドを利用したストレス検出の精度検証 (ITS)
伊藤 良起, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 118, 449, 269, 274, 2019年02月19日
電子情報通信学会, 日本語

トンネル切羽画像を用いた岩盤の穿孔エネルギー推定に関する初期検討 (ITS)
山本 健太郎, 原川 良介, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 118, 449, 291, 294, 2019年02月19日
電子情報通信学会, 日本語

レーザーデータを用いた地下鉄トンネル内の変状検出に関する検討 : 全層畳み込みネットワークを用いた変状領域の可視化 (ITS)
豊田 陽, 原川 良介, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 118, 449, 295, 299, 2019年02月19日
電子情報通信学会, 日本語

複数の推定結果の統合に基づくTwitterのフォロイー推薦の高精度化に関する一検討 (ITS)
滝村 祥司, 原川 良介, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 118, 449, 307, 310, 2019年02月19日
電子情報通信学会, 日本語

3D residual networkに基づくFDG-PET/CT画像を用いた悪性腫瘍候補の自動検出
李 宗曜, 藤後 廉, 小川 貴弘, 平田 健司, 真鍋 治, 志賀 哲, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 118, 449, 311, 314, 2019年02月19日
電子情報通信学会, 英語

Progressive Growing GANに基づく胃炎識別のための画像生成手法に関する一検討 (ITS)
渡邊 はるな, 藤後 廉, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 118, 449, 319, 322, 2019年02月19日
電子情報通信学会, 日本語

視線データを用いた画像注視時に誘起される感情の推定に関する検討 : KDLPCCAに基づく特徴変換の導入による高精度化 (ITS)
松井 太我, 斉藤 直輝, 小川 貴弘, 浅水 仁, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 118, 449, 323, 327, 2019年02月19日
電子情報通信学会, 日本語

OpenPoseに基づくユーザの動作を用いた映像の関心度推定に関する検討 (ITS)
九島 哲哉, 高橋 翔, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 118, 449, 351, 354, 2019年02月19日
電子情報通信学会, 日本語

CCTV映像を用いた河川利用者の行動分類に関する検討 (ITS)
川本 舜, 原川 良介, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 118, 449, 365, 369, 2019年02月19日
電子情報通信学会, 日本語

CCTV映像を用いた河川利用者の異常検出に関する検討 (ITS)
川本 舜, 原川 良介, 小川 貴弘, 長谷山 美紀, 電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 118, 449, 371, 374, 2019年02月19日
電子情報通信学会, 日本語

異種特徴を用いた深層学習に基づく送電鉄塔の劣化レベル分類の高精度化に関する検討
前田圭介, 高橋翔, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 118, 449(ITS2018 59-85), 361‐364, 2019年02月12日
日本語

異種特徴を活用したネットワーク解析に基づく音楽配信サービスにおけるアーティストの人気度予測に関する検討
松本有衣, 原川良介, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 118, 449(ITS2018 59-85), 301‐306, 2019年02月12日
日本語

地下鉄トンネル維持管理支援のための距離計量学習を用いた類似点検データの検索に関する検討
弦間奨, 原川良介, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 118, 449(ITS2018 59-85), 287‐290, 2019年02月12日
日本語

サッカー映像視聴時の視線データを用いた攻撃選手の重要度算出に関する検討
鈴木元樹, 高橋翔, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 118, 449(ITS2018 59-85), 355‐359, 2019年02月12日
日本語

北海道大学における数理・データサイエンス教育の最前線~学部・大学院におけるMDS教育プログラムの基本コンセプト~
長谷山美紀, 長谷山美紀, 大本亨, 大本亨, 湧田雄基, 電子情報通信学会技術研究報告, 118, 449(ITS2018 59-85), 3‐4, 2019年02月12日
日本語

転移学習を用いた胃X線画像における胃炎識別に関する検討
金井美岬, 藤後廉, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 118, 449(ITS2018 59-85), 315‐318, 2019年02月12日
日本語

畳み込みニューラルネットワークを用いた地下鉄トンネルにおける変状検出の高精度化に関する一検討
WANG An, 原川良介, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 118, 449(ITS2018 59-85), 121‐124, 2019年02月12日
日本語

畳み込みニューラルネットワークにおける解釈性向上のための画像の属性分類に関する一検討
堀井風葉, 前田圭介, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 118, 449(ITS2018 59-85), 275‐279, 2019年02月12日
日本語

電子顕微鏡画像および配合量を用いたゴム材料の特性曲線の推定に関する検討
斉藤直輝, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 118, 449(ITS2018 59-85), 261‐264, 2019年02月12日
日本語

地下鉄トンネルの変状画像を用いた技術者の注視領域推定のための初期検討―深層学習に基づく顕著領域の推定手法の適用―
斉藤僚汰, 前田圭介, 小川貴弘, 長谷山美紀, 電子情報通信学会技術研究報告, 118, 449(ITS2018 59-85), 281‐285, 2019年02月12日
日本語

畳み込みニューラルネットワークを用いた地下鉄トンネルにおける変状検出の高精度化に関する一検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
王 安, 原川 良介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 121, 124, 2019年02月
映像情報メディア学会, 日本語

電子顕微鏡画像および配合量を用いたゴム材料の特性曲線の推定に関する検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
斉藤 直輝, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 261, 264, 2019年02月
映像情報メディア学会, 日本語

電子顕微鏡により撮像されたゴム材料からの異常検知に基づく劣化領域の推定に関する一検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 265, 268, 2019年02月
映像情報メディア学会, 日本語

高性能マルチセンシングバンドを利用したストレス検出の精度検証 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
伊藤 良起, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 269, 274, 2019年02月
映像情報メディア学会, 日本語

畳み込みニューラルネットワークにおける解釈性向上のための画像の属性分類に関する一検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
堀井 風葉, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 275, 279, 2019年02月
映像情報メディア学会, 日本語

地下鉄トンネルの変状画像を用いた技術者の注視領域推定のための初期検討 : 深層学習に基づく顕著領域の推定手法の適用 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
斉藤 僚汰, 前田 圭介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 281, 285, 2019年02月
映像情報メディア学会, 日本語

地下鉄トンネル維持管理支援のための距離計量学習を用いた類似点検データの検索に関する検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
弦間 奨, 原川 良介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 287, 290, 2019年02月
映像情報メディア学会, 日本語

トンネル切羽画像を用いた岩盤の穿孔エネルギー推定に関する初期検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
山本 健太郎, 原川 良介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 291, 294, 2019年02月
映像情報メディア学会, 日本語

レーザーデータを用いた地下鉄トンネル内の変状検出に関する検討 : 全層畳み込みネットワークを用いた変状領域の可視化 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
豊田 陽, 原川 良介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 295, 299, 2019年02月
映像情報メディア学会, 日本語

異種特徴を活用したネットワーク解析に基づく音楽配信サービスにおけるアーティストの人気度予測に関する検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
松本 有衣, 原川 良介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 301, 306, 2019年02月
映像情報メディア学会, 日本語

複数の推定結果の統合に基づくTwitterのフォロイー推薦の高精度化に関する一検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
滝村 祥司, 原川 良介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 307, 310, 2019年02月
映像情報メディア学会, 日本語

A Note on Automatic Malignant Tumor Candidate Detection Based on a 3D Deep Residual Network with FDG-PET/CT Images (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
李 宗曜, 藤後 廉, 小川 貴弘, 平田 健司, 真鍋 治, 志賀 哲, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 311, 314, 2019年02月
映像情報メディア学会, 英語

転移学習を用いた胃X線画像における胃炎識別に関する検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
金井 美岬, 藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 315, 318, 2019年02月
映像情報メディア学会, 日本語

Progressive Growing GANに基づく胃炎識別のための画像生成手法に関する一検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
渡邊 はるな, 藤後 廉, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 319, 322, 2019年02月
映像情報メディア学会, 日本語

視線データを用いた画像注視時に誘起される感情の推定に関する検討 : KDLPCCAに基づく特徴変換の導入による高精度化 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
松井 太我, 斉藤 直輝, 小川 貴弘, 浅水 仁, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 323, 327, 2019年02月
映像情報メディア学会, 日本語

OpenPoseに基づくユーザの動作を用いた映像の関心度推定に関する検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
九島 哲哉, 高橋 翔, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 351, 354, 2019年02月
映像情報メディア学会, 日本語

サッカー映像視聴時の視線データを用いた攻撃選手の重要度算出に関する検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
鈴木 元樹, 高橋 翔, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 355, 359, 2019年02月
映像情報メディア学会, 日本語

異種特徴を用いた深層学習に基づく送電鉄塔の劣化レベル分類の高精度化に関する検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
前田 圭介, 高橋 翔, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 361, 364, 2019年02月
映像情報メディア学会, 日本語

CCTV映像を用いた河川利用者の行動分類に関する検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
川本 舜, 原川 良介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 365, 369, 2019年02月
映像情報メディア学会, 日本語

CCTV映像を用いた河川利用者の異常検出に関する検討 (マルチメディアストレージ ヒューマンインフォメーション メディア工学 映像表現&コンピュータグラフィックス)
川本 舜, 原川 良介, 小川 貴弘, 長谷山 美紀, 映像情報メディア学会技術報告 = ITE technical report, 43, 5, 371, 374, 2019年02月
映像情報メディア学会, 日本語

北海道大学におけるデータサイエンス人材育成の取組-創造人材育成基盤構築:大学教育の新機能実装の試み-
長谷山美紀, 長谷山美紀, 大本亨, 大本亨, 湧田雄基, 電子情報通信学会技術研究報告, 118, 449(ITS2018 59-85), 2019年

北海道大学における博士等人材育成の取組-次世代スマートインフラ管理人材育成コンソーシアム-
長谷山美紀, 長谷山美紀, 大本亨, 大本亨, 湧田雄基, 電子情報通信学会技術研究報告, 118, 449(ITS2018 59-85), 2019年

人工知能が医療を変える!医療分野におけるAI研究開発最前線2019 II 領域別・画像診断におけるAI研究開発の最前線 7.その他 3)核医学におけるディープラーニングを用いた画像診断,画像処理
平田健司, 藤後廉, 小川貴弘, 長谷山美紀, 志賀哲, Innervision, 34, 7, 2019年

ユーザの好みを反映した潜在特徴を用いたGCNに基づくTumblrポストの推薦
大友一馬, 原川良介, 小川貴弘, 長谷山美紀, 岩橋政宏, 信号処理シンポジウム講演論文集(CD-ROM), 34th, 2019年

変状分類におけるGrad-CAM++に基づいたCNNの注目領域の可視化に関する検討
小川直輝, 前田圭介, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2019, 2019年

サッカー映像視聴時の視線データを用いた周辺視に基づく攻撃選手の重要度算出に関する検討
鈴木元樹, 高橋翔, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2019, 2019年

画像注視時の視線データを用いた関心の識別に関する検討-LDCCAに基づく特徴量の射影による高精度化-
松本真直, 斉藤直輝, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2019, 2019年

Sparse Bayesian Learningに基づく注視領域の時間変化を考慮したヒトの感情推定に関する検討
諸戸祐哉, 前田圭介, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2019, 2019年

複数の特徴量から得られる類似度の統合に基づくサッカー映像における類似場面検索の高精度化に関する検討
春山知生, 高橋翔, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2019, 2019年

複数変量の生成モデルに基づく画像注視時の脳活動データを用いた画像カテゴリの推定に関する検討
赤松祐亮, 原川良介, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2019, 2019年

Text-to-imageGANに基づくスタイル変換に関する検討-Image-to-textモデル導入による高精度化-
古寺恵, 藤後廉, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2019, 2019年

Semantic Segmentationに基づく地下鉄トンネルにおける変状検出に関する検討
WANG An, 藤後廉, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2019, 2019年

脳活動データを用いた注視画像の再構成における用いる視覚野の領域に関する検討
高田紗弥, 藤後廉, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2019, 2019年

Twitterを用いた異常検知に基づく野球映像の重要シーン検出に関する検討
平澤魁人, 前田圭介, 小川貴弘, 長谷山美紀, 電気・情報関係学会北海道支部連合大会講演論文集(CD-ROM), 2019, 2019年

AI・IoT・ビッグデータ解析に基づいた次世代の社会インフラ維持管理技術
長谷山美紀, 電子情報通信学会大会講演論文集(CD-ROM), 2019, 2019年

生体情報解析を導入した次世代AI技術の構築
長谷山美紀, 小川貴弘, 電子情報通信学会大会講演論文集(CD-ROM), 2019, 2019年

Novel Audio Feature Projection Using KDLPCCA-Based Correlation with EEG Features for Favorite Music Classification.
Ryosuke Sawata, Takahiro Ogawa, Miki Haseyama, IEEE Trans. Affective Computing, 10, 3, 430, 444, 2019年
IEEE A novel audio feature projection using Kernel Discriminative Locality Preserving Canonical Correlation Analysis (KDLPCCA)-based correlation with electroencephalogram (EEG) features for favorite music classification is presented in this paper. The projected audio features reflect individual music preference adaptively since they are calculated by considering correlations with the user & #x0027;s EEG signals during listening to musical pieces that the user likes/dislikes via a novel CCA proposed in this paper. The novel CCA, called KDLPCCA, can consider not only a non-linear correlation but also local properties and discriminative information of each class sample, namely, music likes/dislikes. Specifically, local properties reflect intrinsic data structures of the original audio features, and discriminative information enhances the power of the final classification. Hence, the projected audio features have an optimal correlation with individual music preference reflected in the user & #x0027;s EEG signals, adaptively. If the KDLPCCA-based projection that can transform original audio features into novel audio features is calculated once, our method can extract projected audio features from a new musical piece without newly observing individual EEG signals. Our method therefore has a high level of practicability. Consequently, effective classification of user & #x0027;s favorite musical pieces via a Support Vector Machine (SVM) classifier using the new projected audio features becomes feasible. Experimental results show that our method for favorite music classification using projected audio features via the novel CCA outperforms methods using original audio features, EEG features and even audio features projected by other state-of-the-art CCAs., IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 英語