avatar

Yixiao Zhang 张逸霄

Research Scientist
Seed Music Team, ByteDance
yixiao.97 (at) bytedance.com


About Me

I am currently a Research Scientist at Seed Music Team, ByteDance. My research centers on controllable music generation, music editing, music representation learning, and various MIR tasks.

I received the PhD degree in Artificial Intelligence and Music at Centre for Digital Music (C4DM), Queen Mary University of London. I am honoured to be advised by Prof. Simon Dixon and Dr. Mark Levy. Before that, I was a Research Assistant at the Music X Lab, NYU Shanghai. from 2019 to 2020, supervised by Dr. Gus Xia. I received the Bachelor of Engineering degree in Computer Science and Technology from University of Electronic Science and Technology of China (UESTC).

I write blogs and answers at 知乎. 给MIR新手的学习建议.

I participate in the development of ISMIR website. MIR labs list, Datasets.

News

  • [Nov. 2024] I received my award letter. I am a Doctor now!
  • [Nov. 2024] I will co-organise MIREX 2024 challenge, a satellite event at ISMIR 2024.
  • [Nov. 2024] I have passed my PhD viva!
  • [Oct. 2024] I have joined Seed Music Team @ ByteDance, as a Research Scientist.
  • [Jul. 2024] I have 1 paper accepted to NLPMusA workshop at ISMIR 2024!
  • [Jun. 2024] I have 1 paper accepted to ISMIR 2024.
  • [Apr. 2024] I have 2 papers accepted to IJCAI 2024.
  • [Feb. 2024] I will join Harmonai group at Stability.ai as an intern during May - Oct 2024.
  • [Jan. 2024] My new paper, "MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Models", is now at arxiv.
  • [Oct. 2023] I will join Music Foundation Team at Sony Research as an intern during Oct 2023 - Apr 2024.
  • [Jun. 2023] I will join Yamaha Corporation as an intern during Jun - Sep 2023.

    Education

  1. Queen Mary University of London
    Degree: Ph.D. in Artificial Intelligence and Music (jointly sponsored by QMUL & Apple)
    Duration: 2020 - 2024
    Supervisors: Prof. Simon Dixon, Dr. Mark Levy

  2. University of Electronic Science and Technology of China
    Degree: B.Eng. in Computer Science
    Duration: 2015 - 2019

Selected Publications

    Please see Google Scholar for full publication list.

    First-author papers

  1. NLP4MusA 2024
    The Interpretation Gap in Text-to-Music Generation Models
    Yongyi Zang*, Yixiao Zhang*
    The 3rd Workshop on NLP for Music and Audio (NLP4MusA) at ISMIR 2024
    PDF BibTex

  2. arxiv 2024
    Instruct-MusicGen: Unlocking Text-to-Music Editing for Music Language Models via Instruction Tuning
    Yixiao Zhang, Yukara Ikemiya, Naoki Murata, Woosung Choi, Marco Martínez, Liwei Lin, Gus Xia, Wei-Hsiang Liao, Yuki Mitsufuji, Simon Dixon
    Preprint.
    PDF Code Project Page BibTex Media report by Synced and MarkTechPost.

  3. IJCAI 2024
    MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Models
    Yixiao Zhang, Yukara Ikemiya, Gus Xia, Naoki Murata, Marco Martínez, Wei-Hsiang Liao, Yuki Mitsufuji, Simon Dixon
    The 33rd International Joint Conference on Artificial Intelligence (IJCAI 2024) - AI, Arts & Creativity Track
    PDF Code Project Page BibTex Media report by MarkTechPost.

  4. arxiv 2023
    Loop Copilot: Conducting AI Ensembles for Music Generation and Iterative Editing
    Yixiao Zhang, Akira Maezawa, Gus Xia, Kazuhiko Yamamoto, Simon Dixon
    Preprint.
    PDF Code Project Page BibTex

  5. ISMIR 2022
    Interpreting Song Lyrics with an Audio-Informed Pre-trained Language Model
    Yixiao Zhang, Junyan Jiang, Gus Xia and Simon Dixon
    The 23rd International Society for Music Information Retrieval Conference (ISMIR 2022)
    PDF Code Best Paper Award nomination

    6. Other papers

  6. IJCAI 2024
    Arrange, Inpaint, and Refine: Steerable Long-term Music Audio Generation and Editing via Content-based Controls
    Liwei Lin, Gus Xia, Yixiao Zhang, Junyan Jiang
    The 33rd International Joint Conference on Artificial Intelligence (IJCAI 2024) - AI, Arts & Creativity Track
    PDF Code Project Page BibTex

  7. ISMIR 2024
    Content-based Controls For Music Large Language Modeling
    Liwei Lin, Gus Xia, Junyan Jiang, Yixiao Zhang
    The 25th International Society for Music Information Retrieval Conference (ISMIR 2024)
    PDF Code Project Page BibTex

  8. NeurIPS 2023 Workshop
    The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluation
    Ilaria Manco, Benno Weck, Seungheon Doh, Yixiao Zhang, Dmitry Bogdanov, Yusong Wu, Ke Chen, Philip Tovstogan, Emmanouil Benetos, Elio Quinton, George Fazekas, Juhan Nam, Minz Won
    NeurIPS 2023 Workshop on Machine Learning for Audio
    PDF Code Project Page

    Work

  1. ByteDance
    Team: Seed Music Team
    Role: Research Scientist
    Responsibility: Working on building advanced large-scale music generation models.
    Oct 2024 - Now

    Internships

  1. Stability.ai
    Topics: Research on music large language models
    Advisors: Dr. Jordi Pons, Dr. Julian Parker
    May 2024 - Oct 2024

  2. Sony Research
    Topics: Text-based music editing with diffusion models
    Advisors: Yukara Ikemiya
    Oct 2023 - Apr 2024

  3. Yamaha Corporation
    Topics: Conversational interface for interative music creation
    Advisors: Dr. Akira Maezawa
    Jun 2023 - Sep 2023

  4. Microsoft Research Asia
    Topics: Theoretical research of reinforcement learning
    Advisors: Dr. Weiqing Liu and Dr. Xiao Yang
    Oct 2018 - Jun 2019

Services

Conference Reviewers

Journal Reviewers

Committee Members, Chairs

Teaching

Teaching Assistant

Invited Talks


Powered by Jekyll and Minimal Light theme.