Article count:16439 Read by:87952319

The Central Conservatory of Music recruits its first PhD student in music artificial intelligence! They are serious about studying AI + music

Latest update time：2019-03-04

Reads：

▲Click above Leifeng.com Follow

Text | Huang Shanqing

Report from Leiphone.com (leiphone-sz)

Leifeng.com AI Technology Review: As one of the top music learning institutions in China, the Central Conservatory of Music today released a recruitment notice for a doctoral program in music artificial intelligence. The full name of this major is "Music Artificial Intelligence and Music Information Technology", which is the first time that the Central Conservatory of Music has opened this major. The tutor lineup includes artificial intelligence professors from Tsinghua University and Peking University, who work together with the dean of the Central Conservatory of Music to form a dual tutor training system (music tutor + technology tutor), focusing on cultivating "compound top-notch innovative talents with cross-integration of music and science and engineering."

According to the official website, the major of "Music Artificial Intelligence and Music Information Technology" has a total study period of 3 years, and applicants must be candidates with a background in computer, intelligence and electronic information.

In terms of recommended readings, except for "Foundations of Music Theory" which is related to music theory, the other four recommended books are all related to artificial intelligence theory, namely "Data Structures and Algorithms", "Introduction to Signals and Systems", "Artificial Intelligence: A Modern Approach" and "Neural Networks and Machine Learning".

Since "Music Artificial Intelligence and Music Information Technology" is an interdisciplinary major, the interview will not only assess the professional ability of the subject, but also the musical ability of the candidates - playing a musical instrument or simply singing.

Currently, the three joint training instructors for this major are:

Yu Feng

President of the Central Conservatory of Music, professor, doctoral supervisor, leading talent of the "Ten Thousand Talents Plan", and talent of the "Four Batches". President of the Chinese Conducting Society, deputy director of the National Art Professional Degree Graduate Teaching Committee, member of the 10th National Committee of the China Federation of Literary and Art Circles, and enjoys special government allowances from the State Council.

Sun Maosong

Professor and doctoral supervisor at Tsinghua University, executive vice president of Tsinghua University Artificial Intelligence Research Institute, former director and party secretary of the Department of Computer Science, vice chairman of the Teaching Informatization and Teaching Method Innovation Steering Committee of the Ministry of Education, and member of the 9th National Committee of the China Association for Science and Technology. His main research areas are natural language processing, artificial intelligence, machine learning and computational education. He is the chief scientist of the National 973 Program and the chief expert of the National Social Science Foundation Major Project. In 2017, he led the development of the "Jiuge" artificial intelligence ancient poetry writing system.

Wu Xihong

Professor and doctoral supervisor at Peking University, New Century Excellent Talent of the Ministry of Education. Vice Dean of the School of Information Science and Technology, Director of the Department of Intelligence Science, and Director of the Speech and Hearing Research Center of Peking University. He is committed to the research of machine auditory computing theory, speech information processing, natural language understanding, and music intelligence. He has presided over more than 40 national and provincial and ministerial projects, obtained more than 10 national invention patents, and published more than 200 academic papers. He has made great achievements in the field of intelligent music creation and arrangement.

Candidates interested in enrolling in this major must complete the online registration between March 1 and 15, 2019 (website: http://yz.chsi.com.cn/). The examination will be held at the Central Conservatory of Music in May this year.

For more details, please click: http://www.ccom.edu.cn/xwyhd/xsjd/2019s/201903/t20190301_53856.html.

Were there early signs of the opening of the major?

If you have been following the developments of the Central Conservatory of Music, you will not be surprised by the opening of this major.

As early as May last year, the Central Conservatory of Music signed a cooperation agreement with the School of Information Computing and Engineering at Indiana University, which is well-known for its innovative interdisciplinary research, to build the "Informatics Philharmonic Orchestra" laboratory. The so-called "Informatics Philharmonic Orchestra" refers to a musical artificial intelligence accompaniment system invented by Christopher Raphael, professor and director of the Music Informatics Laboratory at the School of Information Computing and Engineering at Indiana University.

The biggest feature of this system is that it uses mathematical methods to fully interpret and calculate the music itself and the musicians' feelings. Through continuous and active learning, it forms orchestral accompaniment and concerto templates that are more in line with the musicians' personalized performance needs, providing musicians with richer and more flexible performance opportunities.

After the signing, after more than half a year of intensive preparation, the two parties jointly held China's first special concert accompanied by artificial intelligence - "AI Night Concert" on November 26 last year. 12 outstanding soloists from different majors of the Central Conservatory of Music and the "Information Philharmonic" performed 12 Chinese and foreign works of various genres and styles.

It is worth mentioning that this concert included an artificial intelligence concerto of the Chinese music "The Great Wall Capriccio". This is the first time that musical artificial intelligence technology has collided with Chinese folk music.

The picture comes from the official website of the Central Conservatory of Music

Professor Yu Feng, president of the Central Conservatory of Music, said in his speech at the concert: "This is a concert of far-reaching significance. my country's entire music industry will enter an era of "artificial intelligence", which will greatly improve the information level of the entire music industry, especially the music education industry. The combination of artificial intelligence technology and music art will achieve leapfrog development of the entire industry and will definitely become a model for the industrialization of the music industry."

"AI Night Concert" full performance video:

http://video.ccom.edu.cn/index.php?option=weixin,dianbodetail&id=3514

Domestic research enthusiasm is growing

In addition to the Central Conservatory of Music, other institutions that have attempted to achieve results in artificial intelligence + music include the Xinghai Conservatory of Music and Minzu University of China.

On May 16 last year, the "Joint Laboratory for Music Artificial Intelligence Assisted Orchestral Teaching" jointly established by the Orchestral Department of Xinghai Conservatory of Music and the Music Informatics Laboratory of the School of Information Computing and Engineering at Indiana University was officially launched. The two parties will cooperate on the introduction of the "Music Artificial Intelligence Assisted Orchestral Teaching" system into daily teaching.

It is understood that the system allows students to hear the complete music accompaniment of a professional orchestra at any time during their daily professional practice, and at the same time convert the synthesized performance audio of themselves and the orchestra into highly structured, visual, searchable, and comparable music data and bring them to the classroom to discuss with professional teachers; for professional teachers, the system can realize the vertical and horizontal comparison of students' professional learning situation, obtain first-hand information to understand students, and thus improve teaching content and methods.

The picture comes from the WeChat official account of "Xinghai Conservatory of Music"

On December 7 last year, the signing and unveiling ceremony of the "Artificial Intelligence Music Joint Laboratory" jointly established by Minzu University of China and Ping An Technology was held at the Zhixing Hall of Minzu University of China. This cooperation aims to leverage the respective advantages and realize the vision of artificial intelligence music creation from the appreciation stage to the professional stage and then to the expert stage through joint research and development.

Song Min, member of the Standing Committee of the Party Committee and vice president of Minzu University of China, said at the unveiling ceremony that artificial intelligence has been included in the national plan and has entered the stage of gradual implementation. It is constantly integrating with various fields and will undoubtedly lead the development of all walks of life in the future. She hopes that both parties can give full play to their respective advantages through the laboratory platform to improve the level of discipline construction and music creation of Minzu University of China, promote the construction of Beijing's "four centers", especially the cultural center, and actively help China's excellent music culture go global.

The picture comes from the official website of Minzu University of China

In addition, the 6th China Sound and Music Technology Conference CSMT (Conference on Sound and Music Technology), co-founded by Fudan University and Tsinghua University, has been continuously providing academic insights on the multidisciplinary field of sound and music technology to the country since 2013, enriching the research results in the field of artificial intelligence + music in China.

Taking the 2018 conference as an example, the topics for the papers included:

Music Acoustics

Musical instrument acoustics/voice acoustics/psychoacoustics and electroacoustics/spatial music acoustics, etc.

Signal Processing for Sound and Music

Sound signal processing/music signal processing in various fields such as industry, agriculture, animal husbandry, breeding, geography, environment, etc.

Computer hearing

Content analysis, understanding and modeling of sound and music/audio and music information retrieval/sound and music classification, annotation, emotional computing, recommendation, etc./application of artificial intelligence in sound and music computing/application of sound and music computing in entertainment, education, ocean, medicine, equipment, military, information security and other fields

Audio Information Security

Robust Audio Watermarking/Audio Authentication/Audio Forensics

Computer Music and Recording

Computer-aided music composition/Computer-aided music teaching system/Computer music production technology/Computer music software development/Sound and multi-channel sound system/Sound device and related multimedia technology/Sound effect and sound design/Audio human-computer interaction

Auditory Psychology

Multimedia applications combining hearing and vision

It is worth mentioning that last year's CSMT conference specially opened two Special Sessions: one was used to explore computer hearing for general audio, trying to expand the application of Audio + AI artificial intelligence beyond Music in various industries, such as marine ship identification, equipment diagnosis, AI medical care, voice acoustics, audio monitoring, animal identification, agricultural protection, industrial automation, etc.; the other was to explore the cross-integration of Chinese folk music with computer and other scientific and technological technologies, showing the forward-looking nature of this domestic conference.

The current popular AI + music algorithm

Regarding the current research on music artificial intelligence algorithms, Professor Fu Xiaodong of the Department of Musicology at the China Conservatory of Music divided it into "self-discipline" and "heteronomy" in his article "Ethical Thinking on Musical Artificial Intelligence - "Self-discipline" and "Heteronomy" of Algorithmic Composition" published in the May 2018 issue of "Art Exploration".

"Autonomy" refers to the fact that the machine strictly or non-strictly follows the internal structure principles specified in advance, generates musical works corresponding to the sound materials, and the final sound presentation is limited by the autonomy of the internal structure principles; "heteronomy" refers to the fact that the machine strictly or non-strictly follows the external structure principles specified based on human experience, and maps them into sound to generate works. The final sound presentation is limited by the heteronomy of the external structure principles.

The final combing results are as follows:

"Self-discipline" music artificial intelligence algorithm

1. Mathematical Model

The mathematical model composed by mathematical algorithms and random events is used for composition. The algorithm is equivalent to the composition rules, and the random events are equivalent to the musical elements. The various elements in music can be decomposed into a series of random events, such as the four attributes of sound and the three elements of music. The composer (programmer) assigns different weights to them and uses a specific random algorithm to calculate and process them to obtain a sound sequence. The result is non-deterministic. Commonly used random algorithms include Markov chain and Gaussian distribution. At present, the music artificial intelligence works based on mathematical models have a considerable sense of "intelligence" in terms of the speed following of the accompaniment, the intensity processing of the phrases, and the expansion and contraction rhythm of the ending, but there is still a clear lack in the overall audibility of the works.

2. Evolutionary Methods

Evolutionary algorithms are derived from the biological evolution theory revealed by Darwin. They use algorithms to simulate the process of species evolution to construct musical works. Random or artificial sound events are grouped into a population. Through repeated iterations of selection, inheritance, and mutation algorithms, the existing multiple individuals in the population are eliminated and the results are corrected by an audit procedure composed of fitness functions to ensure the quality of their aesthetic significance. The most common evolutionary calculation methods are genetic algorithms and genetic programming. The logic of evolutionary algorithms trying to match the process of species evolution with the process of music generation is not perfect, so the aesthetic recognition of the works is not high. Today, they are often used in harmony configuration and accompaniment tasks.

3. Grammars

The composition rules of music can be compared to the grammatical rules of human language. Human language is composed of words, phrases, etc., which are expressed in units according to certain grammatical rules. The motives, rhythms, and phrases in music also have similar structural characteristics. First, the grammatical rules of a specific musical work are created, and various musical materials such as harmony, rhythm, and pitch are combined to finally generate a musical work. It is true that music and language are isomorphic to a certain extent, but in comparison, the rules of music reflect greater flexibility and variability. The language algorithm that is composed of a fixed grammatical rule and several variable rules produces a musical work that is more or less rigid and dull.

"Heteronomy" music artificial intelligence algorithm

1. Translational Models

Map and transfer the information in non-music media signal sources into musical sound information. The most common is to convert visual information, such as converting lines in an image into melody, color into harmony, chroma into intensity; converting the spatial displacement of a moving object into melody, speed into beat rhythm, etc. It can also be used to transfer non-visual information, such as transferring positive/negative descriptions in literary works into major/minor triads through an automatic sentiment analysis system. In fact, human senses do have a "synesthesia" effect to a certain extent, such as the correspondence between spatial lines and melody trends, but if they are strictly mapped, there is no strong psychological evidence. Therefore, music works generated using the transfer model algorithm often appear in interactive new media art performances, with more aesthetic interest in the relevance and interactivity of on-site events. Once a musical work is separated from its mapped object and presented alone, the audibility of such works will be greatly reduced.

2. Knowledge-based Systems

Taking a certain type of music style as the knowledge base, extracting and encoding the aesthetic characteristics of the music style is called inductive reasoning; using the coding program as an algorithm to create new works of similar style is called deductive reasoning. For example, the Baroque music style coding based on the principle of counterpoint, the classical romantic music style coding based on the major and minor harmony system, the impressionist music style coding with weakened harmony function, and the generation of works of corresponding styles belong to the knowledge inference system algorithm. This algorithm is close to the learning process of composition technology theory in music academies to some extent. The generated music works are very similar to the specific style knowledge base on which they are based, and have high audibility. Its disadvantage is the relative separation of the two links of induction and deduction, that is, the style coding must be provided by the operator, and the program itself is only the execution operation of the coding. The result of the work will be seriously affected by the operator's abstract understanding of the creation rules, and there will be the disadvantages of rigidity and similarity.

3. Machine Learning

The operator inputs a large amount of music sounds into the computer, and the computer effectively "listens and learns" it, that is, it uses statistical methods to learn the laws of music composition. The process is similar to the knowledge inference system, but the operator does not strictly specify the type of music, nor does he provide style coding for the program. This process is automatically completed by the algorithm program, emphasizing its autonomy and "unsupervised" learning. Of course, in essence, the "unsupervised" nature of machine learning can only be within a certain degree and scope, and it is still confined to the knowledge material library provided by the operator. Machine learning is related to the research results of computational sciences such as mathematical optimization and data mining, and is more closely related to the research results of cognitive science and neural network disciplines. The most notable of these is the use of decision trees, artificial neural networks, deep learning and other methods. It is the algorithm with the highest degree of imitation of biological learning processes to date. Machine learning still belongs to bionics, but it goes beyond bionics at the structural and mechanical levels and is a bionics of the human brain's thinking process. Machine learning can be used for music creation in a general sense, as well as for occasions such as improvisation and competition. Although it is possible to generate music works of various specified styles or mixed styles, it still depends on the type of music data provided by the operator, and is a sound prediction after deriving rules through probabilistic statistics of random events.

According to Professor Fu’s classification criteria, we will be able to effectively classify most of the popular artificial intelligence + music research work today.

It is worth mentioning that the paper "XiaoIce Band: A Melody and Arrangement Generation Framework for Pop Music", which was jointly conducted by the University of Science and Technology of China, Microsoft Artificial Intelligence and Research Institute, and Soochow University, and describes the end-to-end melody and arrangement generation framework for song generation. It successfully won the Best Student Paper in the Research Track of KDD 2018.

Leiphone.com AI Technology Review has made corresponding interpretations on this. Interested readers can click https://www.leiphone.com/news/201808/NkobLRDHxZsyadg5.html to review it.

In general, artificial intelligence will play a more important role in the field of music in the future. It can help people analyze works, create, and share a lot of repetitive work, further stimulate creativity, and explore various feasibility in music form and content. It is hoped that this interdisciplinary and integrated cooperation can summarize and improve the logic of various music creations, and make breakthroughs in perception, emotion, etc., so that artificial intelligence can innovate in many fields of music and have an impact on teaching, social services, etc.

- END -

◆ ◆ ◆