Principal Investigators

Amir Zadeh

Amir Zadeh attended University of Tehran to study Computer Engineering. Soon after, he joined the Advanced Robotics Lab under supervision of Professor Hadi Moradi where he was an active researcher in the fields of Machine Learning and Computer Vision. He was admitted as an Artificial Intelligence Ph.D student to Carnegie Mellon University Computer Science program in 2015 under Professor Louis-Philippe Morency. His research spans different areas of Machine Learning and Signal Processing such as Natural Language Processing, Computational Linguistics, Computer Vision, Voice and Acoustic Signal Processing and in general Multimodal Signal Processing. The goal of his research is to extend the capabalities of today's Machine Learning to be able to better model multimodal interactions and for that he has chosen the Deep Neural Networks family as his research focus.

Soujanya Poria

Soujanya Poria received his BEng in Computer Science from Jadavpur University, India, in 2013. In the same year, he received the best undergraduate thesis and researcher award from Jadavpur University and was awarded Gold Plated Silver medal from the University and Tata Consultancy Service for his final year project during his undergraduate course. Soon after, Soujanya joined Nanyang Technological University as a research engineer in the School of Electrical and Electronics Engineering and, later in 2015, he joined NTU Temasek Laboratories, where he is now conducting research on aspect-based sentiment analysis in multiple domains and different modalities. In parallel with his research activities in Singapore, Soujanya is also in the process of finalizing his PhD studies in Computing Science and Mathematics at the University of Stirling, UK. Soujanya has over 1000 citations on Google Scholar and h-index of 20.

Erik Cambria

Erik Cambria received his PhD in Computing Science and Mathematics in 2012 following the completion of an EPSRC project in collaboration with MIT Media Lab, which was selected as impact case study by the University of Stirling for the UK Research Excellence Framework (REF2014). After working at HP Labs India, Microsoft Research Asia, and NUS Temasek Labs, in 2014 Dr Cambria joined NTU School of Computer Science and Engineering as an Assistant Professor. His current affiliations also include Rolls-Royce@NTU, Delta@NTU, A*STAR IHPC, MIT Synthetic Intelligence Lab, and the Brain Sciences Foundation. He is Associate Editor of Elsevier INFFUS, KBS and IPM, Springer AIRE and Cognitive Computation, IEEE CIM, and Editor of the IEEE IS Department on Affective Computing and Sentiment Analysis, fields in which he is one of the most productive authors. Dr Cambria is also recipient of several awards, e.g., the Temasek Research Fellowship, and is involved in many international conferences as Workshop Organizer, e.g., ICDM and KDD, PC Member, e.g., AAAI and ACL, Program Chair, e.g., ELM and FLAIRS, and Keynote Speaker, e.g., CICLing.

Louis-Philippe Morency

Louis-Philippe Morency is a tenure-track Faculty at CMU Language Technology Institute where he leads the Multimodal Communication and Machine Learning Laboratory (MultiComp Lab). He was previously Research Faculty at USC Computer Science Department. He has received my Ph.D. in Computer Science from MIT Computer Science and Artificial Intelligence Laboratory.His research focuses on building the computational foundations to enable computers with the abilities to analyze, recognize and predict subtle human communicative behaviors during social interactions. He has formalized this new research endeavor with the Human Communication Dynamics framework, addressing four key computational challenges: behavioral dynamic, multimodal dynamic, interpersonal dynamic and societal dynamic. This multi-disciplinary research topic overlaps the fields of multimodal interaction, social psychology, computer vision, machine learning and artificial intelligence, and has many applications in areas as diverse as medicine, robotics and education.

Collaborators

Amir Hussain

Amir Hussain is full Professor of Computing Science, and founding Director of the Cognitive Big Data Informatics (CogBID) Research Laboratory (http://cogbid.cs.stir.ac.uk) at the University of Stirling in Scotland. He obtained his BEng in Electronic and Electrical Engineering (with the highest 1st Class Honours, with distinction) and PhD (in novel neural network architectures and algorithms for real-world applications), both from the University of Strathclyde in Glasgow, UK, in 1992 and 1997 respectively. Following a Research Fellowship at the University of Paisley, UK (1996-98), and a Research Lectureship at the University of Dundee, in Scotland, UK (1998-2000), he joined the University of Stirling in Scotland, in 2000, where he is currently full Professor and founding Director of the Cognitive Big Data Informatics (CogBID) Laboratory.

Alexander Gelbukh

Alexander Gelbukh is a research Professor and Head of the Natural Language Laboratory at the Centro de Investigación en Computación (CIC) of the Instituto Politécnico Nacional (IPN), Mexico. His main area of interests is computational linguistics. He is a member of the Mexican Academy of Sciences (AMC), founding member of the Mexican Academy of Computing (AMEXCOMP), and a National Researcher of Mexico at excellence Level 2 (SNI 2). He is the founder and chair of the CICLing International Conference series. He has been Honorary Chair of ENC-2008, Program co-chair of some recent MICAI, CIC, CORE, and some other conferences. He is the founding Editor-in-Chief of the International Journal of Computational Linguistics and Applications (IJCLA) and Editor-in-Chief of the journal Polibits.

Björn W. Schuller

Björn W. Schuller is a Professor & Head of the Chair of Complex and Intelligent Systems, University of Passau, Germany, Reader (Associate Professor) in Machine Learning, Department of Computing, Imperial College London, London/U.K, Chief Executive Officer (CEO) and Co-Founder, audEERING GmbH, Gilching/Germany, Visiting Professor, School of Computer Science and Technology, Harbin Institute of Technology, Harbin/P.R. China.

Stefan Scherer

Stefan Scherer is a Research Assistant Professor in the Department of Computer Science at the University of Southern California (USC) and the USC Institute for Creative Technologies (ICT) where he leads research projects funded by National Science Foundation and the Army Research Laboratory. Stefan Scherer directs the lab for Behavior Analytics and Machine Learning and is the Associate Director of Neural Information Processing at the ICT. His research aims to automatically identify characterize, model, and synthesize individuals’ multimodal nonverbal behavior within both human-machine as well as machine-mediated human-human interaction.

Researchers

Minghai Chen

My name is Minghai Chen. I got my Bachelor’s in Computer Software from Tsinghua University, China. Now I’m a Master student in Language Technologies Institute, CMU. I’m interested in multimodal machine learning. Previously I was working on image captioning and my method obtained top 1 position on 5 of the 14 metrics in the MSCOCO challenge. Recently I am working on multimodal sentiment analysis, in which cues from text, video and audio are combined to predict the speaker’s sentiment. This summer I’m doing my internship in Google on fraud transaction detection.

Edmund Tong

Edmund Tong is an undergraduate who is pursuing a Computer Science major and Language Technologies minor at Carnegie Mellon University. He has been an undergraduate researcher with the Multimodal Communication and Machine Learning Lab since October 2015. Notably, the work that Edmund has done with Amir Zadeh and Prof. Louis-Philippe Morency on applying deep multimodal networks to combat human trafficking will be published for ACL 2017. Edmund’s research interests include multimodal representations and theoretical bases for deep learning, and he can be contacted at edtong AT cmu DOT edu.

Haiyun Peng

Haiyun Peng received his Bachelor of Engineering in automation from Wuhan University in 2013. After that, he obtained his Master of Science in Signal processing from Nanyang Technological University. He is currently a PhD student under the supervision of Erik Cambria in the School of Computer Science and Engineering in Nanyang Technological University. His main research interests are concept-level natural language processing and multi-modal sentiment analysis, both in English and Chinese language.

Devamanyu Hazarika

Devamanyu Hazarika did his major in Computer Science from NIT Warangal, India. He worked at NTU Singapore during the summer of 2016. He received the Institute Gold Medal for best academic performance in his undergraduate course. He is currently pursuing his Ph.D at National University Singapore. His research interests primarily lie in machine learning and deep-learning for natural language processing and its applications.

Prateek Vij

Prateek Vij is currently pursuing his undergraduate studies in Computer Science and Engineering (2014-18) from Indian Institute of Technology, Guwahati. He worked at Temasek Labs, NTU during his research internship in the summers of 2016. His research interests lie in the field of multimodal sentiment analysis and deep learning.

Navonil Majumder

Navonil Majumder is a postgraduate student at the Centro de Investigación en Computación (CIC) of the Instituto Politécnico Nacional, Mexico.

Yukun Ma

Yukun Ma received his Bachelor of Engineering in 2010 from Beijing University of Posts and Telecommunications, China, and Master of Science in 2011 from the University of Edinburgh, UK. He is now a PhD student of Rolls-Royce@NTU Corporate Lab under the supervision of Erik Cambria. His research is mainly about concept recognition from conversational speech, fine-grained named entity recognition and distributional semantics.