KSEM 2021

Abstract: We have recently witnessed unprecedented proliferation of deep learning architectures with impressive performances superseding the state-of-the-arts. The task of optimizing the network parameters is usually handled by BP learning, often with great success. In contrast, the task of optimizing such network structure is usually left to trial-and-error. Moreover, it could be costly if we must start from scratch a brand new round of architectural search for new applications. Needless to say, it is highly desirable that if we could automate such an architecture learning task just like we learn the network parameters. This leads us to "Neural Architecture Search" (NAS), the process of automating architecture engineering.

This talk starts by noting the vital roles of dimensionality in NAS. We shall review the curse/blessing of dimensionality (both depth and width) in deep learning networks. We shall then compare the (very different) design principles behind PNAS (Progressive NAS) and RNAS (Regressive NAS) and explain why do we favor RNAS over PNAS. This is especially so when we harness the vital roles RNAS to mitigate the aforementioned curse of dimensionality.

In this talk, we shall address two technical areas related to the RNAS research:

1. Deep Compression:

It is natural to augment the BP-Learning with a structural learning paradigm, leading to a X-Learning strategy to jointly learn the structure and parameters of the learning models. Based on LASSO-type (i.e. L0-norm) regression we derive a notion of Deleterious Neurons (DNs). This will ultimately lead to the proposed X-learning paradigm where deleterious links (DLs) will be gradually trimmed so as to reach an improved network structure. As to be demonstrated during the talk, X-learn is applicable to both types of application scenarios: Classification-type, e.g. CIFAR or ImageNet. and Regression-type, e.g. super-resolution (SR) hetero-encoder. This talk will also highlight our on-going development on XNAS: an reinforcement-learning autonomous NAS.

2. Data Compression:

For data compression sectors, we have developed a dimension-reduction method called Regression Component Analysis (RCA), with closed-form error analysis. RCA is intimately related to PCA and RCA. For example, while PCA has already proven adequate for auto-encoder, we will need to resort to RCA in order to handle the (more general) hetero-encoder applications, e.g. SR imaging. As another example, we note that DCA and RCA are respectively the only closed-form mathematical tools for subspace analysis, the former for classification and the latter for regression. Note further that DCA has already enjoyed great success in visualization of data from different class labels, an emergent important research field in big data analysis. By the same token, RCA's promising roles for big data analyses can be naturally anticipated as well.

Bio: S.Y. Kung, Life Fellow of IEEE, is a Professor at Department of Electrical Engineering in Princeton University. His research areas include machine learning, data mining, systematic design of (deep-learning) neural networks, statistical estimation, VLSI array processors, signal and multimedia information processing, and most recently compressive privacy. He was a founding member of several Technical Committees (TC) of the IEEE Signal Processing Society. He was elected to Fellow in 1988 and served as a Member of the Board of Governors of the IEEE Signal Processing Society (1989-1991). He was a recipient of IEEE Signal Processing Society's Technical Achievement Award for the contributions on "parallel processing and neural network algorithms for signal processing" (1992); a Distinguished Lecturer of IEEE Signal Processing Society (1994); a recipient of IEEE Signal Processing Society's Best Paper Award for his publication on principal component neural networks (1996); and a recipient of the IEEE Third Millennium Medal (2000). Since 1990, he has been the Editor-In-Chief of the Journal of VLSI Signal Processing Systems. He served as the first Associate Editor in VLSI Area (1984) and the first Associate Editor in Neural Network (1991) for the IEEE Transactions on Signal Processing. He has authored and co-authored more than 500 technical publications and numerous textbooks including "VLSI Array Processors", Prentice-Hall (1988); "Digital Neural Networks", Prentice-Hall (1993) ; "Principal Component Neural Networks", John-Wiley (1996); "Biometric Authentication: A Machine Learning Approach", Prentice-Hall (2004); and "Kernel Methods and Machine Learning”, Cambridge University Press (2014).

Abstract: AI is advancing by leaps and bounds in learning algorithm development, but AI has many challenges when put to practice. One of the major challenges faced by AI is the serious lack of data, which has led to the inability of many good algorithm models to be effectively applied. In this talk, I will present federated learning as a solution designed to connect data silos while protecting user privacy and provide security. I will illustrate some theoretical advances and practical applications.

Bio: Qiang Yang is a chair professor at Hong Kong University of Science and Technology, Computer Science and Engineering Department. He heads the AI group at WeBank, and is a member of the Canadian Academy of Engineering. His research interests are federated learning, transfer learning and AI planning. He has published extensively including the books “Transfer Learning” (Cambridge University Press), “Federated Learning” (Morgan Claypool Publishers) and “Intelligent Planning” (Springer). He is a fellow of AAAI, IEEE, ACM, AAAS, etc.

Abstract: The rapid advancements in Information and Communication Technologies (ICTs) have been greatly changing our society, with clear societal and economic benefits. Mobile technology, Cloud, Big Data, Internet of things, services and technologies that are becoming more and more pervasive and conveniently accessible, towards to the realization of a smart society. At the heart of this evolution is the ability to collect, analyze, process and share an ever-increasing amount of data, to extract knowledge for offering personalized and advanced services. A major concern, and potential obstacle, towards the full realization of such evolution is represented by security and privacy issues. As a matter of fact, the (actual or perceived) loss of control over data and potential compromise of their confidentiality can have a strong detrimental impact on the realization of an open framework for enabling collection, processing, and sharing of data, typically stored or processed by external cloud services. In this talk, I will illustrate some security and privacy issues arising in emerging scenarios, focusing in particular on the problem of managing data while guaranteeing confidentiality and integrity of data stored or processed by external providers.

Bio: Pierangela Samarati is a Professor at the Department of Computer Science of the Università degli Studi di Milano, Italy. Her main research interests are on data and applications security and privacy, especially in emerging scenarios. She has been Computer Scientist at SRI, CA (USA) and visiting researcher at Stanford University, CA (USA), and at George Mason University, VA (USA). She is the chair of the IEEE Systems Council Technical Committee on Security and Privacy in Complex Information Systems (TCSPCIS), of the ERCIM Security and Trust Management Working Group (STM), and of the ACM Workshop on Privacy in the Electronic Society (WPES). She is ACM Distinguished Scientist (named 2009) and IEEE Fellow (named 2012). She has received the ESORICS Outstanding Research Award (2018), the IEEE Computer Society Technical Achievement Award (2016), the IFIP WG 11.3 Outstanding Research Contributions Award (2012), and the IFIP TC11 Kristian Beckman Award (2008). http://www.di.unimi.it/samarati/