Our research effort bridges several areas such as modern wireless networking (sensor/ad-hoc networks, cognitive radio networks), compressed sensing, signal processing, and machine learning. Our research has so far been supported by the following agencies/programs:
We created Odyssey , the most diverse public dataset to date that contains over
3,000 clean and Trojaned models. To generate this dataset, various types of triggers and mappings
(source to target class) have been used. Odyssey contains a total of 3,460 models, over 1,000
models each trained on MNIST, FashionMNIST, and CIFAR10 image datasets.
A recent threat to deep neural networks is the Trojan attack, where an attacker interferes with the training pipeline by inserting triggers into some of the training samples and trains the model to act maliciously only for samples that contain the trigger. We developed Odyssey, the most diverse dataset to date with over 3,000 clean and Trojan models. Our analysis results show that Trojan attacks affect the classifier margin and shape of decision boundary around the manifold of clean data. Exploiting these two factors, we proposed an efficient Trojan detector that operates without any knowledge of the attack and significantly outperforms existing methods.
Related Publications:
Augmenting neural networks with skip connections, as introduced in the so-called ResNet architecture, surprised the community by enabling the training of networks of more than 1,000 layers with significant performance gains. This paper deciphers ResNet by analyzing the effect of skip connections, and puts forward new theoretical results on the advantages of identity skip connections in neural networks. We prove that the skip connections in the residual blocks facilitate preserving the norm of the gradient, and lead to stable back-propagation, which is desirable from optimization perspective. We also show that, perhaps surprisingly, as more residual blocks are stacked, the norm-preservation of the network is enhanced. Our theoretical arguments are supported by extensive empirical evidence.
Can we push for extra norm-preservation? We answer this question by proposing an efficient method to regularize the singular values of the convolution operator and making the ResNet’s transition layers extra norm-preserving.
On Norm Preservation of ResNets Project Page
Related Publications: Deep Learning Theory
Thanks to recent advances in computing, deep learning based systems, which employ very large numbers of inputs, have been developed in the last decade. However, processing/labeling/communication of a large number of input data has remained challenging. Therefore, novel machine learning algorithms that make the best use of a significantly less amount of data are of great interest. We have introduced two novel data selection algorithms with linear complexity and great performance with many applications in different problems such as active leaning, database summarization, video summarization, and training Generative Adversarial Networks (GANs) with less data. For more information on our algorithms, Iterative Projection and Matching (IPM) (CVPR’19) and Spectrum Pursuit (SP) (CVPR’20) please see below:
Related Publications: Data Selection
All source codes for our data selection algorithms (IPM and SP) are available!
In this research, we manipulate the training of ResNets to generate data representations that are robust to outliers and anomalies. Conventional machine learning methods are being designed under closed-set assumptions, meaning that training data contains samples from all the possible classes that the classifier will encounter during testing. Such assumption does not hold in many applications; as it may not be possible to cover every potential input class in the training set. The goal of open-set classifiers is to detect out-of-distribution (OOD) samples; the input instances that do not belong to any of the training classes. We have shown that OOD samples can be detected if the training data is embedded into a low-dimensional space, such that the embedded training samples (or features) lie on a union of 1-dimensional subspaces. A similar idea can also be used for detection and correction of noisy labels, as is currently performed in our group.
Related Publications: Deep Learning
In this research we proposed the general subspace capsule networks (SCNs) that can be successfully applied in generative as well as discriminative tasks. Subspace capsule networks model the possible variations in the appearance of entities trough a group of learned capsule subspaces. Then the capsules are created by projecting the input feature vector onto these learned capsule subspaces using leaned transformations.
Result: We evaluated the effectiveness and generalizability of SCN through a comprehensive set of experiments in three applications namely, high resolution image generation, semi-supervised image classification using the GAN framework and supervised image classification. We demonstrate the scalability of SCN to large datasets and model architectures by applying it on supervised classification of ImageNet dataset.
Related Publications: Deep Learning
Supervised, unsupervised, and reinforcement learning paradigms can be studied as a viable solution for many communication and signal processing problems where the traditional approaches are not efficient due to a model deficit or to an algorithm deficit. Accordingly, our goal in this trust is to revisit some of signal processing and communication problems with data-driven approaches rather than model-based approaches. For example, we extended our research on adaptive non-uniform CS to learn the time-varying signal pattern using long short-term memory (LSTM) and reinforcement learning.
Related Publications: Deep Learning and Compressive Sensing
The goal is to find the spectrum opportunities in temporal, spectral, and spatial domains in the presence of unreliable sensing data and unavailable location information. We proposed a novel probabilistic model to cluster the sensors solely based on their observations, not requiring any other prior. After receiving the sensing data, the base station updates the probability distributions of cluster membership, channel availability, and device reliability. All the update rules are derived mathematically by the variational inference. Then, the distributions are employed to find spectrum opportunities via a multi-label graph cuts method. For mobile CRNs, we proposed a learning algorithm for estimating the primary users (PUs) dynamic based on the mobility of SUs and introduced an energy-efficient and fast coordinated spectrum sensing policy that maximizes the channel discovery ratio for SUs.
Related Publications: Cognitive Radio Networks
The propagated power from the PUs at different locations, time slots, and frequency channels can be modeled as a multi-dimensional propagation tensor. Tensor-based processing of data has several advantageous over the matrix-based processing, as it preserves the data structure and the uniqueness of CANDECOMP/PARAFAC (CP) decomposition for multi-dimensional tensors is guaranteed under milder conditions than in matrix decomposition. Our goal is to recover the original propagation tensor, given the partial observations from a linearly transformed replica of the propagation tensor, i.e., the sensed tensor. A joint problem of tensor CP decomposition and non-negative matrix factorization is introduced. Uniqueness conditions are studied under low-rank assumption on the underlying tensors. Our results show a great promise in the applicability of the proposed algorithm in dynamic spectrum sensing for missing spectrum data recovery. Further, we have employed our tensor framework for spectrum sensor selection as well as generating a dynamic radio environment map (radio spectrum cartography)
Related Publications: Tensor Analysis, Radio Cartography, Spectrum Sensing, and Cognitive Radio Networks
The emerging field of compressive sensing (CS) has overturned the traditional concept of sensing and sampling, established by Nyquist sampling theorem, for signals that have a sparse representation over some proper basis. In contrast with the conventional CS techniques that assumed uniform sparsity and equal important of signal coefficients in their design and theoretical analysis, in my group we avoided the one-size fit-all approach and introduced a generalized CS framework that enables problem-specific design of CS schemes through integration of signal/system properties. Our contributions include designing nonuniform CS (NCS), adaptive NCS using Bayesian inferences, adaptive NCS via reinforcement learning, CS for time-varying signals, CS for clustered and skew-sparse signal recovery, 1-bit CS, and asynchronous distributed sparse signal recovery.
Related Publications: Compressive Sensing
The emerging field of compressive sensing (CS) has overturned the traditional concept of sensing and sampling, established by Nyquist sampling theorem, for signals that have a sparse representation over some proper basis. In contrast with the conventional CS techniques that assumed uniform sparsity and equal important of signal coefficients in their design and theoretical analysis, in my group we avoided the one-size fit-all approach and introduced a generalized CS framework that enables problem-specific design of CS schemes through integration of signal/system properties. Our contributions include designing nonuniform CS (NCS), adaptive NCS using Bayesian inferences, adaptive NCS via reinforcement learning, CS for time-varying signals, CS for clustered and skew-sparse signal recovery, 1-bit CS, and asynchronous distributed sparse signal recovery.
Related Publications: Compressive Sensing
As sparsity is a feature present in many physical phenomena, it is not surprising that CS can be integrated into a flurry of research activities. In my group, we have investigated important problems related to efficient data acquisition, dissemination, and storage in wireless sensor networks by integrating CS with underlying networking protocols for cross-layer optimization to improve the overall performance. Further, utilizing CS, we have devised schemes for efficient spectrum sensing in CRNs, source localization, radio cartography, and post-CMOS A/D designs. Our introduced frameworks lead to smarter, faster and more efficient signal acquisition systems applicable in medical diagnosis devices, structural health monitoring systems, precise indoor localization systems, and spectrum-aware communication among many others.
Related Publications: Compressive Sensing Applications
Rateless codes are a new class of codes that have been invented recently. Rateless codes on lossy channels do not assume any knowledge about the channel, unlike the traditional codes. This feature makes them very interesting in the applications that the channel loss is unknown, time-varying, or nonuniform. In the original work on rateless codes, equal error protection (EEP) of all data was considered. EEP would be sufficient in the applications such as multicasting bulk data. However, in several applications, a portion of data may need more protection than the rest of data. For example, in an MPEG stream, I-frames need more protection that P-frames.
In some other applications, a portion of data may need to be recovered prior to the other parts. An example would be on-demand media streaming, in which the stream should be reconstructed in sequence. Such applications raise a need for having codes with unequal error protection (UEP) or unequal recovery time (URT) property. We developed , for the first time, rateless codes that can provide UEP and URT. We analyzed the proposed codes under both iterative decoding and maximum-likelihood decoding. Results are very promising and show the applicability of UEP-rateless codes in many important applications, such as transferring data frames or video/audio-on-demand streaming.
When multiple sources of data need to transmit their rateless coded symbols through a single relay to a common destination, a distributed rateless code can be employed to encode the source symbols instead of several separated conventional rateless codes to increase the transmission efficiency and flexibility. In our research, we have proposed DU-rateless codes, which are distributed rateless codes that can provide UEP for sources with different data lengths. We have designed degree distributions for DU-rateless codes using genetic-algorithms.
We have also designed several rateless codes similar to LT codes with optimum intermediate recovery rate.
Related Publications: UEP and UEP Rateless Coding
Efficient network-wide broadcasting is an important issue in wireless networks that attracted a lot of attention. In this research, we considered the case that a large amount of packets have to be broadcasted in a multihop wireless network with our main concerns being reliability and energy-efficiency. We proposed a two-phase broadcasting scheme referred as Collaborative Rateless Broadcast (CRBcast). Our two-phase protocol is based on Probabilistic Broadcasting (PBcast) and an application layer rateless coding. At the first phase, the rateless-encoded packets are broadcasted based on PBcast, in which each node probabilistically relays every new received packet. The second recovery phase, which is based on simple collaborations of nodes, ensures that all nodes can recover original data. We showed that CRBcast can provide both reliability and energy efficiency. Simulation results indicate that CRBcast saves at least 72% and 60% energy in comparison with flooding and PBcast, respectively. We implemented CRBcast protocol in a testbed including MICAz motes with TinyOS distributed software operating system. The result showed similar improvement in the energy efficiency while providing reliability. We are currently working on another scheme for reliable and energy-efficient one-to-all broadcasting in multihop wireless networks, where each link is modeled as a packet erasure channel. In this scheme, referred to as Fractional Transmission Scheme (FTS), rateless coding enables each node to send a fraction of the total encoded packets.
We have also proposed two schemes to construct LDPC codes that are suitable for unequal error protection (UEP). The first scheme is based on partially-regular LDPC codes. The proposed ensemble for the second scheme is a combination of two conventional bipartite graphs. We derived density evolution formulas for both the proposed UEP-LDPC ensembles. Using the density evolution formulas, high performance UEP codes were found. The proposed codes were also shown to have linear encoding complexity, which is very desirable for practical applications.
Related Publications: Rateless Coding Applications and UEP-LDPC