Cross-modal clustering (CMC) aims to enhance the clustering performance by exploring complementary information from multiple modalities. However, the performances of existing CMC algorithms are still unsatisfactory due to the conflict of heterogeneous modalities and the high-dimensional non-linear property of individual modality. In this paper, a novel deep mutual information maximin (DMIM) method for cross-modal clustering is proposed to maximally preserve the shared information of multiple modalities while eliminating the superfluous information of individual modalities in an end-to-end manner. Specifically, a multi-modal shared encoder is firstly built to align the latent feature distributions by sharing parameters across modalities. Then, DMIM formulates the complementarity of multi-modalities representations as a mutual information maximin objective function, in which the shared information of multiple modalities and the superfluous information of individual modalities are identified by mutual information maximization and minimization respectively. To solve the DMIM objective function, we propose a variational optimization method to ensure it converge to a local optimal solution. Moreover, an auxiliary overclustering mechanism is employed to optimize the clustering structure by introducing more detailed clustering classes. Extensive experimental results demonstrate the superiority of DMIM method over the state-of-the-art cross-modal clustering methods on IAPR-TC12, ESP-Game, MIRFlickr and NUSWide datasets.

序列化推荐因其实用性和较高推荐精度在近期受到了人们广泛关注.不同于传统推荐方法,序列化推荐的核心在于如何基于用户近期交互行为来捕获用户的短期兴趣.现有工作或者依次考虑用户交互序列中物品之间的成对关系,忽略了更为重要的多对

Multiview clustering (MVC) has recently been the focus of much attention due to its ability to partition data from multiple views via view correlations. However, most MVC methods only learn either interfeature correlations or intercluster correlations, which may lead to unsatisfactory clustering performance. To address this issue, we propose a novel dual-correlated multivariate information bottleneck (DMIB) method for MVC. DMIB is able to explore both interfeature correlations (the relationship among multiple distinct feature representations from different views) and intercluster correlations (the close agreement among clustering results obtained from individual views). For the former, we integrate both view-shared feature correlations discovered by learning a shared discriminative feature subspace and view-specific feature information to fully explore the interfeature correlation. This allows us to attain multiple reliable local clustering results of different views. Following this, we explore the intercluster correlations by learning the shared mutual information over different local clusterings for an improved global partition. By integrating both correlations, we formulate the problem as a unified information maximization function and further design a two-step method for optimization. Moreover, we theoretically prove the convergence of the proposed algorithm, and discuss the relationships between our method and several existing clustering paradigms. The experimental results on multiple datasets demonstrate the superiority of DMIB compared to several state-of-the-art clustering methods.

Multiview clustering (MVC) provides a natural formulation to generate clusters for multiview data, which is fundamental to lots of industrial tasks like autonomous driving, defect detection, and multisensor information fusion, as part of the foundation models. Most existing MVC methods suppose that the data of multiple views are available during the clustering process. However, that is a very strong assumption and is impractical when the views are incremental over time. In addition, if directly applying existing MVC approaches to the clustering setting with incremental views, the massive redundant information in each view might limit the knowledge sharing between historical and newly arrived views. To solve these problems, a continual information bottleneck (CIB) method is presented in this article, which addresses the incremental MVC issue by maximally preserving the consistency of a sequence of views and removing the redundant information in each view. In particular, to facilitate the knowledge transfer from historical views to incoming one, we build a knowledge library to store the representative samples in historical views. When adding a new view, we first construct a view-specific encoder with information-theoretic constraints to learn a compact and discriminative representation, in which redundant information in the new view is eliminated. Then, to capture the consistency information between historical views and the new view, a shared encoder is devised after retrieving the global neighbors in the library for the new view, which is performed by contrasting the cluster assignment and feature representation simultaneously. Finally, a unified objective function is devised to simultaneously optimize the knowledge library and clustering process, in which the knowledge library is updated by maximizing the mutual information between the new view and all historical ones to keep tracking knowledge about the earlier views. Extensive experiment on nine multiview benchmarks has verified the superiority of the CIB method over 19 baselines.

Next-item recommendation has been a hot research topic in academia and industry, which aims to help users discover the next interesting item. In this article, we propose a novel solution, namely graph-augmented social translation model (GAST), which investigates the utility of dynamic social influence for the task of next-item recommendation. Specifically, we introduce a gated graph convolution module to better model long-term user preference. Furthermore, we design a cogating module to capture dynamic patterns at both sequential level and social level. In addition, a social-enhanced translation mechanism is devised to measure the intensity of user-item relationships. Extensive experiments under different recommendation scenarios demonstrate the rationality and effectiveness of our proposed GAST method over several state-ofthe-art methods.

A sequential recommendation has become a hot research topic, which seeks to predict the next interesting item for each user based on his action sequence. While previous methods have made many efforts to capture the dynamics of sequential patterns, we contend that they still suffer from two inherent limitations: 1) they fail to model item transition patterns in an efficient and time-sensitive manner and 2) they are unaware of the importance of dynamically capturing social influence, resulting in suboptimal performance. We introduce a new concept dubbed socio-sequential recommendation, where the challenge mainly lies in dynamically modeling social influences and capturing item-to-item transition patterns in a time-sensitive manner. In light of this, we contribute a novel solution named GCARec (short for graph-augmented co-attention model), which takes into account the joint effect of dynamic sequential patterns and dynamic social influences. GCARec decomposes socio-sequential recommendation workflow into two steps. First, we adopt a light graph embedding module to model long-term user preference. Then, we propose a time-sensitive attention mechanism and a social-aware attention mechanism to capture dynamic patterns at sequential-level and social-level, respectively. Extensive experiments have been conducted on eight real-world datasets from different scenarios, demonstrating the superiority of GCARec against several state-of-the-art methods. The codes and datasets have been released at: https://github.com/wubinzzu/GCARec.

In many practical applications, massive data are observed from multiple sources, each of which contains multiple cohesive views, called hierarchical multiview (HMV) data, such as image-text objects with different types of visual and textual features. Naturally, the inclusion of source and view relationships offers a comprehensive view of the input HMV data and achieves an informative and correct clustering result. However, most existing multiview clustering (MVC) methods can only process single-source data with multiple views or multisource data with single type of feature, failing to consider all the views across multiple sources. Observing the rich closely related multivariate (i.e., source and view) information and the potential dynamic information flow interacting among them, in this article, a general hierarchical information propagation model is first built to address the above challenging problem. It describes the process from optimal feature subspace learning (OFSL) of each source to final clustering structure learning (CSL). Then, a novel self-guided method named propagating information bottleneck (PIB) is proposed to realize the model. It works in a circulating propagation fashion, so that the resulting clustering structure obtained from the last iteration can "self-guide " the OFSL of each source, and the learned subspaces are in turn used to conduct the subsequent CSL. We theoretically analyze the relationship between the cluster structures learned in the CSL phase and the preservation of relevant information propagated from the OFSL phase. Finally, a two-step alternating optimization method is carefully designed for optimization. Experimental results on various datasets show the superiority of the proposed PIB method over several state-of-the-art methods.

Sequential recommendation has become a popular and indispensable component of various online services, which aims to predict the next interested item based on the sequence of a certain user. To deduce users’ actual interests, sequential recommenders concentrate on analyzing the complex transition dependency from the user's recent action sequence. The key types of item transition patterns can be generally divided into item-level and factor-level. However, most existing works directly focus on a one-channel chain of interaction sequence, and only capture item co-occurrence patterns from item-level. They neglect the availability of transitions among items’ latent attributes. Toward this end, we propose a Gating Augmented Capsule Network (GAC), which models both personalized item- and factor-level transitions in a fine-grained manner. Specifically, to distill user-specific information, we present a personalized gating module to replace the convolution operation of the traditional capsule network, so as to augment the links between the user and each item. Moreover, we design an item-routing component and a factor-routing component to build a two-channel routing module for capturing item- and factor-level interactions, respectively, while preserving the relative order of items in the action sequence. Extensive experiments on four public benchmarks demonstrate the effectiveness of our proposed GAC compared to several state-of-the-art baselines. © 2022 Elsevier B.V.

基于深度学习的数字聚合酶链式反应(PCR)液滴识别对PCR图像中的目标进行高阶语义建模,能够减少人工参与特征设计和筛选带来的误差,但忽略了目标的低层物理结构和几何外观细节信息,且在特征建模的过程中重复使用了大量冗余信息,对特征的