Recently, many stochastic variance reduced alternating direction methods of multipliers (ADMMs) (e.g., SAG-ADMM and SVRG-ADMM) have made exciting progress such as linear convergence rate for strongly convex (SC) problems. However, their best-known convergence rate for non-strongly convex (non-SC) problems is O(1/T) as opposed to O(1/T-2) of accelerated deterministic algorithms, where T is the number of iterations. Thus, there remains a gap in the convergence rates of existing stochastic ADMM and deterministic algorithms. To bridge this gap, we introduce a new momentum acceleration trick into stochastic variance reduced ADMM, and propose a novel accelerated SVRG-ADMM method (called ASVRG-ADMM) for the machine learning problems with the constraint Ax + By = c. Then we design a linearized proximal update rule and a simple proximal one for the two classes of ADMM-style problems with B = tau I and B not equal tau I, respectively, where I is an identity matrix and tau is an arbitrary bounded constant. Note that our linearized proximal update rule can avoid solving sub-problems iteratively. Moreover, we prove that ASVRG-ADMM converges linearly for SC problems. In particular, ASVRG-ADMM improves the convergence rate from O(1/T) to O(1/T-2) for non-SC problems. Finally, we apply ASVRG-ADMM to various machine learning problems, e.g., graph-guided fused Lasso, graph-guided logistic regression, graph-guided SVM, generalized graph-guided fused Lasso and multi-task learning, and show that ASVRG-ADMM consistently converges faster than the state-of-the-art methods.

有效的特征表示对深度学习模型的表征能力和图像识别性能至关重要。多尺度特征表示方法能够捕捉不同尺度的丰富信息，从而有助于提高深度学习模型的图像识别性能。然而，当前的多尺度深度学习方法仍存在对图像方向特征建模不明确的局限，导致对具有方向性目标的误识别。为了更好地表示图像中蕴含的多方向特征，本文提出了一种基于多尺度-多方向Transformer的网络框架（MSMDFormer）。该框架首先设计了一种多方向编码器，能够捕获并增强多个方向特征。在此基础上，本文联合不同尺度的Gabor 表征与多头注意力，设计了一种多尺度多方向Transformer编码器，以有效地聚合图像的多尺度和多方向特征。最后，该框架对卷积特征和多尺度-多方向特征进行融合，然后将融合特征用于图像识别。实验结果表明，MSMDFormer 在CIFAR10、CIFAR100和SVHN数据集上分别取得了95.65%、77.46%和96.87%的整体准确率，在与19种基准方法的对比中显示出具有竞争力的图像分类性能。与11种图像分割基准方法相比，MSMDFormer 在ADE20K数据集上展现出0.33%至6.58%mIoU的性能增益。综上所述，本文提出的MSMDFormer在深度学习图像识别任务中展现了卓越的特征表示能力，具有广泛的应用前景。另外，探索更有效的方向特征表示方法将成为未来研究的重要方向。

Modeling contextual relationships in images as graph inference is an interesting and promising research topic. However, existing approaches only perform graph modeling of entities, ignoring the intrinsic geometric features of images. To overcome this problem, a novel multiresolution interpretable contourlet graph network (MICGNet) is proposed in this article. MICGNet delicately balances graph representation learning with the multiscale and multidirectional features of images, where contourlet is used to capture the hyperplanar directional singularities of images and multilevel sparse contourlet coefficients are encoded into graph for further graph representation learning. This process provides interpretable theoretical support for optimizing the model structure. Specifically, first, the superpixel-based region graph is constructed. Then, the region graph is applied to code the nonsubsampled contourlet transform (NSCT) coefficients of the image, which are considered as node features. Considering the statistical properties of the NSCT coefficients, we calculate the node similarity, i.e., the adjacency matrix, using Mahalanobis distance. Next, graph convolutional networks (GCNs) are employed to further learn more abstract multilevel NSCT-enhanced graph representations. Finally, the learnable graph assignment matrix is designed to get the geometric association representations, which accomplish the assignment of graph representations to grid feature maps. We conduct comparative experiments on six publicly available datasets, and the experimental analysis shows that MICGNet is significantly more effective and efficient than other algorithms of recent years.

遥感视频卫星可以实现对整体环境和具体目标连续的全方位实时监控,从而高效率高精度的收集对地观测信息,在大范围交通监控、国防安全监测、军事动态指导等场景有着重要的应用价值,因此在遥感视频卫星数据上实现目标跟踪就显得至关重要。但是由于遥感视频卫星的自身特点使得遥感视频目标跟踪面临着目标尺寸较小、背景极为复杂、遮挡情况严重、目标可识别性较差等难题,现有跟踪器难以应对如此复杂的情况,算法鲁棒性不佳。针对上述情况,本文在孪生网络框架的基础上,研究了无锚跟踪策略、条带波理论、注意力机制、卡尔曼滤波对遥感视频目标跟踪的性能增幅,从而更加高效准确的实现遥感视频目标跟踪,主要工作如下:首先,提出了一种基于Bandelet和孪生网络的无锚遥感视频目标跟踪算法,来解决遥感影像目标尺度变化大,以及目标尺寸较小时提取到的特征判别能力不足等问题。该方法的主要思想是将无锚跟踪器迁移到遥感影像上,更好的适应遥感视频中目标位置和尺度的变化。并且将条带波变换与孪生网络相结合,帮助跟踪网络自适应的对图像的内在几何结构进行追踪,对于尺寸较小的目标也能够更加准确高效的提取到目标特征。实验结果显示,本章方法能够有效提升遥感视频中不同尺度目标跟踪的性能。其次,提出了一种基于多重注意力机制的无锚遥感视频目标跟踪算法。针对遥感视频中目标与背景区分度不高的问题以及互相关操作特征融合不充分的问题,引入了一种基于多重注意力机制的特征提取网络,包括组合注意力机制和交叉注意力机制。组合注意力机制使得跟踪网络能够更加重视与目标相关的有效特征信息,在跟踪过程中更加精确地定位到目标,交叉注意力机制使得孪生网络的两个分支在计算特征期间就能够进行有效的信息融合和交互,这样互相关操作时就能够更加准确的找到搜索区域中与模板帧相似的区域。实验结果显示,多重注意力机制能够帮助网络提取到更加有效的目标特征用于跟踪,从而提升遥感视频目标跟踪的性能。最后,提出了一种基于卡尔曼滤波的无锚遥感视频目标跟踪算法。为了应对遥感视频目标跟踪中常见的遮挡问题,首先提出了一个检测目标被遮挡和目标出遮挡的判断标准,接着使用卡尔曼滤波计算目标在被遮挡前大致的运动状态,在检测到目标被遮挡时以同样的运动速度和方向来移动跟踪框,这样当遮挡结束目标再次出现时,仍然在搜索范围内,使得跟踪器可以重新定位到目标进行跟踪。实验结果显示,采用本章方法对遥感视频进行跟踪,能够大大提升遮挡场景下遥感视频目标跟踪的性能。

随着遥感技术相关领域的飞速发展,遥感影像的应用需求也日益增加。遥感图像语义分割,作为一项理解遥感影像的关键视觉任务,已广泛应用于城市规划、智能农业、环境保护等领域。近年来,深度学习的快速发展促进了遥感图像语义分割性能的不断提升,但由于遥感图像数据本身具有图像尺度大、成像范围广、背景信息复杂多变的特点,现有方法仍然在分割多尺度物体时面临分割精度低的挑战。本文针对这些挑战,对遥感图像语义分割展开了研究,并基于多样化的注意力机制模块和多尺度表征学习对网络进行了设计和改进以提高分割的准确率。本文主要研究内容包括以下几方面:1、提出了一种基于混合注意力和空洞空间金字塔池化的遥感图像语义分割方法。针对遥感图像类内差异大、类间差异小而导致的分类不准确的问题,提出一种基于多尺度表征学习的网络,该网络通过混合注意力模块,从多个维度自适应地对全局像素间关联进行学习。该模块以网络中层输出的粗略分割预测图辅助监督网络训练,强调挖掘鉴别性特征。通过在Potsdam数据集和Vaihingen数据集上进行对比实验和分割效果可视化,验证了该方法的有效性和可行性。2、提出了一种基于多阶段注意力机制的遥感图像语义分割方法。基于编码器-解码器结构的分割方法,在编码过程中进行下采样操作会丢失部分位置信息,通过跳跃连接直接融合编码器、解码器中特征往往不能充分利用不同层级特征信息的优势,特征融合不具有适应性。针对这一问题,提出了一种多阶段的注意力网络框架,在网络的不同阶段使用不同类型的注意力模块对有效特征进行强调;针对空间维度的点积注意力计算复杂度高、运算效率低的问题,提出了一种区域注意力模块,以轻量化的网络对区块间联系进行学习,丰富了上下文信息。通过在Potsdam数据集和Vaihingen数据集上进行对比实验和分割效果可视化,验证了该方法的有效性和可行性。3、提出了一种基于特征金字塔注意力和多尺度精细结构的遥感图像语义分割方法。基于空洞空间金字塔池化的网络可以通过多尺度表征学习提高网络对不同大小物体的分割精确率。但空洞卷积的引入往往使网络的计算效率降低,且由于空洞率的存在,采样点相互独立损害了网络对全局上下文信息的获取。针对该问题,提出了一种特征金字塔注意力模块,使用较小的网络参数对多尺度特征进行学习;针对遥感图像背景信息复杂、分类难度大的问题,在基于多阶段注意力机制的分割框架上,嵌入了多尺度精细网络结构,挖掘类别特征,实现了对分割结果由粗到细的优化。通过在Potsdam数据集和Vaihingen数据集上进行对比实验和分割效果可视化,验证了该方法的有效性和可行性。

目标检测是计算机视觉领域的热点和核心问题之一,其目的是在一幅图像中寻找特定的目标,然后对目标进行精确的定位和分类。它是分析和解译视觉信号的重要基础,已被广泛应用于工业质检、视频监控和无人驾驶等众多领域。近年来,目标检测的研究取得了一系列进展,然而图像目标检测算法仍然面临着许多挑战。本文从提升目标检测性能的三个不同角度出发,使得定位更加准确,分类更加精确,模型更加轻量。首先,本文从目标检测中的定位角度出发,提出了一个简单而有效的共差注意力模块,抑制无关背景干扰,使得定位更加准确。对于输入的中间特征图,共差注意力模块分别沿着通道和空间两个不同的维度并行地计算注意力特征图,然后将这两个注意力特征图与输入特征图相乘得到更加精细的特征。特别地,此注意力模块采用共差运算来削弱无用的背景干扰,从而关注有意义的目标特征。实验在不同数据集上用不同的检测器和骨干网络验证了此注意力模块的有效性。其次,本文从目标检测中的分类角度出发,改进了一个基于外部背景知识图的目标检测框架,使得分类更加精确。与现有的只关注图像内部特征的机器学习和深度学习算法不同,本文利用外部背景知识,如知识图谱等,优化了目标检测框架。该框架采用语义一致性的概念对知识进行量化分析和迁移,将知识图谱等外部知识集成到现有的目标检测器中,再通过重新优化实现了知识感知检测功能,从而提高了目标检测的分类性能。实验验证了此框架能够在保持目标定位精度的同时,大大提高了分类的准确性。最后,本文从目标检测的计算复杂性角度出发,使用了基于小波变换的图神经网络模型,通过图小波变换替代图傅里叶变换,定义了谱图卷积,有效地降低了神经网络计算资源的消耗,使得模型更加轻量。同时,通过图小波变换算法对图片中的目标做边缘提取,使得网络模型可以在不规则区域内提取目标的边缘特征。综上,本文对以上研究工作进行了总结,并对未来值得探索的研究方向进行了梳理。

随着遥感技术的飞速发展,目前可获得的遥感数据越来越丰富,针对遥感数据的场景理解也越发引起关注。语义分割任务作为遥感数据场景理解中的重要任务,有着极其广泛的应用。虽然基于深度学习的方法已在遥感图像分割领域取得了优异的效果,但还存在资源消耗大、边缘分割困难、上下文处理方法仍需优化等问题。针对这些问题,本文结合多任务学习、注意力机制及类别上下文增强思想对遥感图像语义分割展开研究。本文主要研究内容如下:1.提出了一种基于图像超分辨与特征增强的多任务遥感图像语义分割方法。目前主流算法通常面临资源消耗较大的问题。本文从输入端考虑,对低分辨率输入下的语义分割效果进行优化,为解决该问题提供另一种思路。该方法结合多任务学习思想,令同一网络框架下同时包含语义分割和超分辨两个任务分支。超分辨任务作为辅助任务,通过学习图像的高分辨表示来获得额外的细节信息。进而通过特征增强模块学习两个任务特征间的相似性关系进行任务间交互,实现对语义分割任务的辅助。经过实验,所提出方法在三个数据集上均优于各基线网络,验证了该方法的有效性。2.提出了一种基于边缘检测与双注意力的多任务遥感图像语义分割方法。语义分割领域长期存在着边缘部分预测困难的问题。考虑到语义分割与边缘检测耦合性较高,且两个任务间可以有效地相互促进。本文结合多任务学习思想,在语义分割网络框架下引入边缘检测任务。该网络针对边缘检测任务要求精细化结果的特性,引入了双注意力模块对低级别特征进行额外处理,同时使用可学习上采样模块实现更精细的图像上采样。并设计了一种边缘优化模块,对分支间的一致性关系进行学习,从而实现任务间交互。经过实验,所提出方法在三个数据集上均优于各基线网络,验证了该方法的有效性。3.提出了一种基于上下文增强与多任务优化的遥感图像语义分割方法。主要针对上下文处理方法及多任务学习框架再优化问题进行设计。为实现更有效的上下文处理,通过监督学习初步得到特征信息的类别区域表示,并使用特征间的关联关系进行特征增强,实现适应目标区域范围的上下文信息处理。为进一步发挥多任务学习的优势,设计了一种更直接的跨任务交互方法,令边缘特征作为掩膜直接对语义分割特征进行增强。针对多任务学习框架训练过程中的权重适应问题,设计了一种自调节多任务权重方法,协调训练过程中不同任务之间的关系。经过实验,所提出方法在三个数据集上均优于各基线网络,验证了该方法的有效性。

Fusion and interaction of multimodal features are essential for video question answering. Structural information composed of the relationships between different objects in videos is very complex, which restricts understanding and reasoning. In this paper, we propose a quaternion hypergraph network (QHGN) for multimodal video question answering, to simultaneously involve multimodal features and structural information. Since quaternion operations are suitable for multimodal interactions, four components of the quaternion vectors are applied to represent the multimodal features. Furthermore, we construct a hypergraph based on the visual objects detected in the video. Most importantly, the quaternion hypergraph convolution operator is theoretically derived to realize multimodal and relational reasoning. Question and candidate answers are embedded in quaternion space, and a Q & A reasoning module is creatively designed for selecting the answer accurately. Moreover, the unified framework can be extended to other video-text tasks with different quaternion decoders. Experimental evaluations on the TVQA dataset and DramaQA dataset show that our method achieves state-of-the-art performance.

With the increasing accessibility of remote sensing videos, remote sensing tracking is gradually becoming a hot issue. However, accurately detecting and tracking in complex remote sensing scenes is still a challenge. In this article, we propose a collaborative learning tracking network for remote sensing videos, including a consistent receptive field parallel fusion module (CRFPF), dual-branch spatial-channel co-attention (DSCA) module, and geometric constraint retrack strategy (GCRT). Considering the small-size objects of remote sensing scenes are difficult for general forward networks to extract effective features, we propose a CRFPF-module to establish parallel branches with consistent receptive fields to separately extract from shallow to deep features and then fuse hierarchical features adaptively. Since the objects and their background are difficult to distinguish, the proposed DSCA-module uses the spatial-channel co-attention mechanism to collaboratively learn the relevant information, which enhances the saliency of the objects and regresses to precise bounding boxes. Considering the interference of similar objects, we designed a GCRT-strategy to judge whether there is a false detection through the estimated motion trajectory and then recover the correct object by weakening the feature response of interference. The experimental results and theoretical analysis on multiple datasets demonstrate our proposed method's feasibility and effectiveness. Code and net are available at https://github.com/Dawn5786/CoCRF-TrackNet.

With the increasing of the data size and the development of multi-core computers, asynchronous parallel stochastic optimization algorithms such as KroMagnon have gained significant attention. In this paper, we propose a new Sparse approximation and asynchronous parallel Stochastic Variance Reduced Gradient (SSVRG) method for sparse and high-dimensional machine learning problems. Unlike standard SVRG and its asynchronous parallel variant, KroMagnon, the snapshot point of SSVRG is set to the average of all the iterates in the previous epoch, which allows it to take much larger learning rates and also makes it more robust to the choice of learning rates. In particular, we use the sparse approximation of the popular SVRG estimator to perform completely sparse updates at all iterations. Therefore, SSVRG has a much lower per-iteration computational cost than its dense counterpart, SVRG++, and is very friendly to asynchronous parallel implementation. Moreover, we provide the convergence guarantees of SSVRG for both strongly convex and non-strongly convex problems, while existing asynchronous algorithms (e.g., KroMagnon and ASAGA) only have convergence guarantees for strongly convex problems. Finally, we extend SSVRG to non-smooth and asynchronous parallel settings. Numerical experimental results demonstrate that SSVRG converges significantly faster than the state-of-the-art asynchronous parallel methods, e.g., KroMagnon, and is usually more than three orders of magnitude faster than SVRG++.