In this paper, we propose memory-augmented episodic value network (M-EVN) to learn a differentiable planning-based policy with episodic memory in maze games. The episodic memory module associates the environmental state to its corresponding state value function, and outputs a weighted sum of state value functions with similar states to improve the agents' navigation performance in partially observable mazes. In addition, we introduce a Net-in-Net architecture to make M-EVN differentiable by error backpropagation and learn an explicit planning computation. We train M-EVN in 2D maze games, and the experimental results show that the M-EVN agent outperforms the original value iteration network (VIN) in the partially observable maze games. © 2022 IEEE.

While being the de facto standard coordinate representation for human pose estimation, heatmap has not been investigated in-depth. This work fills this gap. For the first time, we find that the process of decoding the predicted heatmaps into the final joint coordinates in the original image space is surprisingly significant for the performance. We further probe the design limitations of the standard coordinate decoding method, and propose a more principled distributionaware decoding method. Also, we improve the standard coordinate encoding process (i.e.Transforming ground-Truth coordinates to heatmaps) by generating unbiased/accurate heatmaps. Taking the two together, we formulate a novel Distribution-Aware coordinate Representation of Keypoints (DARK) method. Serving as a model-Agnostic plug-in, DARK brings about significant performance boost to existing human pose estimation models. Extensive experiments show that DARK yields the best results on two common benchmarks, MPII and COCO. Besides, DARK achieves the 2nd place entry in the ICCV 2019 COCO Keypoints Challenge. The code is available online. © 2020 IEEE.

Multi-Agents are designed at different temporal levels to decomposed a complex task into several simple ones in hierarchical reinforcement learning. At the beginning of neural network training, the great changes of the low-level policy would cause the unstable transitions of the high-level one. In this paper, we propose a hierarchical policy combined with PPO and DDPG to deal with the simultaneous training of multi-Agents. To create an end-To-end policy, neural networks are employed to extract scene features in both low-level and high-level policies. In the meanwhile, a novel internal reward function is designed to enhance the goal achieving ability of low-level policy. A lightweight and fast gridworld Gym environment, MiniGrid, is used to test its validity. We found that the hierarchical policy is able to explore and plan without dense rewards. This attribute has a considerable influence on the study of robot navigation, especially in large and complex environment. © 2020 IEEE.

Generative adversarial networks (GANs) have attracted much attention since it is able to effective learn from an unknown real distribution. However, the instability of the training process greatly affects the quality of the generated images. To address this problem, the network structure-based, loss-based variant model and some training techniques are proposed. Unfortunately, there are some problems with the above methods, such as the limited effect of stabilizing the training process, the complex mathematical derivation, and the lack of universality of training techniques for different tasks. To this end, we propose a novel squeeze criterion GANs. In this method, we design a pseudo real module to synthesize adversarial sample and the double identity discriminator is designed. Then, the generated image and adversarial sample, as well as the generated image and real image form double adversarial learning. Through double adversarial learning, it forms a squeeze criterion to stabilize the training process of generator and discriminator. Finally, experimental results show that the proposed method has well portability and stabilizes the training process of existing GANs, and improves the quality of generated images. © 2020, Springer Nature Switzerland AG.