Adaptive servo system for die-sinking micro-EDM driven by deep Q-network with online-offline combined data

Die-sinking micro electrical discharge machining (micro-EDM) belongs to non-conventional manufacturing methods. However, the process mechanism is complex and it is difficult to describe the process by an accurate mathematical model. Deep reinforcement learning (DRL), the combination of neural network and reinforcement learning (RL), successfully achieves the direct mapping from high-dimension states to scores of different actions, which enables an end-to-end control scheme, from process feedback data to action strategies. Comparing to training methods in traditional deep learning (DL), part or even all datasets for DRL stem from online environment-interactive data, enabling the adaptive ability. This article introduces a RL algorithm based on Deep Q-Network (DQN) and embeds it in the servo system for die-sinking micro-EDM. Based on online-offline combined process data and a priori-knowledge based reward function, the experience tuple for DRL generates automatically after every servo motion step and the Q-network updates for servo strategies. The experiments verify that the proposed DQN driven adaptive servo system for die-sinking micro-EDM can maintain the discharge efficiency more aggressively and avoid short circuits to a much extent, greatly enhancing the machining efficiency.