Hey, I want to let you know that my deep and machine learning library has Q-Learning Neural Networks and SARSA neural network. But if you want the ones with experience replay, you have to take the Beta version of the library (can be either package or module script).
Here’s the link: