Reinforcement Learning Compensation based PD Control for Inverted Pendulum

Guillermo Puriel-Gil; Wen Yu; Humberto Sossa

doi:10.1109/ICEEE.2018.8533946

Reinforcement Learning Compensation based PD Control for Inverted Pendulum

Guillermo Puriel-Gil, Wen Yu, Humberto Sossa

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

11 Scopus citations

Abstract

In this paper, we present a Control Algorithm based on Reinforcement Learning for an inverted pendulum. By implementing the Q-Learning techniques in the PD control scheme, the pendulum is enabled to improve its online performance and adapt to changes in the parameters of the pendulum model. In a first step, Q-Learning is used so that the control can balance the pendulum towards its inverted vertical position; In a second step, we combine hybrid techniques of Q-Learning and PD control. With this combination, we can bring the pendulum to its inverted vertical position, regardless of the applied disturbance. Finally, the results of the simulation show the effectiveness of the proposed controller.

Original language	English
Title of host publication	2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781538670323
DOIs	https://doi.org/10.1109/ICEEE.2018.8533946
State	Published - 13 Nov 2018
Externally published	Yes
Event	15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018 - Mexico City, Mexico Duration: 5 Sep 2018 → 7 Sep 2018

Publication series

Name	2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018

Conference

Conference	15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018
Country/Territory	Mexico
City	Mexico City
Period	5/09/18 → 7/09/18

Keywords

Inverted Pendulum.
Q-Learning
Reinforcement Learning

Access to Document

10.1109/ICEEE.2018.8533946

Cite this

Puriel-Gil, G., Yu, W., & Sossa, H. (2018). Reinforcement Learning Compensation based PD Control for Inverted Pendulum. In 2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018 Article 8533946 (2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICEEE.2018.8533946

Puriel-Gil, Guillermo ; Yu, Wen ; Sossa, Humberto. / Reinforcement Learning Compensation based PD Control for Inverted Pendulum. 2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018. Institute of Electrical and Electronics Engineers Inc., 2018. (2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018).

@inproceedings{8e23f5c2ff504c13818eb26dc8b476d4,

title = "Reinforcement Learning Compensation based PD Control for Inverted Pendulum",

abstract = "In this paper, we present a Control Algorithm based on Reinforcement Learning for an inverted pendulum. By implementing the Q-Learning techniques in the PD control scheme, the pendulum is enabled to improve its online performance and adapt to changes in the parameters of the pendulum model. In a first step, Q-Learning is used so that the control can balance the pendulum towards its inverted vertical position; In a second step, we combine hybrid techniques of Q-Learning and PD control. With this combination, we can bring the pendulum to its inverted vertical position, regardless of the applied disturbance. Finally, the results of the simulation show the effectiveness of the proposed controller.",

keywords = "Inverted Pendulum., Q-Learning, Reinforcement Learning",

author = "Guillermo Puriel-Gil and Wen Yu and Humberto Sossa",

note = "Publisher Copyright: {\textcopyright} 2018 IEEE.; 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018 ; Conference date: 05-09-2018 Through 07-09-2018",

year = "2018",

month = nov,

day = "13",

doi = "10.1109/ICEEE.2018.8533946",

language = "Ingl{\'e}s",

series = "2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018",

address = "Estados Unidos",

}

Puriel-Gil, G, Yu, W & Sossa, H 2018, Reinforcement Learning Compensation based PD Control for Inverted Pendulum. in 2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018., 8533946, 2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018, Institute of Electrical and Electronics Engineers Inc., 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018, Mexico City, Mexico, 5/09/18. https://doi.org/10.1109/ICEEE.2018.8533946

Reinforcement Learning Compensation based PD Control for Inverted Pendulum. / Puriel-Gil, Guillermo; Yu, Wen; Sossa, Humberto.
2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018. Institute of Electrical and Electronics Engineers Inc., 2018. 8533946 (2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Reinforcement Learning Compensation based PD Control for Inverted Pendulum

AU - Puriel-Gil, Guillermo

AU - Yu, Wen

AU - Sossa, Humberto

PY - 2018/11/13

Y1 - 2018/11/13

N2 - In this paper, we present a Control Algorithm based on Reinforcement Learning for an inverted pendulum. By implementing the Q-Learning techniques in the PD control scheme, the pendulum is enabled to improve its online performance and adapt to changes in the parameters of the pendulum model. In a first step, Q-Learning is used so that the control can balance the pendulum towards its inverted vertical position; In a second step, we combine hybrid techniques of Q-Learning and PD control. With this combination, we can bring the pendulum to its inverted vertical position, regardless of the applied disturbance. Finally, the results of the simulation show the effectiveness of the proposed controller.

AB - In this paper, we present a Control Algorithm based on Reinforcement Learning for an inverted pendulum. By implementing the Q-Learning techniques in the PD control scheme, the pendulum is enabled to improve its online performance and adapt to changes in the parameters of the pendulum model. In a first step, Q-Learning is used so that the control can balance the pendulum towards its inverted vertical position; In a second step, we combine hybrid techniques of Q-Learning and PD control. With this combination, we can bring the pendulum to its inverted vertical position, regardless of the applied disturbance. Finally, the results of the simulation show the effectiveness of the proposed controller.

KW - Inverted Pendulum.

KW - Q-Learning

KW - Reinforcement Learning

UR - http://www.scopus.com/inward/record.url?scp=85058433348&partnerID=8YFLogxK

U2 - 10.1109/ICEEE.2018.8533946

DO - 10.1109/ICEEE.2018.8533946

M3 - Contribución a la conferencia

AN - SCOPUS:85058433348

T3 - 2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018

BT - 2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018

Y2 - 5 September 2018 through 7 September 2018

ER -

Puriel-Gil G, Yu W, Sossa H. Reinforcement Learning Compensation based PD Control for Inverted Pendulum. In 2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018. Institute of Electrical and Electronics Engineers Inc. 2018. 8533946. (2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2018). doi: 10.1109/ICEEE.2018.8533946

Reinforcement Learning Compensation based PD Control for Inverted Pendulum

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this