TY - GEN
T1 - Can twitter API be bypassed? A new methodology for collecting chronological information without restrictions
AU - Hernandez-Suarez, Aldo
AU - Sanchez-Perez, Gabriel
AU - Toscano-Medina, Karina
AU - Toscano-Medina, Rocio
AU - Martinez-Hernandez, Victor
AU - Olivares-Mercado, Jesus
AU - Perez-Meana, Hector
AU - Sanchez, Victor
N1 - Publisher Copyright:
© 2018 The authors and IOS Press. All rights reserved.
PY - 2018
Y1 - 2018
N2 - Retrieving information from social networks is a first and primordial step in many data analysis fields such as Natural Language Processing and Machine Learning. Important data science tasks rely on historical data gathering for further predictive results. Recent works use public platforms for collecting public streams of information like Twitter API, which allows querying chronological tweets from periods no longer than three weeks. In this paper, we present Twitter Scrapy, a new methodology for collecting historical tweets from time periods of arbitrary duration using web scraping techniques that bypass Twitter API restrictions.
AB - Retrieving information from social networks is a first and primordial step in many data analysis fields such as Natural Language Processing and Machine Learning. Important data science tasks rely on historical data gathering for further predictive results. Recent works use public platforms for collecting public streams of information like Twitter API, which allows querying chronological tweets from periods no longer than three weeks. In this paper, we present Twitter Scrapy, a new methodology for collecting historical tweets from time periods of arbitrary duration using web scraping techniques that bypass Twitter API restrictions.
KW - Twitter bots
KW - Twitter scrapy
KW - Web crawling
KW - Web scraping
KW - Web spiders
UR - http://www.scopus.com/inward/record.url?scp=85063402491&partnerID=8YFLogxK
U2 - 10.3233/978-1-61499-900-3-453
DO - 10.3233/978-1-61499-900-3-453
M3 - Contribución a la conferencia
T3 - Frontiers in Artificial Intelligence and Applications
SP - 453
EP - 462
BT - New Trends in Intelligent Software Methodologies, Tools and Techniques - Proceedings of the 17th International Conference, SoMeT 2018
A2 - Fujita, Hamido
A2 - Herrera-Viedma, Enrique
PB - IOS Press BV
T2 - 17th International Conference on New Trends in Intelligent Software Methodology Tools and Techniques, SoMeT 2018
Y2 - 26 September 2018 through 28 September 2018
ER -