Web Scraping

Frances Leung

Data Science Fellow 2021-2022
School of Information

Frances Leung is a master’s student at UC Berkeley School of Information where she focuses her studies in information and data science. She has a keen interest in leveraging data-driven insights to better understand consumer behaviors and the world around us. In her professional work as a management consultant, she advises retailers and consumer businesses on digital transformation and creating web/mobile experiences that delight consumers through a human-centered approach. Frances holds a Master in Business Administration from York University, Schulich School...

Sahiba Chopra

Data Science Fellow 2024-2025
Haas School of Business

I'm a PhD student in the Management and Organizations (Macro) group at Berkeley Haas. I have a diverse professional background, primarily as a data scientist across numerous industries, including fintech, cleantech, and media. I hold a BA in Economics from the University of Maryland, an MS in Applied Economics from the University of San Francisco, and an MS in Business Administration from UC Berkeley.

My research focuses on the intersection of inequality, technology, and the labor market. I am particularly interested in understanding how to reduce inequality in...

The Evolving Landscape of Web Scraping on Social Media Platforms

March 11, 2025
Nanqin Ying
by Nanqin Ying. As social media platforms enforce stricter policies against unauthorized data collection, businesses and researchers must adapt to new API-based access models. This shift limits large-scale web scraping, impacting industries reliant on social media insights. The transition to paid API access and stringent compliance measures raises concerns about accessibility, cost, and ethical data collection. This article explores the evolving regulatory landscape, the enforcement of API restrictions, and how organizations can legally and ethically navigate data access in a world where scraping is becoming increasingly difficult. Understanding these changes is crucial for staying compliant while maintaining valuable insights from social media data.

Suraj Nair

Data Science Fellow 2023-2024
School of Information

I am a PhD Student at the School of Information. My research interests lie at the intersection of development economics and machine learning, with a focus on the use of large scale digital data and new computational tools to study pressing issues in global development.

Lauren Chambers

Consultant
School of Information

Lauren Chambers is a Ph.D. student at the Berkeley School of Information, where she studies the intersection of data, technology, and sociopolitical advocacy with Prof. Deirdre Mulligan. Previously Lauren was the staff technologist at the ACLU of Massachusetts, where she explored government data in order to inform citizens and lawmakers about the effects of legislation and political leadership on our civil liberties. Lauren received her Bachelor's from Yale in 2017, where she double-majored in astrophysics and African American studies, and she spent two years after graduation in...

Ini Umosen

Consultant
Economics

Ini is a PhD candidate in the Department of Economics. She studies topics in labor economics and the economics of education using applied econometrics methods. Current work in progress includes evaluating the impact of school choice systems and investigating gender and racial bias on gig platforms. She is a former Graduate Research Fellow at the California Policy Lab. She has also been a tutor for econometrics, labor economics, and macroeconomics.

Tom van Nuenen, Ph.D.

Data/Research Scientist, Senior Consultant, and Senior Instructor
D-Lab
Social Sciences
Digital Humanities

I work as a Lecturer, Data Scientist, and Senior Consultant at UC Berkeley's D-Lab. I lead the curriculum design for D-Lab’s data science workshop portfolio, as well as the Digital Humanities Summer Program at Berkeley.

Former research projects include a Research Associate position in the ‘Discovering and Attesting Digital Discrimination’ project at King’s College London (2019-2022) and a researcher-in-residence role for the UK’s National Research Centre on Privacy, Harm Reduction, and Adversarial Influence Online (2022). My research uses Natural Language Processing methods to...

Stephanie Andrews

Availability: By appointment only

Consulting Areas: Python, SQL, HTML / CSS, Javascript, APIs, Databases & SQL, Data Manipulation and Cleaning, Data Science, Data Sources, Data Visualization, Digital Humanities, Machine Learning, Natural Language Processing, Software Tools, Text Analysis, Web Scraping, Bash or Command Line, Excel, Git or Github, Tableau

Stephanie Andrews

Consultant
Info & Data Science MIDS

Stephanie Andrews is currently studying data science in the MIDS program, having previously majored in Social Welfare as an undergraduate at Cal. After graduating, she worked as an advocate for survivors of gender-based violence, as a public policy analyst focusing on anti-trafficking initiatives, and as a software engineer for progressive and social impact organizations. She is now conducting research with the Human Rights Center's Investigations Lab, using OSINT and data science methods to investigate human rights violations.

Kurt Soncco Sinchi

Consultant
Civil Engineering

First generation student and looking to improve and apply Data Science core concepts into social impactful projects, as well as trying to leverage the information from previous cases for better insights of society. Focused on infrastructure and its impact under natural disasters.