Author(s): Torty Sivill

Originally published on Towards AI.

Image from Unsplash, MOMO Studio

I’ve been publishing articles on Medium and occasionally sharing them on LinkedIn. Naturally, I wanted to know:

Is all this writing actually causing people to follow me on LinkedIn?

Or are my followers growing simply because I post on LinkedIn, regardless of whether there’s a Medium article involved?

To find out, I designed a simple but powerful causal inference experiment.

Medium: Manual Copy and Paste

Now this was actually really hard. Medium (perhaps unsurprisingly in hindsight) doesn’t make it easy for users to extract statistics relating to engagement and performance of their articles. These are the things I tried (and failed)

Medium’s “Download your data” tool (only exports posts, not stats)Automated scrapers (Selenium, Playwright — blocked or invisible DOM)Intercepting network requests (GraphQL payloads hidden behind client-side rendering)

After a while of fruitless attempt at automation I realised I was going to have to do this the much less satisfying way: a classic copy and paste job.

Manually, I scrolled my Medium stats page and copy and pasted into a .txt file. I then parsed that file into structured data using the following Python script.

import pandas as pdimport retext = open("medium_stats_raw.txt").read()pattern = re.compile( r"(.*?)\n([A-Za-z]{3,9} \d{1,2}, \d{4})\n([\d.,K]+)\nViews\n([\d.,K]+)\nReads\n(\$[\d.,]+|-)\nEarnings")def convert_k(val): return int(float(val.replace('K','')) * 1000) if 'K' in val… Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI