Our forum's API

Our MathStat.HWDiscuss forum is powered by open source software called Discourse which has an API that allows us to scrape data right off of it. Check out the following two pages:

The first is just our standard landing page. The second contains the same information, but formatted as JSON. That highly structed format allows us to manipulate it with a computer quite easily. For example, here is the list of categories currently on our forum.

In [1]:
import requests
response = requests.get("https://mathstat.hwdiscuss.com/categories.json")
if response.status_code == 200:
    response_json = response.json()
    print([category['name'] for category in response_json['category_list']['categories']])
else:
    print("Uh-oh")

['About this site', 'Statistics', 'Pre-Calculus', 'Differential Calculus', 'Integral Calculus', 'Multivariable Calculus', 'Linear Algebra', 'Computers and Code', 'Site Feedback']

Now, we can dig a little deeper into the individual categories. For example, here is the list of all the topics in the Statistics category, which happens to have id=11:

In [2]:
response = requests.get("https://mathstat.hwdiscuss.com/c/11.json")
response_json = response.json()
topic_titles = [topic['title'] for topic in response_json['topic_list']['topics']]
topic_titles
Out[2]:
['About the Statistics category',
 'When to use N vs N-1 in equations',
 'Midterm Review',
 'Exam #1 Review Sheet',
 'A hypothesis test for random heights',
 'Random YouTube Videos',
 'What am I doing wrong',
 'CDC BMIs',
 'Stat HW - 6/21',
 'Issue with homework due 6/20',
 'Hypothesis Testing HW',
 'Dadgum HW',
 'Homework issue',
 'My Open math HW',
 'MyOpenMath Homework due 6/13/2018',
 'Random CDC like data',
 'Will You Use Stats',
 'Probability with Z-Score Confusion',
 'Standard deviation',
 'Standard Deviation Practice',
 'Exercise 1.6',
 'Some personal data',
 'Numerical and categorical data']

It turns out that with, an API Key, you can automate just about any process that you would do with a browser. In particular, I can automate the process of checking whether you've done your forum homework and received likes on your posts!