# Note | Statistics for Data Science with Python Week 1

This is a note of the Course [Statistics for Data Science with Python](https://www.coursera.org/learn/statistics-for-data-science-python).

The contents of this week:

- Course Introduction and Python Basics
	- Instructors and Course Overview
	- Python Packages for Data Science
	- Basics of Jupyter Notebooks and Python Review
- Understanding the basics of Descriptive Statistics
	- Statistics introduction 
	- Types of Data
	- Measure of Central Tendency
	- Measure of Dispersion
- Experiments: Descriptive Statistics

## Course Introduction and Python Basics

The instructors are Murtaza Haider(Associate professor @ Ryerson University in Toronto) and Aije Egwaikhide(Senior Data Scientist and Statisticians @ IBM).

> This course consists of five modules: Introduction and Descriptive Statistics, Data Visualization, Introduction to Probability Distribution, Hypothesis Testing, and Regression Analysis.

The main packages relevant to analysis in python are divided into 3 groups:

- Scientific Computing Libraries
	- [Pandas](https://pandas.pydata.org/), offers data structure and tools
	- [NumPy](https://numpy.org/doc/stable/), provides a multidimensional array object, various derived objects and so on.
	- [SciPy](https://scipy.org/), provides fundamental algorithms for scientific computing in Python.
- Visualization Libraries
	- [Matplotlib](https://matplotlib.org/), makes graphs and plots 
	- [Seaborn](https://seaborn.pydata.org/), high level visualization library based on Matplotlib.
- Algorithmic Libraries
	- [scikit-learn](https://scikit-learn.org/stable/)
	- [statsmodels](https://www.statsmodels.org/stable/index.html)

I have experience in  Python, Pandas, NumPy, Matplolib, and scikit-learn. So these are not unfamiliar to me. Here is my repository [gopython](https://github.com/JeremiahZhang/gopython).

## The basics of Descriptive Statistics

### Types of Data

In our daily lives, we are surrounded by data and statistics. The most common data would be a **cross-sectional data**, "which is basically looking at *a measurement taken at one point in time*"[1][1]. 

Compared to the cross-sectional data, there are panel or cross-sectional panel data, "which is essentially asking the same group of individuals the same questions repeatedly over time.[1][1]" 

Another data type is the time series data, as the name indicates, it is "a series of data points indexed (or listed or graphed) in time order.[2:Wikipedia][2]"

### Measure of Central Tendency

We can use mean, median, and mode to measure the central tendency of the data. In Pandas, `dataframe.mean()` and `dataframe.median()` can help us to get the mean and median of the data.

"The *mean* of a set of observations is the arithmetic *average* of the values.[3][3]" It is also called the arithmetic mean. There are other types of means.

"The *median* is the middle number in a sorted, ascending, or descending, list of numbers and can be more descriptive of that data set than the average.[4][4]"

For example, the mean(or average) of this list of numbers `[10, 15, 20, 25, 30]` is (10 + 15 + 20 + 25 + 30) / 5 = 20. The median is 20. If the list of numbers is `[10, 10, 11, 12, 13, 14]`, the median is (11 + 12) / 2 = 11.5.

### Measure of Dispersion

The common measures of dispersion are **standard deviation** and **variance**.

There are sample variance and population variance. We should notice the difference between sample variance([variance of the sampling distribution of the sample mean](https://www.statisticshowto.com/probability-and-statistics/statistics-definitions/sample-mean/)) and [
Population Variance](https://www.statisticshowto.com/population-variance/).

![variance](https://cdn.hashnode.com/res/hashnode/image/upload/v1649842605039/jsQKmkWii.png)

The standard deviation is the square root of the variance. So the standard deviation of the population is different from the standard deviation of the sample.

![std](https://cdn.hashnode.com/res/hashnode/image/upload/v1649842948836/1JW2j62oi.png)

## Experiments: Descriptive Statistics

In this part, we can go over some hands-on exercises using Python to do some descriptive statistics. Here is the [source code](https://github.com/JeremiahZhang/gopython/blob/master/statistics-for-data-science-with-python/week01-descriptive-statistics.ipynb). You can use Google Colab to execute.

[1]: https://www.coursera.org/learn/statistics-for-data-science-python
[2]: https://en.wikipedia.org/wiki/Time_series
[3]: https://en.wikipedia.org/wiki/Mean
[4]: https://www.investopedia.com/terms/m/median.asp