tip 8

Author: Dennis Bakhuis

A_Random_titbits/Assets/3_Clap_Frantically_on_Medium.png

@@ -0,0 +1,134 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Pandas tip #8: Explode your DataFrame\n",
+    "A column in a Pandas DataFrame can practically hold any type. Not all types are ideal but some can be useful as an intermediate value. One of these types is a list which creates some sort of higher order structure in your tabular data. \n",
+    "\n",
+    "For example, I used a list in a DataFrame of words, to store the position the word was mentioned in a large corpus. This nicely groups all relevant data, however, to be useful we need to flatten this again. Flattening in Python is nicely done using a list comprehension. In Pandas I found this nifty method: .explode().\n",
+    "\n",
+    "The .explode() method flattens the list from each row, independed of the length of each list and duplicates the rest of the rows columns. It is probably not used very often but definitely a 'nice to have' method in your toolbox."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Lets generate some random data:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "import pandas as pd\n",
+    "\n",
+    "categories = [list('AB'), list('ABC'), list('ABCD')] \n",
+    "n_samples = 100\n",
+    "\n",
+    "rng = np.random.default_rng()\n",
+    "df = pd.DataFrame({\n",
+    "    'client_id': np.arange(n_samples),    \n",
+    "    'product_category': rng.choice(categories, size=n_samples),\n",
+    "}).set_index('client_id')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "df.head()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Flatten a list in Python is easy using list comprehensions:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "categories"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "[item for sublist in categories for item in sublist]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Flattening a column is called exploding in Pandas:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "df.explode('product_category')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "If you have any questions, comments, or requests, feel free to [contact me on LinkedIn](https://linkedin.com/in/dennisbakhuis)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.7"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}