Traditional loops over lists become a major performance bottleneck when processing large amounts of data.
Learn the fundamentals of linear algebra, where data points are vectors, datasets are matrices, and NumPy arrays provide the data structure.
The k-nearest neighbors algorithm classifies data points by finding the most common label among their closest neighbors in a vector space.
A walkthrough of an unoptimized k-nearest neighbors implementation demonstrates the severe performance issues caused by nested loops.
Replace the inner loop for calculating Manhattan distance with a single vectorized subtraction operation on NumPy arrays.
Use array reshaping and broadcasting to perform all pairwise distance calculations simultaneously, avoiding explicit replication and nested loops.
Gain final performance improvements by replacing Python's list sorting with NumPy's faster sorting algorithms and efficient array slicing.
NumPy arrays are faster because they store data in a contiguous block of memory, which is more efficient for the CPU to process than the scattered memory of Python lists.
Answers common questions about NumPy's underlying C implementation, hyperparameter tuning, memory management, and career paths in data science.
56:46Jodie Burchell
59:43Paul Graham
21:02Andy Terrel
57:46Adrian Schmitt
46:30Anirudh Koul
39:04Bobur Umurzokov
25:29Diana Gastrin
50:38Radovan Kavický
.webp?w=240&auto=compress,format)
Jobs that call for the skills explored in this talk.
CONTIAMO GMBH
Berlin, Germany
