Downloaded 29 times
![Feature Vectors
A data point is represented by a feature vector
Ninja Turtle = [name, weapon, mask_color]
data point 1 = [michelangelo,nunchaku,orange]
data point 2 = [leonardo,katana,blue]
…](https://image.slidesharecdn.com/machine-learning-formoderndevelopers-140607205731-phpapp01/75/Machine-Learning-for-Modern-Developers-27-2048.jpg)
![High-Dimensional Feature Spaces
Most feature vectors are much higher
dimensionality, such as:
FVlaptop = [name,screen size,weight,battery life,
proc,proc speed,ram,price,hard drive,OS]
This means we can’t easily display it visually, but
statistics and matrix math work just fine](https://image.slidesharecdn.com/machine-learning-formoderndevelopers-140607205731-phpapp01/75/Machine-Learning-for-Modern-Developers-30-2048.jpg)
![Demo: Logistic Regression (Scikit-
Learn)
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
iris = load_iris()
# set data
X, y = iris.data, iris.target
# train classifier
clf = LogisticRegression().fit(X, y)
# 'setosa' data point
observed_data_point = [[ 5.0, 3.6, 1.3, 0.25]]
# classify
clf.predict(observed_data_point)
# determine classification probabilities
clf.predict_proba(observed_data_point)](https://image.slidesharecdn.com/machine-learning-formoderndevelopers-140607205731-phpapp01/75/Machine-Learning-for-Modern-Developers-55-2048.jpg)
![Demo: Support Vector Machines
(Scikit-Learn)
from sklearn.datasets import load_iris
from sklearn.svm import LinearSVC
iris = load_iris()
# set data
X, y = iris.data, iris.target
# run regression
clf = LinearSVC().fit(X, y)
# 'setosa' data point
observed_data_point = [[ 5.0, 3.6, 1.3, 0.25]]
# classify
clf.predict(observed_data_point)](https://image.slidesharecdn.com/machine-learning-formoderndevelopers-140607205731-phpapp01/75/Machine-Learning-for-Modern-Developers-62-2048.jpg)
Machine Learning for Modern Developers
- 1. Machine Learning ForModern Developers C. Aaron Cois, PhD
- 2.
- 3. Let’s talk aboutMachine Learning
- 4.
- 5.
- 6.
- 7.
- 8. The Definition “Field ofstudy that gives computers the ability to learn without being explicitly programmed” ~ Arthur Samuel, 1959
- 9. That sounds likeArtificial Intelligence
- 10. That sounds likeArtificial Intelligence True
- 11. That sounds likeArtificial Intelligence Machine Learning is a branch of Artificial Intelligence
- 12. That sounds likeArtificial Intelligence ML focuses on systems that learn from data Many AI systems are simply programmed to do one task really well, such as playing Checkers. This is a solved problem, no learning required.
- 13. Isn’t that howSkynet starts?
- 14. Isn’t that howSkynet starts? Ya, probably
- 15. Isn’t that howSkynet starts?
- 16. But it’s alsohow we do this…
- 17.
- 18.
- 19. Isn’t this juststatistics? Machine Learning can take statistical analyses and make them automated and adaptive Statistical and numerical methods are Machine Learning’s hammer
- 20. Supervised vs. Unsupervised Supervised= System trained on human labeled data (desired output known) Unsupervised = System operates on unlabeled data (desired output unknown)
- 21. Supervised learning isall about generalizing a function or mapping between inputs and outputs
- 22. Supervised Learning Example: ComplementaryColors … Training Data … Test Data
- 23. Supervised Learning Example: ComplementaryColors … Training Data f( ) = … Test Data
- 24. Supervised Learning Example: ComplementaryColors … Training Data f( ) = f( ) = … Test Data
- 25.
- 26. Supervised Learning Example: ComplementaryColors input,output red,green violet,yellow blue,orange orange,blue … training_data.csv red green yellow orange blue … test_data.csv First line indicates data fields
- 27. Feature Vectors A datapoint is represented by a feature vector Ninja Turtle = [name, weapon, mask_color] data point 1 = [michelangelo,nunchaku,orange] data point 2 = [leonardo,katana,blue] …
- 29. Feature Space Feature vectorsdefine a point in an n- dimensional feature space 0 0.1 0.2 0.3 0.4 0.5 0.6 0 0.2 0.4 0.6 0.8 1 1.2 If my feature vectors contain only 2 values, this defines a point in 2-D space: (x,y) = (1.0,0.5)
- 30. High-Dimensional Feature Spaces Mostfeature vectors are much higher dimensionality, such as: FVlaptop = [name,screen size,weight,battery life, proc,proc speed,ram,price,hard drive,OS] This means we can’t easily display it visually, but statistics and matrix math work just fine
- 31. Feature Space Manipulation Featurespaces are important! Many machine learning tasks are solved by selecting the appropriate features to define a useful feature space
- 32. Task: Classification Classification isthe act of placing a new data point within a defined category Supervised learning task Ex. 1: Predicting customer gender through shopping data Ex. 2: From features, classifying an image as a car or truck
- 33. Linear Classification Linear classificationuses a linear combination of features to classify objects
- 34. Linear Classification Linear classificationuses a linear combination of features to classify objects result Weight vector Feature vector Dot product
- 35. Linear Classification Another wayto think of this is that we want to draw a line (or hyperplane) that separates datapoints from different classes
- 36. Sometimes this iseasy Classes are well separated in this feature space Both H1 and H2 accurately separate the classes.
- 37. Other times, lessso This decision boundary works for most data points, but we can see some incorrect classifications
- 38. Example: Iris Data There’sa famous dataset published by R.A. Fisher in 1936 containing measurements of three types of Iris plants You can download it yourself here: http://archive.ics.uci.edu/ml/datasets/Iris
- 39. Example: Iris Data Features: 1.sepal length in cm 2. sepal width in cm 3. petal length in cm 4. petal width in cm 5. class Data: 5.1,3.5,1.4,0.2,Iris-setosa 4.9,3.0,1.4,0.2,Iris-setosa … 7.0,3.2,4.7,1.4,Iris-versicolor … 6.8,3.0,5.5,2.1,Iris-virginica …
- 40. Data Analysis We have4 features in our vector (the 5th is the classification answer) Which of the 4 features are useful for predicting class?
- 41. 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 1 23 4 5 6 7 8 9 sepiawidth sepia length sepia length vs width
- 42. Different feature spacesgive different insight
- 43. 0 1 2 3 4 5 6 7 8 0 1 23 4 5 6 7 8 9 petallength sepia length sepia length vs petal length
- 44. 0 0.5 1 1.5 2 2.5 3 0 1 23 4 5 6 7 8 petalwidth petal length petal length vs petal width
- 45. 0 0.5 1 1.5 2 2.5 3 0 0.5 11.5 2 2.5 3 3.5 4 4.5 5 petalwidth sepia width sepia width vs petal width
- 46. Half the battleis choosing the features that best represent the discrimination you want
- 47. Feature Space Transforms Thegoal is to map data into an effective feature space
- 48.
- 49. Logistic Regression Classification techniquebased on fitting a logistic curve to your data
- 50. Logistic Regression P(Y |b, x) = 1 1+e-(b0+b1x)
- 51. Logistic Regression Class 2 Class1 Probability of data point being in a class Model weights P(Y | b, x) = 1 1+e-(b0+b1x)
- 52. More Dimensions! Extending thelogistic function into N- dimensions:
- 53. More Dimensions! Extending thelogistic function into N- dimensions: Vectors! More weights!
- 54.
- 55. Demo: Logistic Regression(Scikit- Learn) from sklearn.datasets import load_iris from sklearn.linear_model import LogisticRegression iris = load_iris() # set data X, y = iris.data, iris.target # train classifier clf = LogisticRegression().fit(X, y) # 'setosa' data point observed_data_point = [[ 5.0, 3.6, 1.3, 0.25]] # classify clf.predict(observed_data_point) # determine classification probabilities clf.predict_proba(observed_data_point)
- 56. Learning In all casesso far, “learning” is just a matter of finding the best values for your weights Simply, find the function that fits the training data the best More dimensions more features we can consider
- 57. What are wedoing? Logistic regression is actually maximizing the likelihood of the training data This is an indirect method, but often has good results What we really want is to maximize the accuracy of our model
- 58. Support Vector Machines(SVMs) Remember how a large number of lines could separate my classes?
- 59. Support Vector Machines(SVMs) SVMs try to find the optimal classification boundary by maximizing the margin between classes
- 60. Bigger margins meanbetter classification of new data points
- 61. Points on theedge of a class are called Support Vectors Support vectors
- 62. Demo: Support VectorMachines (Scikit-Learn) from sklearn.datasets import load_iris from sklearn.svm import LinearSVC iris = load_iris() # set data X, y = iris.data, iris.target # run regression clf = LinearSVC().fit(X, y) # 'setosa' data point observed_data_point = [[ 5.0, 3.6, 1.3, 0.25]] # classify clf.predict(observed_data_point)
- 63. Want to tryit yourself? Working code from this talk: https://github.com/cacois/ml- classification-examples
- 64. Some great onlinecourses Coursera (Free!) https://www.coursera.org/course/ml Caltech (Free!) http://work.caltech.edu/telecourse Udacity (free trial) https://www.udacity.com/course/ud675
- 66.
Editor's Notes
- #6 What some customers think
- #7 What some people think
- #20 And like any toolbox, the contents are tools – not processes, procedures, or algorithms. Machine Learning provides these components.
- #21 Supervised learning algorithms are trained on labelled examples, i.e., input where the desired output is known. The supervised learning algorithm attempts to generalise a function or mapping from inputs to outputs which can then be used speculatively to generate an output for previously unseen inputs. Unsupervised learning algorithms operate on unlabelled examples, i.e., input where the desired output is unknown. Here the objective is to discover structure in the data (e.g. through a cluster analysis), not to generalise a mapping from inputs to outputs.
- #37 Note: many possible boundaries between black and white dots
- #49 plot_iris.py
- #52 DEMO
- #58 i.e. many logistic models can work the same on training data, some are better than others. We can’t tell.